Monitoring WildFly with Vigilmon

WildFly (formerly JBoss Application Server) is Red Hat's open source, production-ready Jakarta EE application server and the upstream project for Red Hat JBoss EAP. When you self-host WildFly, you're running a sophisticated stack: the Undertow HTTP engine, ActiveMQ Artemis for JMS messaging, Infinispan for distributed caching, JDBC datasource pools, and the Elytron security layer — all managed through a powerful HTTP management API on port 9990. Any silent failure in this stack can bring down your Jakarta EE applications, drain your datasource pool, or leave JMS messages piling up undelivered. Vigilmon gives you end-to-end visibility across every critical subsystem in your WildFly server.

What You'll Set Up

• WildFly management API (port 9990) reachability and server state monitor
• Undertow HTTP listener health check
• Request throughput and error rate monitoring per deployment
• Undertow worker thread pool saturation alert
• JVM heap usage and GC pause monitoring
• Datasource JDBC connection pool health
• ActiveMQ Artemis JMS queue depth and consumer health
• Infinispan cache hit ratio and capacity monitoring
• Application deployment status monitoring
• Domain controller and host controller health (for domain mode)

Prerequisites

• WildFly 26+ running in standalone or domain mode
• Management HTTP interface enabled on port 9990 (default)
• A management user configured via add-user.sh
• A free Vigilmon account

Step 1: Monitor the WildFly Management API

The WildFly HTTP management API on port 9990 is the control plane for everything — configuration, runtime state, and subsystem metrics. If this endpoint goes down or starts rejecting requests, you lose visibility into the entire server.

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. URL: http://your-server:9990/management (or https:// if you've configured TLS on the management interface).
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 401 — an unauthenticated GET to the management API returns 401, confirming the API is alive and enforcing authentication.
6. Click Save.

To verify the actual server state (not just API reachability), you can query the server-state attribute via the management API from a monitoring script and feed the result into a Vigilmon heartbeat:

#!/bin/bash
# Run as a cron job every minute
STATE=$(curl -s -u admin:yourpassword \
  'http://localhost:9990/management?operation=attribute&name=server-state' \
  -H 'Accept: application/json' | jq -r '.result')

if [ "$STATE" = "running" ]; then
  curl -s "https://vigilmon.online/heartbeat/YOUR_HEARTBEAT_ID" > /dev/null
fi

1. Click Add Monitor → Cron Heartbeat.
2. Set expected interval to 2 minutes.
3. Replace YOUR_HEARTBEAT_ID with the heartbeat URL from Vigilmon.
4. Add this script to cron: * * * * * /opt/scripts/wildfly-state-check.sh

Alert fires when server state is not running — catching FAILED, STOPPING, or STOPPED states before they escalate.

Step 2: Monitor Undertow HTTP Listener Health

Undertow is WildFly's embedded HTTP server, replacing JBoss Web/Tomcat since WildFly 8. The default HTTP listener on port 8080 handles all inbound web traffic. A stopped listener means all your deployed applications are unreachable.

1. Click Add Monitor → TCP Port.
2. Host: your WildFly server IP or hostname.
3. Port: 8080 (default HTTP) or 8443 (default HTTPS).
4. Check interval: 1 minute.
5. Click Save.

For a richer check that verifies Undertow is actually serving requests (not just that the port is bound), add an HTTP monitor against a deployed health endpoint:

1. Click Add Monitor → HTTP / HTTPS.
2. URL: http://your-server:8080/your-app/health.
3. Expected HTTP status: 200.
4. Check interval: 1 minute.
5. Click Save.

If your application doesn't expose a /health endpoint, add a minimal one:

@Path("/health")
@GET
@Produces(MediaType.APPLICATION_JSON)
public Response health() {
    return Response.ok("{\"status\":\"UP\"}").build();
}

Step 3: Monitor Request Throughput and Error Rate

WildFly tracks per-deployment request statistics through the Undertow subsystem. A spike in error counts or a sudden drop in request throughput signals application-level failures — unhandled exceptions, missing dependencies, or database connectivity loss.

Use a monitoring script that reads deployment statistics via the management API and pings a Vigilmon heartbeat only when the error rate is within acceptable bounds:

#!/bin/bash
DEPLOY="your-app.war"
BASE="http://localhost:9990/management"
AUTH="admin:yourpassword"

ERRORS=$(curl -s -u "$AUTH" \
  "$BASE/deployment/$DEPLOY/subsystem/undertow?operation=attribute&name=error-count" \
  -H 'Accept: application/json' | jq -r '.result')

REQUESTS=$(curl -s -u "$AUTH" \
  "$BASE/deployment/$DEPLOY/subsystem/undertow?operation=attribute&name=request-count" \
  -H 'Accept: application/json' | jq -r '.result')

# Alert if error ratio exceeds 5%
if [ "$REQUESTS" -gt 0 ] && [ $(( ERRORS * 100 / REQUESTS )) -lt 5 ]; then
  curl -s "https://vigilmon.online/heartbeat/YOUR_HEARTBEAT_ID" > /dev/null
fi

Set the Vigilmon heartbeat interval to 5 minutes. If no ping arrives, the error rate has exceeded your threshold or the management API query failed.

Step 4: Monitor Undertow Worker Thread Pool

Undertow uses XNIO worker thread pools to process HTTP requests. When inbound traffic saturates the pool, requests queue up, response times climb, and eventually the server begins refusing connections. Catching thread pool saturation early lets you scale before users notice.

#!/bin/bash
BASE="http://localhost:9990/management"
AUTH="admin:yourpassword"

WORKER_STATS=$(curl -s -u "$AUTH" \
  "$BASE/core-service/platform-mbean/type/threading?operation=attribute&name=thread-count" \
  -H 'Accept: application/json')

# Also query Undertow worker thread pool utilization
ACTIVE=$(curl -s -u "$AUTH" \
  "$BASE/subsystem/io/worker/default?operation=attribute&name=io-thread-count" \
  -H 'Accept: application/json' | jq -r '.result')

echo "Active IO threads: $ACTIVE"
# Feed into your alerting if active threads approach your configured max

1. In Vigilmon, create a Cron Heartbeat with a 2-minute interval.
2. Have your monitoring script ping the heartbeat when thread utilization is below 80% of configured maximum.
3. Configure alerts: E-mail or Slack notification when the heartbeat misses.

Step 5: Monitor JVM Heap and Garbage Collection

WildFly runs on the JVM, so heap exhaustion and GC pressure are universal concerns regardless of your application type. Extended GC pauses freeze request processing; heap exhaustion causes OutOfMemoryError and crashes the server.

#!/bin/bash
BASE="http://localhost:9990/management"
AUTH="admin:yourpassword"

HEAP_USED=$(curl -s -u "$AUTH" \
  "$BASE/core-service/platform-mbean/type/memory?operation=attribute&name=heap-memory-usage" \
  -H 'Accept: application/json' | jq -r '.result.used')

HEAP_MAX=$(curl -s -u "$AUTH" \
  "$BASE/core-service/platform-mbean/type/memory?operation=attribute&name=heap-memory-usage" \
  -H 'Accept: application/json' | jq -r '.result.max')

HEAP_PCT=$(( HEAP_USED * 100 / HEAP_MAX ))

# Ping heartbeat only when heap usage is below 85%
if [ "$HEAP_PCT" -lt 85 ]; then
  curl -s "https://vigilmon.online/heartbeat/YOUR_HEAP_HEARTBEAT_ID" > /dev/null
fi

1. Create a Cron Heartbeat in Vigilmon with 2-minute interval.
2. Alert triggers when heap usage exceeds 85% for more than 2 consecutive check intervals.
3. Also monitor GC pause time via the java.lang:type=GarbageCollector MBean; alert when any single GC pause exceeds 500ms.

Step 6: Monitor Datasource Connection Pool

WildFly's datasource subsystem manages JDBC connection pools for all database access. Connection pool exhaustion causes your application to block waiting for connections, manifesting as slow responses or timeouts before the root cause is obvious.

#!/bin/bash
BASE="http://localhost:9990/management"
AUTH="admin:yourpassword"
DS="ExampleDS"  # Replace with your datasource name

POOL=$(curl -s -u "$AUTH" \
  "$BASE/subsystem/datasources/data-source/$DS/statistics=pool?operation=attribute&name=ActiveCount" \
  -H 'Accept: application/json' | jq -r '.result')

MAX=$(curl -s -u "$AUTH" \
  "$BASE/subsystem/datasources/data-source/$DS/statistics=pool?operation=attribute&name=MaxUsedCount" \
  -H 'Accept: application/json' | jq -r '.result')

TIMEOUT=$(curl -s -u "$AUTH" \
  "$BASE/subsystem/datasources/data-source/$DS/statistics=pool?operation=attribute&name=TimedOut" \
  -H 'Accept: application/json' | jq -r '.result')

# Alert if any connections have timed out
if [ "$TIMEOUT" -eq 0 ]; then
  curl -s "https://vigilmon.online/heartbeat/YOUR_DS_HEARTBEAT_ID" > /dev/null
fi

Create a Vigilmon Cron Heartbeat with 2-minute interval. A missed ping means connections are timing out, which directly impacts application availability.

Step 7: Monitor ActiveMQ Artemis JMS Broker

WildFly includes ActiveMQ Artemis (replacing HornetQ since WildFly 10) as its embedded JMS messaging provider. If the broker crashes or a queue accumulates messages without consumers draining them, asynchronous workflows grind to a halt.

#!/bin/bash
BASE="http://localhost:9990/management"
AUTH="admin:yourpassword"
QUEUE="YourQueueName"

DEPTH=$(curl -s -u "$AUTH" \
  "$BASE/subsystem/messaging-activemq/server/default/jms-queue/$QUEUE?operation=attribute&name=message-count" \
  -H 'Accept: application/json' | jq -r '.result')

CONSUMERS=$(curl -s -u "$AUTH" \
  "$BASE/subsystem/messaging-activemq/server/default/jms-queue/$QUEUE?operation=attribute&name=consumer-count" \
  -H 'Accept: application/json' | jq -r '.result')

# Alert if messages are accumulating without consumers
if [ "$CONSUMERS" -gt 0 ] && [ "$DEPTH" -lt 1000 ]; then
  curl -s "https://vigilmon.online/heartbeat/YOUR_JMS_HEARTBEAT_ID" > /dev/null
fi

Set the heartbeat interval to 3 minutes. A missed ping alerts you before unprocessed JMS messages cascade into data loss or missed business events.

Step 8: Monitor Infinispan Cache Health

WildFly uses Infinispan for HTTP session clustering, EJB caching, and Hibernate second-level caching. A degraded cache forces expensive database fallbacks and can break session persistence in clustered deployments.

#!/bin/bash
BASE="http://localhost:9990/management"
AUTH="admin:yourpassword"
CACHE_CONTAINER="web"
CACHE_NAME="dist"

HITS=$(curl -s -u "$AUTH" \
  "$BASE/subsystem/infinispan/cache-container/$CACHE_CONTAINER/distributed-cache/$CACHE_NAME/statistics=statistics?operation=attribute&name=hit-ratio" \
  -H 'Accept: application/json' | jq -r '.result')

ENTRIES=$(curl -s -u "$AUTH" \
  "$BASE/subsystem/infinispan/cache-container/$CACHE_CONTAINER/distributed-cache/$CACHE_NAME/statistics=statistics?operation=attribute&name=number-of-entries" \
  -H 'Accept: application/json' | jq -r '.result')

# Alert if cache hit ratio falls below 0.5 (50%)
HIT_INT=$(echo "$HITS" | cut -d'.' -f1)
if [ "${HIT_INT:-0}" -ge 0 ]; then
  curl -s "https://vigilmon.online/heartbeat/YOUR_CACHE_HEARTBEAT_ID" > /dev/null
fi

Monitor hit ratio via heartbeat. Set alerting threshold: hit ratio < 0.5 or eviction rate > 100/min signals the cache working set has exceeded capacity.

Step 9: Monitor Application Deployment Status

WildFly tracks the deployment state of every application — OK, FAILED, STOPPED. A failed deployment means your application is not running, but WildFly itself appears healthy, so TCP and management API checks won't catch it.

#!/bin/bash
BASE="http://localhost:9990/management"
AUTH="admin:yourpassword"

DEPLOYS=$(curl -s -u "$AUTH" \
  "$BASE/deployment?operation=attribute&name=status" \
  -H 'Accept: application/json')

# Check all deployments are in OK state
FAILED=$(echo "$DEPLOYS" | jq '[.[] | select(.result != "OK")] | length')

if [ "$FAILED" -eq 0 ]; then
  curl -s "https://vigilmon.online/heartbeat/YOUR_DEPLOY_HEARTBEAT_ID" > /dev/null
fi

1. Create a Cron Heartbeat with 5-minute interval.
2. The heartbeat only pings when all deployments are in OK state.
3. A missed ping means at least one deployment has FAILED or is STOPPED unexpectedly.

Step 10: Monitor Domain Controller Health (Domain Mode)

If you're running WildFly in domain mode, the Domain Controller is the central configuration and management authority. Host Controllers on each managed host connect to the Domain Controller at startup and remain connected. A Domain Controller outage means you can't push configuration changes or start/stop server instances cluster-wide.

1. Click Add Monitor → TCP Port.
2. Host: your Domain Controller hostname.
3. Port: 9990 (management HTTP) and 9999 (native management protocol).
4. Check interval: 1 minute.
5. Click Save for each port.

For host controller registration count monitoring:

#!/bin/bash
AUTH="admin:yourpassword"

HC_COUNT=$(curl -s -u "$AUTH" \
  'http://domain-controller:9990/management/host?operation=read-children-names&child-type=server' \
  -H 'Accept: application/json' | jq '.result | length')

# Alert if host controller count drops below expected minimum
if [ "$HC_COUNT" -ge 2 ]; then
  curl -s "https://vigilmon.online/heartbeat/YOUR_DC_HEARTBEAT_ID" > /dev/null
fi

Set the heartbeat interval to 2 minutes. A missed ping means a host controller has disconnected from the domain — catching node failures before your load balancer routes traffic to the disconnected host.

Alerting Configuration

Set up alert channels in Vigilmon to reach your team when something goes wrong:

1. Go to Alert Channels in Vigilmon.
2. Add Email alerts for critical monitors (management API, HTTP listener, JMS broker).
3. Add Slack or Teams webhook for deployment status and domain controller alerts.
4. Set alert sensitivity: 2 consecutive failures before alerting (avoids flapping on momentary network blips).

Recommended alert thresholds for WildFly:

| Monitor | Alert Condition | |---|---| | Management API | Not reachable for 2 minutes | | Server state | Any state other than running | | Heap usage | > 85% for 4 minutes | | GC pause | Any pause > 500ms | | Datasource timeouts | Any TimedOut > 0 | | JMS queue depth | > 1000 messages with 0 consumers | | Deployment status | Any deployment not in OK state | | Host controller count | Below expected minimum |

Conclusion

WildFly's rich management API makes it unusually monitorable — every subsystem from Undertow thread pools to Infinispan caches exposes runtime metrics via a consistent HTTP interface. With Vigilmon collecting heartbeats from your management API scripts and running TCP/HTTP checks on your application endpoints, you get end-to-end coverage across the entire WildFly stack: server health, web tier, JMS messaging, distributed caching, datasource pools, and deployment integrity. Set up these monitors once and you'll catch issues in WildFly's subsystems before your users do.