tutorial

How to Monitor Cortex XSOAR Community Edition Health with Vigilmon

Cortex XSOAR server unavailability, playbook automation failures, and integration connector errors silently halt incident response workflows that security teams depend on during active incidents. Learn how to monitor XSOAR server health, playbook execution rates, integration connectivity, and database health with Vigilmon HTTP probes and heartbeat monitors.

Cortex XSOAR Community Edition (formerly Demisto Community Edition) is the free-tier SOAR platform from Palo Alto Networks that automates incident response playbooks across dozens of security tool integrations. When the XSOAR server process crashes due to a PostgreSQL connection pool exhaustion during a high-volume incident wave, playbooks that were mid-execution hang in a running state that never completes — alerts from your SIEM continue arriving, creating new incidents, but no automated enrichment, containment, or notification steps execute; when a VirusTotal integration connector fails because the API key is rate-limited or expired, every phishing investigation playbook that calls VirusTotal-Hash silently returns an error that the playbook treats as inconclusive — analysts receive escalated tickets with empty enrichment fields and must manually re-investigate what automation was supposed to handle; when the Elasticsearch search backend for XSOAR goes down during a shift change, the incident search page returns no results — analysts cannot find in-progress cases from the previous shift and may duplicate effort or miss handoffs entirely.

Vigilmon gives you external visibility into Cortex XSOAR Community Edition through HTTP probe monitoring and heartbeat monitors for server health, playbook execution rates, integration health, and database connectivity. This tutorial covers the full XSOAR monitoring stack.

Why Cortex XSOAR Needs External Monitoring

XSOAR failure modes are operationally dangerous because they halt automated incident response during exactly the moments when it is most needed:

  • XSOAR server crash during incident surge: During a high-volume attack campaign (mass phishing wave, ransomware propagation), XSOAR receives hundreds of incidents from the SIEM simultaneously; if the PostgreSQL connection pool is exhausted or the Python backend hits a memory limit, the server process may crash; all in-flight playbook executions terminate without completing; existing incidents remain open with partially executed playbooks; new incidents from the SIEM queue up with no automation processing them
  • Playbook failure rate spike: When an integration dependency (threat intelligence feed, ticketing system, EDR connector) becomes unavailable, every playbook that calls that integration generates a playbook error; the failure rate climbs from normal background noise to >10%; analysts receive escalated incidents that automation was supposed to handle automatically, and they have no visibility into whether the failure is a one-off or a systemic integration outage
  • Integration connector failure for critical security tools: XSOAR integrations to critical tools (SIEM, EDR, threat intelligence) are the connectors that make automated investigation possible; when a connector fails due to certificate expiry, credential rotation, or API version mismatch, all playbooks depending on that integration fail silently — the XSOAR incident log shows playbook errors but no external alert fires; the security team continues thinking automation is working
  • PostgreSQL failure: XSOAR stores all incident data, evidence, analyst notes, and playbook state in PostgreSQL; a PostgreSQL crash caused by disk full, OOM kill, or corruption during a power event means all incident data is inaccessible; playbooks cannot read or write incident fields; the XSOAR web UI shows incidents but all field reads return errors
  • Incident queue growth during analyst gap: XSOAR Community Edition is limited to 3 users; during nights and weekends with reduced analyst coverage, the open incident queue grows; when analysts return, they face a backlog too large to review manually — incidents from 12 hours ago may have already missed the forensic window for containment
  • Content version drift: XSOAR Content packs provide updated integration versions, new playbooks, and security content; when content is not updated for 30+ days, integrations built against newer API versions of security tools may fail with compatibility errors as the tools update their APIs

External monitoring with Vigilmon adds:

  • XSOAR server reachability checks that fire before analysts discover the server is down during an active incident
  • Playbook execution rate heartbeats that detect when automation has stalled
  • Database connectivity monitoring at the layer below the XSOAR application
  • Integration health assertions through the XSOAR API

Step 1: Build an XSOAR Health Endpoint

XSOAR exposes a REST API that can be used for health checks. Build a lightweight sidecar that queries the XSOAR API and exposes aggregated health over HTTP, or use the XSOAR API directly with Vigilmon's HTTP probe authentication.

Using the XSOAR REST API Directly

XSOAR's REST API is available at https://your-xsoar:443/xsoar/public/v1/. Generate an API key:

  1. Open XSOAR → Settings → API Keys → New Key
  2. Copy the generated key

Use Vigilmon's HTTP probe with the XSOAR API key header.

Python Health Aggregator (Alternative)

# health/xsoar_health.py
import os
import requests
from flask import Flask, jsonify

app = Flask(__name__)

XSOAR_URL     = os.environ.get('XSOAR_URL', 'https://localhost:443')
XSOAR_API_KEY = os.environ.get('XSOAR_API_KEY', '')
XSOAR_HEADERS = {'x-xdr-auth-id': '1', 'x-xdr-nonce': 'nonce', 'Authorization': XSOAR_API_KEY}
VERIFY_TLS    = os.environ.get('XSOAR_VERIFY_TLS', 'false').lower() == 'true'

def xsoar_get(path):
    try:
        r = requests.get(f'{XSOAR_URL}/xsoar/public/v1{path}',
                         headers={'Authorization': XSOAR_API_KEY},
                         verify=VERIFY_TLS,
                         timeout=10)
        return r.json() if r.ok else {'error': f'HTTP {r.status_code}'}
    except Exception as e:
        return {'error': str(e)}

def xsoar_post(path, body):
    try:
        r = requests.post(f'{XSOAR_URL}/xsoar/public/v1{path}',
                          json=body,
                          headers={'Authorization': XSOAR_API_KEY},
                          verify=VERIFY_TLS,
                          timeout=10)
        return r.json() if r.ok else {'error': f'HTTP {r.status_code}'}
    except Exception as e:
        return {'error': str(e)}

def get_server_health():
    result = xsoar_get('/about')
    return {'reachable': 'error' not in result, 'version': result.get('demistoVersion', 'unknown')}

def get_open_incidents():
    result = xsoar_post('/incidents/search', {
        'filter': {'status': [0, 1, 2]},  # 0=pending, 1=active, 2=done pending review
        'size': 0,
    })
    return {'open_count': result.get('total', -1)}

def get_integration_health():
    result = xsoar_get('/settings/integration/search')
    if 'error' in result:
        return result
    instances = result.get('instances', [])
    failed = [i['name'] for i in instances if i.get('lastError')]
    return {
        'total_instances': len(instances),
        'failed_instances': failed,
        'failure_count': len(failed),
    }

@app.route('/health')
def health():
    server = get_server_health()
    incidents = get_open_incidents()
    integrations = get_integration_health()

    open_count = incidents.get('open_count', -1)
    integration_failures = integrations.get('failure_count', 0)

    healthy = (
        server.get('reachable', False) and
        open_count < 500 and
        integration_failures == 0
    )

    return jsonify({
        'healthy': healthy,
        'server': server,
        'incidents': incidents,
        'integrations': integrations,
    }), (200 if healthy else 503)

@app.route('/health/server')
def health_server():
    data = get_server_health()
    return jsonify(data), (200 if data.get('reachable') else 503)

@app.route('/health/integrations')
def health_integrations():
    data = get_integration_health()
    healthy = data.get('failure_count', 1) == 0 and 'error' not in data
    return jsonify({'healthy': healthy, **data}), (200 if healthy else 503)

@app.route('/health/incidents')
def health_incidents():
    data = get_open_incidents()
    healthy = 0 <= data.get('open_count', -1) < 500
    return jsonify({'healthy': healthy, **data}), (200 if healthy else 503)

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8766)

pip install flask requests
XSOAR_URL=https://your-xsoar XSOAR_API_KEY=your-key \
  python health/xsoar_health.py &

Step 2: Monitor XSOAR Server Health

The XSOAR web server must be reachable for analysts to triage incidents and for API-based integrations to submit new incidents.

Add an HTTP Monitor in Vigilmon

  1. Open Vigilmon → Monitors → New Monitor
  2. Set Name: Cortex XSOAR — Server Health
  3. Set URL: http://your-xsoar-host:8766/health/server
  4. Set Method: GET
  5. Set Expected status: 200
  6. Set Check interval: 60 seconds
  7. Set Regions: at least two probe regions

Alternatively, probe the XSOAR API directly:

  • URL: https://your-xsoar/xsoar/public/v1/about
  • Add header: Authorization: <your-api-key>
  • Set Expected status: 200
  • Disable SSL verification if using a self-signed certificate (Vigilmon supports skip_ssl_verify)

Alert Configuration

  1. Open the monitor → Alerts → New Alert
  2. Set Trigger: Status is DOWN
  3. Set Message: Cortex XSOAR server is unreachable — incident response automation has stopped. No new incidents are being processed and all analyst workflows are blocked. Check: systemctl status demisto
  4. Set Recovery message: XSOAR server is back online — incident automation resumed
  5. Add email and Slack channels

Step 3: Monitor Playbook Execution Rate

A stalled playbook execution rate (dropping to zero) indicates automation has halted — new incidents are not being processed by any playbooks.

Heartbeat Monitor for Playbook Activity

XSOAR logs playbook executions. Run a script on the XSOAR host that checks recent playbook execution count and pings Vigilmon only when executions are occurring:

# /opt/xsoar/scripts/check_playbook_rate.sh
#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_PLAYBOOK_KEY"
XSOAR_URL="https://localhost:443"
API_KEY="your-xsoar-api-key"

# Count playbook executions started in the last 10 minutes
COUNT=$(curl -sk \
  -H "Authorization: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"filter":{"fromDate":"'"$(date -d '-10 minutes' -u +%Y-%m-%dT%H:%M:%SZ)"'"},"size":0}' \
  "$XSOAR_URL/xsoar/public/v1/investigation/playbook/search" \
  | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('total',0))")

if [ "$COUNT" -gt "0" ]; then
  curl -sf "$HEARTBEAT_URL" > /dev/null
fi

# /etc/cron.d/xsoar-playbook-heartbeat
*/10 * * * * root /opt/xsoar/scripts/check_playbook_rate.sh

Configure the Heartbeat in Vigilmon

  1. Open Vigilmon → Heartbeats → New Heartbeat
  2. Set Name: Cortex XSOAR — Playbook Execution Rate
  3. Set Expected interval: 15 minutes
  4. Set Grace period: 20 minutes (only alerts if no executions for 20+ minutes during business hours)

Note: In low-volume environments, this heartbeat should only be active during business hours when incident volume is expected. Configure Vigilmon's alert schedule accordingly.

Alert message: XSOAR playbook execution rate has dropped to zero — no incidents have triggered playbook automation in 20+ minutes. Check for XSOAR server issues, PostgreSQL connectivity, or a period of genuine zero incidents.

Step 4: Monitor Playbook Failure Rate

A high playbook failure rate indicates an integration connector outage or a content compatibility issue.

HTTP Monitor with Failure Rate Assertion

Add a failure rate endpoint to the health sidecar:

# Add to xsoar_health.py

def get_playbook_failure_rate():
    import datetime
    since = (datetime.datetime.utcnow() - datetime.timedelta(hours=1)).strftime('%Y-%m-%dT%H:%M:%SZ')
    
    all_result = xsoar_post('/investigation/playbook/search', {
        'filter': {'fromDate': since},
        'size': 0,
    })
    failed_result = xsoar_post('/investigation/playbook/search', {
        'filter': {'fromDate': since, 'status': ['error', 'failed']},
        'size': 0,
    })
    
    total = all_result.get('total', 0)
    failed = failed_result.get('total', 0)
    rate = (failed / total * 100) if total > 0 else 0
    
    return {'total_1h': total, 'failed_1h': failed, 'failure_rate_pct': round(rate, 1)}

@app.route('/health/playbook-failures')
def health_playbook_failures():
    data = get_playbook_failure_rate()
    if 'error' in data:
        return jsonify({'healthy': False, **data}), 503
    healthy = data.get('failure_rate_pct', 100) < 10
    return jsonify({'healthy': healthy, **data}), (200 if healthy else 503)

Create a Vigilmon monitor:

  • URL: http://your-xsoar-host:8766/health/playbook-failures
  • Expected status: 200
  • Check interval: 300 seconds (5 minutes)

Alert message: XSOAR playbook failure rate exceeds 10% — an integration connector or content pack dependency may be failing. Check XSOAR incident logs and integration health.

Step 5: Monitor Integration Health

XSOAR integrations to critical tools (SIEM, threat intel, EDR) must be operational for automated enrichment to work.

  1. Open Vigilmon → Monitors → New Monitor
  2. Set Name: Cortex XSOAR — Integration Health
  3. Set URL: http://your-xsoar-host:8766/health/integrations
  4. Set Expected status: 200
  5. Set Check interval: 300 seconds
  6. Set Response assertion: body must contain "failure_count":0

Alert message: One or more XSOAR integration connectors are reporting errors — playbooks using these integrations will fail. Open XSOAR Settings → Integrations and test each failed integration.

Step 6: Monitor Incident Queue Depth

An incident queue exceeding 500 open incidents indicates analyst capacity issues or automation failure.

  1. Open Vigilmon → Monitors → New Monitor
  2. Set Name: Cortex XSOAR — Incident Queue Depth
  3. Set URL: http://your-xsoar-host:8766/health/incidents
  4. Set Expected status: 200
  5. Set Response assertion: $.open_count is less than 500
  6. Set Check interval: 300 seconds

Alert message: XSOAR open incident queue has exceeded 500 — either automation is failing to close incidents or analyst capacity is insufficient. Review playbook completion rates and escalate if needed.

Step 7: Monitor Database and Engine Health

PostgreSQL Connectivity

Run a PostgreSQL liveness check as a heartbeat:

# /opt/xsoar/scripts/check_postgres.sh
#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_POSTGRES_KEY"
PG_HOST="${XSOAR_PG_HOST:-localhost}"
PG_PORT="${XSOAR_PG_PORT:-5432}"
PG_USER="${XSOAR_PG_USER:-demisto}"
PG_DB="${XSOAR_PG_DB:-demisto}"

if PGPASSWORD="$XSOAR_PG_PASS" pg_isready -h "$PG_HOST" -p "$PG_PORT" -U "$PG_USER" -d "$PG_DB" -q; then
  curl -sf "$HEARTBEAT_URL" > /dev/null
fi

Configure as a Vigilmon heartbeat with a 2-minute interval and 4-minute grace period.

Alert message: XSOAR PostgreSQL database is unreachable — all incident data reads and writes have failed. XSOAR server will not function correctly without database connectivity.

Engine Connectivity

If you run XSOAR remote execution engines (for isolated network segments), monitor each engine with a heartbeat:

# Run on each engine node
*/2 * * * * root curl -sf https://vigilmon.online/heartbeat/YOUR_ENGINE_KEY > /dev/null

Step 8: Alerting Configuration Summary

| Monitor | Type | Interval | Alert Threshold | |---------|------|----------|-----------------| | XSOAR server health | HTTP probe | 60s | Non-200 | | SSL certificate expiry | HTTP probe | 60s | <30 days | | Integration connector health | HTTP assertion | 300s | Any integration with errors | | Playbook failure rate | HTTP assertion | 300s | Failure rate >10% | | Incident queue depth | HTTP assertion | 300s | Open count >500 | | Playbook execution rate | Heartbeat | 15 min | Missing for 20+ min | | PostgreSQL connectivity | Heartbeat | 2 min | Missing for 4+ min | | Engine connectivity (per engine) | Heartbeat | 2 min | Missing for 4+ min |

Conclusion

Cortex XSOAR Community Edition is a force multiplier for small security teams — but when it fails, the automated triage, enrichment, and notification workflows it provides stop silently, and analysts have no way to know that incidents are piling up without automation unless external monitoring is watching. Vigilmon's external HTTP probe and heartbeat monitoring gives your SOC the visibility it needs: server reachability is checked before analysts need the platform, playbook execution rates detect stalled automation, integration health monitoring surfaces connector failures before they cause mass playbook errors, and database connectivity is monitored at the layer below the application.

The monitors above let you catch XSOAR failures during off-hours when reduced analyst coverage makes automation most critical — before an analyst arrives to find hundreds of unprocessed incidents with no investigation context.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles