Monitoring OpenVZ with Vigilmon

OpenVZ is operating-system-level virtualization for Linux: all containers (called Virtual Environments — VEs or CTs) share a single host kernel but have isolated process spaces, network stacks, and filesystems. That shared-kernel architecture is what makes OpenVZ containers so lightweight (no full OS boot, no hypervisor overhead), but it also means a sick host kernel is a sick cluster — a kernel panic takes down every VE simultaneously. Vigilmon lets you monitor OpenVZ from the outside (host and container HTTP endpoints) and from inside the host with scripted checks against vzctl and /proc/user_beancounters, so you catch resource starvation before it becomes a container outage.

What You'll Set Up

• OpenVZ host kernel health and system load monitoring
• Per-container (VE) status and unexpected stop detection
• Container CPU usage vs. configured limit alerting
• Container memory usage and UBC limit monitoring
• Container disk quota tracking
• Network throughput per VE monitoring
• User Beancounter failure detection
• Host disk I/O saturation monitoring
• VE live migration success tracking
• OpenVZ kernel version and security status

Prerequisites

• OpenVZ 7+ (or Virtuozzo) installed on a dedicated Linux server
• vzctl or prlctl available on the host
• SSH access to the OpenVZ host for scripted checks
• A free Vigilmon account

Step 1: Monitor Host Kernel Health

Every OpenVZ container shares the host kernel. If the host goes down — kernel panic, OOM kill of critical processes, or hardware failure — every VE dies with it. External HTTP monitoring gives you the first signal.

If you run a web server on the OpenVZ host itself (even a minimal one for the management panel), add an HTTP monitor:

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter the host management URL or any web service running on the host.
4. Set Check interval to 1 minute.
5. Click Save.

If the host has no web service, use a TCP port monitor on the SSH port:

1. Click Add Monitor → TCP Port.
2. Enter the host IP and port 22.
3. Set Check interval to 1 minute.

This gives you coarse-grained host liveness. For kernel panic detection, use a cron heartbeat that reads dmesg for crash signatures:

#!/bin/bash
# /usr/local/bin/openvz-kernel-check.sh
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_KERNEL_HEARTBEAT_ID"

# Check for kernel panics or oops in the last 5 minutes
PANICS=$(dmesg --since "5 minutes ago" 2>/dev/null | grep -c -i "kernel panic\|oops\|BUG:" || true)

if [ "$PANICS" -eq 0 ]; then
    curl -s "$HEARTBEAT_URL"
fi

Schedule every 5 minutes:

echo "*/5 * * * * root /usr/local/bin/openvz-kernel-check.sh" > /etc/cron.d/openvz-kernel-check

Step 2: Monitor Container (VE) Status

Track the count of running VEs and alert when a container stops unexpectedly.

Create a cron heartbeat for VE status:

1. Click Add Monitor → Cron Heartbeat.
2. Name it OpenVZ VE Status Check.
3. Set the expected interval to 5 minutes.
4. Copy the heartbeat URL.

Script to check all VE states:

#!/bin/bash
# /usr/local/bin/openvz-ve-status.sh
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_VE_STATUS_HEARTBEAT_ID"
EXPECTED_RUNNING=5  # Set to your expected number of running VEs

# Count running VEs
RUNNING=$(vzlist -a -o ctid,status 2>/dev/null | grep "running" | wc -l)

if [ "$RUNNING" -ge "$EXPECTED_RUNNING" ]; then
    curl -s "$HEARTBEAT_URL"
fi

For containers that serve HTTP traffic, add individual Vigilmon HTTP monitors per VE:

# List all running VEs with their IPs
vzlist -o ctid,ip,hostname,status

Add one HTTP monitor per VE that serves web traffic, using the VE's IP or hostname.

Step 3: Track Per-Container CPU Usage

OpenVZ uses CPU units and CPU limits to cap each VE's CPU consumption. A VE stuck at 100% of its CPU limit is throttled and performing poorly — you want to know before users complain.

Check CPU limits and current usage:

# Show CPU limits per VE
vzlist -o ctid,hostname,cpulimit,cpuunits

# Check CPU usage (requires vzstat or /proc/vz/veinfo)
cat /proc/vz/veinfo | awk '{print $1, $3}'  # VEID, CPU time

Create a monitoring script:

#!/bin/bash
# /usr/local/bin/openvz-cpu-check.sh
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_CPU_HEARTBEAT_ID"
CPU_THRESHOLD=90  # Alert if any VE is using >90% of its CPU limit

THROTTLED=$(vzlist -a -o ctid,cpulimit 2>/dev/null | awk 'NR>1 && $2>0 {
    ctid=$1; limit=$2
    cmd="vzctl exec " ctid " grep -c processor /proc/cpuinfo 2>/dev/null"
    cmd | getline count; close(cmd)
    # Simplified check — in practice use vzstat for real CPU %
    print ctid, limit
}' | wc -l)

# Ping only if no critical CPU alerts (extend with real vzstat data in production)
curl -s "$HEARTBEAT_URL"

For detailed per-VE CPU stats, install and use vzstat:

vzstat -t 1 1  # One-shot CPU stats for all VEs

Step 4: Monitor Container Memory and User Beancounters

OpenVZ uses User Beancounters (UBC) to enforce memory limits. UBC failures mean a VE is hitting its resource barriers — a precursor to application OOM kills within the container.

The /proc/user_beancounters file is the canonical UBC data source:

# Show all UBC parameters for all VEs
cat /proc/user_beancounters

Each line shows: VEID resource held maxheld barrier limit failcnt

The critical column is failcnt — a non-zero value means the VE has exceeded its resource limit.

Create a UBC failure detector:

#!/bin/bash
# /usr/local/bin/openvz-ubc-check.sh
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_UBC_HEARTBEAT_ID"

# Check for any UBC failures
FAILURES=$(awk 'NR>1 && $6 != "0" && $6 != "failcnt" {print $1, $2, $6}' \
           /proc/user_beancounters 2>/dev/null | grep -cv "^$" || true)

if [ "$FAILURES" -eq 0 ]; then
    curl -s "$HEARTBEAT_URL"
fi

Schedule every 5 minutes. A missed heartbeat means at least one VE is experiencing resource starvation.

Step 5: Monitor Container Disk Usage

Each VE has a disk quota (soft and hard limits). When a VE hits its hard quota, writes fail, applications crash, and logs stop rolling.

Check disk usage per VE:

# Show disk quota usage for all VEs
vzquota stat -t 2>/dev/null || vzctl exec $CTID df -h /

For a simpler approach, query disk usage inside each running VE:

#!/bin/bash
# /usr/local/bin/openvz-disk-check.sh
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_DISK_HEARTBEAT_ID"
DISK_THRESHOLD=80

OVERFULL=0
for CTID in $(vzlist -o ctid -H 2>/dev/null); do
    USAGE=$(vzctl exec "$CTID" df / 2>/dev/null | awk 'NR==2 {gsub("%",""); print $5}')
    if [ -n "$USAGE" ] && [ "$USAGE" -ge "$DISK_THRESHOLD" ]; then
        OVERFULL=1
        echo "VE $CTID disk at ${USAGE}%"
    fi
done

if [ "$OVERFULL" -eq 0 ]; then
    curl -s "$HEARTBEAT_URL"
fi

Step 6: Monitor Host Disk I/O and Network

All VEs share the host's physical disks. I/O saturation on the host degrades every container simultaneously.

Create a host I/O monitoring heartbeat:

#!/bin/bash
# /usr/local/bin/openvz-io-check.sh
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_IO_HEARTBEAT_ID"
IO_THRESHOLD=90  # Alert if disk I/O utilization >90%

# Use iostat to get utilization
IO_UTIL=$(iostat -dx 1 2 2>/dev/null | awk '/^sd/ || /^vd/ {util=$NF} END {print util}' | \
          tr -d '%' | awk '{if ($1+0 > max) max=$1+0} END {print max}')

if [ -z "$IO_UTIL" ] || [ "$(echo "$IO_UTIL < $IO_THRESHOLD" | bc)" -eq 1 ]; then
    curl -s "$HEARTBEAT_URL"
fi

For network throughput per VE:

# Show network stats per VE interface
vzlist -o ctid,hostname | while read CTID HOSTNAME; do
    echo "=== VE $CTID ($HOSTNAME) ==="
    vzctl exec "$CTID" cat /proc/net/dev 2>/dev/null | grep eth0
done

Step 7: Live Migration Health

If you use OpenVZ live migration to move VEs between hosts, track migration success rate:

#!/bin/bash
# After each migration, log success or failure
MIGRATION_LOG="/var/log/openvz-migrations.log"
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_MIGRATION_HEARTBEAT_ID"

# Wrap vzmigrate with success/failure tracking
migrate_ve() {
    local CTID=$1
    local TARGET=$2
    if vzmigrate --online "$TARGET" "$CTID" 2>&1; then
        echo "$(date): CTID $CTID migrated to $TARGET: SUCCESS" >> "$MIGRATION_LOG"
        curl -s "$HEARTBEAT_URL"
    else
        echo "$(date): CTID $CTID migrated to $TARGET: FAILED" >> "$MIGRATION_LOG"
        # No ping — heartbeat will alert on missed interval
    fi
}

In Vigilmon, set the migration heartbeat interval to slightly longer than your longest migration window to avoid false positives on large VEs.

Step 8: Configure Alert Channels

1. Go to Alert Channels in Vigilmon and add email, Slack, or PagerDuty.
2. For host SSH/HTTP monitors, set Consecutive failures before alert to 2 — a single check failure during a brief network hiccup should not page you.
3. For UBC failure heartbeats, alert on the first missed ping — UBC failures indicate active resource starvation and demand immediate attention.
4. For disk usage heartbeats, set the threshold in the script to 80% and alert immediately — disk quota hard limits cause data loss, not just slowness.
5. For CPU throttling, set consecutive failures to 3 — sustained CPU throttling is a problem; a brief spike is normal.

Group monitors by host with Vigilmon tags so you can see OpenVZ-host-1 vs. OpenVZ-host-2 at a glance in the dashboard.

Conclusion

OpenVZ's shared-kernel architecture is a double-edged sword: extremely efficient when healthy, catastrophically blast-radius-heavy when the host kernel fails. Traditional per-app monitoring misses the host-level signals that precede container failures — User Beancounter exhaustion, disk quota saturation, and I/O saturation affect all VEs simultaneously but show up as individual app failures. With Vigilmon monitoring the host kernel health, per-VE resource UBC failures, disk quotas, and live migration success alongside per-container HTTP endpoints, you build the complete observability picture that OpenVZ operations demand.