tutorial

How to Monitor BIRD Routing Daemon Health with Vigilmon

BIRD daemon crashes drop BGP sessions and withdraw routes from the global table, OSPF adjacency failures partition your network, and memory growth from route leaks silently exhaust RAM until the OOM killer terminates BIRD and takes all routing with it. Learn how to monitor BIRD process health, BGP session states, OSPF adjacency status, and route table integrity with Vigilmon HTTP probes and heartbeat monitors.

BIRD Internet Routing Daemon is the routing software engine for ISPs, IXPs, and large-scale network deployments — managing BGP sessions to hundreds of peers, OSPF adjacencies across internal infrastructure, and BFD sessions for fast failure detection. When the BIRD daemon crashes due to a filter compilation error introduced in a configuration update, all BGP sessions established through BIRD are torn down simultaneously and all routes learned from those peers are withdrawn from the kernel routing table — packets that were flowing through optimized BGP-selected paths are either dropped or fall back to static routes configured outside BIRD, causing partial connectivity failure across the network; when a single BGP peer's session drops from Established to Active because a firewall rule change blocked TCP port 179, BIRD silently removes all routes received from that peer and may trigger NEXT_HOP unreachability for a large prefix set that other peers announced with that peer as the next hop; when BIRD's memory usage grows unboundedly because a full-table BGP peer is re-announcing routes with rapidly rotating MED attributes and BIRD's route table is accumulating suppressed-but-stored entries from all import filters, the process eventually consumes all available RAM and the OOM killer terminates it — taking all BGP and OSPF routing with it. These are network-level routing failures with traffic-impact blast radius that require external monitoring of BIRD process health, session state, and memory to catch before they become outages.

Vigilmon gives you external visibility into BIRD's routing health through HTTP probe monitoring and heartbeat monitors for BGP session state, route counts, and memory usage. This tutorial covers both.

Why BIRD Needs External Monitoring

BIRD failure modes have network-wide routing impact:

  • BIRD daemon crash: The bird process (BIRD2) maintains all BGP/OSPF/RIP sessions and routing tables in memory; a crash from filter bytecode corruption, a null pointer in protocol handling, or an OOM kill removes all dynamically learned routes from the kernel routing table at once; traffic that was routed through BIRD-managed paths is dropped; unlike BGP graceful restart (which requires peer-side support), a sudden BIRD crash is an immediate routing failure
  • BGP session state leaving Established: BIRD BGP sessions cycle through Idle → Connect → Active → OpenSent → OpenConfirm → Established; any state other than Established means the peering is not exchanging routes; a single full-table upstream BGP peer dropping to Active causes BIRD to withdraw that peer's prefixes from the master routing table — the scope of impact is proportional to how many routes that peer was the best path for
  • Route count drop from a peer: Even when a BGP session remains Established, the prefix count can drop — due to peer-side route filtering changes, withdrawals from a route reflector, or a network event upstream; a 10% drop in prefix count from a full-table peer is a significant routing event that may be invisible at the application level until specific destinations become unreachable
  • OSPF adjacency failure: If BIRD is running OSPF for internal routing, a neighbor dropping from Full to Down means LSAs from that segment stop being flooded; routers behind the failed adjacency become unreachable; BIRD logs the adjacency state change but does not automatically notify operators
  • Memory growth from route table bloat: BIRD stores all routes (including non-best paths) from all configured import filters; if import filters accept more routes than expected (misconfigured or missing filter), the route table can grow to consume all available memory; BIRD does not automatically prune or limit table size
  • BFD session failure preceding protocol session failure: BFD (Bidirectional Forwarding Detection) is designed for fast sub-second failure detection; a BFD session going down typically precedes a BGP or OSPF session failure by the BFD detection time; monitoring BFD session state gives early warning of impending routing session failures

External monitoring with Vigilmon adds:

  • Proactive alerting when the BIRD control socket becomes unreachable (daemon crash indicator)
  • BGP session state tracking to catch sessions leaving Established before route withdrawals propagate
  • Route count trending to detect filter changes or upstream withdrawals
  • Memory usage alerting to catch table bloat before OOM termination

Step 1: Build a BIRD Health Endpoint

BIRD does not expose an HTTP health endpoint. Build a sidecar that queries the BIRD control socket via birdc and exposes health over HTTP.

Node.js Health Sidecar

// health/bird.js
const express = require('express');
const { execSync } = require('child_process');
const os = require('os');

const app = express();

const BIRDC_PATH = process.env.BIRDC_PATH || 'birdc';
const BIRD_SOCKET = process.env.BIRD_SOCKET || '/var/run/bird.ctl';
const BGP_MIN_PREFIXES = parseInt(process.env.BGP_MIN_PREFIXES || '0');

function runBirdc(cmd) {
  return execSync(
    `${BIRDC_PATH} -s ${BIRD_SOCKET} '${cmd}' 2>&1`,
    { timeout: 15000 }
  ).toString().trim();
}

function getBgpSessionStates() {
  const output = runBirdc('show protocols all');
  const sessions = [];
  const lines = output.split('\n');
  let currentProto = null;

  for (const line of lines) {
    // Protocol line: "BGP_peer1   BGP      master4    up     2024-01-01  Established"
    const protoMatch = line.match(/^(\S+)\s+BGP\s+\S+\s+\S+\s+\S+\s+(\S+)/);
    if (protoMatch) {
      currentProto = {
        name: protoMatch[1],
        state: protoMatch[2],
        prefixes_imported: 0,
      };
      sessions.push(currentProto);
    }
    // Routes imported line: "  Routes:     823450 imported, ..."
    if (currentProto) {
      const routeMatch = line.match(/Routes:\s+(\d+)\s+imported/);
      if (routeMatch) {
        currentProto.prefixes_imported = parseInt(routeMatch[1]);
      }
    }
  }
  return sessions;
}

function getOspfAdjacencies() {
  try {
    const output = runBirdc('show ospf neighbors');
    const adjacencies = [];
    const lines = output.split('\n');
    for (const line of lines) {
      // "10.0.0.1      2   Full/DR      00:00:07  10.0.0.1     eth0"
      const match = line.match(/(\d+\.\d+\.\d+\.\d+)\s+\d+\s+(\S+)\s+/);
      if (match) {
        adjacencies.push({ neighbor: match[1], state: match[2] });
      }
    }
    return adjacencies;
  } catch {
    return [];
  }
}

function getBirdMemoryMb() {
  try {
    const output = runBirdc('show memory');
    const match = output.match(/Total:\s+(\d+)/i);
    if (match) return Math.round(parseInt(match[1]) / 1024 / 1024);
  } catch {}
  // Fallback: read /proc/PID/status
  const pid = execSync(`pgrep -x bird 2>/dev/null || echo 0`).toString().trim();
  if (pid === '0') return -1;
  const status = require('fs').readFileSync(`/proc/${pid}/status`, 'utf8');
  const vm = status.match(/VmRSS:\s+(\d+)/);
  return vm ? Math.round(parseInt(vm[1]) / 1024) : -1;
}

function getBfdSessions() {
  try {
    const output = runBirdc('show bfd sessions');
    const sessions = [];
    for (const line of output.split('\n')) {
      const match = line.match(/(\d+\.\d+\.\d+\.\d+)\s+\S+\s+(\S+)/);
      if (match) {
        sessions.push({ peer: match[1], state: match[2] });
      }
    }
    return sessions;
  } catch {
    return [];
  }
}

app.get('/health/bird', (req, res) => {
  const checks = {};
  let healthy = true;

  // Control socket / daemon check
  try {
    runBirdc('show status');
    checks.bird_socket = 'ok';
  } catch (err) {
    checks.bird_socket = `unreachable: ${err.message}`;
    healthy = false;
    return res.status(503).json({ status: 'down', checks });
  }

  // BGP session states
  try {
    const bgpSessions = getBgpSessionStates();
    const nonEstablished = bgpSessions.filter(s => s.state !== 'Established');
    checks.bgp_sessions_total = bgpSessions.length;
    checks.bgp_sessions_established = bgpSessions.filter(s => s.state === 'Established').length;
    checks.bgp_sessions_down = nonEstablished.length;
    if (nonEstablished.length > 0) {
      checks.bgp_down_peers = nonEstablished.map(s => `${s.name}(${s.state})`).join(', ');
      healthy = false;
    }
    const totalPrefixes = bgpSessions.reduce((sum, s) => sum + s.prefixes_imported, 0);
    checks.bgp_total_prefixes_imported = totalPrefixes;
    if (BGP_MIN_PREFIXES > 0 && totalPrefixes < BGP_MIN_PREFIXES) {
      checks.bgp_prefix_warning = `Only ${totalPrefixes} prefixes imported (min: ${BGP_MIN_PREFIXES})`;
      healthy = false;
    }
  } catch (err) {
    checks.bgp = `error: ${err.message}`;
  }

  // OSPF adjacencies
  try {
    const adjacencies = getOspfAdjacencies();
    const nonFull = adjacencies.filter(a => !a.state.startsWith('Full'));
    checks.ospf_adjacencies_total = adjacencies.length;
    checks.ospf_adjacencies_full = adjacencies.filter(a => a.state.startsWith('Full')).length;
    if (nonFull.length > 0) {
      checks.ospf_down_neighbors = nonFull.map(a => `${a.neighbor}(${a.state})`).join(', ');
      healthy = false;
    }
  } catch (err) {
    checks.ospf = `error: ${err.message}`;
  }

  // BFD sessions
  try {
    const bfd = getBfdSessions();
    const bfdDown = bfd.filter(s => s.state !== 'Up');
    checks.bfd_sessions_total = bfd.length;
    checks.bfd_sessions_up = bfd.filter(s => s.state === 'Up').length;
    if (bfdDown.length > 0) {
      checks.bfd_down = bfdDown.map(s => `${s.peer}(${s.state})`).join(', ');
      healthy = false;
    }
  } catch (err) {
    checks.bfd = `error: ${err.message}`;
  }

  // Memory usage
  try {
    const memMb = getBirdMemoryMb();
    checks.bird_memory_mb = memMb;
    const totalMb = Math.round(os.totalmem() / 1024 / 1024);
    if (memMb > 0 && memMb > totalMb * 0.5) {
      checks.memory_warning = `BIRD using ${memMb}MB (>${Math.round(memMb / totalMb * 100)}% of RAM)`;
      healthy = false;
    }
  } catch (err) {
    checks.memory = `error: ${err.message}`;
  }

  return res.status(healthy ? 200 : 503).json({
    status: healthy ? 'ok' : 'degraded',
    checks,
  });
});

app.listen(3021, () => console.log('BIRD health sidecar on :3021'));

Python (FastAPI) Alternative

# health/bird_health.py
import os
import re
import subprocess
import json
from fastapi import FastAPI, Response

app = FastAPI()

BIRDC_PATH = os.environ.get("BIRDC_PATH", "birdc")
BIRD_SOCKET = os.environ.get("BIRD_SOCKET", "/var/run/bird.ctl")
BGP_MIN_PREFIXES = int(os.environ.get("BGP_MIN_PREFIXES", "0"))

def run_birdc(cmd: str) -> str:
    result = subprocess.run(
        [BIRDC_PATH, "-s", BIRD_SOCKET, cmd],
        capture_output=True, text=True, timeout=15
    )
    return result.stdout + result.stderr

def check_bird_socket() -> bool:
    try:
        run_birdc("show status")
        return True
    except Exception:
        return False

def get_bgp_sessions() -> list:
    output = run_birdc("show protocols all")
    sessions = []
    current = None
    for line in output.split("\n"):
        proto_match = re.match(r"^(\S+)\s+BGP\s+\S+\s+\S+\s+\S+\s+(\S+)", line)
        if proto_match:
            current = {
                "name": proto_match.group(1),
                "state": proto_match.group(2),
                "prefixes_imported": 0,
            }
            sessions.append(current)
        if current:
            route_match = re.search(r"Routes:\s+(\d+)\s+imported", line)
            if route_match:
                current["prefixes_imported"] = int(route_match.group(1))
    return sessions

def get_ospf_adjacencies() -> list:
    try:
        output = run_birdc("show ospf neighbors")
        adjacencies = []
        for line in output.split("\n"):
            match = re.match(r"(\d+\.\d+\.\d+\.\d+)\s+\d+\s+(\S+)", line)
            if match:
                adjacencies.append({"neighbor": match.group(1), "state": match.group(2)})
        return adjacencies
    except Exception:
        return []

def get_bfd_sessions() -> list:
    try:
        output = run_birdc("show bfd sessions")
        sessions = []
        for line in output.split("\n"):
            match = re.match(r"(\d+\.\d+\.\d+\.\d+)\s+\S+\s+(\S+)", line)
            if match:
                sessions.append({"peer": match.group(1), "state": match.group(2)})
        return sessions
    except Exception:
        return []

def get_bird_memory_mb() -> int:
    try:
        output = run_birdc("show memory")
        match = re.search(r"Total:\s+(\d+)", output, re.IGNORECASE)
        if match:
            return int(match.group(1)) // 1024 // 1024
    except Exception:
        pass
    try:
        pid = subprocess.run("pgrep -x bird", shell=True, capture_output=True,
                             text=True).stdout.strip()
        if pid:
            with open(f"/proc/{pid}/status") as f:
                status = f.read()
            vm = re.search(r"VmRSS:\s+(\d+)", status)
            if vm:
                return int(vm.group(1)) // 1024
    except Exception:
        pass
    return -1

@app.get("/health/bird")
async def bird_health():
    checks = {}
    healthy = True

    if not check_bird_socket():
        checks["bird_socket"] = "unreachable"
        healthy = False
        return Response(
            content=json.dumps({"status": "down", "checks": checks}),
            status_code=503, media_type="application/json"
        )
    checks["bird_socket"] = "ok"

    try:
        sessions = get_bgp_sessions()
        established = [s for s in sessions if s["state"] == "Established"]
        not_established = [s for s in sessions if s["state"] != "Established"]
        checks["bgp_sessions_total"] = len(sessions)
        checks["bgp_sessions_established"] = len(established)
        checks["bgp_sessions_down"] = len(not_established)
        if not_established:
            checks["bgp_down_peers"] = ", ".join(
                f"{s['name']}({s['state']})" for s in not_established
            )
            healthy = False
        total_prefixes = sum(s["prefixes_imported"] for s in sessions)
        checks["bgp_total_prefixes_imported"] = total_prefixes
        if BGP_MIN_PREFIXES > 0 and total_prefixes < BGP_MIN_PREFIXES:
            checks["bgp_prefix_warning"] = f"Only {total_prefixes} prefixes (min: {BGP_MIN_PREFIXES})"
            healthy = False
    except Exception as e:
        checks["bgp"] = f"error: {str(e)}"

    try:
        adjacencies = get_ospf_adjacencies()
        not_full = [a for a in adjacencies if not a["state"].startswith("Full")]
        checks["ospf_adjacencies_total"] = len(adjacencies)
        checks["ospf_adjacencies_full"] = len(adjacencies) - len(not_full)
        if not_full:
            checks["ospf_down_neighbors"] = ", ".join(
                f"{a['neighbor']}({a['state']})" for a in not_full
            )
            healthy = False
    except Exception as e:
        checks["ospf"] = f"error: {str(e)}"

    try:
        bfd = get_bfd_sessions()
        bfd_down = [s for s in bfd if s["state"] != "Up"]
        checks["bfd_sessions_total"] = len(bfd)
        checks["bfd_sessions_up"] = len(bfd) - len(bfd_down)
        if bfd_down:
            checks["bfd_down"] = ", ".join(f"{s['peer']}({s['state']})" for s in bfd_down)
            healthy = False
    except Exception as e:
        checks["bfd"] = f"error: {str(e)}"

    mem_mb = get_bird_memory_mb()
    checks["bird_memory_mb"] = mem_mb

    status_code = 200 if healthy else 503
    return Response(
        content=json.dumps({"status": "ok" if healthy else "degraded", "checks": checks}),
        status_code=status_code, media_type="application/json"
    )

Step 2: Configure Vigilmon Monitoring

HTTP Monitor — BIRD Health Endpoint

In your Vigilmon dashboard, create an HTTP monitor for the BIRD sidecar:

| Field | Value | |-------|-------| | Monitor name | BIRD Routing Daemon Health | | URL | http://router.internal:3021/health/bird | | Method | GET | | Check interval | Every 1 minute | | Expected status | 200 | | Alert threshold | 2 consecutive failures | | Regions | Select 2+ regions for consensus |

Heartbeat Monitor — BGP Session State Tracking

Monitor BGP session states and alert immediately on any non-Established session. Configure a 3-minute timeout heartbeat:

#!/bin/bash
# /etc/cron.d/bird-bgp-sessions — runs every 2 minutes
HEARTBEAT_URL="https://vigilmon.online/hb/your-bird-bgp-sessions-heartbeat"
BIRD_SOCKET="${BIRD_SOCKET:-/var/run/bird.ctl}"

# Get all BGP protocol states
NOT_ESTABLISHED=$(birdc -s "$BIRD_SOCKET" 'show protocols' 2>/dev/null | \
  awk '/BGP/ && $NF != "Established" { print $1, $NF }')

if [ -z "$NOT_ESTABLISHED" ]; then
  curl -fsS --max-time 10 "$HEARTBEAT_URL" > /dev/null 2>&1
else
  echo "BIRD BGP sessions not Established:"
  echo "$NOT_ESTABLISHED"
  # Optionally: post to Slack webhook
fi

Configure with a 3-minute timeout for near-real-time session state alerts.

Heartbeat Monitor — BIRD Memory Usage

Watch for memory growth indicating route table bloat:

#!/bin/bash
# /etc/cron.d/bird-memory — runs every 5 minutes
HEARTBEAT_URL="https://vigilmon.online/hb/your-bird-memory-heartbeat"
MAX_MEMORY_MB="${BIRD_MAX_MEMORY_MB:-4096}"
BIRD_SOCKET="${BIRD_SOCKET:-/var/run/bird.ctl}"

TOTAL_KB=$(birdc -s "$BIRD_SOCKET" 'show memory' 2>/dev/null | \
  grep -i 'Total:' | awk '{ print $2 }' || echo 0)
TOTAL_MB=$((TOTAL_KB / 1024))

if [ "$TOTAL_MB" -lt "$MAX_MEMORY_MB" ]; then
  curl -fsS --max-time 10 "$HEARTBEAT_URL" > /dev/null 2>&1
else
  echo "BIRD memory usage: ${TOTAL_MB}MB (threshold: ${MAX_MEMORY_MB}MB)" >&2
fi

Configure with a 10-minute timeout.

Heartbeat Monitor — Route Flap / Instability

Detect BGP route instability through excessive update messages:

#!/bin/bash
# /etc/cron.d/bird-route-stability — runs every 10 minutes
HEARTBEAT_URL="https://vigilmon.online/hb/your-bird-stability-heartbeat"
BIRD_SOCKET="${BIRD_SOCKET:-/var/run/bird.ctl}"
MAX_DAMPENED=100

# Count routes currently in dampened state (if route flap dampening configured)
DAMPENED=$(birdc -s "$BIRD_SOCKET" 'show route dampened count' 2>/dev/null | \
  grep -oP '\d+' | head -1 || echo 0)

if [ "$DAMPENED" -lt "$MAX_DAMPENED" ]; then
  curl -fsS --max-time 10 "$HEARTBEAT_URL" > /dev/null 2>&1
else
  echo "BIRD: ${DAMPENED} routes currently dampened (threshold: ${MAX_DAMPENED})" >&2
fi

Configure with a 15-minute timeout.

Step 3: Configure Alerting

Alert Policies

BIRD Daemon Down Alert

  • Trigger: Health endpoint returns 503 with bird_socket: unreachable
  • Severity: Critical — P1
  • Channels: PagerDuty (immediate) + Slack #network-critical
  • Message: "BIRD routing daemon is down. All BGP sessions have been torn down and routes withdrawn from the kernel table. Network routing is degraded. Restart with systemctl restart bird and review logs for crash reason."

BGP Session Not Established Alert

  • Trigger: Health endpoint returns 503 with bgp_sessions_down > 0
  • Severity: High
  • Channels: PagerDuty + Slack #network-alerts
  • Message: "One or more BIRD BGP sessions are not in Established state. Routes from affected peers have been withdrawn. Check birdc show protocols all for session details and TCP port 179 connectivity."

BGP Prefix Count Drop Alert

  • Trigger: BGP session heartbeat missed (prefix count below minimum)
  • Severity: High
  • Channels: Slack #network-alerts + email to NOC
  • Message: "BIRD BGP total prefix count has dropped below expected minimum. Route table is incomplete. Specific destinations may be unreachable. Review import filter configuration and upstream peer announcements."

OSPF Adjacency Down Alert

  • Trigger: Health endpoint returns 503 with ospf_down_neighbors > 0
  • Severity: High
  • Channels: PagerDuty + Slack #network-alerts
  • Message: "BIRD OSPF adjacency is not in Full state. Network segments behind the failed adjacency are unreachable. Check physical connectivity and OSPF hello/dead interval configuration."

BIRD Memory Growth Alert

  • Trigger: Memory heartbeat missed (BIRD memory exceeds threshold)
  • Severity: Medium
  • Channels: Slack #network-alerts + email to network team
  • Message: "BIRD routing daemon memory usage has exceeded threshold. Route table may be growing due to missing import filters or route leaks. Review active routes with birdc show route count and check import filter policies."

Key Metrics Summary

| Metric | Alert Threshold | Impact | |--------|----------------|--------| | BIRD daemon / control socket | Any failure | All BGP/OSPF sessions torn down, routes withdrawn | | BGP session state per peer | Any non-Established | Routes from that peer withdrawn from routing table | | BGP prefix count per peer | Drop >10% | Specific destination prefixes becoming unreachable | | OSPF adjacency state | Any non-Full | Network segments behind failed adjacency unreachable | | BFD session state | Any non-Up | Precedes BGP/OSPF session failure by detection time | | BIRD process memory (RSS) | >50% of system RAM | OOM kill risk; route table bloat | | Kernel route table sync | Divergence vs BIRD master | Export filter issue; kernel ignoring BIRD routes | | Routes currently dampened | >100 | Upstream BGP routing instability | | Config reload status | Last reload failure | Running stale routing policy | | BGP notification errors | Any spike | BGP policy violations or misconfigured filters |

Conclusion

BIRD is the routing engine for ISP backbones, IXP route servers, and enterprise BGP deployments — when the daemon crashes or a BGP session drops from Established, the routing impact propagates immediately to the network segments those sessions serve. Vigilmon HTTP probes on the BIRD health sidecar catch daemon failures the moment the control socket becomes unreachable, while heartbeat monitors on BGP session state, prefix counts, and memory usage give you coverage of the gradual routing failures that process monitors miss. Configure the monitors in this tutorial and your network operations team will receive alerts about routing problems before customers report specific destinations becoming unreachable.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles