calricon.
VoIP & Telecom

The Kamailio Timer Process Lockup Trap: Why Heavy Routing Logic

Nov 7, 2026/5 min read/Verified Production Playbook

calricon / field note

Discover why shared timer processes in Kamailio stall under

Nov 7, 2026/VoIP & Telecom
324
Advertisement

The Kamailio Timer Process Lockup Trap: Why Heavy Routing Logic Drops SIP Keepalives

Telecommunications engineers deploying high-capacity Kamailio SIP edge proxies and registrars frequently encounter mysterious spikes in client de-registrations, delayed dispatcher gateway failovers, and dropped SIP OPTIONS pings. Under peak production load, registered endpoints randomly transition to unreachable states, even while Kamailio worker child processes show minimal CPU utilization and network bandwidth remains clear. System logs begin flooding with warnings such as WARNING: core [core/timer.c:646]: utimer_ticker(): utimer task took too much time.

The root cause lies in Kamailio's default single-threaded timer architecture. Core housekeeping tasks (such as SIP transaction retransmissions, NAT keepalive pings via nathelper, dispatcher health checks, and dialog expiration tracking) share a single timer execution loop. When administrators attach custom routing logic, external database synchronization, or blocking API requests to periodic timer routes, the entire global timer process freezes. While the timer is blocked, Kamailio fails to send scheduled SIP keepalives and transaction timeouts, causing endpoints to drop registrations and gateways to trigger false failovers.



Problem Diagnosis & Root Cause Architecture

To understand how periodic background jobs impact real-time signaling, one must examine Kamailio's internal process model and timer scheduling mechanisms.




  • Shared Core Timer Process (Process 1): By default, Kamailio allocates a single dedicated timer process (and a single fast-timer microsecond process) to execute all module callbacks registered for periodic execution. This includes tm transaction retransmissions, usrloc contact expiration cleanup, and nathelper OPTIONS pinging.

  • Blocking I/O in Timer Handlers: When modules like rtimer, sqlops, or http_client run inside a periodic timer route (such as syncing active registration metrics or cleaning up CDRs), any database stall or slow HTTP response blocks the timer process thread.

  • Cascading Ticker Delay: The core timer loop executes tick handlers sequentially. If a custom job consumes 1500 milliseconds, every subsequent timer task is delayed by that duration. Microsecond timer tasks (utimer_ticker) back up rapidly, accumulating a backlog that prevents transaction retransmissions and client probing from dispatching on schedule.

  • Client NAT Pinhole Expiration: Mobile softphones and desk phones behind NAT firewalls depend on Kamailio sending periodic UDP SIP OPTIONS or 4-byte CRLF pings every 20 to 30 seconds to maintain router pinholes. When the timer process locks up for tens of seconds, client firewalls close the UDP NAT mapping, cutting off inbound call delivery and causing instant de-registrations.



Attaching synchronous external lookups or heavy batch routines to Kamailio's default timer pipeline serializes real-time signaling tasks behind slow I/O. Without dedicated timer worker pools, background housekeeping directly cripples active call control.


Why Standard Implementations Fail

Increasing Worker Children Count

Administrators often attempt to resolve timer sluggishness by raising the general SIP listener worker count (children = 32 or children = 64). While this increases concurrency for processing incoming UDP or TCP packets, it has zero impact on timer tasks. Core timers and module periodic routes run in dedicated supervisor processes completely decoupled from the SIP packet worker children.



Shortening Module Timer Intervals

Reducing intervals for tasks (e.g., lowering dispatcher probing from 10 seconds to 2 seconds or NAT pinging from 30 seconds to 10 seconds) actually accelerates failure. A struggling timer process cannot complete its current queue before the next scheduled interval fires, worsening queue contention and multiplying timer task took too much time warnings.



Executing External Shell Scripts via exec_msg

Triggering external shell scripts or background utilities directly from timer routes via exec_msg() or exec_avp() spawns subshells. Process creation overhead and system forks within the single timer thread amplify CPU context switching and lengthen timer delays.



Authenticated Step-by-Step Resolution Playbook

Step 1: Configure Dedicated Timer Processes in kamailio.cfg

Scale the internal timer architecture by defining dedicated timer processes using the core timer_procs parameter, isolating critical signaling timers from user-defined maintenance routines.



# /etc/kamailio/kamailio.cfg
# Core timer concurrency settings
timer_procs = 4

# Increase memory limits for timer work queues
timer_max_loops = 200

# Enable verbose slow-timer execution warnings (threshold in microseconds)
timer_warn_latency = 100000


Step 2: Isolate Periodic Maintenance Using Dedicated rtimer Profiles

Decouple custom database synchronization, metrics reporting, and batch updates from the core timer by binding the rtimer module to an isolated background process.



# Load and configure rtimer with dedicated process execution
loadmodule "rtimer.so"
loadmodule "async.so"
loadmodule "sqlops.so"

# Define isolated timer rule running in its own dedicated process
modparam("rtimer", "timer", "name=sync_job;interval=60;mode=1;")
modparam("rtimer", "exec", "timer=sync_job;route=MAINTENANCE_SYNC")

# Isolate NAT keepalive pinging from custom jobs
modparam("nathelper", "natping_interval", 25)
modparam("nathelper", "natping_processes", 2)
modparam("nathelper", "ping_nated_only", 1)


Step 3: Offload Heavy Tasks to Asynchronous Event Workers

Wrap blocking database queries or external HTTP operations in asynchronous task workers using the async module, preventing timer thread blocking even if the backend database experiences latency.



route[MAINTENANCE_SYNC] {
# Offload execution to asynchronous worker thread pool
async_task_route("ASYNC_DB_SYNC");
}

route[ASYNC_DB_SYNC] {
# Execute database maintenance or metrics push safely
sql_query("cb", "UPDATE active_registrations SET last_seen = NOW() WHERE expires > NOW()", "ra");
sql_free_result("ra");
}

Configure the asynchronous worker pool size in the core configuration:



# Asynchronous framework worker threads
modparam("async", "workers", 4)


Step 4: Verify Timer Responsiveness in Production

Check process status and monitor timer latency metrics via Kamailio RPC (kamcmd) to verify that timers execute cleanly within microsecond tolerances.



# Check active Kamailio process list and timer worker allocations
kamcmd core.psx | grep -i timer

# Inspect core timer latency and warning statistics
kamcmd stats.get_statistics core:timer
kamcmd stats.get_statistics nathelper:

# Verify absence of timer delay warnings in system journals
journalctl -u kamailio -g "timer task took too much time" --since "1 hour ago"

# Confirm NAT keepalive OPTIONS packets dispatch continuously
sngrep --sip-type options "port 5060"

By partitioning core timers across multiple processes, provisioning dedicated NAT pinging workers, and offloading heavy tasks to asynchronous threads, Kamailio maintains steady keepalive intervals and uninterrupted transaction scheduling under peak loads.

Advertisement
Engineering Advisory 48-Hour Audit Turnaround

Experiencing this issue in your production systems?

Calricon conducts senior-level VoIP, SIP trunking, Asterisk, and cloud infrastructure audits. We review your live configs, identify packet drops or contention bottlenecks, and provide exact remediation runbooks.