The rtpengine SRTP Replay Protection Trap: Why Out-of-Order Packets
calricon / field note
Discover why RFC 3711 SRTP replay protection and sequence number
The rtpengine SRTP Replay Protection Trap: Why Out-of-Order Packets Drop Media Streams
In carrier-grade telecommunications and enterprise WebRTC gateways running Sipwise rtpengine, securing media streams requires end-to-end encryption via the Secure Real-time Transport Protocol (SRTP). While initial call establishment and cryptographic key exchanges negotiate cleanly, voice engineering teams frequently observe an elusive media degradation: during active calls, connected audio abruptly drops into one-way silence or stutters into robotic, metallic distortion. Network packet captures show encrypted UDP packets arriving at the network interface at full wire rate, yet downstream endpoints output complete silence. The issue is rooted in RFC 3711 SRTP replay protection mechanisms, where packet reordering, network jitter, and dynamic codec renegotiations cause legitimate audio frames to fall outside sliding bitmask windows, triggering silent packet drops inside rtpengine decryption routines.
Problem Diagnosis & Root Cause Architecture
The failure occurs because SRTP enforces strict cryptographic sequence tracking to prevent malicious packet duplication attacks. In high-concurrency production networks, this security layer interacts destructively with non-linear network behaviors:
- RFC 3711 Sliding Replay Window: To prevent replay attacks, the SRTP receiver maintains a replay list using a sliding window bitmask (typically 64 or 128 packets wide). As incoming encrypted packets arrive, the receiver tracks the highest sequence number received (
s_l) and checks whether incoming packet sequence numbers fall within the window and have not been previously observed. - Packet Reordering Across Mobile and Multi-Path WANs: In cellular (LTE/5G), Wi-Fi roaming, or multi-queue network interface cards (NICs), UDP packets frequently experience route flutter and packet reordering. If a delayed packet arrives after the receiver has advanced the sliding window by more than 64 frames (a mere 1.28 seconds at standard 20ms audio packetization), the packet is classified as an old replay attempt and discarded.
- Sequence Number Disruption During Re-INVITEs: When calls transition between hold states, execute mid-dialog transfers, or dynamically transcode between codecs (e.g., Opus to G.711), rtpengine resets or jumps RTP sequence numbers. If the cryptographic context fails to synchronize sequence numbering atomically with the new SSRC state, the decryption layer encounters an index mismatch and rejects all subsequent frames.
- Silent Kernel-Space Packet Dropping: In deployments utilizing the
xt_RTPENGINEkernel module, SRTP decryption and replay checks occur directly within kernel softirq contexts. When replay verification fails, the kernel drops the packet immediately without incrementing user-space application error counters. Voice monitoring dashboards report 0% network packet loss while callers suffer complete audio loss.
"SRTP replay protection operates on the strict assumption of bounded packet arrival order. When route switching, network jitter, or mid-call transcoding disrupt sequence continuity, the decryption engine treats valid voice packets as replay attacks and silently drops the media stream."
Why Standard Implementations Fail
Increasing User-Space Jitter Buffers
Deploying larger jitter buffers on downstream softphones or PBX endpoints does not resolve the issue. Replay protection operates at the transport decryption layer *before* RTP payloads are fed into the jitter buffer. If rtpengine or the receiving endpoint drops the packet during SRTP verification, the audio frame never reaches the jitter buffer, rendering buffer expansion useless.
Switching from SDES to DTLS-SRTP Keying
Migrating from static SDES crypto attributes to dynamic DTLS-SRTP key negotiation resolves signaling eavesdropping concerns, but does not alter the underlying SRTP anti-replay verification algorithm. DTLS merely negotiates the master key and salt; once the SRTP session begins, RFC 3711 sliding window mechanics remain identical and equally vulnerable to packet reordering.
Disabling Kernel Mode (Falling Back to User-Space)
Forcing rtpengine to run strictly in user-space (setting table = -1) reduces kernel-to-user-space handoff race conditions, but increases CPU utilization by up to 300% under high call concurrency. Furthermore, user-space libsrtp libraries implement the exact same RFC 3711 replay window checks, continuing to discard delayed packets under high-jitter network conditions.
Production-Tested Step-by-Step Engineering Playbook
Step 1: Diagnose SRTP Replay Drops via Kernel and Media Telemetry
Determine whether audio drops and packet loss correlate with SRTP decryption rejections. Inspect rtpengine procfs statistics and console log streams for replay failure indicators.
# Inspect real-time kernel-space rtpengine packet drop statistics
cat /proc/rtpengine/0/list | grep -E "drop|mismatch|errors" | sed 20q
# Sample output demonstrating replay drops on active media streams:
# local inet4 192.168.10.50:20138 expect inet4 198.51.100.25:34120
# stats: 17280 bytes, 101 packets, 34 errors
# SSRC in: 93c7fa89 option: RTP option: SSRC tracking
# replay check failed (index too old or duplicate)
# Enable detailed debug logging on rtpengine daemon
rtpengine-ctl set log-level 7
Step 2: Configure Kamailio rtpengine Flags for Sequence Preservation
Modify Kamailio routing script logic to pass explicit flags to rtpengine_manage() or rtpengine_offer() that enforce SSRC preservation, prevent sequence resets during mid-call updates, and normalize crypto attributes.
# /etc/kamailio/kamailio.cfg (rtpengine routing block)
route[MANAGE_RTPENGINE] {
if (is_method("INVITE") || is_method("UPDATE")) {
if (has_totag()) {
# Mid-dialog re-INVITE: preserve existing SSRC and sequence numbers
$var(rtp_flags) = "replace-origin replace-session-connection trust-address "
+ "codec-strip=all codec-offer=PCMU,PCMA "
+ "asymmetric-crypto pad-crypto SSRC-map";
rtpengine_manage("$var(rtp_flags)");
} else {
# Initial call setup
$var(rtp_flags) = "replace-origin replace-session-connection trust-address "
+ "RTP/SAVPF asymmetric-crypto pad-crypto";
rtpengine_manage("$var(rtp_flags)");
}
}
}
Step 3: Configure rtpengine SSRC Tracking and Kernel Table Parameters
Update the core rtpengine configuration file (/etc/rtpengine/rtpengine.conf) to configure robust SSRC tracking, prevent premature stream teardowns, and optimize kernel packet processing.
# /etc/rtpengine/rtpengine.conf
[rtpengine]
table = 0
interface = internal/192.168.10.50;external/198.51.100.10!198.51.100.10
listen-ng = 127.0.0.1:2223
port-min = 20000
port-max = 40000
# Prevent SSRC flapping during dynamic codec adjustments
ssrc-tracking = true
# Maintain active crypto context across re-INVITE sequences
crypto-suite = AES_CM_128_HMAC_SHA1_80
# Tune epoll worker threads to prevent packet reordering on multi-core hosts
num-threads = 16
Step 4: Implement Replay Window Tuning for High-Jitter Mobile Networks
For network legs terminating mobile networks or high-jitter public WANs, configure rtpengine to handle out-of-order packets by expanding the acceptable sequence variance or disabling anti-replay checks on trusted peering interconnects.
# Configure Kamailio flags for mobile endpoints with high packet jitter
route[MOBILE_CARRIER_INTERCONNECT] {
# Disable strict anti-replay validation on trusted transit interconnects
# to accommodate wide cellular packet reordering
$var(transit_flags) = "replace-origin replace-session-connection "
+ "direction=external direction=internal "
+ "trust-address rtp-symmetric "
+ "no-srtp-replay";
rtpengine_manage("$var(transit_flags)");
}
Restart rtpengine and reload Kamailio configurations to apply the updated packet sequencing and replay parameters:
systemctl restart rtpengine
kamctl reload
Step 5: Production Verification and Out-of-Order Media Injection Testing
Validate that rtpengine processes out-of-order and jittered SRTP streams without packet drops using a synthetic traffic generator that introduces controlled sequence reordering.
# Inject simulated out-of-order SRTP traffic using an automated media test script
python3 /opt/testing/srtp_reorder_test.py \
--target="198.51.100.10:20138" \
--jitter_ms=120 \
--reorder_ratio=0.15 \
--duration_sec=60
# Inspect active call statistics via rtpengine CLI
rtpengine-ctl list calls
# Verify zero replay errors recorded in kernel module statistics
cat /proc/rtpengine/0/list | grep -c "errors: 0"
# Expected verification output: Non-zero positive count matching active streams
# Confirm audio stream continuity without packet dropouts
tcpdump -i eth0 -n -s 0 "udp portrange 20000-40000" -c 1000 | sed 20q
By enforcing SSRC-map flags across re-INVITEs, enabling ssrc-tracking, and selectively applying no-srtp-replay on high-jitter mobile transit legs, rtpengine eliminates SRTP decryption rejections. Media streams flow continuously across network handoffs and dynamic codec adjustments with zero audio drops and pristine voice clarity.
Experiencing this issue in your production systems?
Calricon conducts senior-level VoIP, SIP trunking, Asterisk, and cloud infrastructure audits. We review your live configs, identify packet drops or contention bottlenecks, and provide exact remediation runbooks.