RTP

Overview

This repository contains the unified, lightweight RTP monorepo designed for high-performance WebRTC video conferencing. It consolidates N2O WebSocket signaling pages, session authentication, room process supervisors, Mnesia persistence, and in-process GStreamer compositor port drivers into a single cohesive Erlang/OTP application. The system enforces a centralized Multipoint Control Unit (MCU) model where the GStreamer compositor decodes all upstream feeds, composites them into a single 1920×1080 grid, re-encodes the composite, and broadcasts a single stream to every participant, ensuring O(1) client bandwidth and decoding complexity.

Architecture

The system is organized into three distinct layers: the browser client (SPA), the Erlang/OTP control plane, and the C99 GStreamer media plane. All participants of a given room are bound to the same Erlang node via Ingress sticky sessions, eliminating inter-node cluster traffic and allowing horizontal scaling as share-nothing pods.


SPA ── TCP:8082 WSS SDP/ICE ──► rtp_signaling
                                      │
                             rtp_coordinator
                                      │
                               rtp_broker
                                      │ UNIX pipe stdin JSON
                                 priv/gst (C99)
                                      │ UNIX pipe stdout SDP/ICE
                             rtp_broker
                                      │ Erlang msg via Syn
                                 rtp_signaling ── TCP:8082 WSS push ──► SPA

Browser  ══ UDP dynamic DTLS-SRTP camera ══► priv/gst (MCU)
priv/gst ══ UDP dynamic DTLS-SRTP composite ══► Browser
priv/gst ══ HTTP TCP:8081 HLS segments ══► cast.js Viewer

SPA ··► UDP/TCP:3478 STUN/TURN (optional) ··► relay DTLS-SRTP ··► priv/gst

Three stream types:

1. RTP upstream — each browser's camera and microphone sent as DTLS-SRTP over UDP (dynamic ICE-negotiated port) to the GStreamer MCU webrtcbin.
2. RTP MCU composite — the 1920×1080 H.264 composite and mixed Opus audio broadcast back to each participant as a single DTLS-SRTP downstream, plus HLS segments over HTTP TCP:8081 to passive cast.js viewers.
3. TURN relay (optional) — when direct ICE paths fail behind symmetric NATs, media is relayed through eturnal on UDP/TCP:3478 (TURNS on TCP:5349).

Directory Blueprint


├── c_src/
│   └── gst.c                 GStreamer WebRTC MCU compositor — C99, 668 lines
├── include/
├── priv/
│   ├── gst                   Compiled native C99 binary spawned by Erlang port
│   └── static/
│       ├── app/
│       │   ├── index.htm     Conference participant page — WebRTC + N2O chat
│       │   ├── bcast.htm     HLS broadcast viewer — passive observer
│       │   └── login.htm     Session login page
│       ├── js/
│       │   ├── rtc.js        WebRTC client: peer connection, SDP/ICE, telemetry
│       │   └── cast.js       HLS viewer: hls.js player, retro/live seek, telemetry
│       └── css/
│           ├── blank.css     Base Synrc CSS Layer
│           └── color.css     Custom Synrc CSS Layer
├── lib/
│   ├── hls.ex                HTTP Live Streaming handler — strips ETag, no-store
│   ├── n2o.ex                N2O Bandit WebSocket proxy
│   ├── static.ex             Plug.Static asset server — port 8081
│   └── ws.ex                 WebSocket server — port 8082 → rtp_signaling
└── src/
    ├── rtp.app.src           Erlang/OTP Manifest
    ├── rtp_routes.erl        N2O URL router
    ├── rtp_room.erl          N2O Nitro page: room chat history, member list
    ├── rtp_login.erl         N2O Nitro page: session token issuance and redirect
    ├── rtp_broker.erl        gen_server: GStreamer port lifecycle and Port IPC bridge
    ├── rtp_store.erl         gen_server: Mnesia schema init, per-room chat tables
    ├── rtp_signaling.erl     WebSock handler: SDP/ICE signaling, peer registration
    ├── rtp_coordinator.erl   gen_server: room state, participants, media delegation
    ├── rtp_app.erl           OTP Application: listeners, Syn scopes, session table
    ├── rtp_sup.erl           OTP Supervisor one_for_one: rtp_store worker
    └── rtp_syn.erl           N2O MQ backend: Syn v3 pub/sub as N2O pool registry

Erlang/OTP Control Plane

rtp_app.erl — Application bootstrap. Configures N2O on port 8082, calls kvs:join(), registers Syn scopes (rooms, n2o_mq), spawns Bandit listeners (WS:8082, HTTP:8081), and starts rtp_sup. Startup banner reports MaxRooms = cores × 10, RoomCapacity = 50.

rtp_signaling.erl — WebSock handler implementing Elixir.WebSock. Dispatches incoming JSON frames:


Client Message              Handler Action
─────────────────────────────────────────────────────────────────
{"type":"ready"}            rtp_coordinator:originate_video/3
{"type":"get_room_info"}    push room_info with started_at + hls_format
{"type":"get_peers"}        push peer_list
{"type":"ping"}             no-op keep-alive
{"sdp":{"type":"answer"}}   forward SDP answer to rtp_coordinator
{"candidate":{...}}         forward ICE candidate to rtp_coordinator

On terminate/2, calls rtp_coordinator:peer_left/2 and unregisters from Syn.

rtp_coordinator.erl — Per-room gen_server registered in the rooms Syn scope under the binary room ID.


-record(state, {
    room_id      :: binary(),
    participants = [] :: list(),           % Active member maps: #{id, pid}
    publishers   = [] :: list(),           % Active media publishers
    media_broker = undefined :: pid() | undefined
}).

Handles: join, leave, chat, originate_video, sdp_answer, ice_candidate, peer_left, terminate_room, get_started_at, get_peers. On terminate, unregisters from Syn and stops the media broker.

rtp_broker.erl — Per-room-group gen_server managing the GStreamer OS port lifecycle. Spawns priv/gst on first peer join:


open_port({spawn_executable, Binary}, [
    binary, stream, {args, [OutDir, FormatStr]},
    use_stdio, stderr_to_stdout, exit_status,
    {line, 16384},
    {env, [{"GST_GL_WINDOW", "none"},
           {"GST_PLUGIN_FEATURE_FILTER", "opengl:0,applemedia:0"}]}
])

Decodes stdout JSON lines from GStreamer and dispatches SDP offers and ICE candidates to peer signaling processes via syn:lookup. After recording_started, polls index.m3u8 every 100 ms (up to 100 s) then broadcasts room_info to all peers. Monitors client WebSocket PIDs — a 'DOWN' message triggers clean departure. Last-peer departure closes the port, terminating the GStreamer process.

rtp_store.erl — Singleton gen_server initializing Mnesia. Per-room tables (chat_room_<RoomId>) are created lazily as ordered_set disc_copies, keyed by {room_id, timestamp}.

rtp_syn.erl — N2O MQ backend over Syn v3, replacing Redis:


send(Pool, Message) ->
    syn:publish(n2o_mq, term_to_binary(Pool), Message).
reg(Pool, _Value) ->
    syn:join(n2o_mq, term_to_binary(Pool), self()).

GST MCU IPC Protocol

All inter-process communication between the Erlang control plane and the C99 media plane uses newline-delimited JSON over UNIX stdio. The stdin reader is registered as a GLib IO channel watch, serializing all pipeline mutations on the GLib main loop thread.


Erlang → GStreamer (stdin)
─────────────────────────────────────────────────────────────────
{"type":"peer_joined","peer_id":"..."}     setup_peer()
{"type":"sdp_answer","peer_id":"...","sdp":"..."}
{"type":"ice_candidate","peer_id":"...","candidate":{...}}
{"type":"peer_left","peer_id":"..."}       cleanup_peer()
{"type":"exit"}                            EOS → mux finalize

GStreamer → Erlang (stdout)
─────────────────────────────────────────────────────────────────
{"type":"sdp_offer","peer_id":"...","sdp":"..."}
{"type":"ice_candidate","peer_id":"...","candidate":{...}}
{"type":"recording_started"}

GStreamer Pipeline Variants

The static pipeline backbone maintains a black videotestsrc on compositor.sink_0 and a silent audiotestsrc on audiomixer.sink_0, preventing scheduler stalls when no peers are connected. Three output formats are supported:


Format    Video Encoder                 Audio Encoder     Sink
──────────────────────────────────────────────────────────────────────────────
ts        x264enc→h264parse→rtph264pay  opusenc (WebRTC)  hlssink2
(default) +→hlssink2                    avenc_aac (HLS)   2s segments
fmp4      x264enc→h264parse→rtph264pay  opusenc (WebRTC)  mp4mux
          +→mp4mux                      avenc_aac (mux)   fragment-duration=1000
hevc      x264enc (WebRTC)              opusenc (WebRTC)  hlssink2
          x265enc (HLS)                 avenc_aac (HLS)   H.265 video

HLS caching: Rtp.LiveStream intercepts all .m3u8 requests and injects Cache-Control: no-store, no-cache, must-revalidate, max-age=0 to prevent 304 responses from starving hls.js of new segment announcements.

PTS/DTS integrity: The HLS storage branch tees after h264parse and before rtph264pay. Routing H.264 through an RTP payload/depayload cycle corrupts timestamps, causing MSE decoders to freeze permanently.

Disk-I/O isolation: Storage branches use queue max-size-time=30000000000 leaky=2 (30 s), absorbing filesystem stalls without dropping frames from the live broadcast path.

Configuration and Ports


Port  Protocol         Purpose
──────────────────────────────────────────────────────────────────
8082  WebSocket/TCP    N2O signaling — SDP/ICE exchange
8081  HTTP/TCP         Plug.Static — priv/static/ assets and HLS
3478  UDP/TCP          eturnal STUN/TURN — NAT traversal
5349  UDP/TCP          eturnal TURNS — STUN/TURN over TLS

Mnesia directory: /var/lib/rtp/mnesia (Kubernetes PVC), fallback ./mnesia_data for local development.

ITU-T Standards Alignment


Standard  Description                          System Mapping
────────────────────────────────────────────────────────────────────────────
H.264     AVC video codec                      x264enc — WebRTC + HLS
H.265     HEVC video codec                     x265enc — HEVC HLS pipeline
H.323     Packet multimedia systems            rtp_coordinator mirrors H.323 MCU
H.235.8   SRTP key exchange via signaling      DTLS-SRTP via webrtcbin
H.239     Role management (presenter/viewer)   role field in rtp_signaling state
H.245     Control protocol for multimedia      SDP — RFC 8866 / RFC 3264
G.711     PCM 64 kbit/s audio                  WebRTC baseline audio
G.722     7 kHz wideband audio                 WebRTC HD voice
T.124     Generic conference control           rtp_coordinator gen_server
X.601     Multi-peer communications framework  N2O WebSocket + Syn room scope
X.603     Relayed multicast protocol           Erlang Port IPC stdin/stdout relay

Getting Started

brew install gstreamer libnice libnice-gstreamer json-glib erlang
cc -O3 c_src/gst.c -o priv/gst \
  $(pkg-config --cflags --libs \
    gstreamer-1.0 gstreamer-webrtc-1.0 gstreamer-sdp-1.0 json-glib-1.0)
iex -S mix

╔════════════════════════════════════════════════════════╗
║  ERP/1: RTP Server / Signaling & Telemetry             ║
║  WS  : ws://localhost:8082/ws/app/<page>.htm           ║
║  HTTP: http://localhost:8081/app/login.htm             ║
╚════════════════════════════════════════════════════════╝
  Hardware   : 10 Cores, 16 GB RAM
  Max Rooms  : 100 (heuristic based on cores)
  Capacity   : 5000 max participants (50 per room)
  RTP Codecs : Opus (Audio), VP8, VP9, H.264 (Video)

1> application:which_applications().
[{rtp,"Unified WebRTC Video chat rooms & signaling monolith","0.1.0"},
 {mnesia,"MNESIA  CXC 138 12","4.26.1"},
 {eturnal,"STUN/TURN server","1.12.0"},
 {syn,"A scalable global Process Registry and Process Group manager.","3.4.2"},
 {kvs,"KVS Abstract Chain Database","9.4.1"},
 {nitro,"NITRO Nitrogen Web Framework","7.10.0"},
 {n2o,"N2O MQTT TCP WebSocket","10.3.3"},
 {bandit,"A pure-Elixir HTTP server built for Plug & WebSock apps","1.12.0"},
 {plug,"Compose web applications with functions","1.20.3"},
 {thousand_island,"A simple & modern pure Elixir socket server","1.5.0"},
 {telemetry,"Dynamic dispatching library for metrics","1.4.2"}]

Namdak Tonpa. RTP: A Minimal MCU Gateway of ANSI C99 with GStreamer for High-Density Video Conferencing on Erlang/OTP Control Plane in Alpine Linux under Kubernetes. 2026/

Namdak Tonpa. Deterministic Real-Time Multimedia Synchronization: Architecting Low Latency Topologies in GStreamer and WebRTC. 2026.

Namdak Tonpa. Architecture of GStreamer WebRTC MCU Media Compositor for Chat Applications. 2026.

Namdak Tonpa. NuStream: A Lightweight, Predictable Deterministic Real-Time Media Pipeline with GStreamer-like API for NuttX RTOS. 2026.