Edge and Protocol

Audience: Engine Developer Status: ✅ Ready

The edge tier is telos-gate: it terminates a player’s telnet/TLS connection, decodes the telnet byte stream into semantic frames, runs the auth handshake, and bridges the session to the world shard that owns the player’s zone over the Play gRPC stream. The world speaks in semantic frames (output, prompt, GMCP, a raw screen frame, a cross-shard redirect); the gate renders them for the specific terminal. This split is what lets a dumb telnet client and a rich GMCP client share one protocol.

Related: GMCP Reference, RPC & Protobuf, Cross-Shard Handoff, Accounts & Auth Internals. For deployment/firewall posture see Deployment.

The Play bidi stream

One gRPC bidirectional stream per connected player: rpc Connect(stream ClientFrame) returns (stream ServerFrame) (api/proto/telosmud/play/v1). The gate is the client, the world is the server.

Client → world (ClientFrame oneof): Attach (always the first frame — enforced world-side, else InvalidArgument), InputLine (a sequenced typed line), plus GmcpIn, Resize, Pong, Detach.

World → gate (ServerFrame oneof): Attached (bind ack), Output (markup + OutputClass), GmcpOut, PromptUpdate, Control (echo/mssp), Ping, Redirect, Disconnect, and Screen (raw terminal bytes). One field, ack_input_seq, lives outside the oneof so it piggybacks on every server frame — it is the highest InputLine.seq the shard has applied, and it drives input replay (below).

The Attach payload carries the session_id (gate-generated, stable across a redirect), account_id/character_id, a handoff_token (set only on a re-dial after a Redirect), the input_seq resume point, and the Ed25519-signed session_assertion the world verifies offline. The Screen frame carries pre-formed raw bytes (cursor/erase/scroll) written verbatim, bypassing the gate’s sanitizer and color renderer — safety is by provenance (only engine output or a trust-gated screen.* capability emits it; player text never reaches it).

Some of the proto is still ahead of the implementation. Mark these reserved, not live: Attach.client (ClientInfo) is never populated; the gate sends Attach, InputLine, and GmcpIn (the last only for a whitelisted inbound request — see GMCP Reference → Inbound requests), but never Resize/Pong/Detach (disconnect is CloseSend/EOF); the world reader consumes Input, Detach, and a whitelisted GmcpIn, dropping Resize/Pong; and the gate’s renderFrame has no case for Ping or Control (so the Ping/Pong keepalive and the echo/mssp channel are unwired), never reads Output.preformatted, and does no width/NAWS word-wrapping.

Connect + auth sequence 

sequenceDiagram
    participant C as Player (telnet/TLS)
    participant G as telos-gate
    participant A as telos-account
    participant W as telos-world (zone)
    C->>G: connect
    G->>A: device-code OAuth (browser bridge)
    A-->>G: signed session assertion
    G->>W: Play stream — ClientFrame{Attach: assertion, input_seq}
    W->>W: verify assertion offline, load durable snapshot, bind its durable zone
    W-->>G: ServerFrame{Attached: shard_id} (ack_input_seq)
    loop play
        C->>G: keystrokes
        G->>W: ClientFrame{InputLine: seq, text}
        W-->>G: ServerFrame{Output/Prompt/GmcpOut} (ack_input_seq)
        G-->>C: rendered text / GMCP
    end

Session lifecycle & reconnection

The session is minted once at login (newSession(uuid)), stable across every re-dial; the assertion and color preference are resolved once. The world’s single writer goroutine per stream is the only caller of stream.Send. A single-session lock is heartbeated fleet-wide — a newer login anywhere kicks the older connection (displacedKick).

A draining shard bounces an arriving login, and the gate re-homes it. The zero-drop drain keeps resident players connected, but a brand-new login can still land on a shard that has begun draining — which refuses the attach with codes.Unavailable, expecting the gate to re-resolve. So on the first Recv error the stream returns a structured outcome, and a fresh-login Unavailable (first frame, no handoff token) is not treated as terminal: the gate keeps the socket open, re-resolves the character through the directory (whose zone leases have flipped to the peer), and re-dials — bounded to a few attempts with a short backoff before it gives up and asks the player to reconnect. A token-bearing (handoff) re-dial is never retried: an in-flight handoff has exactly one valid destination, and retrying would race the pending-player TTL.

The gate’s only durable per-player state is the session: a stable id, a nextSeq, an ordered gap-free buf of un-acked input, and a frozen flag. Each typed line is assigned the next seq and appended; every ServerFrame prunes buffered lines at or below its ack_input_seq.

Cross-shard redirect + replay 

sequenceDiagram
    participant C as Player
    participant G as telos-gate
    participant W1 as world A (source)
    participant W2 as world B (dest)
    W1-->>G: ServerFrame{Redirect: target_addr, handoff_token}
    G->>G: freeze() session (buffer live input, don't forward)
    G->>W2: re-dial + Attach{handoff_token, input_seq}
    W2-->>G: Attached (ack_input_seq = resume point)
    G->>W2: doReplay — re-send buffered lines seq > ack, in order
    W2->>W2: dedup by seq (drop seq ≤ appliedSeq) → apply exactly once
    G->>G: drain tail lines, then thaw

The crucial property: the TCP socket never moves — only the gRPC stream re-dials. The connection-scoped comms subscription and mid-session comms watcher live outside the re-dial loop, so a cross-shard walk is transparent to them. Live input during the freeze is buffered but not forwarded, and tail lines that arrived during the freeze are drained before thaw, so a live line can never overtake the replay. The world’s seq dedup guarantees exactly-once application. See Cross-Shard Handoff for the world side.

The telnet codec

internal/telnet is a minimal line-oriented codec. IAC command bytes: IAC=255, WILL/WONT/DO/DONT, SB=250, SE=240; GMCP is option 201.

  • Line assembly (ReadLine): reads a byte at a time; IAC hands off to handleIAC; \r and \0 are dropped; \n terminates. Capped at 4096 bytes — an over-cap line stops appending and drains to the next \n with a one-shot notice, still consuming IAC so negotiation can’t corrupt.
  • Sanitization: assembled input runs through sanitizeLine (UTF-8-aware drop of C0/C1 control runes) after all IAC is consumed. Output runs sanitizeOutput (strips control runes and strong bidi overrides U+202A–202E, preserving CR/LF and a lone 0xFF) → renderColor → doubles a literal 0xFF to IAC IAC. WriteScreen skips sanitize/color but still IAC-escapes.
  • Option negotiation (handleIAC): IAC IAC → literal 0xFF; for option 201 a DO/DONT flips GMCP on/off (no reply owed, since the gate initiated WILL); everything else is refused — a peer’s DO gets WONT, a peer’s WILL gets DONT, and DONT/WONT are answered with silence (avoiding negotiation loops). So the client stays in NVT line mode.

Only GMCP is negotiated. OfferGMCP sends IAC WILL 201 at connect. MCCP2 (compression), NAWS (window size), TTYPE, and CHARSET are refused/skipped — not implemented (there is no zlib path anywhere), and SSH was removed (auth is OAuth-only; TLS telnet is the only encrypted transport). This is why ClientInfo and width-based word-wrapping don’t exist yet.

The TLS-only port sniffs the first byte before handshaking. Rather than a bare tls.Listener, the gate accepts plaintext and peeks the first byte per connection: 0x16 (a TLS ClientHello) is replayed via a peekConn — so the ClientHello stays intact — and wrapped into tls.Server; anything else, or silence past a short read deadline, gets a one-line use TLS message and a close, instead of the silent handshake-failure hang. The sniff runs off the accept loop so a slow or silent client can’t stall it, and the subsequent handshake is driven explicitly under an tlsHandshakeTimeout (HandshakeContext) — closing the resource-exhaustion vector that a deadline-less lazy handshake (which tls.Server / tls.Listen leave open) exposes: a 0x16-then-stall client can no longer pin a goroutine and fd indefinitely.

Slow-client backpressure

Two layers, and neither blocks the zone goroutine:

  • World zone side: send is non-blocking on a buffered out channel (depth 256). On a full buffer the frame is dropped and a counter increments; a sustained wedge logs one “slow client wedged” warning, and a windowed drop-rate warn catches a limping client. The zone never does socket I/O.
  • Gate edge side: a per-write deadline bounds a single Write so a wedged client (full TCP receive window) can’t pin the writer goroutine. A deadline error closes the socket → the world stream ends → the wedged session is reclaimed. This is the reclaim mechanism the zone-side warning defers to.
  • World-side stream watchdog: gRPC keepalive reclaims only transport death. It structurally cannot see a gate whose HTTP/2 stack happily acks PINGs — the transport answers them independently of application flow control — while its application has stopped draining the Play stream. The flow-control window exhausts, stream.Send blocks with no deadline, out fills, the session starts dropping frames, and that stream’s writer, reader, session, entity ownership, and session-lock renewer all leak until the gate’s own write-deadline closes its side — reintroducing exactly the dependency on gate correctness that keepalive was meant to remove. So the writer records the instant each Send begins, and a watchdog tears the stream down once a single Send has been blocked past 45 s — deliberately above both the keepalive budget (~40 s) and the gate’s 30 s telnet write-deadline, so the cheaper, more precise signals still fire first.