Reproducible WebRTC Failure Testing With Playwright and coturn

Testing how a WebRTC application behaves when the network fails is usually done by hand: open two browser windows, turn off Wi-Fi, count to ten, turn it back on, watch what happens. It works, after a fashion. It is also unrepeatable, untimed, impossible to run unattended, and useless for comparing two implementations — the interruption is never quite the same twice, and nothing records what actually occurred.

I hit this while testing recovery behavior across browser engines. I needed the same interruption, applied at the same point in the connection lifecycle, repeated dozens of times, producing machine-readable output. Getting there took longer than expected, mostly because the obvious approach doesn't do what it appears to.

This article describes the approach that worked, the failure modes that shaped it, and the parts worth reusing. The harness is open source; the code, raw trial records, and environment metadata are linked at the end.

The Obvious Approach Is Misleading

Playwright exposes BrowserContext.setOffline(true), which is the natural first reach:

await context.setOffline(true);

It does something — just not the thing you want. setOffline operates at the network layer Playwright controls, so it reliably kills HTTP requests and WebSocket connections. Your signaling channel dies immediately, which looks convincing in the logs.

What it does not reliably do is break an established peer connection whose media path runs over loopback or the local network. ICE keeps exchanging traffic and connectionState stays connected. If you are testing signaling recovery, this is a legitimate tool. If you are testing what an application does when the media path fails, you are testing nothing, and the dead signaling channel makes it look like you are.

There is a second-order problem. Because setOffline blocks the page's WebSocket, any signaling that has to survive the outage — an ICE restart offer, for instance — cannot travel over the browser's own connection. In my harness, this forced signaling to be bridged through the Node process driving the test rather than the page, so that offers and answers could still cross between peers while one of them was offline from the browser's point of view. That is a workaround for a tool limitation, not a property of WebRTC, and it is worth knowing before you build around it.

Make the Media Path Something You Control

The alternative is to stop trying to break the network and instead force all traffic through a process you own.

Setting iceTransportPolicy: "relay" in the RTCConfiguration restricts candidate gathering to relay candidates only — host and server-reflexive candidates are excluded from the pool entirely (MDN). Point that relays at a coturn instance on the local machine, and the entire media path now depends on one process you can signal.

const pc = new RTCPeerConnection({
  iceTransportPolicy: "relay",
  iceServers: [{
    urls: [
      "turn:127.0.0.1:65050?transport=udp",
      "turn:127.0.0.1:65050?transport=tcp"
    ],
    username: "harness",
    credential: "harnesssecret"
  }]
});

Interrupting the connection then becomes process control:

kill -STOP $TURN_PID   # relay stops forwarding

kill -CONT $TURN_PID   # relay resumes

SIGSTOP rather than SIGTERM matters here. The process is suspended, not terminated, so it stops relaying immediately while retaining its port bindings and allocation state. SIGCONT resumes it in place, with no restart and no re-binding race.

The property that makes this useful for experiments is that the outage is *parameterized*. "Restore the relay three seconds after the connection reports failed" becomes a variable rather than a stopwatch and good intentions. Running the same interruption at one, three, and five seconds across two browsers is then just a loop.

Choose the Method Once and Record It

A harness that silently falls back between interruption methods produces data you cannot interpret. If trial 7 paused a relay and trial 8 called setOffline, the comparison is meaningless — and you will not know unless the selection is recorded.

Select once at startup, log what was available alongside what was chosen, and write the selection into every trial record:

{
  "selected": "host_coturn_sigstop",
  "fallbackUsed": false,
  "attempted": [
    {
      "available": true,
      "method": "host_coturn_stop_forward",
      "reason": "turnserver found at /opt/homebrew/opt/coturn/bin/turnserver"
    },
    {
      "available": false,
      "method": "os_wifi_power",
      "reason": "ALLOW_WIFI_TOGGLE not set to 1"
    },
    {
      "available": true,
      "method": "playwright_offline",
      "reason": "always available (may not break loopback WebRTC)"
    }
  ],
  "iceTransportPolicy": "relay"
}

The os_wifi_power entry deserves a comment. Toggling the actual interface via networksetup -setairportpower is closer to a real network event than pausing a relay, and is therefore a better test in principle. It is gated behind an explicit environment variable because a test suite that disables your machine's Wi-Fi without asking is a poor citizen, particularly in CI.

Preflight, and Fail Loudly

A long matrix run that breaks on trial 3 and produces 57 rows of garbage is worse than one that refuses to start. Before any measured trials, verify that each browser can reach every state the experiment depends on, and record what was observed:

{
  "engine": "chromium",
  "browserVersion": "151.0.7922.34",
  "steps": [
    { "step": "ice_disconnected", "ok": true, "waitedMs": 5029 },
    { "step": "ice_failed",       "ok": true, "waitedMs": 9992 },
    { "step": "full_reconnect",   "ok": true, "elapsedMs": 145 }
  ]
}

The preflight also aborts hard on one specific error class: SDP negotiation failures, m-line ordering errors, and InvalidAccessError. These indicate the harness is broken rather than the connection under test, and allowing them through contaminates every downstream result. A run that does not finish with zero of these should be discarded.

The preflight numbers are themselves a result. Under this interruption method, Chrome reached disconnected at 5.0 s and failed at 10.0 s; Firefox took 11.2 s and 19.9 s. Both engines are working within the consent-freshness bounds described in RFC 7675, which sets a 30-second consent expiry with checks roughly every five seconds — but the specific thresholds differ, and that difference is only visible because the interruption is byte-for-byte identical across engines. Manual Wi-Fi toggling cannot produce that comparison.

Attribute Recovery to the Right Connection

This is the detail that determines whether the results mean anything, and it is easy to omit.

When testing whether a connection recovers, "the session is connected again" is not the same claim as "the original connection recovered." A freshly built RTCPeerConnection reaching connected is indistinguishable, in connectionState alone, from the original one returning. Counting both as recovery measures nothing.

Tag every peer connection at construction and compare identity at the moment of recovery:

metrics.originalPeer = (pcInstanceId === iceRestartPcInstanceId);

An epoch counter addresses the related hazard: callbacks from a torn-down connection fire after its replacement exists, and without a guard they write into the current trial's record.

if (epochAtStart !== pcEpoch) return;  // stale callback, ignore

Neither guard is exotic. Both are the difference between a dataset and a pile of numbers.

Not Every Failure Test Needs a Network

One of the two experiments in my harness tests whether a superseded recovery cycle can proceed into a destructive rebuild. That is a concurrency question, not a connectivity one, so it requires no interruption at all — an artificial delay forces the overlap window deterministically.

The result runs identically anywhere, produces the same outcome every time, and completes in seconds. Before building network machinery, it is worth checking which of your failure scenarios are actually about the network.

What the Setup Produces

The stack is Node with Playwright driving two browser contexts, a small WebSocket signaling server, and coturn as the controllable relay. Each trial emits a JSON file with a timestamped event log covering connectionState, iceConnectionState, signalingState, restart and rebuild boundaries, cycle identifiers, and final outcome — plus a CSV summary row.

Alongside those, the run captures an environment record: OS and architecture, Node and Playwright versions, both browser versions, ICE configuration, and the selected interruption method. That last file matters more than it sounds. The distance between "I tested this, and it worked" and a result someone else can check lies almost entirely in whether the environment was recorded next to the numbers.

Two Things I Would Do Differently

The environment record did not capture a git commit hash, so published results cannot be tied to an exact code revision. An obvious gap in hindsight, and the first thing I would add.

More substantively: pausing a TURN relay is not a network interface change. It tests relay failure specifically. The engine timings above may not transfer to a genuine Wi-Fi-to-cellular handoff, where interface teardown, address changes, and gathering behavior all differ. The method buys reproducibility at the cost of realism, and that trade should be stated rather than discovered by a reader.

Running It Yourself

The harness is at github.com/jaynirmal15/webrtc-recovery-harness under an MIT license. The raw trial records, environment metadata, and preflight output from the runs described above are in the results/ directory. If you are testing recovery behavior in your own application, the interruption machinery is the part worth lifting.