Why Playwright Gets Blocked After 200 Requests (And What To Do About It)

The problem was not Playwright. The problem was that every layer of my connection was telling a different story about who I was.

Two Layers, One Identity

Anti-bot systems like Cloudflare, PerimeterX (now HUMAN), and Akamai do not just look at your IP address. They correlate two separate identity signals against each other.

The first signal is the HTTP layer. Your User-Agent string, TLS ClientHello fingerprint, Client Hints headers, JavaScript environment, and navigator properties. This is what your browser claims to be.

The second signal is the network layer. Your IP address, the ASN that owns it, the DNS resolver handling your queries, and the TCP stack fingerprint (p0f). This is what your connection actually is.

When those two layers contradict each other, the session gets flagged. A Chrome User-Agent claiming to be on a Pixel 7 running Android 14, while the IP belongs to a Hetzner datacenter in Germany, and the TCP fingerprint reads as a Linux server kernel. That contradiction is trivial to detect, and Cloudflare catches it within the first hundred requests.

The reason my scraper worked for 180 requests before failing was that Cloudflare runs a scoring system, not a binary gate. Each inconsistency adds risk points. Cross a threshold, and the challenges start. My datacenter proxy accumulated enough risk over those 180 requests for the system to decide it had seen enough.

The Proxy Type Matters More Than the Config

I tried three different proxy types on the same Playwright script with identical fingerprinting configs. The results were consistent across multiple target sites.

Datacenter proxies triggered challenges within 50–100 requests. The ASN belongs to a hosting provider, which is the single strongest bot signal in most detection systems. Residential proxies extended that to roughly 200-300 requests. The ASN belongs to a broadband ISP, which is better, but emulating a mobile device through a Comcast cable IP still creates an ASN-type mismatch that PerimeterX specifically checks for.

Mobile carrier proxies from 4G/5G networks ran thousands of requests without challenges. The ASN belongs to the actual mobile carrier (Verizon, T-Mobile, AT&T). When paired with proper device emulation, every signal layer aligns.

The difference is not about IP "quality" or "cleanliness." It is about whether the network-layer identity matches what the browser-layer identity claims to be.

The Playwright Config That Actually Works

The standard Playwright proxy setup passes --proxy-server at launch and sets credentials in the context. That handles routing but does nothing about fingerprint alignment.

The config below pairs mobile device emulation with CDP-level patches for the signals that Playwright does not set by default.

const { chromium, devices } = require('playwright');

const pixel7 = devices['Pixel 7'];

(async () => {
  const browser = await chromium.launch({
    headless: true,
    args: [
      '--proxy-server=http://your-mobile-proxy:port',
      '--disable-blink-features=AutomationControlled',
      '--disable-features=IsolateOrigins,site-per-process',
    ],
  });

  const context = await browser.newContext({
    ...pixel7,
    proxy: {
      server: 'http://your-mobile-proxy:port',
      username: 'your-username',
      password: 'your-password',
    },
    locale: 'en-US',
    timezoneId: 'America/New_York',
    geolocation: { latitude: 40.7128, longitude: -74.0060 },
    permissions: ['geolocation'],
  });

  const page = await context.newPage();
  const cdpSession = await context.newCDPSession(page);

  // This is the part most guides skip
  await cdpSession.send('Network.setUserAgentOverride', {
    userAgent: pixel7.userAgent,
    platform: 'Linux armv81',
    userAgentMetadata: {
      brands: [
        { brand: 'Chromium', version: '120' },
        { brand: 'Google Chrome', version: '120' },
      ],
      fullVersion: '120.0.6099.144',
      platform: 'Android',
      platformVersion: '14.0.0',
      architecture: '',
      model: 'Pixel 7',
      mobile: true,
    },
  });

  await page.goto('https://target-site.com', { 
    waitUntil: 'networkidle' 
  });
  
  // scraping logic here
  await browser.close();
})();

Client Hints: The Part Everyone Misses

The userAgentMetadata block in that CDP call is doing most of the heavy lifting. Cloudflare checks navigator.userAgentData through JavaScript on the client side. Playwright's built-in device descriptors set the User-Agent string and viewport, but they do not populate Client Hints headers. Without the CDP override, the browser claims to be Chrome on Android through the User-Agent string, but navigator.userAgentData.mobile returns undefined and Sec-CH-UA-Platform is missing from request headers.

That gap alone is enough for Cloudflare's bot scoring to bump the session into challenge territory.

The timezone and geolocation context need to match the proxy IP's actual location, too. I made this mistake early on. A Verizon IP geolocating to Newark, New Jersey, paired with America/Los_Angeles timezone is an instant inconsistency. Run your proxy IP through any geolocation lookup first and set the Playwright context to match.

Additional CDP Headers

For PerimeterX-protected targets, injecting consistent Sec-CH headers through CDP closes more gaps:

await cdpSession.send('Network.setExtraHTTPHeaders', {
  headers: {
    'Accept-Language': 'en-US,en;q=0.9',
    'Sec-CH-UA-Mobile': '?1',
    'Sec-CH-UA-Platform': '"Android"',
  },
});

These should already be present from the userAgentMetadata override, but explicitly setting them as extra headers acts as a safety net. Some detection systems check the raw HTTP headers independently of the JavaScript API responses, and a mismatch between the two is another scoring signal.

DNS: The Leak Nobody Checks

This one burned me for weeks before I figured it out. The proxy was routing HTTP traffic correctly, but DNS queries were leaking to my local resolver. The target site saw a T-Mobile IP on the connection, but Google DNS (8.8.8.8) was handling the domain resolution. That ASN mismatch between the connection IP and the DNS resolver is a known detection signal.

The fix depends on the proxy type. If you are using SOCKS5 in a scripting context, use socks5h:// not socks5://. The "h" means DNS resolves through the proxy, not locally. One character difference, completely different DNS behavior.

Dedicated mobile proxies that run on actual carrier hardware resolve DNS through carrier infrastructure by default, which eliminates the mismatch at the architecture level. With shared proxy pools, DNS behavior varies per exit node, and there is no way to guarantee consistency.

Verify with BrowserLeaks through the proxy before running against production targets. If the DNS resolver ASN does not match the proxy IP's ASN, the leak is active.

The navigator.webdriver Problem

Even with Playwright's newer headless implementation, navigator.webdriver can still return true in certain contexts. The --disable-blink-features=AutomationControlled flag handles the most common case, but some anti-bot scripts use indirect detection methods that survive it.

After setting up the proxy and device emulation, navigate to bot.sannysoft.com or creepjs.com as a first step. There are also standalone fingerprint test tools that break down uniqueness scoring, canvas hashes, and WebGL renderer details in a format that is easier to compare against expected device profiles. Run whatever catches the most signals before touching the actual target.

WebGL renderer strings are another common leak. A headless Chromium instance might report "SwiftShader" as the GPU renderer, which is a known headless indicator. If the target runs Fingerprint's detection or similar services, the WebGL string needs to match what the emulated device would actually report.

Choosing a Mobile Proxy for This

Not every mobile proxy works for this use case. The requirements are specific:

The proxy needs to be on a real carrier ASN, not a datacenter IP labeled as "mobile." Shared pools where hundreds of users rotate through the same IPs carry reputation risk. If someone ran aggressive automation on that IP yesterday, the reputation is already degraded when your session starts.

Providers like IPRoyal, Coronium, and VoidMob offer dedicated mobile proxies on physical SIM hardware. The key differentiator for Playwright use specifically is whether the proxy resolves DNS through carrier infrastructure (eliminates DNS leaks by default) and whether the TCP stack fingerprint matches a real mobile device (passes p0f checks without additional patching).

Test before committing to any provider. Run the proxy through browserleaks.com and verify the IP ASN, DNS resolver ASN, and WebRTC leak status. If any of those show a mismatch, the proxy will fail on well-protected targets regardless of how clean the Playwright config is.

Session Verification Script

I run this before every batch job now. Takes about 5 seconds and catches configuration problems that would otherwise waste hours of debugging time.

async function verifySession(page) {
  // Check IP and ASN
  await page.goto('https://ipinfo.io/json');
  const ipData = JSON.parse(
    await page.textContent('pre')
  );
  console.log('IP:', ipData.ip);
  console.log('ASN:', ipData.org);
  console.log('Location:', ipData.city, ipData.region);

  // Check for WebRTC leaks
  await page.goto('https://browserleaks.com/webrtc');
  // Manual inspection or automated selector check

  // Check DNS resolver
  await page.goto('https://browserleaks.com/dns');
  // Verify DNS ASN matches IP ASN

  // Check bot detection signals
  await page.goto('https://bot.sannysoft.com');
  // Verify all checks pass
}

If the ASN from ipinfo does not say a mobile carrier name, the proxy is wrong. If the DNS resolver ASN is different from the IP ASN, DNS is leaking. If sannysoft flags navigator.webdriver or any other automation signal, the CDP patches need adjustment.

Fix everything this function surfaces before pointing the scraper at the actual target. The five seconds invested here save the session.

Summary

Playwright gets blocked after a few hundred requests because the proxy creates a fingerprint mismatch between what the browser claims and what the network reveals. Datacenter IPs fail fastest. Residential IPs last longer, but still mismatch on the ASN type for mobile emulation. Mobile carrier IPs align all layers when paired with proper CDP configuration.

The config that survives Cloudflare at scale requires: mobile carrier proxy with matching ASN, CDP-level Client Hints and userAgentMetadata overrides, timezone, and geolocation matching the proxy IP, carrier-native DNS resolution, and a session verification step before every batch run.

Every detection layer needs to tell the same story. When they do, the sessions hold.