Playwright vs Puppeteer vs Selenium for web scraping

The three tools side by side

How each tool takes a proxy

Playwright documents a proxy option that can be set globally for the browser or per browser context, with server, username and password. It answers the proxy's authentication challenge itself. One context per task, each with its own sticky-session username, gives each task its own exit and cookie jar. Details and code: Playwright proxy setup.

Puppeteer passes the proxy address to Chrome as --proxy-server or per context as proxyServer; Chrome's flag has no place for credentials, so they go to page.authenticate(), which must be called on every new page before its first navigation. See Puppeteer proxy setup.

Selenium sets host and port through its proxy capability, but neither that capability nor Chrome's flag carries a username and password. The dependable pattern is a small local forwarder that Selenium talks to without credentials and that adds them on the way to the gateway. See Selenium proxy setup.

Waiting, network control and bandwidth

Scrapers fail most often on timing: the element is not there yet. Playwright performs actionability checks before each action and waits for them to pass. Puppeteer's locator API likewise waits — for example for an element to be enabled and to have a stable position — before clicking. Selenium leaves it to you: its documentation describes an implicit wait that defaults to zero and explicit waits for specific conditions, and warns against mixing the two.

Network control decides your bandwidth bill. Blocking images, media and fonts is one line in Playwright's route() and a request-interception handler in Puppeteer; Selenium can intercept requests through WebDriver BiDi, with support that varies by language binding. A rendered page with assets is megabytes against kilobytes for its HTML, so blocking is worth doing in any of them — see how to reduce proxy bandwidth.

Concurrency and sessions

• Playwright contexts are isolated — separate cookies and storage — and cheap to create, so one browser can run many tasks, each with its own proxy username and sticky session.
• Puppeteer browser contexts give the same isolation, and proxyServer per context lets each use its own exit; remember authenticate() on each page.
• Selenium isolates per driver session, which means one browser process per concurrent task and one forwarder or proxy setting per session. Selenium Grid distributes sessions across machines if you already run one.

Which one to choose

• Python or .NET team, new project: Playwright. If you already use Scrapy, scrapy-playwright adds rendering to the requests that need it.
• Node.js team, Chrome-centred, existing Puppeteer code: Puppeteer.
• Existing Selenium test suite or Grid, Java, C# or Ruby codebase: Selenium, with a local forwarder for proxy credentials.
• Data available in HTML or a JSON endpoint: none of them — use an HTTP client. A headless browser is the expensive fallback, not the default.