15% off your first purchase with code WELCOME15, up to 25 GB.

See pricing
ARC Proxies

Integration · HTTP clients

How to use a proxy with Python requests

In Python requests, a proxy is a proxies dict mapping http and https to a proxy URL with the credentials inside it: http://user:pass@host:port. Pass it on every call and the ARC username does the rest: pool, country, city and an optional sticky session.

Reviewed 30 Oct 2026

your proxy, as a URL
http://USERNAME_pool-arcfast_country-de:PASSWORD@proxy.arcproxies.io:8000

The short answer

  • Map both http and https to the same http:// proxy URL; HTTPS targets go through a CONNECT tunnel.
  • Pass proxies= per request: the docs warn that session.proxies can be overwritten by environment proxies.
  • A 407 on an HTTPS target raises ProxyError, and urllib3's Retry will retry it, so filter it out yourself.
  • Always set a timeout; requests waits forever without one.

Setup

Set up Python requests with a proxy, step by step

  1. 01

    Install requests

    The SOCKS extra is only needed if you want SOCKS5 instead of HTTP.

    shell
    pip install requestspip install "requests[socks]"   # optional, for socks5h://
  2. 02

    Send a request through the gateway and check the exit IP

    The echo service returns the address the target sees. With this username it is a German residential exit; without a _session- token, the next new connection can return a different one.

    check_exit.py
    import requestsPROXY = "http://USERNAME_pool-arcfast_country-de:PASSWORD@proxy.arcproxies.io:8000"proxies = {"http": PROXY, "https": PROXY}r = requests.get("https://api.ipify.org?format=json", proxies=proxies, timeout=(5, 30))print(r.json())   # {"ip": "…"}
  3. 03

    Hold one IP for a task with a session handle

    Add _session-<handle>_sesstime-<minutes> and every request with that username uses the same exit for up to that many minutes. Mint a new handle per task: pagination, a search followed by its detail pages, anything that should look like one visitor.

    sticky.py
    import secretsimport requestsdef sticky_proxies(minutes=10):    handle = secrets.token_hex(4)    url = f"http://USERNAME_pool-arcfast_country-de_session-{handle}_sesstime-{minutes}:PASSWORD@proxy.arcproxies.io:8000"    return {"http": url, "https": url}with requests.Session() as s:              # cookies and connections for this task    proxies = sticky_proxies()    for page in range(1, 4):        r = s.get(f"https://example.com/search?q=lamp&page={page}",                  proxies=proxies, timeout=(5, 30))        print(page, r.status_code)
  4. 04

    Target a country or city

    Change the _country- token (lowercase ISO code) and add _city- for a metro from the published list, lowercased without spaces.

    geo.py
    USERS = {    "de-frankfurt": "USERNAME_pool-arcfast_country-de_city-frankfurt",    "nl-amsterdam": "USERNAME_pool-arcfast_country-nl_city-amsterdam",    "au-sydney":    "USERNAME_pool-arcfast_country-au_city-sydney",}for market, user in USERS.items():    url = f"http://{user}:PASSWORD@proxy.arcproxies.io:8000"    r = requests.get("https://example.com/price/123",                     proxies={"http": url, "https": url}, timeout=(5, 30))    print(market, r.status_code)
  5. 05

    Retry rate limits and timeouts, not bad credentials

    Retry 429 and 5xx responses and timeouts with backoff, each attempt on a new exit. Stop on a 407: an HTTPS target raises ProxyError with "407" in the message, a plain-HTTP target returns status 407.

    fetch.py
    import secretsimport timeimport requestsfrom requests.exceptions import ProxyError, Timeout, ConnectionError as ConnErrordef fetch(url, attempts=4):    for attempt in range(attempts):        handle = secrets.token_hex(4)        proxy = f"http://USERNAME_pool-arcfast_country-de_session-{handle}_sesstime-5:PASSWORD@proxy.arcproxies.io:8000"        try:            r = requests.get(url, proxies={"http": proxy, "https": proxy}, timeout=(5, 30))        except ProxyError as err:            if "407" in str(err):                raise RuntimeError("proxy rejected the credentials") from err        except (Timeout, ConnError):            pass                                     # new handle on the next attempt        else:            if r.status_code == 407:                raise RuntimeError("proxy rejected the credentials")            if r.status_code != 429 and r.status_code < 500:                return r            wait = r.headers.get("Retry-After", "")            if wait.isdigit():                time.sleep(int(wait))                continue        time.sleep(2 ** attempt)    raise RuntimeError(f"gave up on {url}")
  6. 06

    Run requests in parallel threads

    requests is synchronous, so concurrency means threads. Give each thread its own Session, since requests makes no thread-safety promise for a shared one, and cap the pool size to what the target tolerates.

    parallel.py
    import threadingfrom concurrent.futures import ThreadPoolExecutorimport requestsPROXY = "http://USERNAME_pool-arcfast_country-de:PASSWORD@proxy.arcproxies.io:8000"local = threading.local()def get(url):    if not hasattr(local, "s"):        local.s = requests.Session()    return local.s.get(url, proxies={"http": PROXY, "https": PROXY}, timeout=(5, 30)).status_codeurls = [f"https://example.com/item/{i}" for i in range(100)]with ThreadPoolExecutor(max_workers=8) as pool:    print(list(pool.map(get, urls)))

Rotating vs sticky with requests: where the IP actually changes

A rotating username lets each new connection take a different exit. A Session reuses connections, and an HTTPS request travels inside a CONNECT tunnel that stays open and stays on one exit. Several requests in one session to the same host can therefore share an IP even without a _session- token.

If you need a different IP per request, use a fresh handle per request, as in the retry example. If you need the same IP across requests, say so explicitly with a sticky handle rather than relying on keep-alive. Rotating vs sticky sessions goes through the trade-offs.

SOCKS5 with requests

The gateway speaks SOCKS5 on the same port with the same credentials. With requests[socks] installed, use socks5h:// so the hostname is resolved at the proxy, not on your machine. Plain socks5:// resolves DNS locally, which leaks the lookups to your resolver and can resolve to a different regional server.

SchemeWho resolves the hostnameInstall
http://The gateway (inside CONNECT)Nothing extra
socks5h://The gatewayrequests[socks]
socks5://Your machinerequests[socks]

Environment variables and precedence

Without proxies=, requests reads HTTP_PROXY, HTTPS_PROXY, NO_PROXY and ALL_PROXY. That is handy for a quick test (HTTPS_PROXY=http://user:pass@host:port python script.py) and a trap in production: a leftover variable on a server can route traffic you did not intend. Explicit proxies= on each call is the documented way to be sure, and the test proxy location guide shows how to confirm where traffic lands.

The problem

Common pitfalls

Setting session.proxies and assuming it sticks

requests' docs warn that session.proxies can be overwritten by environment proxies. Pass proxies= on each request.

No timeout

requests never times out by default. Through any proxy, set timeout=(connect, read), for example (5, 30).

Retrying a 407 with urllib3 Retry

A mounted HTTPAdapter(max_retries=Retry(...)) treats the failed tunnel as a connection error and retries it. Wrong credentials stay wrong; check for 407 first.

Special characters in the password

The password sits inside a URL. If it contains @, : or /, encode it with urllib.parse.quote(password, safe="") first.

FAQ

Questions

Something not covered? Write to contact@arcproxies.io.

How do I use a proxy with authentication in Python requests?

Put the credentials in the proxy URL and pass it for both schemes: proxies = {"http": "http://user:pass@host:port", "https": "http://user:pass@host:port"}, then requests.get(url, proxies=proxies, timeout=30). requests sends them as Basic proxy authentication.

Why is the https key an http:// URL?

The key is the scheme of the site you request; the value is how to reach the proxy. The gateway is reached over plain HTTP, and HTTPS sites are tunnelled through it with CONNECT, so TLS still runs end to end between your code and the site.

How do I rotate IPs with requests?

Use a username without a _session- token and open new connections, or give each request its own _session- handle. Reusing one Session keeps connections open, so consecutive requests to the same host can share an exit.

What does ProxyError: Tunnel connection failed: 407 mean?

The gateway rejected the credentials while opening the HTTPS tunnel. Check that the username and password are copied exactly from the dashboard and that special characters in the password are URL-encoded. Retrying will not help.

Does requests support SOCKS5 proxies?

Yes, with pip install "requests[socks]". Use socks5h://user:pass@host:port so DNS is resolved at the proxy; socks5:// resolves on your machine.

Sources & review

Last reviewed by the ARC Proxies team. Prices are read from the live catalog at build time.

  1. [1]Requests: Advanced usage: Proxies and SOCKS
  2. [2]Requests: Advanced usage: Timeouts
  3. [3]urllib3: urllib3.util.Retry

Next

What people build with it

Use caseProxies for web scraping and data collectionA second provider that just works — Scrapy, requests, Crawlee or Playwright, self-serve, no sales call, bandwidth that never expires.Use caseProxies for price and competitor monitoringSee the price a shopper in Berlin, Lyon or Chicago sees. ISP bandwidth at our lowest price per GB for HTML checks, residential where a marketplace is strict.Use caseProxies for alternative data in financeTurn public listings, pricing and hiring signals into a time series your models can trade on — collected on schedule, from the market you are modelling.GlossarySticky sessionA sticky session is a setting on a rotating proxy that keeps the same exit IP address for a chosen period, usually by attaching a session identifier to the connection, so a multi-step task appears to come from one address from start to finish.GlossarySOCKS5SOCKS5 is a proxy protocol, standardised in RFC 1928 in 1996, that relays TCP connections and UDP datagrams between a client and a destination without interpreting the application traffic, with username/password authentication defined separately in RFC 1929.GuideRotating vs sticky sessionsRotating proxies change exit IP per connection; sticky sessions hold one IP for a set time. How each works, when to use which, and how to set them up.GuideHow to test a proxy's locationCheck a proxy's exit IP and geolocation with curl, why IP geolocation databases disagree, and how to verify what the target site actually sees.IntegrationHTTPXConfigure an authenticated proxy in HTTPX 0.28: the proxy= argument that replaced proxies=, mounts per domain, async clients, SOCKS5 and error handling.

Integrations

Other http clients

HTTPXConfigure an authenticated proxy in HTTPX 0.28: the proxy= argument that replaced proxies=, mounts per domain, async clients, SOCKS5 and error handling.aiohttpUse an authenticated proxy with aiohttp: proxy= per request or per session, why proxy_auth is deprecated, SOCKS5 through aiohttp-socks, and 407 handling.AxiosUse an authenticated proxy with Axios in Node.js: the proxy option on 1.16.1+, https-proxy-agent for older versions, sticky sessions, SOCKS5 and 407s.Node.js fetchNode's built-in fetch has no proxy option. Use undici's ProxyAgent as the dispatcher, or NODE_USE_ENV_PROXY, with sticky sessions, SOCKS5 and 407 handling.curlSend curl through an authenticated proxy with -x and -U, check the exit IP, test sticky sessions and countries, use socks5h, and read 407 failures.GoRoute Go's net/http through an authenticated proxy: Transport.Proxy with url.UserPassword, a sticky session per request context, SOCKS5 and 407 checks.RubyUse an authenticated proxy in Ruby: Net::HTTP proxy arguments, Faraday proxy options, sticky sessions per task, geo-targeting and 407 handling.PHP GuzzleRoute PHP Guzzle and plain cURL through an authenticated proxy: the proxy option, sticky sessions, geo-targeting, retries that stop on a 407.Java HttpClientUse an authenticated proxy with java.net.http.HttpClient: ProxySelector, Authenticator, the Basic-over-CONNECT setting, sticky sessions and 407s.C# HttpClientUse an authenticated proxy in .NET: WebProxy and NetworkCredential on SocketsHttpHandler, a client per sticky session, SOCKS5 and 407 handling.Rust reqwestRoute Rust reqwest through an authenticated proxy: Proxy::all with basic_auth, a Client per sticky session, socks5h, timeouts and 407 errors.wgetRun GNU Wget through an authenticated proxy: https_proxy, --proxy-user, a .wgetrc for credentials, sticky sessions for mirrors and exit codes.PostmanSend Postman requests through an authenticated proxy: the custom proxy settings, Proxy auth, checking the exit IP, sessions, countries and Newman.DockerGive Docker containers and builds an authenticated proxy: HTTP_PROXY, HTTPS_PROXY and NO_PROXY, build args, Compose, and why the daemon proxy differs.

Get started

Get credentials and paste them in.

Sign up, buy a gigabyte, generate a username in the dashboard. Code WELCOME15 takes 15% off your first purchase; bandwidth never expires.

Create your accountQuestions first? contact@arcproxies.io