15% off your first purchase with code WELCOME15, up to 25 GB.

See pricing
ARC Proxies

Integration · HTTP clients

How to use a proxy with Go net/http

In Go, a proxy is the Proxy field of an http.Transport: a function that returns the proxy URL for each request. Put the credentials in that URL with url.UserPassword and net/http sends them as Proxy-Authorization itself. Because the function sees each request, one client can give every task its own sticky session.

Reviewed 30 Oct 2026

your proxy, as a URL
http://USERNAME_pool-arcfast_country-de:PASSWORD@proxy.arcproxies.io:8000

The short answer

  • http.ProxyURL(u) for one fixed proxy; a custom Proxy func to choose the username per request.
  • Build the URL with url.UserPassword so special characters in the password are escaped for you.
  • OnProxyConnectResponse sees the proxy's answer to CONNECT, which makes a 407 easy to detect with errors.Is.
  • Reuse one http.Client: it is safe for concurrent use, and connections are pooled per proxy URL, credentials included.

Setup

Set up Go with a proxy, step by step

  1. 01

    Send a request through the gateway and check the exit

    net/http reads the userinfo from the proxy URL and sends Basic proxy authentication. The echo service returns the exit the target sees.

    main.go
    package mainimport (	"fmt"	"io"	"net/http"	"net/url"	"time")func main() {	proxyURL := &url.URL{		Scheme: "http",		User:   url.UserPassword("USERNAME_pool-arcfast_country-de", "PASSWORD"),		Host:   "proxy.arcproxies.io:8000",	}	client := &http.Client{		Timeout:   30 * time.Second,		Transport: &http.Transport{Proxy: http.ProxyURL(proxyURL)},	}	resp, err := client.Get("https://api.ipify.org?format=json")	if err != nil {		panic(err)	}	defer resp.Body.Close()	body, _ := io.ReadAll(resp.Body)	fmt.Println(resp.StatusCode, string(body)) // 200 {"ip":"…"}}
  2. 02

    Choose the session per request with a Proxy func

    Store a session handle in the request's context; the Proxy func turns it into a sticky username. Requests without a handle use the rotating username. The connection pool is keyed by the full proxy URL, so different handles never share a connection.

    proxy.go
    type sessionKey struct{}// WithSession pins every request made with ctx to one exit IP.func WithSession(ctx context.Context, handle string) context.Context {	return context.WithValue(ctx, sessionKey{}, handle)}func arcProxy(r *http.Request) (*url.URL, error) {	user := "USERNAME_pool-arcfast_country-de"	if h, ok := r.Context().Value(sessionKey{}).(string); ok && h != "" {		user += "_session-" + h + "_sesstime-10"	}	return &url.URL{Scheme: "http", User: url.UserPassword(user, "PASSWORD"), Host: "proxy.arcproxies.io:8000"}, nil}var client = &http.Client{	Timeout: 30 * time.Second,	Transport: &http.Transport{		Proxy:               arcProxy,		MaxIdleConnsPerHost: 16, // the default of 2 is low for concurrent workers	},}
  3. 03

    Run tasks concurrently, each on its own exit

    One client for all goroutines. Each task mints a handle, puts it in the context, and every request it makes leaves from the same IP. A buffered channel caps how many tasks run at once.

    main.go
    func newHandle() string {	b := make([]byte, 4)	rand.Read(b) // crypto/rand	return hex.EncodeToString(b)}func crawlListing(ctx context.Context, start string) error {	ctx = WithSession(ctx, newHandle())	for page := 1; page <= 3; page++ {		req, _ := http.NewRequestWithContext(ctx, http.MethodGet, fmt.Sprintf("%s?page=%d", start, page), nil)		resp, err := client.Do(req)		if err != nil {			return err		}		io.Copy(io.Discard, resp.Body)		resp.Body.Close()	}	return nil}func main() {	starts := []string{"https://example.com/c/lamps", "https://example.com/c/chairs"}	slots := make(chan struct{}, 8)	var wg sync.WaitGroup	for _, s := range starts {		wg.Add(1)		go func(start string) {			defer wg.Done()			slots <- struct{}{}			defer func() { <-slots }()			if err := crawlListing(context.Background(), start); err != nil {				log.Println(start, err)			}		}(s)	}	wg.Wait()}
  4. 04

    Geo-target with a different username

    Country and city are username tokens. Swap the base username per market; cities come from the published list, lowercased without spaces.

    geo.go
    var markets = map[string]string{	"de-cologne": "USERNAME_pool-arcfast_country-de_city-cologne",	"jp-osaka":   "USERNAME_pool-arcfast_country-jp_city-osaka",	"hk":         "USERNAME_pool-arcfast_country-hk",}func clientFor(username string) *http.Client {	u := &url.URL{Scheme: "http", User: url.UserPassword(username, "PASSWORD"), Host: "proxy.arcproxies.io:8000"}	return &http.Client{Timeout: 30 * time.Second, Transport: &http.Transport{Proxy: http.ProxyURL(u)}}}
  5. 05

    Detect 407 and retry the rest

    OnProxyConnectResponse runs when the proxy answers a CONNECT. Returning a sentinel error for 407 lets the caller stop with errors.Is instead of matching strings. Retry 429, 5xx and timeouts with backoff on a new handle.

    retry.go
    var errProxyAuth = errors.New("proxy rejected the credentials (407)")var transport = &http.Transport{	Proxy: arcProxy,	OnProxyConnectResponse: func(ctx context.Context, proxyURL *url.URL, req *http.Request, res *http.Response) error {		if res.StatusCode == http.StatusProxyAuthRequired {			return errProxyAuth		}		return nil	},}var client = &http.Client{Timeout: 30 * time.Second, Transport: transport}func fetch(url string) ([]byte, error) {	for attempt := 0; attempt < 4; attempt++ {		req, _ := http.NewRequestWithContext(WithSession(context.Background(), newHandle()), http.MethodGet, url, nil)		resp, err := client.Do(req)		if errors.Is(err, errProxyAuth) {			return nil, err // wrong credentials: retrying will not help		}		if err == nil {			body, readErr := io.ReadAll(resp.Body)			resp.Body.Close()			if resp.StatusCode != 429 && resp.StatusCode < 500 {				return body, readErr			}		}		time.Sleep(time.Second << attempt) // 1s, 2s, 4s, 8s	}	return nil, fmt.Errorf("gave up on %s", url)}

What Transport.Proxy supports

  • Schemes: http, https, socks5 and socks5h. An empty scheme means http. Go treats socks5 the same as socks5h, so DNS is always resolved at the proxy.
  • Credentials: if the proxy URL has userinfo, net/http sends it as Proxy-Authorization (HTTP proxies) or as SOCKS5 username and password.
  • HTTPS targets go through an HTTP CONNECT tunnel; TLS runs between your program and the site.
  • Environment: http.ProxyFromEnvironment (the default transport's choice) reads HTTP_PROXY, HTTPS_PROXY and NO_PROXY. An explicit Proxy func avoids surprises from a stray variable.

Connection pooling decides when the IP changes

The transport pools connections per proxy URL (credentials included) and target. With a rotating username, consecutive HTTPS requests to one host usually reuse one tunnel and therefore one exit; a new IP needs a new connection. With a handle per task, each task has its own pool entry and keeps its exit for _sesstime- minutes.

MaxIdleConnsPerHost defaults to 2, which forces new connections, and new tunnels, under concurrency. Raise it to roughly your worker count. MaxConnsPerHost caps the total per pool entry if you need a hard ceiling. Pace requests to what each site tolerates; handling 429 rate limits covers the backoff side.

The problem

Common pitfalls

Building the proxy URL by string concatenation

A password with @, : or / breaks a hand-built URL. url.UserPassword escapes it correctly.

A new http.Client per request

Each new transport starts with an empty pool, so every request pays for a new tunnel. Share one client; it is safe for concurrent use.

Not draining the body

A connection only goes back to the pool once the body is read to EOF and closed. Undrained bodies leak connections and force new tunnels.

Matching on the error string

For HTTPS targets a 407 currently surfaces as an error reading "Proxy Authentication Required". OnProxyConnectResponse with a sentinel error is sturdier than string matching.

FAQ

Questions

Something not covered? Write to contact@arcproxies.io.

How do I set an authenticated proxy in Go?

Create the URL with &url.URL{Scheme: "http", User: url.UserPassword(user, pass), Host: "host:port"} and set Transport: &http.Transport{Proxy: http.ProxyURL(u)} on your client. net/http sends the credentials as Proxy-Authorization.

Can Go's http.Client use a SOCKS5 proxy?

Yes. Transport.Proxy accepts socks5:// and socks5h:// URLs, and userinfo in the URL is used for SOCKS5 authentication. Go treats both schemes the same, resolving DNS at the proxy.

How do I use a different proxy per request in Go?

Set Transport.Proxy to your own func(*http.Request) (*url.URL, error). It runs for each request, so it can read a value from the request's context and return a different username, for example a sticky session handle per task.

How do I detect a 407 from the proxy in Go?

For HTTPS targets the proxy answers the CONNECT request, which client.Do reports as an error. Set Transport.OnProxyConnectResponse to return your own error when res.StatusCode == 407, then check it with errors.Is.

Sources & review

Last reviewed by the ARC Proxies team. Prices are read from the live catalog at build time.

  1. [1]Go: net/http: Transport
  2. [2]Go: net/http: ProxyURL
  3. [3]Go: net/url: UserPassword

Next

What people build with it

Use caseProxies for web scraping and data collectionA second provider that just works — Scrapy, requests, Crawlee or Playwright, self-serve, no sales call, bandwidth that never expires.Use caseProxies for cybersecurity and threat intelligenceInvestigate phishing kits, typosquats and malicious infrastructure from an IP that is not your company's, and see the payload the target audience is served.Use caseProxies for alternative data in financeTurn public listings, pricing and hiring signals into a time series your models can trade on — collected on schedule, from the market you are modelling.GlossarySticky sessionA sticky session is a setting on a rotating proxy that keeps the same exit IP address for a chosen period, usually by attaching a session identifier to the connection, so a multi-step task appears to come from one address from start to finish.GlossaryHTTP CONNECTHTTP CONNECT is the HTTP request method a client sends to a proxy to ask for a tunnel to a host and port; once the proxy answers with a 2xx status it blindly forwards bytes in both directions, which is how HTTPS passes through an HTTP proxy with TLS intact end to end (RFC 9110, section 9.3.6).GuideHandling 429 rate limitsWhat HTTP 429 means, how to read Retry-After, and how to retry politely with exponential backoff, jitter and per-host concurrency limits. With code.IntegrationcurlSend curl through an authenticated proxy with -x and -U, check the exit IP, test sticky sessions and countries, use socks5h, and read 407 failures.

Integrations

Other http clients

Python requestsUse an authenticated proxy with Python requests: the proxies argument, sticky sessions per task, geo-targeting, SOCKS5 and retries that skip 407 errors.HTTPXConfigure an authenticated proxy in HTTPX 0.28: the proxy= argument that replaced proxies=, mounts per domain, async clients, SOCKS5 and error handling.aiohttpUse an authenticated proxy with aiohttp: proxy= per request or per session, why proxy_auth is deprecated, SOCKS5 through aiohttp-socks, and 407 handling.AxiosUse an authenticated proxy with Axios in Node.js: the proxy option on 1.16.1+, https-proxy-agent for older versions, sticky sessions, SOCKS5 and 407s.Node.js fetchNode's built-in fetch has no proxy option. Use undici's ProxyAgent as the dispatcher, or NODE_USE_ENV_PROXY, with sticky sessions, SOCKS5 and 407 handling.curlSend curl through an authenticated proxy with -x and -U, check the exit IP, test sticky sessions and countries, use socks5h, and read 407 failures.RubyUse an authenticated proxy in Ruby: Net::HTTP proxy arguments, Faraday proxy options, sticky sessions per task, geo-targeting and 407 handling.PHP GuzzleRoute PHP Guzzle and plain cURL through an authenticated proxy: the proxy option, sticky sessions, geo-targeting, retries that stop on a 407.Java HttpClientUse an authenticated proxy with java.net.http.HttpClient: ProxySelector, Authenticator, the Basic-over-CONNECT setting, sticky sessions and 407s.C# HttpClientUse an authenticated proxy in .NET: WebProxy and NetworkCredential on SocketsHttpHandler, a client per sticky session, SOCKS5 and 407 handling.Rust reqwestRoute Rust reqwest through an authenticated proxy: Proxy::all with basic_auth, a Client per sticky session, socks5h, timeouts and 407 errors.wgetRun GNU Wget through an authenticated proxy: https_proxy, --proxy-user, a .wgetrc for credentials, sticky sessions for mirrors and exit codes.PostmanSend Postman requests through an authenticated proxy: the custom proxy settings, Proxy auth, checking the exit IP, sessions, countries and Newman.DockerGive Docker containers and builds an authenticated proxy: HTTP_PROXY, HTTPS_PROXY and NO_PROXY, build args, Compose, and why the daemon proxy differs.

Get started

Get credentials and paste them in.

Sign up, buy a gigabyte, generate a username in the dashboard. Code WELCOME15 takes 15% off your first purchase; bandwidth never expires.

Create your accountQuestions first? contact@arcproxies.io