15% off your first purchase with code WELCOME15, up to 25 GB.

See pricing

Web scraping · Public tenders

Public tender monitoring across EU and national portals

Public tender monitoring means reading every new contract notice, corrigendum and award that matches your sectors, across the EU's TED and the national portals below it. Use the official API where one exists, and spend proxy bandwidth only on the long tail of portals that have none.

Pay per GB · No subscription · Prices in EUR, excl. VAT

Illustration. The URLs and values are fictional; the connection string is the real grammar.
€0.75
per GB on ARC Basic, excl. VAT
−21%
at 1 TB (€0.59 / GB)
185+
countries, city targeting
Never
expires — no subscription

Written for

  • Bid and proposal teams
  • Tender alert services
  • Public sector sales teams
  • Procurement researchers

The short answer

  • Start with TED's own API. Searching published notices needs no key, and the full notice set is published as open data. A proxy adds nothing there.
  • The gaps are below the EU thresholds and outside the EU. Contracts under the thresholds are advertised nationally, regionally or on a buyer's own profile, and that long tail is where a crawler earns its keep.
  • Collect each national portal from an exit in its country. Several serve a reduced or redirected page to foreign visitors, and all of them are small public services that deserve a slow, steady crawl.
  • Corrigenda move deadlines and change lots. Re-check open notices until they close, or your alert says a tender is due on a date that is no longer true.
  • Tender documents behind a supplier registration are out of scope for automation. Collect the public notice; let the bid team register and download the documents themselves.

The problem

What goes wrong without the right IPs

Below-threshold notices never reach TED

Publication on TED is mandatory above the EU thresholds. Smaller contracts, often the ones an SME can actually win, appear only on national, regional or municipal portals, each with its own layout and search.

Hundreds of small portals, no common format

A single country can run a federal portal, regional platforms and buyer profiles on several vendors' e-procurement systems. Each needs its own parser, and each changes its templates without notice.

Corrigenda silently change the facts

A deadline extension, a new lot or a withdrawn notice arrives as a separate corrigendum. If you only read new notices, your database keeps the original deadline long after it moved.

The same tender appears twice

Above-threshold contracts are often published on TED and on the national portal. Without a matching rule on buyer, title, value and deadline, your alert fires twice and your pipeline counts double.

Public servers, modest capacity

Many regional portals run on infrastructure sized for human visitors. A burst of parallel requests from one address gets throttled, and a throttled night means a missed publication day.

What to collect

What you can collect

Notice ID and notice type
Distinguishes a contract notice from a prior information notice, a corrigendum or an award. The type decides whether the record opens, changes or closes an opportunity.
Buyer name, country and region
The filter most bid teams sort by. Store the buyer's own identifier where the portal prints one, so the same authority matches across portals.
CPV codes
The Common Procurement Vocabulary is how notices are classified by subject. Matching on CPV codes is far more reliable than keyword matching on titles in 24 languages.
Estimated value and lots
Qualifies the opportunity. Lots matter because a supplier can often bid for one lot of a tender it could never deliver whole.
Procedure type
Open, restricted, negotiated or a framework call-off. It tells the bid team how much work a response will take and whether they can still enter.
Submission deadline and its history
The field corrigenda change most often. Keep every value with the date you saw it, not just the latest.
Document and portal links
Where the bid team goes next. Store the link, not the documents; downloads often require a supplier account.
Award details
Winner, value and number of bids received, from award notices. This is the raw material for win-rate and competitor analysis.

How it runs

How a collection run works

  1. 01

    Take TED from the API, not the website

    Query published notices through the TED Search API by CPV code, country and publication date, or load the daily open-data packages. Both are free and neither needs a proxy. Record the notice IDs so you can recognise the same tender on a national portal later.

  2. 02

    List the national and regional portals per country

    For each target country, list the national portal, the regional platforms and any large buyers' own profiles. Check which offer RSS feeds, open-data exports or an API; use those first and crawl only what is left.

  3. 03

    Sweep each portal's latest notices from inside its country

    Fetch the newest-first listing through an exit in the portal's country, page until you reach notices you already hold, and stop. One sticky session per portal keeps any search state on one address.

    Incremental sweep of one national portal
    import timeimport httpxGATEWAY, PASSWORD = "proxy.arcproxies.io:8000", "PASSWORD"def portal_client(country: str, portal: str) -> httpx.Client:    # Sticky ISP exit in the portal's country, one handle per portal    user = f"USERNAME_pool-arc_country-{country}_session-{portal}_sesstime-15"    return httpx.Client(proxy=f"http://{user}:{PASSWORD}@{GATEWAY}", timeout=30)def new_notices(client: httpx.Client, url: str, seen: set[str]):    page = 1    while True:        r = client.get(url, params={"sort": "published-desc", "page": page})        if r.status_code == 429:            time.sleep(int(r.headers.get("Retry-After", 60)))            continue        ids = parse_notice_ids(r.text)  # your parser for this portal        fresh = [i for i in ids if i not in seen]        yield from fresh        if len(fresh) < len(ids) or not ids:            break  # reached notices we already hold        page += 1        time.sleep(3)  # a small public server: be slow
  4. 04

    Re-check open notices for corrigenda

    Once a day, revisit every notice whose deadline has not passed. Store a changed deadline, a new lot or a cancellation as a new version of the record and alert on it as a change, not as a new tender.

  5. 05

    Match across portals, then alert

    Join national notices to TED notices on buyer, normalised title, estimated value and deadline. Alert once per tender, with links to every place it is published.

Setup

Recommended setup

Tender portals are light, public HTML. Keep the bulk on the cheapest pool and use residential exits only where a portal serves foreign or datacentre-looking visitors something else.

PlanRoleSessionTargetingPrice
ARC Basic
Rotating ISP
Default for portal sweeps and corrigendum checks
Our lowest price per GB, with an exit in the portal's own country. Listing pages are small and a daily sweep across dozens of portals is bandwidth, not difficulty.
Sticky, 15 minutes per portalCountry
€0.75 / GB
€0.59 at 1 TB (−21%)
ARC Fast
Rotating residential
For portals that treat ISP ranges differently
Residential exits for the few platforms that answer an ISP range with an error or a reduced page. Move a portal here only after you see it fail.
Sticky, 15 minutes per portalCountry + city
€2.00 / GB
€1.12 at 1 TB (−44%)

Prices are list prices in EUR, excl. VAT. Volume discounts apply automatically at checkout — full ladder on the pricing page.

Connect

One connection string per market

The country, city and sticky session are tokens in the username. Change the token, change the market; keep the session handle, keep the IP.

sticky · Germany · 10 min
http://USERNAME_pool-arcfast_country-de_session-k3f9a1x2_sesstime-10:PASSWORD@proxy.arcproxies.io:8000

Swap _country-de for any market you quote. The full grammar is on the residential proxies page.

Illustration

What daily monitoring of 40 national and regional portals costs

An illustration with round numbers — your pages, payloads and retry rate will differ. The arithmetic is the real price ladder.

Assumptions

Portals
40 across 8 countries
Listing and notice pages per portal per day
800 at 100 KB
Public notice attachments opened
300 a day at 2 MB
TED
via the free API, no proxy
Cadence
daily, 30 days
Bandwidth per month
≈ 114 GB

On ARC Basic

€77.52per month, excl. VAT

114 GB bought in one purchase at €0.68 / GB (−9% volume rung, applied automatically).

The same 114 GB on ARC Fast: €155.04 at €1.36 / GB. Route only the jobs that need it through the pricier pool.

Attachments are the expensive part. Open them only for notices that already match your CPV filter, and never re-download a document whose link has not changed. More in how to reduce proxy bandwidth.

Bandwidth never expires, so a bigger single purchase is only ever cheaper per GB. See the ladder.

In depth

Public tenders, in depth

What TED covers, and what it leaves to national portals

TED (Tenders Electronic Daily) is the online edition of the Supplement to the Official Journal of the EU. According to the European Commission, it carries around 800,000 procurement notices a year, worth more than €815 billion, and the full set is available as open data in XML, with a CSV subset.

Since 25 October 2023, notices above the EU thresholds must use eForms, the structured standard set by Implementing Regulation (EU) 2019/1780. For a data team that is good news: one schema, machine-readable fields, and a public Search API that does not require a key for published notices.

The thresholds themselves are revised every two years; the values for 2026–2027 are set by Commission Delegated Regulation (EU) 2025/2152. Everything below them is published under national rules, which is why a complete feed for a sector always means crawling national and regional portals as well.

TED API or crawler: which source for which notice

SourceWhat you getAccessProxy needed
TED Search APIAll published EU notices, structured eForms fieldsFree, no key for published noticesNo
TED open data packagesBulk XML of all notices, for backfillsFree downloadNo
National portals with feeds or exportsNational notices, often including below-thresholdRSS, CSV or API where offeredRarely
National and regional portals without feedsBelow-threshold and regional noticesPublic HTML pagesYes, an exit in the country
Buyer profile pagesNotices of one authority, sometimes only therePublic HTML pagesYes, slow and polite

Run the free sources first and let the crawler fill only the gaps. That keeps the proxy bill proportional to the part of the market nobody else structures for you. The general crawler patterns are on our web scraping page.

The UK after the Procurement Act 2023

The UK publishes its notices on its own Find a Tender service rather than TED. The Procurement Act 2023 came into force on 24 February 2025, and Find a Tender became the Central Digital Platform that buyers use to meet their transparency duties under the new regime.

Procurements started before that date continue under the old regulations, so for a while a UK feed carries notices under both regimes. Store which regime a notice falls under alongside its type, or your notice-type counts will not compare across the change.

Collecting tenders responsibly

  • Public notices only. Anything behind a supplier login, including many tender document downloads, is for the bid team to fetch with their own account.
  • One request at a time per portal, with a pause between pages. These are public services sized for people, and a crawl that slows them down is a crawl that will be blocked. See handling 429 rate limits.
  • Read the portal's terms and robots.txt before you start; respecting robots.txt covers how to treat it.
  • Contact names and phone numbers in notices are personal data. Store the buyer, not the person, unless you need the contact and have a lawful basis.

Our acceptable use policy applies to every request that goes through the network.

Works with anything that takes a proxy URL

  • Python requests / httpx
  • Scrapy
  • Crawlee
  • n8n
  • Docker

FAQ

Questions

Something not covered? Write to contact@arcproxies.io.

Do I need proxies to collect TED notices?

No. TED offers a public Search API that needs no key for published notices, and the full notice set is available as open data. Proxies are useful for the national, regional and buyer portals that publish below-threshold contracts and offer no feed.

Why are some tenders missing from TED?

Publication on TED is only mandatory for contracts above the EU thresholds, which are revised every two years. Smaller contracts are advertised under national rules, on national or regional portals or on the buyer's own profile, so they never appear on TED.

What are eForms?

eForms are the structured EU standard for procurement notices, introduced by Implementing Regulation (EU) 2019/1780 and mandatory for notices above the EU thresholds since 25 October 2023. They replaced the older standard forms and make notice fields machine-readable.

Can I automate downloading tender documents?

Only where the documents are public without an account. Many e-procurement platforms require supplier registration before download; automating around that is outside what we permit. Collect the public notice and let the bid team download documents with their own account.

How much bandwidth does tender monitoring use?

Listing and notice pages are light, typically around 100 KB. Forty portals swept daily with a few hundred public attachments opened comes to roughly 114 GB a month. Downloading every attachment for every notice is what multiplies it.

Sources & review

Last reviewed by the ARC Proxies team. Prices are read from the live catalog at build time.

  1. [1]European Commission: Tenders Electronic Daily
  2. [2]European Commission: eForms
  3. [3]Publications Office of the European Union: TED API documentation
  4. [4]European Commission, Public Buyers Community: Updated public procurement thresholds for 2026-2027 adopted by the European Commission
  5. [5]UK Government: Transforming Public Procurement

Get started

Buy a gigabyte and try it on your own targets.

No subscription, no sales call. Code WELCOME15 takes 15% off your first purchase, and whatever you do not use is still there next quarter.