Hotel content data collection for mapping and deduplication

Identity

Name, brand or chain, address, geo-coordinates, business phone, website. The keys for matching.

Classification

Star rating as displayed and its source; property type (hotel, aparthotel, guesthouse).

Room types

Raw room name, bed configuration, size, maximum occupancy, view, bathroom type.

Amenities

Property-level and room-level, raw label and mapped code, and whether stated as included or chargeable.

Policies

Check-in and check-out times, children, pets, deposit, accepted payment, smoking.

Photo metadata

Image URL, dimensions, caption or alt text, perceptual hash. Not the image file.

Provenance

Source URL, fetch time, content hash. Lets you show when a field changed and where it came from.

1. 01

   Discover properties from sitemaps

   Hotel and chain sites usually list property pages in their sitemaps. Start there instead of paginating search results; it is lighter and complete.

2. 02

   Read structured data first

   Extract JSON-LD from each page. Where it is present and complete, you need little HTML parsing.

   Pull schema.org Hotel data from a property page24 linescopy

   import json, requestsfrom bs4 import BeautifulSoupPROXIES = {"http": "http://USERNAME_pool-arcfast_country-de:PASSWORD@proxy.arcproxies.io:8000",           "https": "http://USERNAME_pool-arcfast_country-de:PASSWORD@proxy.arcproxies.io:8000"}LODGING = {"Hotel", "LodgingBusiness", "Motel", "Hostel", "Resort", "BedAndBreakfast"}def lodging_jsonld(url: str) -> list[dict]:    html = requests.get(url, proxies=PROXIES, timeout=30).text    out = []    for tag in BeautifulSoup(html, "html.parser").find_all("script", type="application/ld+json"):        try:            data = json.loads(tag.string or "")        except json.JSONDecodeError:            continue        items = data if isinstance(data, list) else data.get("@graph", [data])        for item in items:            types = item.get("@type", [])            types = [types] if isinstance(types, str) else types            if LODGING & set(types):                out.append({k: item.get(k) for k in (                    "name", "address", "geo", "telephone", "starRating", "checkinTime",                    "checkoutTime", "numberOfRooms", "petsAllowed", "amenityFeature")})    return out

3. 03

   Normalise into your vocabulary

   Map amenity labels and room attributes to your own codes, keep raw values beside them, and record the language of the source page.

4. 04

   Match and deduplicate

   Block candidates by geo-distance, then score name, address, phone and room count. The techniques are in deduplicating listings.

5. 05

   Detect changes

   Re-crawl on a schedule, compare content hashes per field, and send only changed fields to review. Most pages will not have changed since last month.

http://USERNAME_pool-arc_country-de:PASSWORD@proxy.arcproxies.io:8000

Prices are list prices in EUR, excl. VAT. Volume discounts apply automatically at checkout — full ladder on the pricing page.