Dubicars Scraper

An Apify actor for scraping car listings from DubiCars.com. It extracts detailed vehicle data from search result pages or specific individual car pages.

Key Features

• Scrape all car data from provided search page URLs.
• Limit scraping to listings posted within the last X days.
• Extract data for specific individual cars by providing their direct URLs.
• Export data in multiple formats (JSON, CSV, etc.) for easy integration.
• Configurable pagination limits and proxy support to manage requests and avoid blocks.

How to Use

Configure the actor using the input fields below. The scraper operates in two main modes: scraping search results or scraping specific car detail pages.

Input

Provide configuration via a JSON object. Here's the main structure:

{
  "run_mode": "prod",
  "start_urls": [
    { "url": "https://www.dubicars.com/dubai/new" }
  ],
  "scrape_page_limit": 5,
  "ad_detail_page_urls": "https://www.dubicars.com/example-car-123.html",
  "proxy_settings": "http://user:pass@proxy:port"
}

• run_mode: Set to "prod" for a full scrape. Default is test mode.
• start_urls: An array of search page URLs. Do not include pagination (page=) or sort parameters in the URL.
• scrape_page_limit: (Optional) Limits how many paginated result pages to scrape from each start_url. Remove this field for no limit.
• ad_detail_page_urls: (Optional) A comma-separated string of specific car detail page URLs to scrape individually. If provided, start_urls are ignored.
• proxy_settings: (Recommended) Proxy configuration string to prevent blocking. For example: http://username:password@hostname:port.

Note: Using a proxy is strongly recommended to avoid being blocked by DubiCars.com.

Output

The actor outputs an array of items, each representing a scraped car listing. Data is available in formats like JSON and CSV via the Apify dataset.

Example Output Item (JSON):

{
  "external_url": "https://www.dubicars.com/2023-ford-raptor-654152.html",
  "updated_at": "24 Jun, 2024",
  "title": "Ford Raptor R Performance 4x4. Local Registration +10%",
  "price": "541000",
  "currency": "AED",
  "brand": "Ford",
  "model": "[Raptor]",
  "years": "2023",
  "vehicle_id": "DUBICARS000654152",
  "item_condition": "NewCondition",
  "transmission_type": "Automatic",
  "fuel_type": "Gasoline"
}

The dataset includes comprehensive fields such as description, color, body_type, features arrays, specifications, and dealer contact information.