🚘 Autotrader.ca Vehicle Scraper Actor Automatically extract vehicle listings from Autotrader.ca with comprehensive details. ## 🌟 What Does This Do? This powerful Apify actor automates the process of scraping vehicle data from Autotrader.ca. It diligently: 1. Visits Autotrader.ca search result URLs you provide. 2. Navigates through search pages to discover all available vehicle listings (ensuring up to 100 listings per page). 3. Extracts detailed information for each vehicle, including: * 🆔 Ad ID, Make, Model, Year * 💲 Price (both formatted string and numeric CAD value) * 🛣️ Mileage (both formatted string and numeric km value) * ✨ Vehicle Status (e.g., "Used", "New") & Posted Age * ⚙️ Key Specifications: Transmission, Drivetrain, Body Type, Exterior Colour, Fuel Type, Doors * 📍 Location: City, Province * 👤 Seller Information: Whether it's a private seller, Seller Name * 🖼️ A list of Image URLs * 📝 Full Vehicle Description * 🔗 The original URL of the listing * 📦 The complete raw JSON data for advanced users (exported as many separate columns in CSV/Excel). ## 🛠️ Configuration Made Simple Provide these details to get started: 1. Start URLs (start_urls) * Paste the Autotrader.ca search result URLs. The actor will automatically adjust them to fetch 100 results per page. * (Example: https://www.autotrader.ca/cars/on/toronto/?rcp=15&rcs=0&srt=35) * Input format: [{ "url": "your_autotrader_url_1" }, { "url": "your_autotrader_url_2" }] 2. Max Pages per Search (max_depth) (Optional) * "How many pages of search results should the actor navigate through for each start URL?" * (Example: Set to 3 to scan the first 3 pages of results. Leave empty or 0 to attempt to get all pages until no new vehicles are found or a repetition threshold is met.) 3. Max Concurrent Detail Page Requests (max_concurrency) (Optional) * How many vehicle detail pages to scrape at the same time. * (Default: 20. Adjust based on your needs.) 4. Custom Proxy URL (proxy_url) (Optional) * If you have a specific proxy you'd like to use. * (If not provided, the actor will use Apify's proxy service.) ## 🚦 Advanced Features - Automatic Pagination: The actor handles the rcs parameter for pagination and sets rcp=100 for maximum results per page. - Duplicate & Stagnation Detection: Stops pagination if no new listings are found or if the same listings repeat, to prevent infinite loops. - Robust Retries: Implements retries for fetching both search result pages and detail pages (up to 5 attempts with backoff). - Comprehensive Data Extraction: Parses the embedded JSON data (window['ngVdpModel']) on detail pages for accurate information. ## 📤 Getting Your Data Your scraped vehicle data is delivered in standard Apify dataset formats: - JSON: Provides the full structured data. Key processed fields are at the top level, image URLs are in a list (image_urls), and the complete raw JSON from Autotrader is nested under the all_data key. - Excel/CSV: Provides a flattened table view. * The key processed fields (like make, model, year, price_cad, mileage_km, description, url, etc.) will typically be found towards the end of the row/columns. * Image URLs will be flattened into separate columns (image_urls/0, image_urls/1, etc.). * The raw all_data JSON will be flattened into many preceding columns (e.g., all_data/adBasicInfo/adId, all_data/hero/make, etc.). You can often ignore these if you only need the main processed fields. Key Processed Fields (found towards the end of CSV/Excel output): url, ad_id, make, model, year, price_str, price_cad, mileage_str, mileage_km, status, posted_age, transmission, drivetrain, body_type, exterior_colour, fuel_type, doors, city, province, is_private_seller, seller_name, image_urls/0 (and subsequent numbers), description. Example Data Snippet (Based on the structure of your CSV output, simplified): ```json { // ... many columns from all_data/ ... "body_type": "SUV", "city": "St-Eustache", "description": "1996 HUMMER H1 6.5L V8 DIESEL 4-PASSENGER OPEN TOP...", "doors": "4", "drivetrain": "4x4", "exterior_colour": "Green", "fuel_type": "Diesel", "image_urls/0": "https://mkt-vehicleimages-prd.autotradercdn.ca/photos/import/202204/0102/0316/0017abd5-654e-460b-8295-980f1d374420.jpg-1024x786", // ... more image_urls/ ... "is_private_seller": false, "make": "Hummer", "mileage_km": 37397, "mileage_str": "37,397 km", "model": "H1", "posted_age": null, // Example data showed null for this field "price_cad": 79990, "price_str": "$79,990", "province": "QC", "seller_name": "Empire Auto", "status": "Used", "transmission": "Automatic", "url": "https://www.autotrader.ca/a/am%20general/hummer/st-eustache/quebec/5_54240768_20060222120117375/", "year": 1996 }