🚗 Carwow UK Car Leasing Scraper This Apify actor allows you to scrape car leasing deals and vehicle data from Carwow UK, one of the UK's leading car buying and leasing platforms. Extract detailed pricing, specifications, and contract information for personal and business car leasing deals. --- ## 🚀 Key Features - 🔎 Scrape car leasing deals from Carwow search results or individual deal pages - 🚙 Filter by brand and model (e.g., Audi Q3, BMW X5) - 📍 Location-based search with UK postcode and distance filtering - 💰 Extract comprehensive pricing data (monthly payments, initial payments, broker fees) - 📊 Deal scoring and rating information - 🔧 Vehicle specifications (transmission, fuel type, engine size) - 📄 Contract details (term length, mileage, maintenance included) - 🖼️ Vehicle images extraction - 🌐 Proxy support for stable, anonymous scraping - ⚙️ Flexible sorting options (best match, price, newest) --- ## ❓ Why Use This Actor - ✅ Automate car leasing deal collection from Carwow UK - ✅ Compare lease prices across multiple vehicles and configurations - ✅ Build datasets for automotive market analysis - ✅ Monitor deal scores and pricing trends over time - ✅ Integrate leasing data into price comparison platforms - ✅ Track availability and special offers for specific models --- ## 👥 Who Is This Actor Suitable For? - 🚗 Car leasing brokers and comparison websites - 📈 Automotive market researchers and analysts - 🧠 Data scientists studying vehicle pricing trends - 🧰 Developers building automotive platforms - 💼 Fleet managers evaluating leasing options - 📚 Academic researchers studying the UK automotive market --- ## 📥 Input Schema The actor accepts two primary input methods: ### Method 1: Direct URLs (Highest Priority) Provide specific Carwow URLs to scrape directly: json { "startUrls": [ { "url": "https://quotes.carwow.co.uk/stock_cars/deal-cards?brand_slug=audi&model_slug=q3-sportback&lease_type=personal" } ], "maxItems": 100, "proxyConfiguration": { "useApifyProxy": false } } ### Method 2: Search Parameters Use structured filters to build search queries: json { "brandSlug": "audi", "modelSlug": "q3-sportback", "postcode": "M1 1AE", "distance": 40, "dealTypeGroup": "leasing", "leaseType": "personal", "vehicleType": "car", "availability": "all", "vehicleStateGroup": "all", "sort": "best_match", "maxItems": 100, "proxyConfiguration": { "useApifyProxy": false } } ### Input Parameters | Parameter | Type | Description | Required | Default | |-----------|------|-------------|----------|---------| | startUrls | array | Direct URLs to scrape. Takes priority over search parameters. | No | - | | brandSlug | string | Brand identifier (e.g., 'audi', 'bmw', 'mercedes-benz') | No | 'audi' | | modelSlug | string | Model identifier (e.g., 'q3-sportback', 'x5'). Leave empty for all models. | No | 'q3-sportback' | | postcode | string | UK postcode for location-based search (e.g., 'M1 1AE', 'SW1A 1AA') | No | 'M1 1AE' | | distance | integer | Search radius in miles from postcode (1-500) | No | 40 | | dealTypeGroup | string | Type of deal: 'leasing', 'finance', or 'cash' | No | 'leasing' | | leaseType | string | Lease type: 'personal' or 'business' | No | 'personal' | | vehicleType | string | Vehicle type: 'car', 'van', 'pickup' | No | 'car' | | availability | string | Availability filter: 'all', 'in_stock', 'factory_order' | No | 'all' | | vehicleStateGroup | string | Vehicle condition: 'all', 'new', 'nearly_new', 'used' | No | 'all' | | sort | string | Sort order: 'best_match', 'price_asc', 'price_desc', 'newest' | No | 'best_match' | | maxItems | integer | Maximum number of deals to scrape. Set to 0 for unlimited. | No | 100 | | proxyConfiguration | object | Proxy settings for scraping | No | { "useApifyProxy": false } | Required if startUrls is not provided. --- ## 📤 Output Schema The actor outputs detailed car leasing deal information in JSON format: json { "id": "12345678", "url": "https://www.carwow.co.uk/deals/audi-q3-sportback-estate-35-tfsi-s-line-5dr", "brand": "Audi", "model": "Q3 Sportback", "derivative": "35 TFSI S Line 5dr", "transmission": "Automatic", "fuelType": "Petrol", "engineSize": "1.5L", "badge": "In Stock", "dealScore": { "score": 8.5, "label": "Great Deal" }, "pricing": { "monthlyPayment": "£299 per month", "monthlyPaymentRaw": 299 }, "contractDetails": { "termMonths": 36, "monthlyPayment": 299, "initialPayment": 2691, "brokerFee": 299, "annualMileage": 10000, "maintenanceIncluded": false, "metallicColourIncluded": true }, "images": [ "https://cdn.carwow.co.uk/images/...", "https://cdn.carwow.co.uk/images/..." ] } ### Output Fields Basic Information: - id: Unique car deal identifier - url: Direct link to the deal page - brand: Vehicle manufacturer - model: Vehicle model name - derivative: Specific trim/variant Specifications: - transmission: Manual or Automatic - fuelType: Petrol, Diesel, Electric, Hybrid, etc. - engineSize: Engine capacity (e.g., "1.5L") - badge: Availability status badge Deal Scoring: - dealScore.score: Numerical deal rating (0-10) - dealScore.label: Deal quality label (e.g., "Great Deal") Pricing: - pricing.monthlyPayment: Formatted monthly cost with currency - pricing.monthlyPaymentRaw: Numeric monthly cost for calculations Contract Details: - termMonths: Contract duration in months - monthlyPayment: Monthly payment amount - initialPayment: Upfront payment required - brokerFee: Broker arrangement fee - annualMileage: Annual mileage allowance - maintenanceIncluded: Whether maintenance is included - metallicColourIncluded: Whether metallic paint is included Media: - images: Array of vehicle image URLs --- ## 🔧 Technical Implementation ### Architecture The actor uses a two-phase scraping strategy: 1. List Phase: Extracts car IDs from search result pages - Paginates through results until maxItems is reached - Identifies cars using Turbo Frame IDs - Queues detail requests for each car 2. Detail Phase: Fetches comprehensive data for each car - Uses Turbo Frame headers for efficient loading - Parses pricing, specifications, and contract details - Extracts images and deal scoring information ### Key Technologies - Runtime: Node.js 20 - Framework: Crawlee 3.0 (CheerioCrawler) - Platform: Apify SDK 3.2.5 - Language: TypeScript 5.5 - Parser: Cheerio (lightweight HTML parsing) ### Source Code Structure carwow/ ├── src/ │ ├── main.ts # Entry point, crawler initialization │ ├── routes.ts # Request handlers (list & detail) │ ├── utils.ts # Helper functions (URL construction, parsing) │ └── interfaces.ts # TypeScript type definitions ├── .actor/ │ ├── actor.json # Actor metadata │ ├── input.json # Input schema definition │ └── output.json # Output view configurations ├── Dockerfile # Multi-stage Docker build ├── package.json # Dependencies and scripts └── tsconfig.json # TypeScript configuration ### Special Features Cookie-based Location Filtering: typescript // Postcode is injected via cookies for accurate distance filtering request.headers = { ...request.headers, Cookie: `user_postcode=${formattedPostcode};` }; Turbo Frame Loading: typescript // Uses Turbo Frame headers for efficient detail page loading request.headers = { ...request.headers, 'turbo-frame': `lazy_deal_card_${request.userData.carId}` }; Batch Processing: typescript // Requests are batched to avoid overwhelming the target server await crawler.addRequests(requests, { batchSize: 10, waitBetweenBatchesMillis: 2000 }); --- ## 🐳 Docker Configuration The actor uses a multi-stage Docker build for optimization: Stage 1 (Builder): - Base: apify/actor-node-puppeteer-chrome:20 - Installs all dependencies (including dev dependencies) - Compiles TypeScript to JavaScript Stage 2 (Production): - Copies only compiled JavaScript files - Installs production dependencies only - Significantly reduces final image size Build & Run: bash # Build the image docker build -t carwow-scraper . # Run locally docker run -it carwow-scraper --- ## 🎯 Usage Examples ### Example 1: Scrape Audi Q3 Personal Leasing Deals json { "brandSlug": "audi", "modelSlug": "q3-sportback", "leaseType": "personal", "maxItems": 50 } ### Example 2: Business Lease Deals Near Manchester json { "brandSlug": "bmw", "modelSlug": "x5", "leaseType": "business", "postcode": "M1 1AE", "distance": 100, "maxItems": 100 } ### Example 3: All BMW Models, In Stock Only json { "brandSlug": "bmw", "availability": "in_stock", "sort": "price_asc", "maxItems": 200 } ### Example 4: Direct URL Scraping json { "startUrls": [ { "url": "https://quotes.carwow.co.uk/stock_cars/deal-cards?brand_slug=mercedes-benz&model_slug=c-class&lease_type=business&sort=price_asc" } ] } --- ## 📊 Data Views The actor provides three pre-configured data views in the Apify platform: ### 1. Overview Displays essential deal information: - Car ID, Brand, Model, Derivative - Deal URL and Images - Monthly Payment and Deal Score - Transmission, Fuel Type, Engine Size ### 2. Pricing Details Focuses on financial information: - Monthly Payment (raw numbers for calculations) - Contract Term, Initial Payment, Broker Fee - Annual Mileage and Deal Score ### 3. Vehicle Specifications Shows technical details: - Transmission and Fuel Type - Engine Size and Availability Badge - Maintenance and Metallic Colour Inclusion --- ## ⚠️ Important Notes 1. startUrls Priority: If startUrls is provided, all other search parameters are ignored. 2. Rate Limiting: The actor implements delays and batch processing to respect server resources. 3. Postcode Format: UK postcodes should include the space (e.g., "M1 1AE" not "M11AE"). 4. Distance Filter: Only works when a valid postcode is provided. 5. Model Slug: Leave empty to get all models for a specific brand. 6. Proxy Recommended: For large scraping jobs, using Apify Proxy is recommended for stability. --- ## 🛠️ Development ### Local Development bash # Install dependencies npm install # Run in development mode npm run start:dev # Build TypeScript npm run build # Run production build npm run start:prod ### Testing bash # Run tests (to be implemented) npm test --- ## 🔍 Looking to Scrape Automotive Websites? Interested in similar scrapers for other automotive platforms? Contact us for custom automotive data extraction solutions: - Car comparison websites - Dealership inventory systems - Vehicle specification databases - Automotive marketplaces --- ## 📞 Contact & Support Author: Yurii License: ISC Need help with customization or have questions? - Create an issue on the Apify platform - Request custom features or modifications - Report bugs or unexpected behavior --- ## 🔗 Related Resources - Carwow UK Official Website - Apify Documentation - Crawlee Documentation - TypeScript Documentation --- This actor is for educational and research purposes. Please review Carwow's terms of service and robots.txt before extensive scraping. Always respect rate limits and server resources.