Mobile.de Scraper

# Mobile.de Scraper A lightweight scraper for extracting car listings from mobile.de. ## Features - Extract car listings from custom search URLs - Control pagination with configurable page limits - Export data in JSON format with detailed vehicle information - Support for multiple search URLs in a single run > Note: Mobile.de limits results to maximum 50 pages per search. For larger datasets, split your searches into smaller chunks using different filter combinations in your URLs. --- ## Configuration ### start_urls (required) An array of URL objects containing Mobile.de search pages to scrape. - Type: Array of objects with url property - Format: Each object must contain a url field with a valid Mobile.de search URL - Example: json "start_urls": [ { "url": "https://suchen.mobile.de/fahrzeuge/search.html?dam=false&isSearchRequest=true&s=Car&sb=rel&vc=Car" } ] How to get search URLs: 1. Go to mobile.de 2. Use the search filters (make, model, price range, location, etc.) 3. Copy the URL from your browser's address bar 4. Add it to the start_urls array ### scrape_page_limit (optional) Limits the number of search result pages to scrape from each start URL. - Type: Number - Default: No limit (scrapes all available pages, up to Mobile.de's maximum of 50) - Example: "scrape_page_limit": 5 (scrapes only the first 5 pages) - Use case: Useful for testing or when you only need recent listings ### detail_page_urls (optional) Array of specific listing URLs to check if ads have been delisted. - Type: Array of strings - Use case: Check delisting status of specific ads - Example: json "detail_page_urls": [ "https://suchen.mobile.de/auto-inserat/volkswagen-t-roc-1-5-tsi-r-line-ledplus-navi-ahk-pano-dab-bayreuth/3911454010.html" ] ### Configuration Examples #### Minimal configuration (scrape all pages) json { "start_urls": [ { "url": "https://suchen.mobile.de/fahrzeuge/search.html?dam=false&isSearchRequest=true&s=Car&sb=rel&vc=Car" } ] } #### With page limit (test mode) json { "start_urls": [ { "url": "https://suchen.mobile.de/fahrzeuge/search.html?dam=false&isSearchRequest=true&s=Car&sb=rel&vc=Car" } ], "scrape_page_limit": 1 } #### Multiple search URLs json { "start_urls": [ { "url": "https://suchen.mobile.de/fahrzeuge/search.html?dam=false&isSearchRequest=true&pageNumber=1&s=Car&sb=rel&vc=Car" }, { "url": "https://suchen.mobile.de/fahrzeuge/search.html?dam=false&isSearchRequest=true&pageNumber=1&s=Motorbike&sb=rel&vc=Bike" } ], "scrape_page_limit": 10 } #### Check for delisted ads json { "detail_page_urls": [ "https://suchen.mobile.de/auto-inserat/volkswagen-t-roc-1-5-tsi-r-line-ledplus-navi-ahk-pano-dab-bayreuth/3911454010.html" ] } --- ## Output ### Output Data Structure Each scraped car listing includes detailed information about: - Basic Info: URL, ID, title, segment, category - Vehicle Details: Make, model, KBA numbers (HSN/TSN) - Pricing: Gross price, net price, VAT, price rating - Technical Specs: Power, fuel type, transmission, displacement, mileage, registration date - Features: Body type, doors, seats, color, emission class - Location: Country, postal code, city - Seller Info: Name, type (dealer/private), logo, phone, rating - Media: Preview images, thumbnails, video availability, image count - Financing: Financing and leasing availability ### Output Example json { "url": "https://suchen.mobile.de/fahrzeuge/details.html?id=395079745", "id": 395079745, "title": "Audi A6 Avant 50 TDI Q sport S line MATRIX ACC 20\" AV", "segment": "Car", "category": "Kombi", "kba": { "hsn": "0588", "tsn": "BNH" }, "make": "Audi", "model": "A6", "previewImage": "https://img.classistatic.de/api/v1/mo-prod/images/2f/2f32eb02-5dea-40cf-ac96-dcc32e5f1200?rule=mo-160.jpg", "previewThumbnails": [ "https://img.classistatic.de/api/v1/mo-prod/images/4c/4c9e4be7-4d2b-43ed-870d-8eb0b248cd60?rule=mo-160.jpg" ], "price": { "gross": 41490, "net": 34865.55, "vat": "19% MwSt." }, "priceRating": "REASONABLE_PRICE", "attributes": [ [ { "value": "EZ 06/2020" }, { "value": "66.600 km" }, { "value": "210 kW (286 PS)" } ], [ { "value": "Kombi" }, { "value": "Unfallfrei", "bold": true }, { "value": "Diesel" }, { "value": "Automatik" }, { "value": "HU Neu" }, { "value": "4/5 Türen" } ] ], "attr": { "cn": "DE", "z": "01917", "loc": "Kamenz", "fr": "06/2020", "pw": "210 kW (286 PS)", "ft": "Diesel", "ml": "66.600 km", "cc": "2.967 cm³", "tr": "Automatik", "gi": "Neu", "ecol": "Weiß", "eu": "Deutsche Ausführung", "door": "4/5", "sc": "5", "c": "EstateCar", "emc": "Euro6", "pvo": "1" }, "isVideoEnabled": true, "imageCount": 25, "isFinancingAvailable": true, "supportedLeasingType": null, "leasingRate": null, "sellerInfo": { "name": "Autohaus Elitzsch GmbH", "sellerType": "DEALER", "country": "DE", "location": "DE-01917 Kamenz", "logo": "https://img.classistatic.de/api/v1/mo-prod/images/04/04bcd3b9-5ff5-4ca3-b21a-06842fd8392a?rule=mo-80.jpg", "phone": "+49 (0)3578 343490", "rating": { "score": 4.6, "count": 35 } } } ### Delisted Ad Output When checking specific URLs with detail_page_urls, delisted ads are identified: json { "url": "https://suchen.mobile.de/auto-inserat/volkswagen-t-roc-1-5-tsi-r-line-ledplus-navi-ahk-pano-dab-bayreuth/3911454010.html", "is_delisted": true } --- ## Limitations - Mobile.de limits search results to maximum 50 pages per query - No built-in proxy support in this version - Single domain support (mobile.de) Workaround for large datasets: Split your searches using different filter combinations: - By price range: €0-10,000, €10,000-20,000, etc. - By location/region - By make/model - By year range - By vehicle type (Car, Motorbike, Caravan, etc.) --- ## Support If this scraper doesn't match your needs or you encounter issues, please contact: - Email: realspidery@gmail.com - X/Twitter: @realspidery For feature requests or bugs, please provide: - Your input configuration - Error messages or logs - Expected vs actual behavior