Yellow Pages Australia Scraper This actor scrapes business listings from Yellow Pages Australia based on search criteria like business type and location. It extracts comprehensive business data including contact details, address, operating hours, social media links, ratings, and more. ## Features - Search for businesses by type (e.g., "electrician", "plumber", "restaurant") - Filter by location (e.g., "Melbourne, VIC", "Sydney, NSW") - Set result limits and pagination - Apply filters like "open now", "local business only", etc. - Sort results by relevance, distance, or rating - Comprehensive proxy support (Apify Proxy or custom proxies) - Transform output to include only fields you need - Custom dataset naming and data storage options - Comprehensive data extraction including: - Business details (name, address, phone, email, website) - Category information - Opening hours - Social media links - Ratings and reviews - Geographic coordinates - And many more fields ## Proxy Support The actor supports proxy configuration to avoid rate limiting and IP-based blocking: ### Using Apify Proxy The recommended option is to use Apify Proxy, which offers multiple IP addresses and country selection: json { "proxy": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"], "apifyProxyCountry": "AU" } } ### Using Custom Proxies Alternatively, you can provide your own proxy URLs: json { "proxy": { "useApifyProxy": false, "proxyUrls": [ "http://username:password@proxy1.example.com:8080", "http://username:password@proxy2.example.com:8080" ] } } ### Proxy Fallback If a proxy connection fails, the actor will try to: 1. Retry connection up to 3 times with the same proxy 2. If all attempts fail, it will try connecting without a proxy 3. The scraper reports all connection attempts in the log ## Usage ### Input Parameters | Parameter | Type | Description | |-----------|------|-------------| | businessType | String | Type of business to search for (e.g., "electrician", "plumber") | | location | String | Location to search in (e.g., "Melbourne, VIC", "Sydney, NSW") | | limit | Integer | Maximum number of business listings to scrape | | pageNumber | Integer | Page number to start scraping from | | sortBy | String | Sort order for the results ("relevance", "distance", "rating") | | display | String | Display mode for the results ("bestMatch", "listView") | | openNow | Boolean | Filter to show only businesses that are currently open | | localBusiness | Boolean | Filter to show only local businesses | | paidProductsOnly | Boolean | Filter to show only paid listings | | proxy | Object | Proxy configuration (see "Proxy Support" section) | ### Advanced Options | Parameter | Type | Description | |-----------|------|-------------| | dev_transform_fields | Array | Transform the output by selecting only specific fields. For nested objects use dot notation (e.g., "address.streetAddress"). For array elements use index (e.g., "images.0.url"). | | dev_dataset_name | String | Custom dataset name with placeholders: {ACTOR} = actor name, {DATE} = date (YYYYMMDD), {TIME} = time (HHMMSS). Use "default" to use Apify's default dataset. | | dev_dataset_clear | Boolean | Clear the dataset before inserting new data | ## Output Format The actor stores results in the default dataset with the following structure (for each business): json { // Basic identification "listingId": "12345", // Unique ID for the listing "name": "Business Name", // Business name "address": "123 Sample St, Melbourne VIC 3000", // Full address as a single string "phone": "03 1234 5678", // Primary phone number "email": "contact@business.com.au", // Business email address "website": "https://www.businessname.com.au", // Business website URL "scrapedFromPage": 1, // Which page number this result came from // Category information "primaryCategoryName": "Electricians", // Just the category name "primaryCategoryDetails": { // Full category object "id": "123", "name": "Electricians" }, "categoryText": "Electricians - Melbourne, VIC", // Full category text string (includes location) // URLs and links "detailsPageUrl": "https://www.yellowpages.com.au/vic/melbourne/business-name-12345", // Yellow Pages listing page "logoUrl": "https://media.yellowpages.com.au/logos/business-name-logo.jpg", // Business logo URL // Opening hours "openingHoursToday": "9:30am - 5:30pm", // String for today's hours "openingHoursDetails": { // Full object of opening hours for all days "monday": "9:30am - 5:30pm", "tuesday": "9:30am - 5:30pm", "wednesday": "9:30am - 5:30pm", "thursday": "9:30am - 5:30pm", "friday": "9:30am - 5:30pm", "saturday": "10:00am - 4:00pm", "sunday": "Closed" }, // Social media links "facebookUrl": "https://www.facebook.com/businessname", // Facebook page URL "twitterUrl": "https://twitter.com/businessname", // Twitter page URL "youtubeUrl": "https://www.youtube.com/businessname", // Youtube channel URL "instagramUrl": "https://www.instagram.com/businessname", // Instagram page URL "linkedinUrl": "https://www.linkedin.com/company/businessname", // LinkedIn page URL // Ratings and reviews "rating": "4.5", // Numeric rating string (e.g., "4.3", "5.0") "reviewCount": 28, // Number of reviews "reviewSummary": { // Full review summary object "displayRating": "4.5", "reviewCount": 28, "ratingDistribution": { "1": 1, "2": 0, "3": 2, "4": 10, "5": 15 } }, // Additional business details "seniorityDate": "2015-06-12", // Date the listing became senior "accreditations": [ // Array of accreditation strings "Licensed Electrical Contractor", "Master Electricians Australia", "Energy Safe Victoria Approved" ], "uniqueSellingPoints": [ // Array of USP text strings "24/7 Emergency Service", "All Work Guaranteed", "No Call Out Fee" ], // Descriptive content "longDescription": "Servicing Melbourne and surrounding suburbs for over 10 years, Business Name provides reliable and professional electrical services for residential and commercial clients. We specialize in electrical repairs, installations, and maintenance with 24/7 emergency service available.", // Longer description/snippet "seoSnippetSummary": "Melbourne electricians offering 24/7 emergency service. Residential & commercial electrical repairs, installations & maintenance. No call out fee!", // SEO snippet summary "textDescriptor": "Electrical Contractors & Electricians - Melbourne, VIC", // Another text snippet // Identifiers and metadata "productCode": "FREE", // Product code (e.g., FREE, PREMIUM) "productId": "12345-FREE", // Product ID "productVersion": "1.2", // Product version "searchableAddress": { // Object containing searchable address details "streetNumber": "123", "streetName": "Sample St", "suburb": "Melbourne", "state": "VIC", "postcode": "3000" }, "salesforceServiceNumber": "SF-12345-AU", // Salesforce service number "searchScore": "0.98", // Search relevance score "distance": "2.5km", // Distance from search location (if provided) "latitude": "-37.8136", // Latitude coordinate "longitude": "144.9631" // Longitude coordinate } ## Output Destinations The scraper stores data in multiple locations: ### 1. Apify Platform Datasets When run as an Apify actor, the results are stored in datasets based on your configuration: #### A. Default Dataset When dev_dataset_name is set to "default" or left blank: - Each business listing is stored as a separate item in the default dataset - Access path: storage/datasets/default/ - Data is accessible through Apify UI and API - This is the recommended option for most users #### B. Custom Named Datasets When dev_dataset_name contains a custom value (e.g., "yellow-pages-{DATE}"): - All business listings are pushed as a batch to this custom dataset - Access path: storage/datasets/[your-custom-name]/ - Useful for organizing multiple runs or searches ### 2. Key-Value Store Additionally, two items are saved to the default key-value store: - metadata: Contains pagination info, result counts, and execution statistics - OUTPUT: Contains the complete results object with both businesses array and metadata ## Field Transformation Examples Use the dev_transform_fields parameter to extract only the fields you need. Here are some examples: ### Basic Contact Information json [ "name", "phone", "email", "website", "address" ] ### Location Data for Mapping json [ "name", "latitude", "longitude", "address", "primaryCategoryName" ] ### Social Media Analysis json [ "name", "facebookUrl", "twitterUrl", "instagramUrl", "linkedinUrl", "youtubeUrl" ]