Costco Product Search Scraper

Overview

This actor automates data extraction from multiple international Costco e-commerce websites (UK, Mexico, South Korea, Taiwan, Australia, Japan). It's built to handle Costco's single-page application (SPA) architecture by directly accessing the underlying API endpoints, bypassing the limitations of traditional HTML scraping. It's designed for developers, analysts, and businesses needing structured product data for market research, price monitoring, and inventory analysis.

Key Features

• Multi-Region Support: Scrapes product data from Costco domains for the UK (.co.uk), Mexico (.com.mx), South Korea (.co.kr), Taiwan, Australia (.com.au), and Japan (.co.jp).
• Comprehensive Data: Extracts over 60 data points per product, including title, price, SKU, specifications, availability, ratings, and images.
• Handles Dynamic Content: Uses advanced techniques to navigate and extract data from JavaScript-rendered category and product listing pages.
• Configurable Scraping: Control the scope via URL lists, item limits, and retry logic.
• Proxy Integration: Supports Apify Proxy with residential IPs to manage request rates and avoid blocks.

How to Use

Configure the actor's input via a JSON object. The primary method is to provide a list of Costco category or search result URLs.

Input Format

The actor accepts a JSON object with the following key parameters:

{
  "max_retries_per_url": 2,
  "proxy": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"],
    "apifyProxyCountry": "JP"
  },
  "max_items_per_url": 20,
  "urls": [
    "https://www.costco.co.uk/Computers/Laptops-MacBooks/c/cos_16.1",
    "https://www.costco.com.au/Electronics/TVs-Projectors/TVs/c/cos_21.1.2"
  ],
  "ignore_url_failures": true
}

• urls: Required. An array of Costco product listing page URLs to scrape.
• max_items_per_url: Optional limit for the number of products to extract per URL.
• max_retries_per_url: Number of retry attempts for a failed URL.
• ignore_url_failures: If true, the run continues even if some URLs fail.
• proxy: Configure Apify Proxy. Set apifyProxyCountry to match your target website's region (e.g., "JP" for costco.co.jp).

Input/Output

Input: A JSON configuration object, primarily containing an array of target URLs.

Output: The actor outputs a dataset of product items, each containing detailed fields. The default storage is a dataset on the Apify platform, which can be downloaded in formats like JSON, CSV, or Excel. Each item in the dataset represents one product with its extracted attributes.

For issues or questions, contact the developer via My profile.