ShopWSS.com Scraper

Overview

This actor scrapes product data from ShopWSS.com (Warehouse Shoe Sale), a US retailer for shoes, clothing, and accessories. It's useful because ShopWSS doesn't offer a free public API. You can scrape collections, search results, and detailed product pages.

Key Features

• Collections: Scrape all products from a specific collection page.
• Search: Scrape all products matching a given keyword.
• Product Details: Extract comprehensive data including title, description, images, variants, and options.
• Pagination Control: Define start and end pages for scraping.
• Item Limits: Optionally limit the total number of items scraped.

How to Use

The actor runs on the Apify platform. You configure it via a JSON input object. Results are stored in an Apify dataset, which you can then access via the Apify API in various programming languages.

Input/Output

Input Parameters (JSON)

The actor accepts the following parameters in its input:

{
  "startUrls": ["https://www.shopwss.com/collections/mens-shoes"],
  "search": "nike air max",
  "endPage": 3,
  "maxItems": 50,
  "proxy": {
    "useApifyProxy": true
  },
  "extendOutputFunction": "($) => { return { 'customField': 'value' }; }",
  "customMapFunction": "(item) => { item.processed = true; return item; }"
}

• startUrls (Optional, Array): List of ShopWSS URLs to scrape. Accepts collection, search, or product detail URLs.
• search (Optional, String): A keyword to search for on the site.
• endPage (Optional, Number): The final page number to scrape for paginated lists (like search results or collections). Default is to scrape all pages.
• maxItems (Optional, Number): A hard limit on the total number of items to scrape.
• proxy (Required, Object): Proxy configuration is mandatory. Use your own proxies or Apify Proxy.
• extendOutputFunction (Optional, String): A JavaScript function to extract custom data from the page using jQuery.
• customMapFunction (Optional, String): A JavaScript function to transform each scraped item object.

Usage Tip: To scrape a specific page range, set a startUrl to the first page and use endPage to define the last page. For example, a startUrl of page 5 with "endPage": 7 will scrape pages 5, 6, and 7.

Output

Each scraped product is saved as a separate item in the run's dataset. The data includes all available product details like title, price, images, variants, and descriptions. You can process this dataset using the Apify API.

Performance Notes

The actor is optimized for speed. Under normal conditions (without frequent blocking), it can scrape approximately 100 product listings in about 2 minutes, consuming ~0.01-0.03 compute units.

Development & Support

This scraper is actively maintained. For bug reports or feature requests, create an issue on the GitHub repository. If the actor receives invalid input, it will stop immediately and describe the error.