Website Checker Workload
by lukaskrivka
Creates reasonable workloads for analyzing any website with the Website Checker actor and combines the resulting data. This is the easiest way to anal...
Opens on Apify.com
About Website Checker Workload
Creates reasonable workloads for analyzing any website with the Website Checker actor and combines the resulting data. This is the easiest way to analyze any website for compute unit usage and anti-scraping blocking.
What does this actor do?
Website Checker Workload is a web scraping and automation tool available on the Apify platform. It's designed to help you extract data and automate tasks efficiently in the cloud.
Key Features
- Cloud-based execution - no local setup required
- Scalable infrastructure for large-scale operations
- API access for integration with your applications
- Built-in proxy rotation and anti-blocking measures
- Scheduled runs and webhooks for automation
How to Use
- Click "Try This Actor" to open it on Apify
- Create a free Apify account if you don't have one
- Configure the input parameters as needed
- Run the actor and download your results
Documentation
Website checker workload Creates reasonable workloads for analyzing any website with Website Checker and combines the resulting data. This is the easiest way to analyze any website for compute units usage and blocking. This actor runs a Website Checker for each proxy group and for both browser/Puppeteer and Cheerio scraper. Those checks are run in parallel with reasonable default values and the output of all checkers in combined into a single output breakdown. This gives you quite a nice idea how difficult and costly will be scraping the site with different methods and can save precious time you would spend with manual checks. ## Input | Field | Type | Default | Description | | ----- | ---- | ------- | ----------- | | website | String | https://apify.com | Website URL where you want to start checking | | runBrowser | Boolean | true | Run the checker with browser | | runCheerio | Boolean | true | Check with Cheerio | | proxyGroups | Array | ['auto', 'BUYPROXIES84958'] | List of proxy groups you want to test. Can be also auto to run with all proxies | | maxPagesPerCheck | Number | 200 | Max pages per each check | | runInParallel | Boolean | true | What to scrape from each page, default is "posts" the other option is "comments" | ## Output The output is saved to the default Key-Value store as OUTPUT record. It is a combined output from all Website Checker runs with added spent compute units. For example for input consisting of json "runBrowser": true, "runCheerio": true, "proxyGroups": ["auto", "BUYPROXIES84958"] The actor will run 4 checkers with all possible combinations: json { "puppeteer/auto": { "computeUnits": 0.45, "pagesPerComputeUnit": 444, "timeouted": 0, "failedToLoadOther": 9, "accessDenied": 0, "recaptcha": 0, "distilCaptcha": 24, "statusCodes": { "200": 3, "401": 2, "403": 5, "405": 24 }, "total": 43 }, "puppeteer/BUYPROXIES84958": { "computeUnits": 0.45, "pagesPerComputeUnit": 444, "timeouted": 0, "failedToLoadOther": 9, "accessDenied": 0, "recaptcha": 0, "distilCaptcha": 24, "statusCodes": { "200": 3, "401": 2, "403": 5, "405": 24 }, "total": 43 }, "cheerio/auto": { "computeUnits": 0.05, "pagesPerComputeUnit": 4000, "timeouted": 0, "failedToLoadOther": 9, "accessDenied": 0, "recaptcha": 0, "distilCaptcha": 24, "statusCodes": { "200": 3, "401": 2, "403": 5, "405": 24 }, "total": 43 }, "cheerio/BUYPROXIES84958": { "computeUnits": 0.05, "pagesPerComputeUnit": 4000, "timeouted": 0, "failedToLoadOther": 9, "accessDenied": 0, "recaptcha": 0, "distilCaptcha": 24, "statusCodes": { "200": 3, "401": 2, "403": 5, "405": 24 }, "total": 43 }, }
Categories
Common Use Cases
Market Research
Gather competitive intelligence and market data
Lead Generation
Extract contact information for sales outreach
Price Monitoring
Track competitor pricing and product changes
Content Aggregation
Collect and organize content from multiple sources
Ready to Get Started?
Try Website Checker Workload now on Apify. Free tier available with no credit card required.
Start Free TrialActor Information
- Developer
- lukaskrivka
- Pricing
- Paid
- Total Runs
- 1,486
- Active Users
- 29
Related Actors
Web Scraper
by apify
Cheerio Scraper
by apify
Website Content Crawler
by apify
Legacy PhantomJS Crawler
by apify
Apify provides a cloud platform for web scraping, data extraction, and automation. Build and run web scrapers in the cloud.
Learn more about ApifyNeed Professional Help?
Couldn't solve your problem? Hire a verified specialist on Fiverr to get it done quickly and professionally.
Trusted by millions | Money-back guarantee | 24/7 Support