Actor – Hacker News Design Scraper Scrape Y Combinator's Hacker News front page and extract only design-oriented posts. The Actor collects: - title - upvotes (points) - rank - source URL This tool is ideal for tracking UX/UI and design-related discussions on Hacker News. ## Features - Scrape the Hacker News homepage (news) quickly via HTTP (Cheerio) - Filter posts by configurable design-related keywords - Store results in the default Apify dataset ## Input Parameters The input is a JSON object with the following fields: - startUrl (Optional, String) - Hacker News page to scrape. Defaults to https://news.ycombinator.com/. - keywords (Optional, Array of String) - Keywords used to match design-related posts in titles. Defaults include: design, ux, ui, usability, accessibility, typography, interaction, product design, interface, figma, sketch, prototype. ### Input Example json { "startUrl": "https://news.ycombinator.com/", "keywords": [ "design", "ux", "ui", "usability", "accessibility" ] } ## During the Run - The Actor loads the provided startUrl (default: HN front page) - It parses the list of posts (tr.athing rows) - For each post, it extracts title, rank, and upvotes (from the adjacent subtext row) - It filters items whose titles include any of the provided keywords - Matching items are stored to the default dataset If incorrect input is provided, the Actor will fail with a descriptive message. ## Output Each dataset item has the following structure: json { "title": "Your vs. My in user interfaces", "upvotes": 166, "rank": 2, "url": "https://adamsilver.io/blog/your-vs-my-in-user-interfaces/", "source": "https://news.ycombinator.com/" } ## Run Locally Use Node 18+ (recommended with nvm): bash nvm install 18 nvm use 18 npm install node src/main.js Local storage paths used by the Actor: - Input: storage/key_value_stores/default/INPUT.json - Output dataset: storage/datasets/default ## Docker bash docker build -f .actor/Dockerfile -t hn-design-scraper . mkdir -p storage docker run --rm -v "$PWD/storage:/usr/src/app/storage" hn-design-scraper ## Apify CLI (optional) bash npm i -g apify-cli apify login apify run ## Deploy to Apify Platform bash apify push --no-prompt apify call ## Notes - This Actor focuses on the front page and performs keyword-based filtering for design topics. - If you need historical pages, Show HN, Ask HN, or comments hierarchy, consider adapting the code or using a more feature-complete HN Actor. - Keep nodejsVersion at 18 in .actor/actor.json for compatibility on the Apify platform. ## Reference - Inspired by the structure of: https://apify.com/epctex/hackernews-scraper