Upwork Job Extractor

Overview

This Actor is a Python-based web scraper built on the Apify platform, designed to extract job listings from Upwork. It uses the Apify SDK for Python as a foundation, providing a structured template to which you add your own scraping logic for Upwork. It handles the core infrastructure like request queuing and data storage, letting you focus on parsing the job data.

Key Features

• Apify SDK for Python: The core toolkit for building and running the scraper on the Apify platform.
• Input Schema: Define and validate configuration parameters (e.g., search keywords, location filters, max results) for each Actor run.
• Request Queue: Manages the list of Upwork search URLs and detail pages to be scraped.
• Dataset Storage: Outputs structured job data (like title, description, budget, client info) in a consistent JSON format.

How to Use

1. Add Your Scraping Logic: Insert your code to navigate and parse Upwork within the main async with Actor: block in src/main.py. You can use the Apify SDK alongside libraries like beautifulsoup4 or playwright.
2. Configure Input: Set your Actor's input via the Apify console, API, or a scheduler. This typically includes Upwork search URLs, pagination limits, and any required filters.
3. Run the Actor: Execute it on the Apify platform. It will process the request queue and store the extracted job listings in the dataset.

Local Development

To develop or test locally, pull the Actor using the Apify CLI:

# Install CLI via npm
npm -g install apify-cli

# Pull the Actor using its ID or name (e.g., "username/upwork-job-extractor")
apify pull <ActorId>

Input/Output

• Input: Configured via the input schema. Common parameters are starting URLs (Upwork search results), maximum pages to scrape, and optional proxies.
• Output: The Actor stores results in a dataset. Each item is a JSON object representing a job, containing fields you define, such as jobTitle, description, postedDate, budget, and clientRating.

Resources

• Apify SDK for Python documentation
• Apify Platform documentation
• Python tutorials in Academy
• Join our developer community on Discord