LinkedIn Job Scraper with AI Analysis A powerful Apify Actor that scrapes LinkedIn job postings and provides intelligent AI analysis using Claude (Anthropic). ## Features - 🔍 Job Search: Search LinkedIn jobs by keywords and location - 📊 Comprehensive Data: Extract job titles, companies, locations, descriptions, and posting dates - 🤖 AI Analysis: Get intelligent insights powered by Claude AI including: - Key requirements analysis - Salary expectations (when mentioned) - Experience level assessment - Technology stack identification - Overall role assessment - 📈 Scalable: Scrape up to 100 jobs per run - 🔒 Secure: API keys stored securely as secrets ## Input Parameters | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | searchQuery | String | Yes | "software engineer" | Keywords to search for jobs | | location | String | Yes | "United States" | Job location filter | | maxJobs | Integer | No | 10 | Maximum number of jobs to scrape (1-100) | | aiAnalysis | Boolean | No | true | Enable AI analysis of job postings | | anthropicApiKey | String | Conditional | - | Your Anthropic API key for Claude AI | | analysisPrompt | String | No | (default prompt) | Custom prompt for AI analysis | Required if aiAnalysis is enabled ## Output The actor stores results in the Apify dataset with the following structure: json { "title": "Senior Software Engineer", "company": "Tech Company Inc.", "location": "San Francisco, CA", "url": "https://www.linkedin.com/jobs/view/...", "postedDate": "2 days ago", "description": "Full job description...", "aiAnalysis": "AI-generated analysis of the role..." } ## Setup Instructions ### 1. Create a New Actor on Apify 1. Go to Apify Console 2. Click Actors → Create new 3. Choose Start from scratch 4. Give it a name (e.g., "linkedin-job-scraper-ai") ### 2. Set Up Project Structure Create the following files in your actor: my-actor/ ├── .actor/ │ ├── actor.json │ └── input_schema.json ├── main.js ├── package.json ├── Dockerfile └── README.md ### 3. Add the Code Copy each artifact into its respective file as shown above. ### 4. Get Your Anthropic API Key 1. Go to Anthropic Console 2. Sign up or log in 3. Navigate to API Keys 4. Create a new API key 5. Copy the key (it starts with sk-ant-) ### 5. Build and Run 1. In Apify Console, click Build 2. Wait for the build to complete 3. Click Start and configure input: - Enter your search query - Enter location - Set max jobs - Enable AI Analysis (checkbox) - Paste your Anthropic API key - Optionally customize the analysis prompt ### 6. View Results - Results appear in the Dataset tab - Download as JSON, CSV, or Excel - View AI analysis for each job ## Example Input json { "searchQuery": "machine learning engineer", "location": "New York, NY", "maxJobs": 20, "aiAnalysis": true, "anthropicApiKey": "sk-ant-api03-...", "analysisPrompt": "Analyze this job and tell me: 1) Is it suitable for mid-level engineers? 2) What's the tech stack? 3) Remote work policy?" } ## Pricing Notes - Apify: Charges based on compute units (CU). Check Apify pricing - Anthropic API: Charges per token. Claude Sonnet is ~$3 per million input tokens. Check Anthropic pricing ## Troubleshooting ### LinkedIn Blocking If you encounter blocking: - Enable residential proxies - Reduce maxJobs - Add delays between requests ### AI Analysis Not Working - Verify your Anthropic API key is correct - Check you have credits in your Anthropic account - Ensure aiAnalysis is set to true ### No Jobs Found - Try broader search terms - Check if LinkedIn structure changed - Verify the location exists on LinkedIn ## Legal & Ethical Use ⚠️ Important: - Respect LinkedIn's Terms of Service - Use responsibly and ethically - Don't scrape personal data - Add appropriate delays - Consider LinkedIn's API for commercial use ## Support For issues or questions: - Check Apify documentation - Review Anthropic API docs - Open an issue in the repository ## License MIT License - feel free to modify and use for your projects!