NVD CVE Scraper An Apify actor that fetches recent CVE (Common Vulnerabilities and Exposures) data from the National Vulnerability Database (NVD) using their official API. ## Features - Fetches CVEs published in the last 7 days - Uses the official NVD API 2.0 for reliable data access - Extracts CVE ID, description, publication date, and metadata - Outputs structured data to Apify dataset ## Why Use the API Instead of Scraping? The NVD website loads CVE data dynamically via JavaScript, making traditional HTML scraping unreliable. The official API provides: - Structured, reliable data - Better performance - No risk of breaking when the website UI changes - Official support from NIST ## Input Configuration You can customize the actor by modifying these parameters in the code: javascript // Adjust the date range (currently set to last 7 days) const sevenDaysAgo = new Date(now.getTime() - 7 * 24 * 60 * 60 * 1000); // Adjust results per page (max 2000) resultsPerPage: '100' ## Output Each scraped CVE includes: - cve: CVE identifier (e.g., CVE-2024-12345) - description: English description of the vulnerability - published: Publication date - lastModified: Last modification date - sourceIdentifier: Source that reported the CVE ### Example Output json { "cve": "CVE-2024-12345", "description": "A buffer overflow vulnerability in XYZ software allows remote attackers to execute arbitrary code...", "published": "2024-11-15T10:15:00.000", "lastModified": "2024-11-15T10:15:00.000", "sourceIdentifier": "security@example.com" } ## Rate Limits The NVD API has rate limits: - Without API key: ~5 requests per 30 seconds - With API key: 50 requests per 30 seconds For this actor (single request), rate limits shouldn't be an issue. ## Getting an API Key (Optional) For higher rate limits or frequent runs: 1. Request a free API key at: https://nvd.nist.gov/developers/request-an-api-key 2. Add it to your code: javascript const response = await fetch(apiUrl, { headers: { 'User-Agent': 'Apify-CVE-Scraper/1.0', 'apiKey': 'YOUR_API_KEY_HERE' } }); ## Installation 1. Create a new Apify actor 2. Copy the code into your actor's main file 3. Deploy and run ## Usage Examples ### Run as Scheduled Actor Set up a schedule to run daily and monitor new CVEs: 1. Go to your actor in Apify Console 2. Click "Schedule" 3. Set to run daily at your preferred time ### Export Data The scraped CVEs are stored in the actor's dataset and can be: - Downloaded as JSON, CSV, or Excel - Accessed via Apify API - Pushed to external services using integrations ## Advanced Customization ### Filter by Severity Add CVSS score filtering to the API request: javascript const params = new URLSearchParams({ pubStartDate: sevenDaysAgo.toISOString(), pubEndDate: now.toISOString(), cvssV3Severity: 'CRITICAL', // or HIGH, MEDIUM, LOW resultsPerPage: '100' }); ### Search by Keyword Filter CVEs containing specific keywords: javascript const params = new URLSearchParams({ keywordSearch: 'buffer overflow', resultsPerPage: '100' }); ## API Documentation Full NVD API documentation: https://nvd.nist.gov/developers/vulnerabilities ## Troubleshooting No CVEs returned: Check that there were CVEs published in your date range. Try expanding the date range. API errors: Ensure you're not hitting rate limits. Add delays between requests if making multiple API calls. Empty descriptions: Some CVEs may not have English descriptions immediately upon publication. ## License This actor uses public data from the National Vulnerability Database. Please review NVD's terms of use. ## Support For issues or questions: - Check NVD API status: https://nvd.nist.gov/general/news - Review API documentation - Contact Apify support for actor-specific issues