Kaggle Dataset Scraper

An Apify actor that scrapes dataset listings from Kaggle based on a search term. It extracts detailed metadata, categories, and usability ratings for each dataset found.

Key Features

• Search-Based Scraping: Finds datasets on Kaggle using a provided search query.
• Comprehensive Metadata: Collects a wide range of details for each dataset, including ownership, licensing, file information, and performance metrics (views, downloads, votes).
• Pagination Control: Lets you specify the number of result pages to scrape via the totalPages input.
• Structured Output: Saves all collected data into an Apify dataset in a consistent JSON format.

How to Use

Run the actor in the Apify console and configure the following input parameters:

• searchTerm (required): The keyword or topic to search for on Kaggle (e.g., "football", "machine learning").
• totalPages (optional, default: 3): The number of search result pages to process.

Example Input

{
  "searchTerm": "football",
  "totalPages": 5
}

Input & Output

Input: A JSON object containing the searchTerm and totalPages parameters.

Output: The actor stores an array of items in the Apify dataset. Each item is a JSON object representing one Kaggle dataset with the following key fields:

• Basic Info: datasetUrl, title, overview, datasetId, datasetSlug, rank.
• Ownership: ownerName, ownerUrl, ownerUserId, ownerTier, creatorName, creatorUrl.
• Activity & Dates: viewCount, downloadCount, totalVotes, dateCreated, dateUpdated.
• Content Details: datasetSize, licenseName, downloadUrl.
• Categories: A list of category objects with id, name, fullPath, and description.
• File Information: commonFileTypes (list of fileType, count, totalSize).
• Usability Rating: An object containing scores like score, overviewScore, tagScore, fileFormatScore, etc.
• Related Resources: scriptCount, scriptsUrl, forumUrl, thumbnailImageUrl, medalUrl.

Example Output Entry

{
  "datasetUrl": "https://www.kaggle.com/datasets/secareanualin/football-events",
  "ownerName": "Alin Secareanu",
  "ownerUrl": "https://www.kaggle.com/secareanualin",
  "title": "Football Events",
  "viewCount": 15000,
  "downloadCount": 4500,
  "licenseName": "CC0: Public Domain",
  "datasetSize": "125.4 MB",
  "categories": [
    { "name": "Sports", "fullPath": "Sports" }
  ],
  "usabilityRating": {
    "score": 7.8,
    "overviewScore": 8.2
  }
}