Food Recipe Scraper ## Comprehensive Recipe Data Extraction Tool Food Recipe Scraper is a powerful web scraping solution designed to extract structured meal data from TheMealDB API. This tool enables users to discover, filter, and collect detailed recipe information including ingredients, cooking instructions, and metadata. Perfect for food enthusiasts, developers, and businesses looking to build recipe databases, meal planning apps, or nutritional analysis tools. Whether you're searching for specific dishes by keyword, filtering by ingredients, categories, or dietary preferences, this scraper delivers enriched, ready-to-use recipe data in a standardized format. ## Table of Contents - Features - Input Parameters - Output Schema - Usage Examples - Configuration Options - How It Works - Limitations - Support ## Features ### Advanced Search Capabilities - Keyword Search: Find recipes by meal name or partial matches - Alphabetical Crawling: Comprehensive catalog exploration when no specific search terms are provided - Ingredient-Based Discovery: Include or exclude recipes based on specific ingredients ### Intelligent Filtering - Category Filters: Narrow results to specific meal categories (e.g., Seafood, Dessert, Chicken) - Cuisine Filters: Focus on recipes from particular regions or areas (e.g., Italian, Mexican, Asian) - Dietary Restrictions: Support for low-sugar, vegetarian, and vegan filtering using smart heuristics ### Data Enrichment - Complete Recipe Details: Extracts ingredients, step-by-step instructions, and serving information - Metadata Collection: Includes cooking times, nutritional data placeholders, and source URLs - Image URLs: Captures high-quality recipe images for visual applications ### Performance & Reliability - Efficient API Usage: Minimizes requests through intelligent caching and batch processing - Configurable Result Limits: Control the number of recipes collected per run - Error Handling: Robust handling of API failures and rate limits ## Input jsonc ## Input Parameters Configure your recipe scraping job using the following input parameters:json { "keyword": "chicken soup", "includeIngredients": ["chicken", "carrot"], "excludeIngredients": ["cream", "butter"], "categories": ["Chicken", "Soup"], "areas": ["American", "British"], "specialDiets": ["Low sugar"], "results_wanted": 50, "mealDbApiKey": "1", "mealDbDebug": false } ### Parameter Details | Parameter | Type | Description | Default | Required | |-----------|------|-------------|---------|----------| | `keyword` | string | Search term for meal names. Performs exact and partial matches via TheMealDB search API. | `""` | No | | `includeIngredients` | array[string] | List of ingredients that must be present in each recipe. Used for targeted discovery. | `[]` | No | | `excludeIngredients` | array[string] | List of ingredients to avoid. Recipes containing any of these will be filtered out. | `[]` | No | | `categories` | array[string] | Meal categories to include (e.g., "Seafood", "Dessert", "Chicken"). Case-insensitive matching. | `[]` | No | | `areas` | array[string] | Geographic regions or cuisines to focus on (e.g., "Italian", "Mexican", "Asian"). | `[]` | No | | `specialDiets` | array[string] | Dietary filters: "Low sugar", "Vegetarian", "Vegan". Uses ingredient-based heuristics. | `[]` | No | | `results_wanted` | integer | Maximum number of recipes to collect and save. | `100` | No | | `mealDbApiKey` | string | API key for TheMealDB. Use "1" for free tier testing. | `"1"` | No | | `mealDbDebug` | boolean | Enable detailed logging of API requests and responses for troubleshooting. | `false` | No | ### Parameters | Name | Type | Description | Default | |------|------|-------------|---------| | keyword | string | Keyword (meal name fragment) to search via search.php. | "" | | includeIngredients | array[string] | Each ingredient must appear in the recipe. Also used to seed ingredient filters. | [] | | excludeIngredients | array[string] | Omit recipes containing any of these ingredients. | [] | | categories | array[string] | Only accept recipes matching these TheMealDB categories. Also seeds category filters. | [] | | areas | array[string] | Only accept recipes from these cuisines/areas. Also seeds area filters. | [] | | specialDiets | array[string] | Optional heuristics for “Low sugar”, “Vegetarian”, “Vegan”. | [] | | results_wanted | integer | Maximum number of recipes to save. | 100 | | mealDbApiKey | string | Custom TheMealDB API key (use 1 for free tests). | "1" | | mealDbDebug | boolean | Log every MealDB request and response size. | false | ## Output Schema Each scraped recipe is saved as a structured JSON object with the following schema: json { "title": "Chicken Handi", "url": "https://www.themealdb.com/meal/52795", "cook_time": null, "special_diets": ["Indian"], "nutrition": {}, "ingredients": [ "2 tsp Chili Powder", "1 kg Chicken", "1 cup Water", "1 tsp Salt" ], "method": [ "Heat oil in a pan...", "Add spices and stir...", "Add chicken and cook...", "Serve hot with rice." ], "serves": null, "image_url": "https://www.themealdb.com/images/media/meals/wyxwsp1486979827.jpg", "description": "Heat oil in a pan and add chopped onions...", "_source": "themealdb", "scraped_at": "2025-11-16T14:00:00.000Z" } ### Field Descriptions - title: Recipe name - url: Direct link to the full recipe on TheMealDB - cook_time: Cooking time (currently null, placeholder for future enhancement) - special_diets: Array of dietary tags from the source - nutrition: Placeholder object for nutritional information - ingredients: List of ingredients with measurements - method: Step-by-step cooking instructions - serves: Number of servings (when available) - image_url: URL to recipe image - description: Full recipe description - _source: Data source identifier - scraped_at: Timestamp of data extraction ## Usage Examples ### Basic Recipe Search Search for recipes containing "pasta" and collect up to 25 results: json { "keyword": "pasta", "results_wanted": 25 } ### Ingredient-Based Filtering Find vegetarian recipes with tomatoes but without cheese: json { "includeIngredients": ["tomato"], "excludeIngredients": ["cheese"], "specialDiets": ["Vegetarian"], "results_wanted": 30 } ### Cuisine-Specific Collection Collect Italian dessert recipes: json { "categories": ["Dessert"], "areas": ["Italian"], "results_wanted": 20 } ### Comprehensive Catalog Gather a large collection of recipes across all categories: json { "results_wanted": 500 } ## Configuration Options ### API Key Management - Use the default key "1" for testing and small-scale scraping - Obtain a personal API key from TheMealDB for higher rate limits in production - Set via mealDbApiKey parameter or environment variable ### Result Limiting - Adjust results_wanted based on your use case - Lower values for quick testing, higher for comprehensive datasets - The scraper stops automatically once the limit is reached ### Debug Mode - Enable mealDbDebug to monitor API calls and response sizes - Useful for troubleshooting filtering issues or API problems ## How It Works 1. Input Processing: Parses and validates search parameters and filters 2. API Querying: Makes targeted requests to TheMealDB API endpoints 3. Data Filtering: Applies ingredient, category, area, and dietary filters 4. Detail Enrichment: Fetches full recipe details for matching summaries 5. Data Standardization: Converts API responses to consistent output format 6. Result Streaming: Saves processed recipes to the output dataset The scraper uses multiple search strategies: - Direct keyword searches for specific recipes - Ingredient-based filtering for component matching - Category and area filtering for thematic collections - Alphabetical crawling for comprehensive catalog extraction ## Limitations - Dependent on TheMealDB API availability and data quality - Free API tier has rate limiting (upgrade with personal API key) - Dietary filtering uses heuristic-based detection, not guaranteed 100% accuracy - Some recipes may lack complete metadata (cooking times, nutritional info) - Maximum 20 ingredients supported per recipe (API limitation) ## Support For issues, feature requests, or questions about this recipe scraper: - Check the input parameters and output schema for configuration guidance - Verify API key validity if encountering rate limit errors - Ensure filter combinations are not overly restrictive (may result in no matches) - Review debug logs when mealDbDebug is enabled for troubleshooting This tool is designed for reliable, efficient recipe data extraction to power your food-related applications and services.