FMCSA DOT Crawler ## Overview The FMCSA DOT Crawler collects structured information about U.S. motor carriers directly from the FMCSA SAFER "Company Snapshot" pages. It supports keyword‑style filtering (DOT range, state, fleet size, etc.) and an optional Premium mode that adds crash statistics, safety ratings and public email addresses. Typical use–cases include: - Lead generation for logistics or insurance companies - Competitor or market‑size analysis - Compliance monitoring and risk assessment - Building datasets of motor carriers for analysis - Monitoring FMCSA updates for compliance or reporting - Integrating carrier data with logistics systems --- ## Key Features * Comprehensive Data Extraction: Scrapes legal names, DBA names, phone numbers, addresses, operational stats, and cargo types. * Flexible filters – pull only the carriers you need by DOT number window or add‑date. * Straight‑forward output – each company snapshot is returned as a single JSON object that is easy to ingest downstream. * Resilient crawling – automatic retries, exponential back‑off and proxy rotation keep the run moving even when SAFER throttles. * Efficient Processing – optimized for crawling millions of records with fast and memory-efficient streaming --- ## Input Schema The scraper reads the parameters below at run‑time. Required fields are marked bold. | Field | Type | Description | | ----------------- | ------------------- | -------------------------------------------------------------- | | dot_start | integer | First DOT number to process (inclusive) | | max_results | integer | Maximum number of records to process (0 → no limit) | | dot_end | integer | Last DOT number to process (inclusive). Defaults to 10,000,000 | | add_after_date | string (YYYY‑MM‑DD) | Include only carriers registered on or after this date | | sp_intended_usage | string | Short sentence describing how you plan to use the data. | | sp_improvement_suggestions | string | Your feedback for future upgrades. | | sp_contact | string | Contact email where we can reach you for feedback/collaboration | ### Example Input json { "dot_start": 2000000, "max_results": 10000, "dot_end": 2100000, "add_after_date": "2020-01-01", "sp_intended_usage": "Outbound prospecting list for carriers", "sp_improvement_suggestions": "Option to filter by cargo type" "sp_contact": "john@example.com" } --- ## Output Fields Each successful company snapshot is returned as a flat JSON object: | Field | Standard | Premium | | ---------------------------------------------------------------- | :------: | :-----: | | DOT_num | ✓ | ✓ | | entity_type, legal_name, dba_name | ✓ | ✓ | | mc_mx_ff_numbers | ✓ | ✓ | | phone, physical_address, mailing_address | ✓ | ✓ | | power_units, drivers | ✓ | ✓ | | cargo_carried, carrier_operation, operation_classification | ✓ | ✓ | | DUNS_num | ✓ | ✓ | | inspections_us (vehicle / driver / hazmat / iep) | ✓ | ✓ | | inspections_ca (vehicle / driver) | ✓ | ✓ | | email | | ✓ | | crashes_us (fatal / injury / tow / total) | | ✓ | | crashes_ca (fatal / injury / tow / total) | | ✓ | | safety_rating (rating_date / review_date / rating / type) | | ✓ | ### Sample (Standard) json { "DOT_num": "2802023", "entity_type": "Carrier", "legal_name": "Example Logistics LLC", "dba_name": "Example Trucks", "mc_mx_ff_numbers": "MC‑123456", "phone": "555‑123‑4567", "physical_address": "123 Main St, Springfield, IL", "mailing_address": "PO Box 456, Springfield, IL", "power_units": "50", "drivers": "75", "cargo_carried": ["General Freight", "Household Goods"], "carrier_operation": ["Interstate"], "operation_classification": ["For Hire"], "DUNS_num": "012345678", "inspections_us": { "vehicle": "10", "driver": "2", "hazmat": "0", "iep": "0" }, "inspections_ca": { "vehicle": "–", "driver": "–" } } | Field | Type | Description | | -------------------------- | ------- | ------------------------------------------------------------- | | DOT_num | string | DOT registration number | | entity_type | string | Business type (CARRIER, BROKER, FREIGHT FORWARDER, etc.) | | legal_name | string | Legal business name | | dba_name | string | "Doing Business As" name | | mcs150_date | string | MCS-150 form update date (MM-DD-YY format) | | mcs150_mileage | string | Annual mileage reported on MCS-150 | | mcs150_mileage_year | string | Year for reported mileage | | mc_mx_ff_numbers | string | MC/MX/FF docket numbers | | phone | string | Primary phone number (digits only) | | cell_phone | string | Cell phone number | | physical_address | string | Physical business address | | mailing_address | string | Mailing address | | power_units | integer | Number of power units (trucks, tractors) | | drivers | integer | Total number of drivers | | truck_units | integer | Number of truck units | | bus_units | integer | Number of bus units | | fleet_size | string | Fleet size category (e.g., "1", "2-3", "4-6", "76-100", etc.) | | cargo_carried | array | Types of cargo transported | | carrier_operation | array | Operation type (Interstate, Intrastate Only, etc.) | | operation_classification | array | Operational classifications | | company_officer_1 | string | Primary company officer | | company_officer_2 | string | Secondary company officer | | DUNS_num | string | Dun & Bradstreet number | | email | string | Email address (when available in source data) | | inspections_us | object | US inspection counts (driver, vehicle, hazmat, iep) | | inspections_ca | object | Canadian inspection counts (currently null) | ### Sample Output json { "DOT_num": "2802023", "entity_type": "CARRIER", "legal_name": "Example Logistics LLC", "dba_name": "Example Trucks", "mcs150_date": "09-06-24", "mcs150_mileage": "120000", "mcs150_mileage_year": "2024", "mc_mx_ff_numbers": "MC123456", "phone": "5551234567", "cell_phone": "5559876543", "physical_address": "123 Main St, Springfield, IL, 62701, US", "mailing_address": "PO Box 456, Springfield, IL, 62701, US", "power_units": 50, "drivers": 75, "truck_units": 45, "bus_units": 0, "fleet_size": "45-55", "cargo_carried": ["General Freight", "Building Materials"], "carrier_operation": ["Interstate"], "operation_classification": ["For Hire"], "company_officer_1": "John Smith", "company_officer_2": "Jane Doe", "DUNS_num": "012345678", "email": "contact@examplelogistics.com", "inspections_us": { "driver": "12", "vehicle": "8", "hazmat": "2", "iep": "0" }, } --- ## Notes & Best Practices * Stick to a single primary filter per run – combining filters is currently not supported. * Records are filtered to include only active carriers (STATUS_CODE = 'A') * Phone numbers are cleaned to contain only digits * MCS-150 dates are formatted from YYYYMMDD to MM-DD-YY for consistency --- ## Need More Features? If you'd like to add new data fields to this scraper or need a custom scraper for another purpose, feel free to file an issue or get in touch! We are open to customizing the scraper to suit your needs. ## Why Choose This Scraper? - Efficient: Processes a high volume of requests with intelligent retries and session management. - Customizable: Tailor input options to suit your scraping needs. - Reliable: Leverages robust error-handling and proxy configurations for uninterrupted scraping. - Flexible: Process only the records you need ## Proxies and Anti-blocking This scraper uses the Apify Proxy or your custom proxy settings to reduce the risk of being blocked. It supports automatic session management for smooth operation. ## Resources - Apify SDK Documentation - CheerioCrawler Documentation --- Enhance your transportation data collection capabilities with the FMCSA DOT Crawler. Get started today!