ONE Line Shipping Price Scraper ## Description The ONE Line Shipping Price Scraper is an Apify Actor that scrapes shipping price data from the ONE Line e-commerce platform (https://ecomm.one-line.com). It retrieves pricing details for specified origin and destination locations, container types, and commodity codes or names. The Actor supports two authentication methods: - Token-Based Authentication (Priority): Provide access_token and refresh_token directly for API access. - Username/Password Authentication (Fallback): Use a username and password to log in via Selenium if tokens are unavailable or invalid. The Actor stores authentication tokens in Apify's key-value store for reuse, refreshes tokens when expired, and handles errors gracefully. It outputs structured data including departure/arrival dates, total price, and detailed freight information (e.g., service codes, charges). ## Features - Supports authentication via tokens or username/password. - Validates and resolves location and commodity codes. - Handles multiple container types (DRY 20, DRY 40, DRY 40H) with quantity and weight. - Provides detailed freight information, including origin/destination charges and transit details. - Displays data in a table view in the Apify UI. ## Prerequisites To use this Actor, you need: - An Apify account (sign up at https://console.apify.com). - Valid ONE Line credentials: - Access and refresh tokens (preferred), or - Username and password for login. - Basic familiarity with the Apify platform for configuring and running Actors. ## Input Schema The Actor expects input with the following fields: | Field Name | Type | Description | | --- | --- | --- | | access_token | string | Access token for ONE Line API authentication (requires refresh_token). | | refresh_token | string | Refresh token for ONE Line API authentication (requires access_token). | | username | string | Username for ONE Line login (requires password, used if tokens are missing). | | password | string | Password for ONE Line login (requires username, used if tokens are missing). | | origin | string | Origin location name or code (e.g., 'dam' or 'EGDAM'). | | destination | string | Destination location name or code (e.g., 'VNCMP' or 'Cai Mep'). | | containers | array | List of container objects with name, quantity, and weight. | | commodity | string | Commodity code (e.g., '091091') or name. | Notes: - Either access_token and refresh_token or username and password must be provided. - containers items are objects with name (string, e.g., "DRY 40"), quantity (integer), and weight (integer). ### Example Input #### Token-Based Authentication json { "access_token": "your_access_token", "refresh_token": "your_refresh_token", "origin": "dam", "destination": "VNCMP", "containers": [ {"name": "DRY 40", "quantity": 5, "weight": 9000}, {"name": "DRY 20", "quantity": 1, "weight": 10000} ], "commodity": "091091" } #### Username/Password Authentication json { "username": "your_username", "password": "your_password", "origin": "dam", "destination": "VNCMP", "containers": [ {"name": "DRY 40", "quantity": 5, "weight": 9000}, {"name": "DRY 20", "quantity": 1, "weight": 10000} ], "commodity": "091091" } ## Output Schema The Actor outputs an array of shipping price records. Key fields are listed below, with nested freightInfos fields included for display in the Apify UI table view: | Field Name | Type | Description | | --- | --- | --- | | departureDateEstimated | string | Estimated departure date (YYYY-MM-DD). | | arrivalDateEstimated | string | Estimated arrival date (YYYY-MM-DD). | | totalPrice | number | Total price in USD. | | freightInfos.serviceCode | string | Service code (e.g., "MD1"). | | freightInfos.serviceName | string | Service name (e.g., "MEDITERRANEAN 1"). | | freightInfos.transportName | string | Name of the transport (e.g., "YM WINNER"). | | freightInfos.price | number | Price for this freight service. | | freightInfos.duration | integer | Duration in days. | | freightInfos.status | string | Status of the service (e.g., "Available"). | | freightInfos.numberOfTransits | integer | Number of transits. | | freightInfos.originCharges | array | Charges at origin (array of charge objects). | | freightInfos.destinationCharges | array | Charges at destination (array of charge objects). | | freightInfos.basicOceanFreightCharges | array | Basic ocean freight charges (array of charge objects). | Notes: - freightInfos is an array of objects; the table view displays fields from the first item. - Charge fields (originCharges, etc.) are arrays of objects with details like chargeCode, chargeAmount, and totalAmountInUSD. ### Example Output json [ { "departureDateEstimated": "2025-04-27", "arrivalDateEstimated": "2025-06-06", "totalPrice": 3644.32, "freightInfos": [ { "serviceCode": "MD1", "serviceName": "MEDITERRANEAN 1", "transportName": "YM WINDOW", "conveyanceNumber": "043E", "duration": 40, "price": 3644.32, "status": "Available", "vgmCutoff": "2025-04-24 13:00", "docCutoff": "2025-04-23 13:00", "cyCutoff": "2025-04-25 17:00", "portCutoff": "2025-04-25 17:00", "routeType": "TRANSIT", "numberOfTransits": 1, "pudoVas": false, "originCharges": [ { "chargeCode": "CSS", "chargeName": "CARRIER SECURITY SURCHARGE", "chargeAmount": "6", "chargeCurrency": "USD", "exchangeRatio": "1.0", "surchargeCondition": "fixed", "totalAmount": 6, "cargoWeight": "10000", "unit": "KGS", "totalAmountInUSD": 6 }, { "chargeCode": "EMS", "chargeName": "EMERGENCY SURCHARGE", "chargeAmount": "250", "chargeCurrency": "USD", "exchangeRatio": "1.0", "surchargeCondition": "fixed", "totalAmount": 250, "cargoWeight": "10000", "unit": "KGS", "totalAmountInUSD": 250 }, { "chargeCode": "CSS", "chargeName": "CARRIER SECURITY SURCHARGE", "chargeAmount": "6", "chargeCurrency": "USD", "exchangeRatio": "1.0", "surchargeCondition": "fixed", "totalAmount": 6, "cargoWeight": "9000", "unit": "KGS", "totalAmountInUSD": 6 }, { "chargeCode": "EMS", "chargeName": "EMERGENCY SURCHARGE", "chargeAmount": "250", "chargeCurrency": "USD", "exchangeRatio": "1.0", "surchargeCondition": "fixed", "totalAmount": 250, "cargoWeight": "9000", "unit": "KGS", "totalAmountInUSD": 250 }, { "chargeCode": "CSS", "chargeName": "CARRIER SECURITY SURCHARGE", "chargeAmount": "6", "chargeCurrency": "USD", "exchangeRatio": "1.0", "surchargeCondition": "fixed", "totalAmount": 6, "cargoWeight": "11000", "unit": "KGS", "totalAmountInUSD": 6 }, { "chargeCode": "EMS", "chargeName": "EMERGENCY SURCHARGE", "chargeAmount": "250", "chargeCurrency": "USD", "exchangeRatio": "1.0", "surchargeCondition": "fixed", "totalAmount": 250, "cargoWeight": "11000", "unit": "KGS", "totalAmountInUSD": 250 } ], "destinationCharges": [ { "chargeCode": "CMC", "chargeName": "CNTR MAINTENANCE CHARGE", "chargeAmount": "315000", "chargeCurrency": "VND", "exchangeRatio": "0.00003918", "surchargeCondition": "fixed", "totalAmount": 315000, "cargoWeight": "10000", "unit": "KGS", "totalAmountInUSD": 12.34 }, { "chargeCode": "CCC", "chargeName": "COLLECTION OF CNTR CLEANING FEE", "chargeAmount": "105000", "chargeCurrency": "VND", "exchangeRatio": "0.00003918", "surchargeCondition": "fixed", "totalAmount": 105000, "cargoWeight": "10000", "unit": "KGS", "totalAmountInUSD": 4.11 }, { "chargeCode": "THD", "chargeName": "TERMINAL HANDLING CHARGE (D)", "chargeAmount": "2800000", "chargeCurrency": "VND", "exchangeRatio": "0.00003918", "surchargeCondition": "fixed", "totalAmount": 2800000, "cargoWeight": "10000", "unit": "KGS", "totalAmountInUSD": 109.7 }, { "chargeCode": "VVN", "chargeName": "VIETNAM TAX (GENERAL)", "chargeAmount": "216835.6", "chargeCurrency": "VND", "exchangeRatio": "0.00003918", "surchargeCondition": "subject to", "totalAmount": 216835.6, "cargoWeight": "10000", "unit": "KGS", "totalAmountInUSD": 8.49 }, { "chargeCode": "CMC", "chargeName": "CNTR MAINTENANCE CHARGE", "chargeAmount": "630000", "chargeCurrency": "VND", "exchangeRatio": "0.00003918", "surchargeCondition": "fixed", "totalAmount": 630000, "cargoWeight": "9000", "unit": "KGS", "totalAmountInUSD": 24.68 }, { "chargeCode": "CCC", "chargeName": "COLLECTION OF CNTR CLEANING FEE", "chargeAmount": "210000", "chargeCurrency": "VND", "exchangeRatio": "0.00003918", "surchargeCondition": "fixed", "totalAmount": 210000, "cargoWeight": "9000", "unit": "KGS", "totalAmountInUSD": 8.22 }, { "chargeCode": "THD", "chargeName": "TERMINAL HANDLING CHARGE (D)", "chargeAmount": "4500000", "chargeCurrency": "VND", "exchangeRatio": "0.00003918", "surchargeCondition": "fixed", "totalAmount": 4500000, "cargoWeight": "9000", "unit": "KGS", "totalAmountInUSD": 176.31 }, { "chargeCode": "VVN", "chargeName": "VIETNAM TAX (GENERAL)", "chargeAmount": "328411.2", "chargeCurrency": "VND", "exchangeRatio": "0.00003918", "surchargeCondition": "subject to", "totalAmount": 328411.2, "cargoWeight": "9000", "unit": "KGS", "totalAmountInUSD": 12.86 }, { "chargeCode": "CMC", "chargeName": "CNTR MAINTENANCE CHARGE", "chargeAmount": "630000", "chargeCurrency": "VND", "exchangeRatio": "0.00003918", "surchargeCondition": "fixed", "totalAmount": 630000, "cargoWeight": "11000", "unit": "KGS", "totalAmountInUSD": 24.68 }, { "chargeCode": "CCC", "chargeName": "COLLECTION OF CNTR CLEANING FEE", "chargeAmount": "210000", "chargeCurrency": "VND", "exchangeRatio": "0.00003918", "surchargeCondition": "fixed", "totalAmount": 210000, "cargoWeight": "11000", "unit": "KGS", "totalAmountInUSD": 8.22 }, { "chargeCode": "THD", "chargeName": "TERMINAL HANDLING CHARGE (D)", "chargeAmount": "4500000", "chargeCurrency": "VND", "exchangeRatio": "0.00003918", "surchargeCondition": "fixed", "totalAmount": 4500000, "cargoWeight": "11000", "unit": "KGS", "totalAmountInUSD": 176.31 }, { "chargeCode": "VVN", "chargeName": "VIETNAM TAX (GENERAL)", "chargeAmount": "328411.2", "chargeCurrency": "VND", "exchangeRatio": "0.00003918", "surchargeCondition": "subject to", "totalAmount": 328411.2, "cargoWeight": "11000", "unit": "KGS", "totalAmountInUSD": 12.86 }, { "chargeCode": "DOF", "chargeName": "DOC FEE (DEST)", "chargeAmount": "900000", "chargeCurrency": "VND", "exchangeRatio": "0.00003918", "surchargeCondition": "subject to", "totalAmount": 900000, "cargoWeight": "", "unit": "", "totalAmountInUSD": 35.26 } ], "basicOceanFreightCharges": [ { "chargeCode": "OFT", "chargeName": "BASIC OCEAN FREIGHT", "chargeAmount": "50", "chargeCurrency": "USD", "exchangeRatio": "1.0", "surchargeCondition": "fixed", "totalAmount": 50, "cargoWeight": "10000", "unit": "KGS", "totalAmountInUSD": 50 }, { "chargeCode": "OFT", "chargeName": "BASIC OCEAN FREIGHT", "chargeAmount": "50", "chargeCurrency": "USD", "exchangeRatio": "1.0", "surchargeCondition": "fixed", "totalAmount": 50, "cargoWeight": "9000", "unit": "KGS", "totalAmountInUSD": 50 }, { "chargeCode": "OFT", "chargeName": "BASIC OCEAN FREIGHT", "chargeAmount": "50", "chargeCurrency": "USD", "exchangeRatio": "1.0", "surchargeCondition": "fixed", "totalAmount": 50, "cargoWeight": "11000", "unit": "KGS", "totalAmountInUSD": 50 } ], "departures": [ { "transportType": "Ocean Vessel", "transportName": "YM WINDOW", "departureDateEstimated": "2025-04-27 18:00", "departureTerminal": "DAMIETTA, EGYPT (DAMIETTA CONTAINER TERMINAL)", "arrivalDateEstimated": "2025-05-30 08:00", "arrivalTerminal": "SINGAPORE (PSA CORPORATION LIMITED)", "departureLoc": "EGDAM", "arrivalLoc": "SGSIN", "transitTimeInHrs": 782 }, { "transportType": "Ocean Vessel", "transportName": "ONE COMPETENCE", "departureDateEstimated": "2025-06-04 18:00", "departureTerminal": "SINGAPORE (PSA CORPORATION LIMITED)", "arrivalDateEstimated": "2025-06-06 09:30", "arrivalTerminal": "CAI MEP (TCIT (TAN CANG CAIMEP INTL TERMINAL))", "departureLoc": "SGSIN", "arrivalLoc": "VNCMT", "transitTimeInHrs": 40 } ], "arrival": { "transportType": "Ocean Vessel", "transportName": "ONE COMPETENCE", "departureDateEstimated": "2025-06-04 18:00", "departureTerminal": "SINGAPORE (PSA CORPORATION LIMITED)", "arrivalDateEstimated": "2025-06-06 09:30", "arrivalTerminal": "CAI MEP (TCIT (TAN CANG CAIMEP INTL TERMINAL))", "departureLoc": "SGSIN", "arrivalLoc": "VNCMT", "transitTimeInHrs": 40 }, "arrivalPod": { "transportType": "Ocean Vessel", "transportName": "ONE COMPETENCE", "departureDateEstimated": "2025-06-04 18:00", "departureTerminal": "SINGAPORE (PSA CORPORATION LIMITED)", "arrivalDateEstimated": "2025-06-06 09:30", "arrivalTerminal": "CAI MEP (TCIT (TAN CANG CAIMEP INTL TERMINAL))", "departureLoc": "SGSIN", "arrivalLoc": "VNCMT", "transitTimeInHrs": 40 } } ] } ] ## Usage 1. Access the Actor: - Find the Actor in the Apify Store or deploy it from your Apify account. 2. Configure Input: - In the Apify UI, go to the Actor's input tab. - Provide either: - access_token and refresh_token, or - username and password. - Specify origin, destination, containers, and commodity as shown in the example input, using names or codes 3. Run the Actor: - Click "Run" in the Apify UI or use the Apify API to execute the Actor. - Monitor logs for authentication status and price fetching progress. 4. View Results: - Check the dataset tab in the Apify UI for the scraped price data. - The table view displays key fields (e.g., departureDateEstimated, totalPrice, freightInfos.serviceCode). - Download the full dataset in JSON, CSV, or other formats for detailed analysis. ## Troubleshooting - Authentication Errors: - Ensure valid access_token and refresh_token or correct username and password.