Capterra Scraper

Features Capterra doesn't provide a very flexible or free API, but this scraper acts as an unofficial Capterra API to help you extract the data you need, when you need it, and at scale. Capterra Scraper supports the following features: - Scrape lists - Scrape resources - Scrape categories - Scrape products - Scrape reviews - Scrape services - Scrape service reviews Capterra provides "Best business software. With software reviews, rating infographics, and the most comprehensive list of the top". Scraping that content and extracting it in a structured format could give you invaluable business insights and an edge over the competition. ## Tutorial Check out this blog post on how to extract data from Capterra with unofficial Capterra API for more information on the scraper. ## Bugs, fixes, updates, and changelog This scraper is under active development. If you have any feature requests, you can create an issue from here. ### Upcoming changes - Retrieve comparisons. - Enrich reviews and output of the products. ## Input parameters The input of this scraper should be JSON containing the list of pages o that should be visited. Possible fields are: - startUrls: (Required) (Array) List of Capterra URLs. You should only provide list or detail URLs. - includeReviews: (Optional) (Boolean) Adding reviews into the product objects is optional and by default, it is false. If you want to scrape the reviews of the companies, then you can set this option as true. - reviewsOnly: (Optional) (Boolean) If you're only interested in reviews and not the full product/service data, set this to true. When this is set along with service URLs, the scraper will directly fetch reviews from the API. - endPage: (Optional) (Number) Final number of page that you want to scrape. The default is Infinite. This applies to all startUrls individually. - endPageForReviews: (Optional) (Number) Maximum number of review pages to scrape for each product/service. Default is Infinite. - maxItems: (Optional) (Number) You can limit scraped products. This should be useful when you search big lists. - proxy: (Required) (Proxy Object) Proxy configuration. - customMapFunction: (Optional) (String) Function that takes each object's handle as an argument and returns the object with executing the function. This solution requires the use of proxy servers, either your own proxy servers or Apify Proxy. ### Tips When you want to scrape over a specific listing URL, just copy and paste the link as one of the startURL. If you would like to scrape only the first page of a list, then add the link for the page and have the endPage as 1. Please also keep in mind that the includeReviews parameter will add multiple requests per product. That's why the number of requests or CUs that are consumed might be higher if you set this option as true. ## Compute unit consumption Capterra Scraper is optimized to run extremely fast and scrape many listings as possible, so it forefronts all listing detail requests. If the actor doesn't get blocked very often, it will scrape 100 listings in 2 minutes and consume ~0.07-0.08 compute units. ### Capterra Scraper input example json { "startUrls": [ "https://www.capterra.com/p/210664/CRYENGINE/", "https://www.capterra.com/services/sp/178/accenture/", "https://www.capterra.com/services/data-analytics/", "https://www.capterra.com/business-intelligence-software/" ], "includeReviews": false, "proxy": { "useApifyProxy": true }, "maxItems": 50 } ## During the run During the run, the actor will output messages letting you know what is going on. Each message always contains a short label specifying which page from the provided list is currently specified. When items are loaded from the page, you should see a message about this event with a loaded item count and total item count for each page. If you provide incorrect input to the actor, it will immediately stop with a failure state and output an explanation of what is wrong. ## Capterra export During the run, the actor stores results into a dataset. Each item is a separate item in the dataset. You can manage the results in any language (Python, PHP, Node.js/NPM). See the FAQ or our API reference to learn more about getting results from this Capterra actor. ## Scraped Capterra Product example The structure of each item in Capterra product looks like this: json { "productId": "19319", "name": "Jira", "productUrl": "https://www.capterra.com/p/19319/JIRA/", "slug": "JIRA", "logoUrl": "https://gdm-catalog-fmapi-prod.imgix.net/ProductLogo/e1b492c5-8a97-4b86-a422-d317b2480afa.png", "reviewCount": 15190, "overallRating": 4.4436800526662275, "easeOfUseRating": 4.1, "customerServiceRating": 4.2, "functionalityRating": null, "valueForMoneyRating": 4.3, "recommendationRating": null, "longDescription": "Jira is particularly well suited to software development and agile teams. Its versatility, however, extends beyond software development to product management, IT operations, and customer support, and it offers robust issue tracking and task management features. While many small businesses take advantage of the more basic free and lower priced versions, Jira's cost increases significantly when adding users, premium features and required plugins. \n", "pricingDetails": { "startingPrice": "$$7.53", "pricingModel": "Per User", "paymentFrequency": "Per Month", "hasFreeTrial": true, "hasFreeVersion": true }, "features": [ { "title": "Task management", "rating": 4.5, "reviewCount": 563 }, { "title": "Project planning/scheduling", "rating": 4.4, "reviewCount": 319 }, { "title": "Task progress tracking", "rating": 4.4, "reviewCount": 332 }, { "title": "Reporting/project tracking", "rating": 4.4, "reviewCount": 153 }, { "title": "Prioritization", "rating": 4.3, "reviewCount": 197 }, { "title": "Project planning", "rating": 4.2, "reviewCount": 166 }, { "title": "Budget Management", "rating": 2.6, "reviewCount": 29 }, { "title": "Task import", "rating": 2.7, "reviewCount": 27 }, { "title": "Email-to-task-conversion", "rating": 2.9, "reviewCount": 23 }, { "title": "Multi-channel communication", "rating": 2.9, "reviewCount": 27 }, { "title": "Client portal", "rating": 3, "reviewCount": 24 }, { "title": "Online time clock", "rating": null, "reviewCount": null }, { "title": "Advanced Roadmaps", "rating": null, "reviewCount": null }, { "title": "Custom Workflows", "rating": null, "reviewCount": null }, { "title": "Advanced reporting and dashboards", "rating": null, "reviewCount": null }, { "title": "Portfolio management", "rating": null, "reviewCount": null }, { "title": "Automation", "rating": null, "reviewCount": null }, { "title": "Atlassian Intelligence", "rating": null, "reviewCount": null }, { "title": "UI Improvements", "rating": null, "reviewCount": null } ], "training": [ { "value": "2", "label": "In Person", "selected": false }, { "value": "3", "label": "Live Online", "selected": true }, { "value": "4", "label": "Webinars", "selected": true }, { "value": "5", "label": "Documentation", "selected": true }, { "value": "6", "label": "Videos", "selected": true } ], "support": [ { "value": "1", "label": "Email/Help Desk", "selected": true }, { "value": "2", "label": "FAQs/Forum", "selected": true }, { "value": "3", "label": "Knowledge Base", "selected": true }, { "value": "4", "label": "Phone Support", "selected": true }, { "value": "5", "label": "24/7 (Live rep)", "selected": true }, { "value": "6", "label": "Chat", "selected": true } ], "targetCompanySizes": [ { "value": "1", "label": "Self-Employed", "selected": true }, { "value": "2", "label": "2-10", "selected": true }, { "value": "3", "label": "11-50", "selected": true }, { "value": "4", "label": "51-200", "selected": true }, { "value": "5", "label": "201-500", "selected": true }, { "value": "6", "label": "501-1000", "selected": true }, { "value": "7", "label": "1000+", "selected": true } ], "deploymentOptions": [ { "value": "1", "label": "Cloud, SaaS, Web-Based", "selected": true }, { "value": "2", "label": "Desktop Mac", "selected": true }, { "value": "3", "label": "Desktop Windows", "selected": true }, { "value": "4", "label": "Desktop Linux", "selected": true }, { "value": "5", "label": "On-Premise Windows", "selected": true }, { "value": "6", "label": "On-Premise Linux", "selected": true }, { "value": "7", "label": "Desktop Chromebook", "selected": false }, { "value": "8", "label": "Mobile Android", "selected": true }, { "value": "9", "label": "Mobile iPhone", "selected": true }, { "value": "10", "label": "Mobile iPad", "selected": true } ], "relatedProducts": [], "category": { "name": "IT Project Management Software", "slug": "/it-project-management-software" }, "bestCategoryShortName": "IT Project Management Software", "bestCategory": "/it-project-management-software", "vendor": {}, "reviewUrl": "https://www.capterra.com/p/19319/JIRA/reviews", "reviews": [] } ## Scraped Capterra Single Review example The structure of each reviews (single review) in Capterra looks like this: json { "reviewId": "3748991", "title": "Useful tool but could be better", "writtenOn": "August 11, 2022", "generalComments": "A really useful tool, but has some kinks that need ironing out.", "incentivized": "NominalGift", "customerSupportRating": "0.0", "easeOfUseRating": "3.0", "functionalityRating": "3.0", "valueForMoneyRating": "0.0", "overallRating": "3.0", "recommendationRating": "6.0", "consText": "The notifications come through to your e-mail inbox quite a long time after comments were made. The latest update, despite making copying comments easier, isn't as straight forward as the old drag and drop interface.", "prosText": "I really like that you can easily comment on a video project, graphic, written piece of work and pin point exactly where you need the changes, the user interface is relatively user friendly, however, I probably preferred it before the latest update.", "sourceSite": "Capterra", "globalReviewId": "Capterra___3748991", "anonymityOn": false, "completenessScore": 1.23, "adviceToOthers": null, "chosenReasons": null, "switchingReasons": null, "reviewSource": { "code": "NGC", "tooltip": "Nominal Gift Card: This reviewer was invited by us to submit an honest review and was offered a nominal incentive as a thank you." }, "reviewer": { "companySize": "10,001+ employees", "industry": "Media Production", "timeUsedProduct": "2+ years", "fullName": "Thom H.", "jobTitle": "Assistant Producer", "verifiedLinkedIn": false, "profilePicUrl": null }, "alternativeProducts": [], "switchedProducts": [], "vendorResponse": { "date": "September 15, 2022", "text": "Hi Thom, thanks for your great feedback!\r\n\r\nWe really appreciate you've taken the time to leave this review.\r\n\r\nWe've shared your feedback with our team and they are looking into it, so you should look out for some exciting news about it very soon!\r\n\r\nThanks again,\r\nFotini Athineli, Social Media Manager ", "name": "Filestage" } } ## Contact Please visit us through epctex.com to see all the products that are available for you. If you are looking for any custom integration or so, please reach out to us through the chat box in epctex.com. In need of support? business@epctex.com is at your service.