What is a web crawler?
A web crawler is a bot—a software program—that systematically visits a website, or sites, and catalogs the data it finds.
Read Article
Crawler
-
-
Algolia Crawler
An overview of what the Algolia Crawler can do for your website.
Watch video
-
Site crawling & federated search: How to make content discoverable
In this article, we will discuss how site crawling and federated search can make your site more organized and your content more discoverable.
Read Article
-
Complete documentation for a seamless integration
Access now -
Master Class: The business value of the Algolia Crawler
Discover how World’s leading brands are using Algolia's crawler to turn their content into manageable and meaningful experiences.
Watch video
-
Algolia Crawler
Algolia Crawler is a hosted and highly customizable web crawler that extracts and structures any content of a website to make it deliverable through a seamless experience.
Read guide
-
30 days to improve our Crawler performance by 50%
This article is about how we reworked the internals of our app crawler, looked for bottlenecks, and streamlined tasks to optimize the processing of this complex parallel & distributed software.
Read Article
