The challenges of crawling the web — and how to overcome them - Samuel Bodin, Algolia

Recorded during Algolia Search Party - Crawling edition - June 2019, Paris The challenges of crawling the web — and how to overcome them - Samuel Bodin, Algolia The world of crawling is dark and full of errors. I will talk about edge cases and present solutions we found at Algolia to overcome them.