Amazon Interview Question

How to build a a scalable web crawling platform