What is a Web Crawler?

My job is to crawl content on the web for Google's index everyday, every night, non stop. I am cute na? Have you ever thought how does the search engines like Google or Bing get so much of information on every possible topic or word you can think of? And from where? A search engine crawler (also known as spider, robot, or simply a bot) helps them do [...]