Apple confirms existence of “Applebot” web crawler, says program indexes the Web for Siri, Spotlight

Sometimes it’s the Web traffic logs that really let you know what Apple is up to.

Last fall, a number of webmasters began seeing web crawler visits from IP addresses starting with 17, an IP address block owned entirely by Apple. Apple has now officially confirmed in a support document that it is operating its own web crawler called “Applebot”.

As of now, Applebot is the web crawler utilized by Apple products such as Siri and Spotlight Suggestions. The program respects the customary robots.txt and robots meta tags used by most crawler programs.

What’s interesting is why Apple might create its own web crawler in the first place, especially when Siri tends to rely on Bing and Wolfram Alpha for its web searches.

One prominent question is if Apple is developing its own search engine with plans to take on outfits like Google, Bing and Yahoo? This seems unlikely, as Apple is a hardware company first and foremost.

Another theory is that the Applebot simply exists to supplement Siri and Spotlight’s search results and may not become its own dominant search engine.

If you have any thoughts on this, let us know in the comments.

Via 9to5Mac