Surf Wiki
Save to docs
general/defunct-internet-search-engines

From Surf Wiki (app.surf) — the open knowledge base

World Wide Web Wanderer

Web crawler

Summary

Web crawler

FieldValue
typeWeb search engine
launch_date
current_statusClosed

The World Wide Web Wanderer, also simply called The Wanderer, was a Perl-based web crawler that was first deployed in June 1993 to measure the size of the World Wide Web. The Wanderer was developed at the Massachusetts Institute of Technology by Matthew Gray, who also created back in 1993 one of the 100 first web servers in history, www.mit.edu. The crawler was used to generate an index called the Wandex later in 1993. The Wanderer charted the growth of the web until late 1995.

The Wanderer was probably the first web robot, and, with its index, clearly had the potential to become a general-purpose WWW search engine. The author, Matthew Gray, does not make this claim. Elsewhere, it is stated that the purpose of the Wanderer was not to be a web search engine.

References

References

  1. SEO, Doctor. (2022). "Evolution of SEO".
  2. [https://www.mit.edu/people/mkgray/index.3.html Matthew Gray's home page] - [https://www.mit.edu/~mkgray/net/background.html Pertinent page on Matthew Gray's section on MIT site]
  3. [https://dspace.mit.edu/bitstream/handle/1721.1/13277/25053550-MIT.pdf Brian LaMacchia's PhD thesis, section 1.2.3]
Wikipedia Source

This article was imported from Wikipedia and is available under the Creative Commons Attribution-ShareAlike 4.0 License. Content has been adapted to SurfDoc format. Original contributors can be found on the article history page.

defunct-internet-search-engines web-crawlers
Want to explore this topic further?

Ask Mako anything about World Wide Web Wanderer — get instant answers, deeper analysis, and related topics.

Research with Mako

Free with your Surf account

Content sourced from Wikipedia, available under CC BY-SA 4.0.

This content may have been generated or modified by AI. CloudSurf Software LLC is not responsible for the accuracy, completeness, or reliability of AI-generated content. Always verify important information from primary sources.

Report