ALIWEB

ALIWEB (Archie-Like Indexing for the Web) is the first Web search engine.

First announced in November 1993 by developer Martijn Koster while working at Nexor, and presented in May 1994 at the First International Conference on the World Wide Web at CERN in Geneva, ALIWEB preceded WebCrawler by several months.

ALIWEB allows users to submit the locations of index files on their sites which enables the search engine to include webpages and add user-written page descriptions and keywords. This empowers webmasters to define the terms that would lead users to their pages, and also avoided setting bots (e.g. the Wanderer, JumpStation) which used up bandwidth.

Martijn Koster, who was also instrumental in the creation of the Robots Exclusion Standard, detailed the background and objectives of ALIWEB with an overview of its functions and framework in the paper he presented at CERN. Koster left Nexor and in the following years development ceased until a new company took over the project. It was discovered that the database file had actually exceeded the range of search so therefore when you entered a search term it would not search the entire database and weight the results, it would only search from the beginning of the database until it either ran out of results to show or exceeded the number of results requested. The new company developers fixed this by weighting the results and searching the entire database before displaying the results.