Getting a share of Net worth

Stock market flotation is what Web data firms really want. By Charles Arthur

The second wave of Internet stock market flotations has begun. On Friday, Yahoo Inc came to the US stock market at a stock price that made it worth $340m. Two weeks earlier, Lycos and Excite had arrived, the former instantly worth $180m - and its value has risen by almost a third since then. That's not bad for companies that have never made a profit, and one of which - Lycos - was last summer in effect worth $1m. The first to arrive was OpenText, which came to the market in January.

But companies such as these, which provide Internet users with "search engines", are likely to be successful, if highly competitive. Everyone wants to be Netscape, which is presently worth around $4bn. And it is likely that search engines are the answer to a couple of questions that have been puzzling marketing executives for a while now - what do people really do with the Internet? And, more importantly, how can they, the marketing executives, make money from that?

It is emerging that people really use the Internet for two things - to communicate with other people and to search for information. You never know when you might want to know the answer to the daftest questions - how high is Mount Everest? Which is the next highest mountain? How big are rhinoceros beetles? How many episodes of the original Star Trek were made? In how many did William Shatner not appear?

You can be fairly sure that those questions are out there right now, somewhere on the Internet, either on the World Wide Web or in the newsgroups. But the Net can seem, to the newcomer and the expert, like an encyclopaedia without an index, with all the topics randomly jumbled up. How do you find what you want?

The answers to those questions can be found using some of the Web's search engines - powerful computers linked to enormous databases that record where such information might be held, both on the World Wide Web and the newsgroups. There are already a number of search engines, competing hard for your attention (see table).

How do search engines get their data? Simply, by getting their computers to do the work that humans could never be bothered to - visiting as many Web pages as they can. A piece of software known as a crawler, worm, spider or robot - but generically as an "agent" - starts from a known set of Web sites, and indexes their contents, just as you might if you were sitting at your screen with your browser on. The software then follows links to succeeding pages and indexes them, adding the features it wants to note to its home database. Humans can then come along and ask for the addresses of pages containing both "Everest" and "height": the database obliges.

Agents can index the text of millions of pages each day, though they run into problems with a number of features that have begun appearing on the Web. Sites that require you to fill in a registration form before your access is granted defeat them, as do pages that contain video or audio clips - there is presently no way to report what is in the clips. Similar problems arise with Java "applets", which are small programs that run on Web pages, and with VRML, the virtual reality modelling language that creates three-dimensional worlds on Web pages.

That hasn't stopped search engines being remarkably popular with advertisers, which have realised that if people keep coming to them, then ads are likely to be noticed. And people do come. Digital's Alta Vista engine, for example, only began operating its service on December 15, but within three weeks it was receiving two million search requests daily. It presently handles four million each day, as people search through its 33-gigabyte index, covering 11 billion words on 22 million Web pages and 13,000 newsgroups.

And what do you see when you first connect to Alta Vista's site? A subtle, text-based ad. It says: "Tip: To find good food: pizza "deep dish" +Chicago." Do any of those words remind you at all of a large American restaurant chain?

So they should. Companies are paying big money - at least $10,000 and $20,000 per month for a slot - to ease their way into your subconscious. That is how the search engine companies aim to recoup some of their costs. Another is by licensing - last week Lycos announced a deal with AT&T, offering its software searching techniques to users of AT&T's WorldNet service.

But the big prize for the search engine companies is flotation on the stock market. That is the only way to get the cash that will allow them to keep ahead of rivals in the technology race. Surfers are notoriously fickle about which engine they use: if they find a new one that turns up what they want more of the time, they'll quickly abandon their old favourite.

The race to Wall Street is hotting up now. After Yahoo, OpenText, Lycos and Excite - formed two years ago and based in Mountain View, California - another rival, Infoseek, plans to hit Wall Street in the next few weeks. Excite intends to sell two million shares at $17 each. But like every other search engine company, it has yet to make a profit: in 1995, it lost $5.34 million on sales of $2.6 million. But that isn't putting these companies off.

Whether it is entirely wise of US investors to be putting their money into these companies is a tricky question. Because tastes can change, there must be a substantial risk of losing everything, or at least never seeing a return on the shares. But technology has always been like that. History, as recorded by the search engines, tends to note the successes rather than the failures.

Here are a few places from which to search the Internet. The list is alphabetical. The order does not imply that one engine is better than another; the same enquiry may give different answers, because each engine indexes data differently.

AltaVista (Digital Equipment): http://www.altavista.digital.com

Excite: http://www.excite.com

Infoseek: http://www.infoseek.com

Inktomi: http://www.inktomi.com/

Lycos: http://www.lycos.com

Open Text: http://www.opentext.com/

WebCrawler: http://www.swebcrawler.com

WWW Worm: http://wwww.cs.colorado.edu/wwww.html

Yahoo: http://www.yahoo.com/

Netscape also maintains a page with a large number of links to these and a number of other search engines at:

http://home.netscape.com/home/internet-search.html