Charles Arthur On Technology

People who write operating systems have realised that we need to be able to search the millions of files on our PCs as easily as we search the web

As my desk is arranged along what is best described as the "deep litter" method (pile it all on top and weed it all out every couple of months; it's a technique from horse stabling, apparently). So if anyone rings up and asks "have you got that very important cutting I sent you?" I can answer in complete honesty: "It's right in front of me." Quite how long it would take to lay my hands on it is another matter. But we live in an on-demand world (as we constantly hear) so if there is a demand for a piece of knowledge that resides on a bit of paper that's near me, then it could be located. Given time, that is, and ample warning.

As my desk is arranged along what is best described as the "deep litter" method (pile it all on top and weed it all out every couple of months; it's a technique from horse stabling, apparently). So if anyone rings up and asks "have you got that very important cutting I sent you?" I can answer in complete honesty: "It's right in front of me." Quite how long it would take to lay my hands on it is another matter. But we live in an on-demand world (as we constantly hear) so if there is a demand for a piece of knowledge that resides on a bit of paper that's near me, then it could be located. Given time, that is, and ample warning.

I probably wouldn't, though. I'd be more likely to look it up on Google. My keyboard is closer than the bits of paper. And even if the information I want is in an e-mail, it's often faster to ask Google: many press releases are online, and there's more than half a gigabyte of e-mail sitting on my computer.

Yet it's strange that it's a lot easier to find something on the web than on my desk; and easier to find something on the web than on my computer. You would think one would store the things that are most important in the closest, most accessible locations; after all, you don't leave your wallet and car keys at the bottom of an unlit stairwell locked in a safe while keeping your entire wardrobe within arm's length.

Instead, Google, halfway around the world, is the default for a lot of the horsework, looking up phone numbers, checking facts, finding references.... But why do we let computers make our lives hard for us? Partly because we've let them remain stupid. The graphical user interface, with its metaphors of "files and folder and desktop", has remained unchanged since 1983 when Apple introduced it with the Lisa, the forerunner to its rather better-selling Macintosh. Computers then were slow and had tiny amounts of memory and disk space. You couldn't store much, so searching was a doddle. It'd either be that folder "Work" or "In".

But that paradigm - that you should be in charge of where your files go - remains today, despite the number of files on our systems having ballooned to tens of thousands. Even Windows XP, whose search facility is quick, and powerful, and will index the contents of documents if you don't mind it chewing up some of your processor from time to time, is still built around the notion that you should have some idea of where you put it.

Now, finally, the people who write operating systems have come to realise that we need to be able to search our millions of files (especially e-mail) with the same ease as we do three billion web pages. Actually, the makers of the Be operating system realised its importance long ago; the BeOS, released in 1990, had an in-built indexing system that built up "metadata" - data about data - on all its files on the fly. That made searching a cinch. While XP does build up metadata, it's not enough to make searching Google-like, where you don't care about location, just content.

That's all about to change. Microsoft is adding a very comprehensive local search facility to its Longhorn operating system, due in 2006. Or maybe 2007. (Well, it's due, anyway.) And last week Apple demonstrated a preview of "Tiger", its name for the fifth iteration of its Mac OSX operating system. Searching will be at the core of Tiger, using a technology dubbed "Spotlight": Steve Jobs, Apple's chief executive, demonstrated it by downloading a PDF map of western California and then searching his hard drive for the word "Yosemite" - which appeared in the map, as well as in many mail messages and other documents. Ken Bereskin, Apple's head of OS marketing, said that the company has been working on this metadata searching for more than a year, since before the release last year of the "Panther" upgrade to OSX. It's no secret that Apple hired some of Be's former staff (the company finally subsided in 2001 and was bought by Palm).

But Tiger isn't exactly scratching at the door; it won't be released until the "first half of 2005", which could be six or 12 months from now. So in the meantime, what can you do to improve your local search life?

The challenge tends to break down into two problems that stand side by side: searching e-mail, and searching everything else. An interesting stab at the e-mail problem is Zoe (www.zoe.nu), which is the closest thing you'll get at present to Gmail on your desktop, but for free. The nice thing here is that you don't necessarily have to give up using Outlook Express, because you can send your mail on to Zoe within your PC; it contains a POP (receiving) mail server as well as an SMTP (sending) server. It has been making hundredth-release steps every so often, and now stands at 0.5.9, and runs on pretty much any platform you care to name, including the latest versions of Windows, Linux and Mac OS. Though the supporting documents could be better, it's got the panoramic view of the e-mail problem that one needs.

But what about all the rest? You might be interested by the Haystack project, now turning out usable code, at MIT (http://haystack.lcs.mit.edu/index.html). This aims to do e-mail and everything else too, by letting you define "relationships" between data on your hard drive (such as someone's address and a web page). Its aim is to let people create "semantic web" pages - ones that will generate meaningful metadata for Google, so that searches won't turn up irrelevant results. But you may find that it's smarter at making sense of your hard drive than it is for the web. After all, nobody feels that searching the web is a problem anymore. And it was only when that was sorted out that we realised our hard drives had got into such a mess while we weren't looking.

network@independent.co.uk