January 22, 2005

The politics of search

As pretty much expected when politics come into play, DMOZ pulled the listing of RubberStamped. Apparently, not enough listings, although when we asked how many listings were required, we didn't get a reply. Que Sera Sera.

When I posted the "No Sleep 'Til Dmoz" thread, the objective wasn't really to get listed. It was to illustrate the slow turn-around time. Suffice to say, if Rich hadn't stepped in, my submission would have rotted along with the hundreds of others that no doubt sit in the queue. DMOZ should be turning around all submissions in far less than eight days. If the submissions are too numerous to deal with, then surely policy and procedures need to change to address that problem? If policies cannot be applied consistently, then what use are they?

The judgement of "quality" or "completedness" of this directory is a red herring. The real issue is that there are many directories on the web that do qualify to be listed but aren't because the queue has been neglected. And because the category isn't fresh, it isn't particularly useful. Will we see the category carefully maintained and kept up to date from now on? I doubt it. It will likely be left dormant like so many others. DMOZ has long since lost sight of it's original vision.

I don't begrudge DMOZ as a whole. They're overwhelmed with submissions and subject to the political problems that come with "free" and volunteerism on that scale. What I don't see is any will or effective strategy to solve those problems.

Those are the real issues that need addressing :)

January 21, 2005

DMOZ come through. Sort of.

Here's how not to do marketing.

There are two main features to the Rubberstamped directory: turn-around time and fresh content. To illustrate this, we decided to show the problem with DMOZ i.e. slow turn around time and lack of fresh content. We submitted to DMOZ, then planned to count the days to acceptance. Of course, this count would go into the hundreds before acceptance. It was highly likely we would never be accepted.

We were wrong.

What happened was Mr DMOZ founder himself, Rich Skrenta, got in touch, personally, and mentioned that we had been added to DMOZ. Days to DMOZ: Erm...eight. And a personal note from the founder.


How embarrassing. The least we could do was approve Mr Skrenta's listing request in RubberStamped :) Rich has also been so good as to agree to an interview, and talk about what he's up to these days. We'll be publishing this on as soon as we're done chatting. That may take some time as Rich is a very interesting chap.

So, DMOZ:1 RubberStamped: 0

Own goal.

January 18, 2005

Mining the deep web

It's interesting trying to find that data beyond the reach of the search engines, or the great sites that, for one reason or another, have been buried in the search results.

Here's a few tools I've been using to look way beneath the surface:

Specialized vertical search engines
Deep Query Manager
Gary Price's compilation of links to deep search interfaces

More to come. Started a deep search category.