deep web

Seems like you gotta run to keep up with webantics these days (and don’t we love it).  As much as it may reflect my lagging a little in this particular  race, the Deep Web is a new one me.  Here’s how those klever folk at Kosmix describe it:

The Deep Web is the portion of the Internet not accessible to traditional search engines.  Social networks, media-sharing sites for photos and videos, library catalogs, airline reservation systems, phone books, and all kinds of scientific databases lurk inside the Web, practically invisible today’s search tools.

The volume of this hidden content is enormous: some estimates have pegged the size of the Deep Web at up to 500 times larger than the slice of the Web we see on search engines today.

The commercial opportunities of getting access to this cornucopia are obvious. But the article also set me to thinking how this sort of capability might play in the local webspace game.

Just this week I found two new local webspaces in my neighbourhood -  an area that’s made up of 1 ward + and half + a bit.  Both are rich in local content in their own ways.  It’s a tantalisingly bountiful local webworld out there. What more could local webfolk do with if we could get access to the gems buried far inside the deep web?

On the flip side of course, the State will also have access. Imagine the power of combining efficient sentiment analysis with effective mining of the deep web. Of course there’s some real public service value creation to be had out of it all, but I have to admit that the thought also leaves me with a frisson of fear.