If the Internet is like a huge mysterious lake full of information, then you could consider the search engines Google and Yahoo as skipping stones, bouncing across and highlighting useful tidbits of data that float to the surface.
Beneath that surface, however, lies a wealth of other knowledge that's harder to get to — stored in databases or hidden within cryptic government Web sites.
In that environment, Santa Fe's Deep Web Technologies functions as something of a big heavy stone. Toss it in the lake and it makes a satisfying plop, then descends into the inner realms of science, medicine and business, finding papers and data on topics that the more well-known search engines don't generally get to.
"Our technology is mostly these days called a 'federated search,' " said Abe Lederman, Deep Web's founder and president. "What it does is searches the deep Web through multiple-information sources, all at the same time and all in real time."
The company operates an assortment of Web-searching sites that would make most technical science types drool. They have similar names, like science.gov, worldwidescience.org and scitopia.org, but each draws on a different assortment of databases and information.
And last week Deep Web launched a new publicly available site that brings data from all of its science-related efforts together in a larger undertaking called scienceresearch.com.
"It searches science.gov, worldwidescience.org and a few of our other sites, in addition to maybe 100 or more databases we're not searching anywhere else," Lederman said. "It's a kind of catch-all."
That site builds on the company's previous efforts, which have gradually expanded and changed to incorporate larger bodies of data.
The first site Deep Web built, shortly after it was founded in 2002, was science.gov. That site, made with a partnership between the company and the Department of Energy's Office of Science and Technology Information, scans about 40 research and development databases from U.S. government agencies.
"Science.gov is probably our best-known site," Lederman said. "We like to say it's aimed at the science-attentive citizen."
The goal is to give the public and researches access to government science that might otherwise go unnoticed.
That site was good for looking up federal research in the United States, but Lederman and others also saw a need for a global site. So, in 2007, they launched worldwidescience.org, under a partnership between DOE's OSTI, the British Library and the International Counsel for Scientific and Technical Information.
"That site is like science.gov, but it searches databases from more than 50 countries around the world," Lederman said.
The site has an international perspective and gives researchers in Third World countries access to a huge range of information that would be hard for them to access otherwise, said Lorrie Johnson, who manages the worldwidescience.org site for DOE.
"One of the main benefits for users is that instead of searching 54 databases, they can just search one and get results from all of them, ranked in order of relevance," Johnson said. "For U.S. researchers, the software allows them to find out what other countries are doing. Researchers in other countries often have a different perspective than we do."
The site has grown from 10 databases representing 12 countries about a year ago into 54 databases representing 56 countries today. And DOE and Deep Web are looking at adding another five more databases in the next few months, Johnson said.
"We're covering about 80 percent of the world population through these databases and countries," Johnson said. "The one we really want to add is Russia, but we haven't found a good English interface yet."
Deep Web is working on an addition to its software package that will translate foreign language research into English as it searches, but so far it's just in the early beta stages, Lederman said.
Beyond the two DOE sites, Deep Web also worked with the Institute of Electrical and Electronics Engineers Inc., or IEEE, on a third big science site called scitopia.org, which looks through databases from the 20 or so major science and technical groups associated with IEEE.
The company's new catch-all site, scienceresearch.com, is like a lot of its other science sites, but it also marks a change for Deep Web.
The prior sites were all paid for by a sponsor organization. But scienceresearch.com is an independent effort by the company, which is still contemplating how to make money from it, Lederman said.
"We're the ones that are doing this one on our own," Lederman said. "We hope to perhaps monetize it with advertising, or perhaps by adding some databases where people can pay to access deeper information, and we get a cut of that. Right now though, it's the largest portal to freely searchable Deep Web science."
In the same vein, the company has also recently launched a medical and a business version of the science site, called mednar.com and biznar.com, respectively.
"Those are also publicly available," Lederman said. "Mednar searches more than 50 medical research sites and is aimed more at medical professionals, and Biznar searches more than 70 sites aimed at business research."
The company is also contemplating new sites for law and energy, among others, he said.
Overall Deep Web Technologies, which employs about 20 people in Santa Fe, has created somewhere between 25-30 sites for both public and private entities so far. One of its private customers is Intel Corp., which uses the software to scan through a host of informational databases the company subscribes to, Lederman said.
And if it continues to be successful — Deep Web made about $2 million in revenues last year — the company could be looking to expand and hire more Santa Fe workers later this year, Lederman said.
"We're working with lots of potential partners, and I'd say it's actually fairly promising that we will be growing in the third or fourth quarter of this year," Lederman said.
On the DOE side, Johnson said she's been very happy with the work Deep Web has done so far. The company has worked with the agency on several sites, some internal and some that are open to the public, she said.
"We have some internal ones that are related to environmental science or biological science, they can be very subject-specific," Johnson said. "Deep Web has been wonderful to work with. They're very innovative, very easy to work with."
Contact Sue Vorenberg at svorenberg@sfnewmexican.com.
You must register with a valid email address and use your real first-and-last name to comment on this forum. Once you've logged into the system, you'll be able to contribute comments. If you need help logging in or establishing your new user name and password, please write us.For information on our community guidelines and updating your username to meet standards, visit http://sfnm.co/sfnmforum.
All users are expected to abide by the forum rules and and be courteous to other users. Comments can be accepted up to eight days following publication. After that, comments can be read but no new submissions made. Send questions to webeditor@sfnewmexican.com
IMPORTANT: Comments must be posted under your own full, real name. Anonymous comments and those posted under a pseudonym can be removed. Please consult the forum rules. If you have questions, e-mail webeditor@sfnewmexican.com.