Just in time or just in case?

Metasearch vs. harvesting & indexing

The other day I gave a presentation for the Assembly of members of the local Amsterdam Libraries Association “Adamnet“, about the Amsterdam Digital Library search portal that we host at the Library of the University of Amsterdam. This portal is built with our MetaLib metasearch tool and offers simultaneous access to, at the moment, 20 local library catalogues.

A large part of this presentation was dedicated to all possible (and very real) technical bottlenecks of this set-up, with the objective of improving coordination and communication between the remote system administrators at the participating libraries and the central portal administration. All MetaLib database connectors/configurations are “home-made”, and the portal highly depends on the availability of the remote cataloging systems.

I took the opportunity to explain to my audience also the “issues” inherent in the concept of metasearch (or “federated search“, “distributed search“, etc.), and compare that to the harvesting & indexing scenario.

Because it was not the first (nor last) time that I had to explain the peculiarities of metasearch, I decided to take the Metasearch vs. Harvesting & Indexing part of the presentation and extend it to a dedicated slideshow. You can see it here, and you are free to use it. Examples/screenshots are taken from our MetaLib Amsterdam Digital Library portal. But everything said applies to other metasearch tools as well, like Webfeat, Muse Global, 360-Search, etc.

The slideshow is meant to be an objective comparison of the two search concepts. I am not saying that Metasearch is bad, and H&I is good, that would be too easy. Some five years ago Metasearch was the best we had, it was a tremendous progress beyond searching numerous individual databases separately. Since then we have seen the emergence of harvesting & indexing tools, combined with “uniform discovery interfaces”, such as Aquabrowser, Primo, Encore, and the OpenSource tools VuFind, SUMMA, Meresco, to name a few.

Anyway,  we can compare the main difference between Metasearch and H&I to the concepts “Just in time” and “Just in case“, used in logistics and inventory management.

With Metasearch, records are fetched on request (Just in time), with the risk of running into logistics and delivery problems. With H&I, all available records are already there (Just in case), but maybe not the most recent ones.

Objectively of course, H&I can solve the problems inherent in Metasearch, and therefore is a superior solution. However, a number of institutions, mainly general academic libraries, will for some time depend on databases that can’t be harvested because of technical, legal or commercial reasons.

In other cases, H&I is the best option, for instance in the case of cooperating local or regional libraries, such as Adamnet, or dedicated academic or research libraries that only depend on a limited number of important databases and catalogs.

But I also believe that the real power of H&I can only be taken advantage of, if institutions cooperate and maintain shared central indexes, instead of building each their own redundant metadata stores. This already happens, for instance in Denmark, where the Royal Library uses Primo to access the national DADS database.

We also see commercial hosted H&I initiatives implemented as SaaS (Software as a Service) by both tool vendors and database suppliers, like Ex Libris’ PrimoCentral, SerialSolutions’ Summon and EBSCOhost Integrated Search.

The funny thing is, that if you want to take advantage of all these hosted harvested indexes, you are likely to end up with a hybrid kind of metasearch situation where you distribute searches to a number of remote H&I databases.

Social networking high and low of the year

Last month the Dutch Advisory Committee on Library Innovation published its report “Innovation with Effect“. The report was commissioned by the Dutch Minister of Education, Culture and Science, the charge was to draw up a plan for library innovation for the period 2009-2012 including a number of required conditions. Priorities that had to be addressed were: provision of digital services, collection policy, marketing, HRM.
The recommendations of the committee are classified in three main areas or “programmatic lines” under a more or less central direction and/or coordination:

• Digital infrastructure (such as: one common information architecture, connection to nationwide and global information infrastructure, one national identity management system)
• Innovation of digital services and products
• Policy innovation

Interesting report, but that is not what I want to point out here. What is very exciting: in the list of consulted sources, amidst official reports and publications, appears the social information professionals network Bibliotheek 2.0, the Dutch equivalent of http://library20.ning.com. This aroused much enthusiasm among the members of the Dutch library blogosphere.
The Committee’s chairperson Josje Calff, deputy director of Leiden University Library, had started a discussion on the topic “One public library catalogue?” in this community, to which I am proud to say I also made a small contribution. The results of this discussion have been used by the committee in formulating their recommendations.

In striking contrast to this success for web 2.0 social networking, there was a lot of outrage in the same Dutch library blogosphere last week about the ban of The Netherlands most popular social network Hyves and YouTube from one of the countries institutes for professional and adult education, reported on by one of its employees (in Dutch). Because of all the protests the school’s management is currently reconsidering their position and a new decision will be made beginning of 2009. Probably YouTube will continue to be permitted, because it is heavily used as a source of information in the lessons.