All Meresco Sources are now moved into one subversion repository at SourceForge.

SourceForge.net is the world's largest Open Source software development web site. One of the improvements is that the sources are now fully accessible for everybody, so including anonymous (non-registered-SourceForge) users.

Before doing this move we have also combined all the sources of the three main Meresco projects into one subversion repository.
Please take a look at the sources at: Meresco at SourceForge

New versions of the following documents have been uploaded:

• Component Library & Application DNA Interface Specification (Core 2.5)
• Meresco Core Public Interfaces

The MERESCO Conference is the place to be for anyone who is interested in MERESCO software suite. Users and developers will present their work and future plans. There will be enough time for mingling around and make plans to infuence the MERESCO roapmap.

The conference is free of charge. It begins at 12h30 and ends around 17h. For more information and registering, contact info@meresco.org.

Browsing, also known as Facetted Search or (Term-) Drilldown, is a fascinating alternative to 'Advanced Search' forms. While the latter allows users to enter (lots of) selection criteria only to find out that 'no matches' were found, a Drilldown or Browse interface tells a user in advance how many results to expect for each and every field.

Faceted Search has been pioneered in the Flamenco project.

However, todays implementations suffer one one or more of the following shortcomings:

1. the number of fields is limited, fixed, or both
2. the number of terms per field is limited
3. the total number of document is limited, often less than 1.000.000
4. the numbers returned are not exact but guesses or estimates
5. the number of concurrent users is limited

These shortcomings prevent the application of this concept to large data sets.

The research project CQ2 has started addresses the problems above. The goals are to research more specialized indexes and better search algoritms in order to design a very scalable Browse or Drilldown implementation that:

1. Flexibly allows browsing on any field, and on many fields
2. Supports unlimited terms per field (i.e. categories)
3. Scales up to 16.000.000 documents on a single node
4. Supports multiple tens of simultanous browse requests
5. Provides exact numbers under all circumstances

It is believed that such a Browse implementation would significantly improve hitrates and recall in networks such as Driver, Oaister and DAREnet.

OpenER-OAI is a self-feeding OAI-Repository created for the Open Universiteit Nederland. It can serve Dublin Core and LOM metadata records based on the RSS feed from EduCommons websites like OpenER.
The Open Source OpenER-OAI 1.0 package was built by Seek You Too and is released on SourceForge

OpenER-OAI is based on the following Meresco-Core Components:

• Self-Learning OAI-Repository
• RSS-Reader

Furthermore OpenER-OAI uses the following Open Source Seek You Too packages.

• Weightless 0.19, a High Performance Asynchronous Networking Library.
• Storage 5, metadata store, with automatic versioning, application level hashing, etc.

The first version of the MERESCO Core OAI-PMH components provides a complete OAI-PMH implementation, including support for sets and multiple metadataprefixes per record. Unfortunately, these components are somewhat elaborate to configure correctly.

To overcome the difficult configuration, CQ2 begun rewriting the OAI-PMH components into a set of components that feature automatic configuration. The end-goal is to have a plug-and-play OAI-PMH component that, when fed records using for example SRU-Record-Update, learns the configuration from these records.

When a record contains information about a set it belongs to, the MERESCO Core OAI-PMH component automatically remembers the set specification and uses this to respond correctly to the OAI verbs ListSets and ListRecords. This will even be the case with nested sets, as these are widely used in OAI data providers.

The same is done with metadataPrefixes. When a new type of record is added, the metadataprefix for this record is registered and used in subsequent OAI-PMH replies to the verbs ListMetadataSchemas and ListRecord.

CQ2 is looking for the sponsors to continue this project and publish the sources.

The RSS Reader is an MERESCO Core component, ready to plug and play with the other components such as OAI-PMH, Storage 5, Lucene etc. It periodically reaps RSS feeds and feeds the records into MERESCO Core for further processing.

The RSS Reader is used for turning the EduCommons' RSS feed into a OAI-PMH data provider, compatible with OAI-PMH, Dublin Core and IEEE LOM v1.0. This makes EduCommons based site pluggable with content services such as LOREnet and EduRep.

Development of the RSS Reader has triggered interesting new features for MERESCO Core. One of them is the upgrade to a completely new Asynchronous I/O Library called Weightless, another is the self-learning OAI repository. See the upcoming news items on these topics.

The document Meresco Core Public Interfaces is published. It describes the SRU, SRW, RSS, HTML, SRU Record Update interfaces as implemented by Meresco Core.

Meresco Architectural Description v0.4 is available. It has been updated with changes since May this years, and includes the latest additions to Meresco:

• the Crawler,
• OAI-PMH data provider
• RSS reader
• Term-Drilldown
• Field-Drilldown

Out-of-the-box Meresco offers well over 50 queries per second (almost 70) on commodity hardware. Today (070807) the SRU interface, running on a 2-CPU, 2-disk, 2-GB amd64 machine, spit out responses varying in size from 500 bytes to 200 kB in an amazing rate. Most responses included 10 copies of LOM metadata records gathered from disk.

During the test, only one CPU and one disk was utilized. At the same time, the Meresco Harvester was ingesting thousands of new records from various repositories.

Given the fact that the particular installation under test replaces a commercial search engine offering just about 50 queries per second on multiple 4-CPU, 8-GB Sun machines, it can be said that Meresco Core delivers real performance very efficiently.