4.16 Extending the repository

This section contains information for people who want to plug in custom Java-based components in the repository server.

4.16.1 Repository plugins

Daisy provides a number of interfaces through which the repository functionality can be extended or customized. The components that do this are called plugins.

4.16.1.1 Anatomy of a plugin

A plugin is basically an implementation of a certain plugin interface. The various plugin interfaces are listed further on.

To deploy the plugin in the repository server, it should be packaged as a container jar for the Daisy Runtime. A container jar is a jar file containing a Spring container definition.

The Spring container definition should contain a bean which (usually upon initialization) registers the plugin implementation with the plugin registry. The bean which performs the registration can simply be the plugin implementation itself.

To gain access to the plugin registry, the Spring container can use the daisy:import-service instruction (a custom tag provided by the Daisy Runtime).

When the Spring container of the plugin is destroyed, it should properly unregister the plugin.

To deploy the plugin, the container jar can be copied to a specific subdirectory of the repository data directory.

This may sound like a lot, but it's not, as is illustrated by this example.

4.16.1.2 Plugin types

4.16.1.2.1 Extensions

Repository extensions are very generic plugins which can implement any sort of functionality. The main advantage of putting this functionality in the form of a repository extension is that the extension can then be easily retrieved using the Repository.getExtension(name) function.

Various non-core functionality in Daisy has been added as extensions, such as the NavigationManager, the Publisher, the EmailNotifier, etc.

For details see repository extensions.

4.16.1.2.2 Pre-save hooks

A pre-save hook is a component which can modify the content of a document right before it is saved. An example is the image pre-save hook included with Daisy, which can generate thumbnails in document parts and extract EXIF data to document fields.

A pre-save hook should implement the following interface:

org.outerj.daisy.repository.spi.local.PreSaveHook

4.16.1.2.3 Authentication schemes

The task of an authentication scheme is to tell if a username/password combination is valid. By default Daisy uses its own authentication based on Daisy's user management. New authentication schemes can be added to check against external systems. Daisy ships with example authentication schemes for LDAP and NTLM which are usable through simple configuration. If you have other needs, you can implement your own scheme. See also user management and an example.

An authentication scheme should implement the following interface:

org.outerj.daisy.authentication.spi.AuthenticationScheme

4.16.1.2.4 Text extractors

Text extractors extract text from various content formats for the purpose of full text indexing. Daisy includes a variety of such text extractors, e.g. for MS Word and PDF.

A text extractor should implement the following interface:

org.outerj.daisy.textextraction.TextExtractor

4.16.1.2.5 Link extractors

Link extractors extract Daisy document links from various content formats. These extracted links are maintained by the repository to enable searching which documents link to a certain document.

A link extractor should implement the following interface:

org.outerj.daisy.linkextraction.LinkExtractor

4.16.1.2.6 HTTP handlers

Adding new functionality to the HTTP interface of the repository server can be done by implementing a request handler:

org.outerj.daisy.httpconnector.spi.RequestHandler

This is mostly useful for extensions which want to support remote invocation.

4.16.1.2.7 Other

It's possible to add it any sort of component to be launched as part of the repository server, it doesn't necessarily need to register something in the plugin registry. Such components could perform all sorts of tasks such as listening to JMS or synchronous repository events, performing timed actions, etc.

4.16.1.3 Plugin registry

To make plugins available, you need to register them with a service called the PluginRegistry.

When registering a plugin, you need to specify the following:

• The plugin type interface, specified as a Java Class object
• A name for the plugin, which should be unique within a particular type of plugins
• The actual plugin instance (an object implementing the plugin type interface)

For example, a text extractor is registered like this:

import org.outerj.daisy.textextraction.TextExtractor;
...

TextExtractor extractor = new MyTextExtractor();

pluginRegistry.addPlugin(TextExtractor.class, "my text extractor", extractor);

To gain access to the PluginRegistry, use daisy:import-service to import it into your spring container:

<beans ...

  <daisy:import-service id="pluginRegistry" service="org.outerj.daisy.plugin.PluginRegistry"/>

For a complete example, see the authentication scheme sample.

It is recommended to unregister the plugin when shutting down.

That's all you need to know about registering plugins. It is the responsibility of the PluginRegistry and the user of the plugins to handle the rest.

4.16.1.4 Deploying plugins

As explained earlier, a plugin should be packaged as a container jar.

The Daisy Runtime configuration for the repository server will automatically include container jars put in the following directories:

<daisy data dir>/plugins/load-before-repository

<daisy data dir>/plugins/load-after-repository

To see how these are included in the runtime configuration, have a look at <DAISY_HOME>/repository-server/conf/runtime-config.xml

As explained in the Daisy Runtime documentation, the container jars put in these directories will be loaded in alphabetical order.

The reason to have two directories, load-before-repository and load-after-repository, is as follows:

For some plugins, it is desirable to register them before the core repository is started because they modify the behavior of the repository. For example, take text extractors (which extract text for the purpose of fulltext indexing). From the moment the repository is started, it can start doing work. For example it might receive a JMS event of an updated document and fulltext-index it. If we would only register text extractor plugins after the repository is started, there will be a (small) amount of time during which the repository server will try to index documents without these additional text extractors being available. Hence documents handled during this period would be treated differently.

Other plugins might be depended on the repository being available (in Daisy Runtime speak: they import services exported by the repository container) , and hence can only be loaded after the repository is available.

In general, plugins which modify the behaviour of the repository server (text extractors, pre-save hooks, authentication schemes, etc.) should be put in the load-before-repository directory.

After copying the new plugin(s) into the appropriate directory, you need to restart the repository server for the plugin to be loaded.

4.16.1.5 Repository Extensions

Repository Extensions are a particular type of plugins that add extra functionality to the repository. An Extension is usually related to the repository, e.g. because it needs access to information in the repository. The extension code has no special privileges, it simply makes use of the repository using the normal APIs and SPIs.

In fact, since an extension is simply an application which makes use of the repository via its API, one could wonder why there is a need for adding this code as an extension to the repository. The reasons are:

• the extension can be retrieved by API users using the getExtension() method shown below, both for the local and remote API implementations (if the extension provides a remote implementation). So API users have a common way to get access to extension functionality.

• the extension automatically has access to the repository object from which it is retrieved. Otherwise one would have to supply the repository object as an argument to extension functions: doSomething(repository, other arguments)

Extensions registered with the repository can be retrieved like this:

MyExtension myExtension = (MyExtension)repository.getExtension("name-of-the-extension");

whereby MyExtension is the interface of the extension.

Examples of extensions delivered with Daisy: the NavigationManager, the EmailSubscriptionManager, the Emailer, the Publisher and the DocumentTaskManager.

To register in an extension you should register an implementation of this interface:

org.outerj.daisy.repository.spi.ExtensionProvider

and add it to the plugin registry.

4.16.1.6 Sample: custom authentication scheme

As an example of how to build a repository plugin, here we will look at how to implement an authentication scheme. Other plugins follow a similar approach.

For the purpose of this example, we'll create a FixedPasswordAuthenticationScheme, i.e. an authentication scheme which accepts just one fixed password regardless of the user.

Components to be deployed in the repository server need to be packaged as a container jar, this is a normal jar file containing at least one Spring bean container definition. This is described in detail in the Daisy Runtime documentation.

4.16.1.6.1 The container jar

The container jar we're going to build will have the following structure:

org
  foobar
    FixedPasswordAuthenticationScheme.class
DAISY-INF
  spring
    applicationContext.xml

So we only need to create two files.

4.16.1.6.2 Implement the necessary code

For a custom authentication scheme, we need to do two things:

• make an implementation of the interface AuthenticationScheme

• register this implementation with the PluginRegistry

We do this here with one class, shown below. Save this code in a file called FixedPasswordAuthenticationScheme.java

package org.foobar;

import org.outerj.daisy.authentication.AuthenticationScheme;
import org.outerj.daisy.authentication.AuthenticationException;
import org.outerj.daisy.plugin.PluginRegistry;
import org.outerj.daisy.repository.Credentials;
import org.outerj.daisy.repository.user.User;
import org.outerj.daisy.repository.user.UserManager;
import org.apache.avalon.framework.configuration.Configuration;

public class FixedPasswordAuthenticationScheme implements AuthenticationScheme {
    private String name;
    private String description;
    private String password;
    private PluginRegistry pluginRegistry;

    public FixedPasswordAuthenticationScheme(Configuration config,
            PluginRegistry pluginRegistry) throws Exception {
        password = config.getChild("password").getValue();
        name = config.getChild("name").getValue();
        description = config.getChild("description").getValue();
        this.pluginRegistry = pluginRegistry;
        pluginRegistry.addPlugin(AuthenticationScheme.class, name, this);
        System.out.println("Scheme " + name + " added!");
    }

    public void destroy() {
        pluginRegistry.removePlugin(AuthenticationScheme.class, name, this);
    }

    public String getName() {
        return name;
    }

    public String getDescription() {
        return description;
    }

    public boolean check(Credentials credentials) throws AuthenticationException {
        // this is the actual password check, very simple in this case
        return password.equals(credentials.getPassword());
    }

    public void clearCaches() {
        // we have nothing cached
    }

    public User createUser(Credentials crendentials, UserManager userManager) throws AuthenticationException {
        // unsupported
        return null;
    }
}

This file can be compiled like this:

Use the method of your choice to compile the code (Ant, Maven, your IDE, ...). When using the command below, we hope you know enough about this that everything should be on one line. For Windows, replace $DAISY_HOME with %DAISY_HOME% and the colons with semicolons)

javac -classpath
          $DAISY_HOME/lib/daisy/jars/daisy-repository-api-<version>.jar:
          $DAISY_HOME/lib/daisy/jars/daisy-repository-server-spi-<version>.jar:
          $DAISY_HOME/lib/daisy/jars/daisy-pluginregistry-api-<version>jar:
          $DAISY_HOME/lib/avalon-framework/jars/avalon-framework-api-4.1.5.jar
      FixedPasswordAuthenticationScheme.java

4.16.1.6.3 Create a Spring bean container definition (applicationContext.xml)

The following is the Spring container definition.

<?xml version="1.0"?>
<beans              xmlns = "http://www.springframework.org/schema/beans"
                xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance"
              xmlns:daisy = "http://outerx.org/daisy/1.0#runtime-springext"
               xmlns:conf = "http://outerx.org/daisy/1.0#config-springext"
       xsi:schemaLocation = "http://www.springframework.org/schema/beans
                             http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
                             http://outerx.org/daisy/1.0#runtime-springext
                             http://daisycms.org/schemas/daisyruntime-springext.xsd
                             http://outerx.org/daisy/1.0#config-springext
                             http://daisycms.org/schemas/config-springext.xsd">

  <daisy:import-service id="configurationManager"
             service="org.outerj.daisy.configuration.ConfigurationManager"/>
  <daisy:import-service id="pluginRegistry"
             service="org.outerj.daisy.plugin.PluginRegistry"/>

  <bean id="foo" class="org.foobar.FixedPasswordAuthenticationScheme" destroy-method="destroy">
    <constructor-arg>
      <conf:configuration group="foobar" name="fixedpwd-auth" source="configurationManager">
        <conf:default xmlns="">
          <name>fixedpwd</name>
          <description>Fixed global password</description>
          <password>jamesbond</password>
        </conf:default>
      </conf:configuration>
    </constructor-arg>
    <constructor-arg ref="pluginRegistry"/>
  </bean>

</beans>

The special <daisy:import> elements are used to import services from other containers. You can assume these services will be available. See the Daisy Runtime for details on how this works.

The component configuration system is also explained elsewhere, for now it suffices to know that the content of the conf:default element will be supplied as default configuration to the component. This configuration could be customized through the myconfig.xml file.

4.16.1.6.4 Deploy the new authentication scheme

Create the jar file (which is a normal zip file) containing the FixedPasswordAuthenticationScheme.class and applicationContext.xml in the directory structure as outlined earlier.

$ find -type f
./org/foobar/FixedPasswordAuthenticationScheme.class
./DAISY-INF/spring/applicationContext.xml

$ jar cvf fixedpwd-auth.jar *

Then copy the jar file to the directory <repo data dir>/plugins/load-before-repository.

Now stop and start the repository server. If everything goes well, the authentication scheme will be loaded and the following line will be printed (to standard out if you're using the wrapper, the wrapper log file)

Scheme fixedpwd added!

If you go to the Administration console in the Daisy Wiki and create or edit a user, you will be able to select the new authentication scheme.

4.16.1.6.5 Follow-up notes

When doing this for real, you would of course use proper logging instead of System.out.println.

For those familiar with Spring, instead of constructor dependency injection, we could as well have used setter dependency injection.

If the implementation of the authentication scheme requires some code on the classpath, than this can be specified by adding a DAISY-INF/classloader.xml file to the container jar. See the Daisy Runtime documentation.

Happy hacking!