Demonstration

This demonstration will introduce the user to the following:

• What aDORe Federation / aDORe Archive configurations exist? What does each define?
• How do I create a new processing profiles?
• What is a processing profile?
• How does the aDORe Federation ingest content?
• How do I retrieve content after ingestion?

aDORe Federation and aDORe Archive ingestion configurations are stored in a properties file similar to $install_dir/adore-federation/etc/adore.properties.

Key configurables include:

* adoreArchive.Organization
- Defines the local OpenURL SID
- (e.g. adoreArchive.Organization=info:sid/library.lanl.gov)

* adoreArchive.tmpDir
- Defines the path to a tmp processing directory
- (e.g. adoreArchive.tmpDir=tmp)

* adoreArchive.threads
- Defines the number of threads used during ingestion
- (e.g. adoreArchive.threads=1)

* adore-arcfile.ResolverURL
- Defines the BaseURL path to the ARCfile Resolver
- (e.g. adore-arcfile.ResolverURL=http://localhost:8080/adore-arcfile-resolver/resolver)

* adore-arcfile.StoreDirectory
- Defines the ARCfile Storage Directory Path 
- (e.g. adore-arcfile.StoreDirectory=$install_dir/adore-archive/data/arc)

* adore-arcfile.IndexDirectory
- Defines the ARCfile Index Directory Path
- (e.g. adore-arcfile.IndexDirectory=$install_dir/adore-archive/data/tape)

* adore-arcfile.PrefixURL
- Defines the Local ARCfile Prefix
- (e.g. adore-arcfile.PrefixURL=info:lanl-repo/arc/)

* adore-arcfile.DataStreamPrefix
- Defines the Local Datastream Prefix
- (e.g. adore-arcfile.DataStreamPrefix=info:lanl-repo/ds/)

* adore-arcfile-registry.OAIURL
- Defines the BaseURL path to the OAI-PMH ARCfile Registry OAI-PMH Service Interface
- (e.g. adore-arcfile-registry.OAIURL=http://localhost:8080/adore-arcfile-registry/OAIHandler)

* adore-arcfile-registry.PutRecordURL
- Defines the BaseURL path to the OAI-PMH Arc Registry PutRecord Service Interface
- (e.g. adore-arcfile-registry.PutRecordURL=http://localhost:8080/adore-arcfile-registry/PutRecordHandler)

* adore-xmltape.PrefixURL
- Defines the Local XMLtape Prefix
- (e.g. adore-xmltape.PrefixURL=adore-xmltape.PrefixURL=info:lanl-repo/xmltape/)

* adore-xmltape.AccessorURL
- Defines the basURL for the multi-tape OAI-PMH Service Interface
- (e.g. adore-xmltape.AccessorURL=http://localhost:8080/adore-archive-accessor/Handler)

* adore-xmltape.StoreDirectory
- Defines the XMLtape Storage Directory Path
- (e.g. adore-xmltape.StoreDirectory=$install_dir/adore-archive/data/tape)

* adore-xmltape.IndexDirectory
- Defines the XMLtape Index Directory Path
- (e.g. adore-xmltape.IndexDirectory=$install_dir/adore-archive/data/tape)

* adore-xmltape-index.Plugin
- Defines the aDORe Archive XMLtape Indexing Plug-in class name
- (e.g. adore-xmltape-index.Plugin=gov.lanl.xmltape.index.BasicTapeIndex)

* adore-xmltape-index.IdIdxPlugin
- Defines the aDORe Federation XMLtape identifier indexing implementation
- (e.g. adore-xmltape-index.IdIdxPlugin=gov.lanl.xmltape.identifier.index.jdbImpl.IdentifierIndex)

* adore-xmltape-registry.OAIURL
- Defines the BaseURL path to the OAI-PMH XMLtape Registry OAI-PMH Service Interface
- (e.g. adore-xmltape-registry.OAIURL=http://localhost:8080/adore-xmltape-registry/OAIHandler)

* adore-xmltape-registry.PutRecordURL
- Defines the BaseURL path to the XMLtape Registry PutRecord Service Interface
- (e.g. adore-xmltape-registry.PutRecordURL=http://localhost:8080/adore-xmltape-registry/PutRecordHandler)

* adore-xmltape.ResolverURL
- Defines the aDORe Archive XMLTape OpenURL Resolver Service BaseURL
- (e.g. adore-xmltape.ResolverURL=http://localhost:8080/adore-xmltape-resolver)

* adore-xmltape-xquery.ResolverURL
- Defines the aDORe Archive XMLTape XQuery OpenURL Resolver Service BaseURL
- (e.g. adore-xmltape-xquery.ResolverURL=http://localhost:8080/adore-xmltape-xquery)

* adore-id-locator.ResolverURL
- Defines the aDORe Federation's Identifier Locator OpenURL Resolver Service BaseURL
- (e.g. adore-id-locator.ResolverURL=http://localhost:8080/adore-id-locator/resolver)

* adore-id-locator.DBProps
- Defines the absolute path to the idlocator.properties file, which contains following <a href="../proj/IdentifierLocator.xml">configurations</a>.
- (e.g. adore-id-locator.DBProps=$install_dir/adore-id-locator/etc/idlocator.properties)

* adore-service-registry.RegistryRecordHandlerURL
- Defines the BaseURL path to the Service Registry PutRecord Service Interface
- (e.g. adore-service-registry.RegistryRecordHandlerURL=http://localhost:8080/adore-service-registry/RegistryRecordHandler)

* adore-service-registry.ResolverURL
- Defines the aDORe Service Registry OpenURL Resolver Service BaseURL
- (e.g. adore-service-registry.ResolverURL=http://localhost:8080/adore-service-registry/resolver)

* adore-service-registry.OAIURL
- Defines the BaseURL path to the OAI-PMH XMLtape Registry OAI-PMH Service Interface
- (e.g. adore-service-registry.OAIURL=http://localhost:8080/adore-service-registry/OAIHandler)

First, lets define a processing profile. A Processing Profile is a configuration grouping which define how source content should be pre-processed to create XMLTapes and ARCfiles for ingestion into the aDORe Archive, and then into the aDORe Federation. The aDORe Archive provides a robust plug-in architecture which allows developers to pre-process content prior to indexing and registration. Our processing profile, which we'll name 'test' will simply copy the source XMLtape/ARCfiles to their respective archive storage directories. For each processing profile, a new profile reference will need to be created in the archive.properties file.

The configuration grouping includes:

* <profilePrefix>.FullName
- Define the full name of Collection Profile
- (e.g. test.FullName=info:sid/library.lanl.gov/sample)

* <profilePrefix>.pmh-defaultproperty
- Define the Collection PMH Default Properties filepath
- (e.g. test.pmh-defaultproperty=$install_dir/adore-archive/etc/moai/test.properties)

* <profilePrefix>.IndexSetSpecProps
- Define the Collection SetSpec XPath Properties filepath
- (e.g. test.IndexSetSpecProps =$install_dir/adore-archive/etc/XPath.properties)

* <profilePrefix>.ProcessorClass
- The pre-processing implementation for processing XMLtape &amp; ARCfiles
- (e.g. test.ProcessorClass=gov.lanl.archive.impl.StandardProcessor)

* <profilePrefix>.identifierIdxRecordPlugin
- The identifier indexing implementation specific to the complex object format (e.g. DIDL, METS, etc.)
- (e.g. test.identifierIdxRecordPlugin=gov.lanl.xmltape.identifier.index.record.didl.DidlRecordDOM)

### Optional Profile Configurations ###

* <profilePrefix>.iesr.subjects
- Add list of unique subjects to Collection-level Service Registry Record (use XQuery Resolver)
- (e.g. test.iesr.subjects=false)

* <profilePrefix>.iesr.itemTypes
- Add list of unique itemTypes to Collection-level Service Registry Record (use XQuery Resolver)
- (e.g. test.iesr.itemTypes=false)

* <profilePrefix>.iesr.itemFormats
- Add list of unique itemFormats (i.e. mimetypes) to Collection-level Service Registry Record (use XQuery Resolver)
- (e.g. test.iesr.itemFormats=false)

* <profilePrefix>.TapeStoreDirectory
- Overrides default storage location for XMLtape Files
- (i.e. test.TapeStoreDirectory=/d1/adore/data/test/tape)

* <profilePrefix>.TapeIndexDirectory
- Overrides default storage location for XMLtape Index Files
- (i.e. test.TapeIndexDirectory=/d1/adore/data/test/idx)

* <profilePrefix>.ArcStoreDirectory
- Overrides default storage location for ARCfiles
- (i.e. test.TapeIndexDirectory=/d1/adore/data/arc)

Before we perform a first content ingestion, let's ensure our Tomcat and MySQL services are running.
To verify your Tomcat instance is running, open http://localhost:8080/. The Tomcat Servlet Engine should appear.
To verify your MySQL instance is running, log in using the arc_registry user using the following command: mysql -u arc_registry -p . Enter the appropriate db password when prompted to do so. MySQL should welcome you and provide you with a mysql query prompt. Type 'exit' to return to your shell prompt.

For this demonstration, we'll ingest a set of pre-constructed XMLTape and ARCFiles. The contents of the XMLTape and ARCFiles were derived from the Library of Congress' American Memory Project. Refer to the aDORe Federation Tutorial for additional information on how to create XMLTapes and ARCfile from existing resources.

The demonstration will perform the following functions:

• Pass file-path refs for configuration file, XMLTape and ARCFile to ingestion application along with the processing profile id.
• Copy the source XMLtape/ARCfiles to their respective archive storage directories.
• Index XMLTape using the specified adore-xmltape-index.Plugin implementation.
• Index ARCfile to create a CDX file.
• Register XMLTape using the defined adore-xmltape-registry.PutRecordURL interface.
• Register ARCfile using the defined adore-arcfile-registry.PutRecordURL interface.

Run the following command:
sh ./demo.sh <install_dir>  (e.g. ./demo.sh /usr/local/adore)

To verify that our new XMLTape was properly registered, let's make an OAI-PMH ListRecords request to the XMLTape Registry:

http://localhost:8080/adore-xmltape-registry/OAIHandler?verb=ListRecords&metadataPrefix=tape

To verify that our new ARCfile was properly registered, let's make an OAI-PMH ListRecords request to the ARCfile Registry:

http://localhost:8080/adore-arcfile-registry/OAIHandler?verb=ListRecords&metadataPrefix=arc

To retrieve the contents of our XMLTape, let's make an OAI-PMH ListRecords request to the Archive Accessor:

http://localhost:8080/adore-archive-accessor/Handler/demoTape?verb=ListRecords&metadataPrefix=native

To retrieve the contents of our ARCfile, let's make an OpenURL request to the ARCfile Resolver:

http://localhost:8080/adore-arcfile-resolver/472de88d-5670-45df-9200-fb135189f00e/openurl-aDORe4?url_ver=Z39.88-2004&rft_id=info%3Alanl-repo%2Fds%2Fa9866bc2-306b-4698-abaf-a0f43c1ea611&svc_id=info:lanl-repo/svc/getDatastream

To retrieve a list of identifiers contained in our ARCfile, we'll make a OpenURL Request for svc_id "info:lanl-repo/svc/identifiers.list":

http://localhost:8080/adore-arcfile-resolver/472de88d-5670-45df-9200-fb135189f00e/openurl-aDORe3?url_ver=Z39.88-2004&rft_id=info%3Alanl-repo%2Farc%2F472de88d-5670-45df-9200-fb135189f00e&svc_id=info:lanl-repo/svc/identifiers.list

To retrieve a list of identifiers contained in our XMLTape, we'll make a OpenURL Request for svc_id "info:lanl-repo/svc/identifiers.list":

http://localhost:8080/adore-xmltape-resolver/demoTape/openurl-aDORe3?rft_id=info%3Alanl-repo%2Fxmltape%2FdemoTape&svc_id=info%3Alanl-repo%2Fsvc%2Fidentifiers.list&url_ver=Z39.88-2004

To obtain a DIDL for a given rft_id using the OpenURL Resolver, we'll make a OpenURL Request for svc_id "info:lanl-repo/svc/getDIDL":

http://localhost:8080/adore-xmltape-resolver/demoTape/openurl-aDORe1?url_ver=Z39.88-2004&rft_id=info%3Adoi%2F10.1016%2Fj.dyepig.2004.06.022&svc_id=info:lanl-repo/svc/getDIDL

To retrieve a list of available locations for a given rft_id using the OpenURL Resolver, we'll make a OpenURL Request for svc_id "info:lanl-repo/svc/locate.sru":

http://localhost:8080/adore-xmltape-resolver/demoTape/openurl-aDORe2?url_ver=Z39.88-2004&rft_id=info%3Adoi%2F10.1016%2Fj.dyepig.2004.06.022&svc_id=info:lanl-repo/svc/locate.sru

The adore-xmltape-xquery OpenURL interface may be used to perform ad hoc of XML repository content:

http://localhost:8080/adore-xmltape-xquery/demoTape/openurl-aDORe7?url_ver=Z39.88-2004&rft_id=info%3Alanl-repo%2Fxmltape%2FdemoTape&svc_id=info:lanl-repo/svc/mimetypes.list

To Obtain the Collection-level Record for repository from the Service Registry:

http://localhost:8080/adore-service-registry/resolver?url_ver=Z39.88-2004&rft_id=info:lanl-repo/xmltape/demoTape&svc_id=info:lanl-repo/svc/ockham

To Obtain a Service-level Record for given service identifier from the Service Registry:

http://localhost:8080/adore-service-registry/resolver?url_ver=Z39.88-2004&rft_id=info:lanl-repo/int/demoTape/openurl-aDORe1&svc_id=info:lanl-repo/svc/ockham

To retrieve a list of all Service Registry Records:

http://localhost:8080/adore-service-registry/OAIHandler?verb=ListRecords&metadataPrefix=oai_iesr

To retrieve a list of Collection-level Service Registry Records:

http://localhost:8080/adore-service-registry/OAIHandler?verb=ListRecords&metadataPrefix=oai_iesr&set=collections

To retrieve a list of Service-level Service Registry Records:

http://localhost:8080/adore-service-registry/OAIHandler?verb=ListRecords&metadataPrefix=oai_iesr&set=services

To query the Identifier Locator for the location of a given identifier:

http://localhost:8080/adore-id-locator/resolver?url_ver=Z39.88-2004&rft_id=info%3Adoi%2F10.1016%2Fj.dyepig.2004.06.022&svc_id=info:lanl-repo/svc/locate.sru

The adore-federator may be used to access multiple aDORe Archive installation through a common OAI-PMH interface:

http://localhost:8080/adore-federator/OAIHandler?verb=ListRecords&metadataPrefix=didl

Finally, the adore-disseminator OpenURL Service interface can access all repository content, as well as perform transformation and dissemation services upon obtained content:

http://localhost:8080/adore-disseminator/service?rft_id=info%3Adoi/10.1016/j.dyepig.2004.06.022&url_ver=Z39.88-2004&rft_val_fmt=http%3A//african.lanl.gov/aDORe/schemas/2007-01/version.xhtml&svc_val_fmt=http%3A//african.lanl.gov/aDORe/schemas/2007-01/chunklist.xhtml&svc_id=info%3Alanl-repo/svc/getDIDL