aDORe djatoka - Overview

djatoka is a Java-based open source image server with an attractive basic feature set and extensibility under control of the community of implementers. Off-the-shelf, djatoka provides compression and region extraction of JPEG 2000 images, URI-addressability of regions, and support for a rich set of input/output image formats (e.g., BMP, GIF, JPG, PNG, PNM, TIF, JPEG 2000). djatoka also comes with a plug-in framework that allows transformations to be applied to regions and resolutions (e.g., watermarking).

aDORe djatoka was derived from a concrete need to introduce a solution to disseminate high-resolution images stored in our aDORe repository. We set out to explore the problem domain and ended up developing the aDORe djatoka dynamic image dissemination solution, an open source package with an attractive feature set capable of disseminating images that reside either in a djatoka environment or that are Web-accessible at arbitrary URIs.

There are hardly any challenges involved in disseminating sizeable images on the Web. Just save the image in a well-supported format on your Web server, give it a URI, and you’re done. But, when the image size reaches the realm of hundreds of megabytes or gigabytes, as is the case with many high-resolution images that result from digitization, dissemination becomes far less trivial. One part of a common solution to the problem is to create a derivative image in a selected service format. This is typically a multi-resolution image file format (e.g., MrSID, JPEG 2000, Pyramid TIFF) that encapsulates a set of resolutions and is derived from the master image, which in many cases is a TIFF file that itself is never used for dissemination. As a benefit, these multi-resolution image formats typically support dynamic region extraction. Another part of the solution is to stream equally sized chunks of the derivative image to a Web client instead of delivering it in its entirety. We have all become familiar with this tiling approach through Google Maps. The tiling approach commonly requires the pre-generation and storage of each tile at all levels of resolution, which significantly increases storage requirements . At the client end, typical solutions use a JavaScript-driven HTML viewer or a viewer based on a browser plug-in such as Flash. Typically, these viewers minimally allow paging (navigation across image sections) and zooming.

We decided to try and address these issues and engaged in the development of the aDORe djatoka (with silent “d”) dynamic image server. These are djatoka’s representative characteristics:

• Use of the ISO-standardized JPEG 2000 format as the service format;
• Java-based open source solution built around the Kakudu JPEG 2000 library;
• Geared towards reuse through URI-addressability of all image disseminations including regions, rotations, and format transformations;
• Provision of a consistent, guessable URI pattern for image disseminations based on the ANSI/NISO OpenURL standard;
• Provision of an extensible service framework for image disseminations enabled by OCLC’s Java OpenURL package;
• Availability of image disseminations in a range of image formats;
• Availability of image disseminations for locally stored JPEG 2000 files, as well as for Web-accessible images in a variety of formats;
• Configurable server-side, file-based caching;
• Ajax-based client reference implementation, based on IIPImage JavaScript Viewer, which allows panning, zooming, and selecting the URI of the current view.

The JPEG 2000 image format has attracted considerable attention among others due to its open specification, and its rich feature set defined in a multi-part ISO standard. Features of interest in the context of this article are:

• Multiple Resolutions: When compressing a master image, JPEG 2000 uses the multi-resolution attributes of the discrete wavelet transform (DWT) to store multiple DWT levels. In djatoka, the number of DWT levels is determined by the number of times an image can be halved from its maximum resolutions longest side to 92 pixels or less.
• Region Extraction: The structure of JPEG 2000 files supports region decoding such that only the image data needed to render a sub-region is decompressed for display.
• Compression: Support for lossless and lossy wavelet-based compression.
• Self-containdness: Support for embedding XML metadata within the image bitstream, which may be used for licensing information, provenance description, descriptive metadata, etc.
• Progressive Transmission: This allows a viewer to display a lower quality version of the image as soon as a small part of the entire image file has been received; the image quality then improves progressively as the downloading proceeds. JPEG 2000 supports progressive transmission in two dimensions: by pixel accuracy and by image resolution.

We describe the overall architecture of the solution by explaining the steps involved in a dissemination request issued from our Ajax-based client reference implementation.

Let’s assume our Ajax client issues the first request to view an image that resides in the djatoka environment. The client starts by issuing an HTTP GET requesting metadata (e.g., width, height, DWT levels) for the image from the djatoka resolver. Details of these metadata-carrying HTTP URIs are provided here, but it suffices to mention that they contain an identifier of the image and some parameters detailing the specific dissemination that is requested for the identified image. In the case of an initial metadata request, the djatoka resolver responds with a JSON (JavaScript Object Notation) object containing image metadata, which the viewer uses for layout purposes. The djatoka reference implementation uses this information to calculate the center of the object and the number of 256x256 pixel tiles that will be necessary to fill a fixed-size viewport region.

The client will initially display a view of the image filling the existing viewport. A user can then zoom-in and explore regions by using common interface tools such as a navigatable reference image, zoom-in, and zoom-out buttons. The process to display a specific viewport always starts with the client determining the y,x offset values and resolution level for each of the 256x256 tiles necessary to fill the viewport. For each tile, an HTTP GET is issued against the djatoka resolver requesting a Region (Step 1). This getRegion HTTP GET request carries the identifier of the image, the x,y offset, tile width and height parameter values, and optionally the desired image format for display (default JPEG) and the rotation degree (default 0).

For each of the incoming HTTP GET requests, the resolver parses off the image identifier and the service parameters, and passes them on to the djatoka API (Step 2). There, the image identifier is resolved to the file path of the corresponding JPEG 2000 image. Then the Kakadu library is called (Step 3). The Kakadu library requests the file handle for the provided JPEG 2000 image file path (Step 4), and subsequently returns the requested Region at the specified resolution level and rotation degree as an uncompressed, ASCII-based PNM (portable anymap) file to the djatoka API (Step 5). The djatoka API then transforms the PNM file to the requested image format (default JPEG) and performs any additional transformations (e.g., watermarking) that might have been specified. The resulting image is then returned to the resolver (Step 6) and from there on to the client. The multiple images resulting from the HTTP GET requests for individual 256x256 tiles are seamlessly joined in the user interface using CSS and JavaScript.

The process described so far assumes the existence of a JPEG 2000 image managed by the djatoka environment, i.e., the HTTP GET request carries an image identifier that the djatoka API can resolve to a file handle. However, djatoka also supports providing the URI of any web-accessible image as the image identifier. In this case, the djatoka API will obtain the image by dereferencing the URI and dynamically converting it to a JPEG 2000 file that is stored locally. In the djatoka environment, the image URI will then be associated with the image’s file path, which allows the image’s URI to be used in the HTTP GET for a requested Region. It is up to the djatoka implementation to decide on the lifetime of the local copy of these Web accessible images.

djatoka is open source Java software that builds upon a rich set of APIs and libraries to provide a service framework for the dynamic dissemination of JPEG 2000 image files.

The djatoka API provides:

• Compression of JPEG 2000 files using the Kakadu JPEG 2000 Library with properties to improve extraction performance and good compression/quality balance;
• Dynamic extraction of multiple resolutions and Regions from JPEG 2000 files;
• Support for a rich set of input/output formats (e.g., BMP, GIF, JPG, PNG, PNM, TIF, JPEG 2000) through the use of the ImageJ, Java Advanced Imaging, and the Kakadu JPEG 2000 Library;
• Extensible interfaces to request image services and manipulations (e.g., watermarking);
• A rich service framework, based on the OCLC OpenURL Resolver, to facilitate the transfer of service parameters via an OpenURL compliant HTTP GET request.
• Configurable File-based Caching for improved performance.

[1] Van de Sompel, H., Chute, R., Hochstenbach, P. (In Press). The aDORe Federation Architecture. International Journal on Digital Libraries. Preprint at arXiv:0803.4511.

[2] International Organization for Standardization. (2004). ISO/IEC 15444-1:2004: JPEG 2000 image coding system: Core coding system. http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=37674

[3] National Information Standardization Organization. (2004). ANSI/NISO Z39.88-2004: The OpenURL Framework for Context-Sensitive Services.
http://www.niso.org/standards/resources/Z39_88_2004.pdf

[4] Marcellin, M.W., Gormish, M.J., Bilgin, A., Boliek, M.P. (2000). An overview of JPEG-2000. Proceedings of the IEEE Data Compression Conference, pp. 523-541
doi:10.1109/DCC.2000.838192

[5] Bernier, R. (2006). An Introduction to JPEG 2000. Library Hi Tech News, 23(7), pp. 26-27
doi:10.1108/07419050610704349

[6] Buonora, P., Liberati, F. (2008). A Format for Digital Preservation of Images: A Study on JPEG 2000 File Robustness. D-Lib Magazine, 14(7/8).
doi:10.1045/july2008-buonora

[7] Lepley. M. (2008). JPEG 2000: fast access to large grayscale images. Proceedings of the International Society for Optical Engineering, Vol. 6943, 69431B
doi:10.1117/12.777178