New open image format

[Benjamin Gilbert]

On 06/26/2013 08:39 AM, Dr. M. Weihrauch wrote:
> Now I would like to define an image format for his company that would be
> completely open for openslide.
>
> 4. Is there already a free, open, well-documented format that we could
> tap into?

There is DICOM supplement 145:

     ftp://medical.nema.org/medical/dicom/final/sup145_ft.pdf

145 is reasonable in itself, but it's essentially a patch to an enormous 
set of specifications containing a large number of options.  As such, 
it'd be a pain to implement, plus there's no guarantee that your 
software would interoperate with other DICOM WSI software (depending on 
the options that you and they supported).  But it's the only open 
specification that tries to anticipate WSI-specific issues.

I'm told that adoption of DICOM 145 has been slow to nonexistent. 
Perhaps others on the list can fill in more details.

AFAICT, the only plausible alternative is to create Yet Another TIFF 
Variant.  Leica SCN does this reasonably well: it uses standard TIFF 
structures and metadata tags where possible, plus a custom XML document 
(contained in a TIFF ImageDescription field) for metadata that doesn't 
fit into standard TIFF tags.

To reduce file size, you may not want to store pixel data for blank 
parts of the image.  However, TIFF doesn't anticipate sparse images.  It 
may be possible to hack around this via a slight abuse of the TIFF 
standard, but that's not ideal.  DICOM does support sparse images.

I recommend reading DICOM 145, the TIFF 6 specification, and TIFF 
supplement 2.

> What is the format inside?   JPEG?  JPEG2000?

If you need more than 8 bits per channel, JPEG is a poor choice.  It 
technically has a 12-bit mode, and some medical imaging formats use it, 
but open-source support for 12-bit JPEG is abysmal.

Conversely, I'm not aware of a standardized way to use JP2K tiles in TIFF.

 From the image-compression perspective, I'm completely unqualified to 
answer this.

> 2. Is it necessary to pretile the 256x256 tiles and how are they
> accessed "from the outside"?

You do need to store tiles into the slide file.  Otherwise the viewer 
software would need to read an entire row in order to display a small 
piece of it.  TIFF supports tiled images, so no problem there.

One caution about tiling.  Scanner hardware often produces tiles which 
are overlapping and misaligned; these then need to be rendered into the 
correct location by software.  Different vendors do this at different times:

- Some formats (e.g. Aperio SVS) align everything before writing the 
file, so the on-disk tiles are in a regular grid.

- Some formats (e.g. Trestle, Ventana BIF) write out the unaligned tiles 
and expect the viewer to do the alignment.  This has the advantage that 
the original scanner data is still available if the alignment algorithms 
improve later, but of course it's harder to read.  TIFF and DICOM both 
require a regular tile grid, so vendors who do this with TIFF have to 
abuse the format.  Also, you still have to do the alignment calculations 
while writing out the file, in order to produce the downsampled levels. 
  (Unless you're MIRAX, but that was a disaster.)

> 3. If there is a complete image pyramid inside - comparable to the
> deepzoom-directory structure, do we need an extra thumbnail?

It's probably not *strictly* necessary, but it'd be nice.  It allows the 
writer to decide what should be shown in the thumbnail, rather than 
forcing each reader to make its own thumbnailing decisions.  (For 
example, how should a reader thumbnail a fluorescence image that 
includes an RGB overview?)

--Benjamin Gilbert