New open image format

Thu Jun 27 03:06:37 EDT 2013

With my very biased DICOM hat on:

On Thu, Jun 27, 2013 at 5:56 AM, Benjamin Gilbert <bgilbert at cs.cmu.edu> wrote:
> On 06/26/2013 08:39 AM, Dr. M. Weihrauch wrote:
>> Now I would like to define an image format for his company that would be
>> completely open for openslide.
>>
>> 4. Is there already a free, open, well-documented format that we could
>> tap into?
>
> There is DICOM supplement 145:
>
>      ftp://medical.nema.org/medical/dicom/final/sup145_ft.pdf

This is also IMHO your very best option by far, if not the only one.

> 145 is reasonable in itself, but it's essentially a patch to an enormous
> set of specifications containing a large number of options.  As such,
> it'd be a pain to implement, plus there's no guarantee that your
> software would interoperate with other DICOM WSI software (depending on
> the options that you and they supported).  But it's the only open
> specification that tries to anticipate WSI-specific issues.

I do not believe this would be a "pain". There are plenty of quality
DICOM toolkit implementation to get you started.
Also if your goal is interoperalbility, then DICOM is the way to go
(think HIS). If you were to build
yet-another-private-format-for-virtual-slides you would not have
interoperality anyway.

> I'm told that adoption of DICOM 145 has been slow to nonexistent.
> Perhaps others on the list can fill in more details.

"DICOM is slow" is basically a tautology ;)
DICOM WG26 group went through the hard path of collecting infos from
multiple vendors and try to do their best to have a common format.
Adoption of new standard is slow compared to vendor-format which
maintains customers within vendor-products ;)

> AFAICT, the only plausible alternative is to create Yet Another TIFF
> Variant.  Leica SCN does this reasonably well: it uses standard TIFF
> structures and metadata tags where possible, plus a custom XML document
> (contained in a TIFF ImageDescription field) for metadata that doesn't
> fit into standard TIFF tags.

When Benjamin use the word "plausible", you should be very worried ;)
I would suggest reading issue #49 [1], excerpts:

"First, we need a custom TIFF parser, libtiff isn't up to the task"

Finally as some other pointed out, OME-TIFF and DICOM went through the
troubles in building up a list of attributes to be stored in your
image data. I would really recommend following this path, instead of
creating your own vocabulary.

> To reduce file size, you may not want to store pixel data for blank
> parts of the image.  However, TIFF doesn't anticipate sparse images.  It
> may be possible to hack around this via a slight abuse of the TIFF
> standard, but that's not ideal.  DICOM does support sparse images.

Olympus .vsi & .wtp also uses nice trick for empty tiles. But if you
are looking for a higher level solution, you may want to check JPEG
2000 and zero-tree coding (EZW).

> I recommend reading DICOM 145, the TIFF 6 specification, and TIFF
> supplement 2.

I would also recommend reading the openslide source code if not done already.

>> What is the format inside?   JPEG?  JPEG2000?
>
> If you need more than 8 bits per channel, JPEG is a poor choice.  It
> technically has a 12-bit mode, and some medical imaging formats use it,
> but open-source support for 12-bit JPEG is abysmal.

JPEG (ITU-81) is indeed not the best choice today. However it's
support is extremely wide, and is a no-op when you have to deal with
web-apps. As mentioned before you will be forced to use the lossy
8bits mode of ITU-81, since support from other part of this standard
is not as widely spread ("abysmal" is a bit strong, when you look at
open-source DICOM implementations).
JPEG 2000 is really designed to handle this task very well, but then
again its support at least in the open source world is abysmal ;)
JPEG-XR may work for you [2]. (why not even JPEG-XT ?).

> Conversely, I'm not aware of a standardized way to use JP2K tiles in TIFF.
>
>  From the image-compression perspective, I'm completely unqualified to
> answer this.
>
>> 2. Is it necessary to pretile the 256x256 tiles and how are they
>> accessed "from the outside"?
>
> You do need to store tiles into the slide file.  Otherwise the viewer
> software would need to read an entire row in order to display a small
> piece of it.  TIFF supports tiled images, so no problem there.

I would rephrase "you do need to store tiles" into:
- you need a fast API to access tiles within your slide. For instance,
you could use RAW files, with multiple levels. You simply do pointer
arithmetic to extract tiles. You never know: what if SSD becomes cheap
and random access is linear.
- You could use JPEG (ITU-81) or JPEG-2000 which provides a
representation for this pyramid type structure. See JPEG hierarchical
encoder or JPEG 2000 'precincts'.

> One caution about tiling.  Scanner hardware often produces tiles which
> are overlapping and misaligned; these then need to be rendered into the
> correct location by software.  Different vendors do this at different times:
>
> - Some formats (e.g. Aperio SVS) align everything before writing the
> file, so the on-disk tiles are in a regular grid.
>
> - Some formats (e.g. Trestle, Ventana BIF) write out the unaligned tiles
> and expect the viewer to do the alignment.  This has the advantage that
> the original scanner data is still available if the alignment algorithms
> improve later, but of course it's harder to read.  TIFF and DICOM both
> require a regular tile grid, so vendors who do this with TIFF have to
> abuse the format.  Also, you still have to do the alignment calculations
> while writing out the file, in order to produce the downsampled levels.
>   (Unless you're MIRAX, but that was a disaster.)
>
>> 3. If there is a complete image pyramid inside - comparable to the
>> deepzoom-directory structure, do we need an extra thumbnail?
>
> It's probably not *strictly* necessary, but it'd be nice.  It allows the
> writer to decide what should be shown in the thumbnail, rather than
> forcing each reader to make its own thumbnailing decisions.  (For
> example, how should a reader thumbnail a fluorescence image that
> includes an RGB overview?)

If you go the DICOM path, then you can store the icon within the DICOM
DataSet. If you go the JPEG-2000 way, your very first pyramid level
could be used as input to produce a thumbnail...

[1] https://github.com/openslide/openslide/issues/49
[2] https://jxrlib.codeplex.com/

2cts
--
Mathieu