<font size=2 face="sans-serif">How about some version information? All
this information only applies to version 1.9, correct? We know there are
small differences (which are not yet interpreted correctly by OpenSlide)
in versions 1.8, 2.0, and 2.1. Maybe a note should also be made about the
distinction b/w brightfield and immuno. How about a reference to the vendor
(3dhistech)?</font>
<br>
<br><font size=2 face="sans-serif">Yves</font>
<br>
<br>
<br>
<br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Benjamin Gilbert <bgilbert@cs.cmu.edu></font>
<br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">OpenSlide Users <openslide-users@lists.andrew.cmu.edu></font>
<br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">24-07-12 07:17</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Introduction
to MIRAX/MRXS</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">openslide-users-bounces+sucaet=histogenex.com@lists.andrew.cmu.edu</font>
<br>
<hr noshade>
<br>
<br>
<br><tt><font size=2>I've been asked to write some introductory material
on the MIRAX format, <br>
since the documentation on the OpenSlide website is incomplete and <br>
occasionally cryptic. Here it is. Comments welcome.<br>
<br>
There are two parts to this story: the tile structure of MIRAX slides <br>
and the layout of the data on disk. Both parts can be fully appreciated
<br>
only with a certain sense of humor or a certain type of beverage. <br>
Throughout, I will be using the CMU-1.mrxs slide to illustrate.<br>
<br>
A note on terminology: the vendor calls the format MRXS. For historical
<br>
reasons, OpenSlide calls it MIRAX.<br>
<br>
Tile structure<br>
--------------<br>
<br>
The scanner works by taking multiple photos of the slide as the camera
<br>
moves past the glass. (Or as the glass moves past the camera; I'm
not <br>
sure which.) The scanner tries to overlap these photos by an amount
<br>
specified in the OVERLAP_X and OVERLAP_Y Slidedat keys, stated in <br>
pixels. However, the camera's movements are not very precise, so
in <br>
fact the position of each photo will be slightly different than the <br>
nominal overlap values would suggest.<br>
<br>
On older scanners, the hardware knew the position of the camera with <br>
high precision, even though it couldn't move it very accurately. These
<br>
positions were recorded in the slide file in the <br>
VIMSLIDE_POSITION_BUFFER.default non-hierarchical section and used to <br>
properly position each photo. I suspect that newer scanners cannot
<br>
detect the position of the camera, relying instead on post-processing to
<br>
detect the degree of overlap between adjacent photos. This would
<br>
explain the format change between version 1.9 and 2.2 slides.<br>
<br>
The camera's photos are fairly high-resolution, too large to be <br>
practically used as image tiles. So, on disk, they are broken up
into <br>
multiple JPEGs, N on a side. In CMU-1.mrxs and many other slides,
N is <br>
4 (it's the GENERAL.CameraImageDivisionsPerSide value in the Slidedat),
<br>
so there are 16 tiles per camera position. This is in level 0, the
<br>
highest-resolution level.<br>
<br>
Each numerically higher (lower-resolution) level concatenates four tiles
<br>
from the previous level in a 2x2 grid and scales the image down by a <br>
factor of 2. So level 1 has 4 tiles per camera position and level
2 has 1.<br>
<br>
Level 3 then has to concatenate tiles corresponding to *different* <br>
camera positions. And indeed it does, in the exact same way: a 2x2
<br>
grid. But those camera positions overlap! So in the middle
of the new <br>
concatenated tile is a block of garbage: 15 pixels, nominally, which are
<br>
redundant with the 15 pixels next to them. Of course the actual number
<br>
of pixels of garbage depends on how much the camera positions overlap,
<br>
which varies from photo to photo.<br>
<br>
This problem gets worse and worse as we move through the levels. By
the <br>
time we get to level 9, each 340x256 pixel tile has 127 blocks of <br>
garbage in each dimension, each of which is nominally 0.234375 pixels <br>
wide. In order to render this tile, we have to separately extract
the <br>
pixels corresponding to each camera position -- many of which are <br>
fractional pixels due to repeated downsampling -- and render them in <br>
their correct positions at sub-pixel resolution. By the nature of
<br>
sub-pixel image manipulation, the result can only be an approximation of
<br>
a cleanly-downsampled image.<br>
<br>
All other slide formats supported by OpenSlide process any overlaps <br>
during the scanning process, before generating reduced-resolution <br>
levels. MIRAX is the only supported format which defers the processing
<br>
of image overlaps to the viewer application, and it is what drove <br>
OpenSlide to depend so extensively on the Cairo graphics library.<br>
<br>
On-disk format<br>
--------------<br>
<br>
The MIRAX on-disk format is complicated, full of <br>
things-pointing-to-other-things. The format stores two types of data:
<br>
hierarchical data (that is, pyramidal images: the actual slide data, <br>
plus some other stuff we don't decode), and non-hierarchical data <br>
(thumbnail images, etc.). Each type of data is stored in a tree <br>
structure dedicated to that type, and finding a block of data requires
<br>
us to traverse a lot of pointers.<br>
<br>
Let's say we want to draw a single JPEG tile from the seventh pyramid <br>
level of CMU-1.mrxs. We do the following:<br>
<br>
1. We start with the [HIERARCHICAL] section in Slidedat.ini. We
want <br>
to read the image pyramid, which is hierarchical data, so we look at the
<br>
HIER_* keys. HIER_COUNT is 3, so there are three hier trees. We
read <br>
each HIER_%d_NAME key, for %d from 0 to 2, until we find one with a <br>
value of "Slide zoom level". We've now discovered that
we want HIER_0.<br>
<br>
2. HIER_0_COUNT is 10, so this hier tree has ten leaves, each <br>
corresponding to a pyramid level. We want to read from the seventh
<br>
pyramid level, so we read the HIER_0_VAL_6_SECTION key to get the name
<br>
of a different Slidedat section: in this case, LAYER_0_LEVEL_6_SECTION.<br>
<br>
3. We look at LAYER_0_LEVEL_6_SECTION. There we find some values
that <br>
may be useful: the nominal camera position overlap for this level (1.875
<br>
pixels), MICROMETER_PER_PIXEL values, etc. But this doesn't help
us <br>
find the image data.<br>
<br>
4. To locate the image data, we need to look at the Index.dat. <br>
Index.dat begins with a version string and a UUID. Immediately after
<br>
that are two 4-byte pointer values (little-endian) which we call the <br>
hier_root and the nonhier_root. They give the locations within the
<br>
index file of, respectively, the hierarchical and non-hierarchical <br>
offset tables. We seek to the location specified by the hier_root.<br>
<br>
5. The offset table is an array of, again, 4-byte little-endian <br>
pointers. Now we need to determine which entry to read. If
we were to <br>
build a flat list of all of the HIER_0 sections in numerical order, <br>
followed by the HIER_1 sections, etc., the entry we need would <br>
correspond to our section's position in that list. In this case we
need <br>
the seventh entry. We seek to that location.<br>
<br>
6. Here we find a linked list of data pages. Each page begins
with two <br>
4-byte values (little-endian as always): the number of data entries in
<br>
the page and the address of the next page (or 0 if this is the end of <br>
the list). For some reason, the initial page in the list always has
0 <br>
data entries. We follow the pointer to the next entry.<br>
<br>
7. Now we have a page with entries in it. Each entry consists
of four <br>
4-byte integers: the tile index, offset, length, and file number. The
<br>
file number is an index into the array of filenames formed by the <br>
[DATAFILE] Slidedat section, and tells us which file to read. The
<br>
offset and length tell us what bytes to read out of that file. So
all <br>
we have to do is traverse the linked list until we find the tile index
<br>
we want. Now we need to calculate that tile index.<br>
<br>
8. The tile index is defined as (y * tiles_across + x), where <br>
tiles_across is really GENERAL.IMAGENUMBER_X from the Slidedat file. <br>
That's fine for level 0. In higher levels, x and y are always multiples
<br>
of 2^level to account for the lower number of JPEG tiles. So if we
want <br>
the tile at position (3, 4) within level 6, we need tile index (4 <<
6) <br>
* 352 + (3 << 6) = 90304. (This tile may not even exist. If
the <br>
scanning software determines that a particular tile is blank, it omits
<br>
the tile entirely. At higher levels, a tile exists if any of the
<br>
constituent level 0 tiles also exist.)<br>
<br>
9. Suppose the tile does exist. Now we can finally read out
the data <br>
for a single 340x256 JPEG. Hooray! Now all we need to do is
extract <br>
and render 1,024 subtiles to account for the 31 overlapped regions on <br>
each axis of the tile. Of course, in order to know exactly *where*
to <br>
render those subtiles within the output image, we need to know the exact
<br>
position of the camera when it produced each subtile.<br>
<br>
10. The camera position map is stored in a non-hierarchical section
<br>
called "default" in a tree called "VIMSLIDE_POSITION_BUFFER".
To find <br>
it, we need to start all the way back at the top, in the Slidedat file.<br>
<br>
Aside: Reading non-hierarchical sections<br>
----------------------------------------<br>
<br>
10a. We again start with the Slidedat [HIERARCHICAL] section. By
<br>
traversing NONHIER_COUNT, NONHIER_%d_NAME, NONHIER_%d_COUNT, and <br>
NONHIER_%d_VAL_%d, we eventually find our nonhier section at <br>
NONHIER_3_VAL_0. So the index into the nonhier offset table is <br>
NONHIER_0_COUNT + NONHIER_1_COUNT + NONHIER_2_COUNT + 0 = 12.<br>
<br>
10b. We read the Index.dat as before: nonhier_root to nonhier offset
<br>
table to linked list head. Again the first page in the linked list
has <br>
no data entries. The second page has one entry and a 0 next pointer.
<br>
The data entry itself is an array of five 4-byte values: 0, 0, offset,
<br>
length, and file number. Good enough! Now we can look up the
file <br>
number in the [DATAFILE] Slidedat section, read out the data, and if we
<br>
were reading the nonhier section for a thumbnail or barcode image, we'd
<br>
be done. But we're not. We still need to decode the slide position
file.<br>
<br>
Tile decoding, part II<br>
----------------------<br>
<br>
11. The slide position file is an array of 9-byte entries, one for
each <br>
camera position, in row-major order. Each entry consists of a flag
byte <br>
of unknown purpose (which is always 0 or 1) and two 4-byte signed <br>
integers representing the level 0 X and Y pixel coordinates of the <br>
camera position. (Negative coordinate values do occasionally occur.)
<br>
If a camera position was omitted from the slide file because its region
<br>
was empty, its coordinate values will be garbage or 0. So, to finally
<br>
draw our tile, all we need to do read the camera positions for each of
<br>
its 1,024 subtiles which have corresponding tiles in level 0, divide <br>
their coordinates by 2^level, and render away!<br>
<br>
Epilogue<br>
--------<br>
<br>
With MIRAX, as with all formats, OpenSlide actually loads all of the <br>
pertinent slide metadata before openslide_open() returns. At runtime
<br>
(that is, during openslide_read_region()) it can simply look up subtile
<br>
positions in memory, do lots and lots of compositing, and return the <br>
desired pixels.<br>
<br>
MRXS files that are generated by the Export function of the vendor's <br>
viewer application don't have any overlaps, because the viewer is kind
<br>
enough to preprocess them away. In this case there is no <br>
VIMSLIDE_POSITION_BUFFER, no nominal overlaps, and OpenSlide skips the
<br>
subtile processing for greater performance. The application also
has a <br>
"Save" command which can produce a downsampled version of a slide;
the <br>
resulting slide simply omits the requisite number of bottom levels, <br>
divides all of the coordinate values in the slide position file by <br>
2^levels_skipped, and updates the IMAGE_CONCAT_FACTOR of the now-lowest
<br>
level to reflect the number of levels that were skipped.<br>
<br>
Please use caution when depending on any of the details described above,
<br>
as some of them are from memory and may have shifted during flight. <br>
Almost all of the above was discovered by Adam Goode, who has more <br>
patience than I do; all errors of narrative are mine; all design choices
<br>
are the original vendor's. Now, if you'll excuse me, I need to go
find <br>
a certain type of beverage.<br>
<br>
--Benjamin Gilbert<br>
<br>
_______________________________________________<br>
openslide-users mailing list<br>
openslide-users@lists.andrew.cmu.edu<br>
</font></tt><a href="https://lists.andrew.cmu.edu/mailman/listinfo/openslide-users"><tt><font size=2>https://lists.andrew.cmu.edu/mailman/listinfo/openslide-users</font></tt></a><tt><font size=2><br>
</font></tt>
<br>
<br>
<HR><BR>WARNING: This e-mail, including any attachments, may contain CONFIDENTIAL INFORMATION, including privileged and/or health information. It is for the sole use of the intended recipients. Any unauthorized copying, disclosure, distribution, reproduction, use or retention of this email or the information in it, is strictly FORBIDDEN. If you are not an intended recipient, please notify the sender immediately (REPLY this e-mail) and permanently DELETE the related e-mail. Please be aware that this email and replies to it may be monitored by the sender's company for quality assurance, policy compliance and/or security purposes.
<br>