3DPathology

We all know pathology, thinking of the healthcare professional examining a tissue section under a microscope, looking for evidence of cancerous cells. Many years ago, this process was digitalised by capturing the glass slides with a scanning device to provide a high-resolution digital image that can be viewed on a computer screen or mobile device, facilitating the acquisition, management, sharing and interpretation of pathology information. This way pathologists can engage, evaluate and collaborate rapidly and remotely, and thereby improve efficiency and productivity, as views, analysis and workflows are improved, while errors and turnaround times are reduced.

In 2015, the digital pathology market was set to multiply by more than five times within five years. This growth, coupled with declining numbers of qualified pathologists, represented a tremendous challenge for pathology departments of clinical and pharmaceutical organisations, and increased the urgency for higher quality diagnostic information to enable more effective and efficient treatments.

Digital images were becoming increasingly more accurate to render 3D shapes of objects, as seen in some medical imaging techniques. Organs structures and contents were already revealed in 3D distribution, but this was not yet the case for tissues, which require microscopic spatial resolution to develop 3D analysis. The main bottleneck to achieving efficient 3D imaging of tissues was to provide a quantitative and global analysis at microscopic resolution.

The ITEA project 3DPathology, headed by Barco and Philips along with knowledge partners and university hospitals from six countries, was set out to create a 3D digital pathology solution, based on a combination of multiple existing pathology modalities, for same-day diagnosis and much more personalised treatment of cancer.

Unique 3D multiple imaging

In order to overcome the anticipated challenges and to achieve this multi-modal 3D quantitative pathology analysis platform, the project had to come to terms with solving five major technological challenges:

1. The first was the 3D acquisition of data using multiple imaging modalities in a fast and automated way.
2. Secondly, the acquired data of some modalities had to be reconstructed and aligned from actual 2D images of pathologic slices into 3D images, combined with different modalities (size, resolution, storage format, spectral bandwidth), including the incorporation of techniques like alternating co-registration, light- or molecule channel-based slide alignment and reconstruction.
3. Then the aligned 3D data from different modalities had to be analysed to improve the quality of diagnosis. This was achieved by extracting and combining the relevant data, using techniques such as quantification, segmentation, machine learning and data mining.
4. Furthermore, the project came up with the development of new 3D visualisation and interaction technologies (equipment and algorithms) optimised for multi-modal 3D pathology.
5. And finally, an IT backbone was created to deal with data of tremendous size, produced by the individual imaging modalities (data sets in the range of Tera- to Petabytes).

A 3D multi-modal pathology demonstrator, the first of its kind in the world, enables unique features such as access to the microscopic organisation of tissue sub-structures in 3D, providing complete chemical information and access to unexplored dimensions of histology. By scaling up 3D microscopic images of bio samples, a better understanding is gained of the relationship between the morphological structures and the molecular, biochemical and metabolic information of tissue while the extraction of quantitative data enhances the molecular/chemical parameters ratio with respect to the structured components of a tissue. Finally, the 3D visualisation of, and interaction with, the relevant data from multiple imaging modalities optimises the presentation of the relevant views and parameters and allows the huge amounts of data (e.g. storage, transfer, processing, rendering) to be handled.

Impressive high-impact results

The 3DPathology project has had a significant impact on JPEG XS standardisation, which focuses on near loss-free, low-latency coding of high-resolution data. Intensive collaboration between imec, ETRO and VUB resulted in the launch of a new extension of JPEG 2000, namely part 15 High-throughput JPEG 2000 that particularly reduces the computational complexity and memory footprint of the EBCOT component of the JPEG 2000 standard, which makes it very interesting for the pathology use case.

In the field of dissemination, the project partners also achieved incredible results. Virchows Archiv European Journal of Pathology has awarded the unique ‘Virchows Archiv prize for the best paper of the year 2019’ to the paper ‘Deep learning for automatic Gleason pattern classification for grade group determination of prostate biopsies’ co-written by one of the project associates Dr. Marit Lucas from Academic Medical Center of the University of Amsterdam.

In addition, by exposing pathologists to more efficient workflows and faster learning, their knowledge and experience increase, enabling them to be more efficient. Increasing the accuracy in pathological examination practice and interpretation has a significant impact on improving quality of life due to personalised treatment, limiting re-occurrence as a result of better treatment outcomes and a reduction in the cost of healthcare from fewer readmissions. Thanks to the insights and results, Academic Medical Center of the University of Amsterdam reported a 10% reduction of re-occurrences/readmissions, considering the cost for a typical re-occurrence/readmission for bladder cancer diagnostics is 2 to 3k euros for every 6 months. Added benefits of workflow efficiency are a lower burden on pathology labs and mitigation of the lack of qualified pathologists in the future.

In respect of exploitation, Philips has been given FDA clearance in the US to market its IntelliSite Pathology Solution for primary diagnostic use there. Philips expects the results of the project to help bring new pathology scanners to market and an innovative multi-layer bright field imaging solution to increase its market share of the bright field pathology. Increased usability range and robustness will address the needs for both small labs and large medical centres.

Prodrive Technologies will support Philips to increase market share and secure Philips’ position in the Pathology market. As a result of the 3DPathology project, Prodrive Technologies finalised the scan engine design and the production tools, which are now available. This emerging technology project will provide better patient care. Furthermore, it will result in significantly higher revenue and contribute to a healthy and stable business relationship. In parallel, the 3D functionality will be improved and released. After finalising the 3D functionality, Prodrive Technologies will resell the 3D camera and parts of the scan engine technology in other application areas.

Slimmer AI, formerly Target Holding, gained a lot of experience in image handling and analysis from 3D molecular image alignment. This has been applied in different customer cases and proofs of concept (PoC), such as cooperations in optical (infrared) product inspection, change detection in sewer levels for water authorities (multi-modal: satellite, aerial photograph, on-spot), in aerial photographs for the land and land-use registration authority, and in technical drawings. It also was a starting point for the successful participation in the ITEA Medolution project.

Currently, the AI-based image analysis line is combined with Slimmer AI’s Natural Language Processing developments to form the PoC-version of an innovative data-room tool, in co-creation with a launching customer. This tool might become Slimmer AI’s next product, leading to a 20 FTE-spin-out within 5 years. Slimmer AI developed the prototype of the 3D Digital Pathology IT backbone infrastructure and provided it to the consortium for the secure exchange of (here: anonymised) medical data. The blueprint is still in use for high-security data-exchange with clients.

Barco has developed optimised display systems that address a variety of pathology lab needs for review, positioning of samples but also for diagnostic purposes. Several hundred of those displays have been sold already worldwide, and in the next few years Barco is expecting a large increase in sales of display systems for Digital Pathology. In addition, Barco prepared a White Paper for the Medical Imaging Working Group of ICC describing the importance of colour calibration for medical imaging and perceptual colour calibration. The White Paper is a first step towards the standardisation of medical colour imaging.

PS-Tech has developed masking technology for extractions within volumetric datasets. This technology, which has now been commercialised in Vesalius3D, is used for preoperative planning in various cardiovascular procedures. Applications for lung and tumour extractions are under development. The technology developed for zooming within large datasets is still under further development and will be included in a Vesalius3d release.

Imec, formerly iMinds, contributed to the standardisation of formal evaluation methodologies to assess the performance of visually loss-free compression systems, for still images and video sequences, and for high dynamic range content, as part of JPEG’s AIC activity (ISO/IEC 29170-1 and ISO/IEC 29170-2). These methodologies can be used as a basis to design a clinical validation test for the compressed 3D pathology data, in cooperation with Barco.

For the Korean partner Xavis, the ITEA 3DPathology project will result in bringing to market new 3D X-ray Microscopy Instrumentation capable of high-resolution, non-destructive imaging and analysis for the quantification of internal structural parameters at submicron to nanometre scale. Although the global and domestic investments have decreased because of the COVID-19 pandemic, XAVIS exerts all possible efforts at continual marketing to a number of medical facilities and national research institutes by sample test reporting and online meetings.

In addition to these strong results, 3DPathology was the first project in ITEA to involve Taiwan with the active participation of Academia Sinica and Bio Ma-tek, who strengthened this project as a solution provider for nano-biomaterial characterisation and analysis.

The novel disease diagnosis mechanism of cell units is expected to be investigated by the nano-biomaterial X-ray microscope system, which is not available in the conventional X-ray system. On the basis of the novel X-ray microscope system, nano-scale medical devices will be adjustable not only for the new disease diagnosis methods but also for the new cancer cell detection methods. As a result, the nano-biomaterial X-ray microscope system is expected to be an innovative development for medical and diagnostic services that contribute to improving the healthcare system on a global scale.

More information