Research Highlights
Clinical Decision Support Using Deep Learning
Deep learning techniques show outstanding success in various areas of computer vision. That success has not translated yet into clinical practice yet, as the medical domain faces particular challenges such as paucity of training data and regulatory and privacy issues. However, deep learning powered clinical decision support systems could provide great value in aiding, for example, the diagnosis process, by mitigating negative effects such as stress, fatigue, or limited experience with certain diseases.
Our research is focusing on the medical application areas dermatology and radiology, ranging from data fusion (for example, clinical images and patient metadata), detecting out of distribution samples, to interpretability of classification results.
Related publications:
- Language model-based labeling of German thoracic radiology reports. Fortschr Röntgenstr 197, 2025 DOI
- German CheXpert Chest X-ray Radiology Report Labeler. Fortschr Röntgenstr 196, 2024 DOI
- Out-of-distribution detection with in-distribution voting using the medical example of chest X-ray classification. Medical Physics 51, 2024 DOI
- WindowNet: Learnable Windows for Chest X-ray Classification. Journal of Imaging 9, 2023 DOI
- Attention-based Saliency Maps Improve Interpretability of Pneumothorax Classification. Radiology: Artificial Intelligence 5, 2023 DOI
- Outlier Detection in Dermatology: Performance of different Convolutional Neural Networks for Binary Classification of Inflammatory Skin Diseases. Journal of the European Academy of Dermatology and Venereology 9, 2023 DOI
- Higher Chest X-ray Resolution Improves Classifi- cation Performance. arXiv:2306.06051, 2023 DOI
- Transformer-based interpretable multi- modal data fusion for skin lesion classification. arXiv:2304.14505, 2023 DOI
- Deep learning-based classification of dermatological lesions given a limited amount of labeled data. Journal of the European Academy of Dermatology and Venereology 36, 2022 DOI
- Risks and benefits of dermatological machine learning healthcare applications - an overview an ethical analysis. Journal of the European Academy of Dermatology and Venereology 36, 2022 DOI
- FusionM4Net: A multi-stage multi-modal learning algorithm for multi-label skin lesion classification. Medical Image Analysis 76, 2022 DOI
Robotic X-ray Computed Tomography
X-ray computed tomography is used in many domains, from medical diagnosis to materials testing. Typically, this means an X-ray source and an X-ray detector rotating around the patient or object of interest in a circular or helical geometry. In order to support more flexible acquisition geometries, for example to avoid high absorbers and the artifacts they create, or to enable advanced imaging modalities, such as AXDT (see below), a flexible robotic setup is very advantegeous.
Our research is focusing on the development of a flexible robotic platform for X-ray computed tomography, ranging from collision detection for safe operations, to path planning, calibration methods to support the high accuracy required, to acquisition optimization strategies.
Related publications:
- Runtime optimization of acquisition trajectories for X-ray computed tomography with a robotic sample holder. Engineering Research Express 5, 2023 DOI
- Spherical acquisition trajectories for X-ray computed tomography with a robotic sample holder. Engineering Research Express 5, 2023 DOI
- Spherical acquisition trajectories for X-ray Computed Tomography with a robotic sample holder. Fully3 Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine (Fully 3D), Stony Brook, USA, July, 2023
- X-ray computed tomography with seven degree of freedom robotic sample holder. Engineering Research Express 4, 2022 DOI
- Geometric calibration of seven degree of freedom Robotic Sample Holder for X-ray CT. International Conference on Image Formation in X-ray Computed Tomography (CT Meeting), Baltimore, USA, June, 2022 DOI
- X-ray Computed Tomography with a Robotic Sample Holder. International Conference on Image Formation in X-ray Computed Tomography (CT Meeting), Regensburg, Germany, August, 2020
X-ray Computed Tomography Reconstruction
X-ray computed tomography is a very useful tool with many applications, ranging from medical diagnosis to materials testing. Solving the associated inverse problem is a challenging task due to the ill-posedness and the problem size. State-of-the-art methods include model-based optimization approaches as well as deep learning based approaches.
Our research is focusing on hybrid approaches that combine model-based optimization with deep learning based approaches, leveraging the strengths of both.
Related publications:
- WNet: A Data-driven Dual-domain Denoising Model for Sparse-view Computed Tomography with a Trainable Reconstruction Layer. IEEE Transactions on Computational Imaging 9, 2023 DOI
- Translating sparse sinogram measurements into reconstruction patches without geometry information via the attention mechanism. Fully3 Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine (Fully 3D), Stony Brook, USA, July, 2023
- Hybrid Reconstruction Using Shearlets and Deep Learning for Sparse X-ray Computed Tomography. International Conference on Image Formation in X-ray Computed Tomography (CT Meeting), Baltimore, USA, June, 2022 DOI
- A Deep Residual Dual Domain Learning Network for Sparse X-ray Computed Tomography. Fully3 Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine (Fully 3D), Leuven, Belgium, July, 2021
- Deep Learning Acceleration of OS-SIRT in X-ray Cone-Beam Computed Tomography. International Conference on Image Formation in X-ray Computed Tomography (CT Meeting), Regensburg, Germany, August, 2020
Light Field Microscopy
Light field imaging is a scanless imaging technique that provides one-shot volumetric information, for example in fluorescent microscopy. It has proven very useful in biologic applications involving fast dynamics, due to its high speed 3D imaging capability. The light field microscope enables scan-less 3D imaging of fluorescent specimens by incorporating an array of micro-lenses into the optical path of a conventional wide-field microscope. Thus, both spatial and directional light field information is captured in a single shot, allowing for subsequent volumetric reconstruction using a wave-based forward model to describe the propagation of light.
Our research is focusing on deriving suitable forward models for light field microscopy, as well as algorithms to solve the resulting inverse problem. In addition, we are investigating deep learning techniques to dramatically accelerate the reconstruction process at inference time.
Related publications:
- Fast light-field 3D microscopy with out-of-distribution detection and adaptation through Conditional Normalizing Flows. Biomedical Optics Express 15, 2024 DOI
- Learning to Reconstruct Confocal Microscopy Stacks From Single Light Field Images. IEEE Transactions on Computational Imaging 7, 2021 DOI
- Real-Time Light Field 3D Microscopy via Sparsity-Driven Learned Deconvolution. IEEE, International Conference on Computational Photography (ICCP), Haifa, Israel, May, 2021 DOI
- What about computational super-resolution in fluorescence Fourier light field microscopy? Optics Express 28, 2020 DOI
- 3D deconvolution in Fourier integral microscopy. SPIE Defense + Commercial Sensing, Computational Imaging V, Anaheim, USA, April, 2020
- Learning to Model and Calibrate Optics via a Differentiable Wave Optics Simulator. IEEE, International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, October, 2020 DOI
- Artifact-free deconvolution in light field microscopy. Optics Express 27, 2019 DOI
- NeuBtracker – an imaging platform for interrogating neurobehavioral dynamics in freely behaving fish. Nature Methods 14, 2017 DOI
Anisotropic X-ray Dark-field Tomography
Conventional X-ray imaging is based on X-rays being absorbed differently by various materials or amounts of material, hence the term absorption contrast.
In the same way as visible light, X-rays can also be refracted (enabling phase contrast) or scattered (enabling dark-field contrast), which can be measured using X-ray grating interferometers.
A unique property of the dark-field contrast is its directional anisotropy, meaning that the signal changes when a sample is rotated in the plane orthogonal to the incoming X-ray beam.
This is due to fibrous microstructures (such as nerve fibers in the brain, or carbon fibers in composite materials) causing scattering orthogonal to the fiber orientation, even though the microstructures themselves are too small to be resolved.
Our research is focusing on deriving suitable forward models for the anisotropic X-ray dark-field contrast, as well as algorithms to solve the resulting inverse problem. In addition, we are investigating practical acquisition strategies to acquire the anisotropic dark-field signal.
Related publications:
- Laplace-Beltrami Regularization for Anisotropic X-ray Dark-field Tomography. Fully3 Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine (Fully 3D), Leuven, Belgium, July, 2021
- Fast Task-Driven Acquisition Optimization for Anisotropic X-ray Dark-field Tomography. International Conference on Image Formation in X-ray Computed Tomography (CT Meeting), Regensburg, Germany, August, 2020
- Task-driven Acquisition in Anisotropic X-ray Dark-Field Tomography. Fully3 Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine (Fully 3D), Philadelphia, USA, June, 2019 DOI
- Brain Connectivity Exposed by Anisotropic X-ray Dark-field Tomography. Scientific Reports 8, 2018 DOI
- Detectability Indices in Anisotropic X-ray Dark-Field Tomography International Conference on Image Formation in X-ray Computed Tomography (CT Meeting), Salt Lake City, USA, May, 2018
- Design of Acquisition Schemes and Setup Geometry for Anisotropic X-ray Dark-Field Tomography (AXDT). Scientific Reports 7, 2017 DOI
- Acquisition Schemes for Directional Dark-field Tomographic Modalities. Fully3 Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine (Fully 3D), Xi'an, China, June, 2017
- Statistical Models for Anisotropic X-Ray Dark- field Tomography. Fully3 Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine (Fully 3D), Xi'an, China, June, 2017
- Micro-structure orientation extraction for Anisotropic X-Ray Dark- Field Tomography. Fully3 Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine (Fully 3D), Xi'an, China, June, 2017
- Anisotropic X-ray Dark-Field Tomography: A Continuous Model and its Discretization. Physical Review Letters 117, 2016 DOI