Viewtify: Next-Generation Medical Image Viewer with Stereoscopic Display
SessionReal-Time Live!
DescriptionViewtify® fully leverages a game engine and GPU to generate and render high-quality 3DCG from CT and MRI images in real time. By using stereoscopic displays, it presents 3D visuals with depth information. Its high-speed processing also enables the real-time display of 4D CT and 4D MRI images as animated 3DCG.
1 Introduction
3DCG reconstruction from volumetric medical data, such as CT or MRI images, is commonly used in medical practice. There are many medical imaging software for it, from free tools like OsiriX (Pixmeo SARL, Switzerland) and 3D Slicer (open source) to commercial products like 3mensio (3mensio Medical Imaging B.V., Netherlands) and Mimics (Materialise NV, Belgium). However, the 3D rendering techniques are often simple and outdated, making the visuals less clear. Recently, some software like syngo.via (Siemens Healthineers, Germany) has applied global illumination for volume rendering, but resolution drops during manipulation. Unclear shading and resolution drop during interaction make interactive operations and understanding complex structures difficult.
Viewtify® (SCIEMENT, Inc., Japan) enables real-time 3DCG adjustments by achieving both fast 3D surface mesh synthesis from CT or MRI images via thresholding and high-quality real-time rendering. It also supports commercial glasses-free, eye-sensing stereoscopic displays (ELF-SR1, ELF-SR2 (SONY, Japan), and SpatialLabs View Pro (Acer Inc., Taiwan)), presenting the 3D visuals as objects with depth information.
2 Technical Overview
Viewtify is developed on Unreal Engine (Epic Games, Inc., U.S.) for high-quality real-time rendering, while the 3D mesh is generated on the GPU using custom development with CUDA. By sharing GPU resources between them, 3D data synthesized by CUDA can be directly read by Unreal Engine. For example, a mesh with about 9.6 million triangles has been produced from 512×512x412 (about 100 million) voxels. With an NVIDIA GeForce RTX 3090, this takes about 20 milliseconds, allowing for the rendering and display of 4D CT images at up to 40 fps in 4K resolution, including user interface elements.
3 High-Quality Rendering
Viewtify leverages Unreal Engine for modern, high-quality rendering, which significantly enhances the understanding of surgical anatomy by emphasizing shading. While a stereoscopic display can help with depth perception, exaggerated shading provides a clearer view, making modern rendering techniques essential for understanding complex anatomy.
4 Clinical Applications
Viewtify is not yet a medical device, but it has facilitated biventricular repair instead of univentricular repair in congenital heart disease by providing a 3D understanding of intracardiac structures. It has also aided in estimating endoleak types by visually analyzing their location and shape after abdominal stent-graft placement. Case reports and retrospective studies utilizing Viewtify are emerging, particularly in cardiac surgery.
5 Conclusion
Viewtify employs the simplest and most classic method of thresholding but can effectively generates high-quality 3DCG in real time.
By supporting stereoscopy, it also faithfully presents the information in CT and MRI images. There still remains significant potential for 3DCG to contribute to the medical field.
1 Introduction
3DCG reconstruction from volumetric medical data, such as CT or MRI images, is commonly used in medical practice. There are many medical imaging software for it, from free tools like OsiriX (Pixmeo SARL, Switzerland) and 3D Slicer (open source) to commercial products like 3mensio (3mensio Medical Imaging B.V., Netherlands) and Mimics (Materialise NV, Belgium). However, the 3D rendering techniques are often simple and outdated, making the visuals less clear. Recently, some software like syngo.via (Siemens Healthineers, Germany) has applied global illumination for volume rendering, but resolution drops during manipulation. Unclear shading and resolution drop during interaction make interactive operations and understanding complex structures difficult.
Viewtify® (SCIEMENT, Inc., Japan) enables real-time 3DCG adjustments by achieving both fast 3D surface mesh synthesis from CT or MRI images via thresholding and high-quality real-time rendering. It also supports commercial glasses-free, eye-sensing stereoscopic displays (ELF-SR1, ELF-SR2 (SONY, Japan), and SpatialLabs View Pro (Acer Inc., Taiwan)), presenting the 3D visuals as objects with depth information.
2 Technical Overview
Viewtify is developed on Unreal Engine (Epic Games, Inc., U.S.) for high-quality real-time rendering, while the 3D mesh is generated on the GPU using custom development with CUDA. By sharing GPU resources between them, 3D data synthesized by CUDA can be directly read by Unreal Engine. For example, a mesh with about 9.6 million triangles has been produced from 512×512x412 (about 100 million) voxels. With an NVIDIA GeForce RTX 3090, this takes about 20 milliseconds, allowing for the rendering and display of 4D CT images at up to 40 fps in 4K resolution, including user interface elements.
3 High-Quality Rendering
Viewtify leverages Unreal Engine for modern, high-quality rendering, which significantly enhances the understanding of surgical anatomy by emphasizing shading. While a stereoscopic display can help with depth perception, exaggerated shading provides a clearer view, making modern rendering techniques essential for understanding complex anatomy.
4 Clinical Applications
Viewtify is not yet a medical device, but it has facilitated biventricular repair instead of univentricular repair in congenital heart disease by providing a 3D understanding of intracardiac structures. It has also aided in estimating endoleak types by visually analyzing their location and shape after abdominal stent-graft placement. Case reports and retrospective studies utilizing Viewtify are emerging, particularly in cardiac surgery.
5 Conclusion
Viewtify employs the simplest and most classic method of thresholding but can effectively generates high-quality 3DCG in real time.
By supporting stereoscopy, it also faithfully presents the information in CT and MRI images. There still remains significant potential for 3DCG to contribute to the medical field.
Contributor
Event Type
Real-Time Live!
TimeFriday, 6 December 20244:00pm - 6:00pm JST
LocationHall C, C Block, Level 4
