Visual Computing for Medicine: Theory, Algorithms, and Applications

By Bernhard Preim and Charl P Botha

Visual Computing for Medicine, Second Edition, offers cutting-edge visualization techniques and their applications in medical diagnosis, education, and treatment. The book includes algorithms, applications, and ideas on achieving reliability of results and clinical evaluation of the techniques covered. Preim and Botha illustrate visualization techniques from research, but also cover the information required to solve practical clinical problems. They base the book on several years of combined teaching and research experience. This new edition includes six new chapters on treatment planning, guidance and training; an updated appendix on software support for visual computing for medicine; and a new global structure that better classifies and explains the major lines of work in the field.

• Complete guide to visual computing in medicine, fully revamped and updated with new developments in the field
• Illustrated in full color
• Includes a companion website offering additional content for professors, source code, algorithms, tutorials, videos, exercises, lessons, and more

• Language: English
• Publisher: Elsevier Science
• Release date: Nov 7, 2013
• ISBN: 9780124159792

Bernhard Preim

Bernhard Preim was born in 1969 in Magdeburg, Germany. He received the diploma in computer science in 1994 (minor in mathematics) and a Ph.D. in 1998 for a thesis on interactive visualization for anatomy education from the Otto-von-Guericke University of Magdeburg. In 1999 he moved to Bremen where he joined the staff of MEVIS and directed the “computer-aided planning in liver surgery” group. Since Mars 2003 he is full professor for Visualization at the computer science department at the Otto-von-Guericke-University of Magdeburg, heading a research group focussed on medical visualization. His research interests include vessel visualization, exploration of blood flow, visual analytics in public health, virtual reality in medical education and since recently narrative visualization. He authored “Visualization in Medicine” (Co-author Dirk Bartz, 2007) and “Visual Computing in Medicine” (Co-author: C. Botha, 2013). Bernhard Preim founded the working group Medical Visualization in the German Society for Computer Science and served as speaker from 2003-2012. He was president of the German Society for Computer- and Robot-Assisted Surgery (www.curac.org). He was Co-Chair and Co-Organizer of the first and second Eurographics Workshop on Visual Computing in Biology and Medicine (VCBM) in 2008 and 2010 and lead the steering committee of that workshop until 2019. He is the chair of the scientific advisory board of ICCAS (International Competence Center on Computer-Assisted Surgery Leipzig, since 2010). From 2011-2018 he was an associate editor of IEEE Transactions on Medical Imaging and and IEEE Transactions on Visualization and Graphics (2017-2022). Currently he serves in the editorial board of Computers & Graphics (since 2019). He was also regularly a Visiting Professor at the University of Bremen where he closely collaborates with Fraunhofer MEVIS (2003-2012) and was Visiting Professor at TU Vienna (2016).

Book preview

Visual Computing for Medicine

Theory, Algorithms, and Applications

Second Edition

Bernhard Preim

Charl Botha

Table of Contents

Cover image

Title page

Copyright

Acknowledgments

Foreword to the Second Edition

Preface to the Second Edition

Author Biography

Chapter 1. Introduction

Abstract

1.1 Visualization in Medicine as a Speciality of Scientific Visualization

1.2 Computerized Medical Imaging

1.3 2D and 3D Visualizations

1.4 Further Information

1.5 Organization

Part I: Acquisition, Analysis, and Interpretation of Medical Volume Data

Part I. Acquisition, Analysis, and Interpretation of Medical Volume Data

Chapter 2. Acquisition of Medical Image Data

Abstract

2.1 Introduction

2.2 Medical Image Data

2.3 Data Artifacts and Signal Processing

2.4 X-Ray Imaging

2.5 Computed Tomography

2.6 Magnetic Resonance Imaging

2.7 Ultrasound

2.8 Imaging in Nuclear Medicine

2.9 Intraoperative Imaging

2.10 Summary

Further Reading

Chapter 3. An Introduction to Medical Visualization in Clinical Practice

Abstract

3.1 Introduction

3.2 Diagnostic Accuracy

3.3 Visual Perception

3.4 Storage of Medical Image Data

3.5 Conventional Film-Based Diagnosis

3.6 Soft-Copy Reading

3.7 Medical Visualization in Nuclear Medicine

3.8 Medical Image Data in Radiation Treatment Planning

3.9 Medical Team Meetings

3.10 Concluding Remarks

Chapter 4. Image Analysis for Medical Visualization

Abstract

4.1 Introduction

4.2 Preprocessing and Filtering

4.3 An Introduction to Image Segmentation

4.4 Graph-Based Segmentation Techniques

4.5 Advanced and Model-Based Segmentation Methods

4.6 Interaction for Segmentation

4.7 Validation of Segmentation Methods

4.8 Registration and Fusion of Medical Image Data

4.9 Summary

Further Reading

Chapter 5. Human-Computer Interaction for Medical Visualization

Abstract

5.1 Introduction

5.2 User and Task Analysis

5.3 Metaphors

5.4 Prototyping

5.5 User Interface Principles and User Experience

5.6 3D Interaction Techniques

5.7 Input Devices

5.8 HCI in the Operating Room

5.9 Mobile Computing

5.10 Evaluation

5.11 Conclusion

Part II: Visualization and Exploration of Medical Volume Data

Part II. Visualization and Exploration of Medical Volume Data

Chapter 6. Surface Rendering

Abstract

6.1 Introduction

6.2 Reconstruction of Surfaces from Contours

6.3 Marching Cubes

6.4 Surface Rendering of Unsegmented Volume Data

6.5 Surface Rendering of Segmented Volume Data

6.6 Advanced Mesh Smoothing

6.7 Mesh Simplification and Web-Based Surface Rendering

6.8 Concluding Remarks

Chapter 7. Direct Volume Visualization

Abstract

7.1 Theoretical Models

7.2 The Volume Rendering Pipeline

7.3 Compositing

7.4 Volume Raycasting

7.5 Efficient Volume Rendering

7.6 Direct Volume Rendering on the GPU

7.7 Summary

Further Reading and Experimentation

Chapter 8. Advanced Direct Volume Visualization

Abstract

8.1 Introduction

8.2 Volumetric Illumination

8.3 Artificial Depth Enhancements

8.4 Concluding Remarks

Further Reading

Chapter 9. Volume Interaction

Abstract

9.1 Introduction

9.2 One-Dimensional Transfer Functions

9.3 Multidimensional Transfer Functions

9.4 Gradient-Based and LH-Based Transfer Functions

9.5 Local and Distance-Based Transfer Functions

9.6 Advanced Picking

9.7 Clipping

9.8 Virtual Resection

9.9 Cutting Medical Volume Data

9.10 Summary

Further Reading

Chapter 10. Labeling and Measurements in Medical Visualization

Abstract

10.1 Introduction

10.2 General Design Issues

10.3 Interactive Measurement of Distances and Volumes

10.4 Automatic Distance Measures

10.5 Angular Measurements

10.6 Measurements in Virtual Reality

10.7 Labeling 2D and 3D Medical Visualizations

10.8 Summary

Further Reading

Part III: Advanced Medical Visualization Techniques

Part III. Advanced Medical Visualization Techniques

Chapter 11. Visualization of Vascular Structures

Abstract

11.1 Introduction

11.2 Enhancing Vascular Structures

11.3 Projection-Based Visualization

11.4 Vessel Analysis

11.5 Model-Based Surface Visualization

11.6 Model-Free Surface Visualization

11.7 Vessel Visualization for Diagnosis

11.8 Summary

Further Reading

Chapter 12. Illustrative Medical Visualization

Abstract

12.1 Introduction

12.2 Medical Applications

12.3 Curvature Approximation

12.4 An Introduction to Feature Lines

12.5 Geometry-Dependent Feature Lines

12.6 Light-Dependent Feature Lines

12.7 Stippling

12.8 Hatching

12.9 Illustrative Shading

12.10 Smart Visibility

12.11 Conclusion

Further Reading

Chapter 13. Virtual Endoscopy

Abstract

13.1 Introduction

13.2 Medical and Technical Background

13.3 Preprocessing

13.4 Rendering for Virtual Endoscopy

13.5 User Interfaces for Virtual Endoscopy

13.6 Applications

13.7 Concluding Remarks

Further Reading

Chapter e14. Projections and Reformations

Abstract

14.1 Introduction

14.2 Overview

14.3 Anatomical Unfolding

14.4 Anatomical Planar Reformation/Projection

14.5 Map Projections

14.6 Conclusion

Part IV: Visualization of High-Dimensional Medical Image Data

Part IV. Visualization of High-Dimensional Medical Image Data

Chapter 15. Visualization of Brain Connectivity

Abstract

15.1 Introduction

15.2 Acquisition of Connectivity Data

15.3 Visualization of Structural Connectivity

15.4 Visualization of Connectivity Matrices

15.5 Summary

Further Reading

Chapter e16. Visual Exploration and Analysis of Perfusion Data

Abstract

16.1 Introduction

16.2 Medical Imaging

16.3 Data Processing and Data Analysis

16.4 Visual Exploration of Perfusion Data

16.5 Visual Analysis of Perfusion Data

16.6 Case Study: Cerebral Perfusion

16.7 Case Study: Breast Tumor Perfusion

16.8 Case Study: Myocardial Perfusion

16.9 Further Application Areas

16.10 Concluding Remarks

Further Reading

Part V: Treatment Planning, Guidance and Training

Part V. Treatment Planning, Guidance and Training

Chapter 17. Computer-Assisted Surgery

Abstract

17.1 Introduction

17.2 General Tasks

17.3 Visualization Techniques

17.4 Guidance Approaches

17.5 Application Areas

17.6 Conclusions

Further Reading

Chapter 18. Image-Guided Surgery and Augmented Reality

Abstract

18.1 Introduction

18.2 Image-Guided Surgery

18.3 Registration

18.4 Calibration and Tracking

18.5 Navigated Control

18.6 Display Modes

18.7 Visualization Techniques for Medical Augmented Reality

18.8 Applications

18.9 Summary

Further Reading

Chapter 19. Visual Exploration of Simulated and Measured Flow Data

Abstract

19.1 Introduction

19.2 Basic Flow Visualization Techniques

19.3 From Medical Image Data to Simulation Models

19.4 Visual Exploration of Measured Cardiac Blood Flow

19.5 Exploration of Simulated Cerebral Blood Flow

19.6 Biomedical Simulation and Modeling

19.7 Concluding Remarks

Further Reading

Chapter e20. Visual Computing for ENT Surgery Planning

Abstract

20.1 Introduction

20.2 Planning and Training Endoscopic Sinus Surgery

20.3 Visual Computing for Inner and Middle Ear Surgery

20.4 Neck Surgery Planning

20.5 Image Analysis for Neck Surgery Planning

20.6 Interactive Visualization for Neck Surgery Planning

20.7 Concluding Remarks

Further Reading

Chapter e21. Computer-Assisted Medical Education

Abstract

21.1 Introduction

21.2 e-Learning in Medicine

21.3 Anatomy Education

21.4 Surgery Education

21.5 Simulation for Surgery and Interventional Radiology

21.6 Simulation for Training Interventional Procedures

21.7 Systems for Training Operative Techniques

21.8 Training Systems Based on Physical Models

21.9 Skills Assessment

21.10 Summary

References

Index

Copyright

Acquiring Editor: Meg Dunkerley

Editorial Project Manager: Heather Scherer

Project Manager: Priya Kumaraguruparan

Designer: Mark Rogers

Morgan Kaufmann is an imprint of Elsevier

225 Wyman Street, Waltham, MA, 02451, USA

© 2014 Elsevier Inc. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods or professional practices, may become necessary. Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information or methods described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

Library of Congress Cataloging-in-Publication Data

Application submitted

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library.

ISBN: 978-0-12-415873-3

For information on all MK publications visit our website at www.mkp.com

Printed in the United States of America

14 15 16 17 10 9 8 7 6 5 4 3 2 1

Acknowledgments

This book was only possible with substantial support from a number of people. First, we want to thank Meg Dunkerley, Heather Scherer, Laura Lewin, and Lauren Mattos from Elsevier for the kind and intensive cooperation starting from the draft of a book proposal to the final stages of writing. Timo Ropinski provided the chapter on advanced volume rendering, making his long-term experience in that area available to the readers. The CAS chapter is based on the significant foundations laid by Thomas Kroes, and is considered to be joint work with him.

Petra Schumann, Petra Specht, and Steffi Quade did proof-reading, improved images, and helped with the generation of indices. We are grateful to all members, past and present, of the (Medical) Visualization group of the Delft University of Technology, who have formed an effective and especially pleasant platform for stimulating research on a number of the topics discussed in this book. We are grateful to the whole visualization group in Magdeburg, in particular Alexandra Baer, Steven Birr, Rocco Gasteiger, Sylvia Glaßer, Kerstin Kellermann, Paul Klemm, Christoph Kubisch, Arno Krüger, Kai Lawonn, Jeanette and Tobias Mönch, Konrad Mühler, Steffen Oeltze, Zein Salah, and Christian Tietjen. Jana and Lars Dornheim as well as Ivo Rössling carried out most of the developmental work described in the chapter on ENT surgery. Simon Adler (Fraunhofer IFF Magdeburg) contributed to surgery simulation. A number of Master students did extraordinary work that is partially reflected in this book: Roland Pfisterer, Daniel Proksch, and Christoph Russ.

The book is largely based on their research; they helped to focus and streamline the discussion of their research results. A number of people from other institutions helped to improve the book primarily by carefully commenting on individual chapters: Christian Rössl and Holger Theisel from the Visual Computing Group (Univ. of Magdeburg), Sebastian Schäfer and Klaus D. Toennies (Image Processing group, Univ. of Magdeburg), Raimund Dachselt (User Interface group, Univ. of Magdeburg), Oliver Speck and Daniel Stucht (Biomedical Magnetic Resonance group, Univ. of Magdeburg), Axel Böse and Georg Rose (Medical Technology group, Univ. of Magdeburg), Philipp Berg, Gabor Janiga and Dominique Thevenin (Fluid Simulation group, Univ. of Magdeburg), Oliver Großer (Radiology group, University of Magdeburg), Werner Korb (University of Applied Sciences, Leipzig), Tobias Isenberg (INRIA-Saclay) and Stefan Schlechtweg (Univ. of Applied Sciences, Koethen), Volker Diehl, Volker Dicken, Christian Hansen, Anja Hennemuth, Jan Klein, and Felix Ritter from Fraunhofer MEVIS Bremen, Ragnar Bade, Tobias Boskamp, and Olaf Konrad from MeVis Medical Solutions, Julian Ang and Alf Ritter from Brainlab, Roy van Pelt and Anna Vilanova (TU Eindhoven), Ralph Brecheisen (Arcus Solutions), Christian Dick (TU Munich), Heinz Handels (Univ. of Luebeck), Stefan Weber (Univ. of Bern), Claes Lindström, (University Linköping), Nigel W. John (Bangor University).

Finally, we want to acknowledge our long-term medical collaborators that provided the motivation for the research described in this book: Andreas Böhm, Andreas Dietz, Stefan Müller, and Gero Strauß (University hospital), Christoph Arens, Oliver Beuing, Jörg Franke, Martin Skalej, Christian and Ulrich Vorwerk (University hospital Magdeburg), Karl Oldhafer (Asklepios hospital Hamburg). We would also like to thank all our colleagues in The Netherlands, both medical and technical, from the Image Processing Division (LKEB), and the departments of Orthopedics, Radiology, Anatomy, and Surgery of the Leiden University Medical Center, and from the departments of Ophthalmology and of Neuroscience and Anatomy of the Erasmus Medical Center in Rotterdam, for the fruitful collaboration over the years.

A special and tender thanks to Uta Preim for providing feedback on all medical issues discussed in this book and for her complementary research, in particular on perfusion imaging.

Foreword to the Second Edition

Visual Computing for Medicine is an excellent textbook for students, researchers, and practitioners in the field of medical visualization. It is an authoritative resource for medical experts and technical personnel as well.

The book is the sequel to the highly successful first edition which immediately established itself as the reference work in this rather new and vibrant research topic in medical informatics. Dirk Bartz as one of the co-authors of the first edition unfortunately and untimely passed away in 2010. This left a sorely felt void in our community and prevented him from collaborating on the second edition.

This second edition provides a substantially updated, restructured, and extended view on the current state of the field. In a recent talk Donald Knuth, the preeminent computer scientist, was asked about relevant open research directions in computer science. After some reflections, he said that medical visualization was one of the important topics in this respect. This was good to hear, though he quickly (and regrettably) added that according to his opinion not many problems have been solved yet. I take the liberty to slightly disagree and put this textbook forward as compelling and written evidence to the contrary. And what an evidence it is! On over one-thousand pages the authors survey the intensive and rapid developments in our area. The relatively short period between first and second edition and the considerable amount of added material in extent and volume are very clear indications of the fast-paced evolution of visualization in medicine.

The book is concerned with diagnosis, treatment, and therapy planning with a focus on the currently most prevalent 2D and 3D imaging modalities. It thoroughly discusses the elaborate pipeline from data acquisition, analysis, and interpretation to advanced volume visualization and exploration techniques. Human computer interaction in the context of medical visualization has been covered in detail and encompasses significant topics like volume interaction, labeling, and measurement. Important application areas and advanced visualization techniques for blood vessels, virtual endoscopy, ENT surgery planning, perfusion, and diffusion data are extensively dealt with.

The book is very well structured, where the 22 chapters are grouped into five focal themes. The authors primarily organize the book according to techniques as most of these are applicable to a variety of medical tasks. Some of the material has been combined into completely new chapters like the one on projection-based medical visualization techniques. Hints at further readings at the end of each chapter point the interested reader to additional useful material not discussed within the chapter. Various advanced topics, which are of interest to the software engineer but are maybe too detailed for the general audience, are included in clearly marked break-out sections. The substantial reference list is another eloquent testimony of the breadth and depth of the topic.

The authors are highly recognized experts in the field of medical visualization. They have achieved the impressive feat of comprehensively covering a dynamic and rapidly emerging subject. The book provides informative, broad, and didactically well-organized information for specialists from diverse areas of expertise. The book will be the standard guide to medical visualization for years to come.

Dr. Eduard Gröller

Vienna University of Technology

Preface to the Second Edition

This second edition of Visualization in Medicine reflects the dynamic development of medical imaging, algorithmic processing and applications in medical research and clinical use after 2006. After the tragic passing of Dirk Bartz in March 2010, Charl Botha stepped in to prepare this new edition. In addition to careful rewriting of all chapters, we added a number of completely new chapters and reorganized and updated others significantly. Advances in imaging technology, e.g., hybrid devices, ultra high field MRI, intraoperative imaging, and the trend towards interventional procedures, are reflected in various parts of the book.

Since more and more advanced applications, e.g., in processing the complex multi-modal data of cardiac or neuroradiological MRI, have entered the stage of routine clinical use, human-computer interaction is becoming increasingly important. A comprehensive chapter was added to introduce HCI concepts with applications in medicine, incorporating recent interaction styles and technology. Also the chapter related to clinical practice was strongly extended by discussing also nuclear medicine, radiation treatment and medical team meetings in addition to the classical diagnostic settings.

Another essential trend is the combination of biomedical simulations with advanced visual exploration. As a consequence, we prepared a chapter that introduces basic techniques, such as the generation of simulation grids from medical imaging data and flow visualization, to explore the results. We study a number of specific applications, such as the simulation of blood flow to better predict the success of treatment options.

While the first edition of this book was focused on visual exploration, we have added discussions of data analysis techniques and their integration in what is widely called visual analytics. This relates, e.g., to cluster analysis and dimension reduction. We discuss these techniques in relation to high-dimensional data, such as perfusion data and diffusion tensor imaging data. They are, however, also relevant for volume classification, the basic process of assigning transfer functions to medical volume data.

Computer-assisted surgery (CAS), one of the most essential applications for medical visualization technology, has matured in the last decade. We use experiences gained in the design and evaluation of such systems to prepare a general introductory chapter on CAS, followed by chapters treating selected application areas, such as orthopedics. Intraoperative imaging and intraoperative guidance have grown in importance in the last years. The chapter devoted to this topic was significantly extended, e.g., with techniques developed for soft-tissue surgery. Even the chapters discussing basic medical visualization techniques, such as surface and direct volume rendering, deserved a careful revision.

Among others, GPU-based techniques play a more prominent role now. GPU-based rendering enables a huge step in improving image quality without compromising performance. We discuss how these improvements are employed, e.g., in virtual endoscopy—another chapter that could be improved by taking advantage of many new and refined techniques.

The increasing size and complexity of medical image data motivated the development of visualization techniques that radically differ from the classical surface and volume rendering techniques. To convey the complex information of medical flow data along with the relevant anatomy, for example, benefits from illustrative techniques that render the anatomy sparsely. Thus, illustrative rendering plays a more prominent role in this second edition discussing the extraction of various features from medical volume data and related meshes as a basis for rendering.

A second radically new class of visualization techniques are map-based techniques. While some isolated techniques, such as stretched curved planar reformations of vascular structures, have been introduced more than a decade ago, we can now discuss this topic in a more general fashion in a separate chapter. DTI was rather new when the first edition was prepared. It is meanwhile a mature technique that is discussed in a wider scope as one out of several techniques to understand brain connectivity.

Medical education in anatomy, interventional radiology and surgery remains an important use case of visual computing. One comprehensive chapter is dedicated to such applications with a focus on recent trends, such as web-based training platforms, and (automatic) skills assessment.

Companion Website

Visit this book’s companion website for this work: http://medvisbook.com/

Author Biography

PROF. DR.-ING. BERNHARD PREIM was born in 1969 in Magdeburg, Germany. He received the diploma in computer science in 1994 (minor in mathematics) and a Ph.D. in 1998 from the Otto-von-Guericke University of Magdeburg (Ph.D. thesis Interactive Illustrations and Animations for the Exploration of Spatial Relations). In 1999 he finished work on a German textbook on Human Computer Interaction which appeared at Springer. He then moved to Bremen where he joined the staff of MeVis (Center for Medical Diagnosis and Visualization Systems, Bremen). In close collaboration with radiologists and surgeons he directed the work on computer-aided planning in liver surgery focusing on virtual resection, automatic resection proposals, visualization of vascular structures, and the integration of measurements in 3D visualizations. This work was largely influenced by Prof. Heinz-Otto Peitgen, the founder and director of MEVIS. In June 2002 Bernhard Preim received the post-doctoral lecture qualification for computer science from the University of Bremen. Since Mars 2003 he is full professor for Visualization at the computer science department at the Otto-von-Guericke-University of Magdeburg, heading a research group which is focussed on medical visualization and applications in surgical education and surgery planning. The focus of this research is illustrative medical visualization, visual exploration of blood flow, virtual endoscopy, and in particular surgery in the ear, nose, throat region. These developments are summarized in a textbook Visualization in Medicine (Co-author Dirk Bartz). His continuous interest in HCI lead to another textbook Interaktive Systeme (Co-author: R. Dachselt) (Springer, 2010). His regular teaching activities include Medical Visualization, Computer-Assisted Diagnosis and Treatment as well as the introductory courses on Visualization and Interactive Systems.

Bernhard Preim founded the working group Medical Visualization in the German Society for Computer Science in 2003 and acted as speaker until 2012. He is also a long-term member of CURAC, the German society for computer-assisted surgery, where he became board member in 2007, and vicepresident in 2009. He was Co-Chair and Co-Organizer of the first and second Eurographics Workshop on Visual Computing in Biology and Medicine (VCBM, together with Charl Botha) and is now member of the steering committee of that workshop. He is the chair of the scientific advisory board of ICCAS (International Competence Center on Computer-Assisted Surgery, since 2010), member of the advisory boards of Fraunhofer Heinrich-Hertz-Institute, Berlin and the Institute for Innovative Surgical Training Technologies (ISTT), Leipzig. He is also regularly a Visiting Professor at the University of Bremen where he closely collaborates with Fraunhofer MEVIS. At the University of Magdeburg, Bernhard Preim is member of the Board (since 2008). Bernhard Preim is married with the radiologist Uta Preim (Medical Doctor), born Hahn and has two children.

DR. CHARL P. BOTHA graduated from the University of Stellenbosch, South Africa, in 1997 with a degree in electronics engineering, followed by an M.Sc. in digital signal processing, in 1999, and finally a Ph.D. in medical visualization from the Delft University of Technology (TU Delft) in the Netherlands, under the supervision of Frits Post, one of the pioneers of scientific visualization in Europe.

After completing his Ph.D., he was appointed (2006) and soon after tenured (2007) as an assistant professor of Visualization at the TU Delft, where he started and headed the medical visualization lab. He also had an appointment at LKEB, the medical image processing section of the Department of Radiology at the Leiden University Medical Center (LUMC), in order to cultivate and expand the fruitful research collaboration between the technical university and the academic hospital.

His research focused on surgical planning and guidance, and visual analysis for medical research. He has published on, among other topics, anatomical modeling, virtual colonoscopy, shoulder replacement, and diffusion tensor imaging. Together with Bernhard Preim he initiated the Eurographics Workshop series on Visual Computing for Biology and Medicine, acted as co-chair in 2008 and 2010, and served as editor together with Prof. Preim of the Computers and Graphics special issue on VCBM.

Prior to his Ph.D. he worked in industry designing embedded image processing systems and algorithms for two different companies. Shortly after the Ph.D., he co-founded Treparel Information Solutions, a company specializing in data mining, and he acts as science advisor to Clinical Graphics, a spin-off company founded by an ex-Ph.D. student to commercialize surgical planning research results. He recently also decided to make the move back into industry full-time, where he has started a company that focuses on bringing computer science, imaging, and visualization research into real-world practice. He remains actively involved with the medical visualization community through the MedVis.org website and its related resources. Charl is married to Stella Botha-Scheepers, MD, Ph.D., a rheumatologist and internist, with whom he has two children.

Chapter 1

Introduction

Abstract

The introductory chapter sets the stage for the whole book by characterizing medical visualization as a speciality of scientific visualization and by briefly discussing the origin and history of medical visualization. Selected examples of clinical applications in diagnosis, treatment planning, intraoperative support, and medical training are discussed. Medical image data, models of surgical instruments and implants, and data from biomedical simulations are described as input for medical visualization. The necessity to carefully combine different views, in particular 2D and 3D views, is discussed. Recent progress in medical image acquisition is considered as a major driving force for a number of recent developments. The introduction provides many links to sources for further information, such as journals, conferences, and blogs and contains an explanation of the outline of the book.

Keywords

Slice-based visualization; 3D visualization; Volume rendering; Biomedical simulation

Visualization refers to the use of computer graphics techniques to create interactive visual representations of data, with the goal of amplifying human cognition. When visualization is applied to medical data, it is called visualization in medicine, or medical visualization for short.

Most medical data has an inherent spatial embedding. For this reason, medical visualization is seen as a special area of scientific visualization. The start of scientific visualization as a research field is considered by many to be the publication of the 1987 report of the NSF on Visualization in Scientific Computing [McCormick et al., 1987]. However, literature reveals instances of medical visualization, following the definition we have set in the first paragraph, as far back as the 1960s.

In an early radiotherapy planning example, patient contours were acquired from line drawings with a mechanical digitizer, and then shown on an oscilloscope display combined with calculated isodose distributions, using a computer especially designed for this purpose [Cox et al., 1966, Holmes, 1970]. Already then this system was put into clinical use. Sunguroff and Greenberg [1978] demonstrated the extraction and visualization of smooth 3D surfaces from CT data. By the end of the 1970s, McShan et al. [1979] had demonstrated the use of 3D graphics for radiotherapy planning. In the early 1980s, 3D visualization was being used clinically for the computer-based preoperative planning of craniofacial surgery [Vannier et al., 1983b].

On the one hand, the long tradition of scientists that illustrate their work by carefully crafted graphics laid the foundation for both scientific and medical visualization. Anatomical illustration, starting with da Vinci’s work, is a prominent example. On the other hand, medical visualization is based on computer graphics that provide algorithms for the efficient rendering of data, with additional influences coming from the world of image processing and medical image analysis.

1.1 Visualization in Medicine as a Speciality of Scientific Visualization

Scientific visualization deals primarily with the visualization, exploration, and analysis of datasets arising from measurements or simulation of real world phenomena. The investigation of air flow around planes and cars is a well-known example. The underlying datasets of scientific visualizations are often very large, which makes it necessary to consider the efficiency and hence the time and space complexity of algorithms. Important goals and research scenarios of scientific visualization are:

to explore data (undirected search without a specific hypothesis),

to test a hypothesis based on measurements or simulations and their visualization, and

the presentation of results.

There are many relevant examples in medical visualization that address these general visualization goals. Whether or not a patient is suffering from a certain disease is a hypothesis to be tested through clinical investigations and medical imaging. If a physician cannot sufficiently assess a disease based on the symptoms described by the patient and by clinical examinations, radiological image data might be acquired without a specific hypothesis. Computer support, in particular image processing, quantitative image analysis, and visualization, may improve the radiologist’s diagnosis.

Finally, if a radiologist has performed a diagnosis, specifying the stage and severity of a disease, certain visualizations are generated to present the diagnosis to the referring physician. Such visualization might include measurements, e.g., the extent of a pathological structure, and annotations, e.g., encircled regions or arrows, to enhance their interpretation. The ultimate goal of such visualizations and the attached report is to support treatment decisions. The presentation goal is also relevant for medical visualizations. Visualizations are generated to be discussed among colleagues, e.g., in a tumor board meeting, to employ them for educational purposes or as part of a publication. Figure 1.1 shows images which have been generated for surgical planning in ear, nose, and throat surgery.

Figure 1.1 Left: A 3D visualization of a neck tumor close to the larynx helps to decide on the resection strategy. A possible larynx infiltration is essential to decide whether the larynx can be preserved. Right: Virtual endoscopy of the paranasal sinus for preparing an endoscopic intervention aiming at polyp removal. With current graphics hardware, realistic rendering may be performed in real time. (Courtesy of Dornheim Medical Images and Christoph Kubisch, University of Magdeburg)

There are several lessons from scientific visualization literature that are inspiring for the design of medical visualization systems. The most important is to consider visualization as a process directed at the understanding of data. The purpose of visualization is insight, not pictures, as McCormick et al. [1987] state in their field-defining report on scientific visualization. Thus, it is essential to understand what kind of insight particular users want to achieve. For medical visualization systems, an in-depth understanding of diagnostic processes, therapeutic decisions, and of intraoperative information needs, is indispensable to provide dedicate computer support. It is also essential to consider organizational and technical constraints, such as sterility and space restrictions in an operating room.

Another consequence is that interaction plays a crucial role in the design of medical visualization systems. Interaction facilities should support the user in navigating within the data, in selecting relevant portions during exploration, in comparing data from different regions or different datasets, in the adjustment and fine-tuning of visualization parameters that define after all the optical properties observable by a human. The whole exploration process should support the interpretation and classification of the data. Examples for this classification in the medical domain are statements such as The patient suffers from a certain disease in a particular stage, The patient can be treated by a certain intervention. A particular surgical strategy was selected. Medical visualization is primarily based on 3D volume data. Our discussion of interaction facilities therefore has a focus on 3D interaction techniques that enable the flexible and efficient exploration of 3D data.

Regarding scientific and medical visualization as an analysis process leads to the conclusion that image generation and visual exploration are not the only way to get insight. Equally important are tools to analyze the data, for example, to characterize the distribution of numerical values in certain regions of the data. Radiological workstations and therapy planning software systems therefore integrate functionality to derive quantitative information concerning the underlying data.

One important aspect that we should keep always in mind are the limitations of the data. These limitations define conditions for interpretation and analysis. Specific structures (i.e., tumors) may not show up at their full size. Other structures are so small, that their analysis might lead to a high error rate, or is highly subjective. Being aware of such limitations is therefore an important key to the successful application of the methods.

1.2 Computerized Medical Imaging

Medical visualization deals with the analysis, visualization, and exploration of medical image data. The main application areas are:

Diagnosis. The diagnosis of radiological data benefits from interactive 2D and 3D visualizations. In particular, if the situation of a particular patient is very unusual (complex fractures, defective positions), 3D visualizations are useful to get an overview of the morphology. More and more, functional and dynamic image data are employed to assess effects, such as blood perfusion or contrast agent enhancement, and metabolism. Various measures are derived from these image data. Appropriate visualizations depict the spatial correlation between these measurements.

Treatment planning. Interactive 3D visualizations of the relevant anatomical and pathological structures may enhance the planning of surgical interventions, radiation treatment, and minimally-invasive interventions. The spatial relations between pathological lesions and life-critical structures at risk may be evaluated better with 3D visualizations. Starting with early work on craniofacial surgery planning [Vannier et al., 1985], the visualization of anatomical structures has been steadily improved due to the progress in image acquisition, graphics and computing hardware, and better rendering. Visualizations may also include information which is not present in radiological data, such as the simulated dose distribution for radiation treatment planning and simulated territories of vascular supply. Treatment planning systems have found their way to many applications, for instance in orthopedic surgery, neurosurgery, abdominal surgery, and craniofacial surgery.

Intraoperative support. Medical visualization based on 3D data is finding increasing application in the operating room. Preoperatively acquired images and intraoperative images are integrated to provide support during an intervention. Flexible and smart displays are needed for such applications (see Fig. 1.4 and Hansen [2012]).

Documentation. Reporting and other documentation tasks benefit from the incorporation of representative visualizations. These visualizations are often annotated with labels and measurements to provide the necessary information to interpret the images. Quantitative analysis of image features, such as tumor extent, may help to fill data necessary for documentation.

Educational purposes. Visualization techniques are the core of anatomy and surgery education systems. As an example, the VOXELMAN, an advanced anatomy education system, combines high-quality surface and volume rendering with 3D interaction facilities and a knowledge base to support anatomy education [Höhne et al., 2003]. More recently surgical simulators were developed on top of these 3D renderings. They support the acquisition and rehearsal of specific skills using tactile input devices and appropriate models of tissue deformation (see Fig. 1.2) More and more, surgical training is performed in special institutions that employ physical and virtual models (see Fig. 1.6).

Medical research. While some kinds of medical image data, e.g., 4D measured blood flow, are (still) too complex for regular clinical use, they are crucial for medical research. In research settings, time is not that strongly limited. Flexible exploration of the data is more important than strict guidance along a workflow. Moreover, in new kinds of applications, there is no defined clinical workflow, (e.g., in studying biomechanical parameters, see Fig. 1.3).

Figure 1.2 Left: 3D visualizations of the dental anatomy as a basis for training drilling procedures. Right: Tactile input devices are employed to provide an experience that is similar to real treatment. With the VOXELMAN dental simulator, it is also possible to automatically assess the skills of the trainee. (Courtesy Institute for Mathematics and Computer Science, University Hospital Hamburg-Eppendorf)

Figure 1.3 Left: The patient’s shoulder anatomy is reconstructed from CT data and shown along with a joint implant and the simulated range of motion after treatment. Right: The range of motion corresponding to the current implant position is compared with the pretreatment range of motion. The comparative visualization highlights the sign and the extent of the changes. (Courtesy of Peter Krekel, Clinical Graphics)

The computer support described above is not intended to replace medical doctors. Instead, physicians should be supported and assisted to perform their tasks more efficiently and/or with increased quality.

Medical Image Data The data, on which medical visualization methods and applications are based, are acquired with scanning devices, such as computed tomography (CT) and magnetic resonance imaging (MRI). These devices have experienced an enormous development in the last 20 years. Although other imaging modalities, such as 3D ultrasound, positron emission tomography (PET), and imaging techniques from nuclear medicine are available, CT and MRI dominate due to their high resolution and their good signal-to-noise-ratio. The image resolution has increased considerably, with the introduction of Multislice CT devices in 1998. Also, the acquisition times have decreased—this development contributes to the quality of medical volume data because motion and breathing artifacts are reduced considerably. The acquisition of time-dependent volume data, which depict dynamic processes in the human body, has been improved with respect to spatial and temporal resolution. Today, also intraoperative imaging becomes a common practice to support difficult interventions, for example, in neurosurgery. Moreover, radiology interventions involving catheters, needles, applicators, and stents, crucially depend on frequent or even real-time imaging to control the position of these instruments and monitor treatment. With the improved quality and wide availability of medical volume data, new and better methods to extract information from such data are feasible and needed.

Figure 1.4 Left: Preoperative planning information is provided in the operating room. Right: With appropriate image registration, essential planning data may be overlaid on current intraoperative images. In the specific example, hepatic vasculature is shown in an illustrative style on top of the liver surface. (Courtesy of Christian Hansen, Fraunhofer MEVIS Bremen)

MRI data experienced a similar development. With improved motion correction and artifact reduction techniques, image quality increased strongly. High-field MRI, such as 7 or even 9.4 Tesla scanners, are expensive and rather rare research installations, but they enable the investigation of future routine possibilities. Figure 1.5 shows images of neurovascular structures that benefit from the high signal-to-noise-ratio of a 7 Tesla MRI scanner (compared to similar images acquired with a 3 Tesla scanner).

Figure 1.5 A maximum-intensity projection of MRI data with a protocol that emphasizes vascular structures. Left: Data are acquired with a 3 Tesla scanner. Right: Data of the same patient acquired with a 7 Tesla scanner and similar settings for the visualization. The data acquired with the 7 Tesla scanner exhibit a better signal-to-noise-ratio but include also venous structures which is often not desired. (Courtesy of Daniel Stucht, Biomedical Magnetic Resonance Group, University of Magdeburg)

Figure 1.6 The surgical planning unit supports the immediate preparation as well as the training of a surgical intervention. High-resolution displays enable a precise rehearsal of preoperatively acquired data. (Courtesy of ICCAS Leipzig and KARL STORZ GmbH & Co. KG Tuttlingen)

Today, a radiologist uses software instead of conventional lightboxes and films to establish a diagnosis. The development of monitors with a sufficient resolution in terms of gray values and spatial resolution was an essential prerequisite for the clinical application of image analysis and visualization techniques. Contrast and brightness may be adjusted with digital image data. This often allows the interpretation of images in a convenient manner even if the data acquisition process was not optimal. More convenient handling, such as touch-based interaction, is further increasing widespread acceptance.

With the increased resolution of the image data, reliable measurements can be derived. For instance, cross-sectional areas and volumes of certain structures can be determined with a reasonable amount of certainty. Measurements of cross-sectional areas are valuable in the diagnosis of vascular diseases (detection of stenosis and aneurysms). Volume measurements of pathological structures are of high relevance to assess the success of a therapy. However, the quality of these measurements depends heavily on the quality of the image data. Specific artifacts (i.e., flow artifacts in MR angiography) may reduce the accuracy significantly.

Note that image analysis and visualization may provide comprehensible views of the data, but the results strongly depend on the original data. Physicians tend to overestimate what can be achieved by processing data with sophisticated algorithms. It is important to cultivate realistic expectations in users. If physicians complain about the results of medical visualization, the problem is often due to deficiencies in the image acquisition process. Structures with a 2 mm diameter cannot be reliably displayed with data that exhibits a 2 mm slice thickness, for example. It is essential that the requirements are stated precisely and that the scanning parameters of the image acquisition are adapted to these requirements.

The increased resolution and improved quality of medical image data also has a tremendous effect on therapy planning. With high-quality data, smaller structures—for example blood vessels and nerves, whose locations are often crucial in the treatment—can be reliably detected. In some cases, this can lead to a better decision of whether or not a particular disease can be successfully treated through surgery, for example whether or not a malignant tumor can be removed entirely. Still, too often such decisions have to be made intraoperatively. In so-called explorative resections, the body is opened and surgery starts to expose the relevant structure to determine whether or not the intervention is feasible. If a resection needs to be cancelled, the patient has been needlessly subjected to a potentially risky intervention. Visualization and computer support for treatment planning aim at reducing such unfavorable situations.

Improvements in Software Support Within the last years, the number of toolkits that directly support medical image analysis and visualization applications has increased significantly. Also the quality of these toolkits has improved, in terms of robustness, functionality, and performance. Among these toolkits are open source software systems, such as DeVIDE (http://graphics.tudelft.nl/Projects/DeVIDE) and Voreen (http://www.voreen.org/), and freely available closed-source systems such as MeVisLab (http://www.mevislab.de/), that enable, e.g., students and academic researchers to make rapid progress and focus on the problems of specific application areas [Botha and Post, 2008, Meyer-Spradow et al., 2009, Ritter et al., 2011].

1.3 2D and 3D Visualizations

Medical imaging started with X-ray imaging at the end of the 19th century. Since that time, diagnosis has been performed by inspecting X-ray films, or more recently, digital X-ray images. With the advent of computed tomography, many slices showing X-ray absorption in a particular region of the body have to be inspected. Slice-by-slice inspection of medical volume data is still common practice. Despite all the efforts to accelerate volume rendering, to employ high-quality reconstruction filters and to ease the adjustment of the necessary parameters, the inspection of 2D slices in radiology is still dominant. A typical explanation of this phenomenon is the assumed ability of a radiologist to mentally fuse the 2D slices in a 3D representation. This ability, however, is not generally accepted and is disputed even between radiologists. In particular, in the case of complex anatomical structures, such as the ear, and in the case of severe fractures or pathological abnormalities, a pure slice-based analysis is probably not adequate [Rodt et al., 2002].

The dominant use of slice data is often attributed to radiological tradition. Well-established techniques are preferred despite obvious disadvantages compared to more recent techniques. However, a thorough analysis of radiological workflows reveals that there are still real benefits of using slice-by-slice inspection. In 2D views of the slices, each and every voxel can be seen and selected (for example to inquire the density value). 2D slice views support precise exploration and analysis of the data. Therefore, radiologists are also legally obliged to inspect every slice. Volume rendering or other 3D visualization, on the other hand, provide an overview. Radiologists use such overviews, for example, if very unfamiliar spatial relations occur, for example, to assess complex fractures. While radiologists rarely rely on 3D visualizations, physicians who carry out interventions (radiation therapy, surgery) strongly benefit from interactive and dynamic 3D visualizations. On the one hand, they do not have the radiological training to mentally imagine complex structures based on a stack of cross-sectional views. On the other hand, they have to understand the 3D spatial relations better than radiologists. While radiologists only describe the data the surgeon actually intervenes in the spatial relations with all the consequences this might have.

Integration of 2D and 3D Visualizations In summary, 2D and 3D visualization techniques are needed. They should be considered independent presentations of radiology data, but should be connected closely, e.g., by synchronizing interaction facilities and display parameters, such as colors. While 3D techniques often provide a more comprehensible overall picture, 2D slice-oriented techniques typically allow a more accurate examination, and hence processing. The integration of slices or parts thereof in a 3D visualization may provide a reference to better understand the 3D spatial relations [Rodt et al., 2002]. An example of a simultaneous employment of 2D and 3D visualizations is shown in Figure 1.7.

Figure 1.7 Left: A lung nodule was segmented and is emphasized as overlay in a slice-based visualization. The quality of the segmentation and the relation between the tumor and the surroundings in this slice are clearly visible. Right: The same lung nodule is shown in a 3D rendering which provides an overview of the nodule’s shape and its location in relation to the surrounding bronchial tree. (Courtesy of Jan-Martin Kunigk, Fraunhofer MEVIS Bremen)

1.4 Further Information

This book hopefully provides the reader with much useful information. However, at least for the in-depth practical or research problems, further information is certainly essential. We provide a paragraph Further Reading at the end of each chapter pointing to relevant publications not discussed within the chapter. The selection of these publications is based on relevance, quality, and availability.

To obtain information that is more recent than this book, we shall give some general hints. The leading visualization conferences IEEE Visualization, EuroVis and more recently IEEE Pacific Vis usually contain one or two Medical Visualization sessions. The journals IEEE Transactions on Visualization and Graphics and Computer Graphics Forum also contain high-quality research results in medical visualization. Medical imaging conferences, such as MICCAI, CARS, and SPIE Medical imaging have a focus on image analysis and medical applications but not on medical visualization algorithms and technology. Occasionally, intraoperative treatment support is covered there.

In the last decade, a small but coherent medical visualization community arose. Its most notable achievement is the Eurographics workshop series Visual Computing in Biology and Medicine¹ that was initiated in 2008. Even before that, two prizes were initiated that are awarded bi-annually. The EUROGRAPHICS Medical price was initiated in 2003 and is now named the Dirk Bartz Medical Prize to honor the co-author of the first edition of this book. The Karl-Heinz Höhne Award for Medical Visualization, honoring the pioneer of medical visualization, was initiated in 2004. Both competitions attract a reasonable spectrum of high-quality medical visualization research that is available online and worth looking at. Finally, we recommend the MedVis blog² that provides event reports, links to recent papers and videos. This blog will be used to inform our readers of new research related to the book, and to classify such new research according to the book’s structure.

Finally, datasets are essential for evaluating medical visualization algorithms. There are various repositories where data may be downloaded via the internet, e.g., http://www.osirix-viewer.com/datasets/ and http://www.volvis.org/. The Osirix-repository is more recently updated and contains even combined PET/CT datasets.

1.5 Organization

A book on medical visualization may be structured primarily according to medical disciplines and tasks, e.g., diagnostic and treatment processes or according to analysis, visualization and interaction techniques. We chose an organization primarily guided by techniques, since most techniques are applicable to a large variety of tasks in medicine. Selected application areas are discussed within several chapters to discuss how the techniques contribute to these specific problems. As an example, we describe the peculiarities of cardiovascular imaging in the medical imaging chapter. Later we discuss, image analysis techniques, such as segmentation and registration, for these specific image data. Based on this discussion, we explain how advanced rendering techniques may be applied to reveal important features of the morphology of cardiac vessels and in further chapters, we discuss how the perfusion in such vessels is measured, analyzed, and visualized and how blood flow is investigated and interpreted as a further essential information for diagnosis diseases of the cardiovascular system. In a similar way, the diagnosis and treatment of tumor diseases plays an essential role in many chapters. The selection of these application areas is not only based by didactic criteria, but primarily by their socio-economic importance, that is by the prevalence and severity of diseases.

This book is structured into five parts.

Part I starts with an introduction into the characteristics of discrete data organized in (uniform) Cartesian grid datasets with scalar values, which is the typical structure of medical image data. In Chapter 2 we introduce the imaging modalities with a focus on computed tomography (CT) and magnetic resonance imaging (MRI) and an overview on other modalities, such as PET, SPECT, and Ultrasound. Chapter 3 deals with the clinical use of medical image data in radiology, radiation treatment planning, and surgery. The software used for the analysis of medical volume data must be carefully integrated in the information processing environment in hospitals and dedicated to usage scenarios, such as patient consult, tumor board discussions, diagnosis, treatment planning, treatment monitoring, and documentation.

This chapter is followed by an overview of medical image analysis (Chap. 4). It illustrates selected image analysis tasks and results. Image segmentation, the identification, and delineation of relevant structures is the focus of this chapter since visualization and many interaction techniques benefit from image segmentation.

We continue with an introduction to human computer interaction (HCI) focusing on the analysis of tasks in a user-centered way, on prototyping solutions and on 3D interaction including a discussion of 3D input devices (Chap. 5). The recent trend toward mobile computing and gesture-based interaction is essential for medical applications, e.g., for bedside use of medical image data. Thus, we carefully discuss the basics and medical applications of this technology.

Part II is devoted to the visualization of medical volume data and to basic interactions with them. Hardware and software aspects, quality, and speed of algorithms are discussed. Volume data can be visualized by directly projecting the data to the screen (direct volume rendering, DVR) or by generating an intermediate representation, which is subsequently rendered (indirect volume rendering). Chapter 6 is devoted to surface-based visualization. Isosurfaces are based on an isovalue selected by the user and display the surface that connects all elements of a volume dataset where this isovalue occurs. Chapter 7 provides an introduction to direct volume rendering including different rendering pipelines and compositing techniques. Chapter 8 describes advanced volume rendering techniques, in particular volume illumination and other techniques that enable an improved shape and depth perception.

After the introduction of volume visualization techniques, we discuss volume interaction (Chap. 9). This includes advanced transfer function design for volume rendering. Without dedicated support, users have to experiment with many possible transfer function settings before an appropriate specification is found. We also discuss clipping and virtual resection. Clipping, virtual resection, and transfer function design are often combined to specify which parts of the data should be displayed.

As another area that deals with the interactive use of medical volume data, we consider labeling and measurement techniques in Chapter 10. Labels and measurements are special kinds of annotations that enhance medical visualizations for diagnosis and particularly for documentation. The integration of these components raises issues of visual design, e.g., appropriate use of layout strategies, color, fonts, and line styles to provide a clear representation of medical image data and related annotations. The qualitative analysis of spatial relations is added through measurements that may directly support treatment decisions. The size and extent of a tumor strongly influences applicable therapies. The angle between bony structures may influence whether the anatomy is regarded as normal or whether treatment is necessary. Interactive measurements and automatic measurements which employ segmentation information are covered.

Part III Advanced visualization techniques starts with the visualization of anatomical tree structures, such as vascular structures (Chap. 11). We describe different methods that produce comprehensible visualizations at different levels of detail and accuracy. While this chapter was restricted to surface-based visualizations in the first edition, we shortened their treatment and provide instead more detail on volume rendering techniques, tailored, e.g., for the diagnosis of vascular diseases.

In Chapter 12, illustrative rendering and emphasis techniques are described. These techniques are essential for medical education and for therapy planning. One scenario is that the user selects an object via its name from a list and this object will be highlighted in the related visualization. In general, emphasis is difficult to carry out because most objects are at least partially occluded.

Chapter 13 is devoted to virtual endoscopy. Virtual endoscopy is inspired by real endoscopic procedures that are carried out for diagnosis (e.g., detection of polyps in the colon) or as minimally-invasive intervention. In real endoscopy, a small camera is inserted in the human body through small incisions or anatomical openings (e.g., the colon) and it is moved to inspect vascular structures or structures filled with air. In virtual endoscopy, similar images are produced through 3D visualization, on the basis of medical volume data. Virtual endoscopy has a great potential for surgery training and treatment planning, as well as for diagnosis and intraoperative navigation, because it has less restrictions than real endoscopy (the virtual camera can go everywhere) and is more comfortable for the patient. Visualization and navigation techniques in the virtual human are the issues which are discussed in this chapter.

In the next chapter, we discuss Projection-based Medical Visualization Techniques (Chap. 14). This chapter is another completely new chapter compared to the first edition and it is motivated by a variety of successful applications that incorporate map-like projections. Vessel flattening, brain and colon flattening, tumor maps, Bull’s Eye plots in cardiology are just some of the examples, where 3D geometries are transformed to map projections to give an overview. Such projections from a higher to a lower dimensional space are related with distortions and a loss of information. We carefully discuss different strategies and their limitations.

Part IV discusses the visualization of high-dimensional medical image data. Thus, in this part we consider time-dependent data, and 3D vector and tensor data.

A special variation of MRI is Diffusion Tensor Imaging (DTI). With this modality, the inhomogeneity of the direction of (water) diffusion can be non-invasively determined. Strongly directed diffusion occurs for example in the whiter matter of the human brain, and thus indicates the direction and location of fiber tracks. This information is highly relevant, in particular in neuroradiology and neurosurgery. The analysis and visualization of DTI data poses many challenges, which are discussed in Chapter 15. In contrast to the first edition, we extend the scope here and consider DTI just one method to explore brain connectivity.

Chapter 16 describes techniques to explore and analyze perfusion data, a special instance of time-dependent volume data. These data have a great potential for medical diagnosis, e.g., for the assessment of tumors, where malignant tumors exhibit a stronger vascularization than benign tumors. This effect cannot be observed in static images. Techniques for the efficient visualization and analysis of such data are important, because the huge amount of dynamic volume data cannot be evaluated without dedicated software support.

Part V Treatment planning, guidance and training covers specific application areas and case studies related to ear-, nose-, and throat surgery. In Chapter 17, we discuss general requirements and solution strategies for computer-assisted surgery. Chapter 18 is dedicated to intraoperative visualization, image-guided surgery, and augmented reality in surgery. Here we will discuss how medical image data is integrated with an intervention itself. We discuss simple techniques that just provide access to medical image data intraoperatively and more advanced techniques combining pre and intraoperative imaging data similar to a car navigation system. The constraints of intraoperative use, interaction techniques appropriate in these settings and limitations with respect to setup times and accuracy will be carefully considered. We go on and discuss the visual exploration of medical flow data, resulting from measurements or simulations (Chap. 19) with a focus on blood flow data, a very active research area in recent years that also is relevant for medical treatment planning, e.g., neurovascular intervention and surgery.

In Chapter 20, we discuss image analysis and visualization for ENT surgery planning (neck dissection, endoscopic sinus surgery). Task analysis and evaluation are carefully described to provide orientation for the development of similar systems. In Chapter 21, the use of medical visualization techniques for educational purposes, in particular for anatomy and surgery education is discussed. Besides describing application areas, this chapter introduces some new techniques, such as labeling and animation of medical volume data, collision detection, and soft tissue deformation for surgical simulation.

---

¹http://vcbm.org/.

²http://medvis.org/.

Part I: Acquisition, Analysis, and Interpretation of Medical Volume Data

Outline

Part I Acquisition, Analysis, and Interpretation of Medical Volume Data

Chapter 2 Acquisition of Medical Image Data

Chapter 3 An Introduction to Medical Visualization in Clinical Practice

Chapter 4 Image Analysis for Medical Visualization

Chapter 5 Human-Computer Interaction for Medical Visualization

Part I

Acquisition, Analysis, and Interpretation of Medical Volume Data

The first part contains introductory chapters for all most topics treated in this book.

In Chapter 2, we introduce the imaging modalities with a focus on computed tomography (CT) and magnetic resonance imaging (MRI) and an overview on other modalities, such as PET, SPECT, and Ultrasound. We aim at a balance between basic aspects of medical imaging and recent developments, such as Dual Source CT, ultra high field MRI, and integrated PET/CT devices. We also discuss intraoperative imaging, a topic that becomes increasingly important to support interventions and minimally-invasive surgery. This chapter should make the reader familiar with the data that serve as input for all kinds of medical visualization.

Chapter 3 deals with the clinical use of medical image data in radiology, radiation treatment planning, and surgery. As background, we describe the conventional reading process of radiologists and later how soft-copy reading may enhance the classic process. The software used for the analysis of medical volume data must be carefully integrated in the information processing environment in hospitals and dedicated to usage scenarios such as patient consult, tumor board discussions, diagnosis, treatment planning, treatment monitoring, and documentation. We use this chapter also to introduce some 3D visualization techniques, not to discuss how they work, but to discuss their value as an addition to slice-based viewing.

This chapter is followed by an overview of medical image analysis (Chap. 4). It illustrates selected image analysis tasks and results starting with noise reduction and other preprocessing techniques. Image segmentation, the identification and delineation of relevant structures is the focus of this chapter since visualization and many interaction techniques benefit from image segmentation. We also discuss registration, the process that aligns different datasets in one coordinate system. Registration is needed to integrate image data from different modalities, or different points in time, e.g., pre- and intraoperative images. Validation is a crucial aspect in image analysis.

We continue with an introduction to human computer interaction (HCI) focusing on the analysis of tasks in a user-centered way, on prototyping solutions, and on 3D interaction including a discussion of 3D input devices (Chap. 5). Scenario descriptions and workflow analysis represent the core of task analysis. Analysis of users is also discussed with a focus on Personas. The recent trend toward mobile computing and gesture-based interaction is essential for medical applications, e.g., for bedside use of medical image data. Thus, we carefully discuss the basics and medical applications of this technology.

Chapter 2

Acquisition of Medical Image Data

Abstract

This chapter provides an introduction into the most essential medical image acquisition processes with a focus on tomographic 3D image data. As a prerequisite sampling processes and interpolation is explained. We briefly discuss major requirements that guide the selection of imaging modalities in practice, e.g., to provide a sufficient spatial resolution for the diagnostic question, to limit the exposure to radiation and to achieve a sufficient image quality in terms of contrast and signal-to-noise ratio (SNR). Readers should understand that trade-offs between these conflicting goals are necessary. Thus, neither optimum spatial resolution nor optimum image quality are relevant goals in clinical practice. In this chapter we focus on tomographic imaging modalities, in particular on CT and MRI data. Typical artifacts that hamper image interpretation, such as inhomogeneity, partial volume effects, and streaking are explained. We consider recent developments, e.g., dual source and dual energy CT and ultra highfield MRI. Hybrid PET/CT and PET/MRI scanners are an exciting development of the last decade. We discuss them as examples for the potential of the complementary use of imaging data and the necessity to fuse the resulting information. Specific examples of diagnostic questions that benefit from certain imaging techniques are given throughout the chapter. Finally, we consider imaging techniques for intraoperative monitoring, such as fluoroscopy and intravascular ultrasound.

Keywords

CT; Signal-to-noise ratio; Artifacts; MRI; Angiography; Hybrid imaging; Ultrasound

2.1 Introduction

Medical image data are acquired for different purposes, such as diagnosis, therapy planning, intraoperative navigation, post-operative monitoring, and biomedical research. Before we start with the description of medical imaging modalities, we briefly discuss major requirements