ITK Lecture 10 Review & Toolbox Part II

ITK Lecture 10Review & Toolbox Part II Methods in Image Analysis CMU Robotics Institute 16-725 U. Pitt Bioengineering 2630 Spring Term, 2006 Damion Shelton

Some thoughts • I’ve collected a bunch of stuff that is either: • Grandiose preaching from the pulpit • Useful filters/tips/tricks I’ve run across • A lot of what I’ll be talking about I’ve mentioned in passing before; hopefully you have a different perspective after having written a filter

Generic programming revisited • Generic programming may be loosely defined as “programming with concepts” (David Musser) • What concepts have we looked at so far?

Concept of an image • An image is rectilinear container in N-space which holds regular samples of some physical space • Each of these regular samples is called a pixel

Concept of a pixel • A pixel is a sample of data in N-space, and may be represented by a variety of data types depending on the modality used to acquire the data

Concept of an iterator • An iterator is a way to move over an image; it provides method that mimics “sequential” access regardless of the actual access method or dimensionality

Concept of a pipeline • There are two main types of objects in the world, data objects and process objects • Typically, we feed a data object to a process object and get a new data object as a result • A sequentially chain of process objects is called a pipeline

Writing generic code • Successful generic programming means that you “ignore” concerns that would be specific to a particular image • pixel type (i.e. the actual data type) • dimensionality • The first way you do this is with templating

Writing generic code, cont. • But... templating alone doesn’t ensure that your code is generic • Avoid loops that maneuver through dimensionality, instead, loop over an iterator • Use data types (VNL vectors, etc.) to make math easier in N-d

Questions to ask yourself • Am I making tradeoffs between: • Speed of coding and reusability? • Level of generality and execution speed? • Compactness and clarity? • ITK seems to lean towards reusable, generic, and clear code; depending on your needs this may be a criticism or a point in favor of the toolkit

When generic programming fails • As much as we’d like, not all algorithms extend to N-d or to all pixel types • But... don’t assume that things won’t work in higher (or lower) dimensions • I was surprised to discover that code I wrote to do medial axis analysis in 3D worked correctly on a 4D hypersphere

The “other” way of reading images • You may recall that I mentioned there was another way of reading images, besides that presented in class • This method relies on ITK’s “factory” architecture

Image reading: first technique • We know what kind of image we have (from a dialog box, for instance) • Create a reader that can read that type of image • Read the image

Image reading: second technique • We know what kinds of images we might have, but don’t assume that the user knows • Register instances of all of reader types we might need • The readers will let us know if they can read the file and will do so if possible

Reading images with the factory m_ImageReader = ImageFileReaderType::New(); itk::MetaImageIOFactory::RegisterOneFactory(); m_ImageReader->SetFileName(m_Filename ); m_ImageReader->Update(); Here I only register one factory, but there’s no reason I couldn’t have more

Factory advantages • The factory model of image reading is a bit more flexible and requires less advance knowledge • If you have a dialog box, you don’t have to parse the filename to decide what type of reader to create • Less risk of being “wrong” about file types

Factory disadvantages • I find it a bit harder to think of conceptually, and you often know what image formats you’re dealing with anyways

Gallery of useful ITK classes • These are classes I have found that solve particularly common problems that arise in image processing • Don’t re-invent the wheel! • This list is not comprehensive (obviously) • I leave specific documentation of these filters as an EFTR

Padding an image Problem: you need to add extra pixels outside of an image (e.g., prior to running a filter) Solution:PadImageFilter & its derived classes

Cropping an image Problem: trimming image data from the outside edges of an image (the inverse of padding) Solution:CropImageFilter

Rescaling image intensity Problem: you need to translate between two different pixel types, or need to shrink or expand the dynamic range of a particular pixel type Solution:RescaleIntensityImageFilter

Computing image derivatives Problem: you need to compute the derivative at each pixel in an image Solution:DerivativeImageFilter, which is a wrapper for the neighborhood tools discussed last week See also LaplacianImageFilter

Compute the mirror image Problem: you want to mirror an image about a particular axis or axes Solution:FlipImageFilter - you specify flipping on a per-axis basis

Rearrange the axes in an image Problem: the coordinate system of your image isn’t what you want; the x axis should be z, and so on Solution:PermuteAxesImageFilter - you specify which input axis maps to which output axis

Resampling an image Problem: you want to apply an arbitrary coordinate transformation to an image, with the output being a new image Solution:ResampleImageFilter - you control the transform and interpolation technique

Getting a lower dimension image Problem: you have read time-series volume data as a single 4D image, and want a 3D “slice” of this data (one frame in time), or want a 2D slice of a 3D image, etc. Solution:ExtractImageFilter - you specify the region to extract and the “index” within the parent image of the extraction region

An(other) introduction to VTK • For the remainder of this class I’ll present a brief summary of how VTK works • It’s useful to know what’s going on behind the scenes in myITKgui • It’s very likely that some of you will want to use VTK in stand-alone mode without the FLTK wrapper

Rendering layout of VTK • vtkRenderWindow defines the window that is displayed on your monitor • vtkRenderers are attached to windows and are responsible for converting more abstract primitives (vtkActors) into displayed images

Actors in VTK • The basic “thing” that can be displayed in VTK is an Actor • Mappers convert raw data to Actors • For example: • boundary points in ITK  VTK pointset  VTK point mask filter  VTK polygon data mapper  VTK actor

vtkRenderWindowInteractors • Interactors are objects that work with RenderWindows to pass mouse and keyboard events back and forth • One particularly useful interactor is vtkFlRenderWindowInteractor, which lets you use VTK windows with the FLTK window manager

Widgets • Widgets are more complicated objects that combine some of the functionality of Interactors and Actors • The ImagePlaneWidget is a mouse-controlled object that provides an arbitrary slice through a 3D volume

Program layout • Create a RenderWindow • Create a Renderer • Create a FlRenderWindowInteractor • Load an image • Create 3 image plane widgets, attach them to the interactor • Enter a message loop to run the program

Adding additional “stuff” • It’s easier than you might think to render additional objects along with the image plane widgets (boundary points for instance) • Starting with some sort of object in ITK, you would do the following...

Arbitrary object visualization • Figure out what type of primitive you have (points/lines/etc.) • Create VTK data representing your primitives • Convert this to poly data • Map this to an actor

Data representation in VTK • For geometric data, you may be interested in the following classes: • vtkPoints stores a list of 3D points • vtkUnstructuredGrid stores a collection of arbitrary cells, where a cell is a primitive such as a vertex or line (vtkLine)

Data representation cont. • This may seem a bit convoluted, but in practice it’s pretty simple once you get the hang of it • VTK has a pipeline, similar to that of ITK, so changing the data/mapper/etc. will affect downstream filters but not upstream ones

Cool side effects • My “favorite”1 added bonus of using VTK is the ability to export scenes to files • Since data and rendering are abstracted away from each other, it’s pretty easy to, for example, dump your entire rendering setup to a Pixar Renderman format file 1 Favorite du jour

ITK Lecture 10 Review & Toolbox Part II