Proposals:Slice contiguous images
ITK uses a contiguous image memory model. All the pixels in an image/volume are in a single contiguous memory block. Many existing medical image analysis systems use a slice contiguous memory model. Here, all the pixels in a slice are in a contiguous memory block but the slices may not be contiguous in memory. A slice contiguous memory model has benefits over a volume contiguous memory model in memory management and cache coherency.
Design
Image virtual address space managed by the PixelContainer
The implementation could be done at the PixelContainer level. You can think of the PixelContainer as providing a 1D contiguous address space for an ND image. The pixel container could then manage the indexing into real memory.
Users should be able to specify maximum number of slices in memory. This would limit the total memory usage and make images and filters that operate on them scalable virtually to any size. Users should also be able to select exactly the range of slices in memory, probably through SetRequestedRegion(). This should be connected to streamed readers/writers.
Which dimensions of an image should be contiguous?
For ND images, we'll need to decide which dimensions are contiguous:
- (N-1)D are contiguous, last dimension sparse across the memory space
- 2D slices are contiguous, (N-2)D dimension sparse across the memory space
Specifying the PixelContainer
At one point in ITK's history, Images were templated over the type of PixelContainer. We could revert back to that design to allow other PixelContainers to be used. Alternatively, we could use inheritance/virtual functions, allowing the existing image to operate on a variety of PixelContainer classes.
Option: Let Iterator be aware of the internal Image Organization
This option will provide a better shot for supporting high computation times. In this option, the iterators will be aware of the internal slice-by-slice memory organization of the image and will manage the final access to the pixel data in memory.
- WithIndex iterators should be easy to adapt to this framework and not API changes.
- PixelWise (not WithIndex) should also require minor modifications and not API changes.
- Neighborhod iterators will be the more challenging ones, but doesn't seem to bea technical reason why they couldn't be converted.
Impact
- Supporting slice contiguous images will require modification to the image iterators and anywhere Image::GetBufferPointer() is called.
- A simplistic implementation may result in a double derefernce of data which could impact performance.
- Using inheritance in the PixelContainer class will most likely result in a virtual function call to dereference a pixel which could impact performance.
Readers and Writers
For completness, this support will make more sense if it is accompanied of modifications in the Readers and Writers that will facilitate to bring isolated slices from images into memory and that will allow to write slice-by-slice into output files.
Image::GetBufferPointer() Usage in Insight Toolkit 3.6.0
Find all "GetBufferPointer", Subfolders, Find Results 1, "Insight Toolkit 3.6.0 Code", "*.c; *.cpp;; *.cxx; *.txx; *.h; *.cmake" Code\Algorithms\itkFFTWComplexConjugateToRealImageFilter.txx(118): Code\Algorithms\itkFFTWComplexConjugateToRealImageFilter.txx(122): Code\Algorithms\itkFFTWRealToComplexConjugateImageFilter.txx(126): Code\Algorithms\itkFFTWRealToComplexConjugateImageFilter.txx(129): Code\Algorithms\itkMattesMutualInformationImageToImageMetric.txx(784): Code\Algorithms\itkMattesMutualInformationImageToImageMetric.txx(907): Code\Algorithms\itkMattesMutualInformationImageToImageMetric.txx(1061): Code\Algorithms\itkMattesMutualInformationImageToImageMetric.txx(1223): Code\Algorithms\itkMattesMutualInformationImageToImageMetric.txx(1254): Code\Algorithms\itkMattesMutualInformationImageToImageMetric.txx(1524): Code\Algorithms\itkRayCastInterpolateImageFunction.txx(1159): Code\Algorithms\itkRayCastInterpolateImageFunction.txx(1163): Code\Algorithms\itkRayCastInterpolateImageFunction.txx(1167): Code\Algorithms\itkRayCastInterpolateImageFunction.txx(1171): Code\Algorithms\itkRayCastInterpolateImageFunction.txx(1192): Code\Algorithms\itkRayCastInterpolateImageFunction.txx(1196): Code\Algorithms\itkRayCastInterpolateImageFunction.txx(1200): Code\Algorithms\itkRayCastInterpolateImageFunction.txx(1204): Code\Algorithms\itkRayCastInterpolateImageFunction.txx(1226): Code\Algorithms\itkRayCastInterpolateImageFunction.txx(1230): Code\Algorithms\itkRayCastInterpolateImageFunction.txx(1234): Code\Algorithms\itkRayCastInterpolateImageFunction.txx(1238): Code\Algorithms\itkSCSLComplexConjugateToRealImageFilter.txx(74): Code\Algorithms\itkSCSLComplexConjugateToRealImageFilter.txx(83): Code\Algorithms\itkSCSLComplexConjugateToRealImageFilter.txx(100): Code\Algorithms\itkSCSLComplexConjugateToRealImageFilter.txx(112): Code\Algorithms\itkSCSLComplexConjugateToRealImageFilter.txx(135): Code\Algorithms\itkSCSLComplexConjugateToRealImageFilter.txx(150): Code\Algorithms\itkSCSLComplexConjugateToRealImageFilter.txx(182): Code\Algorithms\itkSCSLComplexConjugateToRealImageFilter.txx(199): Code\Algorithms\itkSCSLComplexConjugateToRealImageFilter.txx(211): Code\Algorithms\itkSCSLComplexConjugateToRealImageFilter.txx(234): Code\Algorithms\itkSCSLComplexConjugateToRealImageFilter.txx(249): Code\Algorithms\itkSCSLRealToComplexConjugateImageFilter.txx(74): Code\Algorithms\itkSCSLRealToComplexConjugateImageFilter.txx(95): Code\Algorithms\itkSCSLRealToComplexConjugateImageFilter.txx(109): Code\Algorithms\itkSCSLRealToComplexConjugateImageFilter.txx(133): Code\Algorithms\itkSCSLRealToComplexConjugateImageFilter.txx(148): Code\Algorithms\itkSCSLRealToComplexConjugateImageFilter.txx(179): Code\Algorithms\itkSCSLRealToComplexConjugateImageFilter.txx(193): Code\Algorithms\itkSCSLRealToComplexConjugateImageFilter.txx(217): Code\Algorithms\itkSCSLRealToComplexConjugateImageFilter.txx(232): Code\Algorithms\itkVnlFFTComplexConjugateToRealImageFilter.txx(57): Code\Algorithms\itkVnlFFTComplexConjugateToRealImageFilter.txx(69): Code\Algorithms\itkVnlFFTRealToComplexConjugateImageFilter.txx(89): Code\Algorithms\itkVnlFFTRealToComplexConjugateImageFilter.txx(92): Code\Algorithms\itkWatershedSegmenter.txx(533): Code\Algorithms\itkWatershedSegmenter.txx(582): Code\BasicFilters\itkReflectiveImageRegionConstIterator.txx(84): Code\BasicFilters\itkVTKImageExport.txx(374): Code\BasicFilters\itkVTKImageExportBase.cxx(109): Code\BasicFilters\itkVTKImageExportBase.h(99): Code\Common\itkBloxImage.txx(58): Code\Common\itkBSplineDeformableTransform.txx(741): Code\Common\itkBSplineDeformableTransform.txx(917): Code\Common\itkConstNeighborhoodIterator.h(120): Code\Common\itkConstNeighborhoodIterator.txx(360): Code\Common\itkConstNeighborhoodIterator.txx(362): Code\Common\itkConstNeighborhoodIterator.txx(596): Code\Common\itkImage.h(239): Code\Common\itkImage.h(240): Code\Common\itkImage.h(241): Code\Common\itkImage.h(242): Code\Common\itkImageAdaptor.h(232): Code\Common\itkImageAdaptor.h(233): Code\Common\itkImageAdaptor.txx(332): Code\Common\itkImageAdaptor.txx(334): Code\Common\itkImageAdaptor.txx(341): Code\Common\itkImageAdaptor.txx(343): Code\Common\itkImageConstIterator.h(173): Code\Common\itkImageConstIteratorWithIndex.h(230): Code\Common\itkImageConstIteratorWithIndex.txx(66): Code\Common\itkImageConstIteratorWithIndex.txx(81): Code\Common\itkImageConstIteratorWithIndex.txx(144): Code\Common\itkImageConstIteratorWithIndex.txx(229): Code\Common\itkImageContainerInterface.h(77): Code\Common\itkImageDuplicator.txx(79): Code\Common\itkImageRandomConstIteratorWithIndex.txx(110): Code\Common\itkImageRandomNonRepeatingConstIteratorWithIndex.txx(133): Code\Common\itkImageReverseConstIterator.h(177): Code\Common\itkImageReverseConstIterator.h(210): Code\Common\itkImageReverseConstIterator.h(256): Code\Common\itkImportImageContainer.h(86): Code\Common\itkOctree.txx(263): Code\Common\itkSpecialCoordinatesImage.h(248): Code\Common\itkSpecialCoordinatesImage.h(249): Code\Common\itkSpecialCoordinatesImage.h(250): Code\Common\itkSpecialCoordinatesImage.h(251): Code\Common\itkValarrayImageContainer.h(100): Code\Common\itkValarrayImageContainer.h(145): Code\Common\itkValarrayImageContainer.h(147): Code\Common\itkVectorImage.h(250): Code\Common\itkVectorImage.h(251): Code\Common\itkVectorImage.h(252): Code\Common\itkVectorImage.h(253): Code\Common\itkVectorImageNeighborhoodAccessorFunctor.h(70): Code\Common\itkZeroFluxNeumannBoundaryCondition.txx(51): Code\IO\itkImageFileReader.txx(343): Code\IO\itkImageFileReader.txx(395): Code\IO\itkImageFileWriter.txx(283): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(497): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(508): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(573): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(580): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(620): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(631): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(696): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(732): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(776): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(787): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(794): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(803): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(887): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(893): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(951): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(960): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(1047): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(1083): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(1118): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(1210): Code\Review\itkOptMattesMutualInformationImageToImageMetric.txx(1216): Matching lines: 121 Matching files: 38 Total files searched: 2359
Filters
We could consider here at least three types of filters:
- Filters that would directly work with this slice-contiguous memory model. These include some intensity or histogram-based filters for basic operations.
- Filters that would work with this slice-continguous memory model with adaptations. These may include filters that operate on neighbourhoods.
- Filters that definitely won't work with this model, for example because they need all the input to generate the output. This is the case of current RecursiveGaussian filters. This could be check via Concept Checking so these filters could not be instantiated with SliceImages.
Ideally an strategy should be devised that would allow having in memory the region that is currently being processed (current neighborhood). There should be a balance between loading and processing. Loading and processing could even go in separate but synchronized threads, so that, when processing starts the region is already loaded. Ideally this would be transparent to the user.
