|
Notes |
|
|
(0027012)
|
|
David Gobbi
|
|
2011-07-18 10:55
|
|
This is a bad idea. Using a buffer that is the full size of the file will double the amount of memory that VTK uses when reading the whole file.
You should be able to get the same performance boost by calling UpdateWholeExtent() on the reader. |
|
|
|
(0030249)
|
|
Pietro Cerutti
|
|
2013-01-25 12:06
|
|
David,
this is not what I have experienced.
The sample program below reads a directory (containing DICOM files) via vtkImageReader2, using SetDataExtent() and UpdateWholeExtent(), and prints the time taken and the resource usage.
This is the result of loading a directory containing 145 DICOM files (CT scan), for a total of 74 MB.
With my patch:
ImageReader2 : 14.745842 sec
Memory usage : 91.6367 MB
Without my patch:
ImageReader2 : 27.874251 sec
Memory usage : 91.1953 MB
This shows that my patch improves the throughput by ~ 80% and does not affect the memory footprint.
=================================================================================================
#include <cstdio>
#include <sys/types.h>
#include <sys/time.h>
#include <sys/resource.h>
#include <vtkSmartPointer.h>
#include <vtkGlobFileNames.h>
#include <vtkImageReader2.h>
#include <vtkStringArray.h>
#define vtkNew(type, name) vtkSmartPointer<type> name = vtkSmartPointer<type>::New()
int
main(int argc, char **argv)
{
if (argc != 2) {
std::cerr << "Usage: " << argv[0] << " <path>" << std::endl;
return (1);
}
/* time */
struct timeval beg, end, delta;
/* files list */
vtkNew (vtkGlobFileNames, glob);
glob->SetDirectory (argv[1]);
glob->AddFileNames ("*");
vtkIdType nofFiles = glob->GetNumberOfFileNames ();
/* load all files with vtkImageReader2 */
gettimeofday (&beg, NULL);
vtkNew (vtkImageReader2, read);
read->SetFileNames (glob->GetFileNames());
read->SetDataExtent (0, 511, 0, 511, 0, nofFiles);
read->UpdateWholeExtent ();
gettimeofday (&end, NULL);
timersub (&end, &beg, &delta);
std::cout << "ImageReader2 : " << delta.tv_sec << "." << delta.tv_usec << " sec" << std::endl;
/* get current resource footprint */
struct rusage r;
getrusage (RUSAGE_SELF, &r);
std::cout << "Memory usage : " << r.ru_maxrss / 1024. << " MB" << std::endl;
return (0);
} |
|
|
|
(0030251)
|
|
David Gobbi
|
|
2013-01-25 13:40
|
|
The extra memory usage is a temporary and per-file. So if you are reading the image as a whole bunch of slices, you are not going to see much effect.
But let's say someone is loading a single 10GB file (i.e. not a whole bunch of slices, but a single very large file). Then memory usage will temporarily spike at 20GB before dropping down to 10GB.
If you want this patch to be considered, you should add some methods to the class so that people can choose whether or not they want this feature. |
|
|
|
(0030761)
|
|
Pietro Cerutti
|
|
2013-05-14 10:45
|
|
Ok, I figured out that the double amount of memory is due to the fact that the algorithm copies from the allocated buffer into its datastructure.
How would you see this reworked patch? It adds a boolean macro (BufferWholeFile) to enable / disable this feature. In the doxy comments I have made explicit that twice the ram is used, and that this could be of benefit especially over CIFS shares. |
|
|
|
(0030784)
|
|
Pietro Cerutti
|
|
2013-05-16 05:28
|
|
|
I have a revised patch, which more conform to the VTK style. |
|
|
|
(0030899)
|
|
Jean-Christophe Fillion-Robin
|
|
2013-06-05 13:24
|
|
|
|
|
(0031191)
|
|
Dave DeMarle
|
|
2013-07-22 17:46
|
|
|
Please submit the patch via gerrit so that it can be more easily reviewed. |
|
|
|
(0037239)
|
|
Kitware Robot
|
|
2016-08-12 09:55
|
|
Resolving issue as `moved`.
This issue tracker is no longer used. Further discussion of this issue may take place in the current VTK Issues page linked in the banner at the top of this page. |
|
patch-vtkImageReader2.diff (1,369) 2011-07-18 10:35
patch-IO_vtkImageReader2 (2,656) 2013-05-14 10:45