[vtk-developers] [Paraview-developers] change to vtkExodusIIReaderPrivate::ResetCache()

[David Thompson]

Hi Pat,

Are you calling Update() (or at least somehow triggering RequestData()  
calls) many times on the reader? The cache does actually work, just  
not well with a mesh split across many files. Since decreasing the  
cache makes your code take longer, perhaps the problem is that more  
calls to NetCDF (and thus to the filesystem since I believe NetCDF  
instructs the OS not to cache) are being made.

There was talk of adding yet another knob to ParaView to control  
caching from there. Perhaps the best solution would be to expose   
cache size in vtkExodusIIReader and provide a property on the ParaView  
server-manager proxy (hidden or in the GUI) allowing control of the  
cache? I don't know of a way to do anything more clever since I don't  
think any of the popular desktop/server OSes provide usable  
information about memory pressure these days.

	David

On Apr 25, 2011, at 14:29 , Crossno, Patricia J wrote:

> I tried the cache with various values.  Reducing the cache size  
> increases the time to completion.  For a size of 128.0, it finishes  
> in about 17 seconds.  For a size of 10.0, it completes in about 20  
> seconds.  For a size of 1.0, it took about 8 minutes and 15 seconds  
> (timing this with my wristwatch).  I decided to rerun with a size of  
> 0.0, and it took about the same amount of time as for a cache size  
> of 1.0 (though is probably due to my timing method).  I’d never been  
> willing to wait over 8 minutes for it to finish before.   
> Unfortunately, I cannot share the data set with you.  However, the  
> size of the mesh is 637492 cells with 206134 points, which I suspect  
> is larger than your sample files.
>
> I am running this in the Visual Studio 2008 debugger on Windows  
> Vista.  I don’t have Valgrind on my machine, but looking at their  
> list of supported platforms, I don’t see Windows (though apparently  
> there is some complicated work around using another package called  
> Wine, which I’m not going to try).  We’ve also run the 0.0 sized  
> cache on Tim’s Mac and it exhibited similar slow behavior, which we  
> also killed after multiple minutes, but before completion.  So this  
> is not exclusively a Windows issue.
>
> Pat
>
> From: Robert Maynard [mailto:robert.maynard at kitware.com]
> Sent: Monday, April 25, 2011 8:02 AM
> To: Crossno, Patricia J
> Cc: vtk-developers at vtk.org; paraview-developers at paraview.org
> Subject: Re: [Paraview-developers] change to  
> vtkExodusIIReaderPrivate::ResetCache()
>
> Hi,
>
> The changes to the ExodusII reader was made to reduce memory  
> overhead in ParaView. If you set the cache to any positive non zero  
> value does the issue go away? Also would you be able to provide a  
> sample data set that replicates this issue, as all my ExodusII  
> sample datasets have no issue with the cache size set to zero.
>
>
> On Fri, Apr 22, 2011 at 4:45 PM, Crossno, Patricia J <pjcross at sandia.gov 
> > wrote:
> From a pull from master some weeks ago, the previous version of this  
> method looked like this:
>
> void vtkExodusIIReaderPrivate::ResetCache()
> {
>   this->Cache->Clear();
>   this->Cache->SetCacheCapacity( 0. ); // FIXME: Perhaps Cache  
> should have a Reset and a Clear method?
>   this->Cache->SetCacheCapacity( 128. ); // FIXME: Perhaps Cache  
> should have a Reset and a Clear method?
>   this->ClearConnectivityCaches();
> }
>
> I just did a pull and the method has been changed to this:
>
> void vtkExodusIIReaderPrivate::ResetCache()
> {
>   this->Cache->Clear();
>   this->Cache->SetCacheCapacity( 0. ); // FIXME: Perhaps Cache  
> should have a Reset and a Clear method?
>   this->ClearConnectivityCaches();
> }
>
> When reading in an exodus file now, the code disappears into the  
> reader and never finishes (maybe it would eventually, but after a  
> long while I kill it).  Adding back in the line to set the cache  
> capacity to 128 resolves this issue.  I expect that the line was  
> removed for a reason.  Did removing the line resolve a problem for  
> someone else?  Is anyone else having difficulty with the exodus  
> reader as a result of this change?  Or is it something specific to  
> my exodus file/runtime environment?
>
> Pat
>
>
> _______________________________________________
> Paraview-developers mailing list
> Paraview-developers at paraview.org
> http://public.kitware.com/mailman/listinfo/paraview-developers
>
>
>
>
> -- 
> Robert Maynard
> _______________________________________________
> Paraview-developers mailing list
> Paraview-developers at paraview.org
> http://public.kitware.com/mailman/listinfo/paraview-developers