MantisBT - ParaView
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0012818ParaView(No Category)public2011-12-20 20:252012-10-29 17:04
ReporterAlan Scott 
Assigned ToUtkarsh Ayachit 
PriorityhighSeverityminorReproducibilityhave not tried
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version3.14 
Target VersionFixed in Version3.98.0 
ProjectSandia
Topic Name
Typecrash
Summary0012818: CTH memory use grows with compositeIndex
DescriptionCTH AMR memory footprint is growing at a rate that appears proportionate to total compositeIndex. We believe that some type of CTH meta data for a whole dataset is being spread around all pvservers.

This is a showstopper for CTH visualizations beyond about 100 million cells (depending on the block to cell count).

This is easy to test using a dataset that I have access to. (This dataset may not be released to Kitware - sorry. If necessary, I should be able to have someone create a releasable dataset.)

How I replicated the problem:
ParaView master. Remote server, 128 cores (processes). Linux client.
I then did soft links into a dataset, changing the number of files that were read. Max was about 400 files, about 250 million cells. Total was #### blockIns and #### compositeIndexs.

Scaling went as follows, using the new Memory Inspector. Numbers are for one node, 8 processes in 12 GBytes of memory space:

* Nothing open: 1.04 GBytes, CompositeIndex 0
* One file open: 1.61 GBytes, CompositeIndex 844. Empty cores: ?? GBytes
* 8 files open: 1.97 GBytes, CompositeIndex 1.97 Empty cores: 1.41 GBytes
* 32 files open: 2.53 GBytes, Composite index 26242 Empty cores: 2.03 Gbytes
* 128 files open: 4.88 GBytes, Composite index 82717. No empty cores.
* 256 files open: crash. Note that more nodes never work, where fewer cores per node does work.

Note that, in theorie, with 128 cores and 128 files open, I now have one file open per core. I would assume that the size of 128 files (for full cores) would be taking the same as 8 files.

Throwing this in a spread sheet, it appears that every additional compositeIndex adds about 5KBytes of memory loss. Ouch!

Sorry for the confusing bug report - ask me for details.

Listing as a crash, since it does on large data.


Steps To Reproduce
Additional Information
TagsNo tags attached.
Relationships
related to 0012726closed Utkarsh Ayachit CTH variable memory use is inefficient and wrong 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2011-12-20 20:25Alan ScottNew Issue
2011-12-20 20:52Alan ScottStatusbacklog => todo
2012-05-22 18:24Alan ScottRelationship addedrelated to 0012726
2012-05-31 10:23Utkarsh AyachitStatustodo => backlog
2012-05-31 10:24Utkarsh AyachitStatusbacklog => todo
2012-09-11 17:00Utkarsh AyachitStatustodo => gatekeeper review
2012-09-11 17:00Utkarsh AyachitResolutionopen => fixed
2012-09-11 17:00Utkarsh AyachitAssigned To => Utkarsh Ayachit
2012-09-11 17:01Utkarsh AyachitNote Added: 0029173
2012-09-11 17:01Utkarsh AyachitStatusgatekeeper review => customer review
2012-09-11 17:01Utkarsh AyachitFixed in Version => git-master
2012-09-24 20:50Alan ScottNote Added: 0029276
2012-09-24 20:50Alan ScottStatuscustomer review => closed
2012-10-29 17:04Utkarsh AyachitFixed in Versiongit-master => 3.98.0

Notes
(0029173)
Utkarsh Ayachit   
2012-09-11 17:01   
Moving this to customer review since AMR datastructure changes were merged into ParaView and Alan's initial testing has confirmed size improvements. Please reopen if the issue persists.
(0029276)
Alan Scott   
2012-09-24 20:50   
Spectacular. If we decide to do more memory work for CTH, I will open another bug. Tested numerous ways, including Dave's large CTH dataset, and a 400 file, .25 billion cell run. Linux, 8 remote servers, master.