<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi Robert,<br>
<br>
it is been a while since we had this problem but we are running
into it again. Therefore I would like to reopen the discussion on
making use of the file index in the vtkEnSightGoldBinaryReader. I
expect two advantages by doing so, firstly it is more efficient
and secondly, it can be used to enable changing geometries over
time (variable number of points). At the moment, time steps are
skipped (when reading vectors) by assuming a constant number of
points in the mesh over all time steps which is of course not
correct for variable number of points.<br>
<br>
I implemented a preliminary version of reading the file index. It
works fine (and is fast :-)) and enables us to visualize changing
geometries. The implementation is not yet in a style that could be
included in VTK, I therefore wanted to get some pointers what is
the best way to include this feature into VTK.<br>
<br>
I have thought of the following scenarios:<br>
<br>
1)<br>
when reading Scalars, Vectors, Tensors, ... the blocks of data
until timeStep-1 are being skipped one by one assuming a constant
geometry. To skip this we look for a file index and, if present,
this->IFile is set to the correct position in the file.<br>
This has the advantage of being very easy to implement, however it
has to be implemented for every type of field separately. Also one
has to jump to the end of the file for every field in every file
whenever a new time step is requested.<br>
<br>
2)<br>
provide a method in the vtkEnSightGoldBinaryReader, that
implements the reading of the file index and setting
this->IFile to the correct position (for example a
SkipTimestepsUsingFileindex(int timestep) ) or that returns the
address(es) of a(ll) time step(s), which can then be used in the
Read{Scalars,Vectors,..}.. methods.<br>
This way only one implementation that actually reads the file
index is needed, but it still requires the file index to be read
for every field in every file whenever a new time step is
requested.<br>
<br>
3)<br>
Read the file index of all files when the reader is initialized
and store the timestep positions per file in a member of the
vtkEnSightGoldBinaryReader. This way the index has to be read only
once and timesteps can be skipped by referring to the member.
Drawback of this method is, that I need some hints on how this
could be implemented consistent with the rest of the reader (i.e.
map<std::string, std::vectors<long> > vs. string[] and
long[][] or anything in between).<br>
<br>
There are some lines of code attached that show the proof of
concept implementation. I am looking forward to getting some
suggestions.<br>
<br>
Cheers,<br>
<br>
Georg<br>
<br>
<br>
On 19/04/13 14:51, Robert Maynard wrote:<br>
</div>
<blockquote
cite="mid:CAFzjYVq6HW9MHg9cOiwnMrFGXh-YPiJ8kj9iKqThL+8kKbpN=Q@mail.gmail.com"
type="cite">
<meta http-equiv="Context-Type" content="text/html;
charset=windows-1252">
<div dir="ltr">The file index is required by Ensight for all
transient single file formats. The documentation also states
that Ensight automatically writes out the file index
information, so I would consider any transient single file
format that doesn't have it be malformed.</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Fri, Apr 19, 2013 at 7:57 AM, Georg
Hammerl <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:hammerl@lnm.mw.tum.de" target="_blank">hammerl@lnm.mw.tum.de</a>></span>
wrote:<br>
<blockquote class="gmail_quote">
<div>
<div>Hi Robert,<br>
<br>
thanks for the hint with the file index. The ensight
gold files I have available come with a file index. I
have tried to use it in Paraview, however it seems that
the VTK/IO/Ensight/vtkEnSightGoldBinaryReader does not
make use of the index. <br>
Unfortunately, I do not have enough insight into the
inner workings of VTK, so I cannot see myself creating a
clean implementation for using the file index.<br>
<br>
As the file index is just optional, I think it should be
worth implementing an efficient way of reading ensight
gold format even without the file index.<br>
<br>
Cheers,<br>
<br>
Georg
<div>
<div class="h5"><span><br>
</span><br>
<br>
On 18/04/13 20:06, Robert Maynard wrote:<br>
</div>
</div>
</div>
<div>
<div class="h5">
<blockquote type="cite">
<div dir="ltr">Hi Georg,
<div><br>
</div>
<div>You are correct that a
more efficient solution is possible. After
reading the Ensight Gold binary format ( <a
moz-do-not-send="true"
href="http://www-vis.lbl.gov/NERSC/Software/ensight/doc/OnlineHelp/UM-C11.pdf"
target="_blank">http://www-vis.lbl.gov/NERSC/Software/ensight/doc/OnlineHelp/UM-C11.pdf</a>
) I have discovered how you can do this
more efficiently.</div>
<div><br>
</div>
<div>If you are start at 11-37 you will see that
they state:</div>
<div><br>
</div>
<blockquote>
<div>
<div>"Note 3: Efficient reading of each file
(especially binary) is facilitated by </div>
</div>
<div>
<div>appending each file with a file index. A
file index contains appropriate </div>
</div>
<div>
<div>information to access the file byte
positions of each time step in the file.
(EnSight </div>
</div>
<div>
<div>automatically appends a file index to
each file when exporting in transient
single </div>
</div>
<div>
<div>file format.) If used, the file index
must follow the last END TIME STEP</div>
</div>
<div>
<div>wrapper in each file."</div>
</div>
<div>
<div><br>
</div>
</div>
<div>
<div>File Index Usage:</div>
</div>
<div>
<div>ASCII, Binary, Item, Description</div>
</div>
<div>
<div>“%20d\n”, sizeof(int), n, Total number of
data time steps in the file.</div>
</div>
<div>
<div>“%20d\n”, sizeof(long), fb1, File byte
loc for contents of 1st time step*</div>
</div>
<div>
<div>“%20d\n”, sizeof(long), fb2, File byte
loc for contents of 2nd time step*</div>
</div>
<div>
<div>
<div>. . ., . . ., . . ., . . .</div>
</div>
</div>
<div>
<div>
<div>“%20d\n”, sizeof(long), fbn, File byte
loc for contents of nth time step*</div>
</div>
</div>
<div>
<div>
<div>“%20d\n”, sizeof(int), flag,
Miscellaneous flag (= 0 for now)</div>
</div>
</div>
<div>
<div>
<div>“%20d\n”, sizeof(long), fb, of item n
File byte loc for Item n above</div>
</div>
</div>
<div>
<div>
<div>“%s\n”, sizeof(char)*80, “FILE_INDEX”,
File index keyword</div>
</div>
</div>
<div>
<div>* Each file byte location is the first
byte that follows the BEGIN TIME STEP record</div>
</div>
</blockquote>
<div><br>
</div>
<div>To me the best way is to read the file in
reverse looking for FILE_INDEX as the last 80
characters. From that you parse the previous
long as</div>
<div>the new seek position which jumps you to the
start of the file index records. After reading
the file index records you have a lookup table
for all seek positions of the time steps in that
ensight file.</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Wed, Mar 6, 2013 at
4:23 AM, Georg Hammerl <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:hammerl@lnm.mw.tum.de"
target="_blank">hammerl@lnm.mw.tum.de</a>></span>
wrote:<br>
<blockquote class="gmail_quote">Hello,<br>
<br>
I have already posted this issue on the
paraview mailing list, but then I realized
that it is more of a VTK problem.<br>
I can load the first time step of my results
but when I switch to the next step (in which 4
points are added), paraview freezes and htop
shows 100% load for this process.
Surprisingly, this only happens when I use
binary ensight gold format. The same results
written in ascii ensight gold format work. The
ens_checker tells me for both cases that I
have valid output files.<br>
<br>
Meanwhile I had a look into the source code
and discovered the issue. Whenever the
variable-files are read for a time steps
greater than 1, Paraview has to jump in the
binary file to the desired timestep. This is
done by partially parsing and skipping blocks
until the correct "BEGIN TIME STEP" is found.
In order to skip the correct block length,
Paraview uses the number of points (numPts)
from the current geometry step. This breaks
with a varying number of points per timestep.<br>
As an example my first step contains 2 points
and the second step 6 points, therefore when
Paraview skips the first time step, it skips a
block length equivalent to 6 points. The
correct number would be 2 points. It therefore
skips past the next "BEGIN TIME STEP".<br>
<br>
In order to verify my theory I have
implemented a little hack which looks for the
next "BEGIN TIME STEP" by brute force. In case
someone wants to try this I have attached a
patch which can be used in conjunction with
the attached examples.<br>
<br>
A clean solution would be to skip the correct
number of points for every given timestep.
Unfortunately I do not have enough insight
into the VTK ensight reader to implement this
and would appreciate any help in finding a
clean solution to this.<br>
<br>
Cheers,<br>
<br>
Georg<span><br>
<br>
-- <br>
Dipl.-Ing. Georg Hammerl<br>
Lehrstuhl für Numerische Mechanik<br>
Technische Universität München<br>
Boltzmannstrasse 15, D-85747 Garching b.
München<br>
phone <a moz-do-not-send="true"
href="tel:%2B49%2089%20289%2015237"
value="+498928915237" target="_blank">+49
89 289 15237</a><br>
fax <a moz-do-not-send="true"
href="tel:%2B49%2089%20289%2015301"
value="+498928915301" target="_blank">+49
89 289 15301</a><br>
<a moz-do-not-send="true"
href="mailto:hammerl@lnm.mw.tum.de"
target="_blank">hammerl@lnm.mw.tum.de</a><br>
<a moz-do-not-send="true"
href="http://www.lnm.mw.tum.de"
target="_blank">http://www.lnm.mw.tum.de</a><br>
<br>
</span><br>
_______________________________________________<br>
Powered by <a moz-do-not-send="true"
href="http://www.kitware.com"
target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a
moz-do-not-send="true"
href="http://www.kitware.com/opensource/opensource.html"
target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Please keep messages on-topic and check the
VTK FAQ at: <a moz-do-not-send="true"
href="http://www.vtk.org/Wiki/VTK_FAQ"
target="_blank">http://www.vtk.org/Wiki/VTK_FAQ</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a moz-do-not-send="true"
href="http://www.vtk.org/mailman/listinfo/vtkusers"
target="_blank">http://www.vtk.org/mailman/listinfo/vtkusers</a><br>
<br>
</blockquote>
</div>
<br>
<br>
<div><br>
</div>
-- <br>
Robert Maynard </div>
</blockquote>
<br>
<br>
<pre cols="72">--
Dipl.-Ing. Georg Hammerl
Lehrstuhl für Numerische Mechanik
Technische Universität München
Boltzmannstrasse 15, D-85747 Garching b. München
phone <a moz-do-not-send="true" href="tel:%2B49%2089%20289%2015237" value="+498928915237" target="_blank">+49 89 289 15237</a>
fax <a moz-do-not-send="true" href="tel:%2B49%2089%20289%2015301" value="+498928915301" target="_blank">+49 89 289 15301</a>
<a moz-do-not-send="true" href="mailto:hammerl@lnm.mw.tum.de" target="_blank">hammerl@lnm.mw.tum.de</a>
<a moz-do-not-send="true" href="http://www.lnm.mw.tum.de" target="_blank">http://www.lnm.mw.tum.de</a>
</pre>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br>
<div><br>
</div>
-- <br>
Robert Maynard
</div>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Dipl.-Ing. Georg Hammerl
Lehrstuhl für Numerische Mechanik
Technische Universität München
Boltzmannstrasse 15, D-85747 Garching b. München
phone +49 89 289 15237
fax +49 89 289 15301
<a class="moz-txt-link-abbreviated" href="mailto:hammerl@lnm.mw.tum.de">hammerl@lnm.mw.tum.de</a>
<a class="moz-txt-link-freetext" href="http://www.lnm.mw.tum.de">http://www.lnm.mw.tum.de</a>
</pre>
</body>
</html>