<div dir="ltr"><div><div><div><div>Hi Evan,<br><br></div>Thanks for sharing the data. I tried your script on my Linux desktop and the performance I see on both VTK and ParaView is similar and <1 second. This is even without threading enabled. I haven't tried this on Windows yet.<br><br>BTW, there is an error in the script you shared. The inputs to the resample filter are in the wrong order and the "vtkXMLUnstructuredGridWriter" throws an error saying that the data passed to it is not an unstructured grid. I assume you want to resample the data values from the structured grid on to the geometry provided by the unstructured grid. The result will be an unstructured grid, which can be written by the "vtkXMLUnstructuredGridWrite". For this the input should be the "mesh" data and the source should be "image".<br><br></div>So, currently I don't have a good explanation for what is causing the performance degradation for you. It might be some issues with the builds, or the Windows setup. You can maybe try building VTK yourself (make sure to build in "Release" mode), or try another machine and see if the problem persists. I will also try to reproduce this on a Windows machine.<br><br></div>Thanks<br></div>Sujin<br><div><div><div><div><div><br></div></div></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 15, 2018 at 4:47 PM, Evan Kao <span dir="ltr"><<a href="mailto:tossin@gmail.com" target="_blank">tossin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hello Sujin,<div><br></div><div>Using the TimerLog, I got the following time from ParaView:</div><div>
<p style="margin:0px;text-indent:0px;white-space:pre-wrap"><span style="font-family:"Courier New"">Execute vtkResampleWithDataSet id: 10345, 0.42 seconds</span></p><br></div><div>As for the timeit module, you can see how I use it in the attached Python script. I only use timeit's default_timer function to grab the time before and after completion of the vtkResampleWithDataSet method and take the difference as the time elapsed. Regardless, qualitatively ParaView is near-instant while VTK takes a while.</div><div><br></div><div>Google drive links to the datasets themselves are here (hopefully this doesn't trigger any mailing list filters): <a href="https://drive.google.com/open?id=1jvjiDlMJEJihB8OQneOeBzJXFiZKKYsR" target="_blank">Unstructured Grid (35MB)</a> | <a href="https://drive.google.com/open?id=1RYz4eORPWWf23n6G5am-9_F44zxEOHMl" target="_blank">Structured Grid (70MB)</a></div><div><br></div><div>If I get a chance, I'll take a look at using smaller data sets.</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>- Evan</div><div><br></div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 15, 2018 at 12:32 PM, Sujin Philip <span dir="ltr"><<a href="mailto:sujin.philip@kitware.com" target="_blank">sujin.philip@kitware.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Hi Evan,<br><br></div>I tried testing this on my end and I am seeing expected performance from VTK and ParaView. But the performance is dependent on the datasets used. Is it possible for you to share your datasets and scripts with us? Could you try this with smaller versions of your datasets and see if you are able to reproduce this?<br><br></div>I am not familiar with the timeit module in Python. From the documentation it looks like it runs the code multiple times by default and prints the total time. Can you confirm if you have taken this into consideration in your script?<br><br></div><div>A simple way to time operations in ParaView is to refer to the "Timer Log" under the "Tools" menu. You should see a line like:
<p style="margin:0px;text-indent:0px;white-space:pre-wrap"><span style="font-family:monospace">Execute vtkResampleWithDataSet id: 6788, 2.70556 seconds</span></p><br></div><div><br></div>Thanks<span class="m_-717951984441422486gmail-HOEnZb"><font color="#888888"><br></font></span></div><span class="m_-717951984441422486gmail-HOEnZb"><font color="#888888">Sujin<br><br></font></span></div><div class="m_-717951984441422486gmail-HOEnZb"><div class="m_-717951984441422486gmail-h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 15, 2018 at 1:05 PM, Evan Kao <span dir="ltr"><<a href="mailto:tossin@gmail.com" target="_blank">tossin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Shawn and Sujin,<div><br></div><div>Thanks for the quick responses. The CPU on the computer I'm using is an <a href="https://ark.intel.com/products/88196/Intel-Core-i7-6700-Processor-8M-Cache-up-to-4_00-GHz" target="_blank">i7-6700</a> with 4 cores, 8 threads, and 3.4 GHz frequency.</div><div><br></div><div>Multi-threading may be a factor, but it's hard to tell because resampling in ParaView is so quick. ParaView is capable of using 100% of the CPU, while VTK (in Python) will max out at 12-13%. However, for these particular datasets, resampling doesn't appear to stress ParaView that much (11-16% when observing the Windows Task Manager, and some of that may be because of the rendering). However, I was under the impression that at best multi-threading could only reduce the time it takes by N threads (ie 8x), while the speed difference here is almost 1000x. I measured the times for ParaView 5.5, VTK 8.1 (compiled elsewhere), and VTK 7.1 (compiled by our group):</div><div><ol><li>ParaView 5.5 - 1.1s, using a stopwatch, multiple trials. Timing started the moment I clicked "Apply".</li><li>VTK 8.1 - 922.47s, timed using Python's timeit module, measuring only the vtkResampleWithDataSet.Update(<wbr>) method.</li><li>VTK 7.1 - 950.47s, timed the same way as above.</li></ol></div><div>I'm aware of the difference in labeling between VTK and ParaView for Source and Input (which confuses me all the time). I can verify the correct data sets were assigned by saving the output (which should an unstructured grid) and viewing it in ParaView - it looks identical to the resampled data generated in ParaView (although it overwrites the point scalars array and adds some ghost information that needs to be removed).</div><div><br></div><div>Thanks,</div><div>Evan</div></div><div class="m_-717951984441422486gmail-m_144391633153695864HOEnZb"><div class="m_-717951984441422486gmail-m_144391633153695864h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 15, 2018 at 7:38 AM, Sujin Philip <span dir="ltr"><<a href="mailto:sujin.philip@kitware.com" target="_blank">sujin.philip@kitware.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div>Hi Evan,<br><br></div>As Shawn mentioned it could be due to lack of multi-threading. Could you provide us the configuration of the system you are using? Like the number of cores/threads and the CPU frequency? Also please share the actual time that ParaView and VTK are taking. Is it possible for you to try out a slightly older VTK version and see if the performance difference is still there?<br></div><div><br>Which dataset are you setting as input and which as source? The names are unfortunately opposite between VTK-m and ParaView due to legacy reasons. Probing with the unstructured grid as the source is much slower than probing with the structured grid as the source. So please confirm that the VTK pipeline is set up properly.<br><br></div><div>Please let me know if none these seem to be the cause of your problem and I will dig deeper.<br><br></div><div>Thanks<span class="m_-717951984441422486gmail-m_144391633153695864m_-2554281641319285852gmail-HOEnZb"><font color="#888888"><br></font></span></div><span class="m_-717951984441422486gmail-m_144391633153695864m_-2554281641319285852gmail-HOEnZb"><font color="#888888"><div>Sujin<br><br></div><div><br></div></font></span></div><div class="m_-717951984441422486gmail-m_144391633153695864m_-2554281641319285852gmail-HOEnZb"><div class="m_-717951984441422486gmail-m_144391633153695864m_-2554281641319285852gmail-h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 15, 2018 at 9:52 AM, Shawn Waldon <span dir="ltr"><<a href="mailto:shawn.waldon@kitware.com" target="_blank">shawn.waldon@kitware.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div>Hi Evan,<br><br></div>I suspect the differece is that the ParaView binaries were compiled with TBB multithreading support and the Anaconda VTK was not. vtkResampleWithDataSet is set up to use TBB multithreading if available. Check the utilization of the cores on your computer when running each and you will see ParaView using all available cores and Anaconda's VTK probably only using one. It is also possible the cell locator change improved things further but I'm not familiar with that.<br><br></div><div>HTH,<br><br></div>Shawn<br></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="m_-717951984441422486gmail-m_144391633153695864m_-2554281641319285852gmail-m_-4441563830219692306h5">On Mon, May 14, 2018 at 7:54 PM, Evan Kao <span dir="ltr"><<a href="mailto:tossin@gmail.com" target="_blank">tossin@gmail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="m_-717951984441422486gmail-m_144391633153695864m_-2554281641319285852gmail-m_-4441563830219692306h5"><div dir="ltr">Hello all,<div><br></div><div>I am trying to resample a structured grid data (~1.4M points, 1.3M cells) with an unstructured grid (~320K points, 480K cells). In Paraview 5.5, this resampling is nearly instant with the Resample With Dataset filter. Yet in a Python script using vtkResampleWithDataSet from VTK 8.1.0, the same operation takes about 15 minutes (>2 orders of magnitude difference in speed). As far as I can tell from the VTK repository on Gitlab, the only difference between the Paraview/release version and the 8.1.0 or 8.1.1 tagged releases is a switch in the cell locator. Is this enough to explain the difference in the performance? If not, could someone enlighten me as to what the possible factors are here?</div><div><br></div><div>Also, if it matters, this is all on a Windows 7 64-bit machine. Paraview is installed from binaries, while VTK was downloaded from an Anaconda distribution compiled by a third party.</div><div><br></div><div>Thanks for your time,</div><div>Evan Kao</div></div>
<br></div></div><span>______________________________<wbr>_________________<br>
Powered by <a href="http://www.kitware.com" rel="noreferrer" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" rel="noreferrer" target="_blank">http://www.kitware.com/opensou<wbr>rce/opensource.html</a><br>
<br>
Please keep messages on-topic and check the VTK FAQ at: <a href="http://www.vtk.org/Wiki/VTK_FAQ" rel="noreferrer" target="_blank">http://www.vtk.org/Wiki/VTK_FA<wbr>Q</a><br>
<br>
Search the list archives at: <a href="http://markmail.org/search/?q=vtkusers" rel="noreferrer" target="_blank">http://markmail.org/search/?q=<wbr>vtkusers</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="https://vtk.org/mailman/listinfo/vtkusers" rel="noreferrer" target="_blank">https://vtk.org/mailman/listin<wbr>fo/vtkusers</a><br>
<br></span></blockquote></div><br></div>
<br>______________________________<wbr>_________________<br>
Powered by <a href="http://www.kitware.com" rel="noreferrer" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" rel="noreferrer" target="_blank">http://www.kitware.com/opensou<wbr>rce/opensource.html</a><br>
<br>
Please keep messages on-topic and check the VTK FAQ at: <a href="http://www.vtk.org/Wiki/VTK_FAQ" rel="noreferrer" target="_blank">http://www.vtk.org/Wiki/VTK_FA<wbr>Q</a><br>
<br>
Search the list archives at: <a href="http://markmail.org/search/?q=vtkusers" rel="noreferrer" target="_blank">http://markmail.org/search/?q=<wbr>vtkusers</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="https://vtk.org/mailman/listinfo/vtkusers" rel="noreferrer" target="_blank">https://vtk.org/mailman/listin<wbr>fo/vtkusers</a><br>
<br></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>