[vtk-developers] cdash API and VTK's most wanted list

[Bill Lorensen]

Dave,

Very nice. Back in the day, we used to generate a table with a row for each
test and a column for each build. A red/green box showed pass/fail. Made it
easy to see if it was a windows/linux/sun/hp/sgi... issue.

I'll look into TestXMLFIleOutputWindow which seems to be failing strangely
on all windows platform (except cygwin).

For a description of the eary origins of testing for vtk you might look at:
http://www.cdash.org/cdash/project/download/ExtremeTestingTalk.pdf


On Fri, Oct 5, 2012 at 2:38 PM, David E DeMarle <dave.demarle at kitware.com>wrote:

> Howdy y'all,
>
> I've been working on cleaning the VTK dashboard lately and recently
> learned something that is really useful - that CDash's Web API is
> really powerful.
>
> See: http://public.kitware.com/Wiki/CDash:API
>
> Using it I wrote the attached python script which scrapes last night's
> VTK test results and presents them in a test centric view, rather than
> cdash's native submitter centric view.
>
> The test centric view is more useful I think because it helps you to
> prioritize what _code_ or project feature is worst and should be fixed
> soonest, as opposed to what machine is performing worst. It also gives
> you a more realistic view of the state of the project, because it
> won't make you falsely believe, for instance, that there are at most
> the same number of problems as the worst dashboard shows. (20 or so vs
> 70 today)
>
> Also, since it is a script it takes less than a second to generate the
> report, whereas if you build up the same list on your own manually by
> clicking on cdash's web pages it will take you about an hour on a
> project (and project state) like VTK today.
>
> If you run the script you will see that the worst tests are:
>
> vtkIOExportCxx-TestGL2PSExporterVolumeRaster-VerifyRasterizedPNG , 6 ,
> ['Mac10.7.3-clang-release-x86_64-nightly_master', 'Win32-vs9-Shared',
> 'Win64-VS10', 'Mac10.7.3-gcc-release-x86_64-nightly_master',
> 'Win32-vs9-Static', 'Ubuntu-Valgrind']
> vtkInteractionStyleTcl-TestInteractorStyleTerrain , 7 , [...]
> vtkRenderingOpenGLCxx-TestGaussianBlurPass , 7 , [...]
> vtkCommonCore-TestXMLFileOutputWindow , 7 , [...]
> vtkRenderingOpenGLCxx-TestTranslucentLUTDepthPeelingPass , 7 , [...]
> ! vtkRenderingVolumeTcl-gaussian , 10 , [...]
> ! vtkIOGeometryTcl-TestXYZMol , 10 , [...]
> vtkIOSQLCxx-SQLiteTableReadWrite , 10 , [...]
> vtkRenderingOpenGLCxx-TestOffAxisStereo , 14 , [...]
> ! vtkDomainsChemistryCxx-TestProteinRibbon , 19 , [...]
>
> Where the ones I've marked with ! have been fixed and thus should be
> off the most listed list tonight.
>
> David E DeMarle
> Kitware, Inc.
> R&D Engineer
> 21 Corporate Drive
> Clifton Park, NY 12065-8662
> Phone: 518-881-4909
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://www.vtk.org/mailman/listinfo/vtk-developers
>
>
>


-- 
Unpaid intern in BillsBasement at noware dot com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20121005/60dd814f/attachment.html>