[vtk-developers] cdash API and VTK's most wanted list

[Bill Lorensen]

Dave,

You should put this script in Utilities/Maintenance maybe with a better
name:

TestFailures.py or
TestFailSummary.py
or ...

This is a great way to experiment with useful cdash scripts that might be
added to cdash in the future.

Nice,

Bill

On Fri, Oct 5, 2012 at 2:38 PM, David E DeMarle <dave.demarle at kitware.com>wrote:

> Howdy y'all,
>
> I've been working on cleaning the VTK dashboard lately and recently
> learned something that is really useful - that CDash's Web API is
> really powerful.
>
> See: http://public.kitware.com/Wiki/CDash:API
>
> Using it I wrote the attached python script which scrapes last night's
> VTK test results and presents them in a test centric view, rather than
> cdash's native submitter centric view.
>
> The test centric view is more useful I think because it helps you to
> prioritize what _code_ or project feature is worst and should be fixed
> soonest, as opposed to what machine is performing worst. It also gives
> you a more realistic view of the state of the project, because it
> won't make you falsely believe, for instance, that there are at most
> the same number of problems as the worst dashboard shows. (20 or so vs
> 70 today)
>
> Also, since it is a script it takes less than a second to generate the
> report, whereas if you build up the same list on your own manually by
> clicking on cdash's web pages it will take you about an hour on a
> project (and project state) like VTK today.
>
> If you run the script you will see that the worst tests are:
>
> vtkIOExportCxx-TestGL2PSExporterVolumeRaster-VerifyRasterizedPNG , 6 ,
> ['Mac10.7.3-clang-release-x86_64-nightly_master', 'Win32-vs9-Shared',
> 'Win64-VS10', 'Mac10.7.3-gcc-release-x86_64-nightly_master',
> 'Win32-vs9-Static', 'Ubuntu-Valgrind']
> vtkInteractionStyleTcl-TestInteractorStyleTerrain , 7 , [...]
> vtkRenderingOpenGLCxx-TestGaussianBlurPass , 7 , [...]
> vtkCommonCore-TestXMLFileOutputWindow , 7 , [...]
> vtkRenderingOpenGLCxx-TestTranslucentLUTDepthPeelingPass , 7 , [...]
> ! vtkRenderingVolumeTcl-gaussian , 10 , [...]
> ! vtkIOGeometryTcl-TestXYZMol , 10 , [...]
> vtkIOSQLCxx-SQLiteTableReadWrite , 10 , [...]
> vtkRenderingOpenGLCxx-TestOffAxisStereo , 14 , [...]
> ! vtkDomainsChemistryCxx-TestProteinRibbon , 19 , [...]
>
> Where the ones I've marked with ! have been fixed and thus should be
> off the most listed list tonight.
>
> David E DeMarle
> Kitware, Inc.
> R&D Engineer
> 21 Corporate Drive
> Clifton Park, NY 12065-8662
> Phone: 518-881-4909
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://www.vtk.org/mailman/listinfo/vtk-developers
>
>
>


-- 
Unpaid intern in BillsBasement at noware dot com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20121007/efc6eace/attachment.html>