CDash:Testing

CDash integrates a self-testing framework based on Simple Test. The self testing code is located in the CDash/testing directory.

Submitting a CDash Dashboard to the CDash dashboard

Don't worry, it's not a mistake in the title of this section, self-testing involves submitting a CDash submission to CDash.

Testing CDash with CTest

CDash can be tested with CTest from cmake (www.cmake.org.). For this style of testing, you treat CDash much like any other CMake based project.

• Checkout the CDash source code from the svn repository to a directory named "CDashTesting" in your web server's htdocs folder.

• assume that the full path to CDash is /path/to/source/CDashTesting

• Create a build directory

 cd /path/to/source/CDashTesting
 mkdir build

• Run cmake on on CDashTesting

 cd build
 cmake ..

 There are some CMake cache options that configure server names, the default is localhost.

• For coverage, you need to have xdebug installed as a module in your php installation. Once that is done, you need to edit the php.ini file, and add these two lines and restart apache:

 auto_prepend_file =/path/to/source/CDashTesting/prepend_coverage.php
 auto_append_file =/path/to/source/CDashTesting/build/append_coverage.php

For coverage to work, a CMake next newer than 5/25/2010 at 4:00 pm will be needed.

• Note this will create CDashTesting/cdash/config.local.php and CDashTesting/testing/config.test.local.php files in your source tree, so svn will show them as ? files.

• Run ctest on the build tree, like any other ctest project. For example, this will run an experimental coverage build:

 cd /path/to/source/CDashTesting/build
 ctest -D ExperimentalStart -D ExperimentalConfigure -D ExperimentalTest -D ExperimentalCoverage -D ExperimentalSubmit

Configuring the testing

Here are the steps on how to configure the testing without ctest:

• Checkout the CDash source code from the svn repository to a directory named "CDashTesting" in your web server's htdocs folder.

• Add a file named CDashTesting/cdash/config.local.php containing:

<?php
$CDASH_DB_NAME = 'cdash4simpletest';
$CDASH_TESTING_MODE = true;
?>

This file is automatically included by CDash's config.php, but only if it exists.

The database has to be named exactly 'cdash4simpletest' for the tests to run. Part of the test suite involves destroying the existing database of that name and re-creating it from scratch. Because the testing of CDash drops and creates the database you need to make sure this database is not being used for any other purpose than self testing. Also, make sure that the $CDASH_DB_LOGIN has enough privileges to create/drop databases on your SQL server.

• Add a file named CDashTesting/testing/config.test.local.php containing (example values only... adjust to suit your localhost web server):

<?php
$configure = array(
  'urlwebsite'       => 'http://localhost/CDashTesting',
  'outputdirectory'  => 'C:/tmp',  // or /tmp or ~/tmp on a Linux/Mac server
  'type'             => 'Nightly',
  'site'             => 'computername.companyname',
  'buildname'        => 'SVN-Win32-xampp',
  'cdash'            => 'http://www.cdash.org/CDash',
  'svnroot'          => 'C:/xampp/htdocs/CDashTesting'
  );
?>

This file is automatically included by CDash's config.test.php, but only if it exists.

• urlwebsite is the URL to access the tested installation of CDash. Since simpletest emulates a web browser you need to make sure that the urlwebsite is accessible locally. That is, you should be able to visit this URL in a web browser on the testing machine and see the main CDash page.
• outputdirectory is a directory where the testing output can be stored on the system. Make sure that the current user can write to that directory.
• type is the type of submission for CDash: Nightly, Continuous or Experimental.
• site is the name of the site submitting to CDash.
• buildname is name of the build.
• cdash is the URL to the current dashboard. Note that the project should be named CDash, i.e. http://www.cdash.org/CDash/submit.php?project=CDash
• svnroot is the file system path to your testing installation of CDash. The self testing does an svn update to check what files have changed.

Starting the testing

Then to launch the tests and submit a dashboard:

 cd CDashTesting/testing
 php5 alltests.php

For xampp installations on Windows, run:

 cd CDashTesting\testing
 C:\xampp\php\php.exe alltests.php

To run the tests in your browser (without submitting a dashboard) visit the URL:

 http://localhost/CDashTesting/testing/alltests.php