VTK/GitMSBuild

Building VTK using Git and MSBuild

Preliminary steps

We are going to create a script called pull.sh that will automatically pull down onto your computer the files from the VTK and WikiExamples git repositories. This assumes that you have Git installed on your system.

Then we will create file called build.cmd that will build debug and release versions of VTK and WikiExamples. It will use MSBuild to build the software and set the number of CPU's to use to the maximum on your system. This file has the ability to perform separate configurations such as 32-bit or 64-bit. To do this we will create a link to this file called build-x64 that passes a parameter to build.cmd specifying a 64-bit build. Of course you can edit this is as you want, for example, setting a 32-bit build instead.

The key advantage of this process is that the pulling and building process is then largely automated.

By using scripting in this manner we can easily change or add more builds and change build types. In the following discussion we will be doing a 64-bit build. However as mentioned above the scripts have been written so that a 32-bit build can be done by changing just one link.

VTK and the WikiExamples will be built for both release and debug with the release and debug versions being in separate trees. Thus there will be a different build directory for each build configuration. The advantage of this is that there is one tree for each build and we have a clean separation of the build types. You can also extend the scripting to deal with both 32-bit ad 64-bit builds on the same machine. This means that when you are building your own code, using these libraries, you can easily have a debug version and a release version. This will be demonstrated in the WikiExamples build.

If you have express versions of the compiler and want to use the 64-bit version make sure you have the correct Windows SDK installed. This link may provide useful information for VS2008: [1]

Keeping in mind that we are keeping separate source and build/release trees the following directory structure will be used:

%KITWARE_PATH%\build\VTK\Release

%KITWARE_PATH%\build\VTK\Debug

%KITWARE_PATH%\build\WikiExamples\Release

%KITWARE_PATH%\build\WikiExamples\Debug

%KITWARE_PATH%\src\VTK

%KITWARE_PATH%\src\WikiExamples

%KITWARE_PATH%\src\VTKData

%KITWARE_PATH%\src\VTKLargeData

Where %KITWARE_PATH% will be the root path e.g C:\Kitware or C:\Users\<your username>\Code\Kitware, you can choose whatever you prefer here.

Before proceeding you will need to set some environment variables.

Environment variables

Set the following environment variables:

KITWARE_PATH	This will be the root path e.g C:\Kitware or C:\Users\<your username>\Code\Kitware you can choose whatever you prefer here
VTK_BIN	%KITWARE_PATH%\build\VTK\Release\bin
VTK_DATA_ROOT	%KITWARE_PATH%\src\VTKData
VTK_ROOT	%KITWARE_PATH%\src\VTK

The VTK_BIN environment variable will be the default one for all your builds. If you want a debug build then you will change it as outlined below.

You normally want the release variables to be available to the system, so add the following path to the PATH environment variable:

%KITWARE_PATH%\build\VTK\Release\bin\Release

However if you are interested in having debug variables available to the system then you must add the following path to the PATH environment variable:

%KITWARE_PATH%\build\VTK\Release\bin\Debug

You cannot have both paths in the PATH environment variable. Use either one or the other.

Directories

Now create the following directories:

%KITWARE_PATH%\src

%KITWARE_PATH%\build\VTK\Debug

%KITWARE_PATH%\build\VTK\Release

%KITWARE_PATH%\build\WikiExamples\Debug

%KITWARE_PATH%\build\WikiExamples\Release

Clone

Clone VTKsource and VTKData (optionally VTKLargeData) via Git access into %KITWARE_PATH%\src see:

http://www.vtk.org/Wiki/VTK/Git

Clone WikiExamples via Git access into %KITWARE_PATH%\src see:

http://www.vtk.org/Wiki/VTK/Examples

Then in the source directory you should have the following folders:

%KITWARE_PATH%\src\VTK

%KITWARE_PATH%\src\VTKData

%KITWARE_PATH%\src\VTKLargeData

%KITWARE_PATH%\src\WikiExamples

Now create some scripts to facilitate pulling data from the git repositories and building using MSBuild.

Scripts and Batch files

Create the following script, calling it pull.sh and putting it in %KITWARE_PATH%\src:

#!/bin/bash
# Pull all git repositories in the directory this is running in and 
# any specified repositories in subdirectories.

# Author: Andrew Maclean
# Date: 2010-04-30
# Last Modified: 2010-12-17
# Reason: Handles System V version of ls which generates 9 fields
#         and the Berkley version of ls which generates 8 fields. 
# Last Modified: 2011-03-18
# Reason: Made pulling a function so that subdirectories can be accessed.
#         A log directory will be created in each directory. 
# Last Modified: 2012-03-26
# Reason: A single log directory is created.
#         Logs now show a diff --stat between the original head and the updated head. 

pull()
{
	# Runs a git checkout into each of the subdirectories.
	# We are assuming that each of the subdirectories is
	# legitimate and that git has been set up for each directory.
	outDir="$PWD/git_log"
	[ -d "$outDir" ] || mkdir $outDir

	for fn in $dirs
	do
	(
	    output="$outDir/${fn}.log"
	    # remove old log files
	    rm -f $output 2>/dev/null

	    cd $fn || exit 1
	    [ -d .git ] || {
        	echo "Error: no .git directory for $fn"
	        exit 1
	    }

	    echo "Pulling  $fn"
	    echo "Started:  `date`" >> $output
		original_head=$(git rev-parse HEAD) || exit
	    git pull --rebase 2>&1  >> $output
		updated_head=$(git rev-parse HEAD) || exit
		if test "$updated_head" = "$original_head"
        then
			echo "Head unchanged." >> $output
        else
			echo "Head changed, $original_head..$updated_head" >> $output
			( git diff "$original_head" "$updated_head"  --stat >> $output )
        fi 
 	    # update submodules (if they exist)
	    mods=$(git submodule | wc -l)
	    [ "$mods" != 0 ] && git submodule update 2>&1 >> $output

	    # update hooks (if they exist)
	    hooks=$(git branch -a -l | grep hooks | wc -l)
	    if [ "$hooks" != 0 ]
	    then
        	( cd .git/hooks && git pull .. remotes/origin/hooks 2>&1 >> $output )
	    fi

	    echo "Finished: `date`" >> $output
	)
	done
}

pull_data()
{
	# Runs a git checkout into each of the subdirectories.
	# We are assuming that each of the subdirectories is
	# legitimate and that git has been set up for each directory.
	outDir="$PWD/log"
	[ -d "$outDir" ] || mkdir $outDir

	for fn in $dirs
	do
	(
	    output="$outDir/${fn}.log"
	    # remove old log files
	    rm -f $output 2>/dev/null

	    cd $fn || exit 1
	    [ -d .git ] || {
        	echo "Error: no .git directory for $fn"
	        exit 1
	    }

	    echo "Pulling  $fn"
	    echo "Started:  `date`" >> $output
		original_head=$(git rev-parse HEAD) || exit
	    git pull --rebase 2>&1  >> $output
		updated_head=$(git rev-parse HEAD) || exit
		if test "$updated_head" = "$original_head"
        then
			echo "Head unchanged." >> $output
        else
			echo "Head changed, $original_head..$updated_head" >> $output
			( git diff "$original_head" "$updated_head" --stat >> $output )
        fi

	    echo "Finished: `date`" >> $output
	)
	done
}


#---------------- main -------------------
main()
{
	# We need CMake VTK ParaView ITK
	# dirs=$(ls -l |awk '/^d/ {print $NF}' | awk '$0 ~ /(^CMake$|^VTK$|^ParaView$|^ITK$)/ {print $0}')
	dirs=$(ls -l |awk '/^d/ {print $NF}' | awk '$0 ~ /(^VTK$)/ {print $0}')
	pull $dirs
	# We need VTKData ParaViewData wikiexamples ITKApps
	# dirs=$(ls -l |awk '/^d/ {print $NF}' | awk '$0 ~ /(^VTKData$|^VTKLargeData$|^ParaViewData$|^wikiexamples$|^ITKApps$)/ {print $0}')
	dirs=$(ls -l |awk '/^d/ {print $NF}' | awk '$0 ~ /(^VTKData$|^VTKLargeData$|^WikiExamples$)/ {print $0}')
	pull_data $dirs
}

main
echo "Finished pulling."

This script will pull VTK etc. into the directories of the same name. Thus automatically updating any changed files.

To use it go to %KITWARE_PATH% in explorer and right-click on src selecting Git Bash Here, when the window opens, type:

./pull.sh

This script is valid for any linux system and also for Mac OS.

Create the following batch file calling it build.cmd and putting it in %KITWARE_PATH%\build:

echo off

set SETENV_CMD="c:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\setenv.cmd"
if not exist %SETENV_CMD% goto missing
call %SETENV_CMD% /%1

SET FP=%KITWARE_PATH%\build
SET RES=build_log
SET FPRES="%FP%\%RES%"
if not exist "%FPRES%\NUL" md "%FPRES%"
del /Q "%FPRES%\*.log"

SET VTK_DEBUG_CONFIG="VTK.sln" /property:Configuration=Debug /target:ALL_BUILD /maxcpucount:%NUMBER_OF_PROCESSORS% /DetailedSummary
SET VTK_RELEASE_CONFIG="VTK.sln" /property:Configuration=Release /target:ALL_BUILD /maxcpucount:%NUMBER_OF_PROCESSORS% /DetailedSummary
rem For some reason we need to use the project file instead of WikiExamples.sln. 
rem MSBuild is not parsing the BUILD_ALL target in WikiExamples.sln as it does in VTK.
SET WEG_DEBUG_CONFIG="ALL_BUILD.vcxproj" /property:Configuration=Debug /maxcpucount:%NUMBER_OF_PROCESSORS% /DetailedSummary
SET WEG_RELEASE_CONFIG="ALL_BUILD.vcxproj" /property:Configuration=Release /maxcpucount:%NUMBER_OF_PROCESSORS% /DetailedSummary

echo "Building VTK Debug"
cd VTK\Debug
MSBuild %VTK_DEBUG_CONFIG% > "%FPRES%\VTKDebug.log"
cd ..\..

echo "Building VTK Release"
cd VTK\Release
MSBuild %VTK_RELEASE_CONFIG% > "%FPRES%\VTKRelease.log"
cd ..\..

echo "Building WikiExamples Debug"
cd WikiExamples\Debug
MSBuild %WEG_DEBUG_CONFIG% > "%FPRES%\WikiExamplesDebug.log"
cd ..\..

echo "Building WikiExamples Release"
cd WikiExamples\Release
MSBuild %WEG_RELEASE_CONFIG% > "%FPRES%\WikiExamplesRelease.log"
cd ..\..

echo "Finished"
goto :eof

:missing
echo Missing file
echo "%SETENV_CMD%"
goto :eof

Remember to set SETENV_CMD so that setenv.cmd is found. FP points to the the build root directory and is used to create a path to the logging directory.

Notice

1. That build.cmd does not specify which build. We will create a link to this file to do this. The builds available are:

   x86

   x64

   ia64

2. The *_CONFIG variables set the solution file or the project file and the options for MSBuild. Feel free to experiment with the options!

We are going to do a 64-bit build so we will create a shortcut to code.cmd and pass it the x64 parameter. Now create a shortcut to code.cmd in %KITWARE_PATH%. For the following we are assuming that we will be building 64-bit code.

Rename the shortcut to build_x64 right-click on it, selecting Properties and edit the target line to read:

%comspec% /k ""%KITWARE_PATH%\build\build.cmd""  x64

Edit the start in line to read:

%KITWARE_PATH%\build

At this point we have everything necessary to invoke 64-bit builds. However we must first set the necessary environment variables and run CMake in each of the Debug and Release directories.

Running CMake

Use the cmake-gui

Note: Substitute for %KITWARE_PATH% the actual path you used e.g. C:/Kitware or C:/Users/<your username>/Code/Kitware or whatever you selected.

VTK

Select %KITWARE_PATH%/src/VTK as the source code and %KITWARE_PATH%/build/VTK/Debug as where to build the debug binaries.

Select Visual Studio 10 Win64 and then the other options as required, then configure and generate.

Note that we selected Visual Studio 10 Win64 because we are using Visual Studio 10 x64 tools.

• In the case of a debug build for VTK, set:

  CMAKE_CONFIGURATION_TYPE to be Debug.

• In the case of a release build for VTK, set:

  CMAKE_CONFIGURATION_TYPE to be Release.

Proceed in a similar fashion for the release build, setting %KITWARE_PATH%/build/VTK/Release as where to build the release binaries. The rationale here is that we only have a debug build in the debug tree and a release build in the release tree.

WikiExamples

Do the same for WikiExamples remembering to change paths and setting VTK_DIR and CMAKE_CONFIGURATION_TYPE as appropriate for the build type. The source code path will be %KITWARE_PATH%\src\WikiExamples and the binaries path will be either %KITWARE_PATH%\build\WikiExamples\Debug or %KITWARE_PATH%\build\WikiExamples\Release.

• In the case of a debug build for WikiExamples, set:

  VTK_DIR to %KITWARE_PATH%/build/VTK/Debug

  CMAKE_CONFIGURATION_TYPE to be Debug.

• In the case of a release build for VTK, set:

  VTK_DIR to %KITWARE_PATH%/build/VTK/Release

  CMAKE_CONFIGURATION_TYPE to be Release.

Pull and Build

1. Go to %KITWARE_PATH% in explorer and right-click on src selecting Git Bash Here, when the window opens, type:

   ./pull.sh

2. Go to %KITWARE_PATH% in explorer and double-click on the link:

   build-x64

Log files will be found in:

%KITWARE_PATH%\src\log

%KITWARE_PATH%\build\log

Repeat these two steps whenever you want to rebuild VTK and WikiExamples.

Building your own Code

In the following discussion, substitute for %KITWARE_PATH% the actual path you used e.g. C:/Kitware or C:/Users/<your username>/Code/Kitware or whatever you selected.

We are assuming the environment variables have been set up as above. Proceed in a manner similar to the WikiExamples as described above. When you run CMake make sure that Visual Studio 10 Win64 is selected and that:

• In the case of a debug build, set:

  VTK_DIR to %KITWARE_PATH%/build/VTK/Debug

• In the case of a release build, set:

  VTK_DIR to %KITWARE_PATH%/build/VTK/Release

Normally the path to the release dlls will be made available to the system in the PATH environment variable:

%KITWARE_PATH%\build\VTK\Release\bin\Release

However for the debug build you should change this to:

%KITWARE_PATH%\build\VTK\Debug\bin\Debug

Or alternatively copy the VTK debug versions of the dlls into the same directory as the debug executable. This can be automatically done using CMake.