[vtk-developers] VTK ExternalData default object store (was: vtk_common dashboard script)

[David Cole]

-----Original Message-----
From: Brad King <brad.king at kitware.com>
To: David Cole <dlrdave at aol.com>
Cc: vtk-developers <vtk-developers at vtk.org>
Sent: Tue, May 21, 2013 10:48 am
Subject: Re: [vtk-developers] VTK ExternalData default object store 
(was: vtk_common dashboard script)


On 05/21/2013 09:35 AM, Marcus D. Hanwell wrote:
> The expectation is also that data is persistent if we don't change
> the default perhaps we can at least add a warning at configure time to
> advise that data will be downloaded multiple times (the default is
> each build tree?) I would love to see a default parallel to the source
> tree, like VTKExternalData or something, but it seems like you are
> strongly opposed to that.

My original statement was about a general expectation of projects
staying isolated in their source/build tree until "make install" time.
CMake used to have tests that touched the $HOME directory and we got
complaints so we changed the tests to run with a custom HOME env var
pointing at the build tree.

On 05/21/2013 10:34 AM, David Cole wrote:
> Additionally, in favor of a parallel sibling directory to the source
> tree... I think of this as a replacement for a VTKData checkout, 
which
> has traditionally always been recommended to be a parallel sibling
> directory.
>
> Wasn't VTK and VTKData in the same parent directory the recommended
> structure? And didn't the common script do a checkout of VTKData such
> that they ended up that way?

Yes, but both of those required the user to do something local besides
checkout and build VTK: either checkout VTKData manually or write a
dashboard script referencing vtk_common.cmake.

> Seems completely reasonable to me to say: the default is *here*, and
> it's a cache setting... if you want it somewhere else, change the 
cache
> setting.

That requires reading the documentation.  A user or automatic script
that is not familiar with VTK in particular may not do that.

> What I don't want is *this*:
> to have multiple build trees of VTK going on my laptop, and to
> *accidentally* get multiple copies of VTKData and VTKLargeData just
> because I left something at its default value.

Even with the sibling directory you could still end up with multiple
copies if you have VTK sources in separate directories.

A similar discussion a few years ago in ITK led to this feature:

 http://itk.org/gitweb?p=ITK.git;a=commitdiff;h=d80e341a

Just add

 export ExternalData_OBJECT_STORES=$HOME/.ExternalData

to your .bashrc and all your new ITK and VTK build trees will use
that for persistent data.

-Brad
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://www.vtk.org/mailman/listinfo/vtk-developers




OK.

But with your solution, *everybody* has to do work to achieve 
non-duplication of very large data files when they do multiple builds 
of VTK. *Everybody*.

If you would just use a default value in $HOME or %USERPROFILE% 
instead, then *nobody* would have to to work to achieve non-duplication.

And I thought the whole main point of ExternalData was to download each 
item needed *exactly* *once* and to enable non-duplication of each item.

So I guess I'm puzzled by the default value.

But if that's the way it is, I'll set up my machines so that at least I 
don't have duplicated data files.


Thanks,
D