View Issue Details Jump to Notes ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0004931VTK(No Category)public2007-04-25 10:582014-09-15 12:14
ReporterPaul Melis 
Assigned ToDavid Gobbi 
PrioritylowSeverityminorReproducibilityalways
StatusclosedResolutionno change required 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version5.10.0 
Summary0004931: Python wrappers: input/output to and from string broken
Description(This is using a recent (April 2007) CVS version)

The VTK classes vtkDataWriter and vtkDataReader provide methods for writing and reading to/from a string, instead of the usual file. These are really handy for situations where a dataset isn't supposed or doesn't need to hit the disk (like when transferring over a network).

Although VTK's Python wrappers do make the necessary methods available to Python scripts, they are not usable in their current form. This is because embedded NULL characters in input/output strings aren't correctly handled.

Specifically:
* vtkDataSetWriter.GetOutputString will return a Python string _up to the first \x00 in the output_, which for a binary VTK file, is somewhere after the header.
* Working around the limitation by setting the file type to ASCII (so no embedded NULLs are in the output string) usually fails in vtkDataSetWriter.Write(). This seems to be related to the amount of space reserved for the output string.
* vtkDataSetReader.Set(Binary)InputString does not allow to pass a Python string containing an embedded NULL character.
* A work-around exists by passing a vtkCharArray instead of a string for the previous point, but this is really inefficient

The problems at the VTK <-> Python boundary come from the specific Python C/API functions used to handle the string marshalling.
TagsNo tags attached.
Project
Type
Attached Files? file icon write_sphere_to_string.py [^] (1,715 bytes) 1969-12-31 19:00

 Relationships

  Notes
(0027030)
Daniel Lea (reporter)
2011-07-20 12:58

I'd like to note that for me (at least in VTK 5.2 which was probably when I last checked), the ASCII output string received by Python has some variable amount of junk appended to it, but the method works and the data is there. The workaround here is to use the VTK-reported string length to remove the junk:

stringLength = gridWriter.GetOutputStringLength()
string = gridWriter.GetOutputString()[0:stringLength]

I'll also note that I'd still like to be able to access binary string outputs in Python. *bump*
(0027031)
David Gobbi (developer)
2011-07-20 13:27
edited on: 2011-07-20 13:28

Daniel, your solution will not work if the output string contains null bytes. If it does, then the python string will be terminated early, and using [0:stringLength] will not help in that situation.

The problem with vtkDataReader/vtkDataWriter is the method signatures. The vtkDataReader should use this signature, instead of SetBinaryInputString:

SetInputVoidPointer(void *input, int len);

The vtkAbstractArray and vtkImageImport have similar void pointer methods for reading raw data and they work fine.

The python wrappers always treat "char *" args as null-terminated strings and "void *" args as binary data, and that's not something that I'm willing to change. All the other wrapper languages similarly expect that "char *" always points to a null-terminated string.

HOWEVER, if someone submits a properly tested patch that adds void pointer methods to vtkDataReader and vtkDataWriter, I'll consider incorporating them. Use vtkAbstractArray as a reference.

(0027034)
Daniel Lea (reporter)
2011-07-20 15:18

Yes, I was actually pointing out what is essentially a different bug that is nonetheless related to the original contents of this one, i.e. that associated with the *ASCII* string output. I understand that this is not a workaround for the binary string problem, which is something I would still love to see fixed.
(0033342)
David Gobbi (developer)
2014-09-15 12:14

As stated in the notes for this bug, GetOutputString() returns a "char *", and out of necessity the python wrappers always interpret a "char *" as a null-terminated string. Therefore, "char *" should not be used as storage for binary data.

In Sept 2011, commit 85664908 added a GetOutputStdString() method to vtkDataWriter. A std::string is a safe container for binary data and will be properly interpreted by the wrappers. Therefore, I consider this issue to be resolved.

 Issue History
Date Modified Username Field Change
2011-02-26 10:42 David Gobbi Assigned To Will Schroeder => David Gobbi
2011-06-16 13:11 Zack Galbreath Category => (No Category)
2011-07-20 12:58 Daniel Lea Note Added: 0027030
2011-07-20 13:27 David Gobbi Note Added: 0027031
2011-07-20 13:28 David Gobbi Note Edited: 0027031
2011-07-20 15:18 Daniel Lea Note Added: 0027034
2014-09-15 12:14 David Gobbi Note Added: 0033342
2014-09-15 12:14 David Gobbi Status backlog => closed
2014-09-15 12:14 David Gobbi Resolution open => no change required
2014-09-15 12:14 David Gobbi Fixed in Version => 5.10.0


Copyright © 2000 - 2018 MantisBT Team