digitalmars.D.learn - Help Finding Strange Memory Bug (Code linked to C library)
- Craig Dillabaugh (40/40) Dec 07 2013 Hello,
- =?UTF-8?B?UsOpbXkgTW91w6t6YQ==?= (28/62) Dec 07 2013 My strategy here would be to:
- =?UTF-8?B?QWxpIMOHZWhyZWxp?= (7/9) Dec 07 2013 Great sleuthing! :)
- Craig Dillabaugh (7/15) Dec 08 2013 If by "C's Biggest Mistake" you mean "Craig's Biggest Mistake"
- Craig Dillabaugh (10/105) Dec 08 2013 Good catch. Now that you pointed it out I cannot for the life of
- =?UTF-8?B?QWxpIMOHZWhyZWxp?= (3/4) Dec 08 2013 D has already ruined your mind! :p
Hello, I recently wrote bindings to the C-library Shapelib (it reads/writes a common file format used in Geographic Information Systems). I've been trying to write a small test program to make sure my bindings 'work' and I've come across a bizarre memory bug. I THINK I've identified the code that causes the problem, but I have no idea why. My test-suite includes this function: void shapeRead(string filename) { SHPHandle hShp = SHPOpen( std.string.toStringz( filename ), "rb" ); int n, shp_type; double pad_min_bound, pad_max_bound; SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, &pad_max_bound); SHPClose( hShp ); } If I comment out the SHPGetInfo call, then the segmentation fault doesn't happen, but if its there then the program segfaults AFTER the shapeRead function exits (eg. SHPClose runs fine) ). In fact if I call SHPGetInfo as follows, the crash doesn't occur: SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, null); //NULL pointer last arg. So in C the function SHPGetInfo is: void SHPAPI_CALL SHPGetInfo( SHPHandle hSHP, int * pnEntities, int * pnShapeType, double * padfMinBound, double * padfMaxBound ); While my D binding is (pretty much the same): extern( C ) void SHPGetInfo( SHPHandle hSHP, int* pnEntities, int* pnShapeType, double* padfMinBound, double* padfMaxBound ); I have no idea what is going on. The sizes of ints and doubles are 4 and 8 bytes using gcc on my system (which is how I compiled my C library) so those match the sizes of the corresponding D types [I thought maybe there was a type-size mismatch and that was causing something to be overwritten, but it doesn't appear that way]. Any hints on where to look next would be appreciated. Craig
Dec 07 2013
My strategy here would be to: A. run the program in a debugger, say GDB, to get a exhaustive stacktrace for hints about where to look at. B. have a quick look at the library directly (the "Use the Source Luke" strategy). Since I was curious about your problem (you had everything correct - this should not fail), and have no access to your code, I checked out the shapelib code from its cvs public repository and found out that the last pointer, `double * padfMaxBound` is actually a pointer to an array of 4 elements: in shapelib/shpopen.c: SHPGetInfo: starting line 823: for( i = 0; i < 4; i++ ) { if( padfMinBound != NULL ) padfMinBound[i] = psSHP->adBoundsMin[i]; if( padfMaxBound != NULL ) padfMaxBound[i] = psSHP->adBoundsMax[i]; } This also explains why it does not segfault when you pass a null pointer: no array out of bounds happen then. padfMaxBound should point to a `double []` or `double [4]` instead of a mere `double`. I also ended checking the API documentation (http://shapelib.maptools.org/shp_api.html) and they do document that padfMinBound and padfMaxbound are pointers to 4 values X, Y, Z and M in a four entry array. You may want to check if there isn't any other of those "C tricks" hiding in your D bindings. On 12/07/2013 04:29 PM, Craig Dillabaugh wrote:Hello, I recently wrote bindings to the C-library Shapelib (it reads/writes a common file format used in Geographic Information Systems). I've been trying to write a small test program to make sure my bindings 'work' and I've come across a bizarre memory bug. I THINK I've identified the code that causes the problem, but I have no idea why. My test-suite includes this function: void shapeRead(string filename) { SHPHandle hShp = SHPOpen( std.string.toStringz( filename ), "rb" ); int n, shp_type; double pad_min_bound, pad_max_bound; SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, &pad_max_bound); SHPClose( hShp ); } If I comment out the SHPGetInfo call, then the segmentation fault doesn't happen, but if its there then the program segfaults AFTER the shapeRead function exits (eg. SHPClose runs fine) ). In fact if I call SHPGetInfo as follows, the crash doesn't occur: SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, null); //NULL pointer last arg. So in C the function SHPGetInfo is: void SHPAPI_CALL SHPGetInfo( SHPHandle hSHP, int * pnEntities, int * pnShapeType, double * padfMinBound, double * padfMaxBound ); While my D binding is (pretty much the same): extern( C ) void SHPGetInfo( SHPHandle hSHP, int* pnEntities, int* pnShapeType, double* padfMinBound, double* padfMaxBound ); I have no idea what is going on. The sizes of ints and doubles are 4 and 8 bytes using gcc on my system (which is how I compiled my C library) so those match the sizes of the corresponding D types [I thought maybe there was a type-size mismatch and that was causing something to be overwritten, but it doesn't appear that way]. Any hints on where to look next would be appreciated. Craig
Dec 07 2013
On 12/07/2013 03:11 PM, Rémy Mouëza wrote:the last pointer, `double * padfMaxBound` is actually a pointer to anarrayof 4 elements:Great sleuthing! :) This thread is a good example of "C's Biggest Mistake": http://www.drdobbs.com/architecture-and-design/cs-biggest-mistake/228701625 Ali
Dec 07 2013
On Saturday, 7 December 2013 at 23:35:49 UTC, Ali Çehreli wrote:On 12/07/2013 03:11 PM, Rémy Mouëza wrote:If by "C's Biggest Mistake" you mean "Craig's Biggest Mistake" you are incorrect. As hard as it may be to imagine I've done even dumber things:o) It is too bad that when I do do something dumb I tend to post about it on the internet! Cheersthe last pointer, `double * padfMaxBound` is actually apointer to an arrayof 4 elements:Great sleuthing! :) This thread is a good example of "C's Biggest Mistake": http://www.drdobbs.com/architecture-and-design/cs-biggest-mistake/228701625 Ali
Dec 08 2013
On Saturday, 7 December 2013 at 23:11:39 UTC, Rémy Mouëza wrote:My strategy here would be to: A. run the program in a debugger, say GDB, to get a exhaustive stacktrace for hints about where to look at. B. have a quick look at the library directly (the "Use the Source Luke" strategy). Since I was curious about your problem (you had everything correct - this should not fail), and have no access to your code, I checked out the shapelib code from its cvs public repository and found out that the last pointer, `double * padfMaxBound` is actually a pointer to an array of 4 elements: in shapelib/shpopen.c: SHPGetInfo: starting line 823: for( i = 0; i < 4; i++ ) { if( padfMinBound != NULL ) padfMinBound[i] = psSHP->adBoundsMin[i]; if( padfMaxBound != NULL ) padfMaxBound[i] = psSHP->adBoundsMax[i]; } This also explains why it does not segfault when you pass a null pointer: no array out of bounds happen then. padfMaxBound should point to a `double []` or `double [4]` instead of a mere `double`. I also ended checking the API documentation (http://shapelib.maptools.org/shp_api.html) and they do document that padfMinBound and padfMaxbound are pointers to 4 values X, Y, Z and M in a four entry array. You may want to check if there isn't any other of those "C tricks" hiding in your D bindings.Good catch. Now that you pointed it out I cannot for the life of me figure out why I didn't think to check that! To make matters worse I didn't even have to go check the CVS, because I compiled the library from source and had all the C code sitting just a couple of 'cd's away. For some reason it never occurred to me that it might be expecting an array - even though one might reasonably assume that the bounds on a multidimensional dataset would have more than one dimension. Thank you.On 12/07/2013 04:29 PM, Craig Dillabaugh wrote:Hello, I recently wrote bindings to the C-library Shapelib (it reads/writes a common file format used in Geographic Information Systems). I've been trying to write a small test program to make sure my bindings 'work' and I've come across a bizarre memory bug. I THINK I've identified the code that causes the problem, but I have no idea why. My test-suite includes this function: void shapeRead(string filename) { SHPHandle hShp = SHPOpen( std.string.toStringz( filename ), "rb" ); int n, shp_type; double pad_min_bound, pad_max_bound; SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, &pad_max_bound); SHPClose( hShp ); } If I comment out the SHPGetInfo call, then the segmentation fault doesn't happen, but if its there then the program segfaults AFTER the shapeRead function exits (eg. SHPClose runs fine) ). In fact if I call SHPGetInfo as follows, the crash doesn't occur: SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, null); //NULL pointer last arg. So in C the function SHPGetInfo is: void SHPAPI_CALL SHPGetInfo( SHPHandle hSHP, int * pnEntities, int * pnShapeType, double * padfMinBound, double * padfMaxBound ); While my D binding is (pretty much the same): extern( C ) void SHPGetInfo( SHPHandle hSHP, int* pnEntities, int* pnShapeType, double* padfMinBound, double* padfMaxBound ); I have no idea what is going on. The sizes of ints and doubles are 4 and 8 bytes using gcc on my system (which is how I compiled my C library) so those match the sizes of the corresponding D types [I thought maybe there was a type-size mismatch and that was causing something to be overwritten, but it doesn't appear that way]. Any hints on where to look next would be appreciated. Craig
Dec 08 2013
On 12/08/2013 12:16 AM, Craig Dillabaugh wrote:I cannot for the life of me figure out why I didn't think to check that!D has already ruined your mind! :p Ali
Dec 08 2013