www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Help Finding Strange Memory Bug (Code linked to C library)

reply "Craig Dillabaugh" <craig.dillabaugh gmail.com> writes:
Hello,
I recently wrote bindings to the C-library Shapelib (it 
reads/writes a common file format used in Geographic Information 
Systems).

I've been trying to write a small test program to make sure my 
bindings 'work' and I've come across a bizarre memory bug.  I  
THINK I've identified the code that causes the problem, but I 
have no idea why.

My test-suite includes this function:

void shapeRead(string filename) {
   SHPHandle hShp = SHPOpen( std.string.toStringz( filename ), 
"rb" );

   int n, shp_type;
   double pad_min_bound, pad_max_bound;

   SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, 
&pad_max_bound);

   SHPClose( hShp );
}

If I comment out the SHPGetInfo call, then the segmentation fault 
doesn't happen, but if its there then the program segfaults AFTER 
the shapeRead function exits (eg. SHPClose runs fine) ).

In fact if I call SHPGetInfo as follows, the crash doesn't occur:
SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, null); //NULL 
pointer last arg.

So in C the function SHPGetInfo is:

void SHPAPI_CALL
       SHPGetInfo( SHPHandle hSHP, int * pnEntities, int * 
pnShapeType,
                   double * padfMinBound, double * padfMaxBound );

While my D binding is (pretty much the same):

extern( C ) void SHPGetInfo( SHPHandle hSHP, int* pnEntities,
	  int* pnShapeType, double* padfMinBound, double* padfMaxBound );


I have no idea what is going on.  The sizes of ints and doubles 
are 4 and 8 bytes using gcc on my system (which is how I compiled 
my C library) so those match the sizes of the corresponding D 
types [I thought maybe there was a type-size mismatch and that 
was causing something to be overwritten, but it doesn't appear 
that way].

Any hints on where to look next would be appreciated.

Craig
Dec 07 2013
parent reply =?UTF-8?B?UsOpbXkgTW91w6t6YQ==?= <remy.moueza gmail.com> writes:
My strategy here would be to:
A.  run the program in a debugger, say GDB, to get a exhaustive 
stacktrace for hints about where to look at.
B. have a quick look at the library directly (the "Use the Source Luke" 
strategy).

Since I was curious about your problem (you had everything correct - 
this should not fail), and have no access to your code, I checked out 
the shapelib code from its cvs public repository and found out that the 
last pointer, `double * padfMaxBound` is actually a pointer to an array 
of 4 elements: in shapelib/shpopen.c: SHPGetInfo: starting line 823:
     for( i = 0; i < 4; i++ )
     {
         if( padfMinBound != NULL )
             padfMinBound[i] = psSHP->adBoundsMin[i];
         if( padfMaxBound != NULL )
             padfMaxBound[i] = psSHP->adBoundsMax[i];
     }

This also explains why it does not segfault when you pass a null 
pointer: no array out of bounds happen then.

padfMaxBound should point to a `double []` or `double [4]` instead of a 
mere `double`.

I also ended checking the API documentation 
(http://shapelib.maptools.org/shp_api.html) and they do document that 
padfMinBound and padfMaxbound are pointers to 4 values X, Y, Z and M in 
a four entry array.
You may want to check if there isn't any other of those "C tricks" 
hiding in your D bindings.



On 12/07/2013 04:29 PM, Craig Dillabaugh wrote:
 Hello,
 I recently wrote bindings to the C-library Shapelib (it reads/writes a
 common file format used in Geographic Information Systems).

 I've been trying to write a small test program to make sure my bindings
 'work' and I've come across a bizarre memory bug.  I THINK I've
 identified the code that causes the problem, but I have no idea why.

 My test-suite includes this function:

 void shapeRead(string filename) {
    SHPHandle hShp = SHPOpen( std.string.toStringz( filename ), "rb" );

    int n, shp_type;
    double pad_min_bound, pad_max_bound;

    SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, &pad_max_bound);

    SHPClose( hShp );
 }

 If I comment out the SHPGetInfo call, then the segmentation fault
 doesn't happen, but if its there then the program segfaults AFTER the
 shapeRead function exits (eg. SHPClose runs fine) ).

 In fact if I call SHPGetInfo as follows, the crash doesn't occur:
 SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, null); //NULL pointer
 last arg.

 So in C the function SHPGetInfo is:

 void SHPAPI_CALL
        SHPGetInfo( SHPHandle hSHP, int * pnEntities, int * pnShapeType,
                    double * padfMinBound, double * padfMaxBound );

 While my D binding is (pretty much the same):

 extern( C ) void SHPGetInfo( SHPHandle hSHP, int* pnEntities,
        int* pnShapeType, double* padfMinBound, double* padfMaxBound );


 I have no idea what is going on.  The sizes of ints and doubles are 4
 and 8 bytes using gcc on my system (which is how I compiled my C
 library) so those match the sizes of the corresponding D types [I
 thought maybe there was a type-size mismatch and that was causing
 something to be overwritten, but it doesn't appear that way].

 Any hints on where to look next would be appreciated.

 Craig
Dec 07 2013
next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 12/07/2013 03:11 PM, Rémy Mouëza wrote:

 the last pointer, `double * padfMaxBound` is actually a pointer to an 
array
 of 4 elements:
Great sleuthing! :) This thread is a good example of "C's Biggest Mistake": http://www.drdobbs.com/architecture-and-design/cs-biggest-mistake/228701625 Ali
Dec 07 2013
parent "Craig Dillabaugh" <cdillaba cg.scs.carleton.ca> writes:
On Saturday, 7 December 2013 at 23:35:49 UTC, Ali Çehreli wrote:
 On 12/07/2013 03:11 PM, Rémy Mouëza wrote:

 the last pointer, `double * padfMaxBound` is actually a
pointer to an array
 of 4 elements:
Great sleuthing! :) This thread is a good example of "C's Biggest Mistake": http://www.drdobbs.com/architecture-and-design/cs-biggest-mistake/228701625 Ali
If by "C's Biggest Mistake" you mean "Craig's Biggest Mistake" you are incorrect. As hard as it may be to imagine I've done even dumber things:o) It is too bad that when I do do something dumb I tend to post about it on the internet! Cheers
Dec 08 2013
prev sibling parent reply "Craig Dillabaugh" <cdillaba cg.scs.carleton.ca> writes:
On Saturday, 7 December 2013 at 23:11:39 UTC, Rémy Mouëza wrote:
 My strategy here would be to:
 A.  run the program in a debugger, say GDB, to get a exhaustive 
 stacktrace for hints about where to look at.
 B. have a quick look at the library directly (the "Use the 
 Source Luke" strategy).

 Since I was curious about your problem (you had everything 
 correct - this should not fail), and have no access to your 
 code, I checked out the shapelib code from its cvs public 
 repository and found out that the last pointer, `double * 
 padfMaxBound` is actually a pointer to an array of 4 elements: 
 in shapelib/shpopen.c: SHPGetInfo: starting line 823:
     for( i = 0; i < 4; i++ )
     {
         if( padfMinBound != NULL )
             padfMinBound[i] = psSHP->adBoundsMin[i];
         if( padfMaxBound != NULL )
             padfMaxBound[i] = psSHP->adBoundsMax[i];
     }

 This also explains why it does not segfault when you pass a 
 null pointer: no array out of bounds happen then.

 padfMaxBound should point to a `double []` or `double [4]` 
 instead of a mere `double`.

 I also ended checking the API documentation 
 (http://shapelib.maptools.org/shp_api.html) and they do 
 document that padfMinBound and padfMaxbound are pointers to 4 
 values X, Y, Z and M in a four entry array.
 You may want to check if there isn't any other of those "C 
 tricks" hiding in your D bindings.
Good catch. Now that you pointed it out I cannot for the life of me figure out why I didn't think to check that! To make matters worse I didn't even have to go check the CVS, because I compiled the library from source and had all the C code sitting just a couple of 'cd's away. For some reason it never occurred to me that it might be expecting an array - even though one might reasonably assume that the bounds on a multidimensional dataset would have more than one dimension. Thank you.
 On 12/07/2013 04:29 PM, Craig Dillabaugh wrote:
 Hello,
 I recently wrote bindings to the C-library Shapelib (it 
 reads/writes a
 common file format used in Geographic Information Systems).

 I've been trying to write a small test program to make sure my 
 bindings
 'work' and I've come across a bizarre memory bug.  I THINK I've
 identified the code that causes the problem, but I have no 
 idea why.

 My test-suite includes this function:

 void shapeRead(string filename) {
   SHPHandle hShp = SHPOpen( std.string.toStringz( filename ), 
 "rb" );

   int n, shp_type;
   double pad_min_bound, pad_max_bound;

   SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, 
 &pad_max_bound);

   SHPClose( hShp );
 }

 If I comment out the SHPGetInfo call, then the segmentation 
 fault
 doesn't happen, but if its there then the program segfaults 
 AFTER the
 shapeRead function exits (eg. SHPClose runs fine) ).

 In fact if I call SHPGetInfo as follows, the crash doesn't 
 occur:
 SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, null); //NULL 
 pointer
 last arg.

 So in C the function SHPGetInfo is:

 void SHPAPI_CALL
       SHPGetInfo( SHPHandle hSHP, int * pnEntities, int * 
 pnShapeType,
                   double * padfMinBound, double * padfMaxBound 
 );

 While my D binding is (pretty much the same):

 extern( C ) void SHPGetInfo( SHPHandle hSHP, int* pnEntities,
       int* pnShapeType, double* padfMinBound, double* 
 padfMaxBound );


 I have no idea what is going on.  The sizes of ints and 
 doubles are 4
 and 8 bytes using gcc on my system (which is how I compiled my 
 C
 library) so those match the sizes of the corresponding D types 
 [I
 thought maybe there was a type-size mismatch and that was 
 causing
 something to be overwritten, but it doesn't appear that way].

 Any hints on where to look next would be appreciated.

 Craig
Dec 08 2013
parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 12/08/2013 12:16 AM, Craig Dillabaugh wrote:

 I cannot for the life of me figure out why I didn't think to check that!
D has already ruined your mind! :p Ali
Dec 08 2013