www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Having a strange issue with std.net.curl.HTTP as a struct dependency

reply NoBigDeal256 <nobigdeal256 gmail.com> writes:
I'm currently learning D and started working on one of my first 
projects which is an API wrapper. I'm currently having an issue 
with my program getting a InvalidMemoryOperationError upon 
exiting the process on Windows 7. On my Debian VM I get a 
segmentation fault.

I have tried to minimize the code as much as I possibly can while 
still reproducing the error. Here is the code:

import std.net.curl;

struct ThingA {

     HTTP http;

     this(HTTP http) {
         this.http = http;

         arrayOfThingBs();
     }

     ThingB[] arrayOfThingBs() {
         ThingB[] thingBs;

         thingBs ~= ThingB(this);

         return thingBs;
     }

}

struct ThingB {

     ThingA thingA;

     this(ThingA thingA) {
         this.thingA = thingA;
     }

}

void main() {
     auto http = HTTP();
     auto thingA = ThingA(http);
}


If I comment out the HTTP dependency like:

struct ThingA {

     //HTTP http;

     this(HTTP http) {
         //this.http = http;

         arrayOfThingBs();
     }

     ThingB[] arrayOfThingBs() {
         ThingB[] thingBs;

         thingBs ~= ThingB(this);

         return thingBs;
     }

}

The error goes away. The error also goes away if 
ThingA.arrayOfThingBs returns a single instance of ThingB instead 
of an array of ThingB. Removing ThingBs dependency on ThingA also 
gets rid of the error. I'm new to low level languages in general 
so maybe I'm doing something wrong, but this seems like really 
strange behavior.
Jul 09
next sibling parent NoBigDeal256 <nobigdeal256 gmail.com> writes:
Still haven't figured out what's wrong. Any help would be 
appreciated.
Jul 10
prev sibling next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
NoBigDeal256 wrote:

 I'm currently learning D and started working on one of my first projects 
 which is an API wrapper. I'm currently having an issue with my program 
 getting a InvalidMemoryOperationError upon exiting the process on Windows 
 7. On my Debian VM I get a segmentation fault.
known bug in curl finalization on program exit. should be fixed in the next release. for now you have to live with it, or build your own phobos.
Jul 10
parent reply NoBigDeal256 <nobigdeal256 gmail.com> writes:
On Monday, 10 July 2017 at 20:31:12 UTC, ketmar wrote:
 NoBigDeal256 wrote:

 I'm currently learning D and started working on one of my 
 first projects which is an API wrapper. I'm currently having 
 an issue with my program getting a InvalidMemoryOperationError 
 upon exiting the process on Windows 7. On my Debian VM I get a 
 segmentation fault.
known bug in curl finalization on program exit. should be fixed in the next release. for now you have to live with it, or build your own phobos.
Ah. It's interesting that it works perfectly fine when used directly in main, and only becomes an issue when it's a struct member. Do you happen to have a link to the bug report for this specific issue that I could look at? I looked at the known bugs list and couldn't find anything related to this.
Jul 10
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
NoBigDeal256 wrote:

 Do you happen 
 to have a link to the bug report for this specific issue that I could 
 look at? I looked at the known bugs list and couldn't find anything 
 related to this.
nope. it is a kind of "known bug nobody bothered to report". ;-)
Jul 10
parent reply NoBigDeal256 <nobigdeal256 gmail.com> writes:
On Monday, 10 July 2017 at 20:55:19 UTC, ketmar wrote:
 NoBigDeal256 wrote:

 Do you happen to have a link to the bug report for this 
 specific issue that I could look at? I looked at the known 
 bugs list and couldn't find anything related to this.
nope. it is a kind of "known bug nobody bothered to report". ;-)
Okay, so I decided to ditch curl and switch over to dlang-requests, which doesn't depend on curl, and I'm running into this same issue again... I don't think this is a curl issue.
Jul 12
parent reply ikod <geller.garry gmail.com> writes:
On Wednesday, 12 July 2017 at 08:24:12 UTC, NoBigDeal256 wrote:
 On Monday, 10 July 2017 at 20:55:19 UTC, ketmar wrote:
 NoBigDeal256 wrote:

 Do you happen to have a link to the bug report for this 
 specific issue that I could look at? I looked at the known 
 bugs list and couldn't find anything related to this.
nope. it is a kind of "known bug nobody bothered to report". ;-)
Okay, so I decided to ditch curl and switch over to dlang-requests, which doesn't depend on curl, and I'm running into this same issue again... I don't think this is a curl issue.
looks like circular dependency between thingA and thingB lead to problems on exit. this code fails on osx too.
Jul 12
parent ketmar <ketmar ketmar.no-ip.org> writes:
ikod wrote:

 On Wednesday, 12 July 2017 at 08:24:12 UTC, NoBigDeal256 wrote:
 On Monday, 10 July 2017 at 20:55:19 UTC, ketmar wrote:
 NoBigDeal256 wrote:

 Do you happen to have a link to the bug report for this specific issue 
 that I could look at? I looked at the known bugs list and couldn't 
 find anything related to this.
nope. it is a kind of "known bug nobody bothered to report". ;-)
Okay, so I decided to ditch curl and switch over to dlang-requests, which doesn't depend on curl, and I'm running into this same issue again... I don't think this is a curl issue.
looks like circular dependency between thingA and thingB lead to problems on exit. this code fails on osx too.
yet at least with curl there was definitely a bug with "slightly too early curl shutdown".
Jul 12
prev sibling next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 10 July 2017 at 00:10:36 UTC, NoBigDeal256 wrote:
 The error goes away. The error also goes away if 
 ThingA.arrayOfThingBs returns a single instance of ThingB 
 instead of an array of ThingB.
This tells me the problem is in the collection order at the end of the program. The HTTP inside the struct inside the array gets cleaned up by the garbage collector (aka GC) at program termination... and it might be doing that AFTER unloading curl, leading to the segfault, OR it could be running some GC operation in the destructor, leading to InvalidMemoryOperationError (calling a GC function while the GC is running is not allowed). Which you get is apparently dependent on which system you are on, but both are wrong. If it isn't in the array, the destructor gets called at scope exit, which happens before program termination and outside the GC, so everything happens as it is supposed to. HTTP's innards have a RefCounted implementation which has a destructor that calls GC.removeRange. I *think* that can do the invalid mem operation. And the implementation itself can do curl_shutdown, which is your potential segfault. So I'd say the answer is prolly to keep HTTP away from the GC. Manually free its arrays, or keep them outside arrays in the first place.
Jul 12
parent crimaniak <crimaniak gmail.com> writes:
On Wednesday, 12 July 2017 at 13:47:06 UTC, Adam D. Ruppe wrote:
 This tells me the problem is in the collection order at the end 
 of the program.
...
 So I'd say the answer is prolly to keep HTTP away from the GC. 
 Manually free its arrays, or keep them outside arrays in the 
 first place.
I'd try also calling GC.collect() manually before the exit.
Jul 12
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 07/09/2017 05:10 PM, NoBigDeal256 wrote:

     HTTP http;
For what it's worth, defining it as __gshared seems to be a workaround: __gshared HTTP http; Ali
Jul 13
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 07/13/2017 12:01 AM, Ali Çehreli wrote:
 On 07/09/2017 05:10 PM, NoBigDeal256 wrote:

     HTTP http;
For what it's worth, defining it as __gshared seems to be a workaround: __gshared HTTP http;
I sometimes amuse myself. What use is that? :o) What I must have meant is you can have a __gshared array of those and pass pointers to elements to constructors: __gshared HTTP[] https; struct ThingA { HTTP *http; this(HTTP *http) { // ... Not easy but a workaround... Ali
Jul 13