www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Are Fibers just broken in D?

reply Byron Moxie <bheads moxiegroup.com> writes:
I am working on an application that makes heavy use of Fibers 
pools that are processed by a thread pool.

In WIN32 it looks like its leaking memory then it segfaults (I am 
guessing its an OOM).

I am also trying to use Semaphores and/or Conditions on shared 
data between fibers, which results in segfaults as well.

atomicLoad and atomicStore are not working either.

Are Fibers usable at all?  Cause it doesn't seem like.  I already 
re-arched this application to remove vibe.d (was fighting all 
sorts of sync and event errors)

This application is turning into my last straw for D..  I spend 
about 90% of my time fighting the compiler / library. I am really 
debating re-writing this application in c++ over the weekend (I 
have already lost money on this job after convincing by boss that 
D would be better).
Apr 20 2018
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 21/04/2018 6:58 AM, Byron Moxie wrote:

 I am working on an application that makes heavy use of Fibers pools that 
 are processed by a thread pool.
 
 In WIN32 it looks like its leaking memory then it segfaults (I am 
 guessing its an OOM).
 
 I am also trying to use Semaphores and/or Conditions on shared data 
 between fibers, which results in segfaults as well.
 
 atomicLoad and atomicStore are not working either.
 
 Are Fibers usable at all?  Cause it doesn't seem like.  I already 
 re-arched this application to remove vibe.d (was fighting all sorts of 
 sync and event errors)
 
 This application is turning into my last straw for D..  I spend about 
 90% of my time fighting the compiler / library. I am really debating 
 re-writing this application in c++ over the weekend (I have already lost 
 money on this job after convincing by boss that D would be better).


atomicLoad and atomicStore, definitely are working.

It sounds like you have got some pretty big problems that may not be on 
D's end.

But we'll need code to be able to help you further than that.
Apr 20 2018
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/20/18 2:58 PM, Byron Moxie wrote:

 I am working on an application that makes heavy use of Fibers pools that 
 are processed by a thread pool.
 
 In WIN32 it looks like its leaking memory then it segfaults (I am 
 guessing its an OOM).
 
 I am also trying to use Semaphores and/or Conditions on shared data 
 between fibers, which results in segfaults as well.
 
 atomicLoad and atomicStore are not working either.
 
 Are Fibers usable at all?  Cause it doesn't seem like.  I already 
 re-arched this application to remove vibe.d (was fighting all sorts of 
 sync and event errors)
 
 This application is turning into my last straw for D..  I spend about 
 90% of my time fighting the compiler / library. I am really debating 
 re-writing this application in c++ over the weekend (I have already lost 
 money on this job after convincing by boss that D would be better).


It sounds like the problems may be due to Win32 and not the other 
pieces. Have you tried on a Win64 build? Even if that's not your target, 
at least it can help you discover whether that is the problem or not. 
The DMC runtime is the default on Win32, and it's not especially 
thread-safe in all places.

FWIW, I'm using vibe.d on Linux 64 bit with no problems (and I've NEVER 
heard of atomicLoad and atomicStore not working on any arch).

-Steve
Apr 20 2018
parent reply Byron Moxie <bheads moxiegroup.com> writes:
On Friday, 20 April 2018 at 20:46:20 UTC, Steven Schveighoffer 
wrote:

 On 4/20/18 2:58 PM, Byron Moxie wrote:

 [...]


 It sounds like the problems may be due to Win32 and not the 
 other pieces. Have you tried on a Win64 build? Even if that's 
 not your target, at least it can help you discover whether that 
 is the problem or not. The DMC runtime is the default on Win32, 
 and it's not especially thread-safe in all places.

 FWIW, I'm using vibe.d on Linux 64 bit with no problems (and 
 I've NEVER heard of atomicLoad and atomicStore not working on 
 any arch).

 -Steve


I had move the data I wanted to sync with atomicLoad/Store into a 
shared struct and pass a pointer to this struct to the other 
fibers. Not sure if this was an issue with TLS messing with class 
object I was passing around.
Apr 20 2018
parent reply Byron Heads <bheads moxiegroup.com> writes:
On Friday, 20 April 2018 at 20:52:17 UTC, Byron Moxie wrote:

 On Friday, 20 April 2018 at 20:46:20 UTC, Steven Schveighoffer 
 wrote:

 On 4/20/18 2:58 PM, Byron Moxie wrote:

 [...]


 It sounds like the problems may be due to Win32 and not the 
 other pieces. Have you tried on a Win64 build? Even if that's 
 not your target, at least it can help you discover whether 
 that is the problem or not. The DMC runtime is the default on 
 Win32, and it's not especially thread-safe in all places.

 FWIW, I'm using vibe.d on Linux 64 bit with no problems (and 
 I've NEVER heard of atomicLoad and atomicStore not working on 
 any arch).

 -Steve


 I had move the data I wanted to sync with atomicLoad/Store into 
 a shared struct and pass a pointer to this struct to the other 
 fibers. Not sure if this was an issue with TLS messing with 
 class object I was passing around.




Fibers on Win32 have a memory leak for sure:

import core.thread : Fiber;

void main() {

     foreach(ulong i; 0..99_999) {
         auto foo = new Foo();
         foo.call();
         foo.call();
     }
}


class Foo : Fiber {
     this() {
         super(&run);
     }


     void run() {
         Fiber.yield();
     }
}


Running this with -m64 on windows runs without a problem, but 
with -m32 it failes aith a Memory Allocation failed error.
Apr 23 2018
next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/23/18 8:46 PM, Byron Heads wrote:

 Fibers on Win32 have a memory leak for sure:
 
 import core.thread : Fiber;
 
 void main() {
 
      foreach(ulong i; 0..99_999) {
          auto foo = new Foo();
          foo.call();
          foo.call();
      }
 }


It sure looks like this should be fine, the GC should clean up all the 
unused fibers.


 Running this with -m64 on windows runs without a problem, but with -m32 
 it failes aith a Memory Allocation failed error.


How many loops until you get a failure?

Is there a particular reason you need to use 32-bit Windows?

-Steve
Apr 23 2018
prev sibling parent reply Radu <void null.pt> writes:
On Tuesday, 24 April 2018 at 00:46:39 UTC, Byron Heads wrote:

 On Friday, 20 April 2018 at 20:52:17 UTC, Byron Moxie wrote:

 On Friday, 20 April 2018 at 20:46:20 UTC, Steven Schveighoffer 
 wrote:

 On 4/20/18 2:58 PM, Byron Moxie wrote:

 [...]


 It sounds like the problems may be due to Win32 and not the 
 other pieces. Have you tried on a Win64 build? Even if that's 
 not your target, at least it can help you discover whether 
 that is the problem or not. The DMC runtime is the default on 
 Win32, and it's not especially thread-safe in all places.

 FWIW, I'm using vibe.d on Linux 64 bit with no problems (and 
 I've NEVER heard of atomicLoad and atomicStore not working on 
 any arch).

 -Steve


 I had move the data I wanted to sync with atomicLoad/Store 
 into a shared struct and pass a pointer to this struct to the 
 other fibers. Not sure if this was an issue with TLS messing 
 with class object I was passing around.




 Fibers on Win32 have a memory leak for sure:

 import core.thread : Fiber;

 void main() {

     foreach(ulong i; 0..99_999) {
         auto foo = new Foo();
         foo.call();
         foo.call();
     }
 }


 class Foo : Fiber {
     this() {
         super(&run);
     }


     void run() {
         Fiber.yield();
     }
 }


 Running this with -m64 on windows runs without a problem, but 
 with -m32 it failes aith a Memory Allocation failed error.


This is not a fiber issue but a more memory management issue. 
Your run out of address space on win32, the GC will not always 
collect all those 99999 fibers that you allocate in that loop. As 
an exercise replace `auto` with `scope` like `scope foo = new 
Foo();` in that loop - you should see different results.

The issue boils down to the call to VirtualAlloc that the fiber 
constructor makes, which fails as Windows will not allocate any 
more virtual pages for the process. If you really want that many 
fibers you should reconsider 32 bit, and definitely use a 
different allocation strategy.
Apr 24 2018
parent reply bauss <jj_1337 live.dk> writes:
On Tuesday, 24 April 2018 at 07:58:01 UTC, Radu wrote:

 On Tuesday, 24 April 2018 at 00:46:39 UTC, Byron Heads wrote:

 On Friday, 20 April 2018 at 20:52:17 UTC, Byron Moxie wrote:

 On Friday, 20 April 2018 at 20:46:20 UTC, Steven 
 Schveighoffer wrote:

 On 4/20/18 2:58 PM, Byron Moxie wrote:

 [...]


 It sounds like the problems may be due to Win32 and not the 
 other pieces. Have you tried on a Win64 build? Even if 
 that's not your target, at least it can help you discover 
 whether that is the problem or not. The DMC runtime is the 
 default on Win32, and it's not especially thread-safe in all 
 places.

 FWIW, I'm using vibe.d on Linux 64 bit with no problems (and 
 I've NEVER heard of atomicLoad and atomicStore not working 
 on any arch).

 -Steve


 I had move the data I wanted to sync with atomicLoad/Store 
 into a shared struct and pass a pointer to this struct to the 
 other fibers. Not sure if this was an issue with TLS messing 
 with class object I was passing around.




 Fibers on Win32 have a memory leak for sure:

 import core.thread : Fiber;

 void main() {

     foreach(ulong i; 0..99_999) {
         auto foo = new Foo();
         foo.call();
         foo.call();
     }
 }


 class Foo : Fiber {
     this() {
         super(&run);
     }


     void run() {
         Fiber.yield();
     }
 }


 Running this with -m64 on windows runs without a problem, but 
 with -m32 it failes aith a Memory Allocation failed error.


 This is not a fiber issue but a more memory management issue. 
 Your run out of address space on win32, the GC will not always 
 collect all those 99999 fibers that you allocate in that loop. 
 As an exercise replace `auto` with `scope` like `scope foo = 
 new Foo();` in that loop - you should see different results.

 The issue boils down to the call to VirtualAlloc that the fiber 
 constructor makes, which fails as Windows will not allocate any 
 more virtual pages for the process. If you really want that 
 many fibers you should reconsider 32 bit, and definitely use a 
 different allocation strategy.


And in the end of the day it makes no sense to have that many 
fibers as they would probably perform terrible.
Apr 24 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/24/18 5:11 AM, bauss wrote:

 On Tuesday, 24 April 2018 at 07:58:01 UTC, Radu wrote:

 On Tuesday, 24 April 2018 at 00:46:39 UTC, Byron Heads wrote:

 Fibers on Win32 have a memory leak for sure:

 import core.thread : Fiber;

 void main() {

     foreach(ulong i; 0..99_999) {
         auto foo = new Foo();
         foo.call();
         foo.call();
     }
 }


 class Foo : Fiber {
     this() {
         super(&run);
     }


     void run() {
         Fiber.yield();
     }
 }


 Running this with -m64 on windows runs without a problem, but with 
 -m32 it failes aith a Memory Allocation failed error.


 This is not a fiber issue but a more memory management issue. Your run 
 out of address space on win32, the GC will not always collect all 
 those 99999 fibers that you allocate in that loop. As an exercise 
 replace `auto` with `scope` like `scope foo = new Foo();` in that loop 
 - you should see different results.




This shouldn't be a requirement, the 32-bit GC is generally not this bad.


 The issue boils down to the call to VirtualAlloc that the fiber 
 constructor makes, which fails as Windows will not allocate any more 
 virtual pages for the process. If you really want that many fibers you 
 should reconsider 32 bit, and definitely use a different allocation 
 strategy.


 
 And in the end of the day it makes no sense to have that many fibers as 
 they would probably perform terrible.


Let's not forget though, that he's executing the fiber completely within 
the loop. It should be done and collected.

This is not the case of executing 100,000 concurrent fibers, but 
executing 100,000 *sequential* fibers. It should work just fine.

-Steve
Apr 24 2018
next sibling parent reply Radu <void null.pt> writes:
On Tuesday, 24 April 2018 at 13:36:48 UTC, Steven Schveighoffer 
wrote:

 On 4/24/18 5:11 AM, bauss wrote:

 On Tuesday, 24 April 2018 at 07:58:01 UTC, Radu wrote:

 On Tuesday, 24 April 2018 at 00:46:39 UTC, Byron Heads wrote:

 [...]


 This is not a fiber issue but a more memory management issue. 
 Your run out of address space on win32, the GC will not 
 always collect all those 99999 fibers that you allocate in 
 that loop. As an exercise replace `auto` with `scope` like 
 `scope foo = new Foo();` in that loop - you should see 
 different results.




 This shouldn't be a requirement, the 32-bit GC is generally not 
 this bad.


Allocating so many fibers in a loop produces an OOM error on 
win32, that's a fact! Event though it doesn't always happen you 
often get OOM errors with the program above.

Probably the cause is related to how often the collection kicks 
in in relation to the pages allocated via VirtualAlloc. But 
still, the issue OP raised is not a Fiber related issue but a 
memory management issue.


 The issue boils down to the call to VirtualAlloc that the 
 fiber constructor makes, which fails as Windows will not 
 allocate any more virtual pages for the process. If you 
 really want that many fibers you should reconsider 32 bit, 
 and definitely use a different allocation strategy.


 
 And in the end of the day it makes no sense to have that many 
 fibers as they would probably perform terrible.


 Let's not forget though, that he's executing the fiber 
 completely within the loop. It should be done and collected.

 This is not the case of executing 100,000 concurrent fibers, 
 but executing 100,000 *sequential* fibers. It should work just 
 fine.

 -Steve
Apr 24 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/24/18 10:16 AM, Radu wrote:

 On Tuesday, 24 April 2018 at 13:36:48 UTC, Steven Schveighoffer wrote:

 On 4/24/18 5:11 AM, bauss wrote:

 On Tuesday, 24 April 2018 at 07:58:01 UTC, Radu wrote:

 On Tuesday, 24 April 2018 at 00:46:39 UTC, Byron Heads wrote:

 [...]


 This is not a fiber issue but a more memory management issue. Your 
 run out of address space on win32, the GC will not always collect 
 all those 99999 fibers that you allocate in that loop. As an 
 exercise replace `auto` with `scope` like `scope foo = new Foo();` 
 in that loop - you should see different results.




 This shouldn't be a requirement, the 32-bit GC is generally not this bad.


 
 Allocating so many fibers in a loop produces an OOM error on win32, 
 that's a fact! Event though it doesn't always happen you often get OOM 
 errors with the program above.


I'm not saying it doesn't happen, just that it *shouldn't* happen. At 
least not for small sized chunks like this.

I want to emphasize that the program is allocating and releasing to the 
GC 1 fiber at a time -- loop or no loop, this should work (more or less) 
reliably, or Win32 has some more serious issues.

This isn't even a case of multi-threading, there are no extra threads here.


 Probably the cause is related to how often the collection kicks in in 
 relation to the pages allocated via VirtualAlloc. But still, the issue 
 OP raised is not a Fiber related issue but a memory management issue.


Collections usually happen more often than they should. The biggest 
issue with 32-bit arch is that random stack data often keeps large 
blocks of memory from being collected. As those build up, the situation 
gets worse until you have no more memory left.

But smaller chunks like a 16k Fiber stack should work without issues.

It should be simple to prove, instead of allocating a fiber, allocate a 
similar sized chunk of bytes.

-Steve
Apr 24 2018
parent reply Radu <void null.pt> writes:
On Tuesday, 24 April 2018 at 16:05:48 UTC, Steven Schveighoffer 
wrote:

 On 4/24/18 10:16 AM, Radu wrote:

 On Tuesday, 24 April 2018 at 13:36:48 UTC, Steven 
 Schveighoffer wrote:

 On 4/24/18 5:11 AM, bauss wrote:

 On Tuesday, 24 April 2018 at 07:58:01 UTC, Radu wrote:

 On Tuesday, 24 April 2018 at 00:46:39 UTC, Byron Heads 
 wrote:

 [...]


 This is not a fiber issue but a more memory management 
 issue. Your run out of address space on win32, the GC will 
 not always collect all those 99999 fibers that you allocate 
 in that loop. As an exercise replace `auto` with `scope` 
 like `scope foo = new Foo();` in that loop - you should see 
 different results.




 This shouldn't be a requirement, the 32-bit GC is generally 
 not this bad.


 
 Allocating so many fibers in a loop produces an OOM error on 
 win32, that's a fact! Event though it doesn't always happen 
 you often get OOM errors with the program above.


 I'm not saying it doesn't happen, just that it *shouldn't* 
 happen. At least not for small sized chunks like this.

 I want to emphasize that the program is allocating and 
 releasing to the GC 1 fiber at a time -- loop or no loop, this 
 should work (more or less) reliably, or Win32 has some more 
 serious issues.


Changing main to
---
void main(string[] args)
{
     import core.memory;
     foreach(ulong i; 0..99_999) {
         auto foo = new Foo();
         foo.call();
         foo.call();
         if (i % 10000) // <-- this
             GC.collect();
     }
}
---

makes the OOM error go away.
Apr 24 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/24/18 3:45 PM, Radu wrote:

 On Tuesday, 24 April 2018 at 16:05:48 UTC, Steven Schveighoffer wrote:

 On 4/24/18 10:16 AM, Radu wrote:

 On Tuesday, 24 April 2018 at 13:36:48 UTC, Steven Schveighoffer wrote:

 On 4/24/18 5:11 AM, bauss wrote:

 On Tuesday, 24 April 2018 at 07:58:01 UTC, Radu wrote:

 On Tuesday, 24 April 2018 at 00:46:39 UTC, Byron Heads wrote:

 [...]


 This is not a fiber issue but a more memory management issue. Your 
 run out of address space on win32, the GC will not always collect 
 all those 99999 fibers that you allocate in that loop. As an 
 exercise replace `auto` with `scope` like `scope foo = new Foo();` 
 in that loop - you should see different results.




 This shouldn't be a requirement, the 32-bit GC is generally not this 
 bad.


 Allocating so many fibers in a loop produces an OOM error on win32, 
 that's a fact! Event though it doesn't always happen you often get 
 OOM errors with the program above.


 I'm not saying it doesn't happen, just that it *shouldn't* happen. At 
 least not for small sized chunks like this.

 I want to emphasize that the program is allocating and releasing to 
 the GC 1 fiber at a time -- loop or no loop, this should work (more or 
 less) reliably, or Win32 has some more serious issues.


 
 Changing main to
 ---
 void main(string[] args)
 {
      import core.memory;
      foreach(ulong i; 0..99_999) {
          auto foo = new Foo();
          foo.call();
          foo.call();
          if (i % 10000) // <-- this
              GC.collect();
      }
 }
 ---
 
 makes the OOM error go away.


This made it click for me -- VirtualAlloc does NOT allocate from the GC. 
I had mistakenly thought the GC was being used to allocate the stack. 
This means there is little to no pressure on the GC to run a collection 
even though all the memory is being consumed. In other words, the 
runtime has plenty of space to allocate the Fiber class (which is likely 
about 64 or 128 bytes per instance), and is consuming all the memory via 
VirtualAlloc.

I also noticed, Windows default stack size is 32k, not 16k (as it is on 
other systems), so 100,000 stacks in that case is 3.2GB. That's too much 
for sure.

I'll file an issue. We may not be able to solve the problem, but it's 
something we should try and solve.

Thanks

-Steve
Apr 24 2018
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/24/18 4:30 PM, Steven Schveighoffer wrote:


 I'll file an issue. We may not be able to solve the problem, but it's 
 something we should try and solve.


Seems there's already a similar issue in there: 
https://issues.dlang.org/show_bug.cgi?id=3523

-Steve
Apr 24 2018
prev sibling parent reply Byron Heads <bheads moxiegroup.com> writes:
On Tuesday, 24 April 2018 at 13:36:48 UTC, Steven Schveighoffer 
wrote:

 On 4/24/18 5:11 AM, bauss wrote:

 On Tuesday, 24 April 2018 at 07:58:01 UTC, Radu wrote:

 On Tuesday, 24 April 2018 at 00:46:39 UTC, Byron Heads wrote:

 Fibers on Win32 have a memory leak for sure:

 import core.thread : Fiber;

 void main() {

     foreach(ulong i; 0..99_999) {
         auto foo = new Foo();
         foo.call();
         foo.call();
     }
 }


 class Foo : Fiber {
     this() {
         super(&run);
     }


     void run() {
         Fiber.yield();
     }
 }


 Running this with -m64 on windows runs without a problem, 
 but with -m32 it failes aith a Memory Allocation failed 
 error.


 This is not a fiber issue but a more memory management issue. 
 Your run out of address space on win32, the GC will not 
 always collect all those 99999 fibers that you allocate in 
 that loop. As an exercise replace `auto` with `scope` like 
 `scope foo = new Foo();` in that loop - you should see 
 different results.




 This shouldn't be a requirement, the 32-bit GC is generally not 
 this bad.


 The issue boils down to the call to VirtualAlloc that the 
 fiber constructor makes, which fails as Windows will not 
 allocate any more virtual pages for the process. If you 
 really want that many fibers you should reconsider 32 bit, 
 and definitely use a different allocation strategy.


 
 And in the end of the day it makes no sense to have that many 
 fibers as they would probably perform terrible.


 Let's not forget though, that he's executing the fiber 
 completely within the loop. It should be done and collected.

 This is not the case of executing 100,000 concurrent fibers, 
 but executing 100,000 *sequential* fibers. It should work just 
 fine.

 -Steve


Correct, in a normal run of my system there maybe 10-20 fibers 
max alive. I was using threads only before, but found the system 
to execute jobs in a balanced way. But using a few threads to 
process Fibers keep in a queue balances out the work evenly. It 
is also easier to track down bugs by just using 1 thread to 
process the fiber pool.

I would love to use dip1000, Allocators, and shared. But none of 
that stuff really works beyond trivial examples. (Allocators 
probably works fine, but there are forum post about it changing 
and I dont want to refactor it twice..)

I will start ignoring win32 when win64 doesn't require dealing 
with visual studio installs.
Also I have a feeling a client will ask for it.
Apr 24 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/24/18 10:31 AM, Byron Heads wrote:

 On Tuesday, 24 April 2018 at 13:36:48 UTC, Steven Schveighoffer wrote:

 This is not the case of executing 100,000 concurrent fibers, but 
 executing 100,000 *sequential* fibers. It should work just fine.


 
 Correct, in a normal run of my system there maybe 10-20 fibers max 
 alive. I was using threads only before, but found the system to execute 
 jobs in a balanced way. But using a few threads to process Fibers keep 
 in a queue balances out the work evenly. It is also easier to track down 
 bugs by just using 1 thread to process the fiber pool.
 
 I would love to use dip1000, Allocators, and shared. But none of that 
 stuff really works beyond trivial examples. (Allocators probably works 
 fine, but there are forum post about it changing and I dont want to 
 refactor it twice..)


stdx.allocator (https://github.com/dlang-community/stdx-allocator and 
http://code.dlang.org/packages/stdx-allocator) is "stable", you can use 
that instead of the one inside phobos. This way, even if phobos 
introduces breaking changes, you can depend on a specific version of the 
allocators. DIP1000 is very much a work in progress, I'm unsure if/when 
it will become usable. Shared likely isn't going to get any better until 
the main players start focusing on it. Right now, I think they are more 
interested in memory safety.


 I will start ignoring win32 when win64 doesn't require dealing with 
 visual studio installs.
 Also I have a feeling a client will ask for it.


Unfortunately I don't think the VS license will ever allow us to avoid 
installing VS as well.

My recommendation is just to ignore Win32. I wouldn't trust it at all. 
There are serious threading issues there, and the GC is prone to run out 
of memory if you aren't careful.

But I can understand if you can't go that route.

Another thing to try is -m32mscoff, which creates 32-bit binaries, but 
links against Microsoft's runtime instead of DMD. While this is a stated 
problem from you, it may help to determine if it's really the digital 
mars library or something more inherent in the way Fibers or the GC is 
working.

-Steve
Apr 24 2018
parent reply kinke <kinke libero.it> writes:
On Tuesday, 24 April 2018 at 16:22:04 UTC, Steven Schveighoffer 
wrote:

 On 4/24/18 10:31 AM, Byron Heads wrote:

 I will start ignoring win32 when win64 doesn't require dealing 
 with visual studio installs.
 Also I have a feeling a client will ask for it.


 Unfortunately I don't think the VS license will ever allow us 
 to avoid installing VS as well.


DMD doesn't require VS anymore since v2.079.
Apr 24 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/24/18 12:49 PM, kinke wrote:

 On Tuesday, 24 April 2018 at 16:22:04 UTC, Steven Schveighoffer wrote:

 On 4/24/18 10:31 AM, Byron Heads wrote:

 I will start ignoring win32 when win64 doesn't require dealing with 
 visual studio installs.
 Also I have a feeling a client will ask for it.


 Unfortunately I don't think the VS license will ever allow us to avoid 
 installing VS as well.


 
 DMD doesn't require VS anymore since v2.079.


Oh? That's good news. What do you need to install instead? Or do we 
include the SDK library directly?

I had thought there were licensing issues, but glad to be wrong!

-Steve
Apr 24 2018
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 25/04/2018 5:13 AM, Steven Schveighoffer wrote:

 On 4/24/18 12:49 PM, kinke wrote:

 On Tuesday, 24 April 2018 at 16:22:04 UTC, Steven Schveighoffer wrote:

 On 4/24/18 10:31 AM, Byron Heads wrote:

 I will start ignoring win32 when win64 doesn't require dealing with 
 visual studio installs.
 Also I have a feeling a client will ask for it.


 Unfortunately I don't think the VS license will ever allow us to 
 avoid installing VS as well.


 DMD doesn't require VS anymore since v2.079.


 
 Oh? That's good news. What do you need to install instead? Or do we 
 include the SDK library directly?
 
 I had thought there were licensing issues, but glad to be wrong!
 
 -Steve


We are not providing it, its coming straight from MS, no licensing issues.
Apr 24 2018
prev sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Friday, 20 April 2018 at 18:58:36 UTC, Byron Moxie wrote:

 [...]
 In WIN32 it looks like its leaking memory


Unless there is something I'm misunderstanding, it seems that 
Fibers that were not run to completion won't unroll their stack, 
which would mean that some destructors wouldn't be called, and 
possibly, some memory wouldn't be freed:

https://github.com/dlang/druntime/blob/86cd40a036a67d9b1bff6c14e91cba1e5557b119/src/core/thread.d#L4142

Could this have something to do with the problem?
Apr 24 2018
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/24/18 10:12 PM, bitwise wrote:

 On Friday, 20 April 2018 at 18:58:36 UTC, Byron Moxie wrote:

 [...]
 In WIN32 it looks like its leaking memory


 
 Unless there is something I'm misunderstanding, it seems that Fibers 
 that were not run to completion won't unroll their stack, which would 
 mean that some destructors wouldn't be called, and possibly, some memory 
 wouldn't be freed:
 
 https://github.com/dlang/druntime/blob/86cd40a036a67d9b1bff6c14e91cba1e5557b119/src
core/thread.d#L4142 
 
 
 Could this have something to do with the problem?
 


I think that's not the case. In his pathological example deeper in this 
thread, he's running the fibers to completion, and getting OOM error. I 
think the GC isn't cleaning up the fibers because it's not out of new 
memory to allocate yet. But the stack space isn't using GC resources, so 
it consumes all the actual memory in 32-bit land before the GC is ever run.

-Steve
Apr 25 2018