www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - D2 port of Sociomantic CDGC available for early experiments

reply "Dicebot" <public dicebot.lv> writes:
https://github.com/D-Programming-Language/druntime/pull/985

Here is initial port result available for early experiments. It 
can be compiled with make -f posix.mak GC_TYPE=concurrent and 
passes the test suite with only shared library tests disabled 
(ef20b7a).

There are still many issues to be aware of:
1) Documentation is largely missing. Working on it, reading 
 leandro-lucarella-sociomantic old posts 
(http://www.llucax.com.ar/blog/blog/tag/understanding%20the%20current%20gc) 
may help in the meanwhile
2) Code style differs from Phobos standards. To be fixed soon.
3) Shared library support is completely missing. Proxy 
infrastructure similar to one in existing gc needs to be added 
and I don't know if actual implementation will work in such 
environments or more changes will be needed.
4) Deadlock issue 
(http://www.dsource.org/projects/tango/ticket/2087) still 
remains. It is not critical to our code because it almost never 
uses threads so no big effort was put into it but this can be 
huge problem for any other project.

In general this is very far from something that can be merged 
upstream straight away and replace default GC on linux. It can be 
interesting for other projects with similar architecture and 
requirements and probably helpful for anyone else working on 
better D GC.

And contributions are welcome via pull requests towards this PR 
base branch 
(https://github.com/mihails-strasuns-sociomantic/druntime-1/tree/s
ciomantic-cdgc-wip) 
or as e-mail patches (public dicebot.lv)
Oct 06 2014
next sibling parent ketmar via Digitalmars-d-announce <digitalmars-d-announce puremagic.com> writes:
On Mon, 06 Oct 2014 16:51:06 +0000
Dicebot via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 https://github.com/D-Programming-Language/druntime/pull/985
=20
 Here is initial port result available for early experiments. It=20
 can be compiled with make -f posix.mak GC_TYPE=3Dconcurrent and=20
 passes the test suite with only shared library tests disabled=20
 (ef20b7a).
wow, going to try it NOW! ;-)
Oct 06 2014
prev sibling next sibling parent "Kiith-Sa" <kiithsacmp gmail.com> writes:
On Monday, 6 October 2014 at 16:51:07 UTC, Dicebot wrote:
 https://github.com/D-Programming-Language/druntime/pull/985

 Here is initial port result available for early experiments. It 
 can be compiled with make -f posix.mak GC_TYPE=concurrent and 
 passes the test suite with only shared library tests disabled 
 (ef20b7a).

 There are still many issues to be aware of:
 1) Documentation is largely missing. Working on it, reading 
  leandro-lucarella-sociomantic old posts 
 (http://www.llucax.com.ar/blog/blog/tag/understanding%20the%20current%20gc) 
 may help in the meanwhile
 2) Code style differs from Phobos standards. To be fixed soon.
 3) Shared library support is completely missing. Proxy 
 infrastructure similar to one in existing gc needs to be added 
 and I don't know if actual implementation will work in such 
 environments or more changes will be needed.
 4) Deadlock issue 
 (http://www.dsource.org/projects/tango/ticket/2087) still 
 remains. It is not critical to our code because it almost never 
 uses threads so no big effort was put into it but this can be 
 huge problem for any other project.

 In general this is very far from something that can be merged 
 upstream straight away and replace default GC on linux. It can 
 be interesting for other projects with similar architecture and 
 requirements and probably helpful for anyone else working on 
 better D GC.

 And contributions are welcome via pull requests towards this PR 
 base branch 
 (https://github.com/mihails-strasuns-sociomantic/druntime-1/tree/s
ciomantic-cdgc-wip) 
 or as e-mail patches (public dicebot.lv)
Can't really test this right now, just want to say that it's awesome that someone is working on this and I really hope this can (eventually) get merged.
Oct 06 2014
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/6/14, 9:51 AM, Dicebot wrote:
 https://github.com/D-Programming-Language/druntime/pull/985
This is awesome. I recall Don had some solid performance numbers for it in his talk, do you have any in the context of D2? -- Andrei
Oct 06 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 6 October 2014 at 17:23:55 UTC, Andrei Alexandrescu 
wrote:
 On 10/6/14, 9:51 AM, Dicebot wrote:
 https://github.com/D-Programming-Language/druntime/pull/985
This is awesome. I recall Don had some solid performance numbers for it in his talk, do you have any in the context of D2? -- Andrei
No, I didn't get to running any perf test so far. Did PR as soon as test suite passes and commits looked sane. Will do eventually. Any specific project you are interested in? I'd love to see the impact on vibe.d but it is subject to threading/malloc issue right now.
Oct 06 2014
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/6/14, 10:29 AM, Dicebot wrote:
 On Monday, 6 October 2014 at 17:23:55 UTC, Andrei Alexandrescu wrote:
 On 10/6/14, 9:51 AM, Dicebot wrote:
 https://github.com/D-Programming-Language/druntime/pull/985
This is awesome. I recall Don had some solid performance numbers for it in his talk, do you have any in the context of D2? -- Andrei
No, I didn't get to running any perf test so far. Did PR as soon as test suite passes and commits looked sane. Will do eventually. Any specific project you are interested in? I'd love to see the impact on vibe.d but it is subject to threading/malloc issue right now.
vibe.d would be great to test. Generally any realistic project would be relevant. -- Andrei
Oct 06 2014
parent Leandro Lucarella <luca llucax.com.ar> writes:
Andrei Alexandrescu, el  6 de October a las 10:40 me escribiste:
 On 10/6/14, 10:29 AM, Dicebot wrote:
On Monday, 6 October 2014 at 17:23:55 UTC, Andrei Alexandrescu wrote:
On 10/6/14, 9:51 AM, Dicebot wrote:
https://github.com/D-Programming-Language/druntime/pull/985
This is awesome. I recall Don had some solid performance numbers for it in his talk, do you have any in the context of D2? -- Andrei
No, I didn't get to running any perf test so far. Did PR as soon as test suite passes and commits looked sane. Will do eventually. Any specific project you are interested in? I'd love to see the impact on vibe.d but it is subject to threading/malloc issue right now.
vibe.d would be great to test. Generally any realistic project would be relevant. -- Andrei
Probably projects needing low latency are the ones that will benefit the most. Although I was surprised by the results even with single-threaded applications when I did my performance tests back in the days, which got a considerable performance boost in multi-cores, as suddenly you application becomes automatically parallelized (now your program can run in parallel with the collector while before they were serialized). Anyway, we'd love to hear about any numbers people can collect :-) -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- Did you see the frightened ones? Did you hear the falling bombs? Did you ever wonder why we had to run for shelter when the promise of a brave new world unfurled beneath a clear blue sky?
Oct 07 2014
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 6 October 2014 at 17:29:23 UTC, Dicebot wrote:
 No, I didn't get to running any perf test so far. Did PR as 
 soon as test suite passes and commits looked sane. Will do 
 eventually. Any specific project you are interested in?

 I'd love to see the impact on vibe.d but it is subject to 
 threading/malloc issue right now.
SDC ?
Oct 07 2014
prev sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 06.10.2014 19:29, Dicebot wrote:
 On Monday, 6 October 2014 at 17:23:55 UTC, Andrei Alexandrescu wrote:
 On 10/6/14, 9:51 AM, Dicebot wrote:
 https://github.com/D-Programming-Language/druntime/pull/985
This is awesome. I recall Don had some solid performance numbers for it in his talk, do you have any in the context of D2? -- Andrei
No, I didn't get to running any perf test so far. Did PR as soon as test suite passes and commits looked sane. Will do eventually. Any specific project you are interested in? I'd love to see the impact on vibe.d but it is subject to threading/malloc issue right now.
druntime has a small benchmark suite in druntime/benchmark. I'd be very interested to see how it performs against this. (Some files are from Leandros tests.) Just compile and run "runbench.d". I'm benchmarking my Windows version of a concurrent GC with it. It does quite a lot of allocations, and this is causing some serious trouble because marking cannot compete with the rate of allocation, causing some tests to allocate a lot of memory, slowing down marking even more. I'm still looking for a solution...
Oct 08 2014
parent reply "Martin Nowak" <code dawg.eu> writes:
On Wednesday, 8 October 2014 at 18:25:00 UTC, Rainer Schuetze 
wrote:
 I'm benchmarking my Windows version of a concurrent GC with it. 
 It does quite a lot of allocations, and this is causing some 
 serious trouble because marking cannot compete with the rate of 
 allocation, causing some tests to allocate a lot of memory, 
 slowing down marking even more. I'm still looking for a 
 solution...
Well ultimatively you got to throttle your mutators to a throughput that the GC can handle. That's actually a nice thing about the forking GC, the mutator has to pay for the COW page faults, so there is some built-in throttling. How does your GC work?
Oct 29 2014
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 30.10.2014 04:02, Martin Nowak wrote:
 On Wednesday, 8 October 2014 at 18:25:00 UTC, Rainer Schuetze wrote:
 I'm benchmarking my Windows version of a concurrent GC with it. It
 does quite a lot of allocations, and this is causing some serious
 trouble because marking cannot compete with the rate of allocation,
 causing some tests to allocate a lot of memory, slowing down marking
 even more. I'm still looking for a solution...
Well ultimatively you got to throttle your mutators to a throughput that the GC can handle. That's actually a nice thing about the forking GC, the mutator has to pay for the COW page faults, so there is some built-in throttling. How does your GC work?
There's sort of a description here: http://rainers.github.io/visuald/druntime/concurrentgc.html There are actually two variations: - one uses "Memory-Write-Watches" to detect modified pages and rescans all pages modified during a background scan. It could be improved to reschedule the second scan to the background, again, until there are only few modified pages. - the other uses COW protection on the pool memory, but has to copy modified memory back when restoring unprotected memory mappings.
Oct 30 2014
prev sibling next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
06-Oct-2014 20:51, Dicebot пишет:
 https://github.com/D-Programming-Language/druntime/pull/985

 Here is initial port result available for early experiments. It can be
 compiled with make -f posix.mak GC_TYPE=concurrent and passes the test
 suite with only shared library tests disabled (ef20b7a).
Awesome!
 There are still many issues to be aware of:
 1) Documentation is largely missing. Working on it, reading
  leandro-lucarella-sociomantic old posts
 (http://www.llucax.com.ar/blog/blog/tag/understanding%20the%20current%20gc)
 may help in the meanwhile
 2) Code style differs from Phobos standards. To be fixed soon.
 3) Shared library support is completely missing. Proxy infrastructure
 similar to one in existing gc needs to be added and I don't know if
 actual implementation will work in such environments or more changes
 will be needed.
Never used that anyway ;)
 4) Deadlock issue (http://www.dsource.org/projects/tango/ticket/2087)
 still remains. It is not critical to our code because it almost never
 uses threads so no big effort was put into it but this can be huge
 problem for any other project.
Just use different LIBC ? For instance, try Musl: http://www.musl-libc.org/ (Binaries linked statically with Musl can co-exist with GLIBC-based OS just fine) I might try CDGC+musl sometime soonish anyway, but just letting this idea out in case you are interested.
 In general this is very far from something that can be merged upstream
 straight away and replace default GC on linux. It can be interesting for
 other projects with similar architecture and requirements and probably
 helpful for anyone else working on better D GC.

 And contributions are welcome via pull requests towards this PR base
 branch
 (https://github.com/mihails-strasuns-sociomantic/druntime-1/tree/sociomantic-cdgc-wip)
 or as e-mail patches (public dicebot.lv)
-- Dmitry Olshansky
Oct 07 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/6/2014 9:51 AM, Dicebot wrote:
 https://github.com/D-Programming-Language/druntime/pull/985
Thank you. This is great progress! I understand the caveats, but can this be put into a shape where it can be pulled despite being a work in progress? I.e. have the code be disabled by default? That will help encourage the community to help out with the gaps.
Oct 07 2014
next sibling parent reply Leandro Lucarella <luca llucax.com.ar> writes:
Walter Bright, el  7 de October a las 13:06 me escribiste:
 On 10/6/2014 9:51 AM, Dicebot wrote:
https://github.com/D-Programming-Language/druntime/pull/985
Thank you. This is great progress! I understand the caveats, but can this be put into a shape where it can be pulled despite being a work in progress? I.e. have the code be disabled by default? That will help encourage the community to help out with the gaps.
Yes, I think that's the best approach too. Ideally this should be configurable at runtime as Marting suggested, so ANYONE can try it with their applications by just running it like this: D_GC=concurrent=1 ./myapp Then bugs can be filled more easily and people can work on fixing them more easily too. At some point I'd like to see the current GC and the concurrent GC merged, the concurrent GC already support disabling the concurrency. Both GCs diverged with time and both have (different) improvements over the common parent, and it will be a shame to lose any of them. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- Y Gloria Carrá, Gloria Estephan, Gloria Gaynor y Gloria Trevi. -- Peperino Pómoro
Oct 07 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/7/2014 3:27 PM, Leandro Lucarella wrote:
 Walter Bright, el  7 de October a las 13:06 me escribiste:
 On 10/6/2014 9:51 AM, Dicebot wrote:
 https://github.com/D-Programming-Language/druntime/pull/985
Thank you. This is great progress! I understand the caveats, but can this be put into a shape where it can be pulled despite being a work in progress? I.e. have the code be disabled by default? That will help encourage the community to help out with the gaps.
Yes, I think that's the best approach too. Ideally this should be configurable at runtime as Marting suggested, so ANYONE can try it with their applications by just running it like this: D_GC=concurrent=1 ./myapp Then bugs can be filled more easily and people can work on fixing them more easily too. At some point I'd like to see the current GC and the concurrent GC merged, the concurrent GC already support disabling the concurrency. Both GCs diverged with time and both have (different) improvements over the common parent, and it will be a shame to lose any of them.
That's a good idea, but I hate environment variables affecting all D executables. They always wind up being inadvertently being left on, or off, or set for some unrelated purpose. It also would affect all D executables on the system, potentially making a big mess. And there's always: user: "my program used to work, now it is too big / too slow / crashes" n.g.: "did you check that D_GC is not in your environment?" user: "yes I checked, it isn't set" n.g.: "check again" [lots of fruitless back and forth a few times] user: "oops, sorry!" Sort of like: user: "need to fix this on the website" n.g.: "thanks! fixed now!" user: "no it isn't" n.g.: "looks good to me" [lots of fruitless back and forth] n.g.: "did you refresh your browser's cache?" user: "oops, sorry!"
Oct 07 2014
next sibling parent reply "eles" <eles215 gzk.dot> writes:
On Wednesday, 8 October 2014 at 00:18:16 UTC, Walter Bright wrote:
 On 10/7/2014 3:27 PM, Leandro Lucarella wrote:
 Walter Bright, el  7 de October a las 13:06 me escribiste:
 On 10/6/2014 9:51 AM, Dicebot wrote:
 That's a good idea, but I hate environment variables affecting 
 all D executables. They always wind up being inadvertently
LD_LIBRARY_PATH ?
Oct 07 2014
parent Rory McGuire via Digitalmars-d-announce writes:
I like the on/off at runtime idea. Less scary to try on important code. We
can just disable the concurrent gc if we notice dodgy things happening.

On Wed, Oct 8, 2014 at 8:09 AM, eles via Digitalmars-d-announce <
digitalmars-d-announce puremagic.com> wrote:

 On Wednesday, 8 October 2014 at 00:18:16 UTC, Walter Bright wrote:

 On 10/7/2014 3:27 PM, Leandro Lucarella wrote:

 Walter Bright, el  7 de October a las 13:06 me escribiste:

 On 10/6/2014 9:51 AM, Dicebot wrote:
That's a good idea, but I hate environment variables affecting all D
 executables. They always wind up being inadvertently
LD_LIBRARY_PATH ?
Oct 07 2014
prev sibling next sibling parent reply ketmar via Digitalmars-d-announce <digitalmars-d-announce puremagic.com> writes:
On Tue, 07 Oct 2014 17:18:18 -0700
Walter Bright via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 That's a good idea, but I hate environment variables affecting all D=20
 executables. They always wind up being inadvertently being left on,
 or off, or set for some unrelated purpose. It also would affect all D
 executables on the system, potentially making a big mess.
maybe enable envvar-control in alphas/betas and disable in releases? this way people who want to try something new can experiment, but those who need stability will not be affected. after all, those who using alphas/betas knows what they're doing and are aware of possible problems. i assume that not everyone are ready to build dmd from sources, yet many people are ready to try new features, so they can download and install separate alpha/beta to play with.
Oct 07 2014
next sibling parent reply "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Wednesday, 8 October 2014 at 06:58:36 UTC, ketmar via 
Digitalmars-d-announce wrote:
 On Tue, 07 Oct 2014 17:18:18 -0700
 Walter Bright via Digitalmars-d-announce
 <digitalmars-d-announce puremagic.com> wrote:

 i assume that not everyone are ready to build dmd from sources,
Digger? What can be easier? -- /P
Oct 08 2014
next sibling parent ketmar via Digitalmars-d-announce <digitalmars-d-announce puremagic.com> writes:
On Wed, 08 Oct 2014 07:14:29 +0000
Paolo Invernizzi via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 i assume that not everyone are ready to build dmd from sources,
Digger? What can be easier?
i've never used Digger, so i can't tell anything about it. ;-)
Oct 08 2014
prev sibling next sibling parent ketmar via Digitalmars-d-announce <digitalmars-d-announce puremagic.com> writes:
On Wed, 08 Oct 2014 07:14:29 +0000
Paolo Invernizzi via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 i assume that not everyone are ready to build dmd from sources,
Digger? What can be easier?
p.s. i mean that it will be good to have Digger mentioned somewhere at the front page, with some usage samples, so people will know that we have such good tool.
Oct 08 2014
prev sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Wednesday, 8 October 2014 at 07:14:30 UTC, Paolo Invernizzi 
wrote:
 On Wednesday, 8 October 2014 at 06:58:36 UTC, ketmar via 
 Digitalmars-d-announce wrote:
 On Tue, 07 Oct 2014 17:18:18 -0700
 Walter Bright via Digitalmars-d-announce
 <digitalmars-d-announce puremagic.com> wrote:

 i assume that not everyone are ready to build dmd from sources,
Digger? What can be easier?
Yes, this is a perfect use case for Digger. Just scroll down the pull request list, check the GC pull's checkbox, and click "build". What could be easier?
Oct 08 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/7/2014 11:58 PM, ketmar via Digitalmars-d-announce wrote:
 maybe enable envvar-control in alphas/betas and disable in releases?
I thought you were opposed to -release builds?
Oct 08 2014
next sibling parent ketmar via Digitalmars-d-announce <digitalmars-d-announce puremagic.com> writes:
On Wed, 08 Oct 2014 10:33:46 -0700
Walter Bright via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 On 10/7/2014 11:58 PM, ketmar via Digitalmars-d-announce wrote:
 maybe enable envvar-control in alphas/betas and disable in releases?
I thought you were opposed to -release builds?
i'm opposed to "-release" flag which turns off assertions and range checking, not to the builds which can be titled "official releases". we have official DMD releases, alphas, betas and RCs. it's logical to turn on some experimental features in alphas/betas for people to play, but turn such features off for safety reasons in RC/release. turning off assertions and range checks decreases safety. turning off experimental features and features with very unexpected side effects increases safety. it's ok to make alphas/betas less safe. it's not ok to make RCs/releases less safe.
Oct 08 2014
prev sibling parent ketmar via Digitalmars-d-announce <digitalmars-d-announce puremagic.com> writes:
On Wed, 8 Oct 2014 20:47:09 +0300
ketmar via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

p.s. i believe that there shouldn't be any flags that turns off range
checking. if someone wants to do it "fast", he can use pointer
arithmetics (this is exactly the same as using arrays with range checks
off). that's what i'm doing in my code: writing everything using
arrays, which are range-checked. then profiling and rewriting
bottlenecks using pointer arithmetics. and i found that i rarely ever
need to go pointers, 'cause optimizing algorithms has much bigger
impact on performance.
Oct 08 2014
prev sibling next sibling parent reply Leandro Lucarella <luca llucax.com.ar> writes:
Walter Bright, el  7 de October a las 17:18 me escribiste:
 On 10/7/2014 3:27 PM, Leandro Lucarella wrote:
Walter Bright, el  7 de October a las 13:06 me escribiste:
On 10/6/2014 9:51 AM, Dicebot wrote:
https://github.com/D-Programming-Language/druntime/pull/985
Thank you. This is great progress! I understand the caveats, but can this be put into a shape where it can be pulled despite being a work in progress? I.e. have the code be disabled by default? That will help encourage the community to help out with the gaps.
Yes, I think that's the best approach too. Ideally this should be configurable at runtime as Marting suggested, so ANYONE can try it with their applications by just running it like this: D_GC=concurrent=1 ./myapp Then bugs can be filled more easily and people can work on fixing them more easily too. At some point I'd like to see the current GC and the concurrent GC merged, the concurrent GC already support disabling the concurrency. Both GCs diverged with time and both have (different) improvements over the common parent, and it will be a shame to lose any of them.
That's a good idea, but I hate environment variables affecting all D executables. They always wind up being inadvertently being left on, or off, or set for some unrelated purpose. It also would affect all D executables on the system, potentially making a big mess.
I think this is an unjustified fear, there are already many environment variables that can affect your program. That's why they are called... environment variables :) -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- - Mire, don Inodoro! Una paloma con un anillo en la pata! Debe ser mensajera y cayó aquí! - Y... si no es mensajera es coqueta... o casada. -- Mendieta e Inodoro Pereyra
Oct 08 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/8/2014 12:43 AM, Leandro Lucarella wrote:
 I think this is an unjustified fear, there are already many environment
 variables that can affect your program. That's why they are called...
 environment variables :)
Being on the front lines of tech support for 30 years, it is not an unjustified fear nor a hypothetical problem. What you could do is propose a "secret" switch to all dmd generated programs that the druntime switch checks before main() gets control, such as: app --druntimeSet=usexxx ...the regular app args ...
Oct 08 2014
next sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 08.10.2014 19:39, Walter Bright wrote:
 On 10/8/2014 12:43 AM, Leandro Lucarella wrote:
 I think this is an unjustified fear, there are already many environment
 variables that can affect your program. That's why they are called...
 environment variables :)
Being on the front lines of tech support for 30 years, it is not an unjustified fear nor a hypothetical problem. What you could do is propose a "secret" switch to all dmd generated programs that the druntime switch checks before main() gets control, such as: app --druntimeSet=usexxx ...the regular app args ...
We have app --DRT-gcopt=concurrent in gc.config now. There is no perfect plan how an application can change the default options that are used without arguments, though. Doing this in main() is too late for some parameters (e.g. "precise"), because the GC will have to be setup to execute "static this" and other library initialization before reaching main.
Oct 08 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/8/2014 11:19 AM, Rainer Schuetze wrote:
 There is no perfect plan how an application can change the default options that
 are used without arguments, though. Doing this in main() is too late for some
 parameters (e.g. "precise"), because the GC will have to be setup to execute
 "static this" and other library initialization before reaching main.
I meant look for the argument in the C main() which is part of druntime and is the first to get control. You're right that the D main() is too late.
Oct 08 2014
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 08.10.2014 22:10, Walter Bright wrote:
 On 10/8/2014 11:19 AM, Rainer Schuetze wrote:
 There is no perfect plan how an application can change the default
 options that
 are used without arguments, though. Doing this in main() is too late
 for some
 parameters (e.g. "precise"), because the GC will have to be setup to
 execute
 "static this" and other library initialization before reaching main.
I meant look for the argument in the C main() which is part of druntime and is the first to get control. You're right that the D main() is too late.
C main is no longer under user control, because it is auto-generated with D main. I never liked that change, we've just discovered another reason.
Oct 08 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 08/10/14 22:47, Rainer Schuetze wrote:

 C main is no longer under user control, because it is auto-generated
 with D main. I never liked that change, we've just discovered another
 reason.
All platforms have API's to access the command line arguments passed to "main". On OS X that would be "_NSGetArgv" and "_NSGetArgc". -- /Jacob Carlborg
Oct 08 2014
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 09.10.2014 08:29, Jacob Carlborg wrote:
 On 08/10/14 22:47, Rainer Schuetze wrote:

 C main is no longer under user control, because it is auto-generated
 with D main. I never liked that change, we've just discovered another
 reason.
All platforms have API's to access the command line arguments passed to "main". On OS X that would be "_NSGetArgv" and "_NSGetArgc".
Yes, but the problem is not to access command line arguments, but to run code before the GC initialization i.e. before _d_run_main is executed. If we can assume a C++ backend, using static initialization of a C++ global could work: static bool initGC = configureGC(); because the C++ runtime runs configureGC before calling C main. I'd rather like to see a solution using D, though.
Oct 08 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/8/2014 11:43 PM, Rainer Schuetze wrote:
 Yes, but the problem is not to access command line arguments, but to run code
 before the GC initialization i.e. before _d_run_main is executed.

 If we can assume a C++ backend, using static initialization of a C++ global
 could work:

     static bool initGC = configureGC();

 because the C++ runtime runs configureGC before calling C main. I'd rather like
 to see a solution using D, though.
Define a global variable in druntime in its own module: __gshared int druntime_flags = 0; Then, in your app, override it: __gshared int druntime_flags = values...;
Oct 09 2014
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 09.10.2014 10:18, Walter Bright wrote:
 On 10/8/2014 11:43 PM, Rainer Schuetze wrote:
 Yes, but the problem is not to access command line arguments, but to
 run code
 before the GC initialization i.e. before _d_run_main is executed.

 If we can assume a C++ backend, using static initialization of a C++
 global
 could work:

     static bool initGC = configureGC();

 because the C++ runtime runs configureGC before calling C main. I'd
 rather like
 to see a solution using D, though.
Define a global variable in druntime in its own module: __gshared int druntime_flags = 0; Then, in your app, override it: __gshared int druntime_flags = values...;
That's similar to my approach to replace gc.config by a different module with the same module name when compiling the executable: dmd main.d my_gc_config.d where my_gc_config.d contains "module gc.config;" and thus replaces the module in the library. Martin is very much against this, one reason is that it does not work with druntime in a shared library.
Oct 09 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/9/2014 11:25 AM, Rainer Schuetze wrote:
 Martin is very much against this, one reason is that it does not work with
 druntime in a shared library.
I can understand that, but I also don't know what a reasonable use case would be for sharing a library yet using different GC's - it sounds like complexity without purpose.
Oct 09 2014
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 10.10.2014 02:30, Walter Bright wrote:
 On 10/9/2014 11:25 AM, Rainer Schuetze wrote:
 Martin is very much against this, one reason is that it does not work
 with
 druntime in a shared library.
I can understand that, but I also don't know what a reasonable use case would be for sharing a library yet using different GC's - it sounds like complexity without purpose.
It was my impression that phobos in a shared library would soon become the default on linux, i.e. any application would use it by default. In that case, all supported GCs might actually have to be included in the shared library. The options to embed into the application are used to get the best performance for the specific task using any GC available through the shared library. That could include enabling/disabling preciseness, tweaking initial pool sizes and increase strategy or disabling the final collection that even runs if the GC is disabled (and might be useful only for a very limited number of application).
Oct 10 2014
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/10/2014 12:37 AM, Rainer Schuetze wrote:
 It was my impression that phobos in a shared library would soon become the
 default on linux, i.e. any application would use it by default. In that case,
 all supported GCs might actually have to be included in the shared library.

 The options to embed into the application are used to get the best performance
 for the specific task using any GC available through the shared library. That
 could include enabling/disabling preciseness, tweaking initial pool sizes and
 increase strategy or disabling the final collection that even runs if the GC is
 disabled (and might be useful only for a very limited number of application).
If there were multiple GCs, I suspect that having each GC in its own shared library would be a better option than lumping them all in one. But currently (!) I think the use for switching GCs between the concurrent and non-concurrent one is simply for evaluation purposes.
Oct 10 2014
prev sibling parent "Dicebot" <public dicebot.lv> writes:
On Friday, 10 October 2014 at 07:37:49 UTC, Rainer Schuetze wrote:
 It was my impression that phobos in a shared library would soon 
 become the default on linux, i.e. any application would use it 
 by default. In that case, all supported GCs might actually have 
 to be included in the shared library.
This doesn't mean that all GC implementations need to be put into single Phobos distrubution. I doubt it is a good idea. Better approach would be to separate libdruntime.so from libphobos.so again and provide separate libdruntime.so distributions with different GC implementations (with one being the default)
Oct 10 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 09/10/14 08:43, Rainer Schuetze wrote:

 Yes, but the problem is not to access command line arguments, but to run
 code before the GC initialization i.e. before _d_run_main is executed.

 If we can assume a C++ backend, using static initialization of a C++
 global could work:

     static bool initGC = configureGC();

 because the C++ runtime runs configureGC before calling C main. I'd
 rather like to see a solution using D, though.
It's enough with a C function with __attribute__((constructor)). This will run before C main. -- /Jacob Carlborg
Oct 09 2014
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 09.10.2014 15:31, Jacob Carlborg wrote:
 On 09/10/14 08:43, Rainer Schuetze wrote:

 Yes, but the problem is not to access command line arguments, but to run
 code before the GC initialization i.e. before _d_run_main is executed.

 If we can assume a C++ backend, using static initialization of a C++
 global could work:

     static bool initGC = configureGC();

 because the C++ runtime runs configureGC before calling C main. I'd
 rather like to see a solution using D, though.
It's enough with a C function with __attribute__((constructor)). This will run before C main.
This is a gcc extension, which isn't supported under Windows by dmd. Can you add this attribute in GDC/LDC as part of a D file aswell? I recently made some experiments with pragma(data_seg,"SEGMENT_NAME") or pragma(code_seg,"SEGMENT_NAME") that could also be used to move something into the initializer section of the runtime. For COFF it kind of works, but unfortunately there is no way to tell OMF to put a COMDAT (needed for templates) into a specific segment.
Oct 09 2014
next sibling parent reply "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 9 October 2014 at 18:33:25 UTC, Rainer Schuetze 
wrote:
 This is a gcc extension, which isn't supported under Windows by 
 dmd. Can you add this attribute in GDC/LDC as part of a D file 
 aswell?
http://wiki.dlang.org/LDC-specific_language_changes#LDC_global_crt_ctor_and_LDC_global_crt_dtor David
Oct 09 2014
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 09.10.2014 21:38, David Nadlinger wrote:
 On Thursday, 9 October 2014 at 18:33:25 UTC, Rainer Schuetze wrote:
 This is a gcc extension, which isn't supported under Windows by dmd.
 Can you add this attribute in GDC/LDC as part of a D file aswell?
http://wiki.dlang.org/LDC-specific_language_changes#LDC_global_crt_ctor_and_LDC_global_crt_dtor David
Cool. It seems GDC has this, too, as " attribute("constructor") or similar.
Oct 10 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-10-09 20:33, Rainer Schuetze wrote:

 This is a gcc extension, which isn't supported under Windows by dmd.
It seems to be possible in Visual Studio [1], but that still leaves OMF. [1] http://stackoverflow.com/questions/1113409/attribute-constructor-equivalent-in-vc -- /Jacob Carlborg
Oct 09 2014
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 09.10.2014 21:46, Jacob Carlborg wrote:
 On 2014-10-09 20:33, Rainer Schuetze wrote:

 This is a gcc extension, which isn't supported under Windows by dmd.
It seems to be possible in Visual Studio [1], but that still leaves OMF. [1] http://stackoverflow.com/questions/1113409/attribute-constructor-equivalent-in-vc
Interesting. That would also be possible in D with the proposed pragma(data_seg). It still relies on the C++ startup code of the runtime, but dmc also has that (even for the VC++ section AFAICT).
Oct 10 2014
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Wednesday, 8 October 2014 at 17:39:46 UTC, Walter Bright wrote:
 Being on the front lines of tech support for 30 years, it is 
 not an unjustified fear nor a hypothetical problem.

 What you could do is propose a "secret" switch to all dmd 
 generated programs that the druntime switch checks before 
 main() gets control, such as:

     app --druntimeSet=usexxx ...the regular app args ...
I don't think this would work (without introducing extra hacks) when druntime is used as a shared library. Plus, when writing plugins in D, there might not even be a host D application to handle the command line flags. I'm not saying that command line arguments can't be made to work in these cases too, just that doing it will likely introduce quite a bit of extra complexity. David
Oct 08 2014
prev sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Wednesday, 8 October 2014 at 17:39:46 UTC, Walter Bright wrote:
 On 10/8/2014 12:43 AM, Leandro Lucarella wrote:
 I think this is an unjustified fear, there are already many 
 environment
 variables that can affect your program. That's why they are 
 called...
 environment variables :)
Being on the front lines of tech support for 30 years, it is not an unjustified fear nor a hypothetical problem. What you could do is propose a "secret" switch to all dmd generated programs that the druntime switch checks before main() gets control, such as: app --druntimeSet=usexxx ...the regular app args ...
Back when Druntime was called Ares, it was possible to choose the GC at link time. Do we really need to defer the decision to run time? If so, switching GCs after the app has started should work in most cases, though I'm not sure offhand if memory allocated by the prior GC will be scanned through for references within the new GC.
Oct 09 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 9 October 2014 at 14:20:07 UTC, Sean Kelly wrote:
 Back when Druntime was called Ares, it was possible to choose 
 the GC at link time.  Do we really need to defer the decision 
 to run time?  If so, switching GCs after the app has started 
 should work in most cases, though I'm not sure offhand if 
 memory allocated by the prior GC will be scanned through for 
 references within the new GC.
This also forces GC implementations to aware of each other. Not normally an issue but GC traditionally includes many extern(C) functions with common names which will need to be always implemented so that all GC implementations are supported. At the same time I don't see what real benefit such runtime options brings to the table. This is why in my PR garbage collector is currently chosen during compilation time.
Oct 09 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/9/2014 7:25 AM, Dicebot wrote:
 At the same time I don't see what real benefit such runtime options brings to
 the table. This is why in my PR garbage collector is currently chosen during
 compilation time.
Choosing at compile time is probably best.
Oct 09 2014
parent reply Leandro Lucarella <luca llucax.com.ar> writes:
Walter Bright, el  9 de October a las 17:28 me escribiste:
 On 10/9/2014 7:25 AM, Dicebot wrote:
At the same time I don't see what real benefit such runtime options brings to
the table. This is why in my PR garbage collector is currently chosen during
compilation time.
Choosing at compile time is probably best.
This is not (only) about picking a GC implementation, but also about GC *options/configuration*. The fact that right now to select between concurrent or not would mean using a different GC altogether is just an implementation detail. As I said, if at some point we can merge both, this wouldn't be necessary. Right now GDGC can disable the concurrent scanning, among other cool things (like enabling memory stomping, enabling logging of allocations to a file, enable logging of collections to a file, controlling the initial pools of memory when the program starts, etc.). This is very convenient to turn on/off not exactly at *runtime* but what I call *initialization time* or program startup. Because sometimes recompiling the program with different parameters is quite annoying, and as said before, for stuff that needs to be initialized *before* any actual D code is executed, sometimes is not easy to be done *inside* D code in a way that's not horrible and convoluted. I still don't understand why wouldn't we use environment variables for what they've been created for, it's foolish :-) -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- For long you live and high you fly But only if you ride the tide And balanced on the biggest wave You race towards an early grave.
Oct 10 2014
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/10/14, 5:45 PM, Leandro Lucarella wrote:
 I still don't understand why wouldn't we use environment variables for
 what they've been created for, it's foolish:-)
I, too, think envvars are quite appropriate here. -- Andrei
Oct 10 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/10/2014 5:45 PM, Leandro Lucarella wrote:
 I still don't understand why wouldn't we use environment variables for
 what they've been created for, it's foolish :-)
Because using environment variables to tune program X will also affect programs A-Z.
Oct 10 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/10/14, 7:54 PM, Walter Bright wrote:
 On 10/10/2014 5:45 PM, Leandro Lucarella wrote:
 I still don't understand why wouldn't we use environment variables for
 what they've been created for, it's foolish :-)
Because using environment variables to tune program X will also affect programs A-Z.
Nope. Try this at your Unix command prompt: echo $CRAP CRAP=hello echo $CRAP CRAP=world echo $CRAP Andrei
Oct 10 2014
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/10/2014 9:25 PM, Andrei Alexandrescu wrote:
 On 10/10/14, 7:54 PM, Walter Bright wrote:
 On 10/10/2014 5:45 PM, Leandro Lucarella wrote:
 I still don't understand why wouldn't we use environment variables for
 what they've been created for, it's foolish :-)
Because using environment variables to tune program X will also affect programs A-Z.
Nope. Try this at your Unix command prompt: echo $CRAP CRAP=hello echo $CRAP CRAP=world echo $CRAP
Windows doesn't work like that.
Oct 10 2014
prev sibling next sibling parent reply dennis luehring <dl.soluz gmx.net> writes:
Am 11.10.2014 06:25, schrieb Andrei Alexandrescu:
 On 10/10/14, 7:54 PM, Walter Bright wrote:
 On 10/10/2014 5:45 PM, Leandro Lucarella wrote:
 I still don't understand why wouldn't we use environment variables for
 what they've been created for, it's foolish :-)
Because using environment variables to tune program X will also affect programs A-Z.
Nope. Try this at your Unix command prompt: echo $CRAP CRAP=hello echo $CRAP CRAP=world echo $CRAP
in windows there are user-environment-variables (walter talking about) and shell-environment variables (like in your example) setting user-environement variables will affect every program thats why java is not using them
Oct 10 2014
parent reply Paulo Pinto <pjmlp progtools.org> writes:
Am 11.10.2014 um 06:43 schrieb dennis luehring:
 Am 11.10.2014 06:25, schrieb Andrei Alexandrescu:
 On 10/10/14, 7:54 PM, Walter Bright wrote:
 On 10/10/2014 5:45 PM, Leandro Lucarella wrote:
 I still don't understand why wouldn't we use environment variables for
 what they've been created for, it's foolish :-)
Because using environment variables to tune program X will also affect programs A-Z.
Nope. Try this at your Unix command prompt: echo $CRAP CRAP=hello echo $CRAP CRAP=world echo $CRAP
in windows there are user-environment-variables (walter talking about) and shell-environment variables (like in your example) setting user-environement variables will affect every program thats why java is not using them
And lets not forget about OS/400 or any of the other non-POSIX systems out there, unless D is never expected to target such OSs. -- Paulo
Oct 11 2014
parent reply Leandro Lucarella <luca llucax.com.ar> writes:
Paulo Pinto, el 11 de October a las 10:50 me escribiste:
 Am 11.10.2014 um 06:43 schrieb dennis luehring:
Am 11.10.2014 06:25, schrieb Andrei Alexandrescu:
On 10/10/14, 7:54 PM, Walter Bright wrote:
On 10/10/2014 5:45 PM, Leandro Lucarella wrote:
I still don't understand why wouldn't we use environment variables for
what they've been created for, it's foolish :-)
Because using environment variables to tune program X will also affect programs A-Z.
Nope. Try this at your Unix command prompt: echo $CRAP CRAP=hello echo $CRAP CRAP=world echo $CRAP
in windows there are user-environment-variables (walter talking about) and shell-environment variables (like in your example) setting user-environement variables will affect every program thats why java is not using them
And lets not forget about OS/400 or any of the other non-POSIX systems out there, unless D is never expected to target such OSs.
You can use different mechanisms in different OSs. There is no need to force a runtime to be OS-independent. If that were the case, then we should close the concurrent GC pull request now. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- Y vos, me dijiste que soy, un abismo de silencio. Será, porque vos no escuchas, que yo grito por dentro.
Oct 11 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/11/2014 3:59 PM, Leandro Lucarella wrote:
 You can use different mechanisms in different OSs. There is no need to
 force a runtime to be OS-independent. If that were the case, then we
 should close the concurrent GC pull request now.
I still don't see why it can't use a special argument to the C main() function.
Oct 11 2014
parent Leandro Lucarella <luca llucax.com.ar> writes:
Walter Bright, el 11 de October a las 16:39 me escribiste:
 On 10/11/2014 3:59 PM, Leandro Lucarella wrote:
You can use different mechanisms in different OSs. There is no need to
force a runtime to be OS-independent. If that were the case, then we
should close the concurrent GC pull request now.
I still don't see why it can't use a special argument to the C main() function.
You can, but as someone said, sometimes you don't have control over how the program is started or there is no main. Is not the most general mechanism. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- Hey you, out there on your own Sitting naked by the phone Would you touch me?
Oct 12 2014
prev sibling parent reply Leandro Lucarella <luca llucax.com.ar> writes:
Andrei Alexandrescu, el 10 de October a las 21:25 me escribiste:
 On 10/10/14, 7:54 PM, Walter Bright wrote:
On 10/10/2014 5:45 PM, Leandro Lucarella wrote:
I still don't understand why wouldn't we use environment variables for
what they've been created for, it's foolish :-)
Because using environment variables to tune program X will also affect programs A-Z.
Nope. Try this at your Unix command prompt: echo $CRAP CRAP=hello echo $CRAP CRAP=world echo $CRAP
Your example is actually broken, because this is parsed by the shell separately, CRAP=hello is passed as an env variable to the echo command but the $CRAP expansion is evaluated before the command is called, so it will always have the value that had before every echo call (which is empty if you didn't define it before running those 3 commands). But try this for example: LANG=nonexistent perl /dev/null And see how your perl complaints. But anyway, yeah, that's the idea, there are many ways to define environment variables, and usually you never do so globally (except for things that you really want to affect the whole system, like the language). That doesn't mean you can't override it for a particular program. You can also create scripts to run programs, this is how Firefox runs for example: --- $ file /usr/bin/firefox /usr/bin/firefox: symbolic link to `../lib/firefox/firefox.sh' $ file /usr/lib/firefox/firefox.sh /usr/lib/firefox/firefox.sh: POSIX shell script, ASCII text executable $ grep -v '^#' /usr/lib/firefox/firefox.sh | head set -e MOZ_LIBDIR=/usr/lib/firefox MOZ_APP_LAUNCHER=`which $0` MOZ_APP_NAME=firefox MOZ_DEFAULT_PROFILEDIR=.mozilla/firefox MOZ_PROFILEDIR=.mozilla/firefox --- It basically defines a bunch of environment variables and run the binary. This is a super common practice in posix systems. We are not inventing anything here. I don't know how windows or other OSs deal with defining environment variables in a script. Very basic facilities are always configured this way, for example try man 3 mallopt to see how can you change options in the malloc implementation using environment variables... -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- Ladrón no es cualquiera, ladrón es quien usurpa el bien ajeno en beneficio propio, si no, no. -- Ricardo Vaporeso
Oct 11 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/11/2014 4:23 PM, Leandro Lucarella wrote:
 It basically defines a bunch of environment variables and run the
 binary. This is a super common practice in posix systems. We are not
 inventing anything here. I don't know how windows or other OSs deal with
 defining environment variables in a script.

 Very basic facilities are always configured this way, for example try
 man 3 mallopt to see how can you change options in the malloc
 implementation using environment variables...
I don't deny it is common practice on Linux, but defining a bunch of environment variables and then running the app, i.e. using the ev's as command line switches, seems pointless. Just use command line switches. Anyhow, environment variables are a common source of problems on Windows, which is why dmd, for example, uses dmd.conf instead.
Oct 11 2014
next sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 12.10.2014 05:41, Walter Bright wrote:
 On 10/11/2014 4:23 PM, Leandro Lucarella wrote:
 It basically defines a bunch of environment variables and run the
 binary. This is a super common practice in posix systems. We are not
 inventing anything here. I don't know how windows or other OSs deal with
 defining environment variables in a script.

 Very basic facilities are always configured this way, for example try
 man 3 mallopt to see how can you change options in the malloc
 implementation using environment variables...
I don't deny it is common practice on Linux, but defining a bunch of environment variables and then running the app, i.e. using the ev's as command line switches, seems pointless. Just use command line switches. Anyhow, environment variables are a common source of problems on Windows, which is why dmd, for example, uses dmd.conf instead.
C# programs also use app.exe.config files alongside the executable to setup the application. Maybe we could do something similar. I just found this in msbuild/14.0/bin/csc.exe.config: <configuration> <runtime> <gcServer enabled="true" /> <gcConcurrent enabled="false"/> </runtime> </configuration>
Oct 12 2014
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2014-10-12 10:30, Rainer Schuetze wrote:

 C# programs also use app.exe.config files alongside the executable to
 setup the application. Maybe we could do something similar.

 I just found this in msbuild/14.0/bin/csc.exe.config:

 <configuration>
    <runtime>
      <gcServer enabled="true" />
      <gcConcurrent enabled="false"/>
    </runtime>
 </configuration>
Eclipse uses a similar approach. It has an INI file with command line arguments. Mine as 18 arguments passed to the application. -- /Jacob Carlborg
Oct 12 2014
prev sibling parent reply Leandro Lucarella <luca llucax.com.ar> writes:
Rainer Schuetze, el 12 de October a las 10:30 me escribiste:
 On 12.10.2014 05:41, Walter Bright wrote:
On 10/11/2014 4:23 PM, Leandro Lucarella wrote:
It basically defines a bunch of environment variables and run the
binary. This is a super common practice in posix systems. We are not
inventing anything here. I don't know how windows or other OSs deal with
defining environment variables in a script.

Very basic facilities are always configured this way, for example try
man 3 mallopt to see how can you change options in the malloc
implementation using environment variables...
I don't deny it is common practice on Linux, but defining a bunch of environment variables and then running the app, i.e. using the ev's as command line switches, seems pointless. Just use command line switches. Anyhow, environment variables are a common source of problems on Windows, which is why dmd, for example, uses dmd.conf instead.
C# programs also use app.exe.config files alongside the executable to setup the application. Maybe we could do something similar. I just found this in msbuild/14.0/bin/csc.exe.config: <configuration> <runtime> <gcServer enabled="true" /> <gcConcurrent enabled="false"/> </runtime> </configuration>
Windows only please! :-) Also, completely unflexible, so to run 2 instances of the same program with different configuration you have to run one, modify the config file and then run the seconds? OUCH. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- De tan fina la condesa, por no cagarse, reza. -- Ricardo Vaporeso
Oct 12 2014
parent Andrej Mitrovic via Digitalmars-d-announce writes:
On 10/12/14, Leandro Lucarella via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:
 Also, completely unflexible, so to run 2 instances of the same program
 with different configuration you have to run one, modify the config file
 and then run the seconds?
Perhaps there would be a --gc-config=file.conf you could use. But yeah, ultimately it seems to me we're just reinventing what envars are all about..
Oct 12 2014
prev sibling parent Leandro Lucarella <luca llucax.com.ar> writes:
Walter Bright, el 11 de October a las 20:41 me escribiste:
 On 10/11/2014 4:23 PM, Leandro Lucarella wrote:
It basically defines a bunch of environment variables and run the
binary. This is a super common practice in posix systems. We are not
inventing anything here. I don't know how windows or other OSs deal with
defining environment variables in a script.

Very basic facilities are always configured this way, for example try
man 3 mallopt to see how can you change options in the malloc
implementation using environment variables...
I don't deny it is common practice on Linux, but defining a bunch of environment variables and then running the app, i.e. using the ev's as command line switches, seems pointless. Just use command line switches.
Besides the extra flexibility, as mentioned many times, historically command-line parsing is supposed to be owned by the user's code. Libraries and runtimes are configured via environment variables. So, is more flexible and is what a Linux user would expect.
 Anyhow, environment variables are a common source of problems on
 Windows, which is why dmd, for example, uses dmd.conf instead.
Fair enough, then maybe we should support them only on posix systems. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- ¿Cómo estais? ¿Cómo os senteis hoy 29 del membre de 1961 día en que conmemoreramos la nonésima setima nebulización del martir Peperino Pómoro junto al Rolo Puente en la ciudad de Jadad? -- Peperino Pómoro
Oct 12 2014
prev sibling parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Sat, 11 Oct 2014 01:45:48 +0100, Leandro Lucarella <luca llucax.com.ar>  
wrote:

 Walter Bright, el  9 de October a las 17:28 me escribiste:
 On 10/9/2014 7:25 AM, Dicebot wrote:
At the same time I don't see what real benefit such runtime options  
brings to
the table. This is why in my PR garbage collector is currently chosen  
during
compilation time.
Choosing at compile time is probably best.
This is not (only) about picking a GC implementation, but also about GC *options/configuration*. The fact that right now to select between concurrent or not would mean using a different GC altogether is just an implementation detail. As I said, if at some point we can merge both, this wouldn't be necessary. Right now GDGC can disable the concurrent scanning, among other cool things (like enabling memory stomping, enabling logging of allocations to a file, enable logging of collections to a file, controlling the initial pools of memory when the program starts, etc.). This is very convenient to turn on/off not exactly at *runtime* but what I call *initialization time* or program startup. Because sometimes recompiling the program with different parameters is quite annoying, and as said before, for stuff that needs to be initialized *before* any actual D code is executed, sometimes is not easy to be done *inside* D code in a way that's not horrible and convoluted. I still don't understand why wouldn't we use environment variables for what they've been created for, it's foolish :-)
As mentioned this is not a very windows friendly/like solution. Wouldn't it be more generally useful to have another function like main() called init() which if present (optional) is called before/during initialisation. It would be passed the command line arguments. Then a program can chose to implement it, and can use it to configure the GC in any manner it likes. Seems like this could be generally useful in addition to solving this issue. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Oct 14 2014
next sibling parent reply "Dylan Knutson" <tcdknutson gmail.com> writes:
 Wouldn't it be more generally useful to have another function 
 like main() called init() which if present (optional) is called 
 before/during initialisation.  It would be passed the command 
 line arguments.  Then a program can chose to implement it, and 
 can use it to configure the GC in any manner it likes.

 Seems like this could be generally useful in addition to 
 solving this issue.
Isn't this what module constructors are for? As for passed in parameters, I'm sure there's a cross platform way to retrieve them without bring passed them directly, ala how Rust does it.
Oct 16 2014
next sibling parent reply ketmar via Digitalmars-d-announce <digitalmars-d-announce puremagic.com> writes:
On Thu, 16 Oct 2014 08:10:38 +0000
Dylan Knutson via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 I'm sure there's a cross platform way to retrieve=20
 them without bring passed them directly
there isn't.
Oct 16 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 16 October 2014 at 10:56:49 UTC, ketmar via 
Digitalmars-d-announce wrote:
 On Thu, 16 Oct 2014 08:10:38 +0000
 Dylan Knutson via Digitalmars-d-announce
 <digitalmars-d-announce puremagic.com> wrote:

 I'm sure there's a cross platform way to retrieve them without 
 bring passed them directly
there isn't.
Runtime.args?
Oct 16 2014
next sibling parent ketmar via Digitalmars-d-announce <digitalmars-d-announce puremagic.com> writes:
On Thu, 16 Oct 2014 13:54:25 +0000
Sean Kelly via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 I'm sure there's a cross platform way to retrieve them without=20
 bring passed them directly
there isn't.
Runtime.args?
it's good, but it isn't modifiable. so programmer must manually ignore gc-related args. yet yes, i was wrong. there is a way to acces CLI args. but there is no documented way to modify 'em.
Oct 16 2014
prev sibling parent Ben Boeckel via Digitalmars-d-announce writes:
On Thu, Oct 16, 2014 at 22:12:05 +0300, ketmar via Digitalmars-d-announce wrote:
 it's good, but it isn't modifiable. so programmer must manually ignore
 gc-related args.
There's what Haskell (or at least GHC) does: ./my-haskell-program +RTS --runtime-arg1 --runtime-arg2 -RTS --program-arg1 The program doesn't see anything between +RTS and -RTS arguments. --Ben
Oct 16 2014
prev sibling next sibling parent "Regan Heath" <regan netmail.co.nz> writes:
On Thu, 16 Oct 2014 09:10:38 +0100, Dylan Knutson <tcdknutson gmail.com>  
wrote:

 Wouldn't it be more generally useful to have another function like  
 main() called init() which if present (optional) is called  
 before/during initialisation.  It would be passed the command line  
 arguments.  Then a program can chose to implement it, and can use it to  
 configure the GC in any manner it likes.

 Seems like this could be generally useful in addition to solving this  
 issue.
Isn't this what module constructors are for? As for passed in parameters, I'm sure there's a cross platform way to retrieve them without bring passed them directly, ala how Rust does it.
Provided module constructors occur early enough in the process I guess this would work. You would need to ensure the module constructor doing the GC configuration occurred first too I guess. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Oct 16 2014
prev sibling parent reply Leandro Lucarella <luca llucax.com.ar> writes:
Dylan Knutson, el 16 de October a las 08:10 me escribiste:
Wouldn't it be more generally useful to have another function like
main() called init() which if present (optional) is called
before/during initialisation.  It would be passed the command line
arguments.  Then a program can chose to implement it, and can use
it to configure the GC in any manner it likes.

Seems like this could be generally useful in addition to solving
this issue.
Isn't this what module constructors are for?
No, this needs to be configured WAY before any module constructor even runs... -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- Are you such a dreamer? To put the world to rights? I'll stay home forever Where two & two always makes up five
Oct 16 2014
parent reply "thedeemon" <dlang thedeemon.com> writes:
Gentlemen, do I understand correctly that you're trying to find a 
Windows-friendly switch to something that will never see the 
light on Windows (because of being based on fork)?
Oct 16 2014
parent reply "Martin Nowak" <code dawg.eu> writes:
On Friday, 17 October 2014 at 05:38:05 UTC, thedeemon wrote:
 Gentlemen, do I understand correctly that you're trying to find 
 a Windows-friendly switch to something that will never see the 
 light on Windows (because of being based on fork)?
No we want general runtime configuration, not only for the forking GC. https://github.com/D-Programming-Language/druntime/pull/986
Oct 17 2014
parent "Sebastiaan Koppe" <mail skoppe.eu> writes:
On Friday, 17 October 2014 at 08:28:23 UTC, Martin Nowak wrote:
 On Friday, 17 October 2014 at 05:38:05 UTC, thedeemon wrote:
 Gentlemen, do I understand correctly that you're trying to 
 find a Windows-friendly switch to something that will never 
 see the light on Windows (because of being based on fork)?
No we want general runtime configuration, not only for the forking GC. https://github.com/D-Programming-Language/druntime/pull/986
But doesn't that make the goal of testing the concurrent gc dependent on the problem of finding a method for cross-platform runtime configuration? While I do understand the need, I do not see why we need to come up with a perfect solution for one thing, before we start on what we really wanted to do in the first place.
Oct 20 2014
prev sibling parent reply Leandro Lucarella <luca llucax.com.ar> writes:
Regan Heath, el 14 de October a las 11:11 me escribiste:
I still don't understand why wouldn't we use environment variables for
what they've been created for, it's foolish :-)
As mentioned this is not a very windows friendly/like solution.
As mentioned you don't have to use a unique cross-platform solution, you can have different solutions for different OSs. No need to lower down to the worse solution.
 Wouldn't it be more generally useful to have another function like
 main() called init() which if present (optional) is called
 before/during initialisation.  It would be passed the command line
 arguments.  Then a program can chose to implement it, and can use it
 to configure the GC in any manner it likes.
 
 Seems like this could be generally useful in addition to solving
 this issue.
It is nice, but a) a different issue, this doesn't provide "initialization time" configuration. Think of development vs. devops. If devops needs to debug a problem they could potentially re-run the application activating GC logging, or GC stomping. No need to recompile, no need to even have access to the source code. And b) where would this init() function live? You'll have to define it always, even if you don't want to customize anything, otherwise the compiler will have to somehow figure out if one is provided or not and if is not provided, generate a default one. Not a simple solution to implement. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- Refalar: acto de mover el peso de la masa hacia un lugar equivocado pero concreto. Todo refalo es, por cierto, una sucesión de pequeñísimos movimientos a los que un centímetro es la proporción aumentada de miles de porciones de espacio, que, al estar el piso mojado, refala. -- Ricardo Vaporeso
Oct 16 2014
parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Fri, 17 Oct 2014 00:01:39 +0100, Leandro Lucarella <luca llucax.com.ar>  
wrote:

 Regan Heath, el 14 de October a las 11:11 me escribiste:
I still don't understand why wouldn't we use environment variables for
what they've been created for, it's foolish :-)
As mentioned this is not a very windows friendly/like solution.
As mentioned you don't have to use a unique cross-platform solution, you can have different solutions for different OSs. No need to lower down to the worse solution.
You've got it backwards. I'm looking for a *better* solution than environment variables, which are a truly horrid way to control runtime behaviour IMHO. Something built into the language or runtime itself. And, better yet would be something that is more generally useful - not limited to GC init etc.
 Wouldn't it be more generally useful to have another function like
 main() called init() which if present (optional) is called
 before/during initialisation.  It would be passed the command line
 arguments.  Then a program can chose to implement it, and can use it
 to configure the GC in any manner it likes.

 Seems like this could be generally useful in addition to solving
 this issue.
It is nice, but a) a different issue, this doesn't provide "initialization time" configuration.
I don't follow. You want to execute some code A before other code B occurs. This meets that requirement - assuming init() is called at the point you need it to be called.
 Think of development vs. devops. If
 devops needs to debug a problem they could potentially re-run the
 application activating GC logging, or GC stomping. No need to recompile,
 no need to even have access to the source code.
./application -gclog ./application -gcstomp ..code.. init(string[] args) { if .. } Not need to recompile. Some GC options might make sense for all D applications, in that case the compiler default init() could handle those and custom init() functions would simply call it, and handle any extra custom options. Other GC/allocation options might be very application specific i.e. perhaps the application code cannot support RC for some reason, etc.
 And b) where would this init() function live? You'll have to define it
 always
Surely not.
 , even if you don't want to customize anything, otherwise the
 compiler will have to somehow figure out if one is provided or not and
 if is not provided, generate a default one. Not a simple solution to
 implement.
Sounds pretty trivial to me. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Oct 17 2014
parent reply Leandro Lucarella <luca llucax.com.ar> writes:
Regan Heath, el 17 de October a las 10:55 me escribiste:
Regan Heath, el 14 de October a las 11:11 me escribiste:
I still don't understand why wouldn't we use environment variables for
what they've been created for, it's foolish :-)
As mentioned this is not a very windows friendly/like solution.
As mentioned you don't have to use a unique cross-platform solution, you can have different solutions for different OSs. No need to lower down to the worse solution.
You've got it backwards. I'm looking for a *better* solution than environment variables, which are a truly horrid way to control runtime behaviour IMHO.
OK, then this is now a holly war. So I'll drop it here. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- Mi infancia fue en un loft, bien al costado del río Cazabamos correcaminos y lo asábamos en el fogón Después? Después me vine grande y la ciudad me deslumbró Jugando al tejo en Lavalle me hice amigo del bongó
Oct 17 2014
parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Fri, 17 Oct 2014 11:18:27 +0100, Leandro Lucarella <luca llucax.com.ar>  
wrote:

 Regan Heath, el 17 de October a las 10:55 me escribiste:
Regan Heath, el 14 de October a las 11:11 me escribiste:
I still don't understand why wouldn't we use environment variables  
for
what they've been created for, it's foolish :-)
As mentioned this is not a very windows friendly/like solution.
As mentioned you don't have to use a unique cross-platform solution,
you
can have different solutions for different OSs. No need to lower down  
to
the worse solution.
You've got it backwards. I'm looking for a *better* solution than environment variables, which are a truly horrid way to control runtime behaviour IMHO.
OK, then this is now a holly war. So I'll drop it here.
I think you've mistook my tone. I am not "religious" about this. I just think it's a bad idea for a program to alter behaviour based on a largely invisible thing (environment variable). It's far better to have a command line switch staring you in the face. Plus as Walter mentioned the environment variable is a bit like a shotgun, it could potentially affect every program executed from that context. We have a product here which uses env vars for trace flags and (without having separate var for each process) you cannot turn on trace for a single process in an execution tree, instead each child inherits the parent environment and starts to trace. And.. when some of those flags have different meanings in different processes it gets even worse. Especially if one of those flags prints debugging to stdout, and the process is run as a child where input/output are parsed.. it just plain doesn't work. It's a mess. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Oct 17 2014
parent reply Leandro Lucarella <luca llucax.com.ar> writes:
Regan Heath, el 17 de October a las 15:43 me escribiste:
You've got it backwards.  I'm looking for a *better* solution than
environment variables, which are a truly horrid way to control
runtime behaviour IMHO.
OK, then this is now a holly war. So I'll drop it here.
I think you've mistook my tone. I am not "religious" about this. I just think it's a bad idea for a program to alter behaviour based on a largely invisible thing (environment variable). It's far better to have a command line switch staring you in the face.
But it's not the same. I don't mean to be rude, but all you (and Walter) are saying about environment is evidence of not knowing how useful they are in POSIX OSs, what's the history in those OSs and what people expect from them. All these fear about how this can obscurely affect programs is totally unfunded. That just does not happen. Not at least commonly enough to ignore all the other advantages of it. If you keep denying it usefulness and how they are different from command-line arguments, we'll keep going in circles.
 Plus as Walter mentioned the environment variable is a bit like a
 shotgun, it could potentially affect every program executed from
 that context.
 
 We have a product here which uses env vars for trace flags and
 (without having separate var for each process) you cannot turn on
 trace for a single process in an execution tree, instead each child
 inherits the parent environment and starts to trace.
So, your example is a D program, that spawns other D programs, so if you set an environment variable to affect the behaviour of the starting program, you affect also the behaviour of the child programs. This is a good example, and I can argue for environment variables for the same reason. If I want to debug this whole mess, using command-line options there is no way I can affect the whole execution tree to log useful debug information. See, you proved my point, environment variables and command-line arguments are different and thus, useful for different situations.
 And.. when some of those flags have different meanings in different
 processes it gets even worse.
Why would they? Don't create problems where there are any :)
 Especially if one of those flags prints
 debugging to stdout, and the process is run as a child where
 input/output are parsed.. it just plain doesn't work.  It's a mess.
If you write to stdout (without giving the option to write to a log file) then what you did is just broken. Again, there is no point in inventing theoretical situations where you can screw anything up. You can always fabricate those. Let's stay on the domain of reality :) In all the years I've been working in Linux I never, EVER came across problems with environment variables being accidentally set. I find it very hard to believe this is a real problem. On the other hand, they saved my ass several times (LD_PRELOAD I LOVE YOU). -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- Le pedí que me enseñe a usar el mouse Pero solo quiere hablarme del Bauhaus Le pregunté si era chorra o rockera Me dijo "Gertrude Stein era re-tortillera" Me hizo mucho mal la cumbiera intelectual
Oct 17 2014
next sibling parent reply Andrej Mitrovic via Digitalmars-d-announce writes:
On 10/17/14, Leandro Lucarella via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:
 In all the years I've been working in Linux I never, EVER came across
 problems with environment variables being accidentally set. I find it
 very hard to believe this is a real problem. On the other hand, they
 saved my ass several times (LD_PRELOAD I LOVE YOU).
Slightly OT, but having to use Linux for the past 5 months and using almost exclusively Windows before that for the better part of the last ~20 years, I have to say I've become a massive fan of the OS and the way things work. I've come to even hate anything GUI (except maybe the editor), I'd hate to even think about browsing the file system with a mouse anymore. Of course, all of this functionality was available on Windows as well, but what I'm saying is a switch in one's environment (pardon the pun :p) can make them realize that they may have been doing things the slow or inefficient way. So it's not so much about OS A vs OS B, but how used you are to doing things one way. Maybe that's why there's this aversion towards using environment variables. Anyway those were my 2 eurocents..
Oct 17 2014
parent reply "Kagamin" <spam here.lot> writes:
On Friday, 17 October 2014 at 17:53:18 UTC, Andrej Mitrovic via 
Digitalmars-d-announce wrote:
 I've come to even hate anything GUI (except maybe the
 editor), I'd hate to even think about browsing the file system 
 with a
 mouse anymore.
Yes, you can: Windows Explorer does support keyboard input :)
Oct 23 2014
parent ketmar via Digitalmars-d-announce <digitalmars-d-announce puremagic.com> writes:
On Thu, 23 Oct 2014 07:12:25 +0000
Kagamin via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 On Friday, 17 October 2014 at 17:53:18 UTC, Andrej Mitrovic via=20
 Digitalmars-d-announce wrote:
 I've come to even hate anything GUI (except maybe the
 editor), I'd hate to even think about browsing the file system=20
 with a
 mouse anymore.
=20 Yes, you can: Windows Explorer does support keyboard input :)
at least it supports most useful shortcut: alt+f4.
Oct 23 2014
prev sibling parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Fri, 17 Oct 2014 17:54:55 +0100, Leandro Lucarella <luca llucax.com.ar>  
wrote:
 Regan Heath, el 17 de October a las 15:43 me escribiste:
 I think you've mistook my tone.  I am not "religious" about this.  I
 just think it's a bad idea for a program to alter behaviour based on
 a largely invisible thing (environment variable).  It's far better
 to have a command line switch staring you in the face.
But it's not the same. I don't mean to be rude, but all you (and Walter) are saying about environment is evidence of not knowing how useful they are in POSIX OSs
I am aware of how they are used as I have had to deal with them in the past. :)
 what's the history in those OSs and what people expect from them.
D is not simply for these OSs and should be as platform agnostic as possible for core functionality.
 All these fear about how this can obscurely affect programs
 is totally unfunded. That just does not happen. Not at least commonly
 enough to ignore all the other advantages of it.
Sure, but past/current env vars being used are used *privately* to a single program. What you're suggesting here are env vars which will affect *all* D programs that see them. If D takes over the world as we all hope it will then this will be a significantly different situation to what you are used to.
 If you keep denying it usefulness and how they are different from
 command-line arguments, we'll keep going in circles.
I am not denying they are useful. I am denying they are *better* than a command line argument *for this specific use case*
 Plus as Walter mentioned the environment variable is a bit like a
 shotgun, it could potentially affect every program executed from
 that context.

 We have a product here which uses env vars for trace flags and
 (without having separate var for each process) you cannot turn on
 trace for a single process in an execution tree, instead each child
 inherits the parent environment and starts to trace.
So, your example is a D program, that spawns other D programs, so if you set an environment variable to affect the behaviour of the starting program, you affect also the behaviour of the child programs.
Yes. How do you control which of these programs is affected by your global-affects-all-D-programs-env-var?
 This is a good example, and I can argue for environment variables for  
 the same
 reason. If I want to debug this whole mess, using command-line options
 there is no way I can affect the whole execution tree to log useful
 debug information.
Sure you can. You can do whatever you like with an argument, including passing a debug argument to sub-processes as required. Or, you could use custom env vars to do the same thing. What you *do not* want is a global env var that indiscriminately affects every program that sees it, this gives you no control.
 See, you proved my point, environment variables and
 command-line arguments are different and thus, useful for different
 situations.
Sure, but the point is that a global env var that silently controls *all* D programs is a shotgun blast, and we want a needle.
 And.. when some of those flags have different meanings in different
 processes it gets even worse.
Why would they? Don't create problems where there are any :)
Sadly it exists .. I inherited it (the source is 20+ years old).
 Especially if one of those flags prints
 debugging to stdout, and the process is run as a child where
 input/output are parsed.. it just plain doesn't work.  It's a mess.
If you write to stdout (without giving the option to write to a log file) then what you did is just broken. Again, there is no point in inventing theoretical situations where you can screw anything up. You can always fabricate those. Let's stay on the domain of reality :)
Sadly not theoretical. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Oct 20 2014
next sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 20 October 2014 at 10:39:28 UTC, Regan Heath wrote:
 Sure, but past/current env vars being used are used *privately* 
 to a single program.  What you're suggesting here are env vars 
 which will affect *all* D programs that see them.  If D takes 
 over the world as we all hope it will then this will be a 
 significantly different situation to what you are used to.
I'm not advocating the approach, but you could create a "run_d" app that simply set the relevant environment args and then executed the specified app as a child process. The args would be picked up by the app without touching the system environment. This would work on Windows as well as on *nix.
Oct 20 2014
parent "Regan Heath" <regan netmail.co.nz> writes:
On Mon, 20 Oct 2014 18:19:33 +0100, Sean Kelly <sean invisibleduck.org>  
wrote:

 On Monday, 20 October 2014 at 10:39:28 UTC, Regan Heath wrote:
 Sure, but past/current env vars being used are used *privately* to a  
 single program.  What you're suggesting here are env vars which will  
 affect *all* D programs that see them.  If D takes over the world as we  
 all hope it will then this will be a significantly different situation  
 to what you are used to.
I'm not advocating the approach, but you could create a "run_d" app that simply set the relevant environment args and then executed the specified app as a child process. The args would be picked up by the app without touching the system environment. This would work on Windows as well as on *nix.
Sure, but in this case passing an argument is both simpler and clearer (intent). This is basically trying to shoehorn something in where it was never intended to be used, envvars by design are supposed to affect everything running in the environment and they're the wrong tool for the job if you want to target specific processes, which IMO is a requirement we have. A specific example. Imagine we have the equivalent of the windows CRT debug malloc feature bits, i.e. never free or track all allocations etc. These features are very useful, but they are costly. Turning them on for an entire process tree may be unworkable - it may be too slow or consume too much memory. A more targeted approach is required. There are plenty of options, but a global envvar is not one of them. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Oct 21 2014
prev sibling parent reply Leandro Lucarella <luca llucax.com.ar> writes:
Regan Heath, el 20 de October a las 11:39 me escribiste:
 On Fri, 17 Oct 2014 17:54:55 +0100, Leandro Lucarella
 <luca llucax.com.ar> wrote:
Regan Heath, el 17 de October a las 15:43 me escribiste:
I think you've mistook my tone.  I am not "religious" about this.  I
just think it's a bad idea for a program to alter behaviour based on
a largely invisible thing (environment variable).  It's far better
to have a command line switch staring you in the face.
But it's not the same. I don't mean to be rude, but all you (and Walter) are saying about environment is evidence of not knowing how useful they are in POSIX OSs
I am aware of how they are used as I have had to deal with them in the past. :)
OK, then it's even more difficult to understand your concerns or arguments. :S
what's the history in those OSs and what people expect from them.
D is not simply for these OSs and should be as platform agnostic as possible for core functionality.
The runtime is not platform independent AT ALL. Why should you provide a platform agnostic way to configure it? I can understand it if it's free, but if you have to sacrifice something just to get a platform agnostic mechanism, for me it's not worth it at all.
All these fear about how this can obscurely affect programs
is totally unfunded. That just does not happen. Not at least commonly
enough to ignore all the other advantages of it.
Sure, but past/current env vars being used are used *privately* to a single program.
NO, this is completely false, and why I think you are not entirely familiar with env vars in posix. LD_PRELOAD and LD_LIBRARY_PATH affects ALL, EACH and EVERY program for example. D or not D. Every single dynamically linked program.
 What you're suggesting here are env vars which will
 affect *all* D programs that see them.
Yes, *only* D programs, something much less crazy than the standard environment variables that affect every single program (D or not D). :)
 If D takes over the world as we all hope it will then this will be a
 significantly different situation to what you are used to.
No, not at all, not in the posix world.
If you keep denying it usefulness and how they are different from
command-line arguments, we'll keep going in circles.
I am not denying they are useful. I am denying they are *better* than a command line argument *for this specific use case*
OK, fair enough, but I can't buy your arguments, because they are based on false assumptions.
We have a product here which uses env vars for trace flags and
(without having separate var for each process) you cannot turn on
trace for a single process in an execution tree, instead each child
inherits the parent environment and starts to trace.
So, your example is a D program, that spawns other D programs, so if you set an environment variable to affect the behaviour of the starting program, you affect also the behaviour of the child programs.
Yes. How do you control which of these programs is affected by your global-affects-all-D-programs-env-var?
By using setenv() from the spawned program, same as you would pass --command-line-options.
This is a good example, and I can argue for environment variables
for the same
reason. If I want to debug this whole mess, using command-line options
there is no way I can affect the whole execution tree to log useful
debug information.
Sure you can. You can do whatever you like with an argument, including passing a debug argument to sub-processes as required. Or, you could use custom env vars to do the same thing.
But then you have to modify the program that spawns the other processes. In that case, if we assume you can modify that program, you can perfectly use setenv() in the fork()ed process before doing exec().
 What you *do not* want is a global env var that indiscriminately
 affects every program that sees it, this gives you no control.
Why not? You can control it the same as --command-line arguments if you are assuming you control the way programs are started.
See, you proved my point, environment variables and
command-line arguments are different and thus, useful for different
situations.
Sure, but the point is that a global env var that silently controls *all* D programs is a shotgun blast, and we want a needle.
This is what I'm actively denying saying is a myth, you are just scaring children talking about the boogeyman. As I said before, in posix, there are already several env variable that affects each and every program, D or not D, that's dynamically linked. I have the feeling there are env variables that affects glibc too, which would affect every single C/C++/D program, even statically linked programs. I know for a fact there are plenty of libraries that are configured using env variables, so every application using those libraries will be affected by those env variables (libev/libevent or glib[1] meaning each and every gtk+ application is affected, that meaning the whole GNOME desktop environment is affected). This is a super common mechanism. I never ever had problems with this. Did you? Did honestly you even know they existed? [1] https://developer.gnome.org/glib/stable/glib-running.html This is the last time I'll mention this, I'm getting tired to go over and over again about it.
And.. when some of those flags have different meanings in different
processes it gets even worse.
Why would they? Don't create problems where there are any :)
Sadly it exists .. I inherited it (the source is 20+ years old).
OK, someone wrote a bad program 20+ years ago, we should ban env variables! :-)
Especially if one of those flags prints
debugging to stdout, and the process is run as a child where
input/output are parsed.. it just plain doesn't work.  It's a mess.
If you write to stdout (without giving the option to write to a log file) then what you did is just broken. Again, there is no point in inventing theoretical situations where you can screw anything up. You can always fabricate those. Let's stay on the domain of reality :)
Sadly not theoretical.
Anyone can write crappy software, that's besides this point. You are not forced to do it wrong. Just the bare possibility of doing something wrong is not a good reason to not do it right. This problem is fixed by just not doing it wrong. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- "CIRILO" Y "SIRACUSA" DE "SEÑORITA MAESTRA": UNO MUERTO Y OTRO PRESO -- Crónica TV
Oct 21 2014
parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Tue, 21 Oct 2014 23:52:22 +0100, Leandro Lucarella <luca llucax.com.ar>  
wrote:
 The runtime is not platform independent AT ALL.
^ implementation
 Why should you provide a platform agnostic way to configure it?
Because it makes life easier for developers and cross platform development, not to mention documentation. The benefits far outweigh the costs.
 I can understand it if it's free,
 but if you have to sacrifice something just to get a platform agnostic
 mechanism, for me it's not worth it at all.
Reasonable people may disagree.
All these fear about how this can obscurely affect programs
is totally unfunded. That just does not happen. Not at least commonly
enough to ignore all the other advantages of it.
Sure, but past/current env vars being used are used *privately* to a single program.
NO, this is completely false, and why I think you are not entirely familiar with env vars in posix. LD_PRELOAD and LD_LIBRARY_PATH affects ALL, EACH and EVERY program for example. D or not D. Every single dynamically linked program.
True. And the reason these behave this way is because we *always* want them to - the same is NOT true of the proposed vars for D. Which is my point.
 This is a super common mechanism. I never ever had problems with this.
 Did you? Did honestly you even know they existed?
Yes. But this is beside the point, which I hope I have clarified now? Regan -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Oct 22 2014
parent reply Leandro Lucarella <luca llucax.com.ar> writes:
Regan Heath, el 22 de October a las 10:41 me escribiste:
NO, this is completely false, and why I think you are not entirely
familiar with env vars in posix. LD_PRELOAD and LD_LIBRARY_PATH affects
ALL, EACH and EVERY program for example. D or not D. Every single
dynamically linked program.
True. And the reason these behave this way is because we *always* want them to - the same is NOT true of the proposed vars for D.
No, not at all, you very rarely want to change LD_PRELOAD and LD_LIBRARY_PATH globaly.
 Which is my point.
 
This is a super common mechanism. I never ever had problems with this.
Did you? Did honestly you even know they existed?
Yes. But this is beside the point, which I hope I have clarified now?
No. My conclusion is we don't agree mainly on this: I think there are cases where you want runtime configuration to propagate or be set more or less globally. You think no one will ever want some runtime option to propagate. The rest of the argument is based on that difference. Also, I don't have much of a problem with having command-line options to configure the runtime too, although I think in linux/unix is much less natural. Runtime configuration will be most of the time some to be done either by the developer (in which case it would be nicer to have a programatic way to configure it), or on a system level, by a system administrator / devops (in which case for me environment variables are superior for me). Usually runtime options will be completely meaningless for a regular user. Also, will you document them when you use --help? Will you omit them? -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- De tan fina la condesa, por no cagarse, reza. -- Ricardo Vaporeso
Oct 23 2014
parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Thu, 23 Oct 2014 15:27:50 +0100, Leandro Lucarella <luca llucax.com.ar>  
wrote:

 Regan Heath, el 22 de October a las 10:41 me escribiste:
NO, this is completely false, and why I think you are not entirely
familiar with env vars in posix. LD_PRELOAD and LD_LIBRARY_PATH affects
ALL, EACH and EVERY program for example. D or not D. Every single
dynamically linked program.
True. And the reason these behave this way is because we *always* want them to - the same is NOT true of the proposed vars for D.
No, not at all, you very rarely want to change LD_PRELOAD and LD_LIBRARY_PATH globaly.
Sure, but when you do change them you will want them to propagate, by default, which is why envvars are used for these. The same is not true of many potential D GC/allocation/debug flags, we do not necessarily want them to propagate at all and in fact we may want to target a single exe in a process tree i.e. parent <- not this child1 <- this one child2 <- not this
 My conclusion is we don't agree mainly on this:

 I think there are cases where you want runtime configuration to
 propagate or be set more or less globally.
I agree that there are cases we might want it to propagate *from a parent exe downwards* or similar but this is not what I would call "more or less globally" it's very much less than globally. The scope we want is going to be either a single exe, or that exe and some or all of it's children and possibly only for a single execution. Sure, you *could* wrap a single execution in it's own session and only set the envvar within that session but it's far simpler just to pass a command line arg. Likewise, you could set an envvar in a session and run multiple executions within that session, but again it's simpler just to pass an arg each time. Basically, I don't see what positive benefit you get from an envvar over a command line switch, especially if you assume/agree that the most sensible default these switches is 'off' and that they should be enabled specifically. I think what we disagree about here is the scope it should apply and whether propagation should be the default behaviour.
 You think no one will ever want some runtime option to propagate.
Nope, I never said that.
 Also, I don't have much of a problem with having command-line options to
 configure the runtime too, although I think in linux/unix is much less
 natural.
Surely not, unix is the king of command line switches.
 Runtime configuration will be most of the time some to be done
 either by the developer (in which case it would be nicer to have a
 programatic way to configure it)
Agreed.
 or on a system level, by a system
 administrator / devops (in which case for me environment variables are
 superior for me).
Disagree. It's not something we ever want at a system level, it's somewhere within the range of a single session <-> single execution.
 Usually runtime options will be completely meaningless
 for a regular user. Also, will you document them when you use --help?
Or course not, just as you would not document the envvar(s). R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Oct 23 2014
parent Leandro Lucarella <luca llucax.com.ar> writes:
Regan Heath, el 23 de October a las 17:24 me escribiste:
 On Thu, 23 Oct 2014 15:27:50 +0100, Leandro Lucarella
 <luca llucax.com.ar> wrote:
 
Regan Heath, el 22 de October a las 10:41 me escribiste:
NO, this is completely false, and why I think you are not entirely
familiar with env vars in posix. LD_PRELOAD and LD_LIBRARY_PATH affects
ALL, EACH and EVERY program for example. D or not D. Every single
dynamically linked program.
True. And the reason these behave this way is because we *always* want them to - the same is NOT true of the proposed vars for D.
No, not at all, you very rarely want to change LD_PRELOAD and LD_LIBRARY_PATH globaly.
Sure, but when you do change them you will want them to propagate, by default, which is why envvars are used for these.
No, not at all, specially with LD_PRELOAD, I think you almost never want to propagate it. I think LD_PRELOAD is the closest example to what one would want to do with runtime options (and some of the stuff, like replacing the GC, could be done using LD_PRELOAD, actually, but you have to replace it all, you have much less fine grained control, is major surgery).
Also, I don't have much of a problem with having command-line options to
configure the runtime too, although I think in linux/unix is much less
natural.
Surely not, unix is the king of command line switches.
Is not about quantity, is about separation of concerns. Historically in Linux a program only manages the switches it really knows, is not common to hijack programs arguments in Linux (even when is done by some applications, like GTK+, but is all under your control, you explicitly pass the command-line arguments to the library, so it's quite a different case). Anything that you don't control in your program, is handled by environment variables.
or on a system level, by a system
administrator / devops (in which case for me environment variables are
superior for me).
Disagree. It's not something we ever want at a system level, it's somewhere within the range of a single session <-> single execution.
Why? On a single core I want ALL my applications by default NOT to use the concurrent GC, as it will make things slower, while on a multi-core I want to use the concurrent GC by default. You have an use case right there. If I have tons of memory, I would want to have all my programs preallocate 100MiB to speed up things. There might be more in the future, who knows... -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- Parece McGuevara's o CheDonald's
Oct 23 2014
prev sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Wednesday, 8 October 2014 at 00:18:16 UTC, Walter Bright wrote:
 Sort of like:

   user: "need to fix this on the website"
   n.g.: "thanks! fixed now!"
   user: "no it isn't"
   n.g.: "looks good to me"
   [lots of fruitless back and forth]
   n.g.: "did you refresh your browser's cache?"
   user: "oops, sorry!"
OT: I know this is just an example to illustrate a point, but I think this particular story is about the web developer not understanding HTTP caching, or not being aware that their caching headers may cause the fix to not propagate to users indefinitely. :)
Oct 08 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/8/2014 2:55 AM, Vladimir Panteleev wrote:
 On Wednesday, 8 October 2014 at 00:18:16 UTC, Walter Bright wrote:
 Sort of like:

   user: "need to fix this on the website"
   n.g.: "thanks! fixed now!"
   user: "no it isn't"
   n.g.: "looks good to me"
   [lots of fruitless back and forth]
   n.g.: "did you refresh your browser's cache?"
   user: "oops, sorry!"
OT: I know this is just an example to illustrate a point, but I think this particular story is about the web developer not understanding HTTP caching, or not being aware that their caching headers may cause the fix to not propagate to users indefinitely. :)
There are similar recurring discussions about the path to where the D libraries are, such as "I deleted all the old versions of dmd from my system, but it is still using them!" Using environment variables is a recurring support problem, even for people who understand them.
Oct 08 2014
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Tuesday, 7 October 2014 at 20:06:43 UTC, Walter Bright wrote:
 On 10/6/2014 9:51 AM, Dicebot wrote:
 https://github.com/D-Programming-Language/druntime/pull/985
Thank you. This is great progress! I understand the caveats, but can this be put into a shape where it can be pulled despite being a work in progress? I.e. have the code be disabled by default? That will help encourage the community to help out with the gaps.
BTW, why not dub package? GC is pluggable, so can be shipped as a 3rd-party library.
Oct 23 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 23 October 2014 at 07:09:04 UTC, Kagamin wrote:
 On Tuesday, 7 October 2014 at 20:06:43 UTC, Walter Bright wrote:
 On 10/6/2014 9:51 AM, Dicebot wrote:
 https://github.com/D-Programming-Language/druntime/pull/985
Thank you. This is great progress! I understand the caveats, but can this be put into a shape where it can be pulled despite being a work in progress? I.e. have the code be disabled by default? That will help encourage the community to help out with the gaps.
BTW, why not dub package? GC is pluggable, so can be shipped as a 3rd-party library.
Currently druntime is packaged as part of Phobos library binary so I don't see easy way to do it. It could be possible to do the replacement at the startup using GC Proxy instrastructure but CDGC does not currently support it (though it is not complicated to do).
Oct 23 2014
parent reply "Kagamin" <spam here.lot> writes:
On Thursday, 23 October 2014 at 08:46:46 UTC, Dicebot wrote:
 Currently druntime is packaged as part of Phobos library binary 
 so I don't see easy way to do it. It could be possible to do 
 the replacement at the startup using GC Proxy instrastructure 
 but CDGC does not currently support it (though it is not 
 complicated to do).
The idea is that you specify CDGC dub package a dependency for your application and dub links your application with it, so you end up with CDGC in your application.
Oct 23 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 23 October 2014 at 09:53:44 UTC, Kagamin wrote:
 On Thursday, 23 October 2014 at 08:46:46 UTC, Dicebot wrote:
 Currently druntime is packaged as part of Phobos library 
 binary so I don't see easy way to do it. It could be possible 
 to do the replacement at the startup using GC Proxy 
 instrastructure but CDGC does not currently support it (though 
 it is not complicated to do).
The idea is that you specify CDGC dub package a dependency for your application and dub links your application with it, so you end up with CDGC in your application.
Uh, what will it link to? extern(C) function like gc_malloc? druntime links to those statically itself so you will get application that links to different GC than druntime itself. Or am I misunderstanding what you propose?
Oct 23 2014
parent reply "Kagamin" <spam here.lot> writes:
On Thursday, 23 October 2014 at 09:57:38 UTC, Dicebot wrote:
 Uh, what will it link to? extern(C) function like gc_malloc?
 druntime links to those statically itself so you will get 
 application that links to different GC than druntime itself.
The reason is CDGC provides symbols like gc_malloc and after that the linker has no reason to pull default GC from the library: it doesn't pull what's not necessary.
Oct 23 2014
parent reply "Mathias LANG" <geod24 gmail.com> writes:
On Thursday, 23 October 2014 at 11:03:35 UTC, Kagamin wrote:
 On Thursday, 23 October 2014 at 09:57:38 UTC, Dicebot wrote:
 Uh, what will it link to? extern(C) function like gc_malloc?
 druntime links to those statically itself so you will get 
 application that links to different GC than druntime itself.
The reason is CDGC provides symbols like gc_malloc and after that the linker has no reason to pull default GC from the library: it doesn't pull what's not necessary.
It will clash at best, or just ignore cdgc, as objects are considered as a whole, in link order. Now, if druntime defines gc_alloc, it won't link. You have to extract all conflicting symbols out of druntime. However, one could provide a full druntime patched with CDGC as a dub module, and that might work, I suppose.
Oct 23 2014
parent reply "Kagamin" <spam here.lot> writes:
On Thursday, 23 October 2014 at 13:13:06 UTC, Mathias LANG wrote:
 It will clash at best, or just ignore cdgc, as objects are 
 considered as a whole, in link order.
At best, they won't clash :) If the default GC is not pulled by the linker, why should they clash?
 Now, if druntime defines gc_alloc, it won't link. You have to 
 extract all conflicting symbols out of druntime.
Do you consider possibility of breaking changes as a blocker for feasibility of 3rd-party libraries?
Oct 23 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 23 October 2014 at 14:02:33 UTC, Kagamin wrote:
 On Thursday, 23 October 2014 at 13:13:06 UTC, Mathias LANG 
 wrote:
 It will clash at best, or just ignore cdgc, as objects are 
 considered as a whole, in link order.
At best, they won't clash :) If the default GC is not pulled by the linker, why should they clash?
We could experiment with separately linking the GC. It wouldn't be hard to do, though the link line might be a bit weird, since core, rt, and gc are all interdependent in terms of link dependencies.
Oct 23 2014
parent reply "Kagamin" <spam here.lot> writes:
On Thursday, 23 October 2014 at 15:53:19 UTC, Sean Kelly wrote:
 We could experiment with separately linking the GC.  It 
 wouldn't be hard to do, though the link line might be a bit 
 weird, since core, rt, and gc are all interdependent in terms 
 of link dependencies.
Can't it work like any other user library?
Oct 23 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Friday, 24 October 2014 at 06:50:05 UTC, Kagamin wrote:
 On Thursday, 23 October 2014 at 15:53:19 UTC, Sean Kelly wrote:
 We could experiment with separately linking the GC.  It 
 wouldn't be hard to do, though the link line might be a bit 
 weird, since core, rt, and gc are all interdependent in terms 
 of link dependencies.
Can't it work like any other user library?
Well, on linux at least I think you might have to list it twice. Once before and once after libdruntime. I don't know if there's a portable linker flag that indicates that it should try and resolve dependencies in libraries listed later in the link line.
Oct 24 2014
parent "Kagamin" <spam here.lot> writes:
This means, the program should link with object file instead of a 
library. Alternatively it can be pulled eagerly by the proposed 
`init` function if it will explicitly depend on a symbol from a 
gc library - the linker could resolve the symbol early.
Oct 27 2014
prev sibling next sibling parent "Joseph Rushton Wakeling" <joseph.wakeling webdrake.net> writes:
On Monday, 6 October 2014 at 16:51:07 UTC, Dicebot wrote:
 https://github.com/D-Programming-Language/druntime/pull/985

 Here is initial port result available for early experiments. It 
 can be compiled with make -f posix.mak GC_TYPE=concurrent and 
 passes the test suite with only shared library tests disabled
This is fantastic, thanks and congratulations! :-) I'm afraid I'm getting a build error when trying to build lib/libdruntime-linux64so.a. No obvious reasons in the log file, and copy-pasting the dmd call directly into the console doesn't shed any light (it doesn't output any errors, but nor does it create the output file; there are a lot of Deprecation warnings, but those are present also when trying to build the regular runtime). I'm on Ubuntu 14.04, just for reference, and trying to build this with a freshly-built from-git-master dmd.
Oct 07 2014
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
Marginally related: Page fault handling in user space.
http://lwn.net/Articles/615086/

Maybe this can be used as an alternative to forking.
Oct 17 2014
parent "Martin Nowak" <code dawg.eu> writes:
On Friday, 17 October 2014 at 11:30:48 UTC, Marc Schütz wrote:
 Marginally related: Page fault handling in user space.
 http://lwn.net/Articles/615086/

 Maybe this can be used as an alternative to forking.
Definitely good news for moving GCs.
Oct 29 2014