www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.experimental Timeline

reply Jack Stouffer <jack jackstouffer.com> writes:
Currently, we have two modules in std.experimental with a third 
on the way and potentially a single function as well: 
https://github.com/D-Programming-Language/phobos/pull/2945/files

Yet it's a bit concerning that there's no plan on when any of 
these will be moved into the main namespace.

std.logger has been in std.experimental for eight releases and 
eight months. Robert Schadek believes it's blocked by the non 
existant RCString: 
https://github.com/D-Programming-Language/phobos/pull/1500issuecomment-155457980

Is that actually the case? How long can we wait for something 
that might not happen and would take months of work (if that was 
one person's sole focus, longer if they had other things to do)?

Also, knowing when std.allocator will be moved is important, 
because certain functions should integrate std.allocator as soon 
as it's available and have them usable in the same release. 
People who would contribute those changes need to know when that 
is going to happen in order to get the necessary work done before 
hand.
Dec 31 2015
next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 12/31/15 12:45 PM, Jack Stouffer wrote:
 Currently, we have two modules in std.experimental with a third on the
 way and potentially a single function as well:
 https://github.com/D-Programming-Language/phobos/pull/2945/files

 Yet it's a bit concerning that there's no plan on when any of these will
 be moved into the main namespace.
I'm not concerned. Code can live in experimental as long as it's needed to flesh out the API. The idea behind std.experimental is to give us a disclaimer that the API may change at any release despite usage. So the timeline is "when it's ready"
 std.logger has been in std.experimental for eight releases and eight
 months. Robert Schadek believes it's blocked by the non existant
 RCString:
 https://github.com/D-Programming-Language/phobos/pull/1500issuecomment-155457980
I think this is probably something that needs revisiting. I don't remember the rationale behind delaying the movement of logger into std.
 Also, knowing when std.allocator will be moved is important, because
 certain functions should integrate std.allocator as soon as it's
 available and have them usable in the same release. People who would
 contribute those changes need to know when that is going to happen in
 order to get the necessary work done before hand.
This is somewhat tricky. New functions can just be put in std.experimental. Changed functions are not as easy. In the case of allocator, it's quite possible we could start including parameters to functions that are allocators in std, since the behavior of the allocator is implementation, not API. In other words, a function that takes an allocator may not have to be in std.experimental, as long as you know that the parameter is an allocator and the implementation knows how to properly use it (and existing code compiles). -Steve
Dec 31 2015
parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Thursday, 31 December 2015 at 18:26:08 UTC, Steven 
Schveighoffer wrote:
 I'm not concerned. Code can live in experimental as long as 
 it's needed to flesh out the API. The idea behind 
 std.experimental is to give us a disclaimer that the API may 
 change at any release despite usage.

 So the timeline is "when it's ready"
I concerns me because without deadlines things get endlessly delayed. Also, keeping things in experimental for too long encourages a overly cautious mentality IMO.
 In the case of allocator, it's quite possible we could start 
 including parameters to functions that are allocators in std, 
 since the behavior of the allocator is implementation, not API. 
 In other words, a function that takes an allocator may not have 
 to be in std.experimental, as long as you know that the 
 parameter is an allocator and the implementation knows how to 
 properly use it (and existing code compiles).
I was under the impression that any Phobos code importing from experimental was taboo.
Dec 31 2015
next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 12/31/15 1:54 PM, Jack Stouffer wrote:
 On Thursday, 31 December 2015 at 18:26:08 UTC, Steven Schveighoffer wrote:
 I'm not concerned. Code can live in experimental as long as it's
 needed to flesh out the API. The idea behind std.experimental is to
 give us a disclaimer that the API may change at any release despite
 usage.

 So the timeline is "when it's ready"
I concerns me because without deadlines things get endlessly delayed. Also, keeping things in experimental for too long encourages a overly cautious mentality IMO.
Yes, but the alternative is to have a bad API we must maintain. For example, object.destroy used to be called object.clear. It's just now coming out of the requirement to be supported as clear, and I can finally add a method to AAs that clear all elements (I implemented it last year, but put on hold because it was determined that we should wait until we could use the "obvious" name). The idea of having it in std.experimental instead of just on its own repository is so that it goes through the same rigors of testing that all of phobos does, and so that it gets used more. For all intents and purposes, it's "official", just that the API isn't completely baked. I'd say that if people are using it for a good period of time and nobody has any serious objections to it, then we can consider moving it to std.
 In the case of allocator, it's quite possible we could start including
 parameters to functions that are allocators in std, since the behavior
 of the allocator is implementation, not API. In other words, a
 function that takes an allocator may not have to be in
 std.experimental, as long as you know that the parameter is an
 allocator and the implementation knows how to properly use it (and
 existing code compiles).
I was under the impression that any Phobos code importing from experimental was taboo.
I wouldn't say so. If it's an implementation detail, so what? The key is to avoid breaking changes with user code, not breaking changes between std.experimental and std. But I'm not the end-all rule maker. Perhaps there is a rule that it's taboo. -Steve
Dec 31 2015
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, 31 December 2015 at 18:54:05 UTC, Jack Stouffer 
wrote:
 On Thursday, 31 December 2015 at 18:26:08 UTC, Steven 
 Schveighoffer wrote:
 I'm not concerned. Code can live in experimental as long as 
 it's needed to flesh out the API. The idea behind 
 std.experimental is to give us a disclaimer that the API may 
 change at any release despite usage.

 So the timeline is "when it's ready"
I concerns me because without deadlines things get endlessly delayed. Also, keeping things in experimental for too long encourages a overly cautious mentality IMO.
We have no actual policy on the matter, and until we do, I expect that anything that's in std.experimental will indeed sit there indefinitely. Putting stuff in std.experimental to begin with in order to let it be banged on a bit by the community at large before being finalized in std makes sense, but at what point is it actually ready to be put into std? How do we decide that? And do we need to vote again, or have any kind of further review in the newsgroup? Or do we just wait and see if we get bug reports on it and otherwise leave it as-is until we move it over into std due to age or because some of the Phobos devs feel like it? AFAIK, there's no consensus whatsoever on any of that. AFAIK, we don't really even have any proposals on that. We just sort of decided that having stuff put into experimental first would be a good idea so that we can minimize breaking stuff in std or be forced to leave it as-is permanently when a problem is found in the API within a release or two of it being put in Phobos. Probably, we either need someone to propose a process so that we can discuss it and possibly adopt it (either by agreeing to it as a group or by Walter and Andrei approving it). Certainly, until someone pushes the issue, I expect that nothing is going to escape std.experimental.
 In the case of allocator, it's quite possible we could start 
 including parameters to functions that are allocators in std, 
 since the behavior of the allocator is implementation, not 
 API. In other words, a function that takes an allocator may 
 not have to be in std.experimental, as long as you know that 
 the parameter is an allocator and the implementation knows how 
 to properly use it (and existing code compiles).
I was under the impression that any Phobos code importing from experimental was taboo.
I confess that I was assuming that that was the case, but I'm not sure that it's ever been discussed. If there's any risk of the code from std.experimental being thrown away from Phobos rather than moved into std, or if using the stuff from std.experimental would affect the API in std, then it would probably be a bad idea to use it in std. But if it's definitely going to stay in one form or another, and changes to it aren't going to affect the API in std, then maybe it would be okay. I don't know. I was assuming that std.experimental was an island unto itself, and it hadn't occurred to me that anyone would be using it in std before it go into std itself, but that doesn't mean that that's necessarily how it should be. - Jonathan M Davis
Jan 01
parent Jack Stouffer <jack jackstouffer.com> writes:
On Friday, 1 January 2016 at 17:26:17 UTC, Jonathan M Davis wrote:
 On Thursday, 31 December 2015 at 18:54:05 UTC, Jack Stouffer 
 wrote:
 I concerns me because without deadlines things get endlessly 
 delayed. Also, keeping things in experimental for too long 
 encourages a overly cautious mentality IMO.
We have no actual policy on the matter, and until we do, I expect that anything that's in std.experimental will indeed sit there indefinitely. Putting stuff in std.experimental to begin with in order to let it be banged on a bit by the community at large before being finalized in std makes sense, but at what point is it actually ready to be put into std? How do we decide that? And do we need to vote again, or have any kind of further review in the newsgroup? Or do we just wait and see if we get bug reports on it and otherwise leave it as-is until we move it over into std due to age or because some of the Phobos devs feel like it? AFAIK, there's no consensus whatsoever on any of that. AFAIK, we don't really even have any proposals on that. We just sort of decided that having stuff put into experimental first would be a good idea so that we can minimize breaking stuff in std or be forced to leave it as-is permanently when a problem is found in the API within a release or two of it being put in Phobos. Probably, we either need someone to propose a process so that we can discuss it and possibly adopt it (either by agreeing to it as a group or by Walter and Andrei approving it). Certainly, until someone pushes the issue, I expect that nothing is going to escape std.experimental.
I propose this simple policy: Three major releases after the module is included, the author has the right to ask that it be included in std by opening a new PR on Phobos. Voting at this point would be redundant, so if no Phobos contributors have any objections, based on open bugs or concerns in the API, then it MUST be merged. Simple and effective.
Jan 01
prev sibling parent reply Dicebot <public dicebot.lv> writes:
This topic is a bit complicated because idea of introducing 
std.experimental came from Andrei but without any specifics. At 
that time I was doing most of Phobos review queue bookkeeping and 
I had to propose some formal process to respect Andrei desire (I 
personally was against the concept).

Thus the original formal process was looking like this in my head:
     - initial voting only decides if proposed module is needed in 
Phobos at all and if author looks competent enough to get it 
finished
     - any time release beta starts std.experimental module author 
can ask review manager to start voting for final inclusion
     - this voting becomes true quality and stability control 
point - reviewers/voters have to decide if API and implementation 
are good enough to get frozen
     - if voting fails, proposal is kept in std.experimental until 
all issues are addressed and process repeats again

Idea of any "automatic" inclusion is very harmful because by 
current standard std.experimental modules are not required to 
have Phobos quality and can be very far from being good enough.

What makes things most complicated is that I don't do review 
bookkeeping anymore and no one else (including Andrei himself) 
have been willing to take control of the process since than.
Jan 01
parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Friday, 1 January 2016 at 20:10:22 UTC, Dicebot wrote:
 Thus the original formal process was looking like this in my 
 head:
     - initial voting only decides if proposed module is needed 
 in Phobos at all and if author looks competent enough to get it 
 finished
     - any time release beta starts std.experimental module 
 author can ask review manager to start voting for final 
 inclusion
     - this voting becomes true quality and stability control 
 point - reviewers/voters have to decide if API and 
 implementation are good enough to get frozen
     - if voting fails, proposal is kept in std.experimental 
 until all issues are addressed and process repeats again
Can this be adopted formally?
 Idea of any "automatic" inclusion is very harmful because by 
 current standard std.experimental modules are not required to 
 have Phobos quality and can be very far from being good enough.
My proposal was based on the quality of std.logger and std.allocator when they were merged. I wasn't aware that modules that were unfinished could be merged into std.experimental.
 What makes things most complicated is that I don't do review 
 bookkeeping anymore and no one else (including Andrei himself) 
 have been willing to take control of the process since than.
This can be supplemented by people asking for review managers like they do for initial inclusion into std.experimental, no?
Jan 01
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, 2 January 2016 at 04:27:02 UTC, Jack Stouffer wrote:
 On Friday, 1 January 2016 at 20:10:22 UTC, Dicebot wrote:
 Idea of any "automatic" inclusion is very harmful because by 
 current standard std.experimental modules are not required to 
 have Phobos quality and can be very far from being good enough.
My proposal was based on the quality of std.logger and std.allocator when they were merged. I wasn't aware that modules that were unfinished could be merged into std.experimental.
IIRC, the review for std.experimental.logger specifically left some stuff hanging with the idea that it would be sorted out in experimental, but I could be remembering wrong. Regardless, a certain level of quality is required for something to get into std.experimental, because it has to pass the review and vote, but that review and vote is no longer done with the expectation that what's there is necessarily set in stone when it gets merged (otherwise, there's no point to std.experimental). It should be solid, but there is now the potential for redesign on some level before it gets into std, whereas before, it would have just gone into std, and we basically had to get it right up front. Having another review for something like std.experimental.logger before it gets merged into Phobos makes sense, because it would allow us to have an open discussion on anything that needs to be changed now that folks have been using it. And if it's ready to go (potentially decided by a vote), then we could merge it. But without that review, the only stuff that will have changed is stuff that gets reported in bugzilla or that a Phobos dev specifically decided to change. If we want to actually find out how well it's been working for folks, then we need a way to get feedback based on actual usage and discuss it. Another review doesn't necessarily have to be the way that we do that, but it's probably the best mechanism that we have. - Jonathan M Davis
Jan 01
parent Jack Stouffer <jack jackstouffer.com> writes:
On Saturday, 2 January 2016 at 04:47:55 UTC, Jonathan M Davis 
wrote:
 Having another review for something like 
 std.experimental.logger before it gets merged into Phobos makes 
 sense, because it would allow us to have an open discussion on 
 anything that needs to be changed now that folks have been 
 using it. And if it's ready to go (potentially decided by a 
 vote), then we could merge it. But without that review, the 
 only stuff that will have changed is stuff that gets reported 
 in bugzilla or that a Phobos dev specifically decided to 
 change. If we want to actually find out how well it's been 
 working for folks, then we need a way to get feedback based on 
 actual usage and discuss it. Another review doesn't necessarily 
 have to be the way that we do that, but it's probably the best 
 mechanism that we have.
Yeah, unfortunately there is very little communication with a large portion of end users besides comments on reddit and bugzilla reports. I don't count people who post in general because they are largely a self selected group of people who like D enough to participate in the forum, and so we all have a bias. After doing a search on GitHub for repos that were already using std.logger, I found a lot of them were people who post here. Not that good ideas for fixing things can't come out of this forum, but IMO there are just not enough people on here for some of the more obscure of the bad ideas to come to the surface that only show themselves after a lot of stress is put on something, plus the bias mentioned earlier. A good example of this problem is the issue that Sociomantic has with Phobos (I'm going off their 2013 and 2014 Dconf talks, so I apologize if some info is wrong). People were always complaining about Phobos use of the GC, so the Phobos' maintainers solution was to change things to stack allocation with lazy ranges and the like. But it turns out that that was completely missing the mark. Sociomantic uses a lot of fibers so their stacks are rather small, and so they still couldn't use Phobos. The problem wasn't necessarily the GC allocation, it was a lack of choice in allocation, which is a very different problem. Thankfully Andrei then wrote std.allocator which could solve the problem, but that just goes to show how an entire community of people can miss what the root cause of an issue is. Going back to what I said about a lot of the repos being from people who post here, I think this shows an unwillingness from the normal user to import from std. experimental. Paraphrasing someone else's comment on here (I forget who) "importing from std.experimental is asking for trouble because you're essentially importing a ticking time bomb". And really, I can't argue with this mentality. So this is another block on potential feedback from users about bad ideas. As it see it, there are three things that can be done to mitigate these issues: 1. Do the review for final inclusion, as mentioned before 2. Make the inclusion of modules into std.experimental more of an event I knew about PEP 484 way before it was accepted, because I am a professional Python programmer and I browse /r/Python and it was posted there. The review threads should have the same media blast as a D release. 3. Gratuitous use of the deprecation -> undocument -> remove cycle This is the more effective of the two options, because the first idea still has the bias and the lack of interest in experimental problems. Bad ideas are going to slip through the cracks. The best thing to do IMO is to get rid of them ASAP as to minimize the damage done. When very compelling evidence is given that an idea is bad and should be replaced with something else, as soon as the replacement is ready, the cycle should start. I believe this is the best way to mitigate the damage from lack of eyes on a problem.
Jan 02
prev sibling parent reply Dicebot <public dicebot.lv> writes:
On Saturday, 2 January 2016 at 04:27:02 UTC, Jack Stouffer wrote:
 On Friday, 1 January 2016 at 20:10:22 UTC, Dicebot wrote:
 Thus the original formal process was looking like this in my 
 head:
 ...
Can this be adopted formally?
Depends on if you want to volunteer to manage the process and communicate with all interested parties to make it reality - and keep maintaining process in future. Or convince someone else to do the same.
 Idea of any "automatic" inclusion is very harmful because by 
 current standard std.experimental modules are not required to 
 have Phobos quality and can be very far from being good enough.
My proposal was based on the quality of std.logger and std.allocator when they were merged. I wasn't aware that modules that were unfinished could be merged into std.experimental.
Haven't found any issues with std.allocator so far but std.logger definitely is not Phobos ready per my requirements. I have been recently re-evaluating it as possible replacement for old Tango logger we use and found that in several places it forces unnecessary GC allocations and/or immutability (i.e. https://github.com/D-Programming-Language/phobos/blob/master/std/experimenta /logger/core.d#L742 requires that log message must be formatted into immutable string before it can be passed to another logger). This is a deal breaker that can possibly require major API change to fix.
 What makes things most complicated is that I don't do review 
 bookkeeping anymore and no one else (including Andrei himself) 
 have been willing to take control of the process since than.
This can be supplemented by people asking for review managers like they do for initial inclusion into std.experimental, no?
As far as I understand, there is no semi-persistent review manager at all right now. Some people volunteered to handle specific voting threads but to make sure new formal process is accepted one needs much more than that - provide formal spec, ensure approval of language authors, keep poking all involved developers to actually follow the process etc. It doesn't take that much time but it must be dedicated long-term commitment to work.
Jan 03
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/3/16 4:37 PM, Dicebot wrote:
 Haven't found any issues with std.allocator so far but std.logger
 definitely is not Phobos ready per my requirements. I have been recently
 re-evaluating it as possible replacement for old Tango logger we use and
 found that in several places it forces unnecessary GC allocations and/or
 immutability (i.e.
 https://github.com/D-Programming-Language/phobos/blob/master/std/experimental/logger/core.d#L742
 requires that log message must be formatted into immutable string before
 it can be passed to another logger). This is a deal breaker that can
 possibly require major API change to fix.
I guess that's a good example of the benefits of std.experimental. Now you can file bug reports and the module could undergo significant changes before being moved to std. Have you filed requests yet? -- Andrei
Jan 03
parent Dicebot <public dicebot.lv> writes:
On Sunday, 3 January 2016 at 22:37:34 UTC, Andrei Alexandrescu 
wrote:
 I guess that's a good example of the benefits of 
 std.experimental. Now you can file bug reports and the module 
 could undergo significant changes before being moved to std. 
 Have you filed requests yet? -- Andrei
Not yet - I am hoping to come out with some suitable modification proposals before to make bug report less abstract. It didn't look like std.experimental modules are going to be stabilized any time soon thus the issue wasn't on my priority list either.
Jan 03
prev sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
On Sunday, 3 January 2016 at 21:37:28 UTC, Dicebot wrote:
 Haven't found any issues with std.allocator so far but 
 std.logger definitely is not Phobos ready per my requirements. 
 I have been recently re-evaluating it as possible replacement 
 for old Tango logger we use and found that in several places it 
 forces unnecessary GC allocations and/or immutability (i.e. 
 https://github.com/D-Programming-Language/phobos/blob/master/std/experimenta
/logger/core.d#L742 requires that log message must be formatted into immutable
string before it can be passed to another logger). This is a deal breaker that
can possibly require major API change to fix.
I do not follow. 1. Tango passes a const(char)[] around [1]. What is the difference? 2. Have you looked at the doc and the impl. of FileLogger? They both show an easy how to implement a Logger without the GC. With that it is also trivial to make a MultiLogger that does not allocate. BTW, the default implementation is a direct result of the "by default multi-threading safe" requirement brought up multiple times during reviews. [1] https://github.com/SiegeLord/Tango-D2/blob/d2port/tango/util/log/Log.d#L1297
Jan 04
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, 4 January 2016 at 09:24:44 UTC, Robert burner Schadek 
wrote:
 On Sunday, 3 January 2016 at 21:37:28 UTC, Dicebot wrote:
 Haven't found any issues with std.allocator so far but 
 std.logger definitely is not Phobos ready per my requirements. 
 I have been recently re-evaluating it as possible replacement 
 for old Tango logger we use and found that in several places 
 it forces unnecessary GC allocations and/or immutability (i.e. 
 https://github.com/D-Programming-Language/phobos/blob/master/std/experimenta
/logger/core.d#L742 requires that log message must be formatted into immutable
string before it can be passed to another logger). This is a deal breaker that
can possibly require major API change to fix.
I do not follow. 1. Tango passes a const(char)[] around [1]. What is the difference? 2. Have you looked at the doc and the impl. of FileLogger? They both show an easy how to implement a Logger without the GC. With that it is also trivial to make a MultiLogger that does not allocate. BTW, the default implementation is a direct result of the "by default multi-threading safe" requirement brought up multiple times during reviews. [1] https://github.com/SiegeLord/Tango-D2/blob/d2port/tango/util/log/Log.d#L1297
If I understand correctly (based on previous statements by Dicebot), the problem is that at Sociomantic, they reuse buffers heavily. So, they basically don't use string much of anywhere and instead use some combination of char[] and const(char)[] (so that the array elements can be given new values without reallocating), and std.experimental.logger - like a lot of typical D code - uses string, which means that when they log a message with std.experimental.logger, their buffer has to be copied into a string, which means that an allocation occurs, which in their environment is unacceptable. So, to work for them, they would need std.experimental.logger to accept something like const(char)[] or an arbitrary ranges of characters and to not allocate inside of any of its logging functions. Any requirement to convert to string (be it by the user of the logger or inside of the logger itself) doesn't work with their requirements. I'm not particularly familiar with std.experimental.logger as it stands, by I would guess that to fit Sociomantic's requirements, it would need to either manage to log without actually constructing any array of characters (e.g. by printing each of its elements in turn without actually putting them together first), or it would need to reuse a buffer every time it created the line to log (which would incur the cost of copying the characters but wouldn't normally have to allocate). But I don't know how well that fits in with the logger being hierarchical via classes (which solves other design requirements) or how close the current implementation is to that. Certainly, being forced to use classes does prevent the logger from using some of the typical techniques that we use (particularly with regards to ranges). So, the logger definitely presents some challenges that most other Phobos code doesn't have to deal with. :| Though obviously, you'd be more aware of that than anyone. :) - Jonathan M Davis
Jan 04
next sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
On Monday, 4 January 2016 at 10:58:09 UTC, Jonathan M Davis wrote:
 If I understand correctly (based on previous statements by 
 Dicebot), the problem is that at Sociomantic, they reuse 
 buffers heavily. So, they basically don't use string much of 
 anywhere and instead use some combination of char[] and 
 const(char)[] (so that the array elements can be given new 
 values without reallocating), and std.experimental.logger - 
 like a lot of typical D code - uses string, which means that 
 when they log a message with std.experimental.logger, their 
 buffer has to be copied into a string, which means that an 
 allocation occurs, which in their environment is unacceptable.
FileLogger is a Logger that can be used after only one allocation. new FileLogger; Everything else (logging, ...) does not require a single allocation. If you look at the documentation above [1] you will understand why. Building a MultiLogger (Logger of Loggers) that forwards the calls to the log functions to the contained Loggers without allocation is also trivial. Logging into a passed in buffer is also trivial (as long as you buffer is big enough for the data to log, but that is design problem).
 So, to work for them, they would need std.experimental.logger 
 to accept something like const(char)[] or an arbitrary ranges 
 of characters and to not allocate inside of any of its logging 
 functions. Any requirement to convert to string (be it by the 
 user of the logger or inside of the logger itself) doesn't work 
 with their requirements.
as said above, no Logger has to allocate for logging, and no string has to be created. An allocation schema was selected however, to make multi threading work by default. Creating a user specific Logger that fits the Logger interface and that uses a static buffer to write the data to is properly less than 20 lines of code.
 I'm not particularly familiar with std.experimental.logger as 
 it stands, by I would guess that to fit Sociomantic's 
 requirements, it would need to either manage to log without 
 actually constructing any array of characters (e.g. by printing 
 each of its elements in turn without actually putting them 
 together first), or it would need to reuse a buffer every time 
 it created the line to log (which would incur the cost of 
 copying the characters but wouldn't normally have to allocate). 
 But I don't know how well that fits in with the logger being 
 hierarchical via classes (which solves other design 
 requirements) or how close the current implementation is to 
 that. Certainly, being forced to use classes does prevent the 
 logger from using some of the typical techniques that we use 
 (particularly with regards to ranges). So, the logger 
 definitely presents some challenges that most other Phobos code 
 doesn't have to deal with. :| Though obviously, you'd be more 
 aware of that than anyone. :)

 - Jonathan M Davis
If you guess of their requirements is correct, std.experimental.logger fulfills the requirements. [1] https://github.com/D-Programming-language/phobos/blob/master/std/experimental/logger/core.d#L812
Jan 04
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, 4 January 2016 at 13:48:16 UTC, Robert burner Schadek 
wrote:
 On Monday, 4 January 2016 at 10:58:09 UTC, Jonathan M Davis 
 wrote:
 So, to work for them, they would need std.experimental.logger 
 to accept something like const(char)[] or an arbitrary ranges 
 of characters and to not allocate inside of any of its logging 
 functions. Any requirement to convert to string (be it by the 
 user of the logger or inside of the logger itself) doesn't 
 work with their requirements.
as said above, no Logger has to allocate for logging, and no string has to be created. An allocation schema was selected however, to make multi threading work by default. Creating a user specific Logger that fits the Logger interface and that uses a static buffer to write the data to is properly less than 20 lines of code.
Well, looking at your implementation in your other post, you're able to make it work by casting to string when it's not actually a string. So, you're circumventing the type system to make it work. Depending on what the base class does with the resulting string, it may work, but there's no guarantee that it will, especially if you end up deriving your logger class from a 3rd party one. For this code to be safe, msg in LogEntry really needs to be const(char)[] and not string.
 I'm not particularly familiar with std.experimental.logger as 
 it stands, by I would guess that to fit Sociomantic's 
 requirements, it would need to either manage to log without 
 actually constructing any array of characters (e.g. by 
 printing each of its elements in turn without actually putting 
 them together first), or it would need to reuse a buffer every 
 time it created the line to log (which would incur the cost of 
 copying the characters but wouldn't normally have to 
 allocate). But I don't know how well that fits in with the 
 logger being hierarchical via classes (which solves other 
 design requirements) or how close the current implementation 
 is to that. Certainly, being forced to use classes does 
 prevent the logger from using some of the typical techniques 
 that we use (particularly with regards to ranges). So, the 
 logger definitely presents some challenges that most other 
 Phobos code doesn't have to deal with. :| Though obviously, 
 you'd be more aware of that than anyone. :)

 - Jonathan M Davis
If you guess of their requirements is correct, std.experimental.logger fulfills the requirements.
Only by doing a risky cast that happens to work in a reduced example but does not work in the general case. Now, the changes required to avoid the cast should be minimal and straightforward, but as long as a cast to string is required in order to avoid allocating a string, the logger doesn't safely support avoiding allocations. My guess is that it didn't even occur to Dicebot to make the cast, but I doubt that he would be happy with that solution anyway, particularly given the recent PR and related discussion on adding message to Throwable (which returns const(char)[]): https://github.com/D-Programming-Language/druntime/pull/1445 Regardless, Dicebot will obviously have to talk with you about their requirements and why he thinks that std.logger doesn't fit them. - Jonathan M Davis
Jan 04
parent reply Robert burner Schadek <rburners gmail.com> writes:
Yes the cast is ugly, but then I'm writing dynamic content into 
stack arrays.

I actually have a patch in the pipeline for this,. That patch 
require to change the protection for beginLogMsg, logMsgPart, 
finishLogMsg from protected to public and bye bye cast. 
logMsgPart already takes a const(char)[]. I actually thought the 
protection was public to begin with.

Anyway, if people have issues with std.(experimental.)logger, 
please write to the forum, bugzilla, github, or directly to me.
Jan 04
parent reply Dicebot <public dicebot.lv> writes:
Sorry for late reponse, I don't read NG that often now.

On Monday, 4 January 2016 at 17:06:16 UTC, Robert burner Schadek 
wrote:
 I actually have a patch in the pipeline for this,. That patch 
 require to change the protection for beginLogMsg, logMsgPart, 
 finishLogMsg from protected to public and bye bye cast. 
 logMsgPart already takes a const(char)[]. I actually thought 
 the protection was public to begin with.
Very good, this will get rid of one of API problems I was going to mention :)
 Yes the cast is ugly, but then I'm writing dynamic content into 
 stack arrays.
It is not just ugly, it is undefined behavior by D specification. You are casting to immutable data that will inevitably going to change. Most likely it will work on all existing compilers but such code won't pass any decent code review. With protection changes it becomes possible to avoid LogEntry though so this becomes sort of fixed. However ..
 BTW, the default implementation is a direct result of the "by 
 default multi-threading safe" requirement brought up multiple 
 times during reviews.
.. this remains a concern. I know that with some tweaks and lot of custom overriding I can build logging system that works with reusable buffers. However, it would be incompatible with any 3d party logger which is defined in terms of `LogEntry` and that kills the main benefit of having standard API in the first place. I'd like (if it is even possible) to have a fast thread-safe no-allocation logger with thread-local proxies as a default one available because it is both very common demand from a logging system and will influence how derived 3d-party loggers will be designed. But that can be a more problematic task because `const(char)[]` is not implicitly shared and we don't have any good guidelines for sharing mutable data in Phobos :) Right now I have no idea what should be done - it is simply a problem that is likely to result in at least some API changes and a good example why just being in std.experimental for a long period doesn't safeguard against finding unexpected API issues (I have only realized the issue several weeks ago, despite being the review manager of original proposal and reading it in great details).
 Anyway, if people have issues with std.(experimental.)logger, 
 please write to the forum, bugzilla, github, or directly to me.
As I have mentioned in the e-mail, I will do so as soon as I will have any half-decent proposal and will be able to help :)
Jan 05
parent Robert burner Schadek <rburners gmail.com> writes:
On Tuesday, 5 January 2016 at 16:34:27 UTC, Dicebot wrote:
 BTW, the default implementation is a direct result of the "by 
 default multi-threading safe" requirement brought up multiple 
 times during reviews.
.. this remains a concern. I know that with some tweaks and lot of custom overriding I can build logging system that works with reusable buffers. However, it would be incompatible with any 3d party logger which is defined in terms of `LogEntry` and that kills the main benefit of having standard API in the first place. I'd like (if it is even possible) to have a fast thread-safe no-allocation logger with thread-local proxies as a default one available because it is both very common demand from a logging system and will influence how derived 3d-party loggers will be designed.
If you could write this up as a callgraph with some (ascii) art showing thread boundaries, you would help me a lot understanding your requirement.
 But that can be a more problematic task because `const(char)[]` 
 ....
 original proposal and reading it in great details).

 Anyway, if people have issues with std.(experimental.)logger, 
 please write to the forum, bugzilla, github, or directly to me.
As I have mentioned in the e-mail, I will do so as soon as I will have any half-decent proposal and will be able to help :)
You don't need to do my work, it is nice of you to do that, but all I need is a problem description I can work with.
Jan 05
prev sibling parent Robert burner Schadek <rburners gmail.com> writes:
As talk is cheap, here is the code: 
http://dpaste.dzfl.pl/ba4df05339f6

not 20 lines but for 99 lines you get two Loggers and some tests.
Jan 04