www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - What's up with the assert enhancements proposed years ago?

reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
Some ages ago, a whole suite of "assertPred" functions were written 
(giving better diagnostic info, like showing "expected vs actual"), were 
totally awesome, were submitted to phobos...and were rejected because it 
was deemed both easy enough and preferable to get these features by 
modifying DMD to add behind-the-scenes AST magic to "assert".

So...umm...yea...whatever happened to that beefed-up "assert" feature?
Sep 23 2016
next sibling parent reply Seb <seb wilzba.ch> writes:
On Friday, 23 September 2016 at 20:57:49 UTC, Nick Sabalausky 
wrote:
 Some ages ago, a whole suite of "assertPred" functions were 
 written (giving better diagnostic info, like showing "expected 
 vs actual"), were totally awesome, were submitted to 
 phobos...and were rejected because it was deemed both easy 
 enough and preferable to get these features by modifying DMD to 
 add behind-the-scenes AST magic to "assert".

 So...umm...yea...whatever happened to that beefed-up "assert" 
 feature?
http://wiki.dlang.org/DIP83 (needs polishing and submission to the new dip process)
Sep 23 2016
parent reply Jacob Carlborg <doob me.com> writes:
On 2016-09-24 00:02, Seb wrote:

 http://wiki.dlang.org/DIP83

 (needs polishing and submission to the new dip process)
http://wiki.dlang.org/DIP50 :) -- /Jacob Carlborg
Sep 24 2016
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 09/24/2016 06:52 AM, Jacob Carlborg wrote:
 On 2016-09-24 00:02, Seb wrote:

 http://wiki.dlang.org/DIP83

 (needs polishing and submission to the new dip process)
http://wiki.dlang.org/DIP50 :)
+1. Although I haven't given it too much thought, I'd bet that could also be used to provide, in library, my #1 most wanted missing D feature: Input ranges (maybe even forward, too) written as stackless coroutines (C#-style, heck, even C can do it in lib). So we could write input ranges without turning the whole freaking algorithm completely inside-out, or requiring the overhead of a full fiber. Ever try doing LL parsing both traditional procedural-style and range-style? The range version's a complete unholy mess.
Sep 24 2016
parent reply Jacob Carlborg <doob me.com> writes:
On 2016-09-24 15:12, Nick Sabalausky wrote:

 +1.

 Although I haven't given it too much thought, I'd bet that could also be
 used to provide, in library, my #1 most wanted missing D feature: Input
 ranges (maybe even forward, too) written as stackless coroutines
 (C#-style, heck, even C can do it in lib). So we could write input
 ranges without turning the whole freaking algorithm completely
 inside-out, or requiring the overhead of a full fiber.
Yeah, I've been thinking of that as well. It should be fairly easy to implement something like Protothreads [1]. Not sure about stackless coroutines though, if they need to save the state of the function on suspension and restore the state when the function resumes. [1] http://dunkels.com/adam/pt/ -- /Jacob Carlborg
Sep 24 2016
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 09/24/2016 10:20 AM, Jacob Carlborg wrote:
 On 2016-09-24 15:12, Nick Sabalausky wrote:

 +1.

 Although I haven't given it too much thought, I'd bet that could also be
 used to provide, in library, my #1 most wanted missing D feature: Input
 ranges (maybe even forward, too) written as stackless coroutines
 (C#-style, heck, even C can do it in lib). So we could write input
 ranges without turning the whole freaking algorithm completely
 inside-out, or requiring the overhead of a full fiber.
Yeah, I've been thinking of that as well. It should be fairly easy to implement something like Protothreads [1]. Not sure about stackless coroutines though, if they need to save the state of the function on suspension and restore the state when the function resumes. [1] http://dunkels.com/adam/pt/
"Protothreads"! I've been trying to remember the name of that lib and couldn't find it. I used that the last time I was working on a C/C++ thing, and it was very helpful. AIUI though, that IS stackless coroutines, same thing AFAIK. The Protothreads lib refers to itself as "extremely lightweight stackless threads". And the way it's used is definitely what I'd consider a coroutine. Protothreads solves the problem of saving/restoring state by doing two things: 1. You put all your locals (at least, any that need preserved across save/restore) in a struct, together with the appropriate jump label to be jumped to upon resume. Then when yielding, this struct gets passed out, and when resuming you pass it back in. 2. Yielding can ONLY be done by a coroutine, itself, NOT by any functions it calls (unless those are also coroutines and return some flag signaling the first coroutine to also yield). This is the one big key that makes stackless fibers/coroutines/etc possible in the first place. My understanding is that C#'s coroutines, under the hood, work the same way as C's Protothreads library: First, only C#'s coroutines can yield, not regular functions called by a coroutine. Then, the C# compiler "lowers" the coroutine function into a big switch statement, with the current state (values of the locals and the hidden "case" target to jump to) passed in and out as a hidden argument. Since the compiler does the conversion, it can safely deal with saving/restoring the locals, and crossing scope boundaries and such (which C is just plain not concerned with at all). So really, C#'s coroutines (AIUI) are just like C's Protothreads, but with nicer sugar. But about implementing it in D: I'm not so sure that's realistic right now. I mean, I think something like it is *technically* possible, and I would have done so myself long ago, except that I see two major problems: 1. You'd have to write the entire coroutine as a string, which would really mess with IDE/editor features and the compiler's error reporting. And I'm not sure I see any way to do this in D without writing the coroutine as a string (or, you know, using AST macros ;) ). 2. Compared to C, D has much stricter rules about where a switch's "case" labels can do. Ie, they can't really cross scope boundaries. So *all* the coroutine's "yield" statements would have to be within the same scope *and* at the same nesting-level (or at the very least, there'd be some big annoying restrictions). Otherwise, when it gets converted to a "case:", the resulting code would be invalid. We could use "goto" instead of case, but that has many of the same issues, plus (IIRC) the issue of goto-ing past a variable declaration (which IIRC is disallowed, or maybe had some other problem). I'm not sure how you could get around problem #1 without AST macros or just implementing the whole feature in the compiler, like C#. Even if you solved that, the only way I see to get around #2 would require very careful manual adjustments to the coroutine via a CTFE D AST, which (for right now) would be all kinds of bother, and at that point you may as well just add the feature built-into the compiler (like C#) because it would be more efficient than CTFE, AND it would solve problem #1.
Sep 24 2016
next sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Sat, 24 Sep 2016 11:49:24 -0400, Nick Sabalausky wrote:
 My understanding is that C#'s coroutines, under the hood, work the same
 way as C's Protothreads library:
You mean async/await, which is a compiler-provided CPS transformation on top of the Task API. There was a coroutine library for Mono, based on Mono.Tasklets, but I haven't been able to get Mono.Tasklets to work without segfaulting the last couple times I tried.
 First, only C#'s coroutines can yield, not regular functions called by a
 coroutine.
Because it's the continuation passing style, not a full thread of execution. That makes it moderately painful to use under normal circumstances and entirely unsuitable in other cases. For instance, I've got a project that involves simulating a large number of actors (several thousand at a minimum) within a world. If I used CPS, it would be a bit painful. I'd have to keep remembering which functions are asynchronous and which are synchronous. If I needed to turn a function asynchronous, it changes how I call it, and that effect can ripple up. And that's a maintenance problem, but at least I have a type system to help out. I also need a scripting system, which will no doubt be weakly typed (ECMAScript, perhaps), and coordinating those changes there would be a much higher cost. I also want to support people writing scripts who are not cautious about writing those scripts, who don't appreciate the technical innards that I implemented, and who will write scripts that occasionally misbehave. I want to make a pit of success for them and hopefully ensure that their scripts won't typically misbehave too badly. So the CPS transformation model is a nonstarter for me, manual or partially automated.
 But about implementing it in D:
 
 I'm not so sure that's realistic right now.
Look at Node.js and Vert.x. It's not a barrier to the market to force programmers to manually execute the CPS transformation. Fibers are more convenient. They also have a large, upfront cost, but they have the advantage of treating the stack like a stack. It breaks fewer assumptions about performance. Go's multitasking is a middle ground -- it has stacks that can shrink and grow, but the cost is a check for the stack bottom at every function call.
Sep 24 2016
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 09/24/2016 09:14 PM, Chris Wright wrote:
 On Sat, 24 Sep 2016 11:49:24 -0400, Nick Sabalausky wrote:
 My understanding is that C#'s coroutines, under the hood, work the same
 way as C's Protothreads library:
You mean async/await, which is a compiler-provided CPS transformation on top of the Task API.
No, I mean "yield". My C# work was all before async/await.
 First, only C#'s coroutines can yield, not regular functions called by a
 coroutine.
Because it's the continuation passing style, not a full thread of execution. That makes it moderately painful to use under normal circumstances and entirely unsuitable in other cases.
If I understand correctly what you mean by "continuation passing style", then yes, it sort of is, but only behind-the-scenes. The only recognizable effect of that is that yield can only break out of one level of the call stack at a time (just like how return only ever returns from the current function, no further - unless you use Ruby, but well, let's not go there...). The "turning functions inside-out" effect of continuation passing is exactly what input ranges suffer from[1] - and is exactly what stackless coroutines *avoid*.
 Look at Node.js and Vert.x. It's not a barrier to the market to force
 programmers to manually execute the CPS transformation.
Speaking as a long-time web developer, most (not all) web developers are fully-acclimated to eating shit sandwiches and convincing themselves they like it. Hence their tolerance for what Node.js does to their code (among other issues with it, like...using JS...on the server...). Obviously I'm being colorfully figurative there, but I'm sure you knew that ;)
 Fibers are more convenient. They also have a large, upfront cost, but
 they have the advantage of treating the stack like a stack. It breaks
 fewer assumptions about performance.
You may want to give C's protobuf or C#'s yield-based generator functions a try. Yes, true, they're not as convenient as actual fibers, but there's a LOT they let you do painlessly (far more painlessly than writing input ranges[1]) without the fiber overhead - think opApply but without the complete and total incompatibility with functions that operate on ranges. [1] Note I mean actually writing an input range itself. *Using* an input range is wonderfully pleasant. But writing them turns all my logic inside out, somewhat like Node.js-style callback hell (or ActionScript2 back in my day).
Sep 24 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Sun, 25 Sep 2016 01:36:57 -0400, Nick Sabalausky wrote:

 On 09/24/2016 09:14 PM, Chris Wright wrote:
 On Sat, 24 Sep 2016 11:49:24 -0400, Nick Sabalausky wrote:
 My understanding is that C#'s coroutines, under the hood, work the
 same way as C's Protothreads library:
You mean async/await, which is a compiler-provided CPS transformation on top of the Task API.
No, I mean "yield". My C# work was all before async/await.
Ah, I'd heard about people doing that. That's a different transformation. Method to method object, add a state variable to mimic the program counter, and munge things about enough to match the IEnumerator interface. Surprisingly similar despite the differences in implementation, though.
 The "turning functions inside-out" effect of continuation passing is
 exactly what input ranges suffer from[1] - and is exactly what stackless
 coroutines *avoid*.
The compiler inverts them for you.
 Fibers are more convenient. They also have a large, upfront cost, but
 they have the advantage of treating the stack like a stack. It breaks
 fewer assumptions about performance.
You may want to give C's protobuf or C#'s yield-based generator functions a try. Yes, true, they're not as convenient as actual fibers, but there's a LOT they let you do painlessly (far more painlessly than writing input ranges[1]) without the fiber overhead - think opApply but without the complete and total incompatibility with functions that operate on ranges.
Yeah, I really want something transparent.
 [1] Note I mean actually writing an input range itself. *Using* an input
 range is wonderfully pleasant. But writing them turns all my logic
 inside out, somewhat like Node.js-style callback hell (or ActionScript2
 back in my day).
Agreed. It's relatively easy to compose ranges, which is nice, but whenever I have to write one from scratch, I get a little sad.
Sep 25 2016
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 09/25/2016 01:06 PM, Chris Wright wrote:
 On Sun, 25 Sep 2016 01:36:57 -0400, Nick Sabalausky wrote:
 The "turning functions inside-out" effect of continuation passing is
 exactly what input ranges suffer from[1] - and is exactly what stackless
 coroutines *avoid*.
The compiler inverts them for you.
Basically, yea. Soooooo much nicer than writing/maintaining inverted code by hand!
 You may want to give C's protobuf or C#'s yield-based generator
 functions a try. Yes, true, they're not as convenient as actual fibers,
 but there's a LOT they let you do painlessly (far more painlessly than
 writing input ranges[1]) without the fiber overhead - think opApply but
 without the complete and total incompatibility with functions that
 operate on ranges.
Yeah, I really want something transparent.
Ah. Yea, totally transparent they are not. But I find them to be a solid enough abstraction that I have no issue whatsoever. Keep in mind though, fibers and threads aren't really all that transparent either, there's a bunch of behind-the-scenes magic there too. Saving/restoring register state. Your threads will just *stop* at arbitrary points while some other code halfway across the codebase is executed. Etc. But they're solid enough as abstractions that we prefer the abstraction to dealing with the underlying mess transparently. The same applies to nearly all programming constructs, even something as basic as function calls. Bunch of opaque hidden magic there. But it pretty much *just works* so we rarely need/want the transparency.
 [1] Note I mean actually writing an input range itself. *Using* an input
 range is wonderfully pleasant. But writing them turns all my logic
 inside out, somewhat like Node.js-style callback hell (or ActionScript2
 back in my day).
Agreed. It's relatively easy to compose ranges, which is nice, but whenever I have to write one from scratch, I get a little sad.
That's why some solid abstraction magic would be very, very nice! :)
Sep 25 2016
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-09-24 17:49, Nick Sabalausky wrote:

 "Protothreads"! I've been trying to remember the name of that lib and
 couldn't find it. I used that the last time I was working on a C/C++
 thing, and it was very helpful.

 AIUI though, that IS stackless coroutines, same thing AFAIK. The
 Protothreads lib refers to itself as "extremely lightweight stackless
 threads". And the way it's used is definitely what I'd consider a
 coroutine.
I find it quite difficult to find correct definitions with many of these terms like: coroutine, fiber, stackless thread, resumeable function. Because many languages use the same name of different things and they use different names for the same things.
 Protothreads solves the problem of saving/restoring state by doing two
 things:

 1. You put all your locals (at least, any that need preserved across
 save/restore) in a struct, together with the appropriate jump label to
 be jumped to upon resume. Then when yielding, this struct gets passed
 out, and when resuming you pass it back in.
Ah, right. Previously when I looked at this I only saw examples which stored the jump label. That might be possible to automatically rewrite in a AST macro. Would it be possible to store the state in a TLS variable inside the function to avoid having to pass in the state?
 2. Compared to C, D has much stricter rules about where a switch's
 "case" labels can do. Ie, they can't really cross scope boundaries. So
 *all* the coroutine's "yield" statements would have to be within the
 same scope *and* at the same nesting-level (or at the very least,
 there'd be some big annoying restrictions). Otherwise, when it gets
 converted to a "case:", the resulting code would be invalid.
Do you have any examples? -- /Jacob Carlborg
Sep 25 2016
next sibling parent Chris Wright <dhasenan gmail.com> writes:
On Sun, 25 Sep 2016 12:39:21 +0200, Jacob Carlborg wrote:
 On 2016-09-24 17:49, Nick Sabalausky wrote:
 2. Compared to C, D has much stricter rules about where a switch's
 "case" labels can do. Ie, they can't really cross scope boundaries. So
 *all* the coroutine's "yield" statements would have to be within the
 same scope *and* at the same nesting-level (or at the very least,
 there'd be some big annoying restrictions). Otherwise, when it gets
 converted to a "case:", the resulting code would be invalid.
Do you have any examples?
Duff's device works, so case labels can cross scope boundaries.
Sep 25 2016
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 09/25/2016 06:39 AM, Jacob Carlborg wrote:
 I find it quite difficult to find correct definitions with many of these
 terms like: coroutine, fiber, stackless thread, resumeable function.
 Because many languages use the same name of different things and they
 use different names for the same things.
FWIW, and this is at least the way I see it: (BTW, any domain experts please chime in if I'm wrong. I'm thinking it may be worth reposting this as a blog entry, as long as I'm not too far off-base.) Fiber: ------ Like a thread, but lighter-weight. Multiple fibers can run within one thread. Like a thread, a fiber has it's own stack, and upon suspend/resume various state is saved/restored (I assume that "state" means "CPU registers"). Unlike a thread, a fiber's multitasking is sequential and cooperative, NOT simultaneous and preemptive: Ie, One fiber must willfully relinquish control (by calling a "yield" function) before another fiber in the same thread can resume. I'm assuming this sequential/cooperative what allows it to be lighter-weight than a true thread, but I'm not 100% certain. Coroutine: ---------- A function that can "yield" (ie, it can choose to temporarily exit so that it can be resumed from the same place later on). Often a return value is emitted upon yielding, so these are usually used as generators, because they're a convenient way to generate a series of values in an easy procedural style. Coroutines are commonly implemented as fibers, so it is often said that fibers and coroutines are the same thing. That's partially accurate, except for two things: 1. "Coroutine" refers to a specific function, whereas "fiber" refers to the overall sequence of execution. 2. This is probably a debatable matter of semantics, but coroutines don't have to be implemented using fibers. For example, opApply is essentially a coroutine (it yields by calling the delegate that was passed in). Stackless Thread/Fiber: ----------------------- Not a true thread or fiber. It's called a "stackless thread" or "stackless fiber" simply because it's *like* a thread or fiber (closer to a fiber, really), but without a separate stack for each, umm..."fiber". Protothreads is an example of a stackless thread/fiber. A stackless thread/fiber is much more lightweight than even a fiber. But it comes with a limitation: With a true fiber, any function in the callstack can yield. When it yields, it suspends the fiber's entire callstack. So inside a fiber, your coroutine can call a function and have *that* function yield your fiber. That feature requires a separate callstack for each fiber, so you cannot do that with the stackless versions. Instead, in a stackless thread/fiber, yielding will ONLY suspend the current function, not a full callstack (since it doesnt have a callstack of its own to suspend). In other words, yielding a stackless thread/fiber works much like a normal "return" statement: ------------------- void foo() { bar(); } void bar() { return; // ONLY return from bar return from BOTH bar AND foo; // IMPOSSIBLE!!! } ------------------- A stackless thread/fiber works the same (psuedocode): ------------------- void coroutineFoo() { coroutineBar(); } void coroutineBar() { // Stackless fiber: ONLY suspends coroutineBar. // Regular fiber: Suspends ENTIRE callstack. yield; yield from BOTH bar AND foo;// IMPOSSIBLE with stackless // More code here.... } ------------------- Luckily, you can work around it, just like you can with normal returning (psuedocode): ------------------- void coroutineFoo() { bool shouldYield = coroutineBar(); if(shouldYield) yield; // More code here.... } // Yielded value: Should the caller yield? bool coroutineBar() { yield true; // Yield both bar and foo // More code here.... } ------------------- I'd argue opApply's approach technically qualifies as a stackless thread/fiber, since it yields (by calling the delegate it received) and does not involve any extra callstack. Plus, like any other stackless thread/fiber, it cannot suspend any further up the callstack than just itself. However, from what I've seen, a stackless thread/fiber is typically implemented the way Protothreads works: When the "stackless coroutine" function is called, the first thing done inside the function is jump to wherever the function last left off. Then, every "yield" is constructed as an *actual* "return" followed by a jump label. When a function yields, it returns three things: 1. An optional value being yielded (like a normal return value). 2. Which jump label to jump to next time the function is resumed. 3. Any local variables that need to be preserved upon resume (or just all local variables). With a really nice stackless thread/fiber system, like C#'s, this is all handled behind-the-scenes. Stackless Coroutine: -------------------- A coroutine that uses a stackless thread/fiber instead of a normal fiber. Resumable Function: ------------------- This sounds like a vague general term to me. I would think it casually refers to any function that can be suspended/resumed: A Fiber-based coroutine, a stackless coroutine, or whatever else may or may not exist.
 Ah, right. Previously when I looked at this I only saw examples which
 stored the jump label. That might be possible to automatically rewrite
 in a AST macro.

 Would it be possible to store the state in a TLS variable inside the
 function to avoid having to pass in the state?
I don't see why not, but there would be one notable drawback: You could only have one instance existing at a time (per thread). Ex: If you pass the state through params and return values instead of a static function variable, you can do this: ---------------------- // A manual stackless coroutine in D // Would be much simpler with built-in support struct YieldInfo { State state; int yieldedValue; } struct State { int jumpLabel = 1; // Any additional locals here } // Generate three multiples of 'multiplier', repeating forever YieldInfo generateMultiples( State state, int multiplier ) { switch(state.jumpLabel) { case 1: // Yield first multiple return YieldInfo(State(2), 1 * multiplier); case 2: // Yield second multiple return YieldInfo(State(3), 2 * multiplier); case 3: // Yield third multiple return YieldInfo(State(1), 3 * multiplier); } } void main() { // One instance of generateMultiples YieldInfo a; a = generateMultiples(a, 2); assert(a.yieldedValue == 2); a = generateMultiples(a, 2); assert(a.yieldedValue == 4); a = generateMultiples(a, 2); assert(a.yieldedValue == 6); a = generateMultiples(a, 2); assert(a.yieldedValue == 2); // Do another instance at the same time: YieldInfo b; b = generateMultiples(b, 5); assert(b.yieldedValue == 5); a = generateMultiples(a, 2); assert(a.yieldedValue == 4); b = generateMultiples(b, 5); assert(b.yieldedValue == 10); a = generateMultiples(a, 2); assert(a.yieldedValue == 6); b = generateMultiples(b, 5); assert(b.yieldedValue == 15); } ---------------------- Actual stackless coroutine support would make that much simpler of course: ---------------------- CoroutineInputRange!int generateMultiples(int multiple) { while(true) { foreach(i; 1...4) yield i * multiple; } } void main() { auto a = generateMultiples(2); assert(a.front == 2); a.popfront(); assert(a.front == 4); a.popfront(); assert(a.front == 6); a.popfront(); assert(a.front == 2); auto b = generateMultiples(5); assert(b.front == 5); a.popfront(); assert(a.front == 4); b.popfront(); assert(b.front == 10); a.popfront(); assert(a.front == 6); b.popfront(); assert(b.front == 15); } ---------------------- Naturally, if the state was stored as the function's static locals, you woudn't be able to do that. Instance "b" would clobber instance "a".
 2. Compared to C, D has much stricter rules about where a switch's
 "case" labels can do. Ie, they can't really cross scope boundaries. So
 *all* the coroutine's "yield" statements would have to be within the
 same scope *and* at the same nesting-level (or at the very least,
 there'd be some big annoying restrictions). Otherwise, when it gets
 converted to a "case:", the resulting code would be invalid.
Do you have any examples?
I'd have to look it up and refresh my memory. I just remember D has more restrictions regarding jumping to labels than C has. Something like: ------------------ State myCoroutine(State s) { yield "hello"; if(foo) { yield " world"; } yield "!"; } ------------------ A Protothreads-like preprocessing approach that used switch/case would turn that into: ------------------ State myCoroutine(State s) { switch(s.jumpLabel){ case 0: // Start converted code: ////////////// //yield "hello"; -> return+case: return State(1, "hello"); case 1: if(int foo = getFoo()) { //yield " world"; -> return State(2, " world"); return+case: case 2: } //yield "!"; -> return+case: return State(3, "!"); case 3: // End converted code ////////////// break; } } ------------------ That "if" really messed things up. AIUI, I don't think D will allow things like that. C doesn't have so much of a problem with any of this, it famously just permits pretty much anything, which is why Protothreads works there.
Sep 25 2016
parent reply Jacob Carlborg <doob me.com> writes:
On 2016-09-25 20:16, Nick Sabalausky wrote:

 Resumable Function:
 -------------------
 This sounds like a vague general term to me. I would think it casually
 refers to any function that can be suspended/resumed: A Fiber-based
 coroutine, a stackless coroutine, or whatever else may or may not exist.
I think this is the term used in now in C++. It looks like it will have a lower level API for suspending and resuming functions. Then it's possible to build on top of this different concurrency constructs as libraries, like coroutines, stackless threads, goroutines, async/await and so on.
 I don't see why not, but there would be one notable drawback: You could
 only have one instance existing at a time (per thread).
But how does it work in languages which don't pass in the state explicitly, like with "yield" in C#? Or async/await? I guess one doesn't call the function multiple times in the same way instead it's done automatically under the hood, passing in the state which might be stored in the "Task" that is returned?
 I'd have to look it up and refresh my memory. I just remember D has more
 restrictions regarding jumping to labels than C has. Something like:

 ------------------
 State myCoroutine(State s) {
     yield "hello";

     if(foo) {
         yield " world";
     }

     yield "!";
 }
 ------------------

 A Protothreads-like preprocessing approach that used switch/case would
 turn that into:

 ------------------
 State myCoroutine(State s) {
 switch(s.jumpLabel){
     case 0:
     // Start converted code: //////////////

     //yield "hello"; -> return+case:
     return State(1, "hello");
     case 1:

     if(int foo = getFoo()) {
         //yield " world"; ->
         return State(2, " world"); return+case:
         case 2:
     }

     //yield "!"; -> return+case:
     return State(3, "!");
     case 3:

     // End converted code //////////////
     break;
 }
 }
 ------------------

 That "if" really messed things up. AIUI, I don't think D will allow
 things like that.
The above code compiles, after removing the extra "return+case:". -- /Jacob Carlborg
Sep 25 2016
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 09/26/2016 02:42 AM, Jacob Carlborg wrote:
 On 2016-09-25 20:16, Nick Sabalausky wrote:
 I don't see why not, but there would be one notable drawback: You could
 only have one instance existing at a time (per thread).
But how does it work in languages which don't pass in the state explicitly, like with "yield" in C#? Or async/await? I guess one doesn't call the function multiple times in the same way instead it's done automatically under the hood, passing in the state which might be stored in the "Task" that is returned?
I don't know about async/await, because all my C# work was with versions of C# that predated those. But as for C#'s yield functions, yes, that's my understanding: It's hidden magic automatically inserted by the compiler. As a side note, that actually makes D's opApply a very interesting case, really: Instead of doing like the other implementations and "yielding" by removing the top function from the callstack (ie "returning"), opApply does the oppposite: It yields by pushing whatever function it's yielding *to*, *onto* the callstack. So its state is stored on the stack, no need for magic. The downside is, that's exactly what makes it impossible for opApply to be used as a range (at least without introducing a true fiber to save/restore the callstack): popFront *must* return in order for the user's algorithm to call front and obtain the current element - because that's how the range interface works.
 The above code compiles, after removing the extra "return+case:".
Really? I may have to look into this more then. Hmm, well for starters, according to <http://dlang.org/spec/statement.html#GotoStatement>: "It is illegal for a GotoStatement to be used to skip initializations." Although if that's true, I don't know why that example compiled. Maybe because "foo" wasn't used from "case 2:" onward? Maybe it failed to notice the initialization since it was in an if condition? Or maybe the compiler forgot to implement the same "can't skip initialization" check for switch/case?
Sep 26 2016
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2016-09-26 16:31, Nick Sabalausky wrote:

 Really? I may have to look into this more then.
Yes, try compiling it next time ;)
 Hmm, well for starters, according to
 <http://dlang.org/spec/statement.html#GotoStatement>:
 "It is illegal for a GotoStatement to be used to skip initializations."

 Although if that's true, I don't know why that example compiled. Maybe
 because "foo" wasn't used from "case 2:" onward? Maybe it failed to
 notice the initialization since it was in an if condition? Or maybe the
 compiler forgot to implement the same "can't skip initialization" check
 for switch/case?
It's a bug [1]. But it seems that if State is replaced with a plain int, the compiler will initialize the variable, even if it's later jumped over. So I assume, that when it works as it supposed to, it will be possible to jump like in C, but the compiler will still initialize the variables. [1] https://issues.dlang.org/show_bug.cgi?id=16549 -- /Jacob Carlborg
Sep 26 2016
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2016-09-26 16:31, Nick Sabalausky wrote:

 I don't know about async/await, because all my C# work was with versions
 of C# that predated those. But as for C#'s yield functions, yes, that's
 my understanding: It's hidden magic automatically inserted by the compiler.
This talk [1] from cppcon 2016, C++ Coroutines: Under the covers, gave some really nice insight how they implement coroutines in C++. It basically works like in Protothreads. The secret how the state is handled, I believe, is that all coroutines need to return a specific type that, I'm guessing, contains the state. Like generator<T> or task<T>. This return value is later used to resume the function. Directly calling the coroutine again will start a new invocation with a new state. I'm guessing it works the same in C# [2]. The state is hidden in the IEnumerable that is returned, which will later be used in a foreach loop which will resume the coroutine. [1] https://www.youtube.com/watch?v=bzAbe1VzhKk&index=51&list=PLHTh1InhhwT7J5jl4vAhO1WvGHUUFgUQH [2] https://msdn.microsoft.com/en-us/library/9k7k7cf0.aspx -- /Jacob Carlborg
Oct 04 2016
prev sibling next sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Friday, September 23, 2016 16:57:49 Nick Sabalausky via Digitalmars-d 
wrote:
 Some ages ago, a whole suite of "assertPred" functions were written
 (giving better diagnostic info, like showing "expected vs actual"), were
 totally awesome, were submitted to phobos...and were rejected because it
 was deemed both easy enough and preferable to get these features by
 modifying DMD to add behind-the-scenes AST magic to "assert".

 So...umm...yea...whatever happened to that beefed-up "assert" feature?
It was rejected: https://issues.dlang.org/show_bug.cgi?id=5547 Basically, the review that involved assertPred determined that it would be better to hav assert do it, but when someone tried to put in in the compiler, Walter rejected it, saying that it should be a library solution. There are also some potentially issues having assertions print out additional information in the case where the assertion is not in a unit test and there was concern over that. The big problem with assertPred though is that while it's really nice, it's also really expensive. All of those template instantions really added up. So, I don't know if it's ultimately a good idea or not, but I'd fully expect some of the unit testing libraries to have something similar, even if it's not anywhere near as fancy. - Jonathan M Davis
Sep 23 2016
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 09/23/2016 07:57 PM, Jonathan M Davis via Digitalmars-d wrote:
 On Friday, September 23, 2016 16:57:49 Nick Sabalausky via Digitalmars-d
 wrote:
 So...umm...yea...whatever happened to that beefed-up "assert" feature?
[...]
Ugh, so, "It was rejected for being library instead of assert, then it was rejected for being assert instead of library". /facepalm Guess I'll just have to try getting used to awful Java-isms like x.notEqual(y) :(
Sep 23 2016
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 09/23/2016 11:47 PM, Nick Sabalausky wrote:
 On 09/23/2016 07:57 PM, Jonathan M Davis via Digitalmars-d wrote:
 On Friday, September 23, 2016 16:57:49 Nick Sabalausky via Digitalmars-d
 wrote:
 So...umm...yea...whatever happened to that beefed-up "assert" feature?
[...]
Ugh, so, "It was rejected for being library instead of assert, then it was rejected for being assert instead of library". /facepalm Guess I'll just have to try getting used to awful Java-isms like x.notEqual(y) :(
And then that leads too, to the question of whether such third-party asserts are a good idea for the doc unittests I like so much... :/
Sep 23 2016
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Friday, September 23, 2016 23:50:03 Nick Sabalausky via Digitalmars-d 
wrote:
 On 09/23/2016 11:47 PM, Nick Sabalausky wrote:
 On 09/23/2016 07:57 PM, Jonathan M Davis via Digitalmars-d wrote:
 On Friday, September 23, 2016 16:57:49 Nick Sabalausky via Digitalmars-d

 wrote:
 So...umm...yea...whatever happened to that beefed-up "assert" feature?
[...]
Ugh, so, "It was rejected for being library instead of assert, then it was rejected for being assert instead of library". /facepalm
Indeed. :(
 Guess I'll just have to try getting used to awful Java-isms like
 x.notEqual(y) :(
And then that leads too, to the question of whether such third-party asserts are a good idea for the doc unittests I like so much... :/
I'd say not. If you're writing a library for general consumption, I don't think that anyone else who is not actually helping to develop it should have to know or care what unit testing facilities you're using. assert is universal and clear, whereas other stuff is not universal and may or may not be clear to those not familiar with it. Also, my take on it is that ddoc-ed unittest blocks are really there to just give examples for the documentation, not to test your code. You want the examples tested so that you know that they work - which is why ddoc-ed unittest blocks are such a great feature - but it's not their purpose to test your library. It would be wholly unreasonable to have thorough tests in the documentation, and what makes a good example doesn't necessarily make for a very good test (or vice versa). So, you can just use basic assertions in your examples without really impacting your actual tests. You then have separate unittest blocks that have the full set of tests needed to verify full functionality, and those can use whatever unit testing tools you prefer without affecting anyone reading the documentation. - Jonathan M Davis
Sep 24 2016
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 09/24/2016 03:34 AM, Jonathan M Davis via Digitalmars-d wrote:
 On Friday, September 23, 2016 23:50:03 Nick Sabalausky via Digitalmars-d
 wrote:
 And then that leads too, to the question of whether such third-party
 asserts are a good idea for the doc unittests I like so much... :/
I'd say not. If you're writing a library for general consumption, I don't think that anyone else who is not actually helping to develop it should have to know or care what unit testing facilities you're using. assert is universal and clear, whereas other stuff is not universal and may or may not be clear to those not familiar with it.
Yea, honestly, that's my thought as well. :( And it's a big part of why I was (and still am) so disappointed that assertPred didn't become official even if it is technically usable as third-party. We could've already been using that for years by now, standard, if it weren't for D's habit of letting perfect be the enemy of progress.
 Also, my take on it is that ddoc-ed unittest blocks are really there to just
 give examples for the documentation, not to test your code. You want the
 examples tested so that you know that they work - which is why ddoc-ed
 unittest blocks are such a great feature - but it's not their purpose to
 test your library. It would be wholly unreasonable to have thorough tests in
 the documentation, and what makes a good example doesn't necessarily make
 for a very good test (or vice versa).
I think there's a balance (albeit a difficult one). I agree that being pedantic with the examples can make them overly cluttered, so that should be avoided. But all major side-cases should still be documented, and examples help with that as they do with anything else. For example, std.path.buildNormalizedPath: It should not only document typical cases like `buildNormalizedPath("foo","bar")`, but it's very important for its docs to also be clear about various other significant cases: - How does it handle empty strings as arguments? - What does it return for buildNormalizedPath("foo",".."). It used to return empty string which turned out very problematic. Now it returns "." which is much better. - What happens when I do buildNormalizedPath(relativePath,absolutePath)? These are all very important cases that still need to be documented. And examples work excellently as documentation here. (And IIRC, buildNormalizedPath docs are doing a good job of this.) So a good set of examples *do* still test a significant amount of a function, even if it isn't exhaustive. So because of that, and heck, even if your examples *are* overly sparse, sometimes you will get a failure in one. Which, of course, is the whole point of actually testing them. And when you do, it's a big help for the diagnostic output to actually be, well, helpful: The whole cycle of: - Search the stack trace for the relevant file/line number. - Find it: assert(x.foo(blah...) == y); - Insert scaffolding: import std.stdio; writeln("x.foo(blah...): ", x.foo(blah...)); writeln("y: ", y); - Recompile/re-run tests (and maybe go back again and fix any stupid typos in the scaffolding). - Finally view key information that COULD'VE been shown right from the start. Gets very tiresome very quickly, even if it's only in the example tests.
Sep 24 2016
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On Friday, 23 September 2016 at 20:57:49 UTC, Nick Sabalausky 
wrote:
 were rejected because it was deemed both easy enough and 
 preferable to get these features by modifying DMD to add 
 behind-the-scenes AST magic to "assert".

 So...umm...yea...whatever happened to that beefed-up "assert" 
 feature?
assertPred!"=="(a, b); assertPred!"!"(a); assertPred!(std.range.equal)(a, b); Seems to do most of what DIP83 does w/ expensive feature design, and compiler implementation work. Also http://code.dlang.org/packages/unit-threaded comes with a couple of test comparators, though it follows ruby rspec's bad idea of giving every comparator a name (which has to be learnt and documented).
Sep 24 2016
next sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Saturday, September 24, 2016 08:03:15 Martin Nowak via Digitalmars-d wrote:
 On Friday, 23 September 2016 at 20:57:49 UTC, Nick Sabalausky

 wrote:
 were rejected because it was deemed both easy enough and
 preferable to get these features by modifying DMD to add
 behind-the-scenes AST magic to "assert".

 So...umm...yea...whatever happened to that beefed-up "assert"
 feature?
assertPred!"=="(a, b); assertPred!"!"(a); assertPred!(std.range.equal)(a, b); Seems to do most of what DIP83 does w/ expensive feature design, and compiler implementation work. Also http://code.dlang.org/packages/unit-threaded comes with a couple of test comparators, though it follows ruby rspec's bad idea of giving every comparator a name (which has to be learnt and documented).
assertPred as I had implemented it previously worked quite nicely except for the fact that it resulted in a fair bit of compiler overhead with all of the template stuff it was doing (I don't know how much that could be improved). As I recall, it was generally well-liked. It was just that some folks (Andrei included IIRC) thought that assert should be able to do at least the basics of what assertPred did automatically, which is why it was rejected. Since that time, there has been a greater push to do stuff in libraries where we can rather than to make the language do more, so maybe it would go over better today than it did then. - Jonathan M Davis
Sep 24 2016
prev sibling next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 09/24/2016 04:03 AM, Martin Nowak wrote:
 assertPred!"=="(a, b);
 assertPred!"!"(a);
 assertPred!(std.range.equal)(a, b);

 Seems to do most of what DIP83 does w/ expensive feature design, and
 compiler implementation work.
 Also http://code.dlang.org/packages/unit-threaded comes with a couple of
 test comparators, though it follows ruby rspec's bad idea of giving
 every comparator a name (which has to be learnt and documented).
Yea, incidentally, I just started using unit-threaded for the first time this week, and so far, for the most part, I really quite like it a lot. But those comparator functions give me bad flashbacks of old-school Java. Hmm, I wonder if Atila would be amenable to a more assertPred-like function in unit-threaded. Maybe a `should!"=="(leftSide, rightSide)` would fit in well. Or better yet, also adding in something like that that trick used in Andre's really nice recently proposed "dump" function (or whatever the name of it was), where it also prints out the actual argument provided in addition to the argument's value.
Sep 24 2016
parent Atila Neves <atila.neves gmail.com> writes:
On Saturday, 24 September 2016 at 14:08:30 UTC, Nick Sabalausky 
wrote:
 On 09/24/2016 04:03 AM, Martin Nowak wrote:
 [...]
Yea, incidentally, I just started using unit-threaded for the first time this week, and so far, for the most part, I really quite like it a lot. But those comparator functions give me bad flashbacks of old-school Java. Hmm, I wonder if Atila would be amenable to a more assertPred-like function in unit-threaded. Maybe a `should!"=="(leftSide, rightSide)` would fit in well. Or better yet, also adding in something like that that trick used in Andre's really nice recently proposed "dump" function (or whatever the name of it was), where it also prints out the actual argument provided in addition to the argument's value.
See my answer to Martin. "!" next to "==" just looks wrong. Atila
Sep 27 2016
prev sibling parent Atila Neves <atila.neves gmail.com> writes:
On Saturday, 24 September 2016 at 08:03:15 UTC, Martin Nowak 
wrote:
 On Friday, 23 September 2016 at 20:57:49 UTC, Nick Sabalausky 
 wrote:
 were rejected because it was deemed both easy enough and 
 preferable to get these features by modifying DMD to add 
 behind-the-scenes AST magic to "assert".

 So...umm...yea...whatever happened to that beefed-up "assert" 
 feature?
assertPred!"=="(a, b); assertPred!"!"(a); assertPred!(std.range.equal)(a, b);
assertPred!"==" looks like it asserts not equal, which is the 1st problem with this, assertPred!"!=" looks even worse. I considered going down this route but... ugh.
 Seems to do most of what DIP83 does w/ expensive feature 
 design, and compiler implementation work.
 Also http://code.dlang.org/packages/unit-threaded comes with a 
 couple of test comparators, though it follows ruby rspec's bad 
 idea of giving every comparator a name (which has to be learnt 
 and documented).
I didn't want to give every comparator a name, this was the least bad option from my point of view. I'd rather `assert` be magical. Or, be able to have AST macros to get around the limitations of how operator overloading works in D. Don't get me wrong, the reason why it works the way it does is awesome in general, but limiting in this case in particular. I ended up implementing this: foo.should == bar But unfortunately it only works for equality. This is also possible: foo.should.not == bar But I can't do the same with any of the other operators AFAICR so I went with consistency even though the equality check is the most common. Atila
Sep 27 2016