www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - anonymous functions and scope(exit)

reply Luis <Luis.panadero gmail.com> writes:
This is intentional ?

```
         should(function void() {
             auto emptyStack = SimpleStack!int();
             scope(exit) emptyStack.free; // <= This is never 
called

             emptyStack.reserve(16);
             emptyStack.top;
         }).Throw!RangeError;
```

scope(exit) inside of a anonymous functions, it's never called.
Jul 03
next sibling parent frame <frame86 live.com> writes:
On Saturday, 3 July 2021 at 17:20:47 UTC, Luis wrote:
 This is intentional ?

 ```
         should(function void() {
             auto emptyStack = SimpleStack!int();
             scope(exit) emptyStack.free; // <= This is never 
 called

             emptyStack.reserve(16);
             emptyStack.top;
         }).Throw!RangeError;
 ```

 scope(exit) inside of a anonymous functions, it's never called.
Please provide an example code. What lib is this? Normally scope(exit) works also for anonymous functions.
Jul 03
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/3/21 1:20 PM, Luis wrote:
 This is intentional ?
 
 ```
          should(function void() {
              auto emptyStack = SimpleStack!int();
              scope(exit) emptyStack.free; // <= This is never called
 
              emptyStack.reserve(16);
              emptyStack.top;
          }).Throw!RangeError;
 ```
 
 scope(exit) inside of a anonymous functions, it's never called.
In principle, it should technically be called. But in practice, the compiler does not have to clean up anything when an `Error` is thrown. Whether it does or not is defined by the implementation. However, it should *always* work if it's an `Exception` and not an `Error`. ```d import std.stdio; void main() { auto f = function void() { scope(exit) writeln("hi"); throw new Exception("boo"); }; f(); } ``` prints "hi" -Steve
Jul 03
parent reply frame <frame86 live.com> writes:
On Saturday, 3 July 2021 at 17:39:18 UTC, Steven Schveighoffer 
wrote:

 But in practice, the compiler does not have to clean up 
 anything when an `Error` is thrown. Whether it does or not is 
 defined by the implementation.
This should be really mentionend in the docs? "Guard", yeah...
Jul 03
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/3/21 4:08 PM, frame wrote:
 On Saturday, 3 July 2021 at 17:39:18 UTC, Steven Schveighoffer wrote:
 
 But in practice, the compiler does not have to clean up anything when 
 an `Error` is thrown. Whether it does or not is defined by the 
 implementation.
This should be really mentionend in the docs? "Guard", yeah...
Yeah, there isn't a good discussion of the differences between Error and Exception on that page. -Steve
Jul 03
parent reply Luis <Luis.panadero gmail.com> writes:
On Saturday, 3 July 2021 at 20:46:00 UTC, Steven Schveighoffer 
wrote:
 On 7/3/21 4:08 PM, frame wrote:
 On Saturday, 3 July 2021 at 17:39:18 UTC, Steven Schveighoffer 
 wrote:
 
 But in practice, the compiler does not have to clean up 
 anything when an `Error` is thrown. Whether it does or not is 
 defined by the implementation.
This should be really mentionend in the docs? "Guard", yeah...
Yeah, there isn't a good discussion of the differences between Error and Exception on that page. -Steve
On [The D Error Handling Solution](https://dlang.org/spec/errors.html#the_d_error_handling_solution), says :
 If code detects an error like "out of memory," then an Error is 
 thrown with a message
 saying "Out of memory". The function call stack is unwound, 
 looking for a handler for
 the Error. Finally blocks are executed as the stack is unwound. 
 If an error handler is
 found, execution resumes there. If not, the default Error 
 handler is run, which displays
 the message and terminates the program.
scope(exit) it's syntactic sugar for a classic `try {} finally {}` . The documentation says that must be executed.
Jul 03
parent reply frame <frame86 live.com> writes:
On Saturday, 3 July 2021 at 22:04:04 UTC, Luis wrote:

 scope(exit) it's syntactic sugar for a classic `try {} finally 
 {}` . The documentation says that must be executed.
It works if you replace printf() with writeln() or use writeln() after. There must be some buffer issue.
Jul 03
parent reply Luis <Luis.panadero gmail.com> writes:
On Saturday, 3 July 2021 at 22:52:39 UTC, frame wrote:
 On Saturday, 3 July 2021 at 22:04:04 UTC, Luis wrote:

 scope(exit) it's syntactic sugar for a classic `try {} finally 
 {}` . The documentation says that must be executed.
It works if you replace printf() with writeln() or use writeln() after. There must be some buffer issue.
Not works as you expected. Yes, replacing by writeln (better said, putting a writeln) makes it to work. More weird, if I replace the printf(...) by a fprintf(stderr, ...), I don't get anything. To discard depening on checking if it works by the side effect of writing something on the console, now i using malloc/free and checking with valgrind for a lost memory : ```d /+ dub.sdl: dependency "pijamas" version="~>1.1" +/ import core.exception; import core.stdc.stdio; import core.stdc.stdlib; import std.stdio : writeln; void main() { import pijamas; should(() { int* ptr = cast(int*) malloc(int.sizeof * 1000); try { fprintf(stderr, "Hello\n"); throw new RangeError("bla bla"); } finally { // writeln("Bye 1"); fprintf(stderr, "Bye\n"); free(ptr); } }).Throw!RangeError; } ``` Outputs this : ``` $ f.d Hello ``` And valgrind outputs (using the temporal executable generated by dub on /tmp/.dub/...) : ``` $ valgrind --leak-check=full /tmp/.dub/build/f-\~master/application-debug-linux.posix-x86_64-dmd_2097-FB7AFBA927D99FA3DDD7307BACA865DA/f ==18356== Memcheck, a memory error detector ==18356== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==18356== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info ==18356== Command: /tmp/.dub/build/f-~master/application-debug-linux.posix-x86_64-dmd_2097-FB7AFBA927D99FA3DDD7307BACA865DA/f ==18356== --18356-- WARNING: Serious error when reading debug info --18356-- When reading debug info from /tmp/.dub/build/f-~master/application-debug-linux.posix-x86_64-dmd_2097-FB7AFBA927D99FA3DDD7307BACA865DA/f: --18356-- DWARF line info appears to be corrupt - the section is too small --18356-- WARNING: Serious error when reading debug info --18356-- When reading debug info from /tmp/.dub/build/f-~master/application-debug-linux.posix-x86_64-dmd_2097-FB7AFBA927D99FA3DDD7307BACA865DA/f: --18356-- read_filename_table: .debug_line is missing? Hello ==18356== ==18356== HEAP SUMMARY: ==18356== in use at exit: 4,056 bytes in 3 blocks ==18356== total heap usage: 231 allocs, 228 frees, 76,436 bytes allocated ==18356== ==18356== 32 bytes in 1 blocks are possibly lost in loss record 2 of 3 ==18356== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind .... ==18356== 4,000 bytes in 1 blocks are definitely lost in loss record 3 of 3 ==18356== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==18356== by 0x16873D: _D1f4mainFZ9__lambda1FNbZv (f.d:15) ==18356== by 0x168B2F: _D7pijamas9assertion__T9AssertionTPFNbZvZQs__T5ThrowHTC4core9exception10Rang ErrorZQBlMFNeAyamZv (assertion.d:602) ==18356== by 0x168724: _Dmain (assertion.d:598) ==18356== by 0x18C962: _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZ9__lambda2MFZv (in /tmp/.dub/build/f-~master/application-debug-linux.posix-x86_64-dmd_2097-FB7AFBA927D99FA3DDD7307BACA865DA/f) ==18356== by 0x18C804: _D2rt6dmain212_d_run_main2UAAamPUQgZiZ7tryExecMFMDFZvZv (in /tmp/.dub/build/f-~master/application-debug-linux.posix-x86_64-dmd_2097-FB7AFBA927D99FA3DDD7307BACA865DA/f) ==18356== by 0x18C8DE: _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv (in /tmp/.dub/build/f-~master/application-debug-linux.posix-x86_64-dmd_2097-FB7AFBA927D99FA3DDD7307BACA865DA/f) ==18356== by 0x18C804: _D2rt6dmain212_d_run_main2UAAamPUQgZiZ7tryExecMFMDFZvZv (in /tmp/.dub/build/f-~master/application-debug-linux.posix-x86_64-dmd_2097-FB7AFBA927D99FA3DDD7307BACA865DA/f) ==18356== by 0x18C765: _d_run_main2 (in /tmp/.dub/build/f-~master/application-debug-linux.posix-x86_64-dmd_2097-FB7AFBA927D99FA3DDD7307BACA865DA/f) ==18356== by 0x18C4C1: _d_run_main (in /tmp/.dub/build/f-~master/application-debug-linux.posix-x86_64-dmd_2097-FB7AFBA927D99FA3DDD7307BACA865DA/f) ==18356== by 0x1687B9: main (entrypoint.d:29) ==18356== ==18356== LEAK SUMMARY: ==18356== definitely lost: 4,000 bytes in 1 blocks ==18356== indirectly lost: 0 bytes in 0 blocks ==18356== possibly lost: 32 bytes in 1 blocks ==18356== still reachable: 24 bytes in 1 blocks ==18356== suppressed: 0 bytes in 0 blocks ==18356== Reachable blocks (those to which a pointer was found) are not shown. ==18356== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==18356== ==18356== For lists of detected and suppressed errors, rerun with: -s ==18356== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0) ``` But the most funny thing, it's that if I remove the comment on the writeln, then just works and the memory leak dissapers! **DMD , it's doing very weird things here**, only executing the scope (exit) {} / finally {} when there is a side effect with writeln ! **This not happens with my local install of ldc2**. On my original post, I was just searching for a memory leak on a tests of my private library, where I have some nogc containers using std.experimental.allocator, and expecting that the scope (exit) being executed and releasing the allocated memory. And this wasn't happening. dmd --version : DMD64 D Compiler v2.097.0 ldc2 -v : binary /usr/bin/ldc2 version 1.20.1 (DMD v2.090.1, LLVM 10.0.0) config /etc/ldc2.conf (x86_64-pc-linux-gnu)
Jul 04
parent reply jfondren <julian.fondren gmail.com> writes:
On Sunday, 4 July 2021 at 08:24:36 UTC, Luis wrote:
 On Saturday, 3 July 2021 at 22:52:39 UTC, frame wrote:
 It works if you replace printf() with writeln() or use 
 writeln() after. There must be some buffer issue.
Not works as you expected. Yes, replacing by writeln (better said, putting a writeln) makes it to work. More weird,
Dennis's explanation makes the most sense: writeln can throw an Exception, so its presence prevents nothrow inference, which otherwise permits the (not intended to be catchable) RangeError to exit without properly unwinding the stack. By that, what you're running into is an unpleasant interaction between 1. scope(exit)s that you're writing 2. Errors being thrown rather than Exceptions 3. anonymous functions getting inferred as nothrow guards, they have to make it explicit. And while checking for prior issues like this, I found https://issues.dlang.org/show_bug.cgi?id=17494
Not cleaning up after an Error is thrown is allowed by the D 
spec. This enhancement allows much better code to be generated 
for `nothrow` code when `scope` is used. It will also not unwind 
declarations with destructors in `nothrow` code when Errors are 
thrown.
I read this initially as "it is a bug for scope(exit) to ever run after an Error is thrown", but it's an optimization for nothrow code, which fits what you're seeing.
Jul 04
next sibling parent jfondren <julian.fondren gmail.com> writes:
On Sunday, 4 July 2021 at 10:07:08 UTC, jfondren wrote:
 By that, what you're running into is an unpleasant interaction 
 between
 1. scope(exit)s that you're writing
 2. Errors being thrown rather than Exceptions
 3. anonymous functions getting inferred as nothrow


 scope guards, they have to make it explicit.
Although taking the example code and giving the anonymous function a name does not actually change this behavior. It really has to potentially throw a non-Error exception, like ```d if (ptr is null) throw new Exception("null"); ``` There are options like "add a debugging flag to not treat Errors differently"... but you know, you're going to have gaps in what you can reasonably test anyway, like code that calls libc exit(), or code that SIGKILLs itself, or code that enters an infinite loop. You could document some issues with catching non-catchable exceptions.
Jul 04
prev sibling next sibling parent Luis <Luis.panadero gmail.com> writes:
On Sunday, 4 July 2021 at 10:07:08 UTC, jfondren wrote:
 On Sunday, 4 July 2021 at 08:24:36 UTC, Luis wrote:

 Dennis's explanation makes the most sense:

 writeln can throw an Exception, so its presence prevents 
 nothrow inference, which otherwise permits the (not intended to 
 be catchable) RangeError to exit without properly unwinding the 
 stack.

 By that, what you're running into is an unpleasant interaction 
 between
 1. scope(exit)s that you're writing
 2. Errors being thrown rather than Exceptions
 3. anonymous functions getting inferred as nothrow
I did https://issues.dlang.org/show_bug.cgi?id=22099
Jul 04
prev sibling parent frame <frame86 live.com> writes:
On Sunday, 4 July 2021 at 10:07:08 UTC, jfondren wrote:

Not cleaning up after an Error is thrown is allowed by the D 
spec. This enhancement allows much better code to be generated 
for `nothrow` code when `scope` is used. It will also not 
unwind declarations with destructors in `nothrow` code when 
Errors are thrown.
I read this initially as "it is a bug for scope(exit) to ever run after an Error is thrown", but it's an optimization for nothrow code, which fits what you're seeing.
So it's might not a bug but not well documented. ```scope(...)``` still acts as ```try... catch(Throwable)... finally``` as logic people would expect although not written in stone anywhere. Code within that blocks that are not seen by the compiler as 'nothrow' will be executed. But the compiler may silently mark code as 'nothrow' if possible and then this "feature" gets enabled and cleanup will be discarded.
Jul 04
prev sibling next sibling parent jfondren <julian.fondren gmail.com> writes:
On Saturday, 3 July 2021 at 17:20:47 UTC, Luis wrote:
 This is intentional ?
...
 scope(exit) inside of a anonymous functions, it's never called.
``` $ rdmd --eval 'iota(2).map!((int x) { scope(exit) writeln("got: ", x); return x+1; }).array.writeln' got: 0 got: 1 [1, 2] ``` Conclusion: it's not intentional.
Jul 03
prev sibling parent reply Dennis <dkorpel gmail.com> writes:
On Saturday, 3 July 2021 at 17:20:47 UTC, Luis wrote:
 scope(exit) inside of a anonymous functions, it's never called.
I think the compiler infers the function `nothrow` since you don't throw any `Exception`, only an `Error`. Errors represent unrecoverable bugs, after which the program is in an invalid state, so the compiler is free to exit immediately without caring about destructors or `scope(exit)`. Use `Exception` instead of `Error` if you want the stack to properly unwind.
Jul 03
parent Luis <Luis.panadero gmail.com> writes:
On Saturday, 3 July 2021 at 17:47:47 UTC, Dennis wrote:
 On Saturday, 3 July 2021 at 17:20:47 UTC, Luis wrote:
 scope(exit) inside of a anonymous functions, it's never called.
I think the compiler infers the function `nothrow` since you don't throw any `Exception`, only an `Error`. Errors represent unrecoverable bugs, after which the program is in an invalid state, so the compiler is free to exit immediately without caring about destructors or `scope(exit)`. Use `Exception` instead of `Error` if you want the stack to properly unwind.
Indeed, this is happening. I can reproduce with this : ```d /+ dub.sdl: dependency "pijamas" version="~>1.1" +/ import core.exception; void main() { import core.stdc.stdio; import pijamas; should(() { printf("Hello\n"); scope(exit) printf("Bye\n"); throw new RangeError("bla bla"); }).Throw!RangeError; auto f = () { printf("Hello\n"); scope(exit) printf("Bye\n"); throw new RangeError("bla bla"); }; f(); } ``` Outputs this : ``` $ f.d Hello Hello core.exception.RangeError bla bla(20): Range violation ---------------- source/f.d:20 nothrow void f.main().__lambda2() [0x5647d46a17db] source/f.d:22 _Dmain [0x5647d46a1732] Program exited with code 1 ``` If I change the RangeError, by a Exception, then the scope(exit) it's executed.
Jul 03