www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Solving a constraint hiding an error in a function literal

reply Nick Treleaven <nick geany.org> writes:
When a constraint for an alias parameter checks that calling it 
compiles, it instantiates any function literal template. That 
triggers any latent bugs in the user's function literal that 
couldn't be detected before IFTI instantiates it. Those bugs then 
cause the constraint to fail, because the literal does not 
compile. The constraint then removes the otherwise matching 
template from the list of candidates to use.
```d
void f(alias a)() if (is(typeof(a()))) {}

void main()
{
     f!(x => blarg);
}
```
https://issues.dlang.org/show_bug.cgi?id=11907
https://issues.dlang.org/show_bug.cgi?id=14217

This is a common pattern in Phobos.
Initially I thought a constraint expression like 
`__traits(compiles, a())` should be moved to a `static if` test 
with a helpful error message. std.algorithm.all was changed to 
use static assert instead:

https://issues.dlang.org/show_bug.cgi?id=13683
https://github.com/dlang/phobos/pull/6607/files

But that still swallows the actual compile error and just says 
the callable doesn't work without really saying why (why isn't it 
a unary predicate if it accepts `range.front` as an argument?). 
Alternatively, the constraint check could be removed and you will 
get an internal error actually stating the precise error in the 
callable's body. But it doesn't help with overloading (see below).

What if there was a new trait to solve this? Suppose 
`__traits(callable, expr, args)`, that just does IFTI (if `expr` 
is a function template) and then checks that the parameter list 
of `expr` accepts `args`. It does not check that `expr(args)` 
actually compiles, it ignores any errors in the body of `expr`.

It doesn't check the return type, because that's not possible in 
general when the body does not compile and return type inference 
is needed. But overloading on the return type of a callable 
doesn't seem as important compared to overloading on the 
parameters of a callable.

The new trait can be used as a constraint or as a `static if` 
condition. As a constraint it can help to determine which 
overload matches. Imagine two overloads:
```d
// body would call a(int)
void f(alias a)()
if (__traits(callable, a, int()))

// body would call a(int, int)
void f(alias a)()
if (__traits(callable, a, int(), int()))
```
The trait would allow one of the overloads to be selected based 
on the number of parameters that `a` has, *even when* the body of 
`a` contains an error. Then one of the overloads of `f` is 
instantiated and an internal template error will be raised that 
`a` has an error. Despite this being an internal error, it is far 
better than the Phobos status quo because you can see what's 
actually failing with a precise error message.
Feb 03 2023
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/3/23 7:43 AM, Nick Treleaven wrote:


 if (__traits(callable, a, int()))


GMTA?

https://forum.dlang.org/post/rj5hok$c6q$1 digitalmars.com

-Steve
Feb 03 2023
next sibling parent Nick Treleaven <nick geany.org> writes:
On Friday, 3 February 2023 at 13:57:02 UTC, Steven Schveighoffer 
wrote:

 On 2/3/23 7:43 AM, Nick Treleaven wrote:


 if (__traits(callable, a, int()))


 GMTA?

 https://forum.dlang.org/post/rj5hok$c6q$1 digitalmars.com


Thanks, yes I probably remembered it from things read here in the 
past.
Feb 03 2023
prev sibling parent WebFreak001 <d.forum webfreak.org> writes:
On Friday, 3 February 2023 at 13:57:02 UTC, Steven Schveighoffer 
wrote:

 On 2/3/23 7:43 AM, Nick Treleaven wrote:


 if (__traits(callable, a, int()))


 GMTA?

 https://forum.dlang.org/post/rj5hok$c6q$1 digitalmars.com

 -Steve


I remember this thread now and I'm actually amazed we haven't 
been making any progress towards this yet, because this looks 
like such a simple and elegant solution we could use for new code 
and Phobos v2. (because it's a backwards incompatible change when 
suddenly other code paths would attempt to compile I think)

I'll try starting a DIP, maybe that's just everything we need to 
get going.
Feb 09 2023
prev sibling next sibling parent Salih Dincer <salihdb hotmail.com> writes:
On Friday, 3 February 2023 at 12:43:54 UTC, Nick Treleaven wrote:

 
 The new trait can be used as a constraint or as a `static if` 
 condition. As a constraint it can help to determine which 
 overload matches. Imagine two overloads:
 ```d
 // body would call a(int)
 void f(alias a)()
 if (__traits(callable, a, int()))

 // body would call a(int, int)
 void f(alias a)()
 if (__traits(callable, a, int(), int()))


I prefer to use `static if` inside the template to avoid similar 
situations. Otherwise, I have a difficult process to find where 
the error is.

SDB 79
Feb 03 2023
prev sibling next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 3 February 2023 at 12:43:54 UTC, Nick Treleaven wrote:

 Initially I thought a constraint expression like 
 `__traits(compiles, a())` should be moved to a `static if` test 
 with a helpful error message. std.algorithm.all was changed to 
 use static assert instead:

 https://issues.dlang.org/show_bug.cgi?id=13683
 https://github.com/dlang/phobos/pull/6607/files

 But that still swallows the actual compile error and just says 
 the callable doesn't work without really saying why (why isn't 
 it a unary predicate if it accepts `range.front` as an 
 argument?). Alternatively, the constraint check could be 
 removed and you will get an internal error actually stating the 
 precise error in the callable's body. But it doesn't help with 
 overloading (see below).

 What if there was a new trait to solve this? Suppose 
 `__traits(callable, expr, args)`, that just does IFTI (if 
 `expr` is a function template) and then checks that the 
 parameter list of `expr` accepts `args`. It does not check that 
 `expr(args)` actually compiles, it ignores any errors in the 
 body of `expr`.


I think a better solution would be to have `__traits(compiles)` 
return the error message(s) when it fails, instead of just a 
boolean `false`. Then we could write code like:

```d
enum compResult = __traits(compiles, expr);
static if (compResult) // implicitly converts to bool
{
     doStuffWith(expr);
}
else
{
     // print error message
     static assert(0, compResult.message);
}
```

Getting constraints to understand this would require compiler 
changes, but that shouldn't be *too* difficult. Might also be 
worth having `static assert(__traits(compiles, expr))`, without 
an explicit message argument, print the returned message.
Feb 03 2023
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/3/23 1:55 PM, Paul Backus wrote:

 On Friday, 3 February 2023 at 12:43:54 UTC, Nick Treleaven wrote:

 Initially I thought a constraint expression like `__traits(compiles, 
 a())` should be moved to a `static if` test with a helpful error 
 message. std.algorithm.all was changed to use static assert instead:

 https://issues.dlang.org/show_bug.cgi?id=13683
 https://github.com/dlang/phobos/pull/6607/files

 But that still swallows the actual compile error and just says the 
 callable doesn't work without really saying why (why isn't it a unary 
 predicate if it accepts `range.front` as an argument?). Alternatively, 
 the constraint check could be removed and you will get an internal 
 error actually stating the precise error in the callable's body. But 
 it doesn't help with overloading (see below).

 What if there was a new trait to solve this? Suppose 
 `__traits(callable, expr, args)`, that just does IFTI (if `expr` is a 
 function template) and then checks that the parameter list of `expr` 
 accepts `args`. It does not check that `expr(args)` actually compiles, 
 it ignores any errors in the body of `expr`.


 
 I think a better solution would be to have `__traits(compiles)` return 
 the error message(s) when it fails, instead of just a boolean `false`. 
 Then we could write code like:
 
 ```d
 enum compResult = __traits(compiles, expr);
 static if (compResult) // implicitly converts to bool
 {
      doStuffWith(expr);
 }
 else
 {
      // print error message
      static assert(0, compResult.message);
 }
 ```
 
 Getting constraints to understand this would require compiler changes, 
 but that shouldn't be *too* difficult. Might also be worth having 
 `static assert(__traits(compiles, expr))`, without an explicit message 
 argument, print the returned message.


The problem with this is that your constraint then has to become:

```d
foo(alias bar) if (__traits(compiles, 
exprUsingBar).hasOnlyImplementationIssues)
```
That `hasOnlyImplementationIssues` is going to be messy, difficult to 
implement, and basically redoing all that the compiler is already doing. 
Plus, if it's just a string, it might have to change with compiler 
versions. Essentially, changing error messages becomes a breaking change.

-Steve
Feb 03 2023
next sibling parent Paul Backus <snarwin gmail.com> writes:
On Friday, 3 February 2023 at 20:20:48 UTC, Steven Schveighoffer 
wrote:

 The problem with this is that your constraint then has to 
 become:

 ```d
 foo(alias bar) if (__traits(compiles, 
 exprUsingBar).hasOnlyImplementationIssues)
 ```
 That `hasOnlyImplementationIssues` is going to be messy, 
 difficult to implement, and basically redoing all that the 
 compiler is already doing. Plus, if it's just a string, it 
 might have to change with compiler versions. Essentially, 
 changing error messages becomes a breaking change.


Well, it has to become this if you specifically want the 
*semantics* of `__traits(callable)`. If you just want to do a 
better job of surfacing error messages, which is what the 
original post focused on, this is unnecessary.

Either way, even if the semantics of `__traits(callable)` are 
desirable on their own merits, I think it is worth improving 
`__traits(compiles)`, because (1) it is widely used in existing 
code, and (2) sometimes `__traits(compiles)` is actually what you 
want.
Feb 03 2023
prev sibling parent Salih Dincer <salihdb hotmail.com> writes:
On Friday, 3 February 2023 at 20:20:48 UTC, Steven Schveighoffer 
wrote:

 
 The problem with this is that your constraint then has to 
 become:

 ```d
 foo(alias bar) if (__traits(compiles, 
 exprUsingBar).hasOnlyImplementationIssues)
 ```
 That `hasOnlyImplementationIssues` is going to be messy, 
 difficult to implement, and basically redoing all that the 
 compiler is already doing. Plus, if it's just a string, it 
 might have to change with compiler versions. Essentially, 
 changing error messages becomes a breaking change.


I'm trying to understand their, but neither I will understand 
their nor they will understand me!  However, they can see that I 
am struggling with the following non-simple example for sink 
their differences:

```d
// line 2:
auto foo(E)(E value = E.min)
{
    auto a = E.min; /*
    auto a = E.max; //*/
// line 7:
    assert(a == value);
    return 0;
}

auto bar(alias func)(int b = 0)
if (__traits( compiles, func(1) ))
{
   assert(func(b - 1));
   return 0;
}

void main()
{
   enum Eco { False, True }

   Eco eco;
   assert(eco == Eco.min);
//foo(1);    //core.exception.AssertError  source_file.d(7): 
Assertion failure
#line 1
   foo!Eco; // okay

//foo(++eco);//core.exception.AssertError  source_file.d(7): 
Assertion failure
#line 2
   //bar!foo; //core.exception.AssertError  source_file.d(7): 
Assertion failure

   alias fn = function int (int a) {
     import std.stdio;
     "funy a = ".writeln(a);
     return 0;
   };

   bar!fn(42); /* okay: "funy a = 41
                * core.exception.AssertError
                *  source_file.d(14): Assertion failure
                */

}
```
I love and appreciate you all. There is no my question, but I 
don't understand what is being mentioned in this discussion. 
Because I see that doing constraint with __traits is useless...

I can not see! 😀

SDB 79
Feb 03 2023
prev sibling parent FeepingCreature <feepingcreature gmail.com> writes:
On Friday, 3 February 2023 at 12:43:54 UTC, Nick Treleaven wrote:

 When a constraint for an alias parameter checks that calling it 
 compiles, it instantiates any function literal template. That 
 triggers any latent bugs in the user's function literal that 
 couldn't be detected before IFTI instantiates it. Those bugs 
 then cause the constraint to fail, because the literal does not 
 compile. The constraint then removes the otherwise matching 
 template from the list of candidates to use.
 ```d
 void f(alias a)() if (is(typeof(a()))) {}

 void main()
 {
     f!(x => blarg);
 }
 ```
 https://issues.dlang.org/show_bug.cgi?id=11907
 https://issues.dlang.org/show_bug.cgi?id=14217
 ...
 What if there was a new trait to solve this? Suppose 
 `__traits(callable, expr, args)`, that just does IFTI (if 
 `expr` is a function template) and then checks that the 
 parameter list of `expr` accepts `args`. It does not check that 
 `expr(args)` actually compiles, it ignores any errors in the 
 body of `expr`.


Fully agreed. This is one of the biggest "simple" inadequacies in 
D's lambda-based metaprogramming. See also slide 9 from my 2020 
DConf presentation http://dconf.org/2020/online/slides/mathis.odp 
"__traits(compiles) is Satan, the great Deceiver".
Feb 09 2023