www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Is it possible to obtain textual representation of an arbitary code?

reply Oleksii Skidan <al.skidan gmail.com> writes:
Hi,

I wonder if it's possible to convert D language code into a 
string at compile time? C/C++ preprocessor has this feature 
built-in: `#` preprocessing operator allows converting a macro 
argument into a string constant. See the following code snippet 
for example:

```cplusplus
#define ASSERT(EXPR) some_function((EXPR), #EXPR)

// Elsewhere in the code:
void some_function(bool const condition, char const* const expr) {
   if (!condition) {
      printf("Assertion failed for: %s\n", expr);
   }
}

// Usage:
ASSERT(a == b); // Will print "Assertion failed for: a == b"
```

I could imagine a mixin-based solution in D:
```d
// Usage:
ASSERT!"a == b";
```
But it seems a bit alien to me. First of all, it kind of 
stringly-typed one. Secondly, neither IDEs nor advanced text 
editors are able to figure out that the string contains actual D 
code, and so neither syntax highlighting nor code assistance work 
with this approach.

BR,
--
Oleksii
Jan 26
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, January 26, 2018 11:18:21 Oleksii Skidan via Digitalmars-d-learn 
wrote:
 Hi,

 I wonder if it's possible to convert D language code into a
 string at compile time? C/C++ preprocessor has this feature
 built-in: `#` preprocessing operator allows converting a macro
 argument into a string constant. See the following code snippet
 for example:

 ```cplusplus
 #define ASSERT(EXPR) some_function((EXPR), #EXPR)

 // Elsewhere in the code:
 void some_function(bool const condition, char const* const expr) {
    if (!condition) {
       printf("Assertion failed for: %s\n", expr);
    }
 }

 // Usage:
 ASSERT(a == b); // Will print "Assertion failed for: a == b"
 ```

 I could imagine a mixin-based solution in D:
 ```d
 // Usage:
 ASSERT!"a == b";
 ```
 But it seems a bit alien to me. First of all, it kind of
 stringly-typed one. Secondly, neither IDEs nor advanced text
 editors are able to figure out that the string contains actual D
 code, and so neither syntax highlighting nor code assistance work
 with this approach.
You can use stringof on symbols to get their string representation, but you can't get sections of code that way. D makes it very easy to turn strings into code using mixin statements, but it doesn't provide ways to take arbitrary code and turn it into strings. And in many cases, the compiler wouldn't necessarily have access to the code anyway, just like you wouldn't in C/C++. C/C++ can do it with macros but not arbitrary code, and D doesn't have the equivalent of C/C++ macros. Arguably the closest thing that D has to C/C++ macros is the ability to mixin strings, and if you're mixing in a string, you already have the code as a string. And yes, if you use a lot of metaprogramming stuff like string mixins, then you're not going to have the code to see in your text editor, but that's what happens when you generate code at compile time rather than having the code sit in a file. However, if you want syntax highlighting for a string literal that contains code, you can use q{} to contain the string literal rather that "" or `` - e.g. q{auto i = 42;} instead of "auto i = 42;", then most text editors will highlight the string as if it were just code. - Jonathan M Davis
Jan 26
prev sibling next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Friday, 26 January 2018 at 11:18:21 UTC, Oleksii Skidan wrote:

 I could imagine a mixin-based solution in D:
 ```d
 // Usage:
 ASSERT!"a == b";
 ```
 But it seems a bit alien to me. First of all, it kind of 
 stringly-typed one. Secondly, neither IDEs nor advanced text 
 editors are able to figure out that the string contains actual 
 D code, and so neither syntax highlighting nor code assistance 
 work with this approach.
Token strings are intended for this and editors *should* highlight them (don't know if any currently do): https://dlang.org/spec/lex.html#token_strings
Jan 26
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, January 26, 2018 11:32:42 Mike Parker via Digitalmars-d-learn 
wrote:
 On Friday, 26 January 2018 at 11:18:21 UTC, Oleksii Skidan wrote:
 I could imagine a mixin-based solution in D:
 ```d
 // Usage:
 ASSERT!"a == b";
 ```
 But it seems a bit alien to me. First of all, it kind of
 stringly-typed one. Secondly, neither IDEs nor advanced text
 editors are able to figure out that the string contains actual
 D code, and so neither syntax highlighting nor code assistance
 work with this approach.
Token strings are intended for this and editors *should* highlight them (don't know if any currently do): https://dlang.org/spec/lex.html#token_strings
So that's what they're called. I can never remember (though I rarely use them either, since I actually prefer that strings be highlighted as strings and not code even if they contain code). vim definitely highlights them as code, and I would expect most editors that don't understand them to highlight them as if they were code, since q{} looks like code. If anything, it would take a text editor that understood D quite well to highlight it as a string (though really, no editor should be doing that, since the main point of using token strings is for them to be highlighted as code). - Jonathan M Davis
Jan 26
prev sibling next sibling parent reply Oleksii Skidan <al.skidan gmail.com> writes:
On Friday, 26 January 2018 at 11:32:42 UTC, Mike Parker wrote:
 On Friday, 26 January 2018 at 11:18:21 UTC, Oleksii Skidan 
 wrote:

 I could imagine a mixin-based solution in D:
 ```d
 // Usage:
 ASSERT!"a == b";
 ```
 But it seems a bit alien to me. First of all, it kind of 
 stringly-typed one. Secondly, neither IDEs nor advanced text 
 editors are able to figure out that the string contains actual 
 D code, and so neither syntax highlighting nor code assistance 
 work with this approach.
Token strings are intended for this and editors *should* highlight them (don't know if any currently do): https://dlang.org/spec/lex.html#token_strings
Seems like I have to add some context into this conversation: I'm writing a poor man's testing framework, since it's the best and easiest way to learn D ;-) I'm trying to achieve something similar to `Catch2``REQUIRE`macro. To be honest, I did not know about toking strings until today, and I don't know D much. Here's what I came up with so far: ```d string require(string expr)(string file = __FILE__, int line = __LINE__) { import std.array, std.conv; return q{ if (!($expr)) { import std.stdio; writeln("Test failed ", `$file`, ":", $line, "\n", " Expected: `", `$expr`, "` to be `true`.\n"); } }.replace("$expr", expr) .replace("$file", file) .replace("$line", to!string(line)); } ``` That code snippet uses token strings to compose an if statement that basically checks whether the given condition holds. That looks okay-ish to me, the usage of that function is not pretty though: ```d unittest { mixin(require!q{false}); // This test will fail. } ``` It would be awesome if I could write something like the this instead: ```d unittest { require!q{false}; } ``` At first glance it seems like I could have moved the `mixin` statement into the `require` function itself, but that would not really work. Consider the following snippet: ```d unittest { float value = 3f; require!q{value == 3f}; // This line won't compile. } ``` That code won't even compile, since `value` exists in `unittest` scope, which is not visible to the `require` function. -- Oleksii
Jan 26
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, January 26, 2018 12:30:03 Oleksii Skidan via Digitalmars-d-learn 
wrote:
 On Friday, 26 January 2018 at 11:32:42 UTC, Mike Parker wrote:
 On Friday, 26 January 2018 at 11:18:21 UTC, Oleksii Skidan

 wrote:
 I could imagine a mixin-based solution in D:
 ```d
 // Usage:
 ASSERT!"a == b";
 ```
 But it seems a bit alien to me. First of all, it kind of
 stringly-typed one. Secondly, neither IDEs nor advanced text
 editors are able to figure out that the string contains actual
 D code, and so neither syntax highlighting nor code assistance
 work with this approach.
Token strings are intended for this and editors *should* highlight them (don't know if any currently do): https://dlang.org/spec/lex.html#token_strings
Seems like I have to add some context into this conversation: I'm writing a poor man's testing framework, since it's the best and easiest way to learn D ;-) I'm trying to achieve something similar to `Catch2``REQUIRE`macro. To be honest, I did not know about toking strings until today, and I don't know D much. Here's what I came up with so far: ```d string require(string expr)(string file = __FILE__, int line = __LINE__) { import std.array, std.conv; return q{ if (!($expr)) { import std.stdio; writeln("Test failed ", `$file`, ":", $line, "\n", " Expected: `", `$expr`, "` to be `true`.\n"); } }.replace("$expr", expr) .replace("$file", file) .replace("$line", to!string(line)); } ``` That code snippet uses token strings to compose an if statement that basically checks whether the given condition holds. That looks okay-ish to me, the usage of that function is not pretty though: ```d unittest { mixin(require!q{false}); // This test will fail. } ``` It would be awesome if I could write something like the this instead: ```d unittest { require!q{false}; } ``` At first glance it seems like I could have moved the `mixin` statement into the `require` function itself, but that would not really work. Consider the following snippet: ```d unittest { float value = 3f; require!q{value == 3f}; // This line won't compile. } ``` That code won't even compile, since `value` exists in `unittest` scope, which is not visible to the `require` function.
Why are you using strings for any of this? Printing out the expression is kind of pointless. If you have the file and line number (which an AssertError will give you), then you know where the failure is, and you can see the expression. All of this extra machinery is just going to increase your compile times for no benefit. So, what you're doing here is objectively worse than just using assertions. There might be some value if you had something like assertEqual(lhs, rhs); and then on failure, you printed the values that were being compared, since that's not necessarily information that's in the code, but the expressions themselves _are_ already in the code, so printing them out doesn't help any. But even if you have helper functions that take the values separately so that they can be printed, in my experience, the extra template instantiations required to use helper functions like that everywhere in unit tests increases the compilation times (and memory required) enough that it's not worth it, especially when you consider that once the tests are passing, all of that extra machinery does you no good whatsoever. Ultimately, it just costs less to temporarily make an adjustment to the test and rerun it if you need more information. If you don't think that simply using assertions for unit tests is good enough, then I'd suggest that you look at https://code.dlang.org/packages/unit-threaded - Jonathan M Davis
Jan 26
next sibling parent reply Seb <seb wilzba.ch> writes:
On Friday, 26 January 2018 at 13:05:26 UTC, Jonathan M Davis 
wrote:
 If you don't think that simply using assertions for unit tests 
 is good enough, then I'd suggest that you look at 
 https://code.dlang.org/packages/unit-threaded
There's also https://code.dlang.org/packages/fluent-asserts which shows detailed error messages.
Jan 26
parent Oleksii Skidan <al.skidan gmail.com> writes:
On Friday, 26 January 2018 at 13:25:12 UTC, Seb wrote:
 On Friday, 26 January 2018 at 13:05:26 UTC, Jonathan M Davis 
 wrote:
 If you don't think that simply using assertions for unit tests 
 is good enough, then I'd suggest that you look at 
 https://code.dlang.org/packages/unit-threaded
There's also https://code.dlang.org/packages/fluent-asserts which shows detailed error messages.
Thanks, That one look great! I like how it prints the source code of the test, so that you have the context within which the test failed.
Jan 26
prev sibling next sibling parent reply Oleksii Skidan <al.skidan gmail.com> writes:
On Friday, 26 January 2018 at 13:05:26 UTC, Jonathan M Davis 
wrote:
 Why are you using strings for any of this? Printing out the 
 expression is kind of pointless. If you have the file and line 
 number (which an AssertError will give you), then you know 
 where the failure is, and you can see the expression. All of 
 this extra machinery is just going to increase your compile 
 times for no benefit. So, what you're doing here is objectively 
 worse than just using assertions.

 There might be some value if you had something like

 assertEqual(lhs, rhs);

 and then on failure, you printed the values that were being 
 compared, since that's not necessarily information that's in 
 the code, but the expressions themselves _are_ already in the 
 code, so printing them out doesn't help any.
That's actually what I have: ``` // Usage: // mixin(requireEq!q{expected, actual}); // Checks whether the `actual` value is equal to `reauired`. string requireEq(string expr)(string file = __FILE__, int line = __LINE__) { import std.array, std.conv, std.string; auto parts = expr.split(","); return q{ { bool equal = false; static if (__traits(isFloating, $expected) || __traits(isFloating, $actual)) { import std.math; equal = approxEqual($expected, $actual); } else { equal = $expected == $actual; } if (!equal) { import std.stdio, std.conv; writeln("Test failed ", `$file`, ":", $line, "\n", " Expected `", `$actual`, "` to be `", to!string($expected), "`, but got `", to!string($actual), "`\n"); } } }.replace("$expected", parts[0].strip()) .replace("$actual", parts[1].strip()) .replace("$file", file) .replace("$line", to!string(line)); } ``` The sample code I posted before, was way much simpler than this.
 But even if you have helper functions that take the values 
 separately so that they can be printed, in my experience, the 
 extra template instantiations required to use helper functions 
 like that everywhere in unit tests increases the compilation 
 times (and memory required) enough that it's not worth it, 
 especially when you consider that once the tests are passing, 
 all of that extra machinery does you no good whatsoever. 
 Ultimately, it just costs less to temporarily make an 
 adjustment to the test and rerun it if you need more 
 information.
I won't argue against asserts. I agree with your point: they are fine for simple cases, and they are definitely faster to compile than any other framework. My ultimate goal is to implement a BDD framework, so that I could write tests in the most productive (for myself) way. I'm not aiming for a production-ready quality, a toy framework that suits my needs would be just fine.
 If you don't think that simply using assertions for unit tests 
 is good enough, then I'd suggest that you look at 
 https://code.dlang.org/packages/unit-threaded
Thanks, I'll look at it. Also, I have a vague idea that aliases may be the key to the desired functionality. BR -- Oleksii
Jan 26
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, January 26, 2018 14:23:20 Oleksii Skidan via Digitalmars-d-learn 
wrote:
 On Friday, 26 January 2018 at 13:05:26 UTC, Jonathan M Davis

 wrote:
 Why are you using strings for any of this? Printing out the
 expression is kind of pointless. If you have the file and line
 number (which an AssertError will give you), then you know
 where the failure is, and you can see the expression. All of
 this extra machinery is just going to increase your compile
 times for no benefit. So, what you're doing here is objectively
 worse than just using assertions.

 There might be some value if you had something like

 assertEqual(lhs, rhs);

 and then on failure, you printed the values that were being
 compared, since that's not necessarily information that's in
 the code, but the expressions themselves _are_ already in the
 code, so printing them out doesn't help any.
That's actually what I have: ``` // Usage: // mixin(requireEq!q{expected, actual}); // Checks whether the `actual` value is equal to `reauired`. string requireEq(string expr)(string file = __FILE__, int line = __LINE__) { import std.array, std.conv, std.string; auto parts = expr.split(","); return q{ { bool equal = false; static if (__traits(isFloating, $expected) || __traits(isFloating, $actual)) { import std.math; equal = approxEqual($expected, $actual); } else { equal = $expected == $actual; } if (!equal) { import std.stdio, std.conv; writeln("Test failed ", `$file`, ":", $line, "\n", " Expected `", `$actual`, "` to be `", to!string($expected), "`, but got `", to!string($actual), "`\n"); } } }.replace("$expected", parts[0].strip()) .replace("$actual", parts[1].strip()) .replace("$file", file) .replace("$line", to!string(line)); } ``` The sample code I posted before, was way much simpler than this.
If that's what you want, there's still no reason to use strings. Just take the two arguments separately as normal function arguments and print their values when there's a failure. Why are you using strings and mixins for any of this? You could do something like void assertEqual(T, U)(T lhs, U rhs, string file = __FILE__, size_t line = __LINE__) { enforce!AssertError(lhs == rhs, format("assertEqual failed. lhs: %s, rhs: %s", lhs, rhs), file, line); } - Jonathan M Davis
Jan 26
prev sibling parent Oleksii Skidan <al.skidan gmail.com> writes:
On Friday, 26 January 2018 at 13:05:26 UTC, Jonathan M Davis 
wrote:
 On Friday, January 26, 2018 12:30:03 Oleksii Skidan via 
 Digitalmars-d-learn wrote:
 On Friday, 26 January 2018 at 11:32:42 UTC, Mike Parker wrote:
 On Friday, 26 January 2018 at 11:18:21 UTC, Oleksii Skidan

 wrote:
 [...]
Token strings are intended for this and editors *should* highlight them (don't know if any currently do): https://dlang.org/spec/lex.html#token_strings
Seems like I have to add some context into this conversation: I'm writing a poor man's testing framework, since it's the best and easiest way to learn D ;-) I'm trying to achieve something similar to `Catch2``REQUIRE`macro. To be honest, I did not know about toking strings until today, and I don't know D much. Here's what I came up with so far: ```d string require(string expr)(string file = __FILE__, int line = __LINE__) { import std.array, std.conv; return q{ if (!($expr)) { import std.stdio; writeln("Test failed ", `$file`, ":", $line, "\n", " Expected: `", `$expr`, "` to be `true`.\n"); } }.replace("$expr", expr) .replace("$file", file) .replace("$line", to!string(line)); } ``` That code snippet uses token strings to compose an if statement that basically checks whether the given condition holds. That looks okay-ish to me, the usage of that function is not pretty though: ```d unittest { mixin(require!q{false}); // This test will fail. } ``` It would be awesome if I could write something like the this instead: ```d unittest { require!q{false}; } ``` At first glance it seems like I could have moved the `mixin` statement into the `require` function itself, but that would not really work. Consider the following snippet: ```d unittest { float value = 3f; require!q{value == 3f}; // This line won't compile. } ``` That code won't even compile, since `value` exists in `unittest` scope, which is not visible to the `require` function.
Why are you using strings for any of this? Printing out the expression is kind of pointless. If you have the file and line number (which an AssertError will give you), then you know where the failure is, and you can see the expression. All of this extra machinery is just going to increase your compile times for no benefit. So, what you're doing here is objectively worse than just using assertions. There might be some value if you had something like assertEqual(lhs, rhs); and then on failure, you printed the values that were being compared, since that's not necessarily information that's in the code, but the expressions themselves _are_ already in the code, so printing them out doesn't help any. But even if you have helper functions that take the values separately so that they can be printed, in my experience, the extra template instantiations required to use helper functions like that everywhere in unit tests increases the compilation times (and memory required) enough that it's not worth it, especially when you consider that once the tests are passing, all of that extra machinery does you no good whatsoever. Ultimately, it just costs less to temporarily make an adjustment to the test and rerun it if you need more information. If you don't think that simply using assertions for unit tests is good enough, then I'd suggest that you look at https://code.dlang.org/packages/unit-threaded - Jonathan M Davis
I've just realized that I can actually make the test code more pleasant if I use string concatenation. For example, this test: ``` unittest { import testing; { Particle a = Particle(0.9, 0); Particle b = Particle(2.1, 0); Constraint c = Constraint(a, b); c.distance = 1f; c.solve(); mixin(requireEq!q{1f, a.x}); mixin(requireEq!q{0f, a.y}); mixin(requireEq!q{2f, b.x}); mixin(requireEq!q{0f, b.y}); } } ``` could be written as: ``` unittest { import testing; { Particle a = Particle(0.9, 0); Particle b = Particle(2.1, 0); Constraint c = Constraint(a, b); c.distance = 1f; c.solve(); mixin( requireEq!q{1f, a.x} ~ requireEq!q{0f, a.y} ~ requireEq!q{2f, b.x} ~ requireEq!q{0f, b.y} ); } } ``` That code looks a little bit unusual, but I guess I can get used to it. Seems like I can write a test scenario per mixin.
Jan 26
prev sibling parent Basile B. <b2.temp gmx.com> writes:
On Friday, 26 January 2018 at 11:32:42 UTC, Mike Parker wrote:
 On Friday, 26 January 2018 at 11:18:21 UTC, Oleksii Skidan 
 wrote:

 I could imagine a mixin-based solution in D:
 ```d
 // Usage:
 ASSERT!"a == b";
 ```
 But it seems a bit alien to me. First of all, it kind of 
 stringly-typed one. Secondly, neither IDEs nor advanced text 
 editors are able to figure out that the string contains actual 
 D code, and so neither syntax highlighting nor code assistance 
 work with this approach.
Token strings are intended for this and editors *should* highlight them (don't know if any currently do): https://dlang.org/spec/lex.html#token_strings
LOL, all do that: https://imgur.com/a/UoHpz.
Jan 26
prev sibling parent timotheecour <timothee.cour2 gmail.com> writes:
On Friday, 26 January 2018 at 11:18:21 UTC, Oleksii Skidan wrote:
 Hi,

 I wonder if it's possible to convert D language code into a 
 string at compile time? C/C++ preprocessor has this feature 
 built-in: `#` preprocessing operator allows converting a macro 
 argument into a string constant. See the following code snippet 
 for example:
See feature request I suggested here: https://forum.dlang.org/thread/mailman.2377.1516230845.9493.digital ars-d puremagic.com __ARGS__ : allow access to (stringified) arguments, as C's `#arg` macro
Jan 26