www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - CTFE, string mixins & code generation

reply Jan =?UTF-8?B?SMO2bmln?= <hrominium gmail.com> writes:
I am looking for a detailed explanation or showcase regarding 
CTFE and string mixins.
I want to play with D a little bit regarding code generation.
I would like to have a pseudo-AST, consisting of a few classes, 
to represent some calculation. Think of a loop, some statements, 
and expressions.

To do that, I want to be certain that this AST can be computed 
and generated during compile-time, with CTFE and string mixins.

Are there limitations regarding CTFE (GC, global variables, 
static variables, templates, ....)?
Are there any limitations regarding string mixins (which are not 
already included int the CTFE limitations)?
Jan 24 2020
next sibling parent reply Marco de Wild <mdwild sogyo.nl> writes:
On Friday, 24 January 2020 at 16:21:48 UTC, Jan Hönig wrote:

 I am looking for a detailed explanation or showcase regarding 
 CTFE and string mixins.
 I want to play with D a little bit regarding code generation.
 I would like to have a pseudo-AST, consisting of a few classes, 
 to represent some calculation. Think of a loop, some 
 statements, and expressions.

 To do that, I want to be certain that this AST can be computed 
 and generated during compile-time, with CTFE and string mixins.

 Are there limitations regarding CTFE (GC, global variables, 
 static variables, templates, ....)?
 Are there any limitations regarding string mixins (which are 
 not already included int the CTFE limitations)?


For CTFE: functions should be pure. Therefore you cannot use 
global or static variables. Constants (enums) are perfectly fine 
to use though. I don't know the state of the GC and CTFE. I 
recall that there might be some complexity when using the `new` 
keyword.

https://dlang.org/spec/function.html#interpretation

Basically the only limitation of string mixins is that a single 
string should evaluate to valid D code, opposed to C macros. So
int y mixin("= 6");
doesn't compile, while
int y = mixin("6");
or
mixin("int y = 6;");
does. You can use CTFE to compose the string.
Jan 24 2020
parent Jan =?UTF-8?B?SMO2bmln?= <hrominium gmail.com> writes:
On Friday, 24 January 2020 at 16:59:53 UTC, Marco de Wild wrote:

 For CTFE: functions should be pure. Therefore you cannot use 
 global or static variables. Constants (enums) are perfectly 
 fine to use though. I don't know the state of the GC and CTFE. 
 I recall that there might be some complexity when using the 
 `new` keyword.

 https://dlang.org/spec/function.html#interpretation

 Basically the only limitation of string mixins is that a single 
 string should evaluate to valid D code, opposed to C macros. So
 int y mixin("= 6");
 doesn't compile, while
 int y = mixin("6");
 or
 mixin("int y = 6;");
 does. You can use CTFE to compose the string.


Ok, so mixins are really easy.
CTFE is the hard pard.
The link that you send me, i could not find it (or i haven't 
tried hard enough). Thanks!
Jan 24 2020
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
 next sibling parent reply Jacob Carlborg <doob me.com>  writes:


On 2020-01-24 19:43, H. S. Teoh wrote:


 (enums cannot take AA's or
 class objects as values, also, once assigned they are immutable).


AA enums work. Class objects kind of work. One can use static 
const/immutable instead. The following snippet compiles:

class A
{
     int a = 3;
}

const bar = new A;
enum a = bar.a;

enum foo = [1:2];

void main()
{
}

-- 
/Jacob Carlborg



 Jan 24 2020



  parent  "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Fri, Jan 24, 2020 at 08:26:50PM +0100, Jacob Carlborg via
Digitalmars-d-learn wrote:

 On 2020-01-24 19:43, H. S. Teoh wrote:
 

 (enums cannot take AA's or class objects as values, also, once
 assigned they are immutable).


 
 AA enums work.


Ah you're right, it's statically-constructed runtime AA's that don't
work. Mea culpa.



 Class objects kind of work. One can use static const/immutable
 instead. The following snippet compiles:
 
 class A
 {
     int a = 3;
 }
 
 const bar = new A;
 enum a = bar.a;


[...]

Wow.  I'm actually surprised that works!  That's pretty cool.


T

-- 
Almost all proofs have bugs, but almost all theorems are true. -- Paul Pedersen



 Jan 24 2020



prev sibling  parent  Jan =?UTF-8?B?SMO2bmln?= <hrominium gmail.com>  writes:


On Friday, 24 January 2020 at 18:43:14 UTC, H. S. Teoh wrote:

 CTFE in general cannot use global variables.  Any state you 
 need must be created inside a CTFE function, and accessed from 
 within that calling context.  You *can* assign values produced 
 by CTFE to compile-time symbols via 'enum', but there are 
 limitations (enums cannot take AA's or class objects as values, 
 also, once assigned they are immutable).

 As long as you can produce a string (via CTFE or otherwise) 
 whose value is known at compile-time and represents valid D 
 code, it's fair game for string mixins.  However, be aware that 
 there might be scoping and order of declaration issues (they 
 should be rare, but you might perchance run into them).


So as long as i don't have some side effects (aka global state), 
i am good to go with anything. Only trouble might be the output. 
But my output has to be strings, so i am good to go.


 What is your use case?  Maybe describing a bit more details 
 will help us help you.


In the area of my research, we do quite a lot of code generation.
We have some kind of DSL (internal or external) and we generate 
performant code.
Usually we have some "generation" language (e.g. Python) and some 
target language (e.g. C++). The project of my colleague (and my 
master's thesis) is pystencils: 
https://pypi.org/project/pystencils/
A stencil code generator in Python which produces C/C++ code.
It doesn't need to be in Python it can be anything (another 
colleague does something similar in Scala)
Of course there are other approaches. If you have a language 
which performs well on its own, you don't need to know 2 
languages (source and target). There are languages like Common 
Lisp, which have strong (well in case of common lisp best) 
macros, and are still fast enough.
D might be another candidate.
I am just curious and wanted to toy with it a little.


 P.S. If you want to know more about what CTFE / templates can 
 or cannot do, and how they interact, you might find this 
 article helpful:

 	https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time


This article is super helpful, let me reed it and play with it a 
little, before you sacrifice your time more with me :)
I'll come back with more questions soon enough.



 Jan 24 2020



prev sibling  parent  Jesse Phillips <Jesse.K.Phillips+D gmail.com>  writes:


On Friday, 24 January 2020 at 16:21:48 UTC, Jan Hönig wrote:

 I am looking for a detailed explanation or showcase regarding 
 CTFE and string mixins.
 I want to play with D a little bit regarding code generation.
 I would like to have a pseudo-AST, consisting of a few classes, 
 to represent some calculation. Think of a loop, some 
 statements, and expressions.

 To do that, I want to be certain that this AST can be computed 
 and generated during compile-time, with CTFE and string mixins.

 Are there limitations regarding CTFE (GC, global variables, 
 static variables, templates, ....)?
 Are there any limitations regarding string mixins (which are 
 not already included int the CTFE limitations)?


Back in the day I had written CTFE unittests for Protobuf 
generation.

https://github.com/JesseKPhillips/ProtocolBuffer/blob/master/source/dprotobuf/generator/dlang.d#L740

IIRC the mixin was because it needed to compile in a D1 compiler.

I don't recall why I needed these.

https://github.com/JesseKPhillips/ProtocolBuffer/blob/master/source/dprotobuf/generator/dlang.d#L20



 Jan 24 2020