www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - CTFE Status 2

reply Stefan Koch <uplink.coder googlemail.com> writes:
Hi Guys, due to the old CTFE status thread getting to page 30, I 
am now starting a new one.

First let me summerize which features are currently working:
In order of date, the latest features come first.

- fixed continue and break for DoWhileStatements
- non-toplevel Function Pointers as arguments
- basic function pointers
--- phobos, druntime & dmd unit-tests pass ----
- Method Calls (now range foreach will work)
- 1-level Pointers
- recursive function calls
- fixed continue and break for ForStatements (includes foreach 
which is lowered)
- basic Function Calls
- static immutable globals
- fixed continue & break for WhileStatemens
- 1d array literals
- ternary expressions ? :
- struct literals.
- basic struct support.
- Slice and Array boundschecks
- 1d array and slice indexing
- do while statements
- break and continue support
- gotos and labels
- foreach over arrays
- for statements
- integer math
.... (and I few a forgot to list)

unsupported features are
- anything with strings (due to missing unicode handling)
- slicing & concat (due to missing memeory-manager)
- floating point math
- classes (due to missing vtbl and ref support)
- exceptions (due to missing stack-unwinding support and 
side-band channels)
- more complex structs i.e. with struct-members (due to 
incomplete type-handling)
-----------

Currently I am having trouble caused by a bug in dmds inliner 
that only happens on when dmd is compiled as a 32bit executable 
until I have  isolated / fixed this development is slowed down.

--
I hope this thread is informative and will continue to be that 
way.

Cheers,
Stefan (aka UplinkCoder)
Feb 16
next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 Cheers,
 Stefan (aka UplinkCoder)
Stupid question: is your online alias a reference to the game Uplink?
Feb 16
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:19:14 UTC, Jack Stouffer 
wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 Cheers,
 Stefan (aka UplinkCoder)
Stupid question: is your online alias a reference to the game Uplink?
Yes it is. Uplink Hackers Elite is a great game.
Feb 16
prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Feb 16, 2017 at 09:05:51PM +0000, Stefan Koch via Digitalmars-d wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, I am now
 starting a new one.
 
 First let me summerize which features are currently working:
 In order of date, the latest features come first.
 
 - fixed continue and break for DoWhileStatements
 - non-toplevel Function Pointers as arguments
 - basic function pointers
 --- phobos, druntime & dmd unit-tests pass ----
[...] This is great! The fact that you got it to the point unittests pass is awesome. Can't wait for the new CTFE engine to be released! Do you have any estimate as to when it will be mergeable into master? T -- There are three kinds of people in the world: those who can count, and those who can't.
Feb 16
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 23:16:34 UTC, H. S. Teoh wrote:
 On Thu, Feb 16, 2017 at 09:05:51PM +0000, Stefan Koch via 
 Digitalmars-d wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.
 
 First let me summerize which features are currently working: 
 In order of date, the latest features come first.
 
 - fixed continue and break for DoWhileStatements
 - non-toplevel Function Pointers as arguments
 - basic function pointers
 --- phobos, druntime & dmd unit-tests pass ----
[...] This is great! The fact that you got it to the point unittests pass is awesome. Can't wait for the new CTFE engine to be released! Do you have any estimate as to when it will be mergeable into master? T
I estimate that newCTFE 1.0 as outlined in the vision document will be ready around April. This estimate is a lower bound as it assumes that things go smoothly, and the past 6 months have shown that the CTFE-work does not go smoothly most of the time. As an upper bound: I have the personal goal to get it merged before DConf. At this point most parts of the actual engine are done, the remaining work mostly involves either making the runtime ctfeable or building runtime-feature support into dmd. These things are outside of my domain knowledge and therefore take much more time then they should.
Feb 16
prev sibling next sibling parent Andrea Fontana <nospam example.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [...]
 I hope this thread is informative and will continue to be that 
 way.

 Cheers,
 Stefan (aka UplinkCoder)
Yes, it is!
Feb 17
prev sibling next sibling parent Lurker <lurker gmail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 ...
 --
 I hope this thread is informative and will continue to be that 
 way.

 Cheers,
 Stefan (aka UplinkCoder)
Thanks Stefan for your work, and for documenting it. Very exciting to read the progress! I wish more people were as devoted as you are! Keep up the good work!
Feb 17
prev sibling next sibling parent reply Temtaime <temtaime gmail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 First let me summerize which features are currently working:
 In order of date, the latest features come first.

 - fixed continue and break for DoWhileStatements
 - non-toplevel Function Pointers as arguments
 - basic function pointers
 --- phobos, druntime & dmd unit-tests pass ----
 - Method Calls (now range foreach will work)
 - 1-level Pointers
 - recursive function calls
 - fixed continue and break for ForStatements (includes foreach 
 which is lowered)
 - basic Function Calls
 - static immutable globals
 - fixed continue & break for WhileStatemens
 - 1d array literals
 - ternary expressions ? :
 - struct literals.
 - basic struct support.
 - Slice and Array boundschecks
 - 1d array and slice indexing
 - do while statements
 - break and continue support
 - gotos and labels
 - foreach over arrays
 - for statements
 - integer math
 .... (and I few a forgot to list)

 unsupported features are
 - anything with strings (due to missing unicode handling)
 - slicing & concat (due to missing memeory-manager)
 - floating point math
 - classes (due to missing vtbl and ref support)
 - exceptions (due to missing stack-unwinding support and 
 side-band channels)
 - more complex structs i.e. with struct-members (due to 
 incomplete type-handling)
 -----------

 Currently I am having trouble caused by a bug in dmds inliner 
 that only happens on when dmd is compiled as a 32bit executable 
 until I have  isolated / fixed this development is slowed down.

 --
 I hope this thread is informative and will continue to be that 
 way.

 Cheers,
 Stefan (aka UplinkCoder)
Just get LDC. Make it use JIT. And you'll get all the features working. Writing slow interpreter is ... wasting efforts.
Feb 17
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 17 February 2017 at 19:44:10 UTC, Temtaime wrote:
 Just get LDC.
 Make it use JIT.
 And you'll get all the features working.
 Writing slow interpreter is ... wasting efforts.
For your information LLVM takes about 5 milliseconds to start up, it also takes a lot of time to generate code even when completely unoptimized. For usual one-shot CTFE functions this leads to a _HUGE_ pessimisation of performance. For the quite common usecase of returning a literal out of function templates newCTFE takes UNDER a MICROsecond. There is no way LLVM could even come close no matter how many caching layers one adds.
Feb 17
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 2/17/17 8:44 PM, Temtaime wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 I hope this thread is informative and will continue to be that way.

 Cheers,
 Stefan (aka UplinkCoder)
Just get LDC. Make it use JIT. And you'll get all the features working. Writing slow interpreter is ... wasting efforts.
This is a common misconception. LLVM as AoT optimizing compiler is great, but for the JIT compiler it's actually far too slow at codegen to be a sensible choice. --- Dmitry Olshansky
Feb 17
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Friday, 17 February 2017 at 21:01:33 UTC, Dmitry Olshansky 
wrote:
 This is a common misconception. LLVM as AoT optimizing compiler 
 is great, but for the JIT compiler it's actually far too slow 
 at codegen to be a sensible choice.
I don't know how big of a misconception it is. The Dropbox effort on Pyston (LLVM JIT for Python) just recently shut down and nobody expressed a lot of surprise in the HN comments.
Feb 17
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 2/17/17 10:37 PM, jmh530 wrote:
 On Friday, 17 February 2017 at 21:01:33 UTC, Dmitry Olshansky wrote:
 This is a common misconception. LLVM as AoT optimizing compiler is
 great, but for the JIT compiler it's actually far too slow at codegen
 to be a sensible choice.
I don't know how big of a misconception it is. The Dropbox effort on Pyston (LLVM JIT for Python) just recently shut down and nobody expressed a lot of surprise in the HN comments.
It might be that HN just in general are pessimistic ;) w.r.t Pyston I've seen it coming since the very first blog post that described their approach to the problem. However I've seen lots of folks being quite enthusiastic about that effort. ---- Dmitry Olshansky
Feb 17
prev sibling next sibling parent reply Satoshi <satoshi rikarin.org> writes:
Will be possible to make precompiled headers and distribute it 
with static library for example?
Feb 18
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 18 February 2017 at 21:24:18 UTC, Satoshi wrote:
 Will be possible to make precompiled headers and distribute it 
 with static library for example?
anything is possible with ctfe and mixins and templates. certainly header generation as well, you can write your own system for it and do it as a build step, in case the built-in header generator is inadequate.
Feb 19
prev sibling next sibling parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]
Thank you for your continued work on this. I heavily rely on D's CTFE functionality and I try to read all your updates on it.
Feb 18
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 18 February 2017 at 22:40:44 UTC, Moritz Maxeiner 
wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]
Thank you for your continued work on this. I heavily rely on D's CTFE functionality and I try to read all your updates on it.
Hello Moritz, D's ctfe functionality is almost complete. This thread is not about ctfe as it is currently implemented. I am working on/writing about a faster re-implementation of ctfe. When my work is finished nothing will change functionality wise, it will just be much more efficient.
Feb 18
parent Moritz Maxeiner <moritz ucworks.org> writes:
On Sunday, 19 February 2017 at 01:52:07 UTC, Stefan Koch wrote:
 On Saturday, 18 February 2017 at 22:40:44 UTC, Moritz Maxeiner 
 wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 Hi Guys, due to the old CTFE status thread getting to page 
 30, I am now starting a new one.

 [...]
Thank you for your continued work on this. I heavily rely on D's CTFE functionality and I try to read all your updates on it.
Hello Moritz, D's ctfe functionality is almost complete. This thread is not about ctfe as it is currently implemented. I am working on/writing about a faster re-implementation of ctfe. When my work is finished nothing will change functionality wise, it will just be much more efficient.
Hello Stefan, my apologies if I wasn't clear: I'm aware that this isn't about adding anything new in terms of functionality, what I intended to imply with "heavily rely on" was that I use enough of it for it to be noticeable in compile time / memory consumption, which is why I'm very happy about potential speedup and/or memory consumption reduction in CTFE. One public example of that was llvm-d <2.0.
Feb 18
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 Currently I am having trouble caused by a bug in dmds inliner 
 that only happens on when dmd is compiled as a 32bit executable 
 until I have  isolated / fixed this development is slowed down.
I am slowly narrowing down on the source of this bug. Until it is fixed builds of newCTFE might not behave correctly, if they have been complied with -inline!
Feb 20
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 21 February 2017 at 01:57:21 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 Currently I am having trouble caused by a bug in dmds inliner 
 that only happens on when dmd is compiled as a 32bit 
 executable until I have  isolated / fixed this development is 
 slowed down.
I am slowly narrowing down on the source of this bug. Until it is fixed builds of newCTFE might not behave correctly, if they have been complied with -inline!
The culprit that introduced the bug seems to be : https://github.com/dlang/dmd/commit/cd01efb4dd5e32277cb156c3cc2b451bdcb68b52 however more testing is needed before I can be sure.
Feb 21
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 Currently I am having trouble caused by a bug in dmds inliner 
 that only happens on when dmd is compiled as a 32bit executable 
 until I have  isolated / fixed this development is slowed down.
This bug was Issue #17220 and is fixed ~master. Unitl the auto-tester is updated is updated workarounds have to suffice. In this case the workaround is padding a structure such that it's size in no longer 32.
Feb 24
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi Guys, ref variable handing is coming soon! (only int/uint for now)... Also I have suspicions that there is a bug in my ctfe_evaluator in which the first default arguments can overridden with the this pointer :) I will test "this" tomorrow :) Also I submitted my talk proposal a few hours ago; Of course it's about newCTFE. Cheers, Stefan
Feb 25
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [...]
Simple ref calls work now. Meaning the following code will compile :) uint sum(uint[] arr) { uint sum; foreach(uint i;0 .. cast(uint)arr.length) { addToSum(sum, arr[i]); } return sum; } void addToSum(ref uint sum, uint element) { sum = sum + element; // works now as well return ; } static assert([1,2,3,4,5].sum == 15);
Feb 27
parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 27 February 2017 at 23:48:13 UTC, Stefan Koch wrote:
 Simple ref calls work now.
 Meaning the following code will compile :)
Great work.
Feb 28
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi Guys, The implementation of ref parameters by itself was successful. However they hey have one bug, which is a consquence of newCTFE not touching dmd's state. They cannot modify an outer parameter. (where outer means that the parameter does not originate from a ctfe-evaluation further up the call stack) uint add8ret3(ref uint a) {a += 8; return 3;} pragma (msg, (uint outer) { outer += add8ret3(a); return outer;}(1)); // will print 4 pragma (msg, (uint outer) { uint inner = outer; inner += add8ret3(inner); return inner; }(1)); // will print 12 as expected. There are two options I could take: The first one is to disallow having refs of outer parameters taken. (and bailing out in this case) The second option is to implement the logic necessary to modify these expression nodes. Both options require me to detect if we have an outer ref parameter. but that should be easy to do. Cheers, Stefan
Mar 01
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 2 March 2017 at 03:09:56 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
 uint add8ret3(ref uint a) {a += 8; return 3;}

 pragma (msg, (uint outer) { outer += add8ret3(a); return 
 outer;}(1)); // will print 4
was supposed to say: pragma (msg, (uint outer) { outer += add8ret3(outer); return outer;}(1)); // will print 4
Mar 01
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 2 March 2017 at 03:09:56 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
Hi Guys, The implementation of ref parameters by itself was successful. However they hey have one bug, which is a consquence of newCTFE not touching dmd's state. They cannot modify an outer parameter. (where outer means that the parameter does not originate from a ctfe-evaluation further up the call stack) Cheers, Stefan
It turn out that this was not the problem. It was rather old code in the byte-code evaluator that dated from a time where parameters were not considered stack values. causing that code to copy the parameters into a temporary. When I removed the distinction between parameters and normal stack values I deleted the code that kept track of the parameters that were copied into temporaries. Since I forgot to remove the creation of those temporaries. We ended up with the situation described above.
Mar 01
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Time for an update. I am currently working on integrating 64bit values into codegen API. However, a backend may not have native 64bit registers or arithmetic (the x86/arm architectures come to mind) For those a common fallback is to be implemented such that not every architecture has to implement their 64bit arithmetic on their own. Also work is underway to finally support slicing, which is crucial to using phobos algorithms.
Mar 10
next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Mar 10, 2017 at 11:32:14AM +0000, Stefan Koch via Digitalmars-d wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Time for an update. I am currently working on integrating 64bit values into codegen API. However, a backend may not have native 64bit registers or arithmetic (the x86/arm architectures come to mind) For those a common fallback is to be implemented such that not every architecture has to implement their 64bit arithmetic on their own.
Makes sense to me.
 Also work is underway to finally support slicing, which is crucial to
 using phobos algorithms.
This is awesome news! Glad to hear there's steady progress being made on the new CTFE engine. Can't wait for it to be done and merged into master! There are so many awesome things I wanna do with CTFE that currently would be unacceptably slow. Can't wait to take D compile-time capabilities to a whole 'nother level! T -- "A one-question geek test. If you get the joke, you're a geek: Seen on a California license plate on a VW Beetle: 'FEATURE'..." -- Joshua D. Wachs - Natural Intelligence, Inc.
Mar 10
prev sibling next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 10 March 2017 at 11:32:14 UTC, Stefan Koch wrote:
 Also work is underway to finally support slicing, which is 
 crucial to using phobos algorithms.
Incredible diligence.
Mar 10
prev sibling parent reply Kagamin <spam here.lot> writes:
On Friday, 10 March 2017 at 11:32:14 UTC, Stefan Koch wrote:
 I am currently working on integrating 64bit values into codegen 
 API.
How can dmd generate 64-bit code if backend doesn't support 64bit values?
Mar 13
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 13 March 2017 at 11:53:59 UTC, Kagamin wrote:
 On Friday, 10 March 2017 at 11:32:14 UTC, Stefan Koch wrote:
 I am currently working on integrating 64bit values into 
 codegen API.
How can dmd generate 64-bit code if backend doesn't support 64bit values?
I was referring to my newCTFE Bytecode-codegen. It is separate from dmd's backend. Just for completeness sake, it is perfectly possible to generate x86_64 code without supporting 64bit values, you will just have alot of zeros in the upper bits :) dmd does support 64bit values of course.
Mar 13
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Basic Slicing support is in; The following code will now compile with the newCTFE_slicing branch: static immutable uint[] OneToTen = [1,2,3,4,5,6,7,8,9,10]; const(uint[]) SliceOf1to10(uint lwr, uint upr) { return OneToTen[lwr .. upr]; } static assert(SliceOf1to10(2,8) == [3u, 4u, 5u, 6u, 7u, 8u]); Contrary to the old interpreter no copies are made. newCTFE slices reference memory just as runtime slices do. Cheers, Stefan;
Mar 11
next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Saturday, 11 March 2017 at 14:39:54 UTC, Stefan Koch wrote:
 Basic Slicing support is in;
 The following code will now compile with the newCTFE_slicing 
 branch:
I guess this should greatly benefit compile-time parsing, right?
Mar 11
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 11 March 2017 at 17:38:04 UTC, Nordlöw wrote:
 On Saturday, 11 March 2017 at 14:39:54 UTC, Stefan Koch wrote:
 Basic Slicing support is in;
 The following code will now compile with the newCTFE_slicing 
 branch:
I guess this should greatly benefit compile-time parsing, right?
As soon as the UTF8 stuff works properly, yes. Slicing is really just syntactic sugar :)
Mar 11
prev sibling next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Mar 11, 2017 at 02:39:54PM +0000, Stefan Koch via Digitalmars-d wrote:
[...]
 Contrary to the old interpreter no copies are made.
 newCTFE slices reference memory just as runtime slices do.
[...] Awesome! This alone ought to greatly expand the scope of what's feasible in CTFE. Now I really can't wait to see this merged into master. :-) T -- English has the lovely word "defenestrate", meaning "to execute by throwing someone out a window", or more recently "to remove Windows from a computer and replace it with something useful". :-) -- John Cowan
Mar 11
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 11 March 2017 at 14:39:54 UTC, Stefan Koch wrote:
 [ ... Slice Support ... ]
Hi Guys, Since Slice support required an ABI there were a few bugs. Interestingly those bugs where there for a very long time :) The Type-handling I chose uses an index into a specific type-array to represent types. When this index is 0 we consider the type invalid. However I had an off by one bug in the check, causing the first TypeInstance of static arrays to be considered invalid. As an Invalid type the array has the size 0. Which in turn causes the Allocation for that array to allocate zero bytes for it. This zero allocation returns a valid pointer to the current top of the heap. (just without reserving any memory). Then when we slice the array the slice-descriptor has to go onto the heap. And it overwrites the array-descriptor which was allocated with zero size. causing it to point a to a bogus address which is equivalent to the array length. My Sunday was ruined before it began :)
Mar 12
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]
Hey Stefan, you have been working on this for many months now: is someone funding your work? If not, you should put up a bitcoin address or something, I know I'd contribute.
Mar 13
next sibling parent Meta <jared771 gmail.com> writes:
On Monday, 13 March 2017 at 15:49:35 UTC, Joakim wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]
Hey Stefan, you have been working on this for many months now: is someone funding your work? If not, you should put up a bitcoin address or something, I know I'd contribute.
There's Patreon, but they don't support bitcoin currently.
Mar 13
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 13 March 2017 at 15:49:35 UTC, Joakim wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]
Hey Stefan, you have been working on this for many months now: is someone funding your work? If not, you should put up a bitcoin address or something, I know I'd contribute.
Hi Joakim, My work does get sponsored by the D Language Foundation. Therefore any Donations best be directed to them. If someone wants to support my work specifically it's best to contact me personally. I have been thinking about setting up a Patreon account and decided it was too much trouble; considering that there is not a large number of people who even know my of work. Cheers, Stefan
Mar 13
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
I just fixed a bug in ref support; It should now work with slices. I also made it _slightly_ easier to make ABI changes for slice, since I expect the structure to change as soon as concat support is added. (e.g. slices will get a refcount, capacity and an origin ptr)
Mar 13
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 13 March 2017 at 19:40:30 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
I just fixed a bug in ref support; It should now work with slices. I also made it _slightly_ easier to make ABI changes for slice, since I expect the structure to change as soon as concat support is added. (e.g. slices will get a refcount, capacity and an origin ptr)
Bad news. Array expansion via assignment to length regressed. Fixing this is surprisingly time intensive. .... I am just not seeing where it's going wrong. It seems to use completely bogus offsets ... causing it to read from uninitialized memory.
Mar 14
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Mar 14, 2017 at 03:26:44PM +0000, Stefan Koch via Digitalmars-d wrote:
[...]
 Bad news.
 Array expansion via assignment to length regressed.
 Fixing this is surprisingly time intensive.
 .... I am just not seeing where it's going wrong.
 
 It seems to use completely bogus offsets ... causing it to read from
 uninitialized memory.
Sounds like there's a pointer bug / stack overflow / buffer overflow somewhere. Just my gut feeling from having faced similar bugs in my career. Unfortunately, these kinds of bugs are usually very difficult to trace, because the root cause can be very far away from where the symptoms show up, and can come from completely unrelated code. One way that sometimes works (but not always) is to try to shuffle the stack by moving functions / local variables around to see if the symptoms change. That may yield some clues as to the nature of the problem. But that's just a shot in the dark... generally these kinds of bugs are very hard to trace. Or maybe carefully step through the code starting from the length assignment in a debugger and see if any of the variables seem to have strange values. Sometimes the code immediately following is fine (inserting printf's of the buffer may indicate correct values) but it's something that happens afterwards that screws it up. Or, possibly, the state is already messed up before the length assignment... in which case it would be far more difficult to trace. :-( T -- If lightning were to ever strike an orchestra, it'd always hit the conductor first.
Mar 14
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 14 March 2017 at 16:42:01 UTC, H. S. Teoh wrote:
 [ ... ]
H.S.Teoh Thanks for your advice. Since I am running my own bytecode I am quite sure that there is no memory corruption going on. It has to do with the recent ABI changes. -- I just made strings use the same representation as every other slice, Meaning soon we can use std.utf.decode for correct string-foreach :) After that it is not a big step to supporting string concat.
Mar 14
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 3/14/17 4:26 PM, Stefan Koch wrote:
 On Monday, 13 March 2017 at 19:40:30 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
I just fixed a bug in ref support; It should now work with slices. I also made it _slightly_ easier to make ABI changes for slice, since I expect the structure to change as soon as concat support is added. (e.g. slices will get a refcount, capacity and an origin ptr)
Bad news. Array expansion via assignment to length regressed. Fixing this is surprisingly time intensive. .... I am just not seeing where it's going wrong. It seems to use completely bogus offsets ... causing it to read from uninitialized memory.
Time to try valgrind ? Last time I checked it worked just fine with D. --- Dmitry Olshansky
Mar 14
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 14 March 2017 at 21:53:29 UTC, Dmitry Olshansky wrote:
 On 3/14/17 4:26 PM, Stefan Koch wrote:
 On Monday, 13 March 2017 at 19:40:30 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
I just fixed a bug in ref support; It should now work with slices. I also made it _slightly_ easier to make ABI changes for slice, since I expect the structure to change as soon as concat support is added. (e.g. slices will get a refcount, capacity and an origin ptr)
Bad news. Array expansion via assignment to length regressed. Fixing this is surprisingly time intensive. .... I am just not seeing where it's going wrong. It seems to use completely bogus offsets ... causing it to read from uninitialized memory.
Time to try valgrind ? Last time I checked it worked just fine with D. --- Dmitry Olshansky
When I talk about uninitialized memory I mean memory inside the VM, which is untouched. The Problem is a problem with bytecode generation, not the execution. The code executes as it should, but the generated code is invalid.
Mar 14
prev sibling parent Moritz Maxeiner <moritz ucworks.org> writes:
On Tuesday, 14 March 2017 at 21:53:29 UTC, Dmitry Olshansky wrote:
 [...]

 Time to try valgrind ? Last time I checked it worked just fine 
 with D.
Valgrind 3.12.0, at least, works perfectly with dmd&ldc. One just has to remember to suppress the (static) memory leak in druntime[1] [1] https://github.com/dlang/druntime/pull/1557
Mar 14
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
String Slicing is in! The following code will now compile with newCTFE. string slice(string s, uint lwr, uint upr) { return s[lwr .. upr]; } static assert(slice("Hello World", 6, 11) == "World");
Mar 15
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 15 March 2017 at 13:49:43 UTC, Stefan Koch wrote:
 String Slicing is in!
Great! What remains?
Mar 15
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 15 March 2017 at 15:49:59 UTC, Nordlöw wrote:
 On Wednesday, 15 March 2017 at 13:49:43 UTC, Stefan Koch wrote:
 String Slicing is in!
Great! What remains?
Unsupported Features include : - Unicode/Auto-decoding support. NOTE: foreach(char c; string) { ... } will work since it does not auto-decode. - Floating point. - Classes - unions - structs containing other structs. - arrays of structs - arrays of arrays - || and && - ref return (though I doubt it's needed) - references of 64bit values And maybe other I cannot think of right now.
Mar 15
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 15 March 2017 at 15:59:46 UTC, Stefan Koch wrote:
 On Wednesday, 15 March 2017 at 15:49:59 UTC, Nordlöw wrote:
 On Wednesday, 15 March 2017 at 13:49:43 UTC, Stefan Koch wrote:
 String Slicing is in!
Great! What remains?
Unsupported Features include : - Unicode/Auto-decoding support. NOTE: foreach(char c; string) { ... } will work since it does not auto-decode. - Floating point. - Classes - unions - structs containing other structs. - arrays of structs - arrays of arrays - || and && - ref return (though I doubt it's needed) - references of 64bit values And maybe other I cannot think of right now.
slice expansion. and string-foreach caused dmd to miscomplie itself ... *sigh* So no string-foreach for now;
Mar 15
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 15 March 2017 at 16:11:44 UTC, Stefan Koch wrote:
 and string-foreach caused dmd to miscomplie itself ... *sigh*
Guess what ? Things work so much better when you allocate space for the zero terminator :) The segfaults are mysteriously gone :D
Mar 15
prev sibling next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 15 March 2017 at 15:59:46 UTC, Stefan Koch wrote:
 Unsupported Features include :

  - Unicode/Auto-decoding support.
 NOTE: foreach(char c; string) { ... } will work since it does 
 not auto-decode.

  - Floating point.
  - Classes
  - unions
  - structs containing other structs.
  - arrays of structs
  - arrays of arrays
  - || and &&
  - ref return (though I doubt it's needed)
  - references of 64bit values

 And maybe other I cannot think of right now.
How long do you estimate this will take? Can/Will some of your work on newCTFE-branch be merged onto master before _all_ these things have been fixed?
Mar 15
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 15 March 2017 at 19:00:21 UTC, Nordlöw wrote:
 On Wednesday, 15 March 2017 at 15:59:46 UTC, Stefan Koch wrote:
 Unsupported Features include :

  - Unicode/Auto-decoding support.
 NOTE: foreach(char c; string) { ... } will work since it does 
 not auto-decode.

  - Floating point.
  - Classes
  - unions
  - structs containing other structs.
  - arrays of structs
  - arrays of arrays
  - || and &&
  - ref return (though I doubt it's needed)
  - references of 64bit values

 And maybe other I cannot think of right now.
How long do you estimate this will take? Can/Will some of your work on newCTFE-branch be merged onto master before _all_ these things have been fixed?
Yes the plan is to forgo Classes Execptions and FP for newCTFE 1.0.
Mar 15
prev sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Mar 15, 2017 at 03:59:46PM +0000, Stefan Koch via Digitalmars-d wrote:
[...]
 Unsupported Features include :
[...]
  - Floating point.
[...]
  - unions
[...] What are the chances / what's the expected timeframe of unions being implemented? Support for unions in CTFE is a MAJOR milestone in making std.math CTFE-able, which IMO will take D compile-time capabilities to a whole new level, because it will greatly expand the scope of what's computable at compile-time in terms of floating-point constants, lookup tables, etc.. Imagine, for example, a precomputed table of values transcendental functions with some given resolution, for fast runtime lookups. (Of course, this also requires floating-point support in CTFE. But that should be relatively easy(?). As long as both the host and target architectures support the same set of IEEE floating-point types, which is probably the case for our currently-supported platforms.) T -- People say I'm indecisive, but I'm not sure about that. -- YHL, CONLANG
Mar 15
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 15 March 2017 at 18:52:16 UTC, H. S. Teoh wrote:
 On Wed, Mar 15, 2017 at 03:59:46PM +0000, Stefan Koch via 
 Digitalmars-d wrote: [...]
 Unsupported Features include :
[...]
  - Floating point.
[...]
  - unions
[...] What are the chances / what's the expected timeframe of unions being implemented? Support for unions in CTFE is a MAJOR milestone in making std.math CTFE-able, which IMO will take D compile-time capabilities to a whole new level, because it will greatly expand the scope of what's computable at compile-time in terms of floating-point constants, lookup tables, etc.. Imagine, for example, a precomputed table of values transcendental functions with some given resolution, for fast runtime lookups. (Of course, this also requires floating-point support in CTFE. But that should be relatively easy(?). As long as both the host and target architectures support the same set of IEEE floating-point types, which is probably the case for our currently-supported platforms.) T
I am not sure. The CTFE-ABI is different from the ABI at runtime. So more adventurous uses of unions are likely to be surprising. As for floating point, I have yet to find a solution that will work for more numerically inclined people. dmds constant folder apparently does some funky things in that domain as well. I do not anticipate to have any of the fp stuff working before dconf. Currently Unicode correct string-handling is more important. After that comes class support. FP is the very last thing on my list and will like take a lot of time.
Mar 15
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi Guys I am proud to say that newCTFE with string-foreach enabled, is now green! As soon as array-length assignment (and therefore expansion) is working again; You can start writing fast compile-time parsers. Albeit it will be bit a bit awkward without && and ||. :o) Today I feel like I have reached a milestone. A good day to all of you, Cheers Stefan.
Mar 16
next sibling parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
Am 16.03.2017 um 15:48 schrieb Stefan Koch:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi Guys I am proud to say that newCTFE with string-foreach enabled, is now green! As soon as array-length assignment (and therefore expansion) is working again; You can start writing fast compile-time parsers. Albeit it will be bit a bit awkward without && and ||. :o) Today I feel like I have reached a milestone. A good day to all of you, Cheers Stefan.
Congratulations! It's really starting to get exciting. You have no idea how much I look forward to see diet-ng run on the new engine ;-)
Mar 16
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Mar 16, 2017 at 04:18:35PM +0100, Snke Ludwig via Digitalmars-d wrote:
 Am 16.03.2017 um 15:48 schrieb Stefan Koch:
[...]
 Hi Guys I am proud to say that newCTFE with string-foreach enabled,
 is now green!
 
 As soon as array-length assignment (and therefore expansion) is
 working again; You can start writing fast compile-time parsers.
 
 Albeit it will be bit a bit awkward without && and ||. :o)
 
 Today I feel like I have reached a milestone.
 
 A good day to all of you,
 
 Cheers Stefan.
Congratulations! It's really starting to get exciting. You have no idea how much I look forward to see diet-ng run on the new engine ;-)
And I'm so excited to be able to soon write complicated compile-time parsers and parser generators without drastic reduction in compilation time or drastic increase in compiler memory requirements. T -- Fact is stranger than fiction.
Mar 16
prev sibling parent reply Kagamin <spam here.lot> writes:
On Thursday, 16 March 2017 at 14:48:22 UTC, Stefan Koch wrote:
 As soon as array-length assignment (and therefore expansion) is 
 working again;
 You can start writing fast compile-time parsers.
Though for AST one would need unions, like struct Node { int type; union { StringLiteral str; NumberLiteral num; ExpressionNode expr; } }
Mar 17
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 17 March 2017 at 12:55:22 UTC, Kagamin wrote:
 On Thursday, 16 March 2017 at 14:48:22 UTC, Stefan Koch wrote:
 As soon as array-length assignment (and therefore expansion) 
 is working again;
 You can start writing fast compile-time parsers.
Though for AST one would need unions, like struct Node { int type; union { StringLiteral str; NumberLiteral num; ExpressionNode expr; } }
You might as well replace union with struct here.
Mar 17
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Small update I fixed the handling of $ in slices. [1,2,3,4][$-2 .. $] == [3,4] will now work in newCTFE
Mar 17
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Mar 18, 2017 at 03:29:15AM +0000, Stefan Koch via Digitalmars-d wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Small update I fixed the handling of $ in slices. [1,2,3,4][$-2 .. $] == [3,4] will now work in newCTFE
Awesome! It's great to see the new CTFE engine slowly but surely taking shape. Really looking forward to see the final product. T -- The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world. -- Anonymous
Mar 17
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
I just restored Array/Slice-Expansion. I also introduced concat! This will now compile with newCTFE! static immutable uint[] OneToTen = [1,2,3,4,5,6,7,8,9,10]; const(uint[]) SliceOf1to10(uint lwr, uint upr) { return OneToTen[lwr .. upr]; } const(uint)[] testConcat() { return SliceOf1to10(0,4) ~ SliceOf1to10(7,9); } static assert(testConcat == [1,2,3,4,8,9]);
Mar 19
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Oh darn, function pointers regressed! I did not notice because the corresponding tests were commented out. I am working to fix it ASAP.
Mar 20
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 20 March 2017 at 11:48:56 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
Oh darn, function pointers regressed! I did not notice because the corresponding tests were commented out. I am working to fix it ASAP.
It's not function pointers. It's slice extension. It just happened that my function pointer tests relied on slice extension working.
Mar 20
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 20 March 2017 at 12:06:57 UTC, Stefan Koch wrote:
 On Monday, 20 March 2017 at 11:48:56 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
Oh darn, function pointers regressed! I did not notice because the corresponding tests were commented out. I am working to fix it ASAP.
It's not function pointers. It's slice extension. It just happened that my function pointer tests relied on slice extension working.
Actually it's just local variables of slices :) Doged a nasty bullet there.
Mar 20
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi I fixed the local slice issue. It turned out that there was special code in place to deal with this case; But that I forgot to change that to the new slice ABI. The moral of this story is; ABI changes are hard to do if you have an ad-hoc approach to it. However now I made ABI-dependeant values more visible. And it is unlikely that further slice-ABI changes will cause problems. Cheers, Stefan
Mar 21
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
I just fixed a bug in switches, where the fall-trough case would incorrectly jump after the switch. The reason this bug occurred is that none of my tests did cover the fall-trough case. The code that handles switches converts them into a big if-else chain because jump-tables are usually more expensive for small switches.
Mar 22
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Alright guys. I am just fixing the newCTFE LLVM backend such that I can have better guesses as to what performance a JIT could archive. Due to the ABI changes the llvm-backend a little optimization potential. It'll be a while until the llvm backend is on par again. Because functions and such need to be dealt with quite differently. Also the llvm backend recives less testing since I cannot run it at compile time(Yet :)) Cheers, Stefan -- sorry for the short message I am busy fixing codegen bugs as always.
Mar 31
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 31 March 2017 at 14:05:00 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
Alright guys. I am just fixing the newCTFE LLVM backend such that I can have better guesses as to what performance a JIT could archive. Due to the ABI changes the llvm-backend a little optimization potential. It'll be a while until the llvm backend is on par again. Because functions and such need to be dealt with quite differently. Also the llvm backend recives less testing since I cannot run it at compile time(Yet :)) Cheers, Stefan -- sorry for the short message I am busy fixing codegen bugs as always.
oh, yeah ... If you want to checkout the llvm backend fethc the newCTFE_LLVMBackend branch from https://github.com/UplinkCoder/dmd. the posix.mak is hardcoded to use libLLVM-3.5. However you should be able to use any version newer then 3.3.
Mar 31
parent reply Inquie <Inquie data.com> writes:
On Friday, 31 March 2017 at 14:16:15 UTC, Stefan Koch wrote:
 On Friday, 31 March 2017 at 14:05:00 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
Alright guys. I am just fixing the newCTFE LLVM backend such that I can have better guesses as to what performance a JIT could archive. Due to the ABI changes the llvm-backend a little optimization potential. It'll be a while until the llvm backend is on par again. Because functions and such need to be dealt with quite differently. Also the llvm backend recives less testing since I cannot run it at compile time(Yet :)) Cheers, Stefan -- sorry for the short message I am busy fixing codegen bugs as always.
oh, yeah ... If you want to checkout the llvm backend fethc the newCTFE_LLVMBackend branch from https://github.com/UplinkCoder/dmd. the posix.mak is hardcoded to use libLLVM-3.5. However you should be able to use any version newer then 3.3.
How far off until newCTFE is usable to compile the majority of templates out there? I have some new code that doesn't do anything real complex but seems quite slow for no apparent reason and would like to try the newCTFE if it has a good chance of working.
Apr 01
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 1 April 2017 at 17:06:14 UTC, Inquie wrote:
 How far off until newCTFE is usable to compile the majority of 
 templates out there? I have some new code that doesn't do 
 anything real complex but seems quite slow for no apparent 
 reason and would like to try the newCTFE if it has a good 
 chance of working.
Quite far. Templates are usually not slow because of CTFE. But because of O(n^2) ((O(n!) actully in some cases) nature of recursive templates. I am going to fix this after CTFE, (if the D Langauge Foundation or a third-party can provide sponsorship for that).
Apr 01
prev sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Apr 01, 2017 at 05:06:14PM +0000, Inquie via Digitalmars-d wrote:
[...]
 How far off until newCTFE is usable to compile the majority of
 templates out there?
CTFE and templates are two separate things. You may want to read this (draft) article to get a better understanding of how they fit together: https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time T -- Computers shouldn't beep through the keyhole.
Apr 01
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 2 April 2017 at 04:34:34 UTC, H. S. Teoh wrote:
 On Sat, Apr 01, 2017 at 05:06:14PM +0000, Inquie via 
 Digitalmars-d wrote: [...]
 How far off until newCTFE is usable to compile the majority of 
 templates out there?
CTFE and templates are two separate things. You may want to read this (draft) article to get a better understanding of how they fit together: https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time T
some drive-by review --- struct Box!float { int data; } Box!float floatBox; --- Is not what you meant.
Apr 02
prev sibling next sibling parent data pulverizer <data.pulverizer gmail.com> writes:
On Sunday, 2 April 2017 at 04:34:34 UTC, H. S. Teoh wrote:
 On Sat, Apr 01, 2017 at 05:06:14PM +0000, Inquie via 
 Digitalmars-d wrote: [...]
 How far off until newCTFE is usable to compile the majority of 
 templates out there?
CTFE and templates are two separate things. You may want to read this (draft) article to get a better understanding of how they fit together: https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time T
Just read your draft article, very illuminating. Thank you. There was a section "Interleaved AST manipulation and CTFE" which you mention is "one of the keystones of meta-programming in D". I think there should be a separate article on this topic as a way of popularising D's idioms. When can we expect this? :-)
Apr 05
prev sibling next sibling parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Sunday, 2 April 2017 at 04:34:34 UTC, H. S. Teoh wrote:
 On Sat, Apr 01, 2017 at 05:06:14PM +0000, Inquie via 
 Digitalmars-d wrote: [...]
 How far off until newCTFE is usable to compile the majority of 
 templates out there?
CTFE and templates are two separate things. You may want to read this (draft) article to get a better understanding of how they fit together: https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time T
CTFE and template expansion might be more tangled than you thought. For example, you do have access to CTFE during template expansion: http://forum.dlang.org/thread/yaekhryalyxyooaiuakj forum.dlang.org
Apr 05
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Apr 05, 2017 at 11:20:28AM +0000, Yuxuan Shui via Digitalmars-d wrote:
 On Sunday, 2 April 2017 at 04:34:34 UTC, H. S. Teoh wrote:
 On Sat, Apr 01, 2017 at 05:06:14PM +0000, Inquie via Digitalmars-d
 wrote: [...]
 How far off until newCTFE is usable to compile the majority of
 templates out there?
CTFE and templates are two separate things. You may want to read this (draft) article to get a better understanding of how they fit together: https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time T
CTFE and template expansion might be more tangled than you thought. For example, you do have access to CTFE during template expansion: http://forum.dlang.org/thread/yaekhryalyxyooaiuakj forum.dlang.org
Did you read the entire article? There is an entire section dedicated to interleaving of CTFE and templates. And no, you still cannot run CTFE on the same part of the AST that is being template-expanded. But you *can* run CTFE on a subtree that has already been fully expanded. And no, the forum post you linked to has nothing to do with CTFE. The so-called "static foreach" is unrolled at AST expansion time, and is not run through CTFE at all (unless later on you call the expanded function at "compile-time"). And is() expressions are also not CTFE, they are also evaluated at AST expansion time. Read the entire article first. ;-) T -- Guns don't kill people. Bullets do.
Apr 05
parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Wednesday, 5 April 2017 at 16:06:39 UTC, H. S. Teoh wrote:
 On Wed, Apr 05, 2017 at 11:20:28AM +0000, Yuxuan Shui via 
 Digitalmars-d wrote:
 [...]
Did you read the entire article? There is an entire section dedicated to interleaving of CTFE and templates. And no, you still cannot run CTFE on the same part of the AST that is being template-expanded. But you *can* run CTFE on a subtree that has already been fully expanded. And no, the forum post you linked to has nothing to do with CTFE. The so-called "static foreach" is unrolled at AST expansion time, and is not run through CTFE at all (unless later on you call the expanded function at "compile-time"). And is() expressions are also not CTFE, they are also evaluated at AST expansion time. Read the entire article first. ;-) T
I was talking about the use of R.front, R.drop in the template.
Apr 05
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Apr 05, 2017 at 08:08:32PM +0000, Yuxuan Shui via Digitalmars-d wrote:
 On Wednesday, 5 April 2017 at 16:06:39 UTC, H. S. Teoh wrote:
 On Wed, Apr 05, 2017 at 11:20:28AM +0000, Yuxuan Shui via Digitalmars-d
 wrote:
 [...]
Did you read the entire article? There is an entire section dedicated to interleaving of CTFE and templates. And no, you still cannot run CTFE on the same part of the AST that is being template-expanded. But you *can* run CTFE on a subtree that has already been fully expanded. And no, the forum post you linked to has nothing to do with CTFE. The so-called "static foreach" is unrolled at AST expansion time, and is not run through CTFE at all (unless later on you call the expanded function at "compile-time"). And is() expressions are also not CTFE, they are also evaluated at AST expansion time. Read the entire article first. ;-) T
I was talking about the use of R.front, R.drop in the template.
Yes, that's the interleaving I was talking about. In your code example, R is an alias to something outside the template itself, so as long as whatever it aliases can be "finalized" in its AST, it can be handed to the CTFE engine for evaluation. This is really just a more fancy instance of the more common `enum X = f(Y);` idiom, but it's really the same thing. The compiler tries to use CTFE every time it sees a value that's needed at "compile-time"; `enum` is the usual way to trigger this, but a template expansion that depends on the value, as you have here, is also where the same thing happens. What you *cannot* have, though, is if R is in the process of being AST-expanded. E.g., if .front itself requires expandRange() in its definition, then it won't work, because then the compiler can't finalize the AST of .front and CTFE won't be able to run it. T -- Nobody is perfect. I am Nobody. -- pepoluan, GKC forum
Apr 05
prev sibling parent Dragos Carp <dragoscarp gmail.com> writes:
On Sunday, 2 April 2017 at 04:34:34 UTC, H. S. Teoh wrote:
 On Sat, Apr 01, 2017 at 05:06:14PM +0000, Inquie via 
 Digitalmars-d wrote: [...]
 How far off until newCTFE is usable to compile the majority of 
 templates out there?
CTFE and templates are two separate things. You may want to read this (draft) article to get a better understanding of how they fit together: https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time T
Very nice and informative article! Thanks! Still I miss some comments about mixins and template mixins.
Apr 05
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi Guys, I wanted to give a short status update again. No new features; But a few things have changed under the hood. I am currently cleaning up code to get ready for a first publicized preview build. After || und && are implemented (which currently still give me an unreasonable amount of trouble), the preview will go life :) Cheers, Stefan
Apr 06
parent reply crimaniak <crimaniak gmail.com> writes:
Hi!

Is it hard to implement some way to access symbol from UDA 
constructor of symbol?

```
class uda
{
     this()
     {
          // Iterate by Foo members
         foreach (member; __traits(allMembers, __SYMBOL__))
             ...
     }
}


 uda
struct Foo
{

}

```
Apr 08
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 8 April 2017 at 08:50:27 UTC, crimaniak wrote:
 Hi!

 Is it hard to implement some way to access symbol from UDA 
 constructor of symbol?

 ```
 class uda
 {
     this()
     {
          // Iterate by Foo members
         foreach (member; __traits(allMembers, __SYMBOL__))
             ...
     }
 }


  uda
 struct Foo
 {

 }

 ```
This is a question for D.Learn. please post there. Also the question is to terse, I do not know what you are asking.
Apr 08
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 08/04/2017 10:14 AM, Stefan Koch wrote:
 On Saturday, 8 April 2017 at 08:50:27 UTC, crimaniak wrote:
 Hi!

 Is it hard to implement some way to access symbol from UDA constructor
 of symbol?

 ```
 class uda
 {
     this()
     {
          // Iterate by Foo members
         foreach (member; __traits(allMembers, __SYMBOL__))
             ...
     }
 }


  uda
 struct Foo
 {

 }

 ```
This is a question for D.Learn. please post there. Also the question is to terse, I do not know what you are asking.
Nope, not valid for D.learn as it is not currently possible.
Apr 08
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 8 April 2017 at 09:19:03 UTC, rikki cattermole wrote:
 Nope, not valid for D.learn as it is not currently possible.
perfectly valid in D.learn. You could have given the answer that is it not possible there.
Apr 08
prev sibling parent reply crimaniak <crimaniak gmail.com> writes:
On Saturday, 8 April 2017 at 09:14:17 UTC, Stefan Koch wrote:
 This is a question for D.Learn.
 please post there.
Yes, I did it already and know, it is not possible for now: https://forum.dlang.org/post/crkxlbtfhsplxfilakrk forum.dlang.org This is exact reason why I asked, how hard _to implement_ it. I asked you, because UDF constructor executed in CTFE, and you can give the most competent answer, because you are working on it.
 Also the question is to terse, I do not know what you are 
 asking.
How hard to implement something like special keyword __SYMBOL__, which will be alias for Foo (or string with fully qualified name of Foo) in example in my initial letter, in the scope of udf constructor.
Apr 08
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 8 April 2017 at 19:03:52 UTC, crimaniak wrote:
 On Saturday, 8 April 2017 at 09:14:17 UTC, Stefan Koch wrote:
 This is a question for D.Learn.
 please post there.
Yes, I did it already and know, it is not possible for now: https://forum.dlang.org/post/crkxlbtfhsplxfilakrk forum.dlang.org This is exact reason why I asked, how hard _to implement_ it. I asked you, because UDF constructor executed in CTFE, and you can give the most competent answer, because you are working on it.
 Also the question is to terse, I do not know what you are 
 asking.
How hard to implement something like special keyword __SYMBOL__, which will be alias for Foo (or string with fully qualified name of Foo) in example in my initial letter, in the scope of udf constructor.
This is not related to the ctfe subsystem. Therefore take my answer with a grain of salt. I assume that what you want will quite expensive in terms of compiler performance. Also it would require a language change. A facility as you want it would need to modify the constructor behind the scenes, either by adding a hidden template parameter to the constructor. I general I would be weary about adding this facility. Though right now I cannot see any wired corner-cases, I think that you are going to run into trouble with this. Particularly when forward-references are involved.
Apr 08
parent crimaniak <crimaniak gmail.com> writes:
On Sunday, 9 April 2017 at 00:25:45 UTC, Stefan Koch wrote:
...
 I assume that what you want will quite expensive in terms of 
 compiler performance.
... Thanks for your answer!
Apr 09
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi Guys :) I am currently fixing a bug involving complex members of structs (where complex means (slice, struct, array or pointer)) I did expect them to be broken ... but not to be _that_ broken :) struct S { uint[] slice; } uint fn() { S s; s.slice.length = 12; return cast(uint)s.slice.length; } static assert(fn() == 12); This code will not work because s.slice has no elementType :) (which does not mean it has the s.slice[0] has the type void) newCTFE literally looses the type information somewhere. And people wonder why I don't like mondays :)
Apr 10
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 10 April 2017 at 20:49:58 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
Hi Guys :) I am currently fixing a bug involving complex members of structs (where complex means (slice, struct, array or pointer)) I did expect them to be broken ... but not to be _that_ broken :) struct S { uint[] slice; } uint fn() { S s; s.slice.length = 12; return cast(uint)s.slice.length; } static assert(fn() == 12); This code will not work because s.slice has no elementType :) (which does not mean it has the s.slice[0] has the type void) newCTFE literally looses the type information somewhere. And people wonder why I don't like mondays :)
I found out that slice was never allocated :) This is an orthogonal problem, but it's fixed now. The problem from the above post still remains. And I still don't know why it happens.
Apr 11
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
I just found more states we get into, that should be impossible to ever get into. I am stumped. Baffled. And seriously befuddled!
Apr 12
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 12 April 2017 at 09:19:39 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
I just found more states we get into, that should be impossible to ever get into. I am stumped. Baffled. And seriously befuddled!
So .. this is partially because we assume the stack to be zeroed if we have not written to it yet. It is zero-initialized after all, however If we are returning from a function that wrote to the stack and then we are calling another function, that function will see the state the previous function left there... which just means ... we have to zero our temporaries and locals on function entery. implementing this however breaks incremental code-generation. Awwwww ....
Apr 12
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
I just fixed the static assert((null ~ null) is null); Hence I can now enable string-concat!
Apr 12
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Apr 12, 2017 at 01:18:13PM +0000, Stefan Koch via Digitalmars-d wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
I just fixed the static assert((null ~ null) is null); Hence I can now enable string-concat!
[...] Good news! T -- Век живи - век учись. А дураком помрёшь.
Apr 12
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Comma expressions should now work.
Apr 12
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi I want to share another story. I was pretty happy to have recursive function calls working. So happy in fact that I overlooked that they were actually generated twice. Let me illustrate what happend. Suppose we have the following module : void fn(uint rcount) { uint ctr = rcount; while (rcount--) ctr += fn(rcount); return ctr; } pragma(msg, fn(26)); the compiler hits the pragma(msg) and goes on to evaluate fn(26) newCTFE receives the functionBody as an ast-node. it starts processing (function 1) and hits the fn(rcount) it checks if it has the code for fn already. this check returns false since fn has not been completely generated yet. when this check returns it will write the FunctionBody in it's todo list. it wires up the call to the entry in the todo list. (function 2) it then hits the end of fn (function 1)and saves it in its code-cache. now it processes the TODO list it finds that it has to process fn. it starts processing fn (function 2) it hits the call. This time it does find an entry in the code cache for fn (funcion 1) it wires of the call and returns. The generate pseudo-code looks like : ... fn_0 (...) { ... call (fn_1, ...) ... } ... fn_1 (...) { ... call (fn_0, ...) ... } It was very surprised then I saw this :) Cheers, Stefan
Apr 14
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 { ... }
Wonderful news! Most of the Byteocode macros are gone! meaning less templates and faster bytecode generartion!
Apr 14
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]
The llvm backend is back in a fully working state. It's about 2times slower in my then my interpreter ;)
Apr 15
parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Saturday, 15 April 2017 at 10:10:54 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]
The llvm backend is back in a fully working state. It's about 2times slower in my then my interpreter ;)
Huh. In all cases, or only in trivial ones? Because I would have expected the overhead of jitting to become less relevant the more complex stuff you interpret vs jit.
Apr 15
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 15 April 2017 at 10:30:57 UTC, Moritz Maxeiner wrote:
 On Saturday, 15 April 2017 at 10:10:54 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 Hi Guys, due to the old CTFE status thread getting to page 
 30, I am now starting a new one.

 [...]
The llvm backend is back in a fully working state. It's about 2times slower in my then my interpreter ;)
Huh. In all cases, or only in trivial ones? Because I would have expected the overhead of jitting to become less relevant the more complex stuff you interpret vs jit.
It's an average number. on tests the represent the usual usage of ctfe.
Apr 15
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi Guys, I just fixed default initialization of structs. So now a larger portion of code will be compiled and executed by newCTFE. my_struct MyStruct; will now work, before it would trigger a bailout. NOTE: this will create bogus results if the struct does contain complex initializers i.e. anything other then integers. Complex type support will come after dconf.
Apr 16
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi Guys, As you already probably know some work has been done in the past week to get an x86 jit rolling. It is designed to produce very simple code with _any_ optimization at all. Since optimization introduces heavy complexity down the road, even if at first it looks very affordable. My opinion is : "_any_ optimization too much." This stance should make it possible to get some _really_ shiny performance numbers for dconf. Cheers, Stefan
Apr 26
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Apr 27, 2017 at 02:15:30AM +0000, Stefan Koch via Digitalmars-d wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi Guys, As you already probably know some work has been done in the past week to get an x86 jit rolling. It is designed to produce very simple code with _any_ optimization at all. Since optimization introduces heavy complexity down the road, even if at first it looks very affordable. My opinion is : "_any_ optimization too much." This stance should make it possible to get some _really_ shiny performance numbers for dconf.
[...] Is it possible at all to use any of the backend (in particular what parts of the optimizer that are pertinent), or is the API not conducive for this? T -- It always amuses me that Windows has a Safe Mode during bootup. Does that mean that Windows is normally unsafe?
Apr 26
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 27 April 2017 at 03:33:03 UTC, H. S. Teoh wrote:
 Is it possible at all to use any of the backend (in particular 
 what parts of the optimizer that are pertinent), or is the API 
 not conducive for this?


 T
It is of course possible to use dmds backend but not very desirable, dmds backend works on an expression-tree, which would be expensive to build from the linear IR newCTFE uses. Dmds backend is also very hard to debug for anyone who is not Walter. CTFE is the common case will be fastest if executed without any optimizer interfering. modern x86 chips done a very fine job indeed executing crappy code fast. Therefore making it possible to get away with very simple and fast codegen. (Where fast means code-generation speed rather then code execution speed).
Apr 26
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 4/27/17 4:15 AM, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi Guys, As you already probably know some work has been done in the past week to get an x86 jit rolling. It is designed to produce very simple code with _any_ optimization at all. Since optimization introduces heavy complexity down the road, even if at first it looks very affordable. My opinion is : "_any_ optimization too much."
There is also trade-off of spending too much time doing an optimization. That being said simple peep-hole optimizations may be well worth the effort.
 This stance should make it possible to get some _really_ shiny
 performance numbers for dconf.

 Cheers,
 Stefan
Apr 27
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 27 April 2017 at 08:51:17 UTC, Dmitry Olshansky 
wrote:
 On 4/27/17 4:15 AM, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
Hi Guys, As you already probably know some work has been done in the past week to get an x86 jit rolling. It is designed to produce very simple code with _any_ optimization at all. Since optimization introduces heavy complexity down the road, even if at first it looks very affordable. My opinion is : "_any_ optimization too much."
There is also trade-off of spending too much time doing an optimization. That being said simple peep-hole optimizations may be well worth the effort.
 This stance should make it possible to get some _really_ shiny
 performance numbers for dconf.

 Cheers,
 Stefan
I should probably clarify; I made a typo. I was meaning to write "without _any_ optimization at all." Peep-holing would be worth it for wanting to get the last drop of performance; However in the specific case of newCTFE, the crappiest JIT will already be much faster then an optimized interpreter would be. Small peephole optimization quickly turns into and endless source of bugs.
Apr 27
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
After a little of exploration of the JIT, I have now determined that a simple risc architecture is still the best. (codegen for scaled loads is hard :p) I am now back to fixing non-compiling code, such as : struct S { uint[] slice; } uint fn() { S s; s.slice.length = 12; return cast(uint)s.slice.length; } static assert(fn() == 12); This simple test does not compile because; ahm well ... Somewhere along the road we loose the type of s.slice and we cannot tell where to get .length from.
Apr 28
next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 28 April 2017 at 08:47:43 UTC, Stefan Koch wrote:
 After a little of exploration of the JIT, I have now determined 
 that a simple risc architecture is still the best.
 (codegen for scaled loads is hard :p)
Do you mean no Jit?
Apr 28
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 28 April 2017 at 13:03:42 UTC, Nordlöw wrote:
 On Friday, 28 April 2017 at 08:47:43 UTC, Stefan Koch wrote:
 After a little of exploration of the JIT, I have now 
 determined that a simple risc architecture is still the best.
 (codegen for scaled loads is hard :p)
Do you mean no Jit?
Of course there will be a JIT. But currently I am fixing busy bugs in the generated IR. So the implementation of jit will have to wait a little.
Apr 28
parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 28 April 2017 at 13:13:16 UTC, Stefan Koch wrote:
 Do you mean no Jit?
Of course there will be a JIT.
Ah, I misunderstood you formulation.
 But currently I am fixing busy bugs in the generated IR.
 So the implementation of jit will have to wait a little.
Ok. Thanks.
Apr 28
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 28 April 2017 at 08:47:43 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
After a little of exploration of the JIT, I have now determined that a simple risc architecture is still the best. (codegen for scaled loads is hard :p) I am now back to fixing non-compiling code, such as : struct S { uint[] slice; } uint fn() { S s; s.slice.length = 12; return cast(uint)s.slice.length; } static assert(fn() == 12); This simple test does not compile because; ahm well ... Somewhere along the road we loose the type of s.slice and we cannot tell where to get .length from.
I fixed this just now! ABI's are hard :)
Apr 29
prev sibling parent Kagamin <spam here.lot> writes:
An article about LLVM jit: 
https://webkit.org/blog/5852/introducing-the-b3-jit-compiler/
May 03
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi Guys, I just implemented sliceAssigment. meaning the following code will now compile: uint[] assignSlice(uint from, uint to, uint[] stuff) { uint[] slice; slice.length = to + 4; foreach (uint i; 0 .. to + 4) { slice[i] = i + 1; } slice[from .. to] = stuff; return slice; } static assert(assignSlice(1, 4, [9, 8, 7]) == [1, 9, 8, 7, 5, 6, 7, 8]);
Apr 28
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 28 April 2017 at 17:53:04 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
Hi Guys, I just implemented sliceAssigment. meaning the following code will now compile: uint[] assignSlice(uint from, uint to, uint[] stuff) { uint[] slice; slice.length = to + 4; foreach (uint i; 0 .. to + 4) { slice[i] = i + 1; } slice[from .. to] = stuff; return slice; } static assert(assignSlice(1, 4, [9, 8, 7]) == [1, 9, 8, 7, 5, 6, 7, 8]);
as always ... I spoke too soon :( while running test I forgot to specify -bc-ctfe ... Although I use the same code for slicing ... it seems it misbehaves in the usecase.
Apr 28
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 28 April 2017 at 17:53:04 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
Hi Guys, I just implemented sliceAssigment. meaning the following code will now compile: uint[] assignSlice(uint from, uint to, uint[] stuff) { uint[] slice; slice.length = to + 4; foreach (uint i; 0 .. to + 4) { slice[i] = i + 1; } slice[from .. to] = stuff; return slice; } static assert(assignSlice(1, 4, [9, 8, 7]) == [1, 9, 8, 7, 5, 6, 7, 8]);
FIXED!
Apr 29
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Big news! The first step to include debug info has been done. Yes this means you will be able to step through ctfe code while the compiler executes it.
Apr 30
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sun, Apr 30, 2017 at 01:26:09PM +0000, Stefan Koch via Digitalmars-d wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Big news! The first step to include debug info has been done. Yes this means you will be able to step through ctfe code while the compiler executes it.
Wow! Will that be accessible to users in the end? That could be a totally awesome way of debugging CTFE code! T -- Life begins when you can spend your spare time programming instead of watching television. -- Cal Keegan
Apr 30
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 30 April 2017 at 19:52:27 UTC, H. S. Teoh wrote:
 On Sun, Apr 30, 2017 at 01:26:09PM +0000, Stefan Koch via 
 Digitalmars-d wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
Big news! The first step to include debug info has been done. Yes this means you will be able to step through ctfe code while the compiler executes it.
Wow! Will that be accessible to users in the end? That could be a totally awesome way of debugging CTFE code! T
Yes the plan is to make it accessible for the advanced user. probably with a really bad ui, though (since I am awful at UI code).
May 01
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Mon, May 01, 2017 at 06:23:08PM +0000, Stefan Koch via Digitalmars-d wrote:
 On Sunday, 30 April 2017 at 19:52:27 UTC, H. S. Teoh wrote:
 On Sun, Apr 30, 2017 at 01:26:09PM +0000, Stefan Koch via Digitalmars-d
 wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Big news! The first step to include debug info has been done. Yes this means you will be able to step through ctfe code while the compiler executes it.
Wow! Will that be accessible to users in the end? That could be a totally awesome way of debugging CTFE code! T
Yes the plan is to make it accessible for the advanced user. probably with a really bad ui, though (since I am awful at UI code).
I'm not sure about providing a debugger UI inside the compiler itself... it's certainly possible, and could lead to interesting new ways of using a compiler, but I was thinking more along the lines of providing the necessary hooks so that you could attach an external debugger to the CTFE engine. But if the debugger UI is simple enough, perhaps having it built into the compiler may not be a bad thing. It would also avoid potential trouble caused by some platforms not having debuggers capable of plugging into the compiler in that way. But still, I can see people demanding IDE integration for this eventually... :-O T -- Obviously, some things aren't very obvious.
May 01
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 1 May 2017 at 19:06:24 UTC, H. S. Teoh wrote:
 On Mon, May 01, 2017 at 06:23:08PM +0000, Stefan Koch via 
 Digitalmars-d wrote:
 [...]
I'm not sure about providing a debugger UI inside the compiler itself... it's certainly possible, and could lead to interesting new ways of using a compiler, but I was thinking more along the lines of providing the necessary hooks so that you could attach an external debugger to the CTFE engine. But if the debugger UI is simple enough, perhaps having it built into the compiler may not be a bad thing. It would also avoid potential trouble caused by some platforms not having debuggers capable of plugging into the compiler in that way. But still, I can see people demanding IDE integration for this eventually... :-O T
I intended for the debugging functionality to be exposed via a udp socket listening on localhost. Such that a debug-ui does not have to deal with ipc difficulties. I am strictly against building a debugger into dmd.
May 02
next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, May 02, 2017 at 09:55:56AM +0000, Stefan Koch via Digitalmars-d wrote:
[...]
 I intended for the debugging functionality to be exposed via a udp
 socket listening on localhost.
 Such that a debug-ui does not have to deal with ipc difficulties.
 I am strictly against building a debugger into dmd.
Nice, that's a good approach. T -- Bomb technician: If I'm running, try to keep up.
May 02
prev sibling parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Tuesday, 2 May 2017 at 09:55:56 UTC, Stefan Koch wrote:
 [...]

 I intended for the debugging functionality to be exposed via a 
 udp socket listening on localhost.
 Such that a debug-ui does not have to deal with ipc 
 difficulties.
Hm, rationale for UDP over TCP here? I would assume one wouldn't want debugging info to be delivered out of order (or not at all); I mean, I guess it would be ok for localhost only (though one is then depending on implementation specifics vs. protocol semantics), but *if* you use sockets, you will eventually get people who use that over the network (and then the fun begins). Using TCP would just remove any potential future headache from the equation.
May 02
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 2 May 2017 at 22:08:31 UTC, Moritz Maxeiner wrote:
 On Tuesday, 2 May 2017 at 09:55:56 UTC, Stefan Koch wrote:
 [...]

 I intended for the debugging functionality to be exposed via a 
 udp socket listening on localhost.
 Such that a debug-ui does not have to deal with ipc 
 difficulties.
Hm, rationale for UDP over TCP here? I would assume one wouldn't want debugging info to be delivered out of order (or not at all); I mean, I guess it would be ok for localhost only (though one is then depending on implementation specifics vs. protocol semantics), but *if* you use sockets, you will eventually get people who use that over the network (and then the fun begins). Using TCP would just remove any potential future headache from the equation.
I think any ordering should be done explicitly at the debugging protocol level. for example when sending sourceline messages the order is given by the line number and ordering can be done by the application. It's the same for breakpoint setting or for breakpoint trigger notification. As for lost packages, we want to respond intelligently as well. And maybe reduce the amount of data in the package, to make arrival of future packages more likely.
May 02
next sibling parent reply Adrian Matoga <dlang.spam matoga.info> writes:
On Wednesday, 3 May 2017 at 04:22:00 UTC, Stefan Koch wrote:
 On Tuesday, 2 May 2017 at 22:08:31 UTC, Moritz Maxeiner wrote:
 [...]
 Using TCP would just remove any potential future headache from 
 the equation.
I think any ordering should be done explicitly at the debugging protocol level. for example when sending sourceline messages the order is given by the line number and ordering can be done by the application. It's the same for breakpoint setting or for breakpoint trigger notification. As for lost packages, we want to respond intelligently as well. And maybe reduce the amount of data in the package, to make arrival of future packages more likely.
So you're going to reinvent TCP in your debugging protocol?
May 02
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 3 May 2017 at 06:10:22 UTC, Adrian Matoga wrote:
 So you're going to reinvent TCP in your debugging protocol?
No. there is no need for a full blown recovery mechanism. For the typical usecase a lossless orderd connection can be assumed. And most things are not order dependent
May 03
next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 3 May 2017 at 07:35:56 UTC, Stefan Koch wrote:
 On Wednesday, 3 May 2017 at 06:10:22 UTC, Adrian Matoga wrote:
 So you're going to reinvent TCP in your debugging protocol?
No. there is no need for a full blown recovery mechanism. For the typical usecase a lossless orderd connection can be assumed. And most things are not order dependent
What about packet boundaries?
May 03
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 3 May 2017 at 08:23:54 UTC, Nordlöw wrote:
 On Wednesday, 3 May 2017 at 07:35:56 UTC, Stefan Koch wrote:
 On Wednesday, 3 May 2017 at 06:10:22 UTC, Adrian Matoga wrote:
 So you're going to reinvent TCP in your debugging protocol?
No. there is no need for a full blown recovery mechanism. For the typical usecase a lossless orderd connection can be assumed. And most things are not order dependent
What about packet boundaries?
We send source line by line. Packets should be under 1k in most cases.
May 03
prev sibling next sibling parent Moritz Maxeiner <moritz ucworks.org> writes:
On Wednesday, 3 May 2017 at 07:35:56 UTC, Stefan Koch wrote:
 For the typical usecase a lossless orderd connection can be 
 assumed.
- udp is not connection oriented, i.e. there is no connection - udp is not lossless and pretending it is means setting yourself up for a headache down the road - udp datagrams are not guaranteed to arrive in the order they were sent and pretending they do is also setting yourself up for a headache down the road What you've described so far is a classic, textbook use case of tcp.
May 03
prev sibling parent Minty Fresh <mint fresh.com> writes:
On Wednesday, 3 May 2017 at 07:35:56 UTC, Stefan Koch wrote:
 On Wednesday, 3 May 2017 at 06:10:22 UTC, Adrian Matoga wrote:
 So you're going to reinvent TCP in your debugging protocol?
No. there is no need for a full blown recovery mechanism. For the typical usecase a lossless orderd connection can be assumed. And most things are not order dependent
The debugger isn't a massive, real-time system that needs to service thousands of clients and squeeze as much performance out of the network as possible. The overhead incurred by TCP is essentially not worth considering for something that's going to be run over localhost 90%, and service just the one client. Reinventing the wheel adds a far bigger overhead: Maintaining your new protocol. TCP implementations are readily available. They're well maintained, well documented, and are essentially guaranteed to work across platforms. Implementing a new protocol just adds an extra point of breakage for little to no gain. It also incurs the cost of the associated development time, and - down the line - any time spent fixing or iterating. Not to mention that tests need to be written, documentation needs to be put in place. A classic case of premature optimization.
May 03
prev sibling parent Moritz Maxeiner <moritz ucworks.org> writes:
On Wednesday, 3 May 2017 at 04:22:00 UTC, Stefan Koch wrote:
 [...]

 I think any ordering should be done explicitly at the debugging 
 protocol level.
 for example when sending sourceline messages the order is given 
 by the line number and ordering can be done by the application.
 It's the same for breakpoint setting or for breakpoint trigger 
 notification.
Why? If I were to write a client for the debugging protocol, I wouldn't want to write protocol ordering logic (and essentially reimplement part of tcp). I would want to react to protocol messages as they arrive.
 As for lost packages, we want to respond intelligently as well.
The only way I know of to respond intelligently to lost packages using udp - if you care about the information in them (which we do in this use case) - is to implement a retransmit mechanism; i.e. you would be reimplementing another part of tcp.
 And maybe reduce the amount of data in the package, to make 
 arrival of future packages more likely.
You assume a causation between udp datagram size and delivery probability, which - however likely - is an implementation detail.
May 03
prev sibling parent Adrian Matoga <dlang.spam matoga.info> writes:
On Sunday, 30 April 2017 at 19:52:27 UTC, H. S. Teoh wrote:
 On Sun, Apr 30, 2017 at 01:26:09PM +0000, Stefan Koch via 
 Digitalmars-d wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
Big news! The first step to include debug info has been done. Yes this means you will be able to step through ctfe code while the compiler executes it.
Wow! Will that be accessible to users in the end? That could be a totally awesome way of debugging CTFE code!
I used to think the same, but with each new line of code I write that is to be executed in CT, I become more convinced that there's no need to expose such debugging interface to the end user. Why? Because the end user expects that CTFE either gives exactly the same results as run-time execution or that it stops with an explicit error message on something that is not designed to be executed in CT. Any other problem found during CTFE execution must be 100% reproducible in run time or it's an ICE. Any CTFE debugging should be only for compiler maintainers, and the user shouldn't worry that CTFE could mess up something.
May 03
prev sibling parent Swoorup Joshi <swoorupjoshi gmail.com> writes:
On Sunday, 30 April 2017 at 13:26:09 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
Big news! The first step to include debug info has been done. Yes this means you will be able to step through ctfe code while the compiler executes it.
This should have been kept secret before C++ steals it and does not give credit to D. :D
May 03
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Thanks to Daniel Murphy's input; '&&' works now in my experimental version. I hope to get it to pass the auto-tester soon! This is a big step :) Thanks Daniel.
May 10
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi Guys, Outer function arguments are now supperted. meaning this code will now work: int[] filterBy(int[] arr , bool function(uint) fn) { int[] result = []; uint resultLength; result.length = arr.length; foreach(i;0 .. arr.length) { auto e = arr[i]; bool r = true; r = fn(e); if(r) { result[resultLength++] = e; } } int[] filterResult; filterResult.length = resultLength; foreach(i; 0 .. resultLength) { filterResult[i] = result[i]; } return filterResult; } bool isDiv2(uint e) { bool result; result = (e % 2 == 0); return result; } static assert(filterBy([3,4,5], &isDiv2) == [4]); before this would behaved very strangely ;) because isDiv would have been executed instead filterBy. And since after bytecode compilation there is no type checking anymore the arrayPtr would have been interpreter as an integer. (which is always 4byte aligend) that would have caused the isDiv to return 1; which would have been interpreted as address. and whatever was there would have been treated as array descriptor. resulting in mostly the [] return value. ... anyway. I am happy this is fixed now.
May 12
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 12 May 2017 at 11:21:56 UTC, Stefan Koch wrote:

 ...
 anyway.
 I am happy this is fixed now.
Now I am less happy. The fallout of this fix causes code in std.ascii to miscompile. Apperantly we don't make sure our function list is cleared before finalization.
May 12