www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - CTFE Status 2

reply Stefan Koch <uplink.coder googlemail.com> writes:
Hi Guys, due to the old CTFE status thread getting to page 30, I 
am now starting a new one.

First let me summerize which features are currently working:
In order of date, the latest features come first.

- fixed continue and break for DoWhileStatements
- non-toplevel Function Pointers as arguments
- basic function pointers
--- phobos, druntime & dmd unit-tests pass ----
- Method Calls (now range foreach will work)
- 1-level Pointers
- recursive function calls
- fixed continue and break for ForStatements (includes foreach 
which is lowered)
- basic Function Calls
- static immutable globals
- fixed continue & break for WhileStatemens
- 1d array literals
- ternary expressions ? :
- struct literals.
- basic struct support.
- Slice and Array boundschecks
- 1d array and slice indexing
- do while statements
- break and continue support
- gotos and labels
- foreach over arrays
- for statements
- integer math
.... (and I few a forgot to list)

unsupported features are
- anything with strings (due to missing unicode handling)
- slicing & concat (due to missing memeory-manager)
- floating point math
- classes (due to missing vtbl and ref support)
- exceptions (due to missing stack-unwinding support and 
side-band channels)
- more complex structs i.e. with struct-members (due to 
incomplete type-handling)
-----------

Currently I am having trouble caused by a bug in dmds inliner 
that only happens on when dmd is compiled as a 32bit executable 
until I have  isolated / fixed this development is slowed down.

--
I hope this thread is informative and will continue to be that 
way.

Cheers,
Stefan (aka UplinkCoder)
Feb 16
next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 Cheers,
 Stefan (aka UplinkCoder)
Stupid question: is your online alias a reference to the game Uplink?
Feb 16
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:19:14 UTC, Jack Stouffer 
wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 Cheers,
 Stefan (aka UplinkCoder)
Stupid question: is your online alias a reference to the game Uplink?
Yes it is. Uplink Hackers Elite is a great game.
Feb 16
prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Feb 16, 2017 at 09:05:51PM +0000, Stefan Koch via Digitalmars-d wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, I am now
 starting a new one.
 
 First let me summerize which features are currently working:
 In order of date, the latest features come first.
 
 - fixed continue and break for DoWhileStatements
 - non-toplevel Function Pointers as arguments
 - basic function pointers
 --- phobos, druntime & dmd unit-tests pass ----
[...] This is great! The fact that you got it to the point unittests pass is awesome. Can't wait for the new CTFE engine to be released! Do you have any estimate as to when it will be mergeable into master? T -- There are three kinds of people in the world: those who can count, and those who can't.
Feb 16
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 23:16:34 UTC, H. S. Teoh wrote:
 On Thu, Feb 16, 2017 at 09:05:51PM +0000, Stefan Koch via 
 Digitalmars-d wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.
 
 First let me summerize which features are currently working: 
 In order of date, the latest features come first.
 
 - fixed continue and break for DoWhileStatements
 - non-toplevel Function Pointers as arguments
 - basic function pointers
 --- phobos, druntime & dmd unit-tests pass ----
[...] This is great! The fact that you got it to the point unittests pass is awesome. Can't wait for the new CTFE engine to be released! Do you have any estimate as to when it will be mergeable into master? T
I estimate that newCTFE 1.0 as outlined in the vision document will be ready around April. This estimate is a lower bound as it assumes that things go smoothly, and the past 6 months have shown that the CTFE-work does not go smoothly most of the time. As an upper bound: I have the personal goal to get it merged before DConf. At this point most parts of the actual engine are done, the remaining work mostly involves either making the runtime ctfeable or building runtime-feature support into dmd. These things are outside of my domain knowledge and therefore take much more time then they should.
Feb 16
prev sibling next sibling parent Andrea Fontana <nospam example.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [...]
 I hope this thread is informative and will continue to be that 
 way.

 Cheers,
 Stefan (aka UplinkCoder)
Yes, it is!
Feb 17
prev sibling next sibling parent Lurker <lurker gmail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 ...
 --
 I hope this thread is informative and will continue to be that 
 way.

 Cheers,
 Stefan (aka UplinkCoder)
Thanks Stefan for your work, and for documenting it. Very exciting to read the progress! I wish more people were as devoted as you are! Keep up the good work!
Feb 17
prev sibling next sibling parent reply Temtaime <temtaime gmail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 First let me summerize which features are currently working:
 In order of date, the latest features come first.

 - fixed continue and break for DoWhileStatements
 - non-toplevel Function Pointers as arguments
 - basic function pointers
 --- phobos, druntime & dmd unit-tests pass ----
 - Method Calls (now range foreach will work)
 - 1-level Pointers
 - recursive function calls
 - fixed continue and break for ForStatements (includes foreach 
 which is lowered)
 - basic Function Calls
 - static immutable globals
 - fixed continue & break for WhileStatemens
 - 1d array literals
 - ternary expressions ? :
 - struct literals.
 - basic struct support.
 - Slice and Array boundschecks
 - 1d array and slice indexing
 - do while statements
 - break and continue support
 - gotos and labels
 - foreach over arrays
 - for statements
 - integer math
 .... (and I few a forgot to list)

 unsupported features are
 - anything with strings (due to missing unicode handling)
 - slicing & concat (due to missing memeory-manager)
 - floating point math
 - classes (due to missing vtbl and ref support)
 - exceptions (due to missing stack-unwinding support and 
 side-band channels)
 - more complex structs i.e. with struct-members (due to 
 incomplete type-handling)
 -----------

 Currently I am having trouble caused by a bug in dmds inliner 
 that only happens on when dmd is compiled as a 32bit executable 
 until I have  isolated / fixed this development is slowed down.

 --
 I hope this thread is informative and will continue to be that 
 way.

 Cheers,
 Stefan (aka UplinkCoder)
Just get LDC. Make it use JIT. And you'll get all the features working. Writing slow interpreter is ... wasting efforts.
Feb 17
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 17 February 2017 at 19:44:10 UTC, Temtaime wrote:
 Just get LDC.
 Make it use JIT.
 And you'll get all the features working.
 Writing slow interpreter is ... wasting efforts.
For your information LLVM takes about 5 milliseconds to start up, it also takes a lot of time to generate code even when completely unoptimized. For usual one-shot CTFE functions this leads to a _HUGE_ pessimisation of performance. For the quite common usecase of returning a literal out of function templates newCTFE takes UNDER a MICROsecond. There is no way LLVM could even come close no matter how many caching layers one adds.
Feb 17
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 2/17/17 8:44 PM, Temtaime wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 I hope this thread is informative and will continue to be that way.

 Cheers,
 Stefan (aka UplinkCoder)
Just get LDC. Make it use JIT. And you'll get all the features working. Writing slow interpreter is ... wasting efforts.
This is a common misconception. LLVM as AoT optimizing compiler is great, but for the JIT compiler it's actually far too slow at codegen to be a sensible choice. --- Dmitry Olshansky
Feb 17
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Friday, 17 February 2017 at 21:01:33 UTC, Dmitry Olshansky 
wrote:
 This is a common misconception. LLVM as AoT optimizing compiler 
 is great, but for the JIT compiler it's actually far too slow 
 at codegen to be a sensible choice.
I don't know how big of a misconception it is. The Dropbox effort on Pyston (LLVM JIT for Python) just recently shut down and nobody expressed a lot of surprise in the HN comments.
Feb 17
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 2/17/17 10:37 PM, jmh530 wrote:
 On Friday, 17 February 2017 at 21:01:33 UTC, Dmitry Olshansky wrote:
 This is a common misconception. LLVM as AoT optimizing compiler is
 great, but for the JIT compiler it's actually far too slow at codegen
 to be a sensible choice.
I don't know how big of a misconception it is. The Dropbox effort on Pyston (LLVM JIT for Python) just recently shut down and nobody expressed a lot of surprise in the HN comments.
It might be that HN just in general are pessimistic ;) w.r.t Pyston I've seen it coming since the very first blog post that described their approach to the problem. However I've seen lots of folks being quite enthusiastic about that effort. ---- Dmitry Olshansky
Feb 17
prev sibling next sibling parent reply Satoshi <satoshi rikarin.org> writes:
Will be possible to make precompiled headers and distribute it 
with static library for example?
Feb 18
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 18 February 2017 at 21:24:18 UTC, Satoshi wrote:
 Will be possible to make precompiled headers and distribute it 
 with static library for example?
anything is possible with ctfe and mixins and templates. certainly header generation as well, you can write your own system for it and do it as a build step, in case the built-in header generator is inadequate.
Feb 19
prev sibling next sibling parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]
Thank you for your continued work on this. I heavily rely on D's CTFE functionality and I try to read all your updates on it.
Feb 18
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 18 February 2017 at 22:40:44 UTC, Moritz Maxeiner 
wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]
Thank you for your continued work on this. I heavily rely on D's CTFE functionality and I try to read all your updates on it.
Hello Moritz, D's ctfe functionality is almost complete. This thread is not about ctfe as it is currently implemented. I am working on/writing about a faster re-implementation of ctfe. When my work is finished nothing will change functionality wise, it will just be much more efficient.
Feb 18
parent Moritz Maxeiner <moritz ucworks.org> writes:
On Sunday, 19 February 2017 at 01:52:07 UTC, Stefan Koch wrote:
 On Saturday, 18 February 2017 at 22:40:44 UTC, Moritz Maxeiner 
 wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 Hi Guys, due to the old CTFE status thread getting to page 
 30, I am now starting a new one.

 [...]
Thank you for your continued work on this. I heavily rely on D's CTFE functionality and I try to read all your updates on it.
Hello Moritz, D's ctfe functionality is almost complete. This thread is not about ctfe as it is currently implemented. I am working on/writing about a faster re-implementation of ctfe. When my work is finished nothing will change functionality wise, it will just be much more efficient.
Hello Stefan, my apologies if I wasn't clear: I'm aware that this isn't about adding anything new in terms of functionality, what I intended to imply with "heavily rely on" was that I use enough of it for it to be noticeable in compile time / memory consumption, which is why I'm very happy about potential speedup and/or memory consumption reduction in CTFE. One public example of that was llvm-d <2.0.
Feb 18
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 Currently I am having trouble caused by a bug in dmds inliner 
 that only happens on when dmd is compiled as a 32bit executable 
 until I have  isolated / fixed this development is slowed down.
I am slowly narrowing down on the source of this bug. Until it is fixed builds of newCTFE might not behave correctly, if they have been complied with -inline!
Feb 20
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 21 February 2017 at 01:57:21 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 Currently I am having trouble caused by a bug in dmds inliner 
 that only happens on when dmd is compiled as a 32bit 
 executable until I have  isolated / fixed this development is 
 slowed down.
I am slowly narrowing down on the source of this bug. Until it is fixed builds of newCTFE might not behave correctly, if they have been complied with -inline!
The culprit that introduced the bug seems to be : https://github.com/dlang/dmd/commit/cd01efb4dd5e32277cb156c3cc2b451bdcb68b52 however more testing is needed before I can be sure.
Feb 21
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 Currently I am having trouble caused by a bug in dmds inliner 
 that only happens on when dmd is compiled as a 32bit executable 
 until I have  isolated / fixed this development is slowed down.
This bug was Issue #17220 and is fixed ~master. Unitl the auto-tester is updated is updated workarounds have to suffice. In this case the workaround is padding a structure such that it's size in no longer 32.
Feb 24
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi Guys, ref variable handing is coming soon! (only int/uint for now)... Also I have suspicions that there is a bug in my ctfe_evaluator in which the first default arguments can overridden with the this pointer :) I will test "this" tomorrow :) Also I submitted my talk proposal a few hours ago; Of course it's about newCTFE. Cheers, Stefan
Feb 25
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [...]
Simple ref calls work now. Meaning the following code will compile :) uint sum(uint[] arr) { uint sum; foreach(uint i;0 .. cast(uint)arr.length) { addToSum(sum, arr[i]); } return sum; } void addToSum(ref uint sum, uint element) { sum = sum + element; // works now as well return ; } static assert([1,2,3,4,5].sum == 15);
Feb 27
parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 27 February 2017 at 23:48:13 UTC, Stefan Koch wrote:
 Simple ref calls work now.
 Meaning the following code will compile :)
Great work.
Feb 28
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Hi Guys, The implementation of ref parameters by itself was successful. However they hey have one bug, which is a consquence of newCTFE not touching dmd's state. They cannot modify an outer parameter. (where outer means that the parameter does not originate from a ctfe-evaluation further up the call stack) uint add8ret3(ref uint a) {a += 8; return 3;} pragma (msg, (uint outer) { outer += add8ret3(a); return outer;}(1)); // will print 4 pragma (msg, (uint outer) { uint inner = outer; inner += add8ret3(inner); return inner; }(1)); // will print 12 as expected. There are two options I could take: The first one is to disallow having refs of outer parameters taken. (and bailing out in this case) The second option is to implement the logic necessary to modify these expression nodes. Both options require me to detect if we have an outer ref parameter. but that should be easy to do. Cheers, Stefan
Mar 01
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 2 March 2017 at 03:09:56 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
 uint add8ret3(ref uint a) {a += 8; return 3;}

 pragma (msg, (uint outer) { outer += add8ret3(a); return 
 outer;}(1)); // will print 4
was supposed to say: pragma (msg, (uint outer) { outer += add8ret3(outer); return outer;}(1)); // will print 4
Mar 01
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 2 March 2017 at 03:09:56 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
Hi Guys, The implementation of ref parameters by itself was successful. However they hey have one bug, which is a consquence of newCTFE not touching dmd's state. They cannot modify an outer parameter. (where outer means that the parameter does not originate from a ctfe-evaluation further up the call stack) Cheers, Stefan
It turn out that this was not the problem. It was rather old code in the byte-code evaluator that dated from a time where parameters were not considered stack values. causing that code to copy the parameters into a temporary. When I removed the distinction between parameters and normal stack values I deleted the code that kept track of the parameters that were copied into temporaries. Since I forgot to remove the creation of those temporaries. We ended up with the situation described above.
Mar 01
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Time for an update. I am currently working on integrating 64bit values into codegen API. However, a backend may not have native 64bit registers or arithmetic (the x86/arm architectures come to mind) For those a common fallback is to be implemented such that not every architecture has to implement their 64bit arithmetic on their own. Also work is underway to finally support slicing, which is crucial to using phobos algorithms.
Mar 10
next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Mar 10, 2017 at 11:32:14AM +0000, Stefan Koch via Digitalmars-d wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Time for an update. I am currently working on integrating 64bit values into codegen API. However, a backend may not have native 64bit registers or arithmetic (the x86/arm architectures come to mind) For those a common fallback is to be implemented such that not every architecture has to implement their 64bit arithmetic on their own.
Makes sense to me.
 Also work is underway to finally support slicing, which is crucial to
 using phobos algorithms.
This is awesome news! Glad to hear there's steady progress being made on the new CTFE engine. Can't wait for it to be done and merged into master! There are so many awesome things I wanna do with CTFE that currently would be unacceptably slow. Can't wait to take D compile-time capabilities to a whole 'nother level! T -- "A one-question geek test. If you get the joke, you're a geek: Seen on a California license plate on a VW Beetle: 'FEATURE'..." -- Joshua D. Wachs - Natural Intelligence, Inc.
Mar 10
prev sibling next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 10 March 2017 at 11:32:14 UTC, Stefan Koch wrote:
 Also work is underway to finally support slicing, which is 
 crucial to using phobos algorithms.
Incredible diligence.
Mar 10
prev sibling parent reply Kagamin <spam here.lot> writes:
On Friday, 10 March 2017 at 11:32:14 UTC, Stefan Koch wrote:
 I am currently working on integrating 64bit values into codegen 
 API.
How can dmd generate 64-bit code if backend doesn't support 64bit values?
Mar 13
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 13 March 2017 at 11:53:59 UTC, Kagamin wrote:
 On Friday, 10 March 2017 at 11:32:14 UTC, Stefan Koch wrote:
 I am currently working on integrating 64bit values into 
 codegen API.
How can dmd generate 64-bit code if backend doesn't support 64bit values?
I was referring to my newCTFE Bytecode-codegen. It is separate from dmd's backend. Just for completeness sake, it is perfectly possible to generate x86_64 code without supporting 64bit values, you will just have alot of zeros in the upper bits :) dmd does support 64bit values of course.
Mar 13
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
Basic Slicing support is in; The following code will now compile with the newCTFE_slicing branch: static immutable uint[] OneToTen = [1,2,3,4,5,6,7,8,9,10]; const(uint[]) SliceOf1to10(uint lwr, uint upr) { return OneToTen[lwr .. upr]; } static assert(SliceOf1to10(2,8) == [3u, 4u, 5u, 6u, 7u, 8u]); Contrary to the old interpreter no copies are made. newCTFE slices reference memory just as runtime slices do. Cheers, Stefan;
Mar 11
next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Saturday, 11 March 2017 at 14:39:54 UTC, Stefan Koch wrote:
 Basic Slicing support is in;
 The following code will now compile with the newCTFE_slicing 
 branch:
I guess this should greatly benefit compile-time parsing, right?
Mar 11
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 11 March 2017 at 17:38:04 UTC, Nordlöw wrote:
 On Saturday, 11 March 2017 at 14:39:54 UTC, Stefan Koch wrote:
 Basic Slicing support is in;
 The following code will now compile with the newCTFE_slicing 
 branch:
I guess this should greatly benefit compile-time parsing, right?
As soon as the UTF8 stuff works properly, yes. Slicing is really just syntactic sugar :)
Mar 11
prev sibling next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Mar 11, 2017 at 02:39:54PM +0000, Stefan Koch via Digitalmars-d wrote:
[...]
 Contrary to the old interpreter no copies are made.
 newCTFE slices reference memory just as runtime slices do.
[...] Awesome! This alone ought to greatly expand the scope of what's feasible in CTFE. Now I really can't wait to see this merged into master. :-) T -- English has the lovely word "defenestrate", meaning "to execute by throwing someone out a window", or more recently "to remove Windows from a computer and replace it with something useful". :-) -- John Cowan
Mar 11
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 11 March 2017 at 14:39:54 UTC, Stefan Koch wrote:
 [ ... Slice Support ... ]
Hi Guys, Since Slice support required an ABI there were a few bugs. Interestingly those bugs where there for a very long time :) The Type-handling I chose uses an index into a specific type-array to represent types. When this index is 0 we consider the type invalid. However I had an off by one bug in the check, causing the first TypeInstance of static arrays to be considered invalid. As an Invalid type the array has the size 0. Which in turn causes the Allocation for that array to allocate zero bytes for it. This zero allocation returns a valid pointer to the current top of the heap. (just without reserving any memory). Then when we slice the array the slice-descriptor has to go onto the heap. And it overwrites the array-descriptor which was allocated with zero size. causing it to point a to a bogus address which is equivalent to the array length. My Sunday was ruined before it began :)
Mar 12
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]
Hey Stefan, you have been working on this for many months now: is someone funding your work? If not, you should put up a bitcoin address or something, I know I'd contribute.
Mar 13
next sibling parent Meta <jared771 gmail.com> writes:
On Monday, 13 March 2017 at 15:49:35 UTC, Joakim wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]
Hey Stefan, you have been working on this for many months now: is someone funding your work? If not, you should put up a bitcoin address or something, I know I'd contribute.
There's Patreon, but they don't support bitcoin currently.
Mar 13
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 13 March 2017 at 15:49:35 UTC, Joakim wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]
Hey Stefan, you have been working on this for many months now: is someone funding your work? If not, you should put up a bitcoin address or something, I know I'd contribute.
Hi Joakim, My work does get sponsored by the D Language Foundation. Therefore any Donations best be directed to them. If someone wants to support my work specifically it's best to contact me personally. I have been thinking about setting up a Patreon account and decided it was too much trouble; considering that there is not a large number of people who even know my of work. Cheers, Stefan
Mar 13
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
I just fixed a bug in ref support; It should now work with slices. I also made it _slightly_ easier to make ABI changes for slice, since I expect the structure to change as soon as concat support is added. (e.g. slices will get a refcount, capacity and an origin ptr)
Mar 13
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 13 March 2017 at 19:40:30 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
I just fixed a bug in ref support; It should now work with slices. I also made it _slightly_ easier to make ABI changes for slice, since I expect the structure to change as soon as concat support is added. (e.g. slices will get a refcount, capacity and an origin ptr)
Bad news. Array expansion via assignment to length regressed. Fixing this is surprisingly time intensive. .... I am just not seeing where it's going wrong. It seems to use completely bogus offsets ... causing it to read from uninitialized memory.
Mar 14
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Mar 14, 2017 at 03:26:44PM +0000, Stefan Koch via Digitalmars-d wrote:
[...]
 Bad news.
 Array expansion via assignment to length regressed.
 Fixing this is surprisingly time intensive.
 .... I am just not seeing where it's going wrong.
 
 It seems to use completely bogus offsets ... causing it to read from
 uninitialized memory.
Sounds like there's a pointer bug / stack overflow / buffer overflow somewhere. Just my gut feeling from having faced similar bugs in my career. Unfortunately, these kinds of bugs are usually very difficult to trace, because the root cause can be very far away from where the symptoms show up, and can come from completely unrelated code. One way that sometimes works (but not always) is to try to shuffle the stack by moving functions / local variables around to see if the symptoms change. That may yield some clues as to the nature of the problem. But that's just a shot in the dark... generally these kinds of bugs are very hard to trace. Or maybe carefully step through the code starting from the length assignment in a debugger and see if any of the variables seem to have strange values. Sometimes the code immediately following is fine (inserting printf's of the buffer may indicate correct values) but it's something that happens afterwards that screws it up. Or, possibly, the state is already messed up before the length assignment... in which case it would be far more difficult to trace. :-( T -- If lightning were to ever strike an orchestra, it'd always hit the conductor first.
Mar 14
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 14 March 2017 at 16:42:01 UTC, H. S. Teoh wrote:
 [ ... ]
H.S.Teoh Thanks for your advice. Since I am running my own bytecode I am quite sure that there is no memory corruption going on. It has to do with the recent ABI changes. -- I just made strings use the same representation as every other slice, Meaning soon we can use std.utf.decode for correct string-foreach :) After that it is not a big step to supporting string concat.
Mar 14
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 3/14/17 4:26 PM, Stefan Koch wrote:
 On Monday, 13 March 2017 at 19:40:30 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
I just fixed a bug in ref support; It should now work with slices. I also made it _slightly_ easier to make ABI changes for slice, since I expect the structure to change as soon as concat support is added. (e.g. slices will get a refcount, capacity and an origin ptr)
Bad news. Array expansion via assignment to length regressed. Fixing this is surprisingly time intensive. .... I am just not seeing where it's going wrong. It seems to use completely bogus offsets ... causing it to read from uninitialized memory.
Time to try valgrind ? Last time I checked it worked just fine with D. --- Dmitry Olshansky
Mar 14
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 14 March 2017 at 21:53:29 UTC, Dmitry Olshansky wrote:
 On 3/14/17 4:26 PM, Stefan Koch wrote:
 On Monday, 13 March 2017 at 19:40:30 UTC, Stefan Koch wrote:
 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
I just fixed a bug in ref support; It should now work with slices. I also made it _slightly_ easier to make ABI changes for slice, since I expect the structure to change as soon as concat support is added. (e.g. slices will get a refcount, capacity and an origin ptr)
Bad news. Array expansion via assignment to length regressed. Fixing this is surprisingly time intensive. .... I am just not seeing where it's going wrong. It seems to use completely bogus offsets ... causing it to read from uninitialized memory.
Time to try valgrind ? Last time I checked it worked just fine with D. --- Dmitry Olshansky
When I talk about uninitialized memory I mean memory inside the VM, which is untouched. The Problem is a problem with bytecode generation, not the execution. The code executes as it should, but the generated code is invalid.
Mar 14
prev sibling parent Moritz Maxeiner <moritz ucworks.org> writes:
On Tuesday, 14 March 2017 at 21:53:29 UTC, Dmitry Olshansky wrote:
 [...]

 Time to try valgrind ? Last time I checked it worked just fine 
 with D.
Valgrind 3.12.0, at least, works perfectly with dmd&ldc. One just has to remember to suppress the (static) memory leak in druntime[1] [1] https://github.com/dlang/druntime/pull/1557
Mar 14
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
 [ ... ]
String Slicing is in! The following code will now compile with newCTFE. string slice(string s, uint lwr, uint upr) { return s[lwr .. upr]; } static assert(slice("Hello World", 6, 11) == "World");
Mar 15
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 15 March 2017 at 13:49:43 UTC, Stefan Koch wrote:
 String Slicing is in!
Great! What remains?
Mar 15
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 15 March 2017 at 15:49:59 UTC, Nordlöw wrote:
 On Wednesday, 15 March 2017 at 13:49:43 UTC, Stefan Koch wrote:
 String Slicing is in!
Great! What remains?
Unsupported Features include : - Unicode/Auto-decoding support. NOTE: foreach(char c; string) { ... } will work since it does not auto-decode. - Floating point. - Classes - unions - structs containing other structs. - arrays of structs - arrays of arrays - || and && - ref return (though I doubt it's needed) - references of 64bit values And maybe other I cannot think of right now.
Mar 15
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 15 March 2017 at 15:59:46 UTC, Stefan Koch wrote:
 On Wednesday, 15 March 2017 at 15:49:59 UTC, Nordlöw wrote:
 On Wednesday, 15 March 2017 at 13:49:43 UTC, Stefan Koch wrote:
 String Slicing is in!
Great! What remains?
Unsupported Features include : - Unicode/Auto-decoding support. NOTE: foreach(char c; string) { ... } will work since it does not auto-decode. - Floating point. - Classes - unions - structs containing other structs. - arrays of structs - arrays of arrays - || and && - ref return (though I doubt it's needed) - references of 64bit values And maybe other I cannot think of right now.
slice expansion. and string-foreach caused dmd to miscomplie itself ... *sigh* So no string-foreach for now;
Mar 15
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 15 March 2017 at 16:11:44 UTC, Stefan Koch wrote:
 and string-foreach caused dmd to miscomplie itself ... *sigh*
Guess what ? Things work so much better when you allocate space for the zero terminator :) The segfaults are mysteriously gone :D
Mar 15
prev sibling next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 15 March 2017 at 15:59:46 UTC, Stefan Koch wrote:
 Unsupported Features include :

  - Unicode/Auto-decoding support.
 NOTE: foreach(char c; string) { ... } will work since it does 
 not auto-decode.

  - Floating point.
  - Classes
  - unions
  - structs containing other structs.
  - arrays of structs
  - arrays of arrays
  - || and &&
  - ref return (though I doubt it's needed)
  - references of 64bit values

 And maybe other I cannot think of right now.
How long do you estimate this will take? Can/Will some of your work on newCTFE-branch be merged onto master before _all_ these things have been fixed?
Mar 15
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 15 March 2017 at 19:00:21 UTC, Nordlöw wrote:
 On Wednesday, 15 March 2017 at 15:59:46 UTC, Stefan Koch wrote:
 Unsupported Features include :

  - Unicode/Auto-decoding support.
 NOTE: foreach(char c; string) { ... } will work since it does 
 not auto-decode.

  - Floating point.
  - Classes
  - unions
  - structs containing other structs.
  - arrays of structs
  - arrays of arrays
  - || and &&
  - ref return (though I doubt it's needed)
  - references of 64bit values

 And maybe other I cannot think of right now.
How long do you estimate this will take? Can/Will some of your work on newCTFE-branch be merged onto master before _all_ these things have been fixed?
Yes the plan is to forgo Classes Execptions and FP for newCTFE 1.0.
Mar 15
prev sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Mar 15, 2017 at 03:59:46PM +0000, Stefan Koch via Digitalmars-d wrote:
[...]
 Unsupported Features include :
[...]
  - Floating point.
[...]
  - unions
[...] What are the chances / what's the expected timeframe of unions being implemented? Support for unions in CTFE is a MAJOR milestone in making std.math CTFE-able, which IMO will take D compile-time capabilities to a whole new level, because it will greatly expand the scope of what's computable at compile-time in terms of floating-point constants, lookup tables, etc.. Imagine, for example, a precomputed table of values transcendental functions with some given resolution, for fast runtime lookups. (Of course, this also requires floating-point support in CTFE. But that should be relatively easy(?). As long as both the host and target architectures support the same set of IEEE floating-point types, which is probably the case for our currently-supported platforms.) T -- People say I'm indecisive, but I'm not sure about that. -- YHL, CONLANG
Mar 15
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 15 March 2017 at 18:52:16 UTC, H. S. Teoh wrote:
 On Wed, Mar 15, 2017 at 03:59:46PM +0000, Stefan Koch via 
 Digitalmars-d wrote: [...]
 Unsupported Features include :
[...]
  - Floating point.
[...]
  - unions
[...] What are the chances / what's the expected timeframe of unions being implemented? Support for unions in CTFE is a MAJOR milestone in making std.math CTFE-able, which IMO will take D compile-time capabilities to a whole new level, because it will greatly expand the scope of what's computable at compile-time in terms of floating-point constants, lookup tables, etc.. Imagine, for example, a precomputed table of values transcendental functions with some given resolution, for fast runtime lookups. (Of course, this also requires floating-point support in CTFE. But that should be relatively easy(?). As long as both the host and target architectures support the same set of IEEE floating-point types, which is probably the case for our currently-supported platforms.) T
I am not sure. The CTFE-ABI is different from the ABI at runtime. So more adventurous uses of unions are likely to be surprising. As for floating point, I have yet to find a solution that will work for more numerically inclined people. dmds constant folder apparently does some funky things in that domain as well. I do not anticipate to have any of the fp stuff working before dconf. Currently Unicode correct string-handling is more important. After that comes class support. FP is the very last thing on my list and will like take a lot of time.
Mar 15