www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - CTFE Status

reply Stefan Koch <uplink.coder googlemail.com> writes:
Hi Guys, since I got a few complaints about giving minor status 
updates in the announce group, I am opening this thread.

I will start with giving an overview of what works and what does 
not work.

Currently the only basic type you can do arithmetic on is int.
Altough you can compare longs since a few days.

These are the constructs that will work.

- foreach on static arrays strings and range-foreach  (those 
kinds (0 .. 64)).
- switches (even deeply nested ones)
- for and while loops
- ternary expressions (? :)
- if and else statements (as long as you don't use && and || )
- lables and gotos
- arithmetic expressions as well as post and pre increment and 
decrement

Constructs that will not work (but are actively worked on)

- assignment to static array cells
- long ulong arithmetic.
- function calls
- dynamic arrays and slices
- pointers
- structs
- && and ||
- sliceing

Constructs that will not work and are futher down the list.

- classes
- closures
- boundschecks
- asserts

Please note that there will probably be bugs all over the place.
So even the working features might not be working completely.
Oct 31 2016
next sibling parent Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: CTFE Status
References: <tdhdsvmknnpiugnngkex forum.dlang.org>
In-Reply-To: <tdhdsvmknnpiugnngkex forum.dlang.org>

--iKI927Tn283L8H4qPVenp66IJJbESsVN5
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Thank you and keep doing awesome stuff ;)


--iKI927Tn283L8H4qPVenp66IJJbESsVN5--
Oct 31 2016
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 10/31/16 9:29 AM, Stefan Koch wrote:
 Hi Guys, since I got a few complaints about giving minor status updates
 in the announce group, I am opening this thread.

 I will start with giving an overview of what works and what does not work.
Awesome work, I can't wait for efficient streamlined CTFE. Note to those not aware, Stefan is working on replacing the CTFE engine in D with a bytecode generated one. So although he mentions things that "don't work", he's talking about them in the context of the new CTFE engine. Many of these things already work in current CTFE implementation. -Steve
Oct 31 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 31 October 2016 at 20:49:58 UTC, Steven Schveighoffer 
wrote:
 On 10/31/16 9:29 AM, Stefan Koch wrote:
 Hi Guys, since I got a few complaints about giving minor 
 status updates
 in the announce group, I am opening this thread.

 I will start with giving an overview of what works and what 
 does not work.
Awesome work, I can't wait for efficient streamlined CTFE. Note to those not aware, Stefan is working on replacing the CTFE engine in D with a bytecode generated one. So although he mentions things that "don't work", he's talking about them in the context of the new CTFE engine. Many of these things already work in current CTFE implementation. -Steve
All of them work in current CTFE which is why the mark is quite high. I am glad to hear you guys appriciate my work. It does help a ton to hear it from time to time. --- Very recent news: NEW CTFE PASSES THE DRUNTIME UNITESTS!!!!
Nov 01 2016
next sibling parent Daniel Kozak via Digitalmars-d <digitalmars-d puremagic.com> writes:
Dne 1.11.2016 v 16:50 Stefan Koch via Digitalmars-d napsal(a):

 On Monday, 31 October 2016 at 20:49:58 UTC, Steven Schveighoffer wrote:
 On 10/31/16 9:29 AM, Stefan Koch wrote:
 Hi Guys, since I got a few complaints about giving minor status updates
 in the announce group, I am opening this thread.

 I will start with giving an overview of what works and what does not 
 work.
Awesome work, I can't wait for efficient streamlined CTFE. Note to those not aware, Stefan is working on replacing the CTFE engine in D with a bytecode generated one. So although he mentions things that "don't work", he's talking about them in the context of the new CTFE engine. Many of these things already work in current CTFE implementation. -Steve
All of them work in current CTFE which is why the mark is quite high. I am glad to hear you guys appriciate my work. It does help a ton to hear it from time to time. --- Very recent news: NEW CTFE PASSES THE DRUNTIME UNITESTS!!!!
This is so awesome, really good work :)
Nov 01 2016
prev sibling next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 1 November 2016 at 15:50:06 UTC, Stefan Koch wrote:
 Very recent news:
  NEW CTFE PASSES THE DRUNTIME UNITESTS!!!!
Great work!
Nov 01 2016
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 1 November 2016 at 15:50:06 UTC, Stefan Koch wrote:

 Very recent news:
  NEW CTFE PASSES THE DRUNTIME UNITESTS!!!!
Now a phobos unittest miscompiles :( Again passing the unittests does not mean too much. I just means I bail out before I generate invalid code :)
Nov 01 2016
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 1 November 2016 at 19:28:03 UTC, Stefan Koch wrote:
 Now a phobos unittest miscompiles :(

 Again passing the unittests does not mean too much.
 I just means I bail out before I generate invalid code :)
Keep up!
Nov 01 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 1 November 2016 at 21:12:06 UTC, Nordlöw wrote:
 On Tuesday, 1 November 2016 at 19:28:03 UTC, Stefan Koch wrote:
 Now a phobos unittest miscompiles :(

 Again passing the unittests does not mean too much.
 I just means I bail out before I generate invalid code :)
Keep up!
I am now passing phobos unittests! However please take it with a grain of salt ;) The following line resbonsible for baling out of functions I know are failing. if (fd.ident == Identifier.idPool("isRooted") || fd.ident == Identifier.idPool("__lambda2") || fd.ident == Identifier.idPool("divideRoundUp") || fd.ident == Identifier.idPool("isSameLength") || fd.ident == Identifier.idPool("wrapperParameters") || fd.ident == Identifier.idPool("wrap") || fd.ident == Identifier.idPool("args")) { IGaveUp = true; return ; } I am working on reducing test-cases to figure out why this is failing. The code is burried in templates makeing it hard to see what is going on. Still passing the tests is passing the tests :)
Nov 02 2016
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 2 November 2016 at 11:01:55 UTC, Stefan Koch wrote:
 I am now passing phobos unittests!
I can't wait to see this in action! Keep on going and I'll keep praying... 😊
Nov 02 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 2 November 2016 at 23:48:46 UTC, Nordlöw wrote:
 On Wednesday, 2 November 2016 at 11:01:55 UTC, Stefan Koch 
 wrote:
 I am now passing phobos unittests!
I can't wait to see this in action! Keep on going and I'll keep praying... 😊
No need for prayer. I am trying my best to be doing solid engineering work. Neither god nor the devil will paly any role. I am currently working on making the DMD-AST-facing side of the code more solid. With a bit of luck I can role out a preview release soon. But I will only do so when there are no known bugs left.
Nov 02 2016
parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 3 November 2016 at 00:02:46 UTC, Stefan Koch wrote:
 No need for prayer.

 I am trying my best to be doing solid engineering work.

 Neither god nor the devil will paly any role.
Got it.
Nov 03 2016
prev sibling next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 31 October 2016 at 13:29:01 UTC, Stefan Koch wrote:
 These are the constructs that will work.

 - foreach on static arrays strings and range-foreach  (those 
 kinds (0 .. 64)).
 - switches (even deeply nested ones)
 - for and while loops
 - ternary expressions (? :)
 - if and else statements (as long as you don't use && and || )
 - lables and gotos
 - arithmetic expressions as well as post and pre increment and 
 decrement

 Constructs that will not work (but are actively worked on)

 - assignment to static array cells
 - long ulong arithmetic.
 - function calls
 - dynamic arrays and slices
 - pointers
 - structs
 - && and ||
 - sliceing

 Constructs that will not work and are futher down the list.

 - classes
 - closures
 - boundschecks
 - asserts
Do all these work in current implementation?
Nov 01 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 1 November 2016 at 16:44:14 UTC, Nordlöw wrote:
 On Monday, 31 October 2016 at 13:29:01 UTC, Stefan Koch wrote:
 These are the constructs that will work.

 - foreach on static arrays strings and range-foreach  (those 
 kinds (0 .. 64)).
 - switches (even deeply nested ones)
 - for and while loops
 - ternary expressions (? :)
 - if and else statements (as long as you don't use && and || )
 - lables and gotos
 - arithmetic expressions as well as post and pre increment and 
 decrement

 Constructs that will not work (but are actively worked on)

 - assignment to static array cells
 - long ulong arithmetic.
 - function calls
 - dynamic arrays and slices
 - pointers
 - structs
 - && and ||
 - sliceing

 Constructs that will not work and are futher down the list.

 - classes
 - closures
 - boundschecks
 - asserts
Do all these work in current implementation?
Yes of they do! There is a lot of work behind the implementation! I intend to keep the current implemntation around as a fallback. For things that are used rarely. like floating point and others :)
Nov 01 2016
parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Tuesday, 1 November 2016 at 17:41:35 UTC, Stefan Koch wrote:
 I intend to keep the current implemntation around as a fallback.
 For things that are used rarely.
 like floating point and others :)
Is that a good idea? It means that there will be tons of really complex code that is rarely if ever used and therefore gets no testing...
Nov 03 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 3 November 2016 at 15:44:20 UTC, Marc SchĂŒtz wrote:
 On Tuesday, 1 November 2016 at 17:41:35 UTC, Stefan Koch wrote:
 I intend to keep the current implemntation around as a 
 fallback.
 For things that are used rarely.
 like floating point and others :)
Is that a good idea? It means that there will be tons of really complex code that is rarely if ever used and therefore gets no testing...
Passing the whole test-suite is as good as it gets.
Nov 03 2016
parent Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Thursday, 3 November 2016 at 16:35:08 UTC, Stefan Koch wrote:
 On Thursday, 3 November 2016 at 15:44:20 UTC, Marc SchĂŒtz wrote:
 On Tuesday, 1 November 2016 at 17:41:35 UTC, Stefan Koch wrote:
 I intend to keep the current implemntation around as a 
 fallback.
 For things that are used rarely.
 like floating point and others :)
Is that a good idea? It means that there will be tons of really complex code that is rarely if ever used and therefore gets no testing...
Passing the whole test-suite is as good as it gets.
But for operations that your new engine supports, the test-suite will not normally test the old engine, right? Except in certain edge cases where that the new engine happens not to support, so it falls back to the old one. But these edge cases are probably not well-defined and hard to test exhaustively...
Nov 04 2016
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Thu, 03 Nov 2016 15:44:20 +0000, Marc SchĂŒtz wrote:

 On Tuesday, 1 November 2016 at 17:41:35 UTC, Stefan Koch wrote:
 I intend to keep the current implemntation around as a fallback.
 For things that are used rarely.
 like floating point and others :)
Is that a good idea? It means that there will be tons of really complex code that is rarely if ever used and therefore gets no testing...
I think the intent is that the current implementation will always be used for specific features, not that the compiler will attempt to run CTFE with the bytecode version and resort to the previous implementation if something goes wrong.
Nov 03 2016
parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Friday, 4 November 2016 at 01:19:36 UTC, Chris Wright wrote:
 On Thu, 03 Nov 2016 15:44:20 +0000, Marc SchĂŒtz wrote:

 On Tuesday, 1 November 2016 at 17:41:35 UTC, Stefan Koch wrote:
 I intend to keep the current implemntation around as a 
 fallback.
 For things that are used rarely.
 like floating point and others :)
Is that a good idea? It means that there will be tons of really complex code that is rarely if ever used and therefore gets no testing...
I think the intent is that the current implementation will always be used for specific features, not that the compiler will attempt to run CTFE with the bytecode version and resort to the previous implementation if something goes wrong.
In this case better make sure to remove the resulting dead code from the old interpreter, otherwise it will become and unmaintainable mess in the long run.
Nov 04 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 4 November 2016 at 10:59:25 UTC, Marc SchĂŒtz wrote:
 
 In this case better make sure to remove the resulting dead code 
 from the old interpreter, otherwise it will become and 
 unmaintainable mess in the long run.
If you ask me the old interpreter is already an unmaintainble mess. It will not be possible to remove code since the old interpreter and the new one do not share any common state or code-paths. Therefore all evaluation-capabilty the old interpreter has must be retained otherwise it would not be useful anymore. I thought about it some more, and decided that I will replace the old interpreter completely in the long run. However in order to get something release before 2017, I have to rely on it being there. I recently lost 3 days of work because of my git-skills. Now I am working on simplyfing the cases the interpreter can get into. Before I would do some optimisation to avoid temporaries and the like. Since I now have the llvm-backend to verify the code is correct it's no longer crucial in order to be able to debug. This simplification will (hopefully) finally enable me to proper handling of boolean-expressions. I suspect that most of the invalid code I generated came from a bug in the "temporaries-avoidance optimisation" Also I will put some effort into documenting the new engine better. However code-gen bugs take priority over documentation.
Nov 04 2016
next sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Saturday, November 05, 2016 01:21:48 Stefan Koch via Digitalmars-d wrote:
 I thought about it some more, and decided that I will replace the
 old interpreter completely in the long run.
 However in order to get something release before 2017, I have to
 rely on it being there.
That seems reasonable, but it definitely seems messy to leave the old stuff there in the long run, especially if only part of it is really used and tested. - Jonathan M Davis
Nov 05 2016
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 06/11/2016 1:57 AM, Jonathan M Davis via Digitalmars-d wrote:
 On Saturday, November 05, 2016 01:21:48 Stefan Koch via Digitalmars-d wrote:
 I thought about it some more, and decided that I will replace the
 old interpreter completely in the long run.
 However in order to get something release before 2017, I have to
 rely on it being there.
That seems reasonable, but it definitely seems messy to leave the old stuff there in the long run, especially if only part of it is really used and tested. - Jonathan M Davis
Based upon some help I gave Stefan early on with the current implementation, it will be left alone as is. It doesn't need to change in any way beyond the entry point. So not at all messy and it certainly is already well tested. Especially since it is mostly localized to a single file.
Nov 05 2016
prev sibling next sibling parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Saturday, 5 November 2016 at 01:21:48 UTC, Stefan Koch wrote:
 I thought about it some more, and decided that I will replace 
 the old interpreter completely in the long run.
 However in order to get something release before 2017, I have 
 to rely on it being there.
That's better of course. Parallel use of new and old just shouldn't stay there indefinitely. Look at the problems with the AA implementation as a similar example.
 I recently lost 3 days of work because of my git-skills.
Unless you haven't committed your work yet, almost everything in Git can be undone. Make a copy of your entire project directory (including .git) and then have a look at `git reflog` around the time the disaster happened. It will show you commit IDs that you can check out.
Nov 05 2016
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Saturday, 5 November 2016 at 15:48:19 UTC, Marc SchĂŒtz wrote:
 I recently lost 3 days of work because of my git-skills.
Unless you haven't committed your work yet, almost everything in Git can be undone. Make a copy of your entire project directory (including .git) and then have a look at `git reflog` around the time the disaster happened. It will show you commit IDs that you can check out.
Solid advice right there ^
Nov 05 2016
prev sibling next sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Saturday, 5 November 2016 at 15:48:19 UTC, Marc SchĂŒtz wrote:
 Unless you haven't committed your work yet, almost everything 
 in Git can be undone. Make a copy of your entire project 
 directory (including .git) and then have a look at `git reflog` 
 around the time the disaster happened. It will show you commit 
 IDs that you can check out.
especially if one will turn off automatic git gc.
Nov 05 2016
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 11/05/2016 11:48 AM, Marc SchĂŒtz wrote:
 On Saturday, 5 November 2016 at 01:21:48 UTC, Stefan Koch wrote:
 I recently lost 3 days of work because of my git-skills.
Unless you haven't committed your work yet, almost everything in Git can be undone. Make a copy of your entire project directory (including .git) and then have a look at `git reflog` around the time the disaster happened. It will show you commit IDs that you can check out.
Yea, but unless you're a git-fu master, sometimes figuring out how to fix whatever got messed up can lose you 3 days of work ;) I really want to make a saner CLI front-end for git, but that would require learning more about git than I really ever want to know :(
Nov 08 2016
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Nov 08, 2016 at 11:40:31AM -0500, Nick Sabalausky via Digitalmars-d
wrote:
next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 11/08/2016 11:57 AM, H. S. Teoh via Digitalmars-d wrote:
 The thing about git is that at its core, it's really very simple. Dumb,
 even.  It's basically a program for managing a directed acyclic graph
 (DAG).  That's all there is to it. The rest is just frills.

 Trying to rationalize git in terms of traditional version control
 systems is what usually causes lots of confusion, incomprehension, and
 frustration.  To truly grok git, you have to just forget about
 traditional version control concepts, and think purely in terms of DAGs.
 Once you do, everything falls into place and it all makes sense in its
 own peculiar way (including all the flaws :-P).
I don't suppose you have a handy link to an "Understanding Git as a DAG manager instead of VCS" document?
Nov 08 2016
next sibling parent Antonio Corbi <amcb ggmail.com> writes:
On Tuesday, 8 November 2016 at 17:12:34 UTC, Nick Sabalausky 
wrote:
 On 11/08/2016 11:57 AM, H. S. Teoh via Digitalmars-d wrote:
[...]
I don't suppose you have a handy link to an "Understanding Git as a DAG manager instead of VCS" document?
Hi Nick, Maybe this one is useful for you: http://eagain.net/articles/git-for-computer-scientists/ Antonio
Nov 08 2016
prev sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Nov 08, 2016 at 12:12:34PM -0500, Nick Sabalausky via Digitalmars-d
wrote:
 On 11/08/2016 11:57 AM, H. S. Teoh via Digitalmars-d wrote:
 
 The thing about git is that at its core, it's really very simple.
 Dumb, even.  It's basically a program for managing a directed
 acyclic graph (DAG).  That's all there is to it. The rest is just
 frills.
 
 Trying to rationalize git in terms of traditional version control
 systems is what usually causes lots of confusion, incomprehension,
 and frustration.  To truly grok git, you have to just forget about
 traditional version control concepts, and think purely in terms of
 DAGs.  Once you do, everything falls into place and it all makes
 sense in its own peculiar way (including all the flaws :-P).
 
I don't suppose you have a handy link to an "Understanding Git as a DAG manager instead of VCS" document?
Nothing immediately comes to mind, but thanks to Dr. Google, I found this page that's sorta helpful: http://ericsink.com/vcbe/html/directed_acyclic_graphs.html And perhaps these: http://eagain.net/articles/git-for-computer-scientists/ http://marklodato.github.io/visual-git-guide/index-en.html Or maybe I should write my own article about git being based on a DAG. :-P T -- Leather is waterproof. Ever see a cow with an umbrella?
Nov 08 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Back to CTFE.
The 64Bit bug is fixed.
And now there is one bailout-line less.

I am onto the next bug.
Let's hunt them all down!
Nov 08 2016
next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 8 November 2016 at 21:56:32 UTC, Stefan Koch wrote:
 I am onto the next bug.
 Let's hunt them all down!
Marvellous!
Nov 08 2016
prev sibling next sibling parent deadalnix <deadalnix gmail.com> writes:
On Tuesday, 8 November 2016 at 21:56:32 UTC, Stefan Koch wrote:
 Back to CTFE.
 The 64Bit bug is fixed.
 And now there is one bailout-line less.

 I am onto the next bug.
 Let's hunt them all down!
https://media.giphy.com/media/FmNXeuoadNTpe/giphy.gif
Nov 08 2016
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 8 November 2016 at 21:56:32 UTC, Stefan Koch wrote:
 Back to CTFE.
 The 64Bit bug is fixed.
 And now there is one bailout-line less.

 I am onto the next bug.
 Let's hunt them all down!
More 64bit bugs appeared that were shadowed by the one I fixed. Also a new bugs related to labeled breaks appeared.
Nov 09 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
After the bad news now a bit of good news.
I just finished the strCat algorithm;

The reason this took so much time is because of the 
representation the heap has in the elevator.
it is a uint[] meaning you are getting into trouble with offsets 
if the first string is not a multiple of 4 characters long:)
This is the code for strcat :

uint[] strcat(const uint[] a, const uint[] b, uint aLength = 0, 
uint bLength = 0)
{
   aLength = a[0];
   bLength = b[0];
   uint resultLength = a[0] + b[0];
   uint[] result;
   result.length = neededArraySize(resultLength);
   assert(result.length == (a.length + b.length) - 2);
   result[0] = resultLength;
   auto resultPosition = 1;

   auto aMinusOne = !(aLength & 3);
   foreach(p, ca; a[1 .. $ - aMinusOne])
   {
     result[resultPosition++] = ca;
   }
   // this one is easy we just copy
     auto bMinusOne = !(bLength & 3);

   uint offset = aLength & 3;
   if (offset)
   {
     assert(offset == 2);
     resultPosition--;
     foreach(p, cb; b[1 .. $])
     {
       result[resultPosition++] |= (cb & ((1U << (offset*8)) -1)) 
<< (offset*8);
       if (resultPosition == result.length) break;
       result[resultPosition++] |= (cb & ~((1U << (offset*8)) -1)) 
 (offset*8);
resultPosition--; } } else { foreach(p, cb; b[1 .. $ - bMinusOne]) { result[resultPosition++] = cb; } } return result; } uint neededArraySize(uint size) { return (((size - 1) / 4) + 2) + (!(size & 3)); }
Nov 09 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 10 November 2016 at 01:17:48 UTC, Stefan Koch wrote:
 After the bad news now a bit of good news.
 I just finished the strCat algorithm;

 [...]
As expected this code is slightly wrong and will fail on the offset == 3 case. It will need a little bit of tinkering until it's ready to go :)
Nov 10 2016
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thursday, November 10, 2016 18:40:03 Stefan Koch via Digitalmars-d wrote:
 On Thursday, 10 November 2016 at 01:17:48 UTC, Stefan Koch wrote:
 After the bad news now a bit of good news.
 I just finished the strCat algorithm;

 [...]
As expected this code is slightly wrong and will fail on the offset == 3 case. It will need a little bit of tinkering until it's ready to go :)
And aren't you glad that you have all of these unit tests to verify that what you're working works properly? They don't necessarily catch everything, but it would be _way_ worse to completely replace something like this without them. :) The way that D promotes unit testing is definitely one of its strong points. - Jonathan M Davis
Nov 10 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 10 November 2016 at 20:15:49 UTC, Jonathan M Davis 
wrote:
 On Thursday, November 10, 2016 18:40:03 Stefan Koch via 
 Digitalmars-d wrote:
 On Thursday, 10 November 2016 at 01:17:48 UTC, Stefan Koch 
 wrote:
 After the bad news now a bit of good news.
 I just finished the strCat algorithm;

 [...]
As expected this code is slightly wrong and will fail on the offset == 3 case. It will need a little bit of tinkering until it's ready to go :)
And aren't you glad that you have all of these unit tests to verify that what you're working works properly? They don't necessarily catch everything, but it would be _way_ worse to completely replace something like this without them. :) The way that D promotes unit testing is definitely one of its strong points. - Jonathan M Davis
I love compile-time unitests. Who does not like them ? The interpret3.d test in the compiler acceptance testsuite is almost a work of art. After you pass that one you can be pretty sure to have a complainant d compiler. The current Status of stringCat can be found here : https://gist.github.com/UplinkCoder/fed000f5ca41ad3d65d59a7a63d5ef6f
Nov 10 2016
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 10 November 2016 at 20:51:01 UTC, Stefan Koch wrote:
 On Thursday, 10 November 2016 at 20:15:49 UTC, Jonathan M Davis 
 wrote:
 On Thursday, November 10, 2016 18:40:03 Stefan Koch via 
 Digitalmars-d wrote:
 On Thursday, 10 November 2016 at 01:17:48 UTC, Stefan Koch 
 wrote:
 After the bad news now a bit of good news.
 I just finished the strCat algorithm;

 [...]
As expected this code is slightly wrong and will fail on the offset == 3 case. It will need a little bit of tinkering until it's ready to go :)
And aren't you glad that you have all of these unit tests to verify that what you're working works properly? They don't necessarily catch everything, but it would be _way_ worse to completely replace something like this without them. :) The way that D promotes unit testing is definitely one of its strong points. - Jonathan M Davis
I love compile-time unitests. Who does not like them ? The interpret3.d test in the compiler acceptance testsuite is almost a work of art. After you pass that one you can be pretty sure to have a complainant d compiler. The current Status of stringCat can be found here : https://gist.github.com/UplinkCoder/fed000f5ca41ad3d65d59a7a63d5ef6f
StringConcat works and will be translated into bytecode soon. I am going to get a few big things done during next week. Namely support for dynamic arrays. At least I hope so :) You never know what bugs might rain on your programming parade.
Nov 11 2016
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Assignment to Array.length works now :)
the following code :

uint[] makeArr()
{
     uint[] arr;
     arr.length = 5;
     return arr;
}


pragma(msg, makeArr());

results in :

CTFE_DEBUG_MESSAGE : building Array of Length 5
[0u, 0u, 0u, 0u, 0u]
Nov 13 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 14 November 2016 at 05:27:15 UTC, Stefan Koch wrote:
 Assignment to Array.length works now :)
 the following code :

 uint[] makeArr()
 {
     uint[] arr;
     arr.length = 5;
     return arr;
 }


 pragma(msg, makeArr());

 results in :

 CTFE_DEBUG_MESSAGE : building Array of Length 5
 [0u, 0u, 0u, 0u, 0u]
Assignment now also works :) uint[] MakeAndInitArr(uint length) { uint[] arr; arr.length = length; foreach(i;0 .. length) { arr[i] = i + 3; } return arr; } pragma(msg, MakeAndInitArr(12)); Output : [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]
Nov 13 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 14 November 2016 at 07:57:20 UTC, Stefan Koch wrote:
 On Monday, 14 November 2016 at 05:27:15 UTC, Stefan Koch wrote:
 Assignment to Array.length works now :)
 the following code :

 uint[] makeArr()
 {
     uint[] arr;
     arr.length = 5;
     return arr;
 }


 pragma(msg, makeArr());

 results in :

 CTFE_DEBUG_MESSAGE : building Array of Length 5
 [0u, 0u, 0u, 0u, 0u]
Assignment now also works :) uint[] MakeAndInitArr(uint length) { uint[] arr; arr.length = length; foreach(i;0 .. length) { arr[i] = i + 3; } return arr; } pragma(msg, MakeAndInitArr(12)); Output : [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]
This for a length values in the range of short.max/4. Which is the maximal heapSize for now. This is up to 1.5 times faster then the old engine. Once the heap-limit is lifted you can see this rock :)
Nov 14 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
To more bullet points cross the border from unsupported to 
supported.
- assignment to static array cells
- long ulong arithmetic.

However there is a a bug inside the code that does 
bounds-checking for array assignment.
In rare cases it can trigger a out-bounds-error on newly created 
arrays.

Dynamic arrays work partially.

The only way to create them is to assign to length.
For initialization fill the cells in a for-loop.
Nov 14 2016
next sibling parent reply Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Tuesday, 15 November 2016 at 01:35:42 UTC, Stefan Koch wrote:
 Two more bullet points cross the border from unsupported to 
 supported.
Nice, Stefan. I always come back to this thread to see your progress. Thanks for making it easy to follow it.
Nov 15 2016
parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 15 November 2016 at 19:02:29 UTC, Bastiaan Veelo 
wrote:
 Nice, Stefan. I always come back to this thread to see your 
 progress. Thanks for making it easy to follow it.
I agree.
Nov 15 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 15 November 2016 at 01:35:42 UTC, Stefan Koch wrote:
 However there is a a bug inside the code that does 
 bounds-checking for array assignment.
 In rare cases it can trigger a out-bounds-error on newly 
 created arrays.
This raise all kind of red flags to me. What are the design decision that lead to this ?
Nov 15 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 15 November 2016 at 22:50:49 UTC, deadalnix wrote:
 On Tuesday, 15 November 2016 at 01:35:42 UTC, Stefan Koch wrote:
 However there is a a bug inside the code that does 
 bounds-checking for array assignment.
 In rare cases it can trigger a out-bounds-error on newly 
 created arrays.
This raise all kind of red flags to me. What are the design decision that lead to this ?
I am still figuring out when and why the bug is caused. The Byte-code for slice allocation is rather complex. Because it has to deal with resizing slices as well. I suspect that somewhere the heapPtr is not bumped or the length is not set correctly.
Nov 15 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 15 November 2016 at 23:46:51 UTC, Stefan Koch wrote:
 On Tuesday, 15 November 2016 at 22:50:49 UTC, deadalnix wrote:
 On Tuesday, 15 November 2016 at 01:35:42 UTC, Stefan Koch 
 wrote:
 However there is a a bug inside the code that does 
 bounds-checking for array assignment.
 In rare cases it can trigger a out-bounds-error on newly 
 created arrays.
This raise all kind of red flags to me. What are the design decision that lead to this ?
I am still figuring out when and why the bug is caused. The Byte-code for slice allocation is rather complex. Because it has to deal with resizing slices as well. I suspect that somewhere the heapPtr is not bumped or the length is not set correctly.
Indeed the length was not set on a code-path meant for resizeing. The problem is fixed :) The HeapLimit has been raised to a more reasonable limit of 2 ^^ 24 Addresses. (Which means that you'll have 2 ^^ 24 Bytes in practice.)
Nov 16 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 16 November 2016 at 09:22:01 UTC, Stefan Koch wrote:
 On Tuesday, 15 November 2016 at 23:46:51 UTC, Stefan Koch wrote:

 I suspect that somewhere the heapPtr is not bumped or the 
 length is not set correctly.
Indeed the length was not set on a code-path meant for resizeing. The problem is fixed :) The HeapLimit has been raised to a more reasonable limit of 2 ^^ 24 Addresses. (Which means that you'll have 2 ^^ 24 Bytes in practice.)
In fact, there is a single design decision that fosters these kinds of problems. And that is to go with a low-level IR. However although it's a tough route. It's also the only solution, (the only one I could think of), that will really enable CTFE to scale gracefully. My latest measurements show that even for relatively small arrays (2 ^^ 15) bytes. There is a 2x speedup. When the interpreter backend. As soon as my own jit backend is in place the performance will be a factor of 6 better. Makeing newCTFE 12x faster then the current engine, even on small one-shot functions!
Nov 16 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Here is a small demostration of the performance increase :

[root localhost dmd]# time src/dmd -c testSettingArrayLength.d > 
x 2> x

real	0m0.199s
user	0m0.180s
sys	0m0.017s
[root localhost dmd]# time src/dmd -c testSettingArrayLength.d 
-bc-ctfe > x 2> x

real	0m0.072s
user	0m0.050s
sys	0m0.020s

Please note that newCTFE only spends 15 ms inside the evaluation, 
most time is spent clearing the 2^^24 Bytes of heap-memory to 
zero.

The sourcode of testSettingArrayLength is
uint[] MakeAndInitArr(uint length)
{
     uint[] arr;
     arr.length = length;

     foreach(i;0 .. length)
     {
         arr[i] = i + 3;
     }
     return arr;
}

static assert(MakeAndInitArr(ushort.max).length == ushort.max);
Nov 16 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 16 November 2016 at 09:45:24 UTC, Stefan Koch wrote:
 Here is a small demostration of the performance increase :

 [root localhost dmd]# time src/dmd -c testSettingArrayLength.d
 x 2> x
real 0m0.199s user 0m0.180s sys 0m0.017s [root localhost dmd]# time src/dmd -c testSettingArrayLength.d -bc-ctfe > x 2> x real 0m0.072s user 0m0.050s sys 0m0.020s Please note that newCTFE only spends 15 ms inside the evaluation, most time is spent clearing the 2^^24 Bytes of heap-memory to zero. The sourcode of testSettingArrayLength is uint[] MakeAndInitArr(uint length) { uint[] arr; arr.length = length; foreach(i;0 .. length) { arr[i] = i + 3; } return arr; } static assert(MakeAndInitArr(ushort.max).length == ushort.max);
A more accurate breakdown : Initializing Heap: 18.6 ms Generating Bytecode: 1.2 ms Executing Bytecode: 13.2 ms Converting to CTFE-EXp: 9.1 ms For a second execution of the same function with the same arguments within the same file the numbers look like : Initializing Heap: 16.7 ms Generating Bytecode: 0.6 ms Executing Bytecode: 13.2 ms Converting to CTFE-EXp: 9.3 ms
Nov 16 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 16 November 2016 at 10:07:06 UTC, Stefan Koch wrote:
 A more accurate breakdown :

 Initializing Heap:     18.6 ms
 Generating Bytecode:    1.2 ms
 Executing Bytecode:    13.2 ms
 Converting to CTFE-EXp: 9.1 ms

 For a second execution of the same function with the same 
 arguments within the same file the numbers look like :

 Initializing Heap:     16.7 ms
 Generating Bytecode:    0.6 ms
 Executing Bytecode:    13.2 ms
 Converting to CTFE-EXp: 9.3 ms
The above numbers were obtained using a debug build made with dmd. The following numbers are from a optimized build with ldmd2 First Execution (cold cache) : Initializing Heap: 17.4 ms Generating Bytecode: 0.7 ms Executing Bytecode: 5.3 ms Converting to CTFE-EXp: 5.1 ms Second run (warmer cache) : Initializing Heap: 16.9 ms Generating Bytecode: 0.3 ms Executing Bytecode: 5.3 ms Converting to CTFE-EXp: 4.9 ms
Nov 16 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 16 November 2016 at 10:25:30 UTC, Stefan Koch wrote:

 First Execution (cold cache) :

 Initializing Heap:     17.4 ms
 Generating Bytecode:    0.7 ms
 Executing Bytecode:     5.3 ms
 Converting to CTFE-EXp: 5.1 ms

 Second run (warmer cache) :

 Initializing Heap:     16.9 ms
 Generating Bytecode:    0.3 ms
 Executing Bytecode:     5.3 ms
 Converting to CTFE-EXp: 4.9 ms
And Again a bit of bad news. Due to problems in the lowering of function arguments the implementation of strcat is delayed again.
Nov 16 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 16 November 2016 at 14:44:06 UTC, Stefan Koch wrote:
 And Again a bit of bad news.
 Due to problems in the lowering of function arguments the 
 implementation of strcat is delayed again.
The bug does not affect strings. Since strings are not build up out of multiple sub-expressions. strcat is on it's way. I have begun the process of macrofication.
Nov 16 2016
parent reply Andrea Fontana <nospam example.com> writes:
On Thursday, 17 November 2016 at 05:35:33 UTC, Stefan Koch wrote:
 On Wednesday, 16 November 2016 at 14:44:06 UTC, Stefan Koch 
 wrote:
 And Again a bit of bad news.
 Due to problems in the lowering of function arguments the 
 implementation of strcat is delayed again.
The bug does not affect strings. Since strings are not build up out of multiple sub-expressions. strcat is on it's way. I have begun the process of macrofication.
I follow this thread every day. I hope you'll write an article on dlang blog when the work will be completed :)
Nov 17 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 17 November 2016 at 08:39:57 UTC, Andrea Fontana 
wrote:
 I follow this thread every day. I hope you'll write an article 
 on dlang blog when the work will be completed :)
Mike Parker is going to write an short article about it based on information I gave him via mail. I am afraid my blog-writing skills are a bit under-developed. On the topic of CTFE : My attempts of half-automatically generating a string-concat macro have yet to succeed. I am currently busy fixing bugs :) I apologize for the seemingly slow progress. However, such is the nature of low-level code. There is really no way around it, if we are aiming for performance.
Nov 17 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 17 November 2016 at 09:19:57 UTC, Stefan Koch wrote:
 On Thursday, 17 November 2016 at 08:39:57 UTC, Andrea Fontana 
 wrote:
 I follow this thread every day. I hope you'll write an article 
 on dlang blog when the work will be completed :)
Mike Parker is going to write an short article about it based on information I gave him via mail. I am afraid my blog-writing skills are a bit under-developed. On the topic of CTFE : My attempts of half-automatically generating a string-concat macro have yet to succeed. I am currently busy fixing bugs :) I apologize for the seemingly slow progress. However, such is the nature of low-level code. There is really no way around it, if we are aiming for performance.
I just fixed the bug with multiple ArraysLiterals as arguments. We are not out of the mud yet. This will most likely not play nice with nested arrays. But let's cross that bridge when we come to it :)
Nov 18 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 18 November 2016 at 15:09:02 UTC, Stefan Koch wrote:

 I just fixed the bug with multiple ArraysLiterals as arguments.
This means the following code will now compile and properly execute: uint Sum3Arrays (uint[] a1, uint[] a2, uint[] a3) { uint result; for(int i; i != a1.length; i++) { result += a1[i]; } for(int i; i != a2.length; i++) { result += a2[i]; } for(int i; i != a3.length; i++) { result += a3[i]; } return result; } static assert(Sum3Arrays([2,3],[],[]) == 5); static assert(Sum3Arrays([2],[],[3]) == 5); static assert(Sum3Arrays([3],[1],[1]) == 5);
Nov 18 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 18 November 2016 at 17:21:10 UTC, Stefan Koch wrote:
 On Friday, 18 November 2016 at 15:09:02 UTC, Stefan Koch wrote:

 I just fixed the bug with multiple ArraysLiterals as arguments.
This means the following code will now compile and properly execute: uint Sum3Arrays (uint[] a1, uint[] a2, uint[] a3) { uint result; for(int i; i != a1.length; i++) { result += a1[i]; } for(int i; i != a2.length; i++) { result += a2[i]; } for(int i; i != a3.length; i++) { result += a3[i]; } return result; } static assert(Sum3Arrays([2,3],[],[]) == 5); static assert(Sum3Arrays([2],[],[3]) == 5); static assert(Sum3Arrays([3],[1],[1]) == 5);
Another small update. I just fixed the bailout mechanism. Before it would ignore failiures in argument processing and start the interpreter with bogus as arguments. This of course looked like miscompiled code. I am happy it was just the bailout :)
Nov 18 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Another small update.
I got rid of the heapClearing overhead.
By using ddmds allocator.
Because of that I was able to shave off 20ms overhead.
Nov 19 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 19 November 2016 at 09:42:33 UTC, Stefan Koch wrote:
 Another small update.
 I got rid of the heapClearing overhead.
 By using ddmds allocator.
 Because of that I was able to shave off 20ms overhead.
Allocation of dynamic arrays works :) The following code will work just fine with the new engine. uint[] repeat(uint[] a, uint cnt) { uint[] result = []; uint pos; result.length = cast(uint) (a.length * cnt); while(cnt--) foreach(c;a) { result[pos++] = c; } return result; } static assert(repeat([1,2,3,4], 4) == [1,2,3,4, 1,2,3,4, 1,2,3,4, 1,2,3,4] );
Nov 19 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 19 November 2016 at 11:22:18 UTC, Stefan Koch wrote:
 On Saturday, 19 November 2016 at 09:42:33 UTC, Stefan Koch 
 wrote:
 Another small update.
 I got rid of the heapClearing overhead.
 By using ddmds allocator.
 Because of that I was able to shave off 20ms overhead.
Allocation of dynamic arrays works :) The following code will work just fine with the new engine. uint[] repeat(uint[] a, uint cnt) { uint[] result = []; uint pos; result.length = cast(uint) (a.length * cnt); while(cnt--) foreach(c;a) { result[pos++] = c; } return result; } static assert(repeat([1,2,3,4], 4) == [1,2,3,4, 1,2,3,4, 1,2,3,4, 1,2,3,4] );
There is an important change coming up. I am going to rewrite the 64 bit support. The after the change the interpreter will be slower when working with 64bit values. However it will be much easier for the jit. Also I have a lead on howto fix the bugs in break and continue handling. A nice weekend to all of you!
Nov 20 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 20 November 2016 at 09:02:30 UTC, Stefan Koch wrote:
 On Saturday, 19 November 2016 at 11:22:18 UTC, Stefan Koch 
 wrote:
 On Saturday, 19 November 2016 at 09:42:33 UTC, Stefan Koch 
 wrote:
 Another small update.
 I got rid of the heapClearing overhead.
 By using ddmds allocator.
 Because of that I was able to shave off 20ms overhead.
Allocation of dynamic arrays works :) The following code will work just fine with the new engine. uint[] repeat(uint[] a, uint cnt) { uint[] result = []; uint pos; result.length = cast(uint) (a.length * cnt); while(cnt--) foreach(c;a) { result[pos++] = c; } return result; } static assert(repeat([1,2,3,4], 4) == [1,2,3,4, 1,2,3,4, 1,2,3,4, 1,2,3,4] );
There is an important change coming up. I am going to rewrite the 64 bit support. The after the change the interpreter will be slower when working with 64bit values. However it will be much easier for the jit. Also I have a lead on howto fix the bugs in break and continue handling. A nice weekend to all of you!
I discovered a new bug in the handling of do-while statements. Which I subsequently fixed. I also fixed the C backend. It would interpret a certain conditional different from all other back-ends. This makes it oblivious that better documentation of the code-gen interface is needed. Furthermore I need to extend my bc_tests. to make sure the interpretation is the same. Such that this bug cannot happen again. As always a happy weekend to all of you. - I really need a drink ... preferably root-beer.
Nov 20 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 20 November 2016 at 15:37:50 UTC, Stefan Koch wrote:
 Furthermore I need to extend my bc_tests. to make sure the 
 interpretation is the same.
 Such that this bug cannot happen again.
I have extended my test-suite to test this case. It is crucial that the c backend and the interpreter back-ends do always return the same correct value. As those are both testable at CTFE. I have plans to implement a small CTFEable instruction-set-simulator for the intel 486 processor. That way I can make sure that my x86-backend does what it is supposed to do. However this is further down the road :)
Nov 20 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 20 November 2016 at 16:27:24 UTC, Stefan Koch wrote:
 On Sunday, 20 November 2016 at 15:37:50 UTC, Stefan Koch wrote:
 Furthermore I need to extend my bc_tests. to make sure the 
 interpretation is the same.
 Such that this bug cannot happen again.
before. I have extended my test-suite to test this case. It is crucial that the c backend and the interpreter back-ends do always return the same correct value. As those are both testable at CTFE. I have plans to implement a small CTFEable instruction-set-simulator for the intel 486 processor. That way I can make sure that my x86-backend does what it is supposed to do. However this is further down the road :)
A bug in Do-While handling appeared that was hidden by the previously incorrect behavior. When writing do { ... } while(true/false) The IR generated would tell the interpreter to evaluate the condition flag. However the load of a constant does not modify the condition flag and therefore if the loop is executed or not, depends on the result of a previous comparison. The fix is simple check for the (false/true) constants. (DMD has a shortcut for this, awkwardly named isBool() ) with an even more awkward usage. if (expression.isBool(true)) { // this expression is a constant which evaluates to true } else if (expression.isBool(false)) {// this expression is a constant false else { // this value is not a constant }
Nov 21 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 21 November 2016 at 10:51:01 UTC, Stefan Koch wrote:
 When writing do { ... } while(true/false)
 The IR generated would tell the interpreter to evaluate the 
 condition flag.
This Bug is now fixed. At the same time a superfluous jump could be eliminated. I have also fixed another bug in the bailout system. Such that we now bail out when we could not interpret the right-hand-side of an arrayLength Assignment.
Nov 21 2016
prev sibling parent reply Anonymouse <asdf asdf.net> writes:
On Sunday, 20 November 2016 at 09:02:30 UTC, Stefan Koch wrote:
 On Saturday, 19 November 2016 at 11:22:18 UTC, Stefan Koch 
 wrote:
[...] I would buy you a beer but the Internet is in the way.
Nov 21 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 21 November 2016 at 19:11:58 UTC, Anonymouse wrote:
 On Sunday, 20 November 2016 at 09:02:30 UTC, Stefan Koch wrote:
 On Saturday, 19 November 2016 at 11:22:18 UTC, Stefan Koch 
 wrote:
[...] I would buy you a beer but the Internet is in the way.
I appreciate the offer. During the next days I will hopefully be able to have the first arrays of structures working. Those are a bit tricky since they require doubly-indirected RMW.
Nov 21 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 21 November 2016 at 19:19:50 UTC, Stefan Koch wrote:
 On Monday, 21 November 2016 at 19:11:58 UTC, Anonymouse wrote:
 On Sunday, 20 November 2016 at 09:02:30 UTC, Stefan Koch wrote:
 On Saturday, 19 November 2016 at 11:22:18 UTC, Stefan Koch 
 wrote:
[...] I would buy you a beer but the Internet is in the way.
I appreciate the offer. During the next days I will hopefully be able to have the first arrays of structures working. Those are a bit tricky since they require doubly-indirected RMW.
Progress is still stalled on this. It is very important to get pointers and handling of other qusi-references right. There are still known bugs in switch processing, also the llvm-backend need to be fixed.
Nov 24 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
I have just discovered breakage of basic operations.
I will need regression tests, some time soon.
Nov 24 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 24 November 2016 at 17:16:33 UTC, Stefan Koch wrote:
 I have just discovered breakage of basic operations.
 I will need regression tests, some time soon.
Hey Guys, I fixed the regression. (At the expense, of removing more optimization) Work continues on struct and (dynamic) array support. Also I am going to fix another regression I just discoverd that prevents multiple string-arguments to work properly. Those regression are seriously annoying.
Nov 25 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 25 November 2016 at 09:14:16 UTC, Stefan Koch wrote:
 On Thursday, 24 November 2016 at 17:16:33 UTC, Stefan Koch 
 wrote:
 I have just discovered breakage of basic operations.
 I will need regression tests, some time soon.
Hey Guys, I fixed the regression. (At the expense, of removing more optimization) Work continues on struct and (dynamic) array support. Also I am going to fix another regression I just discoverd that prevents multiple string-arguments to work properly. Those regression are seriously annoying.
String-Arguments are fixed. The known regressions are now resolved.
Nov 25 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 25 November 2016 at 10:53:50 UTC, Stefan Koch wrote:
 On Friday, 25 November 2016 at 09:14:16 UTC, Stefan Koch wrote:
 On Thursday, 24 November 2016 at 17:16:33 UTC, Stefan Koch 
 wrote:
 I have just discovered breakage of basic operations.
 I will need regression tests, some time soon.
Hey Guys, I fixed the regression. (At the expense, of removing more optimization) Work continues on struct and (dynamic) array support. Also I am going to fix another regression I just discoverd that prevents multiple string-arguments to work properly. Those regression are seriously annoying.
String-Arguments are fixed. The known regressions are now resolved.
I discovered a nasty bug in goto handling. I am working on fixed it; This could take a while, since it likely requires structural changes.
Nov 28 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 28 November 2016 at 17:02:37 UTC, Stefan Koch wrote:
 On Friday, 25 November 2016 at 10:53:50 UTC, Stefan Koch wrote:

 I discovered a nasty bug in goto handling.
 I am working on fixed it;
 This could take a while, since it likely requires structural 
 changes.
I fixed a bug in continue break handling. The dreaded structural changes for this were easier then expected. Perhaps fixing the goto-handling will be as straight-forward ? The following code compiles now: int bug4448() { int n = 2; L1: do { switch(n) { case 5: return 7; default: n = 5; break L1; return n; } int w = 7; } while (0); return 3; } static assert(bug4448() == 3);
Nov 29 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 29 November 2016 at 23:00:08 UTC, Stefan Koch wrote:
 I fixed a bug in continue break handling.
For the record it was an off by one error. The value for unresolvedGotos would add one referencing jump. But the count of referencing would be initialized to zero instead of one.
Nov 29 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 29 November 2016 at 23:09:08 UTC, Stefan Koch wrote:
 On Tuesday, 29 November 2016 at 23:00:08 UTC, Stefan Koch wrote:
 I fixed a bug in continue break handling.
For the record it was an off by one error. The value for unresolvedGotos would add one referencing jump. But the count of referencing would be initialized to zero instead of one.
I just added support for assert in newCTFE. Also I noticed a performance regression in the bytecode generator. (compared to an earlier version that handled much less) Nontheless I get a solid 4x performance improvement over the old evaluator when compiled with ldc. Furthermore struct support regressed. It has to be rewritten anyway in order to support method-calls properly.
Nov 30 2016
next sibling parent reply Andrea Fontana <nospam example.com> writes:
On Wednesday, 30 November 2016 at 13:49:01 UTC, Stefan Koch wrote:
 On Tuesday, 29 November 2016 at 23:09:08 UTC, Stefan Koch wrote:
 On Tuesday, 29 November 2016 at 23:00:08 UTC, Stefan Koch 
 wrote:
 I fixed a bug in continue break handling.
For the record it was an off by one error. The value for unresolvedGotos would add one referencing jump. But the count of referencing would be initialized to zero instead of one.
I just added support for assert in newCTFE. Also I noticed a performance regression in the bytecode generator. (compared to an earlier version that handled much less) Nontheless I get a solid 4x performance improvement over the old evaluator when compiled with ldc. Furthermore struct support regressed. It has to be rewritten anyway in order to support method-calls properly.
Every now and then it would be interesting to have a list of things that work and a list of things still to do. Keep up the good work!
Nov 30 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 30 November 2016 at 15:02:21 UTC, Andrea Fontana 
wrote:

 Every now and then it would be interesting to have a list of 
 things that work and a list of things still to do.
That list changes. On both ends. I constantly discover things that I thought were dealt with are indeed still work in progress. I can give a rough overview in a few day when things have (Hopefully!) stabilized a little more.
 Keep up the good work!
I will!
Nov 30 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
I just fixed  a bug cause the non-debug build to fail on a 
expression that it should have bailed out on.

Currently I have troubles with this  code :

int fn(int y)
{
   int x = void;
   ++x;
   return x+y;
}

Since x is uninitialized the ctfe interpreter should fail on it.
I will need to reserve a certain value as the uninitialized flag.
since stackAddress zero can never be used it'd be perfect for the 
job.
This bug should be fixed by tomorrow.
Dec 01 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 1 December 2016 at 17:52:02 UTC, Stefan Koch wrote:
 I just fixed  a bug cause the non-debug build to fail on a 
 expression that it should have bailed out on.

 Currently I have troubles with this  code :

 int fn(int y)
 {
   int x = void;
   ++x;
   return x+y;
 }

 Since x is uninitialized the ctfe interpreter should fail on it.
 I will need to reserve a certain value as the uninitialized 
 flag.
 since stackAddress zero can never be used it'd be perfect for 
 the job.
 This bug should be fixed by tomorrow.
I opted for bailing out as soon as void initialization is in the function. This will need to handled properly at some point but now is not the time.
Dec 02 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Currently I am working on the ability of returning structs.
Although those are "merely" heterogeneous arrays.
This is a very important step towards having a usable 
CTFE-reimplementation.

the regressed struct support.
seems to be restored. (Although I have no idea why)
Once this is done the more complex composite types have to be fit 
in the current model.

I have a very flat representation of values.
They are treated almost exactly as run-time values.
Which means that the ctfe engine itself needs a well defined abi.
Since I was aware of this, I have already put alot of thought 
into this.
However the recent onslaught of bugs have shown that I can not 
assume that my code is bug-free.
I expect a few hard issues in the memory-management and 
ABI-dependent code.

I will keep you updated,
Cheers,
Stefan
Dec 02 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 3 December 2016 at 02:07:06 UTC, Stefan Koch wrote:
 Currently I am working on the ability of returning structs.

 I will keep you updated,
This is coming along rather nicely. StructLiterals as Arguments regressed. But they did rather dubious stuff in the first place. So supported types are. StringLiterals. Integral Array Literals. Dynamic int[] and uint[] Arrays. Structs with int or uint members. and StructLiterals with int/uint members (regressed currently.) 64bit integrals (long) do work but only if the interpreter backend is used. Supported operations. All basic math. (+ - * / %) ...++ ...-- +++... --... while(...) {...} do ... while(...) label : goto label; break; break label; continue; continue label; for(...;...;...) ... ? ... : ... switch (...) { case ... , ... } (unreliable due to differences in case-sorting :( ) for-each on strings and arrays. Assigning to array length. (currently without copying on reallocation :( ) Bounds-Checked : Array-Indexing Left-Shift Right-Shift. (always unsigned! :( ) Unsupported : && || classes (unlikely to be added in the near future) function calls (plain and closure) slices pointers static arrays. floating point math (unlikely to be added in the near future)
Dec 03 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 3 December 2016 at 18:00:10 UTC, Stefan Koch wrote:
 On Saturday, 3 December 2016 at 02:07:06 UTC, Stefan Koch wrote:
 [...]
switch (...) { case ... , ... } (unreliable due to differences in case-sorting :( ) for-each on strings and arrays.
if (...) { ... } else { ... } (if(__ctfe) && if(!__ctfe) are special cased and non-ctfe code is removed) assert(..., "...")
 Assigning to array length. (currently without copying on 
 reallocation :( )
 Bounds-Checked :
 Array-Indexing
 Left-Shift Right-Shift. (always unsigned! :( )

 [...]
Forgot to add assert, I forget that I implemented it a few days ago.
Dec 03 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 3 December 2016 at 18:16:21 UTC, Stefan Koch wrote:
 [...]
StructLiteral regressions fixed. That one was hard to find. Basically the for Array arguments interfered with this one. Arguments are technically the same expression as everything else. But in order to support targets that do have native function calls, and in order to archive more performance for the interpreter they have to be treated quite differently from non-argument expressions. For now they should work, but it's brittle at best.
Dec 03 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 3 December 2016 at 19:28:52 UTC, Stefan Koch wrote:
 On Saturday, 3 December 2016 at 18:16:21 UTC, Stefan Koch wrote:
 [...]
StructLiteral regressions fixed. That one was hard to find. Basically the for Array arguments interfered with this one. Arguments are technically the same expression as everything else. But in order to support targets that do have native function calls, and in order to archive more performance for the interpreter they have to be treated quite differently from non-argument expressions. For now they should work, but it's brittle at best.
The following code now compiles with newCTFE: struct S { uint u1; uint u2; } S makeTenTen() { return S(10, 10); } S makeS(uint a, uint b) { return S(a, b); } uint getu2(S s) { return s.u2; } static assert(getu2(S(10, 14)) == 14); static assert(makeTenTen() == makeS(10, 10));
Dec 03 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Another bug in continue break handling has appeared.

uint fn() {
   for (;;)
   {
     if (true) break;
     return 0;
   }
   return 1;
}

static assert(fn());

The above code is enough to trigger it.
Apparently the fixup jump for the break is never inserted.
I doubt that this is an off-by-one error again.

It's going to be tricky.
Dec 03 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 3 December 2016 at 23:30:58 UTC, Stefan Koch wrote:
 Another bug in continue break handling has appeared.

 uint fn() {
   for (;;)
   {
     if (true) break;
     return 0;
   }
   return 1;
 }

 static assert(fn());

 The above code is enough to trigger it.
 Apparently the fixup jump for the break is never inserted.
 I doubt that this is an off-by-one error again.

 It's going to be tricky.
I figured out part of the reason for this bug. baically it's like this.
 uint fn() {
   for (;;)
beginBlock: //causes and unconditonal jump (UC1) after the end of the block
   {
     if (true) break;
// causes an unconditional jump(UC2) here to the end of the block UC2: goto endBlock;
     return 0;
endBlock: UC1: goto BeginBlock;
   }
   return 1;
 }
Dec 03 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 4 December 2016 at 00:14:26 UTC, Stefan Koch wrote:
 On Saturday, 3 December 2016 at 23:30:58 UTC, Stefan Koch wrote:
 Another bug in continue break handling has appeared.

 uint fn() {
   for (;;)
   {
     if (true) break;
     return 0;
   }
   return 1;
 }

 static assert(fn());

 The above code is enough to trigger it.
 Apparently the fixup jump for the break is never inserted.
 I doubt that this is an off-by-one error again.

 It's going to be tricky.
I figured out part of the reason for this bug. baically it's like this.
 uint fn() {
   for (;;)
beginBlock: //causes and unconditonal jump (UC1) after the end of the block
   {
     if (true) break;
// causes an unconditional jump(UC2) here to the end of the block UC2: goto endBlock;
     return 0;
endBlock: UC1: goto BeginBlock;
   }
   return 1;
 }
This can be fixed by setting a state-flag whenever we are going to process the body of an infinite loop. If anyone tried at any point to break out of the loop we need to introduce a conditional that becomes false and allows us to exit. It's ugly but it will work regardless of nesting-level .... I hope.
Dec 03 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 4 December 2016 at 00:23:32 UTC, Stefan Koch wrote:
 On Sunday, 4 December 2016 at 00:14:26 UTC, Stefan Koch wrote:
 On Saturday, 3 December 2016 at 23:30:58 UTC, Stefan Koch 
 wrote:
 [...]
I figured out part of the reason for this bug. baically it's like this.
   [...]
beginBlock: //causes and unconditonal jump (UC1) after the end of the block
     [...]
// causes an unconditional jump(UC2) here to the end of the block UC2: goto endBlock;
     [...]
endBlock: UC1: goto BeginBlock;
 [...]
Some progress had been made on this. There is still a little hiccup with (true) and (false) conditions but those can be eliminated soon. For the most part breaking out of infinite loops works now.
Dec 04 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
I just had a hunch about the continue statement within for-loops 
being treated incorrectly.

let's see where it leads.

Also the fix for infinite loops has apparently cost us a lot of 
performance.
I am not sure where this loss is coming from.
Maybe I am just reading my scales wrong ;)

While those codegen remain new features and documentation have 
reduced priority.

I am doing a screening regarding the structural strength of the 
architecture.

I would be quite happy to read any thoughts on that matter.
Dec 04 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 4 December 2016 at 23:10:34 UTC, Stefan Koch wrote:
 Also the fix for infinite loops has apparently cost us a lot of 
 performance.
 I am not sure where this loss is coming from.
 Maybe I am just reading my scales wrong ;)
it seemed to have t odo with the hardware being particularly busy at the point of measurement. I fixed a bug in ifStatements. so now all the constant expression trouble should be resolved.
Dec 04 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
I just improved the handling of void initializations.
Now the code is less pessimistic and will allow them if they are 
assigned to before use.
However using void variables at ctfe will not result in any 
performance wins.
Dec 04 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 December 2016 at 04:26:35 UTC, Stefan Koch wrote:
 I just improved the handling of void initializations.
 Now the code is less pessimistic and will allow them if they 
 are assigned to before use.
 However using void variables at ctfe will not result in any 
 performance wins.
I found an easily fixable performance problem inside the byte-code generator. Causing it to allocate 800K per discovery of a new type. Reducing this will probably make IR generation 10 times faster in the average case. Clearing 800K takes quite some time. Furthermore I will revise some design decisions inside the IR generator. Those are related to how break and continue are handled. Because continue is still broken. Hopefully I can fix this without spreading the responsible code around everywhere :)
Dec 04 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 December 2016 at 07:48:31 UTC, Stefan Koch wrote:
 I found an easily fixable performance problem inside the 
 byte-code generator.
 Causing it to allocate 800K per discovery of a new type.
 Reducing this will probably make IR generation 10 times faster 
 in the average case.
 Clearing 800K takes quite some time.
I just fixed this. As predicted, the taken time to generate byte-code is now greatly reduced. This has a really huge impact. It looks like the performance wins brought by the new ctfe engine might be higher then I predicted.
Dec 05 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/05/2016 11:28 AM, Stefan Koch wrote:
 It looks like the performance wins brought by the new ctfe engine might
 be higher then I predicted.
That's awesome!! -- Andrei
Dec 05 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 December 2016 at 16:47:33 UTC, Andrei Alexandrescu 
wrote:
 On 12/05/2016 11:28 AM, Stefan Koch wrote:
 It looks like the performance wins brought by the new ctfe 
 engine might
 be higher then I predicted.
That's awesome!! -- Andrei
After discovering this performance bottleneck I have now changed my mind about how I will tackle concatenation of arrays and strings in particular. All con-cat operations should be done as intrinsic calls. Because tight cooperation with the CTFE-Memory-Management-Subsystem is needed.
Dec 05 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 December 2016 at 18:47:13 UTC, Stefan Koch wrote:
 On Monday, 5 December 2016 at 16:47:33 UTC, Andrei Alexandrescu 
 wrote:
 On 12/05/2016 11:28 AM, Stefan Koch wrote:
 It looks like the performance wins brought by the new ctfe 
 engine might
 be higher then I predicted.
That's awesome!! -- Andrei
After discovering this performance bottleneck I have now changed my mind about how I will tackle concatenation of arrays and strings in particular. All con-cat operations should be done as intrinsic calls. Because tight cooperation with the CTFE-Memory-Management-Subsystem is needed.
I just implemented a bytecode cache, however the bytecode generation is so fast now that we only safe a couple micro seconds by using the cache. It is not noticeable at all :P.
Dec 06 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/06/2016 11:27 AM, Stefan Koch wrote:
 On Monday, 5 December 2016 at 18:47:13 UTC, Stefan Koch wrote:
 On Monday, 5 December 2016 at 16:47:33 UTC, Andrei Alexandrescu wrote:
 On 12/05/2016 11:28 AM, Stefan Koch wrote:
 It looks like the performance wins brought by the new ctfe engine might
 be higher then I predicted.
That's awesome!! -- Andrei
After discovering this performance bottleneck I have now changed my mind about how I will tackle concatenation of arrays and strings in particular. All con-cat operations should be done as intrinsic calls. Because tight cooperation with the CTFE-Memory-Management-Subsystem is needed.
I just implemented a bytecode cache, however the bytecode generation is so fast now that we only safe a couple micro seconds by using the cache. It is not noticeable at all :P.
Just give us time :o). -- Andrei
Dec 06 2016
prev sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 6 December 2016 at 16:27:38 UTC, Stefan Koch wrote:
 I just implemented a bytecode cache, however the bytecode 
 generation is so fast now that we only safe a couple micro 
 seconds by using the cache.
 It is not noticeable at all :P.
I can't wait to try this out.
Dec 06 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 December 2016 at 21:40:47 UTC, Nordlöw wrote:
 On Tuesday, 6 December 2016 at 16:27:38 UTC, Stefan Koch wrote:
 I just implemented a bytecode cache, however the bytecode 
 generation is so fast now that we only safe a couple micro 
 seconds by using the cache.
 It is not noticeable at all :P.
I can't wait to try this out.
Go ahead. Many features should already be working correctly. The missing important ones are slices and concat.
Dec 06 2016
next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 6 December 2016 at 22:21:59 UTC, Stefan Koch wrote:
 Go ahead.
 Many features should already be working correctly.
This https://github.com/UplinkCoder/dmd/commits/newCTFE I presume?
Dec 06 2016
prev sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 6 December 2016 at 22:21:59 UTC, Stefan Koch wrote:
 Go ahead.
 Many features should already be working correctly.

 The missing important ones are slices and concat.
Ok, great. I was thinking about benchmarking parser generators such as Pegged. Is there another way of concatenating strings than with the builtin ~ operator?
Dec 06 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 December 2016 at 22:44:34 UTC, Nordlöw wrote:
 On Tuesday, 6 December 2016 at 22:21:59 UTC, Stefan Koch wrote:
 Go ahead.
 Many features should already be working correctly.

 The missing important ones are slices and concat.
Ok, great. I was thinking about benchmarking parser generators such as Pegged. Is there another way of concatenating strings than with the builtin ~ operator?
Yes there is. Create a char[] give it the length of string a + string b; copy string a into the char[] then copy string b into the resulting array. offsetting it by the length of string a.
Dec 06 2016
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 December 2016 at 23:16:16 UTC, Stefan Koch wrote:
 On Tuesday, 6 December 2016 at 22:44:34 UTC, Nordlöw wrote:
 On Tuesday, 6 December 2016 at 22:21:59 UTC, Stefan Koch wrote:
 Go ahead.
 Many features should already be working correctly.

 The missing important ones are slices and concat.
Ok, great. I was thinking about benchmarking parser generators such as Pegged. Is there another way of concatenating strings than with the builtin ~ operator?
Yes there is. Create a char[] give it the length of string a + string b; copy string a into the char[] then copy string b into the resulting array. offsetting it by the length of string a.
But keep in mind function calls are unsupported right now. Meaning you have to put everything in one function. However function calls should be working some time soon. I am still in the middle of restructuring how blocks or handled, because right now I will miscompile for-loops with continues in them.
Dec 06 2016
prev sibling next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 6 December 2016 at 23:16:16 UTC, Stefan Koch wrote:
 Yes there is.
 Create a char[] give it the length of string a + string b;
 copy string a into the char[]
 then copy string b into the resulting array.
 offsetting it by the length of string a.
Sounds like you have your concat implementation right there ;)
Dec 06 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 December 2016 at 23:55:06 UTC, Nordlöw wrote:
 On Tuesday, 6 December 2016 at 23:16:16 UTC, Stefan Koch wrote:
 Yes there is.
 Create a char[] give it the length of string a + string b;
 copy string a into the char[]
 then copy string b into the resulting array.
 offsetting it by the length of string a.
Sounds like you have your concat implementation right there ;)
not really, concat needs to be smart about allocation and memory reuse. when used on char arrays the memory access will be different. and use more instructions.
Dec 06 2016
prev sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 6 December 2016 at 23:16:16 UTC, Stefan Koch wrote:
 Yes there is...
Thanks.
Dec 06 2016
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Monday, 5 December 2016 at 04:26:35 UTC, Stefan Koch wrote:
 I just improved the handling of void initializations.
 Now the code is less pessimistic and will allow them if they 
 are assigned to before use.
 However using void variables at ctfe will not result in any 
 performance wins.
Void initialization are allowed at CTFE ?
Dec 04 2016
next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Monday, 5 December 2016 at 07:55:32 UTC, deadalnix wrote:
 On Monday, 5 December 2016 at 04:26:35 UTC, Stefan Koch wrote:
 I just improved the handling of void initializations.
 Now the code is less pessimistic and will allow them if they 
 are assigned to before use.
 However using void variables at ctfe will not result in any 
 performance wins.
Void initialization are allowed at CTFE ?
not now, but it looks like needless limitation. any void initialization can be converted to "fill the things with zeroes" in CTFE engine.
Dec 05 2016
parent Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
On Monday, 5 December 2016 at 08:07:11 UTC, ketmar wrote:
 On Monday, 5 December 2016 at 07:55:32 UTC, deadalnix wrote:
 On Monday, 5 December 2016 at 04:26:35 UTC, Stefan Koch wrote:
 I just improved the handling of void initializations.
 Now the code is less pessimistic and will allow them if they 
 are assigned to before use.
 However using void variables at ctfe will not result in any 
 performance wins.
Void initialization are allowed at CTFE ?
not now, but it looks like needless limitation. any void initialization can be converted to "fill the things with zeroes" in CTFE engine.
On the contrary. If something is NOT initialized, it uses its default (e.g. zeroes), but if it is =void it should be an error to use it before any assignment.
Dec 05 2016
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 December 2016 at 07:55:32 UTC, deadalnix wrote:
 On Monday, 5 December 2016 at 04:26:35 UTC, Stefan Koch wrote:
 I just improved the handling of void initializations.
 Now the code is less pessimistic and will allow them if they 
 are assigned to before use.
 However using void variables at ctfe will not result in any 
 performance wins.
Void initialization are allowed at CTFE ?
the following code will compile just fine. uint fn(uint a) { int b = void; if (a == 2) { b = 1; } return b; // only fine is a was 2; } static assert(fn(2) == 1);
Dec 05 2016
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 5 December 2016 at 04:26:35 UTC, Stefan Koch wrote:
 I just improved the handling of void initializations.
 Now the code is less pessimistic and will allow them if they 
 are assigned to before use.
 However using void variables at ctfe will not result in any 
 performance wins.
Oh it's broken. It does not detect the ... maybe not void case. I found out why certain switches are broken. Essentially this is fixable by putting a guarding jump around the braces that holds the cases. Which is also the reason why code in a switch that does not belong to any case is unreachable code :)
Dec 05 2016
prev sibling parent reply Andrea Fontana <nospam example.com> writes:
On Saturday, 3 December 2016 at 18:00:10 UTC, Stefan Koch wrote:
 classes (unlikely to be added in the near future)
 floating point math (unlikely to be added in the near future)
What's the problem with classes and floating point math? Andrea
Dec 07 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 December 2016 at 08:33:22 UTC, Andrea Fontana 
wrote:
 On Saturday, 3 December 2016 at 18:00:10 UTC, Stefan Koch wrote:
 classes (unlikely to be added in the near future)
 floating point math (unlikely to be added in the near future)
What's the problem with classes and floating point math? Andrea
Honestly the main problem is that I am not familiar with FloatingPoint-Math. Therefore I would may not even notice when I introduce a subtle bug. Classes build on top of a functioning memory-management and require a mechanism for indirect calls or function pointers. Both are currently not implemented. I will worry about classes when the rest of the engine runs cleanly.
Dec 07 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
I have an update about ctfe performance.
I wondered for quite a while why newCTFE had a 5 millisecond 
overhead, when compared the old interpreter.
Since on my charts it had comparable or better stats.
I finally figured it out.

Because the interpreter is supposed to be ctfe-able itself it 
uses a gc-allocated stack.
And that in turn call envokes the gc.
The mark phase of the garbage-collector matches up exactly with 
the extra time taken.

I have no idea, why this happens though since the GC is supposed 
to be disabled.
Dec 07 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 December 2016 at 18:00:40 UTC, Stefan Koch wrote:
 I have an update about ctfe performance.
 I wondered for quite a while why newCTFE had a 5 millisecond 
 overhead, when compared the old interpreter.
 Since on my charts it had comparable or better stats.
 I finally figured it out.

 Because the interpreter is supposed to be ctfe-able itself it 
 uses a gc-allocated stack.
 And that in turn call envokes the gc.
 The mark phase of the garbage-collector matches up exactly with 
 the extra time taken.

 I have no idea, why this happens though since the GC is 
 supposed to be disabled.
It turns out that almost classes in dmd that are em-placed. Whereas by bytecode visitor uses the regular new, because it is fairly sizable. The gc is much more visible. Although that does still not explain why GC.disable seems to have no effect.
Dec 07 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 December 2016 at 18:50:20 UTC, Stefan Koch wrote:
 On Wednesday, 7 December 2016 at 18:00:40 UTC, Stefan Koch 
 wrote:
 I have an update about ctfe performance.
 I wondered for quite a while why newCTFE had a 5 millisecond 
 overhead, when compared the old interpreter.
 Since on my charts it had comparable or better stats.
 I finally figured it out.

 Because the interpreter is supposed to be ctfe-able itself it 
 uses a gc-allocated stack.
 And that in turn call envokes the gc.
 The mark phase of the garbage-collector matches up exactly 
 with the extra time taken.

 I have no idea, why this happens though since the GC is 
 supposed to be disabled.
It turns out that almost classes in dmd that are em-placed. Whereas by bytecode visitor uses the regular new, because it is fairly sizable. The gc is much more visible. Although that does still not explain why GC.disable seems to have no effect.
After a few more iterations the observed slow down seems to disappear. I am suspicious.
Dec 07 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
I did the change in continue and break handling.

Now ForStatements should be handled correctly for the most part.
I think there could still be a bug if you do labeled continue on 
a for-statement.
I will do a change later that takes that into account as well.

Then we should be all set to implement call handling I believe.
Dec 07 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 December 2016 at 20:56:08 UTC, Stefan Koch wrote:
 I did the change in continue and break handling.

 Now ForStatements should be handled correctly for the most part.
 I think there could still be a bug if you do labeled continue 
 on a for-statement.
 I will do a change later that takes that into account as well.

 Then we should be all set to implement call handling I believe.
Array-literals with ctfe-known Variables in them are now supported; uint[] fn(uint n) { return [n, n, n]; } static assert(fn(3).length == 3); static assert(fn(3)[0] == 3); static assert(fn(3)[1] == 3); static assert(fn(3)[2] == 3);
Dec 07 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 8 December 2016 at 01:52:13 UTC, Stefan Koch wrote:
 Array-literals with ctfe-known Variables in them are now 
 supported;
 uint[] fn(uint n)
 {
   return [n, n, n];
 }

 static assert(fn(3).length == 3);
 static assert(fn(3)[0] == 3);
 static assert(fn(3)[1] == 3);
 static assert(fn(3)[2] == 3);
This works as well with interspersed constant. uint[] itrspary(uint n) { return [1, n, 3]; } static assert(itrspary(7).length == 3); static assert(itrspary(7)[0] == 1); static assert(itrspary(7)[1] == 7); static assert(itrspary(7)[2] == 3);
Dec 07 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 8 December 2016 at 02:22:31 UTC, Stefan Koch wrote:
 On Thursday, 8 December 2016 at 01:52:13 UTC, Stefan Koch wrote:
 Array-literals with ctfe-known Variables in them are now 
 supported;
 uint[] fn(uint n)
 {
   return [n, n, n];
 }

 static assert(fn(3).length == 3);
 static assert(fn(3)[0] == 3);
 static assert(fn(3)[1] == 3);
 static assert(fn(3)[2] == 3);
This works as well with interspersed constant. uint[] itrspary(uint n) { return [1, n, 3]; } static assert(itrspary(7).length == 3); static assert(itrspary(7)[0] == 1); static assert(itrspary(7)[1] == 7); static assert(itrspary(7)[2] == 3);
I fixed the bug in switchStatements (I think.). It was not a mismatch with dmd's sorting but rather. My own sloppy block-exit detection. Using the dmd BlockExit visitor fixed it.
Dec 07 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
I found the biggest performance bottleneck in newCTFE!

oldCtfe :
[root localhost dmd]# time src/dmd -c ctfeTest.d testStringEq.d 
testStringLength.d testStruct.d testMultipleArrayLiterals.d

real	0m0.026s
user	0m0.020s
sys	0m0.003s

[root localhost dmd]# time src/dmd -c ctfeTest.d testStringEq.d 
testStringLength.d testStruct.d testMultipleArrayLiterals.d 
-bc-ctfe

real	0m0.025s
user	0m0.020s
sys	0m0.003s

After Fixing

[root localhost dmd]# time src/dmd -c ctfeTest.d testStringEq.d 
testStringLength.d testStruct.d testMultipleArrayLiterals.d  
-bc-ctfe

real	0m0.019s
user	0m0.017s
sys	0m0.000s
Dec 08 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 8 December 2016 at 19:13:23 UTC, Stefan Koch wrote:
 I found the biggest performance bottleneck in newCTFE!

 oldCtfe :
 [root localhost dmd]# time src/dmd -c ctfeTest.d testStringEq.d 
 testStringLength.d testStruct.d testMultipleArrayLiterals.d

 real	0m0.026s
 user	0m0.020s
 sys	0m0.003s
Before Fixing :
 [root localhost dmd]# time src/dmd -c ctfeTest.d testStringEq.d 
 testStringLength.d testStruct.d testMultipleArrayLiterals.d 
 -bc-ctfe

 real	0m0.025s
 user	0m0.020s
 sys	0m0.003s

 After Fixing

 [root localhost dmd]# time src/dmd -c ctfeTest.d testStringEq.d 
 testStringLength.d testStruct.d testMultipleArrayLiterals.d  
 -bc-ctfe

 real	0m0.019s
 user	0m0.017s
 sys	0m0.000s
Please note that the ctfe interpretation, with newCTFE, inside the frontend takes 10% of the compilation time Whereas with the old interpreter it takes 50%
Dec 08 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
I just wanted to post another performance comparision that does 
not test dmd's memory allocator more then anything else :)

[root localhost dmd]# time src/dmd -c testSettingArrayLength.d  
-bc-ctfe
2147385345u
536821761u
4294639619u

real	0m0.114s
user	0m0.110s
sys	0m0.003s
[root localhost dmd]# time src/dmd -c testSettingArrayLength.d
2147385345u
536821761u
4294639619u

real	0m0.921s
user	0m0.843s
sys	0m0.077s

Results are obtained running the following code
uint MakeInitAndSumArray(uint length)
{
     uint result;
     uint[] arr;
     arr.length = length;

     while(length--)
     {
         arr[length] = length;
     }
     foreach(e;arr)
     {
       result += e;
     }

     return result;
}

pragma(msg, MakeInitAndSumArray(ushort.max));
pragma(msg, MakeInitAndSumArray(ushort.max/2));
pragma(msg, MakeInitAndSumArray(ushort.max*2));
Dec 08 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/08/2016 04:24 PM, Stefan Koch wrote:
 Results are obtained running the following code
 uint MakeInitAndSumArray(uint length)
 {
     uint result;
     uint[] arr;
     arr.length = length;

     while(length--)
     {
         arr[length] = length;
     }
     foreach(e;arr)
     {
       result += e;
     }

     return result;
 }

 pragma(msg, MakeInitAndSumArray(ushort.max));
 pragma(msg, MakeInitAndSumArray(ushort.max/2));
 pragma(msg, MakeInitAndSumArray(ushort.max*2));
That's pretty awesome. The new CTFE engine release will soon rival Tesla Model 3 in terms of interest raised :o). In addition to artificial corpora like the above, I suggest comparing against real-world code - starting with simpler things such as bitfields instantiations. Andrei
Dec 08 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 9 December 2016 at 02:10:58 UTC, Andrei Alexandrescu 
wrote:
 On 12/08/2016 04:24 PM, Stefan Koch wrote:
 Results are obtained running the following code
 uint MakeInitAndSumArray(uint length)
 {
     uint result;
     uint[] arr;
     arr.length = length;

     while(length--)
     {
         arr[length] = length;
     }
     foreach(e;arr)
     {
       result += e;
     }

     return result;
 }

 pragma(msg, MakeInitAndSumArray(ushort.max));
 pragma(msg, MakeInitAndSumArray(ushort.max/2));
 pragma(msg, MakeInitAndSumArray(ushort.max*2));
That's pretty awesome. The new CTFE engine release will soon rival Tesla Model 3 in terms of interest raised :o). In addition to artificial corpora like the above, I suggest comparing against real-world code - starting with simpler things such as bitfields instantiations. Andrei
That will not work currently. Since function and methods calls are still "in the works". They will require another restructuring as well. Therefore artificial tests are the best I can do to gunge performance.
Dec 08 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 9 December 2016 at 03:06:49 UTC, Stefan Koch wrote:
 On Friday, 9 December 2016 at 02:10:58 UTC, Andrei Alexandrescu 
 wrote:
 That's pretty awesome. The new CTFE engine release will soon 
 rival Tesla Model 3 in terms of interest raised :o).

 In addition to artificial corpora like the above, I suggest 
 comparing against real-world code - starting with simpler 
 things such as bitfields instantiations.


 Andrei
That will not work currently. Since function and methods calls are still "in the works". They will require another restructuring as well. Therefore artificial tests are the best I can do to gunge performance.
Progress has been made on this front. However I discovered that some math is translated incorrectly. This means I have to fix the code-gen again. So you'll have to wait a bit until this works :).
Dec 09 2016
prev sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 30 November 2016 at 13:49:01 UTC, Stefan Koch wrote:
 I just added support for assert in newCTFE.
Cool.
 Nontheless I get a solid 4x performance improvement over the 
 old evaluator when compiled with ldc.
Wow.
 Furthermore struct support regressed.
 It has to be rewritten anyway in order to support method-calls 
 properly.
Keep up!
Nov 30 2016
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 11/08/2016 02:11 PM, Antonio Corbi wrote:
 Maybe this one is useful for you:

 http://eagain.net/articles/git-for-computer-scientists/
On 11/08/2016 03:01 PM, H. S. Teoh via Digitalmars-d wrote:
 Nothing immediately comes to mind, but thanks to Dr. Google, I found
 this page that's sorta helpful:

 	http://ericsink.com/vcbe/html/directed_acyclic_graphs.html

 And perhaps these:

 	http://eagain.net/articles/git-for-computer-scientists/
 	http://marklodato.github.io/visual-git-guide/index-en.html
Ok, so it looks like each node in the DAG is a commit. I'll definitely have to read further. Thanks, both. Although I have my doubts it would explain all the issues I've hit upon with git's CLI. For example: I don't see why annotated tags aren't the default. Or why non-annotated ones even exist at all. When I made <https://github.com/Abscissa/gen-package-version>, I noticed that it'll *only* work if the version tags are "annotated" tags. Because otherwise "git describe" doesn't retrieve the tag. Doesn't even matter one bit *what* the tag's message is, just that it exists. Why? Who knows! It's git! So it means that unlike hg users, who can just make their tags with "hg tag v1.2.0" and everything's fine, git users can't just say "git tag v1.2.0". In git, making a tag like that will work for SOME use-cases but not all. Instead, git users have to use the less-obvious "git tag -a v1.2.0 -m '...whatever...'" incantation or else "git describe" won't freaking work right, therefore gen-package-version won't be able to determine the working copy's current tag name. So then why do the non-annotated tags even exist? Why not just have "annotated" tags with *optional* messages? Why even have the "-a" flag for "git tag" at all? Why not have "git tag tagname" make a tag with no message, and "git tag tagname -m '...'" to make a tag with a message, and ditch the whole "-a" nonsense? Because it wouldn't be stupid enough that way? I don't get it. Its things like that. I'd be surprised if that has much to do with git's nature as a "dumb" DAG tool. It's just the general good-design principle of "The thing you *want to* or *should* do or *expect* 99% of the time should be the DEFAULT, not the obscure incantation".
 Or maybe I should write my own article about git being based on a DAG.
 :-P
You should! :) I've given a lot of thought over the years to the concepts of learning and teaching. As I see it, the big key to both is taking seemingly complex things, finding a model that makes them simple, and then finding a way to communicate that model in a way that preserves its inherent simplicity. The big key there is actually having a simple, well-understood mental model, and it sounds like you have that on this, so I'd love to read anything you'd have to write about it.
Nov 09 2016
next sibling parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: [OT] Re: CTFE Status
References: <gcuuyujyweibqkpgdqiu forum.dlang.org>
 <bdlvsfrnohpffoiofmtt forum.dlang.org> <sjinpbubrbzfhmuxobzu forum.dlang.org>
 <nvgnn8$233l$1 digitalmars.com> <dcciyzoojqgueoqymbfq forum.dlang.org>
 <hbvpgnjsvioiamjerije forum.dlang.org> <mqhyiwiehgxnmnoxmbtd forum.dlang.org>
 <nvsv5v$2k40$1 digitalmars.com>
 <mailman.39.1478624757.3405.digitalmars-d puremagic.com>
 <nvt122$2nba$1 digitalmars.com>
 <mailman.43.1478635777.3405.digitalmars-d puremagic.com>
 <o00rpp$n1a$1 digitalmars.com>
In-Reply-To: <o00rpp$n1a$1 digitalmars.com>

--Q7KlMMvq4dqWtX6BBMWdm3Sa4XtAwWRS7
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 11/10/2016 06:07 AM, Nick Sabalausky wrote:
 On 11/08/2016 02:11 PM, Antonio Corbi wrote:
 Maybe this one is useful for you:

 http://eagain.net/articles/git-for-computer-scientists/
On 11/08/2016 03:01 PM, H. S. Teoh via Digitalmars-d wrote:
 Nothing immediately comes to mind, but thanks to Dr. Google, I found
 this page that's sorta helpful:

     http://ericsink.com/vcbe/html/directed_acyclic_graphs.html

 And perhaps these:

     http://eagain.net/articles/git-for-computer-scientists/
     http://marklodato.github.io/visual-git-guide/index-en.html
=20 Ok, so it looks like each node in the DAG is a commit. I'll definitely have to read further. Thanks, both. =20 Although I have my doubts it would explain all the issues I've hit upon=
 with git's CLI. For example: I don't see why annotated tags aren't the
 default. Or why non-annotated ones even exist at all. When I made
 <https://github.com/Abscissa/gen-package-version>, I noticed that it'll=
 *only* work if the version tags are "annotated" tags. Because otherwise=
 "git describe" doesn't retrieve the tag. Doesn't even matter one bit
 *what* the tag's message is, just that it exists. Why? Who knows! It's =
git! Studying git object model in more details actually explains that. Annotated tag is a first class git object (like commit), while plain tag is simply a reference to existing object (like branch). From usability PoV difference can be negligible but impact on git internals is huge. The fact that the default is non-annotated tag is still just plain bad decision though :)
 Its things like that. I'd be surprised if that has much to do with git'=
s
 nature as a "dumb" DAG tool. It's just the general good-design principl=
e
 of "The thing you *want to* or *should* do or *expect* 99% of the time
 should be the DEFAULT, not the obscure incantation".
I think it is related, but is not necessary consequence. My understanding is that for a long time command line design was given zero thoughts on its own - it was directly exposing whatever git does internally with no usability considerations. Which is why it is so awkward to use unless you really do know internal object model in great details. --Q7KlMMvq4dqWtX6BBMWdm3Sa4XtAwWRS7--
Nov 10 2016
next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 11/10/2016 09:31 AM, Dicebot wrote:
 On 11/10/2016 06:07 AM, Nick Sabalausky wrote:
 Its things like that. I'd be surprised if that has much to do with git's
 nature as a "dumb" DAG tool. It's just the general good-design principle
 of "The thing you *want to* or *should* do or *expect* 99% of the time
 should be the DEFAULT, not the obscure incantation".
I think it is related, but is not necessary consequence. My understanding is that for a long time command line design was given zero thoughts on its own - it was directly exposing whatever git does internally with no usability considerations. Which is why it is so awkward to use unless you really do know internal object model in great details.
Yea, see that's the stuff I mean when I say I dislike git's CLI, and would really like to fix with a front-end. A few years back, someone here compared it to driving a car by leaning under the hood pulling on various wires. I like that comparison. Git always did give off that kinda feel to me. Somebody else pointed out gitless. At a quick glance, it looks like it has at least some potential. I'll have to take a closer look. Although I suspect I'll find a bunch of "oh, hmm, I'd have done that differently" just because...well, I tend to be like that ;)
Nov 10 2016
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2016-11-10 06:31, Dicebot wrote:

 I think it is related, but is not necessary consequence. My
 understanding is that for a long time command line design was given zero
 thoughts on its own - it was directly exposing whatever git does
 internally with no usability considerations. Which is why it is so
 awkward to use unless you really do know internal object model in great
 details.
It has the porcelain commands now, compared to the more low level pluming commands. I guess at one point only the pluming commands existed. -- /Jacob Carlborg
Nov 12 2016
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-11-09 20:07, Nick Sabalausky wrote:

 Although I have my doubts it would explain all the issues I've hit upon
 with git's CLI. For example: I don't see why annotated tags aren't the
 default. Or why non-annotated ones even exist at all. When I made
 <https://github.com/Abscissa/gen-package-version>, I noticed that it'll
 *only* work if the version tags are "annotated" tags. Because otherwise
 "git describe" doesn't retrieve the tag. Doesn't even matter one bit
 *what* the tag's message is, just that it exists. Why? Who knows! It's git!
If you add the --tags flag to "describe" it will work. -- /Jacob Carlborg
Nov 12 2016
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 11/12/2016 11:02 AM, Jacob Carlborg wrote:
 On 2016-11-09 20:07, Nick Sabalausky wrote:

 Although I have my doubts it would explain all the issues I've hit upon
 with git's CLI. For example: I don't see why annotated tags aren't the
 default. Or why non-annotated ones even exist at all. When I made
 <https://github.com/Abscissa/gen-package-version>, I noticed that it'll
 *only* work if the version tags are "annotated" tags. Because otherwise
 "git describe" doesn't retrieve the tag. Doesn't even matter one bit
 *what* the tag's message is, just that it exists. Why? Who knows! It's
 git!
If you add the --tags flag to "describe" it will work.
Ok, I see (sorta). But it does work without --tags if it's an annotated tag. Clear as mud. Yea, see, these are the reasons I consider git's cli to be god-awful.
Nov 12 2016
prev sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Tuesday, 8 November 2016 at 16:57:27 UTC, H. S. Teoh wrote:
 On Tue, Nov 08, 2016 at 11:40:31AM -0500, Nick Sabalausky via 
 Digitalmars-d wrote:
 On 11/05/2016 11:48 AM, Marc SchĂŒtz wrote:
 On Saturday, 5 November 2016 at 01:21:48 UTC, Stefan Koch 
 wrote:
 
 I recently lost 3 days of work because of my git-skills.
Unless you haven't committed your work yet, almost everything in Git can be undone. Make a copy of your entire project directory (including .git) and then have a look at `git reflog` around the time the disaster happened. It will show you commit IDs that you can check out.
Yea, but unless you're a git-fu master, sometimes figuring out how to fix whatever got messed up can lose you 3 days of work ;) I really want to make a saner CLI front-end for git, but that would require learning more about git than I really ever want to know :(
If you study carefully the data model of git you will then come to the conclusion that its CLI is not that bad.
 The thing about git is that at its core, it's really very 
 simple. Dumb, even.  It's basically a program for managing a 
 directed acyclic graph (DAG).  That's all there is to it. The 
 rest is just frills.

 Trying to rationalize git in terms of traditional version 
 control systems is what usually causes lots of confusion, 
 incomprehension, and frustration.  To truly grok git, you have 
 to just forget about traditional version control concepts, and 
 think purely in terms of DAGs. Once you do, everything falls 
 into place and it all makes sense in its own peculiar way 
 (including all the flaws :-P).
I describe git not as a vcs but as exacly what Linus Torvalds described it when he first presented it the Kernel mailing list: a directory content state recorder optimized for text files. The DAG is already an abstraction above i.e. a mean to that end.
Nov 08 2016
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 11/08/2016 02:30 PM, Patrick Schluter wrote:
 I describe git not as a vcs but as exacly what Linus Torvalds described
 it when he first presented it the Kernel mailing list: a directory
 content state recorder optimized for text files.
...that's incapable of comprehending the concept of empty directories, contrary to every major file system in use. I never bought the argument about "b/c it tracks content, not files". If that really did explain the limitation, then it wouldn't be capable of handling empty *files* either, and yet it does.
Nov 08 2016
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 11/08/2016 05:54 PM, Nick Sabalausky wrote:
 On 11/08/2016 02:30 PM, Patrick Schluter wrote:
 I describe git not as a vcs but as exacly what Linus Torvalds described
 it when he first presented it the Kernel mailing list: a directory
 content state recorder optimized for text files.
...that's incapable of comprehending the concept of empty directories, contrary to every major file system in use. I never bought the argument about "b/c it tracks content, not files". If that really did explain the limitation, then it wouldn't be capable of handling empty *files* either, and yet it does.
(Pardon the OT)
Nov 08 2016
prev sibling parent Iakh <iaktakh gmail.com> writes:
On Tuesday, 8 November 2016 at 16:40:31 UTC, Nick Sabalausky 
wrote:
 On 11/05/2016 11:48 AM, Marc SchĂŒtz wrote:
 On Saturday, 5 November 2016 at 01:21:48 UTC, Stefan Koch 
 wrote:
 I recently lost 3 days of work because of my git-skills.
Unless you haven't committed your work yet, almost everything in Git can be undone. Make a copy of your entire project directory (including .git) and then have a look at `git reflog` around the time the disaster happened. It will show you commit IDs that you can check out.
Yea, but unless you're a git-fu master, sometimes figuring out how to fix whatever got messed up can lose you 3 days of work ;) I really want to make a saner CLI front-end for git, but that would require learning more about git than I really ever want to know :(
http://gitless.com/
Nov 08 2016
prev sibling next sibling parent deadalnix <deadalnix gmail.com> writes:
On Saturday, 5 November 2016 at 01:21:48 UTC, Stefan Koch wrote:
 I thought about it some more, and decided that I will replace 
 the old interpreter completely in the long run.
 However in order to get something release before 2017, I have 
 to rely on it being there.
Good thinking.
Nov 05 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/4/16 9:21 PM, Stefan Koch wrote:
 I thought about it some more, and decided that I will replace the old
 interpreter completely in the long run.
 However in order to get something release before 2017, I have to rely on
 it being there.
That's good thinking - leave short term to the short term and long term to the long term. As the Romanian proverb goes: "Don't sell the skin of the bear before you shoot it." -- Andrei
Nov 05 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 5 November 2016 at 20:57:06 UTC, Andrei Alexandrescu 
wrote:
 On 11/4/16 9:21 PM, Stefan Koch wrote:
 I thought about it some more, and decided that I will replace 
 the old
 interpreter completely in the long run.
 However in order to get something release before 2017, I have 
 to rely on
 it being there.
That's good thinking - leave short term to the short term and long term to the long term. As the Romanian proverb goes: "Don't sell the skin of the bear before you shoot it." -- Andrei
I would interpret this proverb as referring to sequential consistency. A just a small update today: I just introduced questionable code that enables the interpreter-backed to pass the phobos unitests but will cause the llvm backend to fail. Also, the list of manual bail-outs just grew by a few members. Currently work on Function-Call support is halted in favor of investigating why the bail-outs are necessary. I begun some work on returning Error-Information for the BC-backends. This is rather difficult due to the high degree of separation between the ctfe-engine and the rest of dmd. A nice weekend to all of you. P.S. Phobos and druntime unittests do compile now. After the ability for returning errors is added all unitests will be passed. (Most likely if there are not more bugs inside the engine)
Nov 06 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 6 November 2016 at 15:31:24 UTC, Stefan Koch wrote:
 Phobos and druntime unittests do compile now.
 After the ability for returning errors is added all unitests 
 will be passed. (Most likely if there are not more bugs inside 
 the engine)
I just have just implemented the ability to error out. And I am still not passing interpret3.d. That is one hard unit-test. Currently I pass around 25% of interpret3.d.
Nov 06 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 6 November 2016 at 22:07:05 UTC, Stefan Koch wrote:
 On Sunday, 6 November 2016 at 15:31:24 UTC, Stefan Koch wrote:
 Phobos and druntime unittests do compile now.
 After the ability for returning errors is added all unitests 
 will be passed. (Most likely if there are not more bugs inside 
 the engine)
I just have just implemented the ability to error out. And I am still not passing interpret3.d. That is one hard unit-test. Currently I pass around 25% of interpret3.d.
I consider the engine ready to be merged into the feature branch. There are still a few obscure failures. But nothing preventing an alpha release.
Nov 07 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 7 November 2016 at 10:02:37 UTC, Stefan Koch wrote:
 On Sunday, 6 November 2016 at 22:07:05 UTC, Stefan Koch wrote:
 On Sunday, 6 November 2016 at 15:31:24 UTC, Stefan Koch wrote:
 Phobos and druntime unittests do compile now.
 After the ability for returning errors is added all unitests 
 will be passed. (Most likely if there are not more bugs 
 inside the engine)
I just have just implemented the ability to error out. And I am still not passing interpret3.d. That is one hard unit-test. Currently I pass around 25% of interpret3.d.
I consider the engine ready to be merged into the feature branch. There are still a few obscure failures. But nothing preventing an alpha release.
Update I tracked down a bug that prevented a function from std.allocator to execute correctly. Slowly but steadily new ctfe becomes more robust.
Nov 07 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 7 November 2016 at 14:13:04 UTC, Stefan Koch wrote:
 On Monday, 7 November 2016 at 10:02:37 UTC, Stefan Koch wrote:
 On Sunday, 6 November 2016 at 22:07:05 UTC, Stefan Koch wrote:
 On Sunday, 6 November 2016 at 15:31:24 UTC, Stefan Koch wrote:
 Phobos and druntime unittests do compile now.
 After the ability for returning errors is added all unitests 
 will be passed. (Most likely if there are not more bugs 
 inside the engine)
I just have just implemented the ability to error out. And I am still not passing interpret3.d. That is one hard unit-test. Currently I pass around 25% of interpret3.d.
I consider the engine ready to be merged into the feature branch. There are still a few obscure failures. But nothing preventing an alpha release.
Update I tracked down a bug that prevented a function from std.allocator to execute correctly. Slowly but steadily new ctfe becomes more robust.
Another bug was identified. There are severe issues with the interpreter back-end and it's 32bit only legacy. Fixing these issues will degrade performance of the interpreter by roughly 7-15%. Maybe there is still a way around it, but so far my more performance-oriented approaches are failing to fix the problems. Errors now know their location, and most of the time assertions thrown be the new engine are close enough to the old interpreters one. However my errors have less information in them due to design limitations. It is possible to lift those but only with significant effort. Slightly OT. I feel like I hit every bump in the road to a faster interpreter, and I know there are a a lot more ahead. Today I spent 3 hours to find a bug which caused a division by zero. It turned out that 64bit large function arguments where silently skipped. Causing the interpreter to read zero initialized memory instead of the actual arguments. However I suspected the division code to be at fault and wasted a lot of time of second-guessing correct code. Having these implementation challenges intermixed with design and performance considerations can be taxing sometimes. Without my bytecode-unitests which are ctfeable it would be almost impossible to for me to work at the pace I am working. Simply because I get direct feedback if something goes wrong because the compile will abort after less then a second. These kinds of things are the true strength of ctfe. CTFE is the one of the greatest development assets one can have. Because it transforms the compile-debug-edit cycle into a compile-edit cycle.
Nov 07 2016
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 11/05/2016 04:57 PM, Andrei Alexandrescu wrote:
 That's good thinking - leave short term to the short term and long term
 to the long term. As the Romanian proverb goes: "Don't sell the skin of
 the bear before you shoot it." -- Andrei
I really wish Google would take that to heart. They seem to make a habit of ripping things out *before* having replacements in place. I think they just simply love deleting code.
Nov 08 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Tue, 08 Nov 2016 11:44:50 -0500, Nick Sabalausky wrote:
 I really wish Google would take that to heart. They seem to make a habit
 of ripping things out *before* having replacements in place.
 
 I think they just simply love deleting code.
I've seen this more internally than externally. The joke was that there are two versions of everything: the deprecated one, and the one that doesn't work yet.
Nov 08 2016
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 11/08/2016 11:40 PM, Chris Wright wrote:
 On Tue, 08 Nov 2016 11:44:50 -0500, Nick Sabalausky wrote:
 I really wish Google would take that to heart. They seem to make a habit
 of ripping things out *before* having replacements in place.

 I think they just simply love deleting code.
I've seen this more internally than externally. The joke was that there are two versions of everything: the deprecated one, and the one that doesn't work yet.
It's REALLY poor project management.
Nov 09 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 10 November 2016 at 04:22:10 UTC, Nick Sabalausky 
wrote:
 On 11/08/2016 11:40 PM, Chris Wright wrote:
 On Tue, 08 Nov 2016 11:44:50 -0500, Nick Sabalausky wrote:
 I really wish Google would take that to heart. They seem to 
 make a habit
 of ripping things out *before* having replacements in place.

 I think they just simply love deleting code.
I've seen this more internally than externally. The joke was that there are two versions of everything: the deprecated one, and the one that doesn't work yet.
It's REALLY poor project management.
Please cut down on the OT guys. This thread serves as my development log :) As well as providing a place for discussion of the new engine. Though that discussion seems to be very quiet at the moment.
Nov 09 2016
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 11/09/2016 11:27 PM, Stefan Koch wrote:
 Please cut down on the OT guys.

 This thread serves as my development log :)
 As well as providing a place for discussion of the new engine.

 Though that discussion seems to be very quiet at the moment.
Although the default web-based front-end for this newsgroup suggests otherwise, this *is* a tree-based message system. Uninteresting sub-trees can simply be ignored without resorting to content policing.
Nov 09 2016
prev sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Friday, 4 November 2016 at 10:59:25 UTC, Marc SchĂŒtz wrote:
 In this case better make sure to remove the resulting dead code 
 from the old interpreter, otherwise it will become and 
 unmaintainable mess in the long run.
interpreter is used all over the code, and it is a set of visitors doing various things. that is, it is not possible to remove "dead code" from it, only replace it as a whole. which is a great achievement, but completely unpractical. it is way, way easier to build the basic VM that runs alongside the old interpreter, and call old one if new VM found something it can't process yet. this way new VM can be used to speed up some parts, but nothing will be broken, and VM can be refined gradually. also, it is great for testing: one can call new and old interpreters and compare results. as new VM is completely independent of old interpreter, the code won't be any messier than it is now. ;-) from a technical POV CTFE interpreter is just a codegen: it receiving lowered AST, visit nodes and does some actions. one can have many codegens in frontend code, they won't hurt each other. ;-)
Nov 04 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 5 November 2016 at 03:01:48 UTC, ketmar wrote:
 On Friday, 4 November 2016 at 10:59:25 UTC, Marc SchĂŒtz wrote:
 In this case better make sure to remove the resulting dead 
 code from the old interpreter, otherwise it will become and 
 unmaintainable mess in the long run.
interpreter is used all over the code, and it is a set of visitors doing various things. that is, it is not possible to remove "dead code" from it, only replace it as a whole.
Yes what ketmar said. It is looking good, current testsuite-failures are caused by the new engines missing checks for ub, and the inability to signal those errors. Figuring out a good mechanism for error-signaling is the next challenge.
Nov 05 2016
prev sibling next sibling parent reply Faux Amis <faux amis.com> writes:
Any reason for the infinite depth update posting style?
I would have loved to see each update to be a child of the root post 
with its own discussions tree.
Currently, the posts are quite unreadable in tree view (thunderbird).


On 2016-10-31 14:29, Stefan Koch wrote:
 Hi Guys, since I got a few complaints about giving minor status updates
 in the announce group, I am opening this thread.
Dec 08 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 8 December 2016 at 19:49:47 UTC, Faux Amis wrote:
 Any reason for the infinite depth update posting style?
 I would have loved to see each update to be a child of the root 
 post with its own discussions tree.
 Currently, the posts are quite unreadable in tree view 
 (thunderbird).


 On 2016-10-31 14:29, Stefan Koch wrote:
 Hi Guys, since I got a few complaints about giving minor 
 status updates
 in the announce group, I am opening this thread.
The reason it that I don't use the tree view. I just answer to myself every time I have an update. It looks fine on the web-frontend.
Dec 08 2016
parent Faux Amis <faux amis.com> writes:
On 2016-12-09 00:00, Stefan Koch wrote:
 On Thursday, 8 December 2016 at 19:49:47 UTC, Faux Amis wrote:
 Any reason for the infinite depth update posting style?
 I would have loved to see each update to be a child of the root post
 with its own discussions tree.
 Currently, the posts are quite unreadable in tree view (thunderbird).


 On 2016-10-31 14:29, Stefan Koch wrote:
 Hi Guys, since I got a few complaints about giving minor status updates
 in the announce group, I am opening this thread.
The reason it that I don't use the tree view. I just answer to myself every time I have an update. It looks fine on the web-frontend.
Might you consider making use of the tree data structure? It really has quite nice properties with regard to discussions.. (check the threaded view mode, everybody else seems to use it like that) Anyway, I kinda missed the updates because I thought it was just a very deep discussion until I noticed it was mostly you posting ;)
Dec 09 2016
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 31 October 2016 at 13:29:01 UTC, Stefan Koch wrote:
 Hi Guys, since I got a few complaints about giving minor status 
 updates in the announce group, I am opening this thread.

 I will start with giving an overview of what works and what 
 does not work.

 Currently the only basic type you can do arithmetic on is int.
 Altough you can compare longs since a few days.

 These are the constructs that will work.

 - foreach on static arrays strings and range-foreach  (those 
 kinds (0 .. 64)).
 - switches (even deeply nested ones)
 - for and while loops
 - ternary expressions (? :)
 - if and else statements (as long as you don't use && and || )
 - lables and gotos
 - arithmetic expressions as well as post and pre increment and 
 decrement

 Constructs that will not work (but are actively worked on)

 - assignment to static array cells
 - long ulong arithmetic.
 - function calls
 - dynamic arrays and slices
 - pointers
 - structs
 - && and ||
 - sliceing

 Constructs that will not work and are futher down the list.

 - classes
 - closures
 - boundschecks
 - asserts

 Please note that there will probably be bugs all over the place.
 So even the working features might not be working completely.
FunctionPointers and plain function calls are currently in progress! Also I made further performance improvements to the Bytecode-generator. As of now it needs around 50 nano-seconds per generated instruction. Generation + Interpretation execution of trivial functions now takes LESS then a microsecond on contemporary machines!
Dec 09 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 10 December 2016 at 07:24:10 UTC, Stefan Koch wrote:
 FunctionPointers and plain function calls are currently in 
 progress!
 Also I made further performance improvements to the 
 Bytecode-generator.
 As of now it needs around 50 nano-seconds per generated 
 instruction.
 Generation + Interpretation execution of trivial functions now 
 takes LESS then a microsecond on contemporary machines!
Function-call support has broken the bailout capability. I am working on restoring it.
Dec 10 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 10 December 2016 at 18:51:32 UTC, Stefan Koch wrote:
 On Saturday, 10 December 2016 at 07:24:10 UTC, Stefan Koch 
 wrote:
 FunctionPointers and plain function calls are currently in 
 progress!
 Also I made further performance improvements to the 
 Bytecode-generator.
 As of now it needs around 50 nano-seconds per generated 
 instruction.
 Generation + Interpretation execution of trivial functions now 
 takes LESS then a microsecond on contemporary machines!
Function-call support has broken the bailout capability. I am working on restoring it.
I just fixed the bailout. certain switch-statements. And made the creation of bytecode even cheaper.
Dec 10 2016
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 31 October 2016 at 13:29:01 UTC, Stefan Koch wrote:
 Hi Guys, since I got a few complaints about giving minor status 
 updates in the announce group, I am opening this thread.

 I will start with giving an overview of what works and what 
 does not work.

 Currently the only basic type you can do arithmetic on is int.
 Altough you can compare longs since a few days.

 These are the constructs that will work.

 - foreach on static arrays strings and range-foreach  (those 
 kinds (0 .. 64)).
 - switches (even deeply nested ones)
 - for and while loops
 - ternary expressions (? :)
 - if and else statements (as long as you don't use && and || )
 - lables and gotos
 - arithmetic expressions as well as post and pre increment and 
 decrement

 Constructs that will not work (but are actively worked on)

 - assignment to static array cells
 - long ulong arithmetic.
 - function calls
 - dynamic arrays and slices
 - pointers
 - structs
 - && and ||
 - sliceing

 Constructs that will not work and are futher down the list.

 - classes
 - closures
 - boundschecks
 - asserts

 Please note that there will probably be bugs all over the place.
 So even the working features might not be working completely.
I just fixed a really nasty bug that prevented the following code from working in newCTFE. int computeFib(int n) { int t = 1; int result = 0; while(n--) { result = t - result; t = t + result; } return result; } static assert(computeFib(12) == 144); The line result = t - result is the one triggering the bug. Because newCTFE broke it down like this result = t; result -= result; This was not found by the compilers testsuite! I guess we should add it.
Dec 10 2016
parent Basile B. <b2.temp gmx.com> writes:
On Saturday, 10 December 2016 at 11:03:11 UTC, Stefan Koch wrote:
 This was not found by the compilers testsuite!
Have you already thought to use fuzz testing ? I'm sure it would find some weird failure cases. The other day I've found that `enum a=null+null;` triggers a CTFE ICE. It's on the old engine but a tool with a "mean and nasty" approach could have found it.
Dec 10 2016
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 31 October 2016 at 13:29:01 UTC, Stefan Koch wrote:
 Hi Guys, since I got a few complaints about giving minor status 
 updates in the announce group, I am opening this thread.

 I will start with giving an overview of what works and what 
 does not work.

 Currently the only basic type you can do arithmetic on is int.
 Altough you can compare longs since a few days.

 These are the constructs that will work.

 - foreach on static arrays strings and range-foreach  (those 
 kinds (0 .. 64)).
 - switches (even deeply nested ones)
 - for and while loops
 - ternary expressions (? :)
 - if and else statements (as long as you don't use && and || )
 - lables and gotos
 - arithmetic expressions as well as post and pre increment and 
 decrement

 Constructs that will not work (but are actively worked on)

 - assignment to static array cells
 - long ulong arithmetic.
 - function calls
 - dynamic arrays and slices
 - pointers
 - structs
 - && and ||
 - sliceing

 Constructs that will not work and are futher down the list.

 - classes
 - closures
 - boundschecks
 - asserts

 Please note that there will probably be bugs all over the place.
 So even the working features might not be working completely.
Remember those numbers ? [root localhost dmd]# time src/dmd -c testSettingArrayLength.d -bc-ctfe 2147385345u 536821761u 4294639619u real 0m0.114s user 0m0.110s sys 0m0.003s This is how they look like after my performance fixes: real 0m0.047s user 0m0.043s sys 0m0.000s
Dec 11 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 11 December 2016 at 08:33:49 UTC, Stefan Koch wrote:
 On Monday, 31 October 2016 at 13:29:01 UTC, Stefan Koch wrote:
 Hi Guys, since I got a few complaints about giving minor 
 status updates in the announce group, I am opening this thread.

 I will start with giving an overview of what works and what 
 does not work.

 Currently the only basic type you can do arithmetic on is int.
 Altough you can compare longs since a few days.

 These are the constructs that will work.

 - foreach on static arrays strings and range-foreach  (those 
 kinds (0 .. 64)).
 - switches (even deeply nested ones)
 - for and while loops
 - ternary expressions (? :)
 - if and else statements (as long as you don't use && and || )
 - lables and gotos
 - arithmetic expressions as well as post and pre increment and 
 decrement

 Constructs that will not work (but are actively worked on)

 - assignment to static array cells
 - long ulong arithmetic.
 - function calls
 - dynamic arrays and slices
 - pointers
 - structs
 - && and ||
 - sliceing

 Constructs that will not work and are futher down the list.

 - classes
 - closures
 - boundschecks
 - asserts

 Please note that there will probably be bugs all over the 
 place.
 So even the working features might not be working completely.
Remember those numbers ? [root localhost dmd]# time src/dmd -c testSettingArrayLength.d -bc-ctfe 2147385345u 536821761u 4294639619u real 0m0.114s user 0m0.110s sys 0m0.003s This is how they look like after my performance fixes: real 0m0.047s user 0m0.043s sys 0m0.000s
Disregard That! I wasn't paying attention The lower numbers are produced by ldc! The performance fixes did lower the overall overhead even more though. Which means bytecode generation will not even show up among the top 50 functions in dmd,
Dec 11 2016
parent reply Anonymouse <asdf asdf.net> writes:
On Sunday, 11 December 2016 at 08:37:28 UTC, Stefan Koch wrote:
 Disregard That!
 I wasn't paying attention
 The lower numbers are produced by ldc!
 The performance fixes did lower the overall overhead even more 
 though.
 Which means bytecode generation will not even show up among the 
 top 50 functions in dmd,
Would you say it has ended up being more or less (or roughly equal) work than you initially expected? And keep up the good work!
Dec 11 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 11 December 2016 at 09:05:26 UTC, Anonymouse wrote:
 Would you say it has ended up being more or less (or roughly 
 equal) work than you initially expected?

 And keep up the good work!
I did expect a lot of work. But with debugging it exceeded my expectations. Originally I planned for this to over in 3 months. Now I am going finish the 6th month and it won't be done completely. I am just happy that I could keep the bytecode-overhead so low that no one will notice a slow-down when we switch engines. Work on function-calls is still progressing.
Dec 11 2016
next sibling parent Ethan Watson <gooberman gmail.com> writes:
On Sunday, 11 December 2016 at 09:13:41 UTC, Stefan Koch wrote:
 Originally I planned for this to over in 3 months.
 Now I am going finish the 6th month and it won't be done 
 completely.
I hear that. As an anecdote, by Binderoo work has been functional for months; but it's only now becoming usable by normal people. And then once normal people use it, there's gonna be tons of bugs and edge cases that I need to blitz through. But I will have a framework that people can use to do Unity-style development of code using D as their rapid iteration language at the end of it. I've been moving more stuff out of templates and in to CTFE in anticipation for this work hitting main.
Dec 11 2016
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 11 December 2016 at 09:13:41 UTC, Stefan Koch wrote:
 On Sunday, 11 December 2016 at 09:05:26 UTC, Anonymouse wrote:
 Would you say it has ended up being more or less (or roughly 
 equal) work than you initially expected?

 And keep up the good work!
I did expect a lot of work. But with debugging it exceeded my expectations. Originally I planned for this to over in 3 months. Now I am going finish the 6th month and it won't be done completely. I am just happy that I could keep the bytecode-overhead so low that no one will notice a slow-down when we switch engines. Work on function-calls is still progressing.
I just fixed labled for and foreach! uint fn1() { uint result; L: foreach (i; 0 .. 24) { if (i != 4) continue L; result += i; } return result; } static assert(fn1() == 4); This will work now. Whereas before it ignored the continue.
Dec 12 2016
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Hi Guys, I just fixed the LLVM-Backend a little.
It's about 4000 times slower to start up then the interpreter.
And has 1000 microseconds overhead per evaluation.

If you don't want to run a raytracer at compiletime I doubt that 
the llvm backend is the right one for you.

That said, it's a great fit for getting started in compiler 
development.
It can also be used to show potential optimization our own 
peephole optimizer could do.

The llvm backend will currently not run the whole test-suite 
because it fails on ++ (because of a missing load-store cycle I 
suspect)...
The strange thing here is, that it only fails sometimes :)
Dec 12 2016
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-12-13 08:21, Stefan Koch wrote:
 Hi Guys, I just fixed the LLVM-Backend a little.
 It's about 4000 times slower to start up then the interpreter.
 And has 1000 microseconds overhead per evaluation.

 If you don't want to run a raytracer at compiletime I doubt that the
 llvm backend is the right one for you.
IIRC, the Higgs JavaScript JIT compiler doesn't always use JIT because sometimes it faster to interpret because JIT has some overhead. Not sure if this is still the case and how it chooses when to interpret or when to JIT. -- /Jacob Carlborg
Dec 12 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 13 December 2016 at 07:53:56 UTC, Jacob Carlborg 
wrote:
 On 2016-12-13 08:21, Stefan Koch wrote:
 Hi Guys, I just fixed the LLVM-Backend a little.
 It's about 4000 times slower to start up then the interpreter.
 And has 1000 microseconds overhead per evaluation.

 If you don't want to run a raytracer at compiletime I doubt 
 that the
 llvm backend is the right one for you.
IIRC, the Higgs JavaScript JIT compiler doesn't always use JIT because sometimes it faster to interpret because JIT has some overhead. Not sure if this is still the case and how it chooses when to interpret or when to JIT.
Higgs IR is high-level which makes a translation step to a lower ir and then x86 unavoidable. My architecture allows to emit assembly directly (while everything is still hot and steamy in the cache). Therefore JIT will _ALWAYS_ be faster then interpretation. Given that a executeble-page is already allocated. Since getting the page from the OS can take a long time.
Dec 13 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 13 December 2016 at 07:21:07 UTC, Stefan Koch wrote:
 Hi Guys, I just fixed the LLVM-Backend a little.
 It's about 4000 times slower to start up then the interpreter.
 And has 1000 microseconds overhead per evaluation.

 If you don't want to run a raytracer at compiletime I doubt 
 that the llvm backend is the right one for you.

 That said, it's a great fit for getting started in compiler 
 development.
 It can also be used to show potential optimization our own 
 peephole optimizer could do.

 The llvm backend will currently not run the whole test-suite 
 because it fails on ++ (because of a missing load-store cycle I 
 suspect)...
 The strange thing here is, that it only fails sometimes :)
Do you use MCJIT or some special sauce made with Orc ? Anyway, yes, LLVM's JIT is heavy duty, good for long running code but probably not so much for CTFE which are mostly one shot.
Dec 14 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 15 December 2016 at 06:04:12 UTC, deadalnix wrote:
 On Tuesday, 13 December 2016 at 07:21:07 UTC, Stefan Koch wrote:

 Do you use MCJIT or some special sauce made with Orc ?

 Anyway, yes, LLVM's JIT is heavy duty, good for long running 
 code but probably not so much for CTFE which are mostly one 
 shot.
MCJIT, mostly orc is not available with the old set of sdc-llvm headers I happen to use. Orc is too much of a pain to setup if you are not planning on rolling with llvm to begin with. Also LLVM has a but of weirdness to it. Where it'll miscompile perfectly valid code. (I happen to use llvm 3.4)
Dec 15 2016
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
I made good progress on function calls today.
By deferring them we get both better bytecode-generator 
performance and simpler code.
Since the deferring step seems to work, you can expect calls to 
work by the end of next week.

When function-calls work, I will direct efforts towards correct 
handling of || and &&,
since it is very annoying to work around the lack of them :)

Also start-up times for ctfe are still dropping.
When the old ctfe is completely replaced, (hopefully around 
February)
I can reasonably expect compile-times to be reduced by over 65% 
(given that ctfe was the bottleneck)

Cheers,
Stefan
Dec 13 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 13 December 2016 at 12:04:31 UTC, Stefan Koch wrote:
 I made good progress on function calls today.
 By deferring them we get both better bytecode-generator 
 performance and simpler code.
 Since the deferring step seems to work, you can expect calls to 
 work by the end of next week.

 When function-calls work, I will direct efforts towards correct 
 handling of || and &&,
 since it is very annoying to work around the lack of them :)

 Also start-up times for ctfe are still dropping.
 When the old ctfe is completely replaced, (hopefully around 
 February)
 I can reasonably expect compile-times to be reduced by over 65% 
 (given that ctfe was the bottleneck)

 Cheers,
 Stefan
String-Switches are supported now. And just to make a point why I am pissed about templates sometimes : The following paste contains the IR generated by a Phobos template which was miscompiled until very recently. oh ... pastie does not accept more then 64K :( Ah pastebin does. http://pastebin.com/Jfh1NH3m
Dec 14 2016
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 14 December 2016 at 08:40:34 UTC, Stefan Koch wrote:
 String-Switches are supported now.
Cool, this is a crucial requirement for pattern matching in parser generators, like Pegged and alikes. Is it worth giving newCTFE a try with these things now?
Dec 15 2016
next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 15 December 2016 at 09:21:10 UTC, Nordlöw wrote:
 Cool, this is a crucial requirement for pattern matching in 
 parser generators, like Pegged and alikes. Is it worth giving 
 newCTFE a try with these things now?
BTW, does newCTFE string-switches use O(1) lookups?
Dec 15 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 15 December 2016 at 09:23:07 UTC, Nordlöw wrote:
 On Thursday, 15 December 2016 at 09:21:10 UTC, Nordlöw wrote:
 Cool, this is a crucial requirement for pattern matching in 
 parser generators, like Pegged and alikes. Is it worth giving 
 newCTFE a try with these things now?
BTW, does newCTFE string-switches use O(1) lookups?
Nope. Currently it lowers to an if else chain. Which is O(N), though still faster then calling the runtime :) I just fixed Assigning static arrays from ArrayLiterals. .... And now the phobos-compiletime tests pass!
Dec 15 2016
next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 15 December 2016 at 12:06:46 UTC, Stefan Koch wrote:
 I just fixed Assigning static arrays  from ArrayLiterals.
 ....
 And now the phobos-compiletime tests pass!
Wonderful! I just realized that fast string-switch is only necessary for parse generators when the generated parse is evaluated at compile-time :)
Dec 15 2016
prev sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 15 December 2016 at 12:06:46 UTC, Stefan Koch wrote:
 And now the phobos-compiletime tests pass!
How do you run the phobos compile-time tests? make unittest does do it all right?
Dec 15 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 15 December 2016 at 13:38:36 UTC, Nordlöw wrote:
 On Thursday, 15 December 2016 at 12:06:46 UTC, Stefan Koch 
 wrote:
 And now the phobos-compiletime tests pass!
How do you run the phobos compile-time tests? make unittest does do it all right?
yes. I run make -f posix.mak unittest.
Dec 15 2016
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 12/15/16 10:21 AM, Nordlöw wrote:
 On Wednesday, 14 December 2016 at 08:40:34 UTC, Stefan Koch wrote:
 String-Switches are supported now.
Cool, this is a crucial requirement for pattern matching in parser generators, like Pegged and alikes. Is it worth giving newCTFE a try with these things now?
Afaik string switches are implemented as a fairly slow hash table. Optimal solutions like building a trie would be a nice enhancement. --- Dmitry Olshansky
Dec 15 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 15 December 2016 at 23:05:55 UTC, Dmitry Olshansky 
wrote:
 Afaik string switches are implemented as a fairly slow hash 
 table. Optimal solutions like building a trie would be a nice 
 enhancement.

 ---
 Dmitry Olshansky
Tries take up alot of memory for sparse tables. I would like to avoid them. Futhermore, CTFE is targeted towards smaller functions with low iteration count. Therefore producing bytecode as fast as possible is priority. This function from std.uri does now compile and execute correctly. private enum { URI_Alpha = 1, URI_Reserved = 2, URI_Mark = 4, URI_Digit = 8, URI_Hash = 0x10, // '#' } immutable ubyte[128] uri_flags = // indexed by character ({ ubyte[128] uflags; // Compile time initialize uflags['#'] |= URI_Hash; foreach (c; 'A' .. 'Z' + 1) { uflags[c] |= URI_Alpha; uflags[c + 0x20] |= URI_Alpha; // lowercase letters } foreach (c; '0' .. '9' + 1) uflags[c] |= URI_Digit; foreach (c; ";/?: &=+$,") uflags[c] |= URI_Reserved; foreach (c; "-_.!~*'()") uflags[c] |= URI_Mark; return uflags; })(); I have to say I was surprised myself :) It takes around 30 us to execute, which can be improved for sure :)
Dec 15 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 15 December 2016 at 23:37:50 UTC, Stefan Koch wrote:
 On Thursday, 15 December 2016 at 23:05:55 UTC, Dmitry Olshansky 
 wrote:
 Afaik string switches are implemented as a fairly slow hash 
 table. Optimal solutions like building a trie would be a nice 
 enhancement.

 ---
 Dmitry Olshansky
Tries take up alot of memory for sparse tables. I would like to avoid them. Futhermore, CTFE is targeted towards smaller functions with low iteration count. Therefore producing bytecode as fast as possible is priority. This function from std.uri does now compile and execute correctly. private enum { URI_Alpha = 1, URI_Reserved = 2, URI_Mark = 4, URI_Digit = 8, URI_Hash = 0x10, // '#' } immutable ubyte[128] uri_flags = // indexed by character ({ ubyte[128] uflags; // Compile time initialize uflags['#'] |= URI_Hash; foreach (c; 'A' .. 'Z' + 1) { uflags[c] |= URI_Alpha; uflags[c + 0x20] |= URI_Alpha; // lowercase letters } foreach (c; '0' .. '9' + 1) uflags[c] |= URI_Digit; foreach (c; ";/?: &=+$,") uflags[c] |= URI_Reserved; foreach (c; "-_.!~*'()") uflags[c] |= URI_Mark; return uflags; })(); I have to say I was surprised myself :) It takes around 30 us to execute, which can be improved for sure :)
Hey Guys, I just had a mental breakthrough in regards to handling doubly in-directed read-modify-write operations. such as changing the length of an array through a pointer. Refernce values will carry a typed pointer to their origin thereby allowing them to be treated like normal values most of the time, and only receive special treatment on assignment. (Which is where they have to emit a Store32 Instead of a Set) As this has design implications, I will resolve this before continuing function-call support. In fact, I think I will have to delay call support until most of the ABI issues are worked out.
Dec 15 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Hey Guys bad news, Regressions.
The handling of default constructed Struct-Literals inside of 
newCTFE is broken.

I have no idea were this came from all of a sudden.
I was in crunch-mode to get the phobos unittest to compile, I 
don't remember the last 72 hours at all :)
So it might take a while until the regression is resolved.
I apologize for the inconvenience.
Dec 16 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 16 December 2016 at 09:32:13 UTC, Stefan Koch wrote:
 Hey Guys bad news, Regressions.
 The handling of default constructed Struct-Literals inside of 
 newCTFE is broken.

 I have no idea were this came from all of a sudden.
 I was in crunch-mode to get the phobos unittest to compile, I 
 don't remember the last 72 hours at all :)
 So it might take a while until the regression is resolved.
 I apologize for the inconvenience.
Apparently the regression was introduce in I confirmed that the regression was introduced in https://github.com/dlang/dmd/pull/6186/commits/33dbf5aee7d42b1929fd62ef176bd299c4da8ac5 Which suggests that it only worked by chance before :(
Dec 16 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 16 December 2016 at 10:26:15 UTC, Stefan Koch wrote:
 On Friday, 16 December 2016 at 09:32:13 UTC, Stefan Koch wrote:
 Hey Guys bad news, Regressions.
 The handling of default constructed Struct-Literals inside of 
 newCTFE is broken.

 I have no idea were this came from all of a sudden.
 I was in crunch-mode to get the phobos unittest to compile, I 
 don't remember the last 72 hours at all :)
 So it might take a while until the regression is resolved.
 I apologize for the inconvenience.
Apparently the regression was introduce in I confirmed that the regression was introduced in https://github.com/dlang/dmd/pull/6186/commits/33dbf5aee7d42b1929fd62ef176bd299c4da8ac5 Which suggests that it only worked by chance before :(
It's confirmed. The test was not thorough enough. The gist with the current newCTFE compatible code with was updated. https://gist.github.com/UplinkCoder/89faa06311e417aa93ea99bc92934d3e
Dec 16 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 16 December 2016 at 10:33:32 UTC, Stefan Koch wrote:
 On Friday, 16 December 2016 at 10:26:15 UTC, Stefan Koch wrote:
 On Friday, 16 December 2016 at 09:32:13 UTC, Stefan Koch wrote:
 Hey Guys bad news, Regressions.
 The handling of default constructed Struct-Literals inside of 
 newCTFE is broken.

 I have no idea were this came from all of a sudden.
 I was in crunch-mode to get the phobos unittest to compile, I 
 don't remember the last 72 hours at all :)
 So it might take a while until the regression is resolved.
 I apologize for the inconvenience.
Apparently the regression was introduce in I confirmed that the regression was introduced in https://github.com/dlang/dmd/pull/6186/commits/33dbf5aee7d42b1929fd62ef176bd299c4da8ac5 Which suggests that it only worked by chance before :(
It's confirmed. The test was not thorough enough. The gist with the current newCTFE compatible code with was updated. https://gist.github.com/UplinkCoder/89faa06311e417aa93ea99bc92934d3e
Another bug is going to be fixed soon, causing infinite recursion on structs that have pointer of their own type. This will allow me to remove a few bail-outs and compile more of phobos.
Dec 16 2016
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
I just fixed gotos again.
While researching unusually large performance drops, I found the 
cause for a rather nasty bug. (It was an off-by-one error again 
:))

It turned out that there were circumstances where some gotos 
would be dropped and the corresponding fixup jumps where never 
emitted.

Due to a curious situation in the dmd ast, I had to make jumps 
without addresses into nops.
This was hiding the bug.

BTW supporting NOPS has been a huge pain for some time now.
I will look into fixing my dmd-ast handling to make them 
unnecessary.
Jumps that don't know where they are supposed to go should not be 
ignored but an error instead!
Dec 16 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 17 December 2016 at 04:14:29 UTC, Stefan Koch wrote:
 I just fixed gotos again.
 While researching unusually large performance drops, I found 
 the cause for a rather nasty bug. (It was an off-by-one error 
 again :))

 It turned out that there were circumstances where some gotos 
 would be dropped and the corresponding fixup jumps where never 
 emitted.

 Due to a curious situation in the dmd ast, I had to make jumps 
 without addresses into nops.
 This was hiding the bug.

 BTW supporting NOPS has been a huge pain for some time now.
 I will look into fixing my dmd-ast handling to make them 
 unnecessary.
 Jumps that don't know where they are supposed to go should not 
 be ignored but an error instead!
I rectified a few more performance problems. The StringEquals macro put alot of pressure on the bytecode generator. Therefore I implemented it as an instruction. Interestingly this did not speed up the execution by as much as I would have expected, maybe I need better test-cases ?
Dec 16 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Miscompiling phobos code is down to 4 functions.
std.range.primitives.{front, empty, back}
and std.traits.defaultParameters.
Note, StructLiterals are still regressed.
I am looking into it.
Dec 17 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 17 December 2016 at 08:53:07 UTC, Stefan Koch wrote:
 Miscompiling phobos code is down to 4 functions.
 std.range.primitives.{front, empty, back}
 and std.traits.defaultParameters.
 Note, StructLiterals are still regressed.
 I am looking into it.
Fixed the regression. It was an off-by-one error in the check of the member-count. causing the last field of a struct to be considered not part of the struct :p
Dec 17 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 17 December 2016 at 09:39:02 UTC, Stefan Koch wrote:
 On Saturday, 17 December 2016 at 08:53:07 UTC, Stefan Koch 
 wrote:
 Miscompiling phobos code is down to 4 functions.
 std.range.primitives.{front, empty, back}
 and std.traits.defaultParameters.
 Note, StructLiterals are still regressed.
 I am looking into it.
Fixed the regression. It was an off-by-one error in the check of the member-count. causing the last field of a struct to be considered not part of the struct :p
Many of the forced bailouts could be removed. Soon newCTFE will compile Phobos without any blacklisted functions!
Dec 19 2016
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 19 December 2016 at 08:26:00 UTC, Stefan Koch wrote:
 On Saturday, 17 December 2016 at 09:39:02 UTC, Stefan Koch 
 wrote:
 On Saturday, 17 December 2016 at 08:53:07 UTC, Stefan Koch 
 wrote:
 Miscompiling phobos code is down to 4 functions.
 std.range.primitives.{front, empty, back}
 and std.traits.defaultParameters.
 Note, StructLiterals are still regressed.
 I am looking into it.
Fixed the regression. It was an off-by-one error in the check of the member-count. causing the last field of a struct to be considered not part of the struct :p
Many of the forced bailouts could be removed. Soon newCTFE will compile Phobos without any blacklisted functions!
Excellent! Keep up the great work!
Dec 19 2016
prev sibling next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 19 December 2016 at 08:26:00 UTC, Stefan Koch wrote:
 Soon newCTFE will compile Phobos without any blacklisted 
 functions!
Awesome!
Dec 19 2016
prev sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 19 December 2016 at 08:26:00 UTC, Stefan Koch wrote:
 Soon newCTFE will compile Phobos without any blacklisted 
 functions!
Which functions are still blacklisted?
Dec 19 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 19 December 2016 at 10:11:18 UTC, Nordlöw wrote:
 On Monday, 19 December 2016 at 08:26:00 UTC, Stefan Koch wrote:
 Soon newCTFE will compile Phobos without any blacklisted 
 functions!
Which functions are still blacklisted?
Check here https://github.com/UplinkCoder/dmd/blob/newCTFE/src/ctfe_bc.d#L120 currently it's still the range primitives and __lambda2 because of ParameterTypeTuple stuff.
Dec 19 2016
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Hey Guys I wanted to give a quick update.


I recently fixed an interesting bug that happened de-referencing 
null pointers to the heap.
That would cause arrays to be overwritten, when passed as 
arguments.
That bug had me stuck for a few hours.

Getting Function-Calls right will be impossible,
if the ABI issues remain.

Currently it's about the handling of pointers.
My architecture forbids pointers to the stack, therefore every 
time you take the address of something it needs to be pushed onto 
the heap.
And then the heap-value and the stack-value needs to be kept in 
sync with each other.
This is rather complicated, so it will take a while to get right.
At this point it might take another month until function calls 
are working.

--
On a different note.
Thanks for your continued moral support.
It is nice to know that my work is eagerly awaited by people.
As always I am sorry for the long time it takes.
I hope that my reports give a little insight into what issues I 
am facing, and why it is taking so long.
Dec 20 2016
next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 20 December 2016 at 16:53:21 UTC, Stefan Koch wrote:
 At this point it might take another month until function calls 
 are working.
I admire your endurance.
Dec 20 2016
prev sibling next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 20 December 2016 at 16:53:21 UTC, Stefan Koch wrote:
 On a different note.
 Thanks for your continued moral support.
 It is nice to know that my work is eagerly awaited by people.
 As always I am sorry for the long time it takes.
 I hope that my reports give a little insight into what issues I 
 am facing, and why it is taking so long.
I find it all very interesting. Keep up the good work.
Dec 20 2016
prev sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 20 December 2016 at 16:53:21 UTC, Stefan Koch wrote:
 I hope that my reports give a little insight into what issues I 
 am facing, and why it is taking so long.
There's a left parenthesis too much on line 3320 in ctfe_bc.d ;)
Dec 20 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 20 December 2016 at 20:30:43 UTC, Nordlöw wrote:
 On Tuesday, 20 December 2016 at 16:53:21 UTC, Stefan Koch wrote:
 I hope that my reports give a little insight into what issues 
 I am facing, and why it is taking so long.
There's a left parenthesis too much on line 3320 in ctfe_bc.d ;)
You're right. I was sleepy when committing that, so I forgot to check if it compiles. It had flags to allow stack pointers for integers to make slice handling cheaper in the interpreter, but it turns out that this decision hurt performance overall, and now I am removing it. This unmatched paren was part of the i32Ptr check :)
Dec 20 2016
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Hey Guys,

Pointers support is coming!

The following code compiles now:

uint* fn(uint v)
{
     return new uint(v + 6);
}


static assert(*fn(5) == 11);
Dec 22 2016
next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 22 December 2016 at 12:47:12 UTC, Stefan Koch wrote:
 Hey Guys,

 Pointers support is coming!
Awesome! Are there any D language features working in newCTFE that doesn't work on current upstream master?
Dec 22 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 22 December 2016 at 15:21:08 UTC, Nordlöw wrote:
 On Thursday, 22 December 2016 at 12:47:12 UTC, Stefan Koch 
 wrote:
 Hey Guys,

 Pointers support is coming!
Awesome! Are there any D language features working in newCTFE that doesn't work on current upstream master?
None at the moment, the current work is aimed to provide a fast replacement for the engine. It does not aim to add any functionality to CTFE. At least not yet. I am planning to support closures with multiple contexts at CTFE, but that can only be worked on after everything else is working.
Dec 22 2016
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
I just fixed the "" case for StringLiterals.
However the fix is quite hackish and needs to be revisited soon.
Furthermore I have made a few more preparations for function call 
support.

The reason why I am a bit reculant to enable function calls is 
the vast increase of bug-attack-surface.

Also there has been a slight speed-increase in the interpreter 
due to the removal of stack-pointers which were a left-over from 
the time where the heap was unimplemented.

I may restore the llvm-backend temporarily to verify the 
function-call handling (one I am ready to enable it).
Dec 22 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 22 December 2016 at 20:34:50 UTC, Stefan Koch wrote:
 I just fixed the "" case for StringLiterals.
This exposed bugs in the code-generator because now phobos-unittests with "" are run. (Which would previously be counted as uncompilable) Apparently we run into a situation where the interpreter tries to fetch instructions that are out-of-range. This can only be caused by miscompiled code. I am investigating.
Dec 23 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 23 December 2016 at 10:25:57 UTC, Stefan Koch wrote:
 On Thursday, 22 December 2016 at 20:34:50 UTC, Stefan Koch 
 wrote:
 I just fixed the "" case for StringLiterals.
This exposed bugs in the code-generator because now phobos-unittests with "" are run. (Which would previously be counted as uncompilable) Apparently we run into a situation where the interpreter tries to fetch instructions that are out-of-range. This can only be caused by miscompiled code. I am investigating.
Seems like that one was triggerd by random garbage being written into empty strings :) Empty string literals are now fixed for good.
Dec 23 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 23 December 2016 at 11:31:14 UTC, Stefan Koch wrote:
 On Friday, 23 December 2016 at 10:25:57 UTC, Stefan Koch wrote:
 On Thursday, 22 December 2016 at 20:34:50 UTC, Stefan Koch 
 wrote:
 I just fixed the "" case for StringLiterals.
This exposed bugs in the code-generator because now phobos-unittests with "" are run. (Which would previously be counted as uncompilable) Apparently we run into a situation where the interpreter tries to fetch instructions that are out-of-range. This can only be caused by miscompiled code. I am investigating.
Seems like that one was triggerd by random garbage being written into empty strings :) Empty string literals are now fixed for good.
I just made global enums and static immutables visible to newCTFE. Before they would trigger a bailout since they are not defined in the functions stack-frame. meaning this compiles now: uint[] arrayLiteralReturn() { return [1,2,3]; } static immutable alr = arrayLiteralReturn(); static assert(alr == [1,2,3]);
Dec 23 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
 static immutable alr = arrayLiteralReturn();
 static assert(alr == [1,2,3]);
There were a few problems with that. Namely some arrays-literals would not have a length. This is because of literals coercing to slices. This bug is now fixed, at the expense of registering more types then is strictly necessary. Also it creates null holes in the TypeArrays. This may might be painful in the future. But for now, global Array-literals work.
Dec 25 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 25 December 2016 at 11:27:07 UTC, Stefan Koch wrote:
 static immutable alr = arrayLiteralReturn();
 static assert(alr == [1,2,3]);
There were a few problems with that. Namely some arrays-literals would not have a length. This is because of literals coercing to slices. This bug is now fixed, at the expense of registering more types then is strictly necessary. Also it creates null holes in the TypeArrays. This may might be painful in the future. But for now, global Array-literals work.
There are sill problems when you try to index those arrays for function calls. This can be fixed by proving a different code-path. If those arrays are indexed as function-parameters at the top level.
Dec 25 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 25 December 2016 at 12:21:42 UTC, Stefan Koch wrote:
 On Sunday, 25 December 2016 at 11:27:07 UTC, Stefan Koch wrote:
 static immutable alr = arrayLiteralReturn();
 static assert(alr == [1,2,3]);
There were a few problems with that. Namely some arrays-literals would not have a length. This is because of literals coercing to slices. This bug is now fixed, at the expense of registering more types then is strictly necessary. Also it creates null holes in the TypeArrays. This may might be painful in the future. But for now, global Array-literals work.
There are sill problems when you try to index those arrays for function calls. This can be fixed by proving a different code-path. If those arrays are indexed as function-parameters at the top level.
I reverted my ArrayLiteral fixes for now, they broke the druntime build.
Dec 25 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 25 December 2016 at 15:26:36 UTC, Stefan Koch wrote:

 I reverted my ArrayLiteral fixes for now,
 they broke the druntime build.
So far by attempts to fix this situation, proved fruitless. However this getting global static variables to work correctly is a high-priority goal, as it is both important for the typical use-cases of ctfe and it affects the overall design of the engine. I'll be taking a short rest during the next days, hopefully I will know how to move forward after that. Also I need to prepare a talk-proposal for next dconf.
Dec 27 2016
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Some progress on && has been made.
It can now be used within an if.
It is only enabled if debug = andnd is given.
Dec 24 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 24 December 2016 at 16:10:04 UTC, Stefan Koch wrote:
 Some progress on && has been made.
 It can now be used within an if.
 It is only enabled if debug = andnd is given.
This introduced a bug which is now fixed. Furthermore returning of 64bit integral values has now been enabled. I just thought about a way to enable slices with less "CTFE-runtime" support. This is a good thing since the CTFE runtime currently is made up of a few pretty long macros. That bloat the bytecode beyond recognition. As soon as call are supported they can be turned into function-calls. However function calls are pretty expensive. Therefore it would be good if lightweight slice handling would work.
Dec 24 2016
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Hi Guys,

I just figured out why array constants did not work as function 
arguments.
It's because the array-constant undergoes a cast when used as 
slice, while an array literal can be taken as is.

The currently newCTFE does not really provide the capabilities to 
handle casts.
This is another issue that has to be addressed before 
function-calls come into play.

Within the engine dynamic and static arrays are represented in 
the same structure.
Therefore we don't need to take action on such a cast as long as 
the element types are the same.

While simply ignoring the cast-expression is correct in the case, 
I need to systemize the handling of casts better.
Dec 29 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 29 December 2016 at 18:21:30 UTC, Stefan Koch wrote:
 Hi Guys,

 I just figured out why array constants did not work as function 
 arguments.
 It's because the array-constant undergoes a cast when used as 
 slice, while an array literal can be taken as is.

 The currently newCTFE does not really provide the capabilities 
 to handle casts.
 This is another issue that has to be addressed before 
 function-calls come into play.

 Within the engine dynamic and static arrays are represented in 
 the same structure.
 Therefore we don't need to take action on such a cast as long 
 as the element types are the same.

 While simply ignoring the cast-expression is correct in the 
 case, I need to systemize the handling of casts better.
This is now partially fixed. Also all the latest work was merged in the feature branch. The current timeline is still the same, I suspect that newCTFE will be usable in February.
Dec 30 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
global immutable integer Arrays are now completely functional!

This makes the following code work:
uint echo (uint val)
{
     return val;
}

const(uint) fastLog10(const uint val) pure nothrow  nogc {
     return      (val < 10) ? 0 :
                 (val < 100) ? 1 :
                 (val < 1000) ? 2 :
                 (val < 10000) ? 3 :
                 (val < 100000) ? 4 :
                 (val < 1000000) ? 5 :
                 (val < 10000000) ? 6 :
                 (val < 100000000) ? 7 :
                 (val < 1000000000) ? 8 : 9;
}

/* unique*/ static immutable uint[10] fastPow10tbl = [
         1,
         10,
         100,
         1000,
         10000,
         100000,
         1000000,
         10000000,
         100000000,
         1000000000,
] ;

static assert(fastLog10(fastPow10tbl[2]) == 2);
static assert(fastLog10(fastPow10tbl[3]) == 3);
static assert(fastLog10(1000) == 3);

static assert(echo(fastPow10tbl[1]) == 10);
Dec 31 2016
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Saturday, 31 December 2016 at 16:32:42 UTC, Stefan Koch wrote:
 global immutable integer Arrays are now completely functional!
Great work!
 /* unique*/ static immutable uint[10] fastPow10tbl = [
BTW: What role does unique play here?
Dec 31 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 31 December 2016 at 17:12:22 UTC, Nordlöw wrote:
 On Saturday, 31 December 2016 at 16:32:42 UTC, Stefan Koch 
 wrote:
 global immutable integer Arrays are now completely functional!
Great work!
 /* unique*/ static immutable uint[10] fastPow10tbl = [
BTW: What role does unique play here?
No role at all. It's a comment. I have these comments in my tests, to remind myself of certain properties. That, when properly detected, can lead to better generated code.
Dec 31 2016
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
So guys.

basic function-calls are supported.

Hell YEAH!
Jan 02
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 2 January 2017 at 18:40:44 UTC, Stefan Koch wrote:
 So guys.

 basic function-calls are supported.

 Hell YEAH!
Meaning this will compile : uint caller(uint n) { return callee(n, 2); } uint callee(uint a, uint b) { return a*b; } static assert(caller(3) == 6); static assert(caller(24) == 48); uint rfn(uint n) { if (n) return n + rfn(n-1); return 1; } pragma(msg, rfn(2000));
Jan 02
parent reply Meta <jared771 gmail.com> writes:
On Monday, 2 January 2017 at 19:01:43 UTC, Stefan Koch wrote:
 On Monday, 2 January 2017 at 18:40:44 UTC, Stefan Koch wrote:
 So guys.

 basic function-calls are supported.

 Hell YEAH!
Meaning this will compile : uint caller(uint n) { return callee(n, 2); } uint callee(uint a, uint b) { return a*b; } static assert(caller(3) == 6); static assert(caller(24) == 48); uint rfn(uint n) { if (n) return n + rfn(n-1); return 1; } pragma(msg, rfn(2000));
There's something I've been wondering about. NewCTFE is basically just another backend, right? Does this mean that you'll have to implement various optimizations such as tail-call elision independently from what DMD with the default backend currently does?
Jan 02
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 2 January 2017 at 21:52:05 UTC, Meta wrote:
 There's something I've been wondering about. NewCTFE is 
 basically just another backend, right? Does this mean that 
 you'll have to implement various optimizations such as 
 tail-call elision independently from what DMD with the default 
 backend currently does?
Yes, I could borrow analysis passes from dmd however to check if the optimization is applicable. However I do not intend to do so. CTFE is mostly used for small one-shot things. There is no point in eliding calls. Executing the code as soon as possible is often faster then doing the optimization.
Jan 02
prev sibling next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 2 January 2017 at 18:40:44 UTC, Stefan Koch wrote:
 Hell YEAH!
Nice work.
Jan 02
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 2 January 2017 at 18:40:44 UTC, Stefan Koch wrote:
 So guys.

 basic function-calls are supported.

 Hell YEAH!
Disabled for now. Since it pushes some buffers/caches to their limit. A more flexible solution is needed. However the good news is that the ABI seems to work better than expected. Now it's time to deal with this-pointers and method calls. After remaining issues with nested switches have to be fixed.
Jan 04
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 5 January 2017 at 06:04:00 UTC, Stefan Koch wrote:
 On Monday, 2 January 2017 at 18:40:44 UTC, Stefan Koch wrote:
 So guys.

 basic function-calls are supported.

 Hell YEAH!
Disabled for now. Since it pushes some buffers/caches to their limit. A more flexible solution is needed.
This is now fixed by simply extending the limits. Now more failures start to show and more manual bailouts are required again. Although, I expected this to happen it bums me. :(
Jan 08
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 5 January 2017 at 06:04:00 UTC, Stefan Koch wrote:
 On Monday, 2 January 2017 at 18:40:44 UTC, Stefan Koch wrote:
 So guys.

 basic function-calls are supported.

 Hell YEAH!
Disabled for now. Since it pushes some buffers/caches to their limit. A more flexible solution is needed. However the good news is that the ABI seems to work better than expected. Now it's time to deal with this-pointers and method calls. After remaining issues with nested switches have to be fixed.
Basic Function call support reenabled. Phobos unittests pass.
Jan 16
next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 16 January 2017 at 16:31:25 UTC, Stefan Koch wrote:
 Basic Function call support reenabled.
 Phobos unittests pass.
Nice!
Jan 16
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 16 January 2017 at 16:31:25 UTC, Stefan Koch wrote:
 On Thursday, 5 January 2017 at 06:04:00 UTC, Stefan Koch wrote:
 On Monday, 2 January 2017 at 18:40:44 UTC, Stefan Koch wrote:
 So guys.

 basic function-calls are supported.

 Hell YEAH!
Disabled for now. Since it pushes some buffers/caches to their limit. A more flexible solution is needed. However the good news is that the ABI seems to work better than expected. Now it's time to deal with this-pointers and method calls. After remaining issues with nested switches have to be fixed.
Basic Function call support reenabled. Phobos unittests pass.
Just found a bug with recursive functions, which is now fixed. So time for some perf: -- t.d uint rfn(uint n) { if (n > 2) return n + rfn(n-1) + rfn(n-2); return 1; } pragma(msg, rfn(30)); -- uplink uplink-pc:~/d/dmd$ time src/dmd t.d -c 4674396u real 0m0.396s user 0m0.392s sys 0m0.000s uplink uplink-pc:~/d/dmd$ time dmd t.d -c 4674396u real 0m1.993s user 0m1.872s sys 0m0.120s
Jan 16
next sibling parent Martin Nowak <code dawg.eu> writes:
On Tuesday, 17 January 2017 at 06:36:58 UTC, Stefan Koch wrote:
 real	0m1.993s
 user	0m1.872s
 sys	0m0.120s
Just a tip, if you use (maybe install) /bin/time, you also get memory usage. 0.00user 0.00system 0:00.00elapsed 85%CPU (0avgtext+0avgdata 6372maxresident)k 0inputs+0outputs (0major+361minor)pagefaults 0swaps
Jan 17
prev sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 17 January 2017 at 06:36:58 UTC, Stefan Koch wrote:
 Just found a bug with recursive functions, which is now fixed.
 So time for some perf:
Impressive speedup! What remains? Are you adding more tests to the Phobos unittests?
Jan 17
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 17 January 2017 at 20:06:46 UTC, Nordlöw wrote:
 On Tuesday, 17 January 2017 at 06:36:58 UTC, Stefan Koch wrote:
 Just found a bug with recursive functions, which is now fixed.
 So time for some perf:
Impressive speedup! What remains? Are you adding more tests to the Phobos unittests?
There are still features to add : - floating point. - UTF-de/re-encodung. - slicing - concat - pointer/ref - member-function calls But after this is done all the 1.0 goals are reached.
Jan 17
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 17 January 2017 at 21:47:51 UTC, Stefan Koch wrote:
 On Tuesday, 17 January 2017 at 20:06:46 UTC, Nordlöw wrote:
 On Tuesday, 17 January 2017 at 06:36:58 UTC, Stefan Koch wrote:
 Just found a bug with recursive functions, which is now fixed.
 So time for some perf:
Impressive speedup! What remains? Are you adding more tests to the Phobos unittests?
There are still features to add : - floating point. - UTF-de/re-encodung. - slicing - concat - pointer/ref - member-function calls But after this is done all the 1.0 goals are reached.
Forget something, && and || also needs to be implemented. As well as various interactions of pointers/arrays/slices and structures.
Jan 17
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 17 January 2017 at 21:50:42 UTC, Stefan Koch wrote:
 There are still features to add :

 - floating point.
 - UTF-de/re-encodung.
 - slicing
 - concat
 - pointer/ref
 - member-function calls

 But after this is done all the 1.0 goals are reached.
Forget something, && and || also needs to be implemented. As well as various interactions of pointers/arrays/slices and structures.
So I reckon the Phobos unittests are quite incomplete, then?
Jan 17
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 17 January 2017 at 22:06:47 UTC, Nordlöw wrote:
 So I reckon the Phobos unittests are quite incomplete, then?
Not really, I still use a fallback to the old interpreter for the unimplemented features.
Jan 17
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 17 January 2017 at 22:11:21 UTC, Stefan Koch wrote:
 Not really, I still use a fallback to the old interpreter for 
 the unimplemented features.
Does this mean that the newCTFE branch is usable now? And that I get performance improvements for the expression nodes you have implemented in newCTFE but not for the others?
Jan 17
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 17 January 2017 at 22:19:39 UTC, Nordlöw wrote:
 On Tuesday, 17 January 2017 at 22:11:21 UTC, Stefan Koch wrote:
 Not really, I still use a fallback to the old interpreter for 
 the unimplemented features.
Does this mean that the newCTFE branch is usable now? And that I get performance improvements for the expression nodes you have implemented in newCTFE but not for the others?
Exactly. Modulo bugs of course. If you have code gives you a different output with my engine then with the old one, please open an issue in my github repo.
Jan 17
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
Hi Guys,

I just enabled the handling for with(Type).
makes the following code work :

import core.time;
import core.sys.linux.time;

  auto _posixClock(ClockType clockType)
     {
             with(ClockType) final switch(clockType)
             {
             case bootTime: return CLOCK_BOOTTIME;
             case coarse: return CLOCK_MONOTONIC_COARSE;
             case normal: return CLOCK_MONOTONIC;
             case precise: return CLOCK_MONOTONIC;
             case processCPUTime: return CLOCK_PROCESS_CPUTIME_ID;
             case raw: return CLOCK_MONOTONIC_RAW;
             case threadCPUTime: return CLOCK_THREAD_CPUTIME_ID;
             case second: assert(0);
             }
}

static assert(_posixClock(ClockType.bootTime) == CLOCK_BOOTTIME);
Jan 03
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
I have fixed a few bugs in the handling of casts.
And fixed an oversight which would cause Xors to bailout.
However Phobos-unittests are still regressed.

I am now working on Method-Calls or member-function-calls.
Jan 09
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Druntime and phobos broke, the reason being that,
I did test against an outdated version of druntime/phobos.

I am working on getting them to compile again.
Adding features is stalled until then.
Jan 20
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 20 January 2017 at 19:08:28 UTC, Stefan Koch wrote:
 Druntime and phobos broke, the reason being that,
 I did test against an outdated version of druntime/phobos.

 I am working on getting them to compile again.
 Adding features is stalled until then.
Phobos and druntime unittests pass again. this was _NOT_ a proper fix. only a really crude bailout when we detect that we jump to code that does not exist.
Jan 20
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 21 January 2017 at 05:59:32 UTC, Stefan Koch wrote:
 On Friday, 20 January 2017 at 19:08:28 UTC, Stefan Koch wrote:
 Druntime and phobos broke, the reason being that,
 I did test against an outdated version of druntime/phobos.

 I am working on getting them to compile again.
 Adding features is stalled until then.
Phobos and druntime unittests pass again. this was _NOT_ a proper fix. only a really crude bailout when we detect that we jump to code that does not exist.
The cause seems to be fixed now.
Jan 21
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
I just fixed a bug related to the unsupported floating point.
~master phobos compiles again and runs the unit-tests.

I will probably have to spent the rest of the month with fixing 
bugs that function-call support has uncovered.

Since now newCTFE sees much more code before bailing out.

I have also determined the cause of most of the miscompiled code.
The cause is the incorrect treatment of utf8.
(currently the evaluator really just does (dchar c= (_c & 7f))
which leads to corrupted strings in mixins;
and that has pretty crazy effects.

It was very hard to find this.
Jan 22
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 23 January 2017 at 07:57:35 UTC, Stefan Koch wrote:
 It was very hard to find this.
Good job.
Jan 23
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 23 January 2017 at 09:06:49 UTC, Nordlöw wrote:
 Good job.
There are more good news uint fn() { uint x = 7; modx(&x); return x; } void modx(uint* x) { *x = 12; } static assert(fn() == 12); This code does now compile with newCTFE :)
Jan 23
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
interpret3.d passes!!!

The only remaining issues that cause miscompiled code are 
UTF(8/16/32) encoding and decoding issues.
Jan 23
next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 23 January 2017 at 17:42:00 UTC, Stefan Koch wrote:
 interpret3.d passes!!!

 The only remaining issues that cause miscompiled code are 
 UTF(8/16/32) encoding and decoding issues.
Wow! Getting close know!
Jan 23
prev sibling next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 1/23/17 6:42 PM, Stefan Koch wrote:
 interpret3.d passes!!!
Now that's something!
 The only remaining issues that cause miscompiled code are UTF(8/16/32)
 encoding and decoding issues.
--- Dmitry Olshansky
Jan 23
prev sibling next sibling parent Daniel =?UTF-8?B?S296w6Fr?= via Digitalmars-d writes:
V Mon, 23 Jan 2017 17:42:00 +0000
Stefan Koch via Digitalmars-d <digitalmars-d puremagic.com> napsĂĄno:

 interpret3.d passes!!!
 
 The only remaining issues that cause miscompiled code are 
 UTF(8/16/32) encoding and decoding issues.
WOW
Jan 23
prev sibling parent reply David Nadlinger <code klickverbot.at> writes:
On Monday, 23 January 2017 at 17:42:00 UTC, Stefan Koch wrote:
 interpret3.d passes!!!

 The only remaining issues that cause miscompiled code are 
 UTF(8/16/32) encoding and decoding issues.
Is that without bailing out? — David
Jan 23
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 23 January 2017 at 22:09:27 UTC, David Nadlinger wrote:
 On Monday, 23 January 2017 at 17:42:00 UTC, Stefan Koch wrote:
 interpret3.d passes!!!

 The only remaining issues that cause miscompiled code are 
 UTF(8/16/32) encoding and decoding issues.
Is that without bailing out? — David
No I still bail a lot :). And I will be for some time, until I have written a memory-manager and various other runtime-libarary equivalents which are required for CTFE. As well as my own soft-floating point library to ensure proper output during cross-compilation. However this is an important milestone because it can now magically speed up a subset of ctfe without breaking things... modulo bugs not covered by the unittests.
Jan 23
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
The blacklisted functions are now down to only two.

Those are the bitswap function in druntime.
Because the interpreter does handle all values as 64bit 
internally it tends to miscompile code that uses xor on 32bit 
values.

And the second one is the "to" template from std.conv;
Because of the aforementioned UTF issues.

NOTE: that the blacklist is by nature transitive meaning a 
function that calls a blacklisted function will also not be 
interpreted by newCTFE
Jan 23
next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 24 January 2017 at 06:35:56 UTC, Stefan Koch wrote:
 And the second one is the "to" template from std.conv;
 Because of the aforementioned UTF issues.

 NOTE: that the blacklist is by nature transitive meaning a 
 function that calls a blacklisted function will also not be 
 interpreted by newCTFE
`to` is so common that I suspect you'll see a wave of bugs when you fix the UTF issues, yes? Either way, thanks for spending so much time on this, I'm excited about these improvements both from a professional and personal standpoint.
Jan 24
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 24 January 2017 at 09:27:52 UTC, John Colvin wrote:
 On Tuesday, 24 January 2017 at 06:35:56 UTC, Stefan Koch wrote:
 And the second one is the "to" template from std.conv;
 Because of the aforementioned UTF issues.

 NOTE: that the blacklist is by nature transitive meaning a 
 function that calls a blacklisted function will also not be 
 interpreted by newCTFE
`to` is so common that I suspect you'll see a wave of bugs when you fix the UTF issues, yes? Either way, thanks for spending so much time on this, I'm excited about these improvements both from a professional and personal standpoint.
NEW CTFE IS GREEN ON 64 BIT! GREEN!
Jan 24
next sibling parent reply Claude <no no.no> writes:
On Tuesday, 24 January 2017 at 09:52:13 UTC, Stefan Koch wrote:
 NEW CTFE IS GREEN ON 64 BIT!
 GREEN!
I felt first a bit surprised about that "green" thing, but somehow if the new-ctfe engine consumes less CPU, it means it consumes less power, and therefore less carbone dioxyde is released into the atmosphere. So indeed, we can assert that new-ctfe is eco-friendly! Good job! :) I really enjoy reading this thread everyday. It's kind of like a Dungeon & Dragon novel where the hero slashes its way through monsters, and bugs and features, up to the Holy Grail: the New-CTFE Engine implementation! This is awesome work! Thanks.
Jan 24
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 24 January 2017 at 10:34:58 UTC, Claude wrote:
 On Tuesday, 24 January 2017 at 09:52:13 UTC, Stefan Koch wrote:
 NEW CTFE IS GREEN ON 64 BIT!
 GREEN!
I felt first a bit surprised about that "green" thing, but somehow if the new-ctfe engine consumes less CPU, it means it consumes less power, and therefore less carbone dioxyde is released into the atmosphere. So indeed, we can assert that new-ctfe is eco-friendly!
Green means it passes all tests on auto-tester.
Jan 24
next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 24 January 2017 at 11:01:46 UTC, Stefan Koch wrote:
 Green means it passes all tests on auto-tester.
Wonderful! Time to try out Pegged with newCTFE, then!
Jan 24
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 24 January 2017 at 12:43:28 UTC, Nordlöw wrote:
 On Tuesday, 24 January 2017 at 11:01:46 UTC, Stefan Koch wrote:
 Green means it passes all tests on auto-tester.
Wonderful! Time to try out Pegged with newCTFE, then!
There will be NO permanence improvement for pegged. For the 7th time. It is bogged down by templates, ctfe is not the problem there.
Jan 24
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 24 January 2017 at 14:16:10 UTC, Stefan Koch wrote:
 There will be NO permanence improvement for pegged.
 For the 7th time.

 It is bogged down by templates, ctfe is not the problem there.
Sorry for that. Got it. What's the essence of this suboptimal performance when using templates? Non-O(1) lookup times? Mangling?
Jan 24
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 24 January 2017 at 15:16:29 UTC, Nordlöw wrote:
 On Tuesday, 24 January 2017 at 14:16:10 UTC, Stefan Koch wrote:
 There will be NO permanence improvement for pegged.
 For the 7th time.

 It is bogged down by templates, ctfe is not the problem there.
Sorry for that. Got it. What's the essence of this suboptimal performance when using templates? Non-O(1) lookup times? Mangling?
The lookup time is one issue but not the most important, The instanciation of a template is a non-trivial operation within the compiler. Type-deduction ast rewriting expression expansion all of that takes time and a huge chunk at that. Also binary code needs to be generated and that code is generated for every instance regardless of how similar they are.
Jan 24
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 24 January 2017 at 15:33:37 UTC, Stefan Koch wrote:
 The lookup time is one issue but not the most important,
 The instanciation of a template is a non-trivial operation 
 within the compiler.
 Type-deduction ast rewriting expression expansion
 all of that takes time and a huge chunk at that.
Interesting. Is that the major cause of high memory usage in DMD?
 Also binary code needs to be generated and that code is 
 generated for every instance regardless of how similar they are.
Thanks.
Jan 24
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 24 January 2017 at 15:48:19 UTC, Nordlöw wrote:
 On Tuesday, 24 January 2017 at 15:33:37 UTC, Stefan Koch wrote:
 The lookup time is one issue but not the most important,
 The instanciation of a template is a non-trivial operation 
 within the compiler.
 Type-deduction ast rewriting expression expansion
 all of that takes time and a huge chunk at that.
Interesting. Is that the major cause of high memory usage in DMD?
 Also binary code needs to be generated and that code is 
 generated for every instance regardless of how similar they 
 are.
Thanks.
I would have to investigate the specific case. It's also because the recursive nature of templates that you end up with +100_000 instances It those numbers even small allocations have a huge impact.
Jan 24
prev sibling parent Claude <no no.no> writes:
On Tuesday, 24 January 2017 at 11:01:46 UTC, Stefan Koch wrote:
 Green means it passes all tests on auto-tester.
Yes, I understood. I was playing with the idea of eco-friendliness as well... :)
Jan 24
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 24 January 2017 at 09:52:13 UTC, Stefan Koch wrote:
 NEW CTFE IS GREEN ON 64 BIT!
 GREEN!
I just fixed a regression found by compiling vibe.d. We are now even greener!
Jan 24
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 24 January 2017 at 06:35:56 UTC, Stefan Koch wrote:
 The blacklisted functions are now down to only two.

 Those are the bitswap function in druntime.
 Because the interpreter does handle all values as 64bit 
 internally it tends to miscompile code that uses xor on 32bit 
 values.

 And the second one is the "to" template from std.conv;
 Because of the aforementioned UTF issues.

 NOTE: that the blacklist is by nature transitive meaning a 
 function that calls a blacklisted function will also not be 
 interpreted by newCTFE
The blacklisting of bitswap was removed, because we now bailout on bit-level operations on 32bit values. However I did have to add a unittest-specific bailout for fail_compilation/fail14304.d which requires the ctfe interpeter to detect a semantically invalid situation that should have been detected before we ever begin ctfe.
Jan 25
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
newCTFE is green now on all platforms!
Jan 25
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch wrote:
 newCTFE is green now on all platforms!
<3
Jan 25
prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch wrote:
 newCTFE is green now on all platforms!
Congrats. How much work remains?
Jan 25
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 25 January 2017 at 15:26:39 UTC, jmh530 wrote:
 On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch 
 wrote:
 newCTFE is green now on all platforms!
Congrats. How much work remains?
Quite a lot. - Slicing - Appending - || - String support (implying utf conversions) - Floating-Point - classes - exceptions - function pointers - closures and proper handling of more complex types such as arrays of structures
Jan 25
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 1/25/17 4:44 PM, Stefan Koch wrote:
 On Wednesday, 25 January 2017 at 15:26:39 UTC, jmh530 wrote:
 On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch wrote:
 newCTFE is green now on all platforms!
Congrats. How much work remains?
Quite a lot. - Slicing - Appending - || - String support (implying utf conversions) - Floating-Point - classes - exceptions - function pointers - closures and proper handling of more complex types such as arrays of structures
Will note that std.regex will requires at least basic support for classes (construct and return from a function) and slicing. --- Dmitry Olshansky
Jan 29
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 29 January 2017 at 13:21:47 UTC, Dmitry Olshansky 
wrote:
 On 1/25/17 4:44 PM, Stefan Koch wrote:
 On Wednesday, 25 January 2017 at 15:26:39 UTC, jmh530 wrote:
 On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch 
 wrote:
 newCTFE is green now on all platforms!
Congrats. How much work remains?
Quite a lot. - Slicing - Appending - || - String support (implying utf conversions) - Floating-Point - classes - exceptions - function pointers - closures and proper handling of more complex types such as arrays of structures
Will note that std.regex will requires at least basic support for classes (construct and return from a function) and slicing. --- Dmitry Olshansky
I can support non-polymorphic classes in near future (mid of march ?). Slicing needs to be considered together with concatenation esentially it will require a memory-manager. I still need to figure out if druntime could be reused here.
Jan 29
prev sibling next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch wrote:
 newCTFE is green now on all platforms!
Great!
Jan 25
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch wrote:
 newCTFE is green now on all platforms!
I just found an interesting bug just now. The following code would cause newCTFE to segfault. char* initPtr() { return cast(char*) size_t.max; } static assert(cast(size_t)initPtr() == size_t.max); Because it would try to return the char-value at heap-address size_t.max; newCTFE allocates a 16MB heap by default and therefore the address was hopelessly out-of-bounds. The reason this was not detected before is because: `cast(char*) size_t.max;` does not actually produce a cast expression. But rather an IntegerLiteral with the type char*. After dealing with dmd for 7 months, I could figure out this bug in a day. If I had run into this earlier I would have been stuck for weeks :)
Jan 25
parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Thursday, 26 January 2017 at 07:27:13 UTC, Stefan Koch wrote:
 On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch 
 wrote:
 newCTFE is green now on all platforms!
I just found an interesting bug just now. The following code would cause newCTFE to segfault. char* initPtr() { return cast(char*) size_t.max; } static assert(cast(size_t)initPtr() == size_t.max); Because it would try to return the char-value at heap-address size_t.max; newCTFE allocates a 16MB heap by default and therefore the address was hopelessly out-of-bounds. The reason this was not detected before is because: `cast(char*) size_t.max;` does not actually produce a cast expression. But rather an IntegerLiteral with the type char*. After dealing with dmd for 7 months, I could figure out this bug in a day. If I had run into this earlier I would have been stuck for weeks :)
I do not understand why it should dereference the pointer? Do you mean that the bug was that newCTFE dereferenced but it shouldn't or that oldCTFE didn't dereference but it should have?
Jan 26
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 26 January 2017 at 11:59:48 UTC, Patrick Schluter 
wrote:
 On Thursday, 26 January 2017 at 07:27:13 UTC, Stefan Koch wrote:
 On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch 
 wrote:
 char* initPtr()
 {
     return cast(char*) size_t.max;
 }

 static assert(cast(size_t)initPtr() == size_t.max);

 Because it would try to return the char-value at heap-address 
 size_t.max;
 newCTFE allocates a 16MB heap by default and therefore the 
 address was hopelessly out-of-bounds.
I do not understand why it should dereference the pointer? Do you mean that the bug was that newCTFE dereferenced but it shouldn't or that oldCTFE didn't dereference but it should have?
newCTFE should not dereference that pointer. However since all other pointers need to be dereferenced, this bug came totally out of the blue.
Jan 26
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
Member-Function calls work now!

this code will compile and properly execute with newCTFE:

struct iota_range
{
   int current;
   int end;
   int step;

   uint front()
   {
      return current;
   }

   void popFront()
   {
     current += step;
     // we need this return otherwise we generate invalid code :(
     return ;
   }

   bool empty()
   {
     return current > end;
   }

   this(uint end, uint begin = 0, uint step = 1) pure
   {
     assert(step != 0, "cannot have a step of 0");
     this.step = step;
     this.current = begin;
     this.end = end;
   }
}

auto Iota(int end)
{
   return iota_range(end);
}

uint testThisCall(uint end)
{
   uint result;

   foreach(n;Iota(end))
   {
     result += n;
   }

   return result;
}


static assert(testThisCall(12) == 78);


Neat, huh ?
Jan 26
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
I just fixed the void initialization fail compilation tests.
By detecting the possibility of returning a void initialized 
field and bailing out.
dmd testsuite does now compile and pass(!) again.

I also fixed an obscure bug related to the recently introduced 
conecpt of heap references.
Jan 28
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 28 January 2017 at 19:08:42 UTC, Stefan Koch wrote:
 I just fixed the void initialization fail compilation tests.
 By detecting the possibility of returning a void initialized 
 field and bailing out.
 dmd testsuite does now compile and pass(!) again.

 I also fixed an obscure bug related to the recently introduced 
 conecpt of heap references.
Druntime and phobos compile again and pass their respective unittests. I also fixed a little performace bottleneck in the function-call code. Although it is still far from optimal and can still be speed-up significantly. However I would like to get more feature completeness before I am fixing anymore perf issues, except when they are jumping in my face :) Also my ctfe engine still requires utf8 support, for string-foreach. Currently there are methods for that in druntime, I hope to simply call them at ctfe, rather then re-implement them. If someone volunteers I can provide help to make them ctfeable, however at the moment my free mental bandwidth is too low to deal with utf details. This approach will require the druntime-source containing the utf8 methods to be present and visible at every compiler run that uses string foreach. If you consider this an issue please state why.
Jan 28
parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 29 January 2017 at 02:17:12 UTC, Stefan Koch wrote:
 Also my ctfe engine still requires utf8 support, for 
 string-foreach.
 Currently there are methods for that in druntime, I hope to 
 simply call them at ctfe, rather then re-implement them. If 
 someone volunteers I can provide help to make them ctfeable, 
 however at the moment my free mental bandwidth is too low to 
 deal with utf details.

 This approach will require the druntime-source containing the 
 utf8 methods to be present and visible at every compiler run 
 that uses string foreach.

 If you consider this an issue please state why.
1: I think it's good to avoid this if possible, though I'm not fiercely opposed 2: I wrote this for mach: https://github.com/pineapplemachine/mach.d/tree/master/mach/text/utf If you want to borrow code or if I can provide an implementation meeting different requirements then I am at your disposal. 3: Can you clarify what string-foreach refers to? Surely not `foreach(ch; some_string){}`, which enumerates code units and not code points?
Jan 28
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 29 January 2017 at 02:46:16 UTC, pineapple wrote:
 On Sunday, 29 January 2017 at 02:17:12 UTC, Stefan Koch wrote:
 Also my ctfe engine still requires utf8 support, for 
 string-foreach.
 [ ... ]
 Can you clarify what string-foreach refers to? Surely not 
 `foreach(ch; some_string){}`, which enumerates code units and 
 not code points?
Yes exactly that. many times in phobos foreach(dchar ch; some_string) which requires me to encode the utf8 string temporarily into utf32 and then when it is appending to some other string I need to reencode it into utf8.
Jan 28
next sibling parent pineapple <meapineapple gmail.com> writes:
On Sunday, 29 January 2017 at 02:52:51 UTC, Stefan Koch wrote:
 Yes exactly that. many times in phobos foreach(dchar ch; 
 some_string)
 which requires me to encode the utf8 string temporarily into 
 utf32
 and then when it is appending to some other string I need to 
 reencode it into utf8.
Oooh, I never realized foreach acted specially like that. I can't decide whether I approve or disapprove. You'll also have to support UTF-16 for the case `foreach(wchar ch; some_string){}`. As before just let me know if I can help.
Jan 29
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 1/29/17 3:52 AM, Stefan Koch wrote:
 On Sunday, 29 January 2017 at 02:46:16 UTC, pineapple wrote:
 On Sunday, 29 January 2017 at 02:17:12 UTC, Stefan Koch wrote:
 Also my ctfe engine still requires utf8 support, for string-foreach.
 [ ... ]
 Can you clarify what string-foreach refers to? Surely not `foreach(ch;
 some_string){}`, which enumerates code units and not code points?
Yes exactly that. many times in phobos foreach(dchar ch; some_string) which requires me to encode the utf8 string temporarily into utf32 and then when it is appending to some other string I need to reencode it into utf8.
Why encode the whole string to utf32? You can lower it to use std.utf.decode in a loop. Essentially this: foreach(dchar ch; some_string){ ... } Becomes: import std.utf; size_t i=0; while(i<some_string.length){ dchar ch = decode(some_string, i); ... } --- Dmitry Olshansky
Jan 29
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 29 January 2017 at 13:18:11 UTC, Dmitry Olshansky 
wrote:
 Essentially this:

 foreach(dchar ch; some_string){ ... }

 Becomes:

 import std.utf;
 size_t i=0;
 while(i<some_string.length){
 	dchar ch = decode(some_string, i);
 	...
 }

 ---
 Dmitry Olshansky
Won't work right now, since ref is not supported. Also I really want to minimize dependency and phobos.
Jan 29
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 29 January 2017 at 02:46:16 UTC, pineapple wrote:

 https://github.com/pineapplemachine/mach.d/tree/master/mach/text/utf

 If you want to borrow code or if I can provide an 
 implementation meeting different requirements then I am at your 
 disposal.
That code is a great help, understanding the issue, thanks!
Jan 28
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
New bugs incoming:

uint fn(uint a)
{
     final switch(a)
     {
         case 1 : {

             while(a < 20)
//          bool whileCondition;
//            WhileBlockEvalCond :
//            whileCondition = (a < 20);
             {
//                WhileBlockBegin:
                 a++;
                 if (a == 17) break;
/*
                 WhlieBlockEnd:
                     if(!whileCondition)
                        goto WhileBlockAfter
                 // break seems to go here :-/
                 goto WhileBlockEvalCond;
                 // break should go here!
*/
             }
//             WhileBlockAfter:
                 return a;
         }
     }

     return 1;
}

pragma(msg, fn(1)); // should print 17
when executed with with newCTFE this will print 20
Jan 30
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 30 January 2017 at 15:15:55 UTC, Stefan Koch wrote:
 New bugs incoming:

 uint fn(uint a)
 {
     final switch(a)
     {
         case 1 : {

             while(a < 20)
 //          bool whileCondition;
 //            WhileBlockEvalCond :
 //            whileCondition = (a < 20);
             {
 //                WhileBlockBegin:
                 a++;
                 if (a == 17) break;
 /*
                 WhlieBlockEnd:
                     if(!whileCondition)
                        goto WhileBlockAfter
                 // break seems to go here :-/
                 goto WhileBlockEvalCond;
                 // break should go here!
 */
             }
 //             WhileBlockAfter:
                 return a;
         }
     }

     return 1;
 }

 pragma(msg, fn(1)); // should print 17
 when executed with with newCTFE this will print 20
This bug is fixed now!! it was a simple oversight were the mechanism I build to deal with this was simply not invoked. Luckily not a failure of the mechanism itself.
Jan 31
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
newCTFE is now green on all platforms on travis as well.

function pointer support are coming!
with it most of std.algorithm will be supported.
Jan 31
next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 31 January 2017 at 16:21:27 UTC, Stefan Koch wrote:
 newCTFE is now green on all platforms on travis as well.

 function pointer support are coming!
 with it most of std.algorithm will be supported.
Cool!
Jan 31
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 31 January 2017 at 16:21:27 UTC, Stefan Koch wrote:
 function pointer support is coming!
This is more difficult then I expected, since the argument handling system was build without considering that we could have functions as arguments, therefore when trying to evaluate the function-argument it will overwrite the data-structures that are supposed to hold the information for the function we currently processing the arguments for. As you can imagine this bug took me two days to find :) maybe this can be fixed by handling the arguments before handling the function itself.
Feb 01
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 1 February 2017 at 11:53:09 UTC, Stefan Koch wrote:
 On Tuesday, 31 January 2017 at 16:21:27 UTC, Stefan Koch wrote:
 function pointer support is coming!
This is more difficult then I expected, since the argument handling system was build without considering that we could have functions as arguments, therefore when trying to evaluate the function-argument it will overwrite the data-structures that are supposed to hold the information for the function we currently processing the arguments for. maybe this can be fixed by handling the arguments before handling the function itself.
I have not fixed the bug with top-level function arguments yet, However I have enabled function-pointer support for non-toplevel use such as in : int[] filterBy(int[] arr , bool function(uint) fn) { int[] result; uint resultLength; result.length = arr.length; foreach(i;0 .. arr.length) { auto e = arr[i]; bool r = true; r = fn(e); if(r) { result[resultLength++] = e; } } int[] filterResult; filterResult.length = resultLength; foreach(i; 0 .. resultLength) { filterResult[i] = result[i]; } return filterResult; } bool isDiv2(uint e) { bool result_; result_ = (e % 2 == 0); return result_; } bool isNotDiv2(uint e) { bool result_; result_ = (e % 2 != 0); return result_; } int[] run(int[] arr, bool div2) { return filterBy(arr, div2 ? &isDiv2 : &isNotDiv2); } static assert(run([3,4,5], true) == [4]); static assert(run([3,4,5], false) == [3,5]); //static assert(filterBy([3,4,5], &isDiv2) == [4]); // top-level function arguments do currently not work with newCTFE Enjoy!
Feb 03
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
std.conv.to is no longer blacklisted.
since I now bail on every string foreach.
As soon as ctfeable utf-conversion functions are in place, 
newCTFE can be used for fast low-memory overhead comptiletime 
parsing.
Feb 04
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 4 February 2017 at 10:27:58 UTC, Stefan Koch wrote:
 std.conv.to is no longer blacklisted.
 since I now bail on every string foreach.
 As soon as ctfeable utf-conversion functions are in place, 
 newCTFE can be used for fast low-memory overhead comptiletime 
 parsing.
Also I just enabled &&. It's still experimental but it seems to work sofar.
Feb 04
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 4 February 2017 at 11:32:52 UTC, Stefan Koch wrote:
 On Saturday, 4 February 2017 at 10:27:58 UTC, Stefan Koch wrote:
 std.conv.to is no longer blacklisted.
 since I now bail on every string foreach.
 As soon as ctfeable utf-conversion functions are in place, 
 newCTFE can be used for fast low-memory overhead comptiletime 
 parsing.
Also I just enabled &&. It's still experimental but it seems to work sofar.
experiment failed. Not necessarily because AndAnd itself is wrong but rather because something that is now not bailed out on, puts us in a bad state. I will disable && for the time being and continue working on the function argument problem. I can present you a nice snippet that just compiled with newCTFE
Feb 04
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 4 February 2017 at 16:30:30 UTC, Stefan Koch wrote:
 I can present you a nice snippet that just compiled with newCTFE
int[] makeAndInitArray(int until) { int[] result; result.length = until; foreach(i; 0 .. until) { result[i] = i; } return result; } int[] filter(alias filterFn)(int[] arr) { int[] result; result.length = arr.length; uint resultLength; foreach(i;0 .. arr.length) { auto e = arr[i]; if (filterFn(e)) { result[resultLength++] = e; } } result.length = resultLength; // I cannot remember implementing it but shrinking seems to work return result; } enum arr_500_000 = makeAndInitArray(500_000); static assert(arr_500_000.filter!(e => !(e % 3))); static assert([1,2,3,4,5,6,7,8,9,10,11,12,13].filter!(e => !(e % 3)) == [3,6,9,12]); newCTFE takes a quarter of the time, and only half of the memory. It would go even faster if we did not create the enum arr_500_000 but instead do that in a function-local variable. because then the array does not allocate 500_000 expression nodes.
Feb 04
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
Function pointers and other advanced features interfere with the 
ability to cache bytecode I just never noticed because the 
bytecode caching feature was disabled a long time.
Since I do not know how much of it is bitrot I removed it 
alltogether.
This feature will return and be more performant, when the outside 
function-arguments are getting fixed.
When this is done we can also remove a few special cases for the 
interpreter backend which is always good.
Feb 08
parent reply Kagamin <spam here.lot> writes:
Do I understand it right that union at compile time takes space 
equal to the sum of sizes of its members instead of the size of 
the largest member?
Feb 08
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 8 February 2017 at 14:01:33 UTC, Kagamin wrote:
 Do I understand it right that union at compile time takes space 
 equal to the sum of sizes of its members instead of the size of 
 the largest member?
generally it takes as much space as the largest member. Why do you ask ? And in what context ?
Feb 08
parent Kagamin <spam here.lot> writes:
On Wednesday, 8 February 2017 at 14:03:13 UTC, Stefan Koch wrote:
 generally it takes as much space as the largest member.
 Why do you ask ?
 And in what context ?
Asking about memory consumption. I thought it allowed access to overlapped fields, now I see it doesn't.
Feb 08
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
I just re-enabled << and >> for 32bit values, as they should not 
behave differently when applied to mixtures of 32bit and 64bit 
values.

Furthermore I fixed a few performance issues in the interaction 
between the interpreter and the bytecode-gen.

I also took the first steps for utf8/32 support.

32bit linux now fails on travis and the autotester.
I cannot reproduce this on my local setup therefore I deduce that 
it is a regression in the dmd version that is used on those 
platforms.
Feb 09
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
I just encountered a familiar problem again ...

uint fn(uint a)
{
     final switch(a)
     {
         case 1 :
             do
             {
                 a++;
                 if (a == 17) break;
             } while(a < 20);
                 return a;

         case 2 : return 1;
     }
}

static assert(fn(1) == 17); // fails because the return is 20
static assert(fn(2) == 1);

It turns out I forgot use the facility I implemented to handle 
this on DoStatements as well.
This is now fixed.
Feb 10
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
I am currently working on && again,
So far I hit a curious situation when handling
(b1 && b2 && b3)
everything works fine except that b3 will be evaluated 3 times.
It's a puzzler :)
Feb 13
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 13 February 2017 at 12:26:14 UTC, Stefan Koch wrote:
 I am currently working on && again,
 So far I hit a curious situation when handling
 (b1 && b2 && b3)
 everything works fine except that b3 will be evaluated 3 times.
 It's a puzzler :)
I just reverted every change of last week I should have a clean building branch after that. Tomorrow I will rewrite the removed functionality more cautiously.
Feb 13
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 13 February 2017 at 21:09:40 UTC, Stefan Koch wrote:
 On Monday, 13 February 2017 at 12:26:14 UTC, Stefan Koch wrote:
 I am currently working on && again,
 So far I hit a curious situation when handling
 (b1 && b2 && b3)
 everything works fine except that b3 will be evaluated 3 times.
 It's a puzzler :)
I just reverted every change of last week I should have a clean building branch after that. Tomorrow I will rewrite the removed functionality more cautiously.
I have now rebuild the new structures partially. It turns out that I was missing a few bailouts. However those should have been necessary before as well, so I am still puzzled.
Feb 14
parent Jacob Carlborg <doob me.com> writes:
On 2017-02-14 10:51, Stefan Koch wrote:

 I have now rebuild the new structures partially.
 It turns out that I was missing a few bailouts.
 However those should have been necessary before as well, so I am still
 puzzled.
301 posts (302 with this one). Time for a new thread? -- /Jacob Carlborg
Feb 14
prev sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
Jacob Carlborg wrote:
301 posts (302 with this one). Time for a new thread?
please, no! i just downloaded 4500+ posts only to get the root of this thread! don't turn it into useless effort!
Feb 14