www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - CTFE Status

reply Stefan Koch <uplink.coder googlemail.com> writes:
Hi Guys, since I got a few complaints about giving minor status 
updates in the announce group, I am opening this thread.

I will start with giving an overview of what works and what does 
not work.

Currently the only basic type you can do arithmetic on is int.
Altough you can compare longs since a few days.

These are the constructs that will work.

- foreach on static arrays strings and range-foreach  (those 
kinds (0 .. 64)).
- switches (even deeply nested ones)
- for and while loops
- ternary expressions (? :)
- if and else statements (as long as you don't use && and || )
- lables and gotos
- arithmetic expressions as well as post and pre increment and 
decrement

Constructs that will not work (but are actively worked on)

- assignment to static array cells
- long ulong arithmetic.
- function calls
- dynamic arrays and slices
- pointers
- structs
- && and ||
- sliceing

Constructs that will not work and are futher down the list.

- classes
- closures
- boundschecks
- asserts

Please note that there will probably be bugs all over the place.
So even the working features might not be working completely.
Oct 31 2016
next sibling parent Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: CTFE Status
References: <tdhdsvmknnpiugnngkex forum.dlang.org>
In-Reply-To: <tdhdsvmknnpiugnngkex forum.dlang.org>

--iKI927Tn283L8H4qPVenp66IJJbESsVN5
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Thank you and keep doing awesome stuff ;)


--iKI927Tn283L8H4qPVenp66IJJbESsVN5--
Oct 31 2016
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 10/31/16 9:29 AM, Stefan Koch wrote:

 Hi Guys, since I got a few complaints about giving minor status updates
 in the announce group, I am opening this thread.

 I will start with giving an overview of what works and what does not work.


Awesome work, I can't wait for efficient streamlined CTFE.

Note to those not aware, Stefan is working on replacing the CTFE engine 
in D with a bytecode generated one. So although he mentions things that 
"don't work", he's talking about them in the context of the new CTFE 
engine. Many of these things already work in current CTFE implementation.

-Steve
Oct 31 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 31 October 2016 at 20:49:58 UTC, Steven Schveighoffer 
wrote:

 On 10/31/16 9:29 AM, Stefan Koch wrote:

 Hi Guys, since I got a few complaints about giving minor 
 status updates
 in the announce group, I am opening this thread.

 I will start with giving an overview of what works and what 
 does not work.


 Awesome work, I can't wait for efficient streamlined CTFE.

 Note to those not aware, Stefan is working on replacing the 
 CTFE engine in D with a bytecode generated one. So although he 
 mentions things that "don't work", he's talking about them in 
 the context of the new CTFE engine. Many of these things 
 already work in current CTFE implementation.

 -Steve


All of them work in current CTFE which is why the mark is quite 
high.

I am glad to hear you guys appriciate my work.
It does help a ton to hear it from time to time.


---

Very recent news:
  NEW CTFE PASSES THE DRUNTIME UNITESTS!!!!
Nov 01 2016
next sibling parent Daniel Kozak via Digitalmars-d <digitalmars-d puremagic.com> writes:
Dne 1.11.2016 v 16:50 Stefan Koch via Digitalmars-d napsal(a):


 On Monday, 31 October 2016 at 20:49:58 UTC, Steven Schveighoffer wrote:

 On 10/31/16 9:29 AM, Stefan Koch wrote:

 Hi Guys, since I got a few complaints about giving minor status updates
 in the announce group, I am opening this thread.

 I will start with giving an overview of what works and what does not 
 work.


 Awesome work, I can't wait for efficient streamlined CTFE.

 Note to those not aware, Stefan is working on replacing the CTFE 
 engine in D with a bytecode generated one. So although he mentions 
 things that "don't work", he's talking about them in the context of 
 the new CTFE engine. Many of these things already work in current 
 CTFE implementation.

 -Steve


 All of them work in current CTFE which is why the mark is quite high.

 I am glad to hear you guys appriciate my work.
 It does help a ton to hear it from time to time.


 ---

 Very recent news:
  NEW CTFE PASSES THE DRUNTIME UNITESTS!!!!


This is so awesome, really good work :)
Nov 01 2016
prev sibling next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 1 November 2016 at 15:50:06 UTC, Stefan Koch wrote:

 Very recent news:
  NEW CTFE PASSES THE DRUNTIME UNITESTS!!!!


Great work!
Nov 01 2016
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 1 November 2016 at 15:50:06 UTC, Stefan Koch wrote:


 Very recent news:
  NEW CTFE PASSES THE DRUNTIME UNITESTS!!!!


Now a phobos unittest miscompiles :(

Again passing the unittests does not mean too much.
I just means I bail out before I generate invalid code :)
Nov 01 2016
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 1 November 2016 at 19:28:03 UTC, Stefan Koch wrote:

 Now a phobos unittest miscompiles :(

 Again passing the unittests does not mean too much.
 I just means I bail out before I generate invalid code :)


Keep up!
Nov 01 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 1 November 2016 at 21:12:06 UTC, Nordlöw wrote:

 On Tuesday, 1 November 2016 at 19:28:03 UTC, Stefan Koch wrote:

 Now a phobos unittest miscompiles :(

 Again passing the unittests does not mean too much.
 I just means I bail out before I generate invalid code :)


 Keep up!


I am now passing phobos unittests!
However please take it with a grain of salt ;)
The following line resbonsible for baling out of functions I know 
are failing.

	if (fd.ident == Identifier.idPool("isRooted") || fd.ident == 
Identifier.idPool("__lambda2") || fd.ident == 
Identifier.idPool("divideRoundUp") || fd.ident == 
Identifier.idPool("isSameLength") || fd.ident == 
Identifier.idPool("wrapperParameters") || fd.ident == 
Identifier.idPool("wrap") || fd.ident == 
Identifier.idPool("args"))
		{
			IGaveUp = true;
			return ;
		}

I am working on reducing test-cases to figure out why this is 
failing.
The code is burried in templates makeing it hard to see what is 
going on.

Still passing the tests is passing the tests :)
Nov 02 2016
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 2 November 2016 at 11:01:55 UTC, Stefan Koch wrote:

 I am now passing phobos unittests!


I can't wait to see this in action!

Keep on going and I'll keep praying... 😊
Nov 02 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 2 November 2016 at 23:48:46 UTC, Nordlöw wrote:

 On Wednesday, 2 November 2016 at 11:01:55 UTC, Stefan Koch 
 wrote:

 I am now passing phobos unittests!


 I can't wait to see this in action!

 Keep on going and I'll keep praying... 😊


No need for prayer.

I am trying my best to be doing solid engineering work.

Neither god nor the devil will paly any role.

I am currently working on making the DMD-AST-facing side of the 
code more solid.

With a bit of luck I can role out a preview release soon.

But I will only do so when there are no known bugs left.
Nov 02 2016
parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 3 November 2016 at 00:02:46 UTC, Stefan Koch wrote:

 No need for prayer.

 I am trying my best to be doing solid engineering work.

 Neither god nor the devil will paly any role.


Got it.
Nov 03 2016
prev sibling next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 31 October 2016 at 13:29:01 UTC, Stefan Koch wrote:

 These are the constructs that will work.

 - foreach on static arrays strings and range-foreach  (those 
 kinds (0 .. 64)).
 - switches (even deeply nested ones)
 - for and while loops
 - ternary expressions (? :)
 - if and else statements (as long as you don't use && and || )
 - lables and gotos
 - arithmetic expressions as well as post and pre increment and 
 decrement

 Constructs that will not work (but are actively worked on)

 - assignment to static array cells
 - long ulong arithmetic.
 - function calls
 - dynamic arrays and slices
 - pointers
 - structs
 - && and ||
 - sliceing

 Constructs that will not work and are futher down the list.

 - classes
 - closures
 - boundschecks
 - asserts


Do all these work in current implementation?
Nov 01 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 1 November 2016 at 16:44:14 UTC, Nordlöw wrote:

 On Monday, 31 October 2016 at 13:29:01 UTC, Stefan Koch wrote:

 These are the constructs that will work.

 - foreach on static arrays strings and range-foreach  (those 
 kinds (0 .. 64)).
 - switches (even deeply nested ones)
 - for and while loops
 - ternary expressions (? :)
 - if and else statements (as long as you don't use && and || )
 - lables and gotos
 - arithmetic expressions as well as post and pre increment and 
 decrement

 Constructs that will not work (but are actively worked on)

 - assignment to static array cells
 - long ulong arithmetic.
 - function calls
 - dynamic arrays and slices
 - pointers
 - structs
 - && and ||
 - sliceing

 Constructs that will not work and are futher down the list.

 - classes
 - closures
 - boundschecks
 - asserts


 Do all these work in current implementation?


Yes of they do!

There is a lot of work behind the implementation!
I intend to keep the current implemntation around as a fallback.
For things that are used rarely.
like floating point and others :)
Nov 01 2016
parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Tuesday, 1 November 2016 at 17:41:35 UTC, Stefan Koch wrote:

 I intend to keep the current implemntation around as a fallback.
 For things that are used rarely.
 like floating point and others :)


Is that a good idea? It means that there will be tons of really 
complex code that is rarely if ever used and therefore gets no 
testing...
Nov 03 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 3 November 2016 at 15:44:20 UTC, Marc Schütz wrote:

 On Tuesday, 1 November 2016 at 17:41:35 UTC, Stefan Koch wrote:

 I intend to keep the current implemntation around as a 
 fallback.
 For things that are used rarely.
 like floating point and others :)


 Is that a good idea? It means that there will be tons of really 
 complex code that is rarely if ever used and therefore gets no 
 testing...


Passing the whole test-suite is as good as it gets.
Nov 03 2016
parent Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Thursday, 3 November 2016 at 16:35:08 UTC, Stefan Koch wrote:

 On Thursday, 3 November 2016 at 15:44:20 UTC, Marc Schütz wrote:

 On Tuesday, 1 November 2016 at 17:41:35 UTC, Stefan Koch wrote:

 I intend to keep the current implemntation around as a 
 fallback.
 For things that are used rarely.
 like floating point and others :)


 Is that a good idea? It means that there will be tons of 
 really complex code that is rarely if ever used and therefore 
 gets no testing...


 Passing the whole test-suite is as good as it gets.


But for operations that your new engine supports, the test-suite 
will not normally test the old engine, right? Except in certain 
edge cases where that the new engine happens not to support, so 
it falls back to the old one. But these edge cases are probably 
not well-defined and hard to test exhaustively...
Nov 04 2016
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Thu, 03 Nov 2016 15:44:20 +0000, Marc Schütz wrote:


 On Tuesday, 1 November 2016 at 17:41:35 UTC, Stefan Koch wrote:

 I intend to keep the current implemntation around as a fallback.
 For things that are used rarely.
 like floating point and others :)


 
 Is that a good idea? It means that there will be tons of really complex
 code that is rarely if ever used and therefore gets no testing...


I think the intent is that the current implementation will always be used 
for specific features, not that the compiler will attempt to run CTFE 
with the bytecode version and resort to the previous implementation if 
something goes wrong.
Nov 03 2016
parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Friday, 4 November 2016 at 01:19:36 UTC, Chris Wright wrote:

 On Thu, 03 Nov 2016 15:44:20 +0000, Marc Schütz wrote:


 On Tuesday, 1 November 2016 at 17:41:35 UTC, Stefan Koch wrote:

 I intend to keep the current implemntation around as a 
 fallback.
 For things that are used rarely.
 like floating point and others :)


 
 Is that a good idea? It means that there will be tons of 
 really complex code that is rarely if ever used and therefore 
 gets no testing...


 I think the intent is that the current implementation will 
 always be used for specific features, not that the compiler 
 will attempt to run CTFE with the bytecode version and resort 
 to the previous implementation if something goes wrong.


In this case better make sure to remove the resulting dead code 
from the old interpreter, otherwise it will become and 
unmaintainable mess in the long run.
Nov 04 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 4 November 2016 at 10:59:25 UTC, Marc Schütz wrote:

 
 In this case better make sure to remove the resulting dead code 
 from the old interpreter, otherwise it will become and 
 unmaintainable mess in the long run.


If you ask me the old interpreter is already an unmaintainble 
mess.
It will not be possible to remove code since the old interpreter 
and the new one do not share any common state or code-paths.
Therefore all evaluation-capabilty the old interpreter has must 
be retained otherwise it would not be useful anymore.

I thought about it some more, and decided that I will replace the 
old interpreter completely in the long run.
However in order to get something release before 2017, I have to 
rely on it being there.

I recently lost 3 days of work because of my git-skills.
Now I am working on simplyfing the cases the interpreter can get 
into.
Before I would do some optimisation to avoid temporaries and the 
like.
Since I now have the llvm-backend to verify the code is correct 
it's no longer crucial in order to be able to debug.
This simplification will (hopefully) finally enable me to proper 
handling of boolean-expressions.
I suspect that most of the invalid code I generated came from a 
bug in the "temporaries-avoidance optimisation"

Also I will put some effort into documenting the new engine 
better.
However code-gen bugs take priority over documentation.
Nov 04 2016
next sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Saturday, November 05, 2016 01:21:48 Stefan Koch via Digitalmars-d wrote:

 I thought about it some more, and decided that I will replace the
 old interpreter completely in the long run.
 However in order to get something release before 2017, I have to
 rely on it being there.


That seems reasonable, but it definitely seems messy to leave the old stuff
there in the long run, especially if only part of it is really used and
tested.

- Jonathan M Davis
Nov 05 2016
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 06/11/2016 1:57 AM, Jonathan M Davis via Digitalmars-d wrote:

 On Saturday, November 05, 2016 01:21:48 Stefan Koch via Digitalmars-d wrote:

 I thought about it some more, and decided that I will replace the
 old interpreter completely in the long run.
 However in order to get something release before 2017, I have to
 rely on it being there.


 That seems reasonable, but it definitely seems messy to leave the old stuff
 there in the long run, especially if only part of it is really used and
 tested.

 - Jonathan M Davis


Based upon some help I gave Stefan early on with the current 
implementation, it will be left alone as is. It doesn't need to change 
in any way beyond the entry point.

So not at all messy and it certainly is already well tested.
Especially since it is mostly localized to a single file.
Nov 05 2016
prev sibling next sibling parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Saturday, 5 November 2016 at 01:21:48 UTC, Stefan Koch wrote:

 I thought about it some more, and decided that I will replace 
 the old interpreter completely in the long run.
 However in order to get something release before 2017, I have 
 to rely on it being there.


That's better of course. Parallel use of new and old just 
shouldn't stay there indefinitely. Look at the problems with the 
AA implementation as a similar example.


 I recently lost 3 days of work because of my git-skills.


Unless you haven't committed your work yet, almost everything in 
Git can be undone. Make a copy of your entire project directory 
(including .git) and then have a look at `git reflog` around the 
time the disaster happened. It will show you commit IDs that you 
can check out.
Nov 05 2016
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Saturday, 5 November 2016 at 15:48:19 UTC, Marc Schütz wrote:

 I recently lost 3 days of work because of my git-skills.


 Unless you haven't committed your work yet, almost everything 
 in Git can be undone. Make a copy of your entire project 
 directory (including .git) and then have a look at `git reflog` 
 around the time the disaster happened. It will show you commit 
 IDs that you can check out.


Solid advice right there ^
Nov 05 2016
prev sibling next sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Saturday, 5 November 2016 at 15:48:19 UTC, Marc Schütz wrote:

 Unless you haven't committed your work yet, almost everything 
 in Git can be undone. Make a copy of your entire project 
 directory (including .git) and then have a look at `git reflog` 
 around the time the disaster happened. It will show you commit 
 IDs that you can check out.


especially if one will turn off automatic git gc.
Nov 05 2016
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 11/05/2016 11:48 AM, Marc Schütz wrote:

 On Saturday, 5 November 2016 at 01:21:48 UTC, Stefan Koch wrote:

 I recently lost 3 days of work because of my git-skills.


 Unless you haven't committed your work yet, almost everything in Git can
 be undone. Make a copy of your entire project directory (including .git)
 and then have a look at `git reflog` around the time the disaster
 happened. It will show you commit IDs that you can check out.


Yea, but unless you're a git-fu master, sometimes figuring out how to 
fix whatever got messed up can lose you 3 days of work ;)

I really want to make a saner CLI front-end for git, but that would 
require learning more about git than I really ever want to know :(
Nov 08 2016
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Nov 08, 2016 at 11:40:31AM -0500, Nick Sabalausky via Digitalmars-d
wrote:

 next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 11/08/2016 11:57 AM, H. S. Teoh via Digitalmars-d wrote:

 The thing about git is that at its core, it's really very simple. Dumb,
 even.  It's basically a program for managing a directed acyclic graph
 (DAG).  That's all there is to it. The rest is just frills.

 Trying to rationalize git in terms of traditional version control
 systems is what usually causes lots of confusion, incomprehension, and
 frustration.  To truly grok git, you have to just forget about
 traditional version control concepts, and think purely in terms of DAGs.
 Once you do, everything falls into place and it all makes sense in its
 own peculiar way (including all the flaws :-P).


I don't suppose you have a handy link to an "Understanding Git as a DAG 
manager instead of VCS" document?



 Nov 08 2016



 next sibling parent  Antonio Corbi <amcb ggmail.com>  writes:


On Tuesday, 8 November 2016 at 17:12:34 UTC, Nick Sabalausky 
wrote:

 On 11/08/2016 11:57 AM, H. S. Teoh via Digitalmars-d wrote:

[...]


 I don't suppose you have a handy link to an "Understanding Git 
 as a DAG manager instead of VCS" document?


Hi Nick,

Maybe this one is useful for you:

http://eagain.net/articles/git-for-computer-scientists/

Antonio



 Nov 08 2016



prev sibling  parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Tue, Nov 08, 2016 at 12:12:34PM -0500, Nick Sabalausky via Digitalmars-d
wrote:

 On 11/08/2016 11:57 AM, H. S. Teoh via Digitalmars-d wrote:

 
 The thing about git is that at its core, it's really very simple.
 Dumb, even.  It's basically a program for managing a directed
 acyclic graph (DAG).  That's all there is to it. The rest is just
 frills.
 
 Trying to rationalize git in terms of traditional version control
 systems is what usually causes lots of confusion, incomprehension,
 and frustration.  To truly grok git, you have to just forget about
 traditional version control concepts, and think purely in terms of
 DAGs.  Once you do, everything falls into place and it all makes
 sense in its own peculiar way (including all the flaws :-P).
 


 
 I don't suppose you have a handy link to an "Understanding Git as a
 DAG manager instead of VCS" document?


Nothing immediately comes to mind, but thanks to Dr. Google, I found
this page that's sorta helpful:

	http://ericsink.com/vcbe/html/directed_acyclic_graphs.html

And perhaps these:

	http://eagain.net/articles/git-for-computer-scientists/
	http://marklodato.github.io/visual-git-guide/index-en.html

Or maybe I should write my own article about git being based on a DAG.
:-P


T

-- 
Leather is waterproof.  Ever see a cow with an umbrella?



 Nov 08 2016



 next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Back to CTFE.
The 64Bit bug is fixed.
And now there is one bailout-line less.

I am onto the next bug.
Let's hunt them all down!



 Nov 08 2016



 next sibling parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 8 November 2016 at 21:56:32 UTC, Stefan Koch wrote:

 I am onto the next bug.
 Let's hunt them all down!


Marvellous!



 Nov 08 2016



prev sibling next sibling parent  deadalnix <deadalnix gmail.com>  writes:


On Tuesday, 8 November 2016 at 21:56:32 UTC, Stefan Koch wrote:

 Back to CTFE.
 The 64Bit bug is fixed.
 And now there is one bailout-line less.

 I am onto the next bug.
 Let's hunt them all down!


https://media.giphy.com/media/FmNXeuoadNTpe/giphy.gif



 Nov 08 2016



prev sibling  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 8 November 2016 at 21:56:32 UTC, Stefan Koch wrote:

 Back to CTFE.
 The 64Bit bug is fixed.
 And now there is one bailout-line less.

 I am onto the next bug.
 Let's hunt them all down!


More 64bit bugs appeared that were shadowed by the one I fixed.
Also a new bugs related to labeled breaks appeared.



 Nov 09 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


After the bad news now a bit of good news.
I just finished the strCat algorithm;

The reason this took so much time is because of the 
representation the heap has in the elevator.
it is a uint[] meaning you are getting into trouble with offsets 
if the first string is not a multiple of 4 characters long:)
This is the code for strcat :

uint[] strcat(const uint[] a, const uint[] b, uint aLength = 0, 
uint bLength = 0)
{
   aLength = a[0];
   bLength = b[0];
   uint resultLength = a[0] + b[0];
   uint[] result;
   result.length = neededArraySize(resultLength);
   assert(result.length == (a.length + b.length) - 2);
   result[0] = resultLength;
   auto resultPosition = 1;

   auto aMinusOne = !(aLength & 3);
   foreach(p, ca; a[1 .. $ - aMinusOne])
   {
     result[resultPosition++] = ca;
   }
   // this one is easy we just copy
     auto bMinusOne = !(bLength & 3);

   uint offset = aLength & 3;
   if (offset)
   {
     assert(offset == 2);
     resultPosition--;
     foreach(p, cb; b[1 .. $])
     {
       result[resultPosition++] |= (cb & ((1U << (offset*8)) -1)) 
<< (offset*8);
       if (resultPosition == result.length) break;
       result[resultPosition++] |= (cb & ~((1U << (offset*8)) -1)) 

 (offset*8);




       resultPosition--;
     }
   }
   else
   {
     foreach(p, cb; b[1 .. $ - bMinusOne])
     {
       result[resultPosition++] = cb;
     }
   }

   return result;
}

uint neededArraySize(uint size)
{
   return (((size - 1) / 4) + 2) + (!(size & 3));
}



 Nov 09 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 10 November 2016 at 01:17:48 UTC, Stefan Koch wrote:

 After the bad news now a bit of good news.
 I just finished the strCat algorithm;

 [...]


As expected this code is slightly wrong and will fail on the 
offset == 3 case.
It will need a little bit of tinkering until it's ready to go :)



 Nov 10 2016



  parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On Thursday, November 10, 2016 18:40:03 Stefan Koch via Digitalmars-d wrote:

 On Thursday, 10 November 2016 at 01:17:48 UTC, Stefan Koch wrote:

 After the bad news now a bit of good news.
 I just finished the strCat algorithm;

 [...]


 As expected this code is slightly wrong and will fail on the
 offset == 3 case.
 It will need a little bit of tinkering until it's ready to go :)


And aren't you glad that you have all of these unit tests to verify that
what you're working works properly? They don't necessarily catch everything,
but it would be _way_ worse to completely replace something like this
without them. :)

The way that D promotes unit testing is definitely one of its strong points.

- Jonathan M Davis



 Nov 10 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 10 November 2016 at 20:15:49 UTC, Jonathan M Davis 
wrote:

 On Thursday, November 10, 2016 18:40:03 Stefan Koch via 
 Digitalmars-d wrote:

 On Thursday, 10 November 2016 at 01:17:48 UTC, Stefan Koch 
 wrote:

 After the bad news now a bit of good news.
 I just finished the strCat algorithm;

 [...]


 As expected this code is slightly wrong and will fail on the
 offset == 3 case.
 It will need a little bit of tinkering until it's ready to go 
 :)


 And aren't you glad that you have all of these unit tests to 
 verify that what you're working works properly? They don't 
 necessarily catch everything, but it would be _way_ worse to 
 completely replace something like this without them. :)

 The way that D promotes unit testing is definitely one of its 
 strong points.

 - Jonathan M Davis



I love compile-time unitests.
Who does not like them ?

The interpret3.d test in the compiler acceptance testsuite is 
almost a work of art.
After you pass that one you can be pretty sure to have a 
complainant d compiler.

The current Status of stringCat can be found here :
https://gist.github.com/UplinkCoder/fed000f5ca41ad3d65d59a7a63d5ef6f



 Nov 10 2016



 next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 10 November 2016 at 20:51:01 UTC, Stefan Koch wrote:

 On Thursday, 10 November 2016 at 20:15:49 UTC, Jonathan M Davis 
 wrote:

 On Thursday, November 10, 2016 18:40:03 Stefan Koch via 
 Digitalmars-d wrote:

 On Thursday, 10 November 2016 at 01:17:48 UTC, Stefan Koch 
 wrote:

 After the bad news now a bit of good news.
 I just finished the strCat algorithm;

 [...]


 As expected this code is slightly wrong and will fail on the
 offset == 3 case.
 It will need a little bit of tinkering until it's ready to go 
 :)


 And aren't you glad that you have all of these unit tests to 
 verify that what you're working works properly? They don't 
 necessarily catch everything, but it would be _way_ worse to 
 completely replace something like this without them. :)

 The way that D promotes unit testing is definitely one of its 
 strong points.

 - Jonathan M Davis



 I love compile-time unitests.
 Who does not like them ?

 The interpret3.d test in the compiler acceptance testsuite is 
 almost a work of art.
 After you pass that one you can be pretty sure to have a 
 complainant d compiler.

 The current Status of stringCat can be found here :
 https://gist.github.com/UplinkCoder/fed000f5ca41ad3d65d59a7a63d5ef6f


StringConcat works and will be translated into bytecode soon.
I am going to get a few big things done during next week.
Namely support for dynamic arrays.
At least I hope so :) You never know what bugs might rain on your 
programming parade.



 Nov 11 2016



prev sibling  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Assignment to Array.length works now :)
the following code :

uint[] makeArr()
{
     uint[] arr;
     arr.length = 5;
     return arr;
}


pragma(msg, makeArr());

results in :

CTFE_DEBUG_MESSAGE : building Array of Length 5
[0u, 0u, 0u, 0u, 0u]



 Nov 13 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 14 November 2016 at 05:27:15 UTC, Stefan Koch wrote:

 Assignment to Array.length works now :)
 the following code :

 uint[] makeArr()
 {
     uint[] arr;
     arr.length = 5;
     return arr;
 }


 pragma(msg, makeArr());

 results in :

 CTFE_DEBUG_MESSAGE : building Array of Length 5
 [0u, 0u, 0u, 0u, 0u]


Assignment now also works :)

uint[] MakeAndInitArr(uint length)
{
     uint[] arr;
     arr.length = length;

     foreach(i;0 .. length)
     {
         arr[i] = i + 3;
     }
     return arr;
}

pragma(msg, MakeAndInitArr(12));

Output :
[3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]



 Nov 13 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 14 November 2016 at 07:57:20 UTC, Stefan Koch wrote:

 On Monday, 14 November 2016 at 05:27:15 UTC, Stefan Koch wrote:

 Assignment to Array.length works now :)
 the following code :

 uint[] makeArr()
 {
     uint[] arr;
     arr.length = 5;
     return arr;
 }


 pragma(msg, makeArr());

 results in :

 CTFE_DEBUG_MESSAGE : building Array of Length 5
 [0u, 0u, 0u, 0u, 0u]


 Assignment now also works :)

 uint[] MakeAndInitArr(uint length)
 {
     uint[] arr;
     arr.length = length;

     foreach(i;0 .. length)
     {
         arr[i] = i + 3;
     }
     return arr;
 }

 pragma(msg, MakeAndInitArr(12));

 Output :
 [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]


This for a length values in the range of short.max/4.
Which is the maximal heapSize for now.
This is up to 1.5 times faster then the old engine.

Once the heap-limit is lifted you can see this rock :)



 Nov 14 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


To more bullet points cross the border from unsupported to 
supported.
- assignment to static array cells
- long ulong arithmetic.

However there is a a bug inside the code that does 
bounds-checking for array assignment.
In rare cases it can trigger a out-bounds-error on newly created 
arrays.

Dynamic arrays work partially.

The only way to create them is to assign to length.
For initialization fill the cells in a for-loop.



 Nov 14 2016



 next sibling parent reply Bastiaan Veelo <Bastiaan Veelo.net>  writes:


On Tuesday, 15 November 2016 at 01:35:42 UTC, Stefan Koch wrote:

 Two more bullet points cross the border from unsupported to 
 supported.


Nice, Stefan. I always come back to this thread to see your 
progress. Thanks for making it easy to follow it.



 Nov 15 2016



  parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 15 November 2016 at 19:02:29 UTC, Bastiaan Veelo 
wrote:

 Nice, Stefan. I always come back to this thread to see your 
 progress. Thanks for making it easy to follow it.


I agree.



 Nov 15 2016



prev sibling  parent reply deadalnix <deadalnix gmail.com>  writes:


On Tuesday, 15 November 2016 at 01:35:42 UTC, Stefan Koch wrote:

 However there is a a bug inside the code that does 
 bounds-checking for array assignment.
 In rare cases it can trigger a out-bounds-error on newly 
 created arrays.


This raise all kind of red flags to me. What are the design 
decision that lead to this ?



 Nov 15 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 15 November 2016 at 22:50:49 UTC, deadalnix wrote:

 On Tuesday, 15 November 2016 at 01:35:42 UTC, Stefan Koch wrote:

 However there is a a bug inside the code that does 
 bounds-checking for array assignment.
 In rare cases it can trigger a out-bounds-error on newly 
 created arrays.


 This raise all kind of red flags to me. What are the design 
 decision that lead to this ?


I am still figuring out when and why the bug is caused.
The Byte-code for slice allocation is rather complex.
Because it has to deal with resizing slices as well.
I suspect that somewhere the heapPtr is not bumped or the length 
is not set correctly.



 Nov 15 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 15 November 2016 at 23:46:51 UTC, Stefan Koch wrote:

 On Tuesday, 15 November 2016 at 22:50:49 UTC, deadalnix wrote:

 On Tuesday, 15 November 2016 at 01:35:42 UTC, Stefan Koch 
 wrote:

 However there is a a bug inside the code that does 
 bounds-checking for array assignment.
 In rare cases it can trigger a out-bounds-error on newly 
 created arrays.


 This raise all kind of red flags to me. What are the design 
 decision that lead to this ?


 I am still figuring out when and why the bug is caused.
 The Byte-code for slice allocation is rather complex.
 Because it has to deal with resizing slices as well.
 I suspect that somewhere the heapPtr is not bumped or the 
 length is not set correctly.


Indeed the length was not set on a code-path meant for resizeing.
The problem is fixed :)

The HeapLimit has been raised to a more reasonable limit of 2 ^^ 
24 Addresses. (Which means that you'll have 2 ^^ 24 Bytes in 
practice.)



 Nov 16 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 16 November 2016 at 09:22:01 UTC, Stefan Koch wrote:

 On Tuesday, 15 November 2016 at 23:46:51 UTC, Stefan Koch wrote:


 I suspect that somewhere the heapPtr is not bumped or the 
 length is not set correctly.


 Indeed the length was not set on a code-path meant for 
 resizeing.
 The problem is fixed :)

 The HeapLimit has been raised to a more reasonable limit of 2 
 ^^ 24 Addresses. (Which means that you'll have 2 ^^ 24 Bytes in 
 practice.)


In fact, there is a single design decision that fosters these 
kinds of problems.
And that is to go with a low-level IR.

However although it's a tough route. It's also the only solution, 
(the only one I could think of), that will really enable CTFE to 
scale gracefully.

My latest measurements show that even for relatively small arrays 
(2 ^^ 15) bytes.
There is a 2x speedup.
When the interpreter backend.
As soon as my own jit backend is in place the performance will be 
a factor of 6 better.
Makeing newCTFE 12x faster then the current engine, even on small 
one-shot functions!



 Nov 16 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Here is a small demostration of the performance increase :


x 2> x

real	0m0.199s
user	0m0.180s
sys	0m0.017s

-bc-ctfe > x 2> x

real	0m0.072s
user	0m0.050s
sys	0m0.020s

Please note that newCTFE only spends 15 ms inside the evaluation, 
most time is spent clearing the 2^^24 Bytes of heap-memory to 
zero.

The sourcode of testSettingArrayLength is
uint[] MakeAndInitArr(uint length)
{
     uint[] arr;
     arr.length = length;

     foreach(i;0 .. length)
     {
         arr[i] = i + 3;
     }
     return arr;
}

static assert(MakeAndInitArr(ushort.max).length == ushort.max);



 Nov 16 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 16 November 2016 at 09:45:24 UTC, Stefan Koch wrote:

 Here is a small demostration of the performance increase :



 x 2> x


 real	0m0.199s
 user	0m0.180s
 sys	0m0.017s

 -bc-ctfe > x 2> x

 real	0m0.072s
 user	0m0.050s
 sys	0m0.020s

 Please note that newCTFE only spends 15 ms inside the 
 evaluation, most time is spent clearing the 2^^24 Bytes of 
 heap-memory to zero.

 The sourcode of testSettingArrayLength is
 uint[] MakeAndInitArr(uint length)
 {
     uint[] arr;
     arr.length = length;

     foreach(i;0 .. length)
     {
         arr[i] = i + 3;
     }
     return arr;
 }

 static assert(MakeAndInitArr(ushort.max).length == ushort.max);


A more accurate breakdown :

Initializing Heap:     18.6 ms
Generating Bytecode:    1.2 ms
Executing Bytecode:    13.2 ms
Converting to CTFE-EXp: 9.1 ms

For a second execution of the same function with the same 
arguments within the same file the numbers look like :

Initializing Heap:     16.7 ms
Generating Bytecode:    0.6 ms
Executing Bytecode:    13.2 ms
Converting to CTFE-EXp: 9.3 ms



 Nov 16 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 16 November 2016 at 10:07:06 UTC, Stefan Koch wrote:

 A more accurate breakdown :

 Initializing Heap:     18.6 ms
 Generating Bytecode:    1.2 ms
 Executing Bytecode:    13.2 ms
 Converting to CTFE-EXp: 9.1 ms

 For a second execution of the same function with the same 
 arguments within the same file the numbers look like :

 Initializing Heap:     16.7 ms
 Generating Bytecode:    0.6 ms
 Executing Bytecode:    13.2 ms
 Converting to CTFE-EXp: 9.3 ms


The above numbers were obtained using a debug build made with dmd.
The following numbers are from a optimized build with ldmd2

First Execution (cold cache) :

Initializing Heap:     17.4 ms
Generating Bytecode:    0.7 ms
Executing Bytecode:     5.3 ms
Converting to CTFE-EXp: 5.1 ms

Second run (warmer cache) :

Initializing Heap:     16.9 ms
Generating Bytecode:    0.3 ms
Executing Bytecode:     5.3 ms
Converting to CTFE-EXp: 4.9 ms



 Nov 16 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 16 November 2016 at 10:25:30 UTC, Stefan Koch wrote:


 First Execution (cold cache) :

 Initializing Heap:     17.4 ms
 Generating Bytecode:    0.7 ms
 Executing Bytecode:     5.3 ms
 Converting to CTFE-EXp: 5.1 ms

 Second run (warmer cache) :

 Initializing Heap:     16.9 ms
 Generating Bytecode:    0.3 ms
 Executing Bytecode:     5.3 ms
 Converting to CTFE-EXp: 4.9 ms



And Again a bit of bad news.
Due to problems in the lowering of function arguments the 
implementation of strcat is delayed again.



 Nov 16 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 16 November 2016 at 14:44:06 UTC, Stefan Koch wrote:

 And Again a bit of bad news.
 Due to problems in the lowering of function arguments the 
 implementation of strcat is delayed again.


The bug does not affect strings.
Since strings are not build up out of multiple sub-expressions.
strcat is on it's way.

I have begun the process of macrofication.



 Nov 16 2016



  parent reply Andrea Fontana <nospam example.com>  writes:


On Thursday, 17 November 2016 at 05:35:33 UTC, Stefan Koch wrote:

 On Wednesday, 16 November 2016 at 14:44:06 UTC, Stefan Koch 
 wrote:

 And Again a bit of bad news.
 Due to problems in the lowering of function arguments the 
 implementation of strcat is delayed again.


 The bug does not affect strings.
 Since strings are not build up out of multiple sub-expressions.
 strcat is on it's way.

 I have begun the process of macrofication.


I follow this thread every day. I hope you'll write an article on 
dlang blog when the work will be completed :)



 Nov 17 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 17 November 2016 at 08:39:57 UTC, Andrea Fontana 
wrote:

 I follow this thread every day. I hope you'll write an article 
 on dlang blog when the work will be completed :)


Mike Parker is going to write an short article about it based on 
information I gave him via mail.

I am afraid my blog-writing skills are a bit under-developed.

On the topic of CTFE :
My attempts of half-automatically generating a string-concat 
macro have yet to succeed.

I am currently busy fixing bugs :)
I apologize for the seemingly slow progress.
However,
such is the nature of low-level code.
There is really no way around it,
if we are aiming for performance.



 Nov 17 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 17 November 2016 at 09:19:57 UTC, Stefan Koch wrote:

 On Thursday, 17 November 2016 at 08:39:57 UTC, Andrea Fontana 
 wrote:

 I follow this thread every day. I hope you'll write an article 
 on dlang blog when the work will be completed :)


 Mike Parker is going to write an short article about it based 
 on information I gave him via mail.

 I am afraid my blog-writing skills are a bit under-developed.

 On the topic of CTFE :
 My attempts of half-automatically generating a string-concat 
 macro have yet to succeed.

 I am currently busy fixing bugs :)
 I apologize for the seemingly slow progress.
 However,
 such is the nature of low-level code.
 There is really no way around it,
 if we are aiming for performance.


I just fixed the bug with multiple ArraysLiterals as arguments.
We are not out of the mud yet. This will most likely not play 
nice with nested arrays.
But let's cross that bridge when we come to it :)



 Nov 18 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 18 November 2016 at 15:09:02 UTC, Stefan Koch wrote:


 I just fixed the bug with multiple ArraysLiterals as arguments.


This means the following code will now compile and properly 
execute:

uint Sum3Arrays (uint[] a1, uint[] a2, uint[] a3)
{
   uint result;
   for(int i; i != a1.length; i++)
   {
     result += a1[i];
   }
   for(int i; i != a2.length; i++)
   {
     result += a2[i];
   }
   for(int i; i != a3.length; i++)
   {
     result += a3[i];
   }

   return result;
}

static assert(Sum3Arrays([2,3],[],[]) == 5);
static assert(Sum3Arrays([2],[],[3]) == 5);
static assert(Sum3Arrays([3],[1],[1]) == 5);



 Nov 18 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 18 November 2016 at 17:21:10 UTC, Stefan Koch wrote:

 On Friday, 18 November 2016 at 15:09:02 UTC, Stefan Koch wrote:


 I just fixed the bug with multiple ArraysLiterals as arguments.


 This means the following code will now compile and properly 
 execute:

 uint Sum3Arrays (uint[] a1, uint[] a2, uint[] a3)
 {
   uint result;
   for(int i; i != a1.length; i++)
   {
     result += a1[i];
   }
   for(int i; i != a2.length; i++)
   {
     result += a2[i];
   }
   for(int i; i != a3.length; i++)
   {
     result += a3[i];
   }

   return result;
 }

 static assert(Sum3Arrays([2,3],[],[]) == 5);
 static assert(Sum3Arrays([2],[],[3]) == 5);
 static assert(Sum3Arrays([3],[1],[1]) == 5);


Another small update.
I just fixed the bailout mechanism.
Before it would ignore failiures in argument processing and start 
the interpreter with bogus as arguments.
This of course looked like miscompiled code.
I am happy it was just the bailout :)



 Nov 18 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Another small update.
I got rid of the heapClearing overhead.
By using ddmds allocator.
Because of that I was able to shave off 20ms overhead.



 Nov 19 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 19 November 2016 at 09:42:33 UTC, Stefan Koch wrote:

 Another small update.
 I got rid of the heapClearing overhead.
 By using ddmds allocator.
 Because of that I was able to shave off 20ms overhead.


Allocation of dynamic arrays works :)
The following code will work just fine with the new engine.
uint[] repeat(uint[] a, uint cnt)
{
   uint[] result = [];
   uint pos;
   result.length = cast(uint) (a.length * cnt);
   while(cnt--) foreach(c;a)
   {
     result[pos++] = c;
   }

   return result;
}

static assert(repeat([1,2,3,4], 4) ==
   [1,2,3,4, 1,2,3,4, 1,2,3,4, 1,2,3,4]
);



 Nov 19 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 19 November 2016 at 11:22:18 UTC, Stefan Koch wrote:

 On Saturday, 19 November 2016 at 09:42:33 UTC, Stefan Koch 
 wrote:

 Another small update.
 I got rid of the heapClearing overhead.
 By using ddmds allocator.
 Because of that I was able to shave off 20ms overhead.


 Allocation of dynamic arrays works :)
 The following code will work just fine with the new engine.
 uint[] repeat(uint[] a, uint cnt)
 {
   uint[] result = [];
   uint pos;
   result.length = cast(uint) (a.length * cnt);
   while(cnt--) foreach(c;a)
   {
     result[pos++] = c;
   }

   return result;
 }

 static assert(repeat([1,2,3,4], 4) ==
   [1,2,3,4, 1,2,3,4, 1,2,3,4, 1,2,3,4]
 );


There is an important change coming up.
I am going to rewrite the 64 bit support.

The after the change the interpreter will be slower when working 
with 64bit values.
However it will be much easier for the jit.

Also I have a lead on howto fix the bugs in break and continue 
handling.

A nice weekend to all of you!



 Nov 20 2016



 next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 20 November 2016 at 09:02:30 UTC, Stefan Koch wrote:

 On Saturday, 19 November 2016 at 11:22:18 UTC, Stefan Koch 
 wrote:

 On Saturday, 19 November 2016 at 09:42:33 UTC, Stefan Koch 
 wrote:

 Another small update.
 I got rid of the heapClearing overhead.
 By using ddmds allocator.
 Because of that I was able to shave off 20ms overhead.


 Allocation of dynamic arrays works :)
 The following code will work just fine with the new engine.
 uint[] repeat(uint[] a, uint cnt)
 {
   uint[] result = [];
   uint pos;
   result.length = cast(uint) (a.length * cnt);
   while(cnt--) foreach(c;a)
   {
     result[pos++] = c;
   }

   return result;
 }

 static assert(repeat([1,2,3,4], 4) ==
   [1,2,3,4, 1,2,3,4, 1,2,3,4, 1,2,3,4]
 );


 There is an important change coming up.
 I am going to rewrite the 64 bit support.

 The after the change the interpreter will be slower when 
 working with 64bit values.
 However it will be much easier for the jit.

 Also I have a lead on howto fix the bugs in break and continue 
 handling.

 A nice weekend to all of you!


I discovered a new bug in the handling of do-while statements.
Which I subsequently fixed.

I also fixed the C backend.
It would interpret a certain conditional different from all other 
back-ends.

This makes it oblivious that better documentation of the code-gen 
interface is needed.
Furthermore I need to extend my bc_tests. to make sure the 
interpretation is the same.
Such that this bug cannot happen again.

As always a happy weekend to all of you.
- I really need a drink ... preferably root-beer.



 Nov 20 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 20 November 2016 at 15:37:50 UTC, Stefan Koch wrote:

 Furthermore I need to extend my bc_tests. to make sure the 
 interpretation is the same.
 Such that this bug cannot happen again.


I have extended my test-suite to test this case.
It is crucial that the c backend and the interpreter back-ends  
do always return the same correct value. As those are both 
testable at CTFE.

I have plans to implement a small CTFEable 
instruction-set-simulator for the intel 486 processor.
That way I can make sure that my x86-backend does what it is 
supposed to do.
However this is further down the road :)



 Nov 20 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 20 November 2016 at 16:27:24 UTC, Stefan Koch wrote:

 On Sunday, 20 November 2016 at 15:37:50 UTC, Stefan Koch wrote:

 Furthermore I need to extend my bc_tests. to make sure the 
 interpretation is the same.
 Such that this bug cannot happen again.


before.
 I have extended my test-suite to test this case.
 It is crucial that the c backend and the interpreter back-ends  
 do always return the same correct value. As those are both 
 testable at CTFE.

 I have plans to implement a small CTFEable 
 instruction-set-simulator for the intel 486 processor.
 That way I can make sure that my x86-backend does what it is 
 supposed to do.
 However this is further down the road :)


A bug in Do-While handling appeared that was hidden by the 
previously incorrect behavior.
When writing do { ... } while(true/false)
The IR generated would tell the interpreter to evaluate the 
condition flag.
However the load of a constant does not modify the condition flag 
and therefore if the loop is executed or not, depends on the 
result of a previous comparison.
The fix is simple check for the (false/true) constants.
(DMD has a shortcut for this, awkwardly named isBool() )
with an even more awkward usage.

if (expression.isBool(true)) { // this expression is a constant 
which evaluates to true }
else if (expression.isBool(false)) {// this expression is a 
constant false
else { // this value is not a constant }



 Nov 21 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 21 November 2016 at 10:51:01 UTC, Stefan Koch wrote:

 When writing do { ... } while(true/false)
 The IR generated would tell the interpreter to evaluate the 
 condition flag.


This Bug is now fixed.
At the same time a superfluous jump could be eliminated.
I have also fixed another bug in the bailout system.
Such that we now bail out when we could not interpret the 
right-hand-side of an arrayLength Assignment.



 Nov 21 2016



prev sibling  parent reply Anonymouse <asdf asdf.net>  writes:


On Sunday, 20 November 2016 at 09:02:30 UTC, Stefan Koch wrote:

 On Saturday, 19 November 2016 at 11:22:18 UTC, Stefan Koch 
 wrote:


[...]

I would buy you a beer but the Internet is in the way.



 Nov 21 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 21 November 2016 at 19:11:58 UTC, Anonymouse wrote:

 On Sunday, 20 November 2016 at 09:02:30 UTC, Stefan Koch wrote:

 On Saturday, 19 November 2016 at 11:22:18 UTC, Stefan Koch 
 wrote:


 [...]

 I would buy you a beer but the Internet is in the way.


I appreciate the offer.

During the next days I will hopefully be able to have the first 
arrays of structures working.
Those are a bit tricky since they require doubly-indirected RMW.



 Nov 21 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 21 November 2016 at 19:19:50 UTC, Stefan Koch wrote:

 On Monday, 21 November 2016 at 19:11:58 UTC, Anonymouse wrote:

 On Sunday, 20 November 2016 at 09:02:30 UTC, Stefan Koch wrote:

 On Saturday, 19 November 2016 at 11:22:18 UTC, Stefan Koch 
 wrote:


 [...]

 I would buy you a beer but the Internet is in the way.


 I appreciate the offer.

 During the next days I will hopefully be able to have the first 
 arrays of structures working.
 Those are a bit tricky since they require doubly-indirected RMW.


Progress is still stalled on this.
It is very important to get pointers and handling of other 
qusi-references right.
There are still known bugs in switch processing, also the 
llvm-backend need to be fixed.



 Nov 24 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


I have just discovered breakage of basic operations.
I will need regression tests, some time soon.



 Nov 24 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 24 November 2016 at 17:16:33 UTC, Stefan Koch wrote:

 I have just discovered breakage of basic operations.
 I will need regression tests, some time soon.


Hey Guys,
I fixed the regression.
(At the expense, of removing more optimization)
Work continues on struct and (dynamic) array support.
Also I am going to fix another regression I just discoverd that 
prevents multiple string-arguments to work properly.
Those regression are seriously annoying.



 Nov 25 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 25 November 2016 at 09:14:16 UTC, Stefan Koch wrote:

 On Thursday, 24 November 2016 at 17:16:33 UTC, Stefan Koch 
 wrote:

 I have just discovered breakage of basic operations.
 I will need regression tests, some time soon.


 Hey Guys,
 I fixed the regression.
 (At the expense, of removing more optimization)
 Work continues on struct and (dynamic) array support.
 Also I am going to fix another regression I just discoverd that 
 prevents multiple string-arguments to work properly.
 Those regression are seriously annoying.


String-Arguments are fixed.
The known regressions are now resolved.



 Nov 25 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 25 November 2016 at 10:53:50 UTC, Stefan Koch wrote:

 On Friday, 25 November 2016 at 09:14:16 UTC, Stefan Koch wrote:

 On Thursday, 24 November 2016 at 17:16:33 UTC, Stefan Koch 
 wrote:

 I have just discovered breakage of basic operations.
 I will need regression tests, some time soon.


 Hey Guys,
 I fixed the regression.
 (At the expense, of removing more optimization)
 Work continues on struct and (dynamic) array support.
 Also I am going to fix another regression I just discoverd 
 that prevents multiple string-arguments to work properly.
 Those regression are seriously annoying.


 String-Arguments are fixed.
 The known regressions are now resolved.


I discovered a nasty bug in goto handling.
I am working on fixed it;
This could take a while, since it likely requires structural 
changes.



 Nov 28 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 28 November 2016 at 17:02:37 UTC, Stefan Koch wrote:

 On Friday, 25 November 2016 at 10:53:50 UTC, Stefan Koch wrote:

 I discovered a nasty bug in goto handling.
 I am working on fixed it;
 This could take a while, since it likely requires structural 
 changes.


I fixed a bug in continue break handling.
The dreaded structural changes for this were easier then expected.
Perhaps fixing the goto-handling will be as straight-forward ?

The following code compiles now:

int bug4448()
{
     int n = 2;
L1:
     do
     {
         switch(n)
         {
         case 5:
             return 7;
         default:
             n = 5;
             break L1;
             return n;
         }
         int w = 7;
     } while (0);
     return 3;
}
static assert(bug4448() == 3);



 Nov 29 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 29 November 2016 at 23:00:08 UTC, Stefan Koch wrote:

 I fixed a bug in continue break handling.


For the record it was an off by one error.
The value for unresolvedGotos would add one referencing jump.
But the count of referencing would be initialized to zero instead 
of one.



 Nov 29 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 29 November 2016 at 23:09:08 UTC, Stefan Koch wrote:

 On Tuesday, 29 November 2016 at 23:00:08 UTC, Stefan Koch wrote:

 I fixed a bug in continue break handling.


 For the record it was an off by one error.
 The value for unresolvedGotos would add one referencing jump.
 But the count of referencing would be initialized to zero 
 instead of one.


I just added support for assert in newCTFE.
Also I noticed a performance regression in the bytecode generator.
(compared to an earlier version that handled much less)

Nontheless I get a solid 4x performance improvement over the old 
evaluator when compiled with ldc.

Furthermore struct support regressed.
It has to be rewritten anyway in order to support method-calls 
properly.



 Nov 30 2016



 next sibling parent reply Andrea Fontana <nospam example.com>  writes:


On Wednesday, 30 November 2016 at 13:49:01 UTC, Stefan Koch wrote:

 On Tuesday, 29 November 2016 at 23:09:08 UTC, Stefan Koch wrote:

 On Tuesday, 29 November 2016 at 23:00:08 UTC, Stefan Koch 
 wrote:

 I fixed a bug in continue break handling.


 For the record it was an off by one error.
 The value for unresolvedGotos would add one referencing jump.
 But the count of referencing would be initialized to zero 
 instead of one.


 I just added support for assert in newCTFE.
 Also I noticed a performance regression in the bytecode 
 generator.
 (compared to an earlier version that handled much less)

 Nontheless I get a solid 4x performance improvement over the 
 old evaluator when compiled with ldc.

 Furthermore struct support regressed.
 It has to be rewritten anyway in order to support method-calls 
 properly.


Every now and then it would be interesting to have a list of 
things that work and a list of things still to do.

Keep up the good work!



 Nov 30 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 30 November 2016 at 15:02:21 UTC, Andrea Fontana 
wrote:


 Every now and then it would be interesting to have a list of 
 things that work and a list of things still to do.


That list changes. On both ends.  I constantly discover things 
that I thought were dealt with are indeed still work in progress.

I can give a rough overview in a few day when things have 
(Hopefully!) stabilized a little more.


 Keep up the good work!


I will!



 Nov 30 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


I just fixed  a bug cause the non-debug build to fail on a 
expression that it should have bailed out on.

Currently I have troubles with this  code :

int fn(int y)
{
   int x = void;
   ++x;
   return x+y;
}

Since x is uninitialized the ctfe interpreter should fail on it.
I will need to reserve a certain value as the uninitialized flag.
since stackAddress zero can never be used it'd be perfect for the 
job.
This bug should be fixed by tomorrow.



 Dec 01 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 1 December 2016 at 17:52:02 UTC, Stefan Koch wrote:

 I just fixed  a bug cause the non-debug build to fail on a 
 expression that it should have bailed out on.

 Currently I have troubles with this  code :

 int fn(int y)
 {
   int x = void;
   ++x;
   return x+y;
 }

 Since x is uninitialized the ctfe interpreter should fail on it.
 I will need to reserve a certain value as the uninitialized 
 flag.
 since stackAddress zero can never be used it'd be perfect for 
 the job.
 This bug should be fixed by tomorrow.


I opted for bailing out as soon as void initialization is in the 
function.
This will need to handled properly at some point but now is not 
the time.



 Dec 02 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Currently I am working on the ability of returning structs.
Although those are "merely" heterogeneous arrays.
This is a very important step towards having a usable 
CTFE-reimplementation.

the regressed struct support.
seems to be restored. (Although I have no idea why)
Once this is done the more complex composite types have to be fit 
in the current model.

I have a very flat representation of values.
They are treated almost exactly as run-time values.
Which means that the ctfe engine itself needs a well defined abi.
Since I was aware of this, I have already put alot of thought 
into this.
However the recent onslaught of bugs have shown that I can not 
assume that my code is bug-free.
I expect a few hard issues in the memory-management and 
ABI-dependent code.

I will keep you updated,
Cheers,
Stefan



 Dec 02 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 3 December 2016 at 02:07:06 UTC, Stefan Koch wrote:

 Currently I am working on the ability of returning structs.

 I will keep you updated,


This is coming along rather nicely.
StructLiterals as Arguments regressed.
But they did rather dubious stuff in the first place.

So supported types are.

StringLiterals.
Integral Array Literals.
Dynamic int[] and uint[] Arrays.
Structs with int or uint members.
and
StructLiterals with int/uint members (regressed currently.)

64bit integrals (long) do work but only if the interpreter 
backend is used.

Supported operations.
All basic math. (+ - * / %)
...++ ...-- +++... --...
while(...) {...}
do ... while(...)
label :
goto label;
break;
break label;
continue;
continue label;
for(...;...;...)
... ? ... : ...
switch (...) { case ... , ... } (unreliable due to differences in 
case-sorting :( )
for-each on strings and arrays.
Assigning to array length. (currently without copying on 
reallocation :( )
Bounds-Checked :
Array-Indexing
Left-Shift Right-Shift. (always unsigned! :( )

Unsupported :
&& ||
classes (unlikely to be added in the near future)
function calls (plain and closure)
slices
pointers
static arrays.

floating point math (unlikely to be added in the near future)



 Dec 03 2016



 next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 3 December 2016 at 18:00:10 UTC, Stefan Koch wrote:

 On Saturday, 3 December 2016 at 02:07:06 UTC, Stefan Koch wrote:

 [...]


 switch (...) { case ... , ... } (unreliable due to differences 
 in case-sorting :( )
 for-each on strings and arrays.


if (...) { ... } else { ... } (if(__ctfe) && if(!__ctfe) are 
special cased and non-ctfe code is removed)
assert(..., "...")

 Assigning to array length. (currently without copying on 
 reallocation :( )
 Bounds-Checked :
 Array-Indexing
 Left-Shift Right-Shift. (always unsigned! :( )

 [...]


Forgot to add assert, I forget that I implemented it a few days 
ago.



 Dec 03 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 3 December 2016 at 18:16:21 UTC, Stefan Koch wrote:

 [...]


StructLiteral regressions fixed.
That one was hard to find.
Basically the for Array arguments interfered with this one.
Arguments are technically the same expression as everything else.
But in order to support targets that do have native function 
calls,
and in order to archive more performance for the interpreter they 
have to be treated quite differently from non-argument 
expressions.

For now they should work, but it's brittle at best.



 Dec 03 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 3 December 2016 at 19:28:52 UTC, Stefan Koch wrote:

 On Saturday, 3 December 2016 at 18:16:21 UTC, Stefan Koch wrote:

 [...]


 StructLiteral regressions fixed.
 That one was hard to find.
 Basically the for Array arguments interfered with this one.
 Arguments are technically the same expression as everything 
 else.
 But in order to support targets that do have native function 
 calls,
 and in order to archive more performance for the interpreter 
 they have to be treated quite differently from non-argument 
 expressions.

 For now they should work, but it's brittle at best.


The following code now compiles with newCTFE:
struct S
{
  uint u1;
  uint u2;
}

S makeTenTen()
{
   return S(10, 10);
}

S makeS(uint a, uint b)
{
   return S(a, b);
}

uint getu2(S s)
{
   return s.u2;
}

static assert(getu2(S(10, 14)) == 14);
static assert(makeTenTen() == makeS(10, 10));



 Dec 03 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Another bug in continue break handling has appeared.

uint fn() {
   for (;;)
   {
     if (true) break;
     return 0;
   }
   return 1;
}

static assert(fn());

The above code is enough to trigger it.
Apparently the fixup jump for the break is never inserted.
I doubt that this is an off-by-one error again.

It's going to be tricky.



 Dec 03 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 3 December 2016 at 23:30:58 UTC, Stefan Koch wrote:

 Another bug in continue break handling has appeared.

 uint fn() {
   for (;;)
   {
     if (true) break;
     return 0;
   }
   return 1;
 }

 static assert(fn());

 The above code is enough to trigger it.
 Apparently the fixup jump for the break is never inserted.
 I doubt that this is an off-by-one error again.

 It's going to be tricky.


I figured out part of the reason for this bug.
baically it's like this.



 uint fn() {
   for (;;)


beginBlock:
//causes and unconditonal jump (UC1) after the end of the block

   {
     if (true) break;


// causes an unconditional jump(UC2) here to the end of the block
UC2: goto endBlock;

     return 0;


endBlock:
UC1: goto BeginBlock;

   }
   return 1;
 }





 Dec 03 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 4 December 2016 at 00:14:26 UTC, Stefan Koch wrote:

 On Saturday, 3 December 2016 at 23:30:58 UTC, Stefan Koch wrote:

 Another bug in continue break handling has appeared.

 uint fn() {
   for (;;)
   {
     if (true) break;
     return 0;
   }
   return 1;
 }

 static assert(fn());

 The above code is enough to trigger it.
 Apparently the fixup jump for the break is never inserted.
 I doubt that this is an off-by-one error again.

 It's going to be tricky.


 I figured out part of the reason for this bug.
 baically it's like this.



 uint fn() {
   for (;;)


 beginBlock:
 //causes and unconditonal jump (UC1) after the end of the block

   {
     if (true) break;


 // causes an unconditional jump(UC2) here to the end of the 
 block
 UC2: goto endBlock;

     return 0;


 endBlock:
 UC1: goto BeginBlock;

   }
   return 1;
 }




This can be fixed by setting a state-flag whenever we are going 
to process the body of an infinite loop.
If anyone tried at any point to break out of the loop we need to 
introduce a conditional that becomes false and allows us to exit.
It's ugly but it will work regardless of nesting-level .... I 
hope.



 Dec 03 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 4 December 2016 at 00:23:32 UTC, Stefan Koch wrote:

 On Sunday, 4 December 2016 at 00:14:26 UTC, Stefan Koch wrote:

 On Saturday, 3 December 2016 at 23:30:58 UTC, Stefan Koch 
 wrote:

 [...]


 I figured out part of the reason for this bug.
 baically it's like this.



   [...]


 beginBlock:
 //causes and unconditonal jump (UC1) after the end of the block

     [...]


 // causes an unconditional jump(UC2) here to the end of the 
 block
 UC2: goto endBlock;

     [...]


 endBlock:
 UC1: goto BeginBlock;

 [...]






Some progress had been made on this.
There is still a little hiccup with (true) and (false) conditions 
but those can be eliminated soon.
For the most part breaking out of infinite loops works now.



 Dec 04 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


I just had a hunch about the continue statement within for-loops 
being treated incorrectly.

let's see where it leads.

Also the fix for infinite loops has apparently cost us a lot of 
performance.
I am not sure where this loss is coming from.
Maybe I am just reading my scales wrong ;)

While those codegen remain new features and documentation have 
reduced priority.

I am doing a screening regarding the structural strength of the 
architecture.

I would be quite happy to read any thoughts on that matter.



 Dec 04 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 4 December 2016 at 23:10:34 UTC, Stefan Koch wrote:

 Also the fix for infinite loops has apparently cost us a lot of 
 performance.
 I am not sure where this loss is coming from.
 Maybe I am just reading my scales wrong ;)


it seemed to have t odo with the hardware being particularly busy 
at the point of measurement.

I fixed a bug in ifStatements. so now all the constant expression 
trouble should be resolved.



 Dec 04 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


I just improved the handling of void initializations.
Now the code is less pessimistic and will allow them if they are 
assigned to before use.
However using void variables at ctfe will not result in any 
performance wins.



 Dec 04 2016



 next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 5 December 2016 at 04:26:35 UTC, Stefan Koch wrote:

 I just improved the handling of void initializations.
 Now the code is less pessimistic and will allow them if they 
 are assigned to before use.
 However using void variables at ctfe will not result in any 
 performance wins.


I found an easily fixable performance problem inside the 
byte-code generator.
Causing it to allocate 800K per discovery of a new type.
Reducing this will probably make IR generation 10 times faster in 
the average case.
Clearing 800K takes quite some time.

Furthermore I will revise some design decisions inside the IR 
generator.
Those are related to how break and continue are handled.
Because continue is still broken.

Hopefully I can fix this without spreading the responsible code 
around everywhere :)



 Dec 04 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 5 December 2016 at 07:48:31 UTC, Stefan Koch wrote:

 I found an easily fixable performance problem inside the 
 byte-code generator.
 Causing it to allocate 800K per discovery of a new type.
 Reducing this will probably make IR generation 10 times faster 
 in the average case.
 Clearing 800K takes quite some time.


I just fixed this.
As predicted, the taken time to generate byte-code is now greatly 
reduced.
This has a really huge impact.
It looks like the performance wins brought by the new ctfe engine 
might be higher then I predicted.



 Dec 05 2016



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 12/05/2016 11:28 AM, Stefan Koch wrote:

 It looks like the performance wins brought by the new ctfe engine might
 be higher then I predicted.


That's awesome!! -- Andrei



 Dec 05 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 5 December 2016 at 16:47:33 UTC, Andrei Alexandrescu 
wrote:

 On 12/05/2016 11:28 AM, Stefan Koch wrote:

 It looks like the performance wins brought by the new ctfe 
 engine might
 be higher then I predicted.


 That's awesome!! -- Andrei


After discovering this performance bottleneck I have now changed 
my mind about how I will tackle concatenation of arrays and 
strings in particular.
All con-cat operations should be done as intrinsic calls.
Because tight cooperation with the 
CTFE-Memory-Management-Subsystem is needed.



 Dec 05 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 5 December 2016 at 18:47:13 UTC, Stefan Koch wrote:

 On Monday, 5 December 2016 at 16:47:33 UTC, Andrei Alexandrescu 
 wrote:

 On 12/05/2016 11:28 AM, Stefan Koch wrote:

 It looks like the performance wins brought by the new ctfe 
 engine might
 be higher then I predicted.


 That's awesome!! -- Andrei


 After discovering this performance bottleneck I have now 
 changed my mind about how I will tackle concatenation of arrays 
 and strings in particular.
 All con-cat operations should be done as intrinsic calls.
 Because tight cooperation with the 
 CTFE-Memory-Management-Subsystem is needed.


I just implemented a bytecode cache, however the bytecode 
generation is so fast now that we only safe a couple micro 
seconds by using the cache.
It is not noticeable at all :P.



 Dec 06 2016



 next sibling parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 12/06/2016 11:27 AM, Stefan Koch wrote:

 On Monday, 5 December 2016 at 18:47:13 UTC, Stefan Koch wrote:

 On Monday, 5 December 2016 at 16:47:33 UTC, Andrei Alexandrescu wrote:

 On 12/05/2016 11:28 AM, Stefan Koch wrote:

 It looks like the performance wins brought by the new ctfe engine might
 be higher then I predicted.


 That's awesome!! -- Andrei


 After discovering this performance bottleneck I have now changed my
 mind about how I will tackle concatenation of arrays and strings in
 particular.
 All con-cat operations should be done as intrinsic calls.
 Because tight cooperation with the CTFE-Memory-Management-Subsystem is
 needed.


 I just implemented a bytecode cache, however the bytecode generation is
 so fast now that we only safe a couple micro seconds by using the cache.
 It is not noticeable at all :P.


Just give us time :o). -- Andrei



 Dec 06 2016



prev sibling  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 6 December 2016 at 16:27:38 UTC, Stefan Koch wrote:

 I just implemented a bytecode cache, however the bytecode 
 generation is so fast now that we only safe a couple micro 
 seconds by using the cache.
 It is not noticeable at all :P.


I can't wait to try this out.



 Dec 06 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 6 December 2016 at 21:40:47 UTC, Nordlöw wrote:

 On Tuesday, 6 December 2016 at 16:27:38 UTC, Stefan Koch wrote:

 I just implemented a bytecode cache, however the bytecode 
 generation is so fast now that we only safe a couple micro 
 seconds by using the cache.
 It is not noticeable at all :P.


 I can't wait to try this out.


Go ahead.
Many features should already be working correctly.

The missing important ones are slices and concat.



 Dec 06 2016



 next sibling parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 6 December 2016 at 22:21:59 UTC, Stefan Koch wrote:

 Go ahead.
 Many features should already be working correctly.


This https://github.com/UplinkCoder/dmd/commits/newCTFE I presume?



 Dec 06 2016



prev sibling  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 6 December 2016 at 22:21:59 UTC, Stefan Koch wrote:

 Go ahead.
 Many features should already be working correctly.

 The missing important ones are slices and concat.


Ok, great.

I was thinking about benchmarking parser generators such as 
Pegged.

Is there another way of concatenating strings than with the 
builtin ~ operator?



 Dec 06 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 6 December 2016 at 22:44:34 UTC, Nordlöw wrote:

 On Tuesday, 6 December 2016 at 22:21:59 UTC, Stefan Koch wrote:

 Go ahead.
 Many features should already be working correctly.

 The missing important ones are slices and concat.


 Ok, great.

 I was thinking about benchmarking parser generators such as 
 Pegged.

 Is there another way of concatenating strings than with the 
 builtin ~ operator?


Yes there is.
Create a char[] give it the length of string a + string b;
copy string a into the char[]
then copy string b into the resulting array.
offsetting it by the length of string a.



 Dec 06 2016



 next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 6 December 2016 at 23:16:16 UTC, Stefan Koch wrote:

 On Tuesday, 6 December 2016 at 22:44:34 UTC, Nordlöw wrote:

 On Tuesday, 6 December 2016 at 22:21:59 UTC, Stefan Koch wrote:

 Go ahead.
 Many features should already be working correctly.

 The missing important ones are slices and concat.


 Ok, great.

 I was thinking about benchmarking parser generators such as 
 Pegged.

 Is there another way of concatenating strings than with the 
 builtin ~ operator?


 Yes there is.
 Create a char[] give it the length of string a + string b;
 copy string a into the char[]
 then copy string b into the resulting array.
 offsetting it by the length of string a.


But keep in mind function calls are unsupported right now.
Meaning you have to put everything in one function.
However function calls should be working some time soon.
I am still in the middle of restructuring how blocks or handled, 
because right now I will miscompile for-loops with continues in 
them.



 Dec 06 2016



prev sibling next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 6 December 2016 at 23:16:16 UTC, Stefan Koch wrote:

 Yes there is.
 Create a char[] give it the length of string a + string b;
 copy string a into the char[]
 then copy string b into the resulting array.
 offsetting it by the length of string a.


Sounds like you have your concat implementation right there ;)



 Dec 06 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 6 December 2016 at 23:55:06 UTC, Nordlöw wrote:

 On Tuesday, 6 December 2016 at 23:16:16 UTC, Stefan Koch wrote:

 Yes there is.
 Create a char[] give it the length of string a + string b;
 copy string a into the char[]
 then copy string b into the resulting array.
 offsetting it by the length of string a.


 Sounds like you have your concat implementation right there ;)


not really, concat needs to be smart about allocation and memory 
reuse.
when used on char arrays the memory access will be different.
and use more instructions.



 Dec 06 2016



prev sibling  parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 6 December 2016 at 23:16:16 UTC, Stefan Koch wrote:

 Yes there is...


Thanks.



 Dec 06 2016



prev sibling next sibling parent reply deadalnix <deadalnix gmail.com>  writes:


On Monday, 5 December 2016 at 04:26:35 UTC, Stefan Koch wrote:

 I just improved the handling of void initializations.
 Now the code is less pessimistic and will allow them if they 
 are assigned to before use.
 However using void variables at ctfe will not result in any 
 performance wins.


Void initialization are allowed at CTFE ?



 Dec 04 2016



 next sibling parent reply ketmar <ketmar ketmar.no-ip.org>  writes:


On Monday, 5 December 2016 at 07:55:32 UTC, deadalnix wrote:

 On Monday, 5 December 2016 at 04:26:35 UTC, Stefan Koch wrote:

 I just improved the handling of void initializations.
 Now the code is less pessimistic and will allow them if they 
 are assigned to before use.
 However using void variables at ctfe will not result in any 
 performance wins.


 Void initialization are allowed at CTFE ?


not now, but it looks like needless limitation. any void 
initialization can be converted to "fill the things with zeroes" 
in CTFE engine.



 Dec 05 2016



  parent  Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com>  writes:


On Monday, 5 December 2016 at 08:07:11 UTC, ketmar wrote:

 On Monday, 5 December 2016 at 07:55:32 UTC, deadalnix wrote:

 On Monday, 5 December 2016 at 04:26:35 UTC, Stefan Koch wrote:

 I just improved the handling of void initializations.
 Now the code is less pessimistic and will allow them if they 
 are assigned to before use.
 However using void variables at ctfe will not result in any 
 performance wins.


 Void initialization are allowed at CTFE ?


 not now, but it looks like needless limitation. any void 
 initialization can be converted to "fill the things with 
 zeroes" in CTFE engine.


On the contrary. If something is NOT initialized, it uses its 
default (e.g. zeroes), but if it is =void it should be an error 
to use it before any assignment.



 Dec 05 2016



prev sibling  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 5 December 2016 at 07:55:32 UTC, deadalnix wrote:

 On Monday, 5 December 2016 at 04:26:35 UTC, Stefan Koch wrote:

 I just improved the handling of void initializations.
 Now the code is less pessimistic and will allow them if they 
 are assigned to before use.
 However using void variables at ctfe will not result in any 
 performance wins.


 Void initialization are allowed at CTFE ?


the following code will compile just fine.
uint fn(uint a) {
  int b = void;
  if (a == 2)
  {
    b = 1;
  }

  return b; // only fine is a was 2;
}

static assert(fn(2) == 1);



 Dec 05 2016



prev sibling  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 5 December 2016 at 04:26:35 UTC, Stefan Koch wrote:

 I just improved the handling of void initializations.
 Now the code is less pessimistic and will allow them if they 
 are assigned to before use.
 However using void variables at ctfe will not result in any 
 performance wins.


Oh it's broken.
It does not detect the ... maybe not void case.

I found out why certain switches are broken.
Essentially this is fixable by putting a guarding jump around the 
braces that holds the cases.
Which is also the reason why code in a switch that does not 
belong to any case is unreachable code :)



 Dec 05 2016



prev sibling  parent reply Andrea Fontana <nospam example.com>  writes:


On Saturday, 3 December 2016 at 18:00:10 UTC, Stefan Koch wrote:

 classes (unlikely to be added in the near future)
 floating point math (unlikely to be added in the near future)


What's the problem with classes and floating point math?

Andrea



 Dec 07 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 7 December 2016 at 08:33:22 UTC, Andrea Fontana 
wrote:

 On Saturday, 3 December 2016 at 18:00:10 UTC, Stefan Koch wrote:

 classes (unlikely to be added in the near future)
 floating point math (unlikely to be added in the near future)


 What's the problem with classes and floating point math?

 Andrea


Honestly the main problem is that I am not familiar with 
FloatingPoint-Math.
Therefore I would may not even notice when I introduce a subtle 
bug.

Classes build on top of a functioning memory-management and 
require a mechanism for indirect calls or function pointers.
Both are currently not implemented.

I will worry about classes when the rest of the engine runs 
cleanly.



 Dec 07 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


I have an update about ctfe performance.
I wondered for quite a while why newCTFE had a 5 millisecond 
overhead, when compared the old interpreter.
Since on my charts it had comparable or better stats.
I finally figured it out.

Because the interpreter is supposed to be ctfe-able itself it 
uses a gc-allocated stack.
And that in turn call envokes the gc.
The mark phase of the garbage-collector matches up exactly with 
the extra time taken.

I have no idea, why this happens though since the GC is supposed 
to be disabled.



 Dec 07 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 7 December 2016 at 18:00:40 UTC, Stefan Koch wrote:

 I have an update about ctfe performance.
 I wondered for quite a while why newCTFE had a 5 millisecond 
 overhead, when compared the old interpreter.
 Since on my charts it had comparable or better stats.
 I finally figured it out.

 Because the interpreter is supposed to be ctfe-able itself it 
 uses a gc-allocated stack.
 And that in turn call envokes the gc.
 The mark phase of the garbage-collector matches up exactly with 
 the extra time taken.

 I have no idea, why this happens though since the GC is 
 supposed to be disabled.


It turns out that almost classes in dmd that are em-placed.
Whereas by bytecode visitor uses the regular new, because it is 
fairly sizable. The gc is much more visible.

Although that does still not explain why GC.disable seems to have 
no effect.



 Dec 07 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 7 December 2016 at 18:50:20 UTC, Stefan Koch wrote:

 On Wednesday, 7 December 2016 at 18:00:40 UTC, Stefan Koch 
 wrote:

 I have an update about ctfe performance.
 I wondered for quite a while why newCTFE had a 5 millisecond 
 overhead, when compared the old interpreter.
 Since on my charts it had comparable or better stats.
 I finally figured it out.

 Because the interpreter is supposed to be ctfe-able itself it 
 uses a gc-allocated stack.
 And that in turn call envokes the gc.
 The mark phase of the garbage-collector matches up exactly 
 with the extra time taken.

 I have no idea, why this happens though since the GC is 
 supposed to be disabled.


 It turns out that almost classes in dmd that are em-placed.
 Whereas by bytecode visitor uses the regular new, because it is 
 fairly sizable. The gc is much more visible.

 Although that does still not explain why GC.disable seems to 
 have no effect.


After a few more iterations the observed slow down seems to 
disappear.
I am suspicious.



 Dec 07 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


I did the change in continue and break handling.

Now ForStatements should be handled correctly for the most part.
I think there could still be a bug if you do labeled continue on 
a for-statement.
I will do a change later that takes that into account as well.

Then we should be all set to implement call handling I believe.



 Dec 07 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 7 December 2016 at 20:56:08 UTC, Stefan Koch wrote:

 I did the change in continue and break handling.

 Now ForStatements should be handled correctly for the most part.
 I think there could still be a bug if you do labeled continue 
 on a for-statement.
 I will do a change later that takes that into account as well.

 Then we should be all set to implement call handling I believe.


Array-literals with ctfe-known Variables in them are now 
supported;
uint[] fn(uint n)
{
   return [n, n, n];
}

static assert(fn(3).length == 3);
static assert(fn(3)[0] == 3);
static assert(fn(3)[1] == 3);
static assert(fn(3)[2] == 3);



 Dec 07 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 8 December 2016 at 01:52:13 UTC, Stefan Koch wrote:

 Array-literals with ctfe-known Variables in them are now 
 supported;
 uint[] fn(uint n)
 {
   return [n, n, n];
 }

 static assert(fn(3).length == 3);
 static assert(fn(3)[0] == 3);
 static assert(fn(3)[1] == 3);
 static assert(fn(3)[2] == 3);


This works as well with interspersed constant.
uint[] itrspary(uint n)
{
   return [1, n, 3];
}

static assert(itrspary(7).length == 3);
static assert(itrspary(7)[0] == 1);
static assert(itrspary(7)[1] == 7);
static assert(itrspary(7)[2] == 3);



 Dec 07 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 8 December 2016 at 02:22:31 UTC, Stefan Koch wrote:

 On Thursday, 8 December 2016 at 01:52:13 UTC, Stefan Koch wrote:

 Array-literals with ctfe-known Variables in them are now 
 supported;
 uint[] fn(uint n)
 {
   return [n, n, n];
 }

 static assert(fn(3).length == 3);
 static assert(fn(3)[0] == 3);
 static assert(fn(3)[1] == 3);
 static assert(fn(3)[2] == 3);


 This works as well with interspersed constant.
 uint[] itrspary(uint n)
 {
   return [1, n, 3];
 }

 static assert(itrspary(7).length == 3);
 static assert(itrspary(7)[0] == 1);
 static assert(itrspary(7)[1] == 7);
 static assert(itrspary(7)[2] == 3);


I fixed the bug in switchStatements (I think.).
It was not a mismatch with dmd's sorting but rather.
My own sloppy block-exit detection.
Using the dmd BlockExit visitor fixed it.



 Dec 07 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


I found the biggest performance bottleneck in newCTFE!

oldCtfe :

testStringLength.d testStruct.d testMultipleArrayLiterals.d

real	0m0.026s
user	0m0.020s
sys	0m0.003s


testStringLength.d testStruct.d testMultipleArrayLiterals.d 
-bc-ctfe

real	0m0.025s
user	0m0.020s
sys	0m0.003s

After Fixing


testStringLength.d testStruct.d testMultipleArrayLiterals.d  
-bc-ctfe

real	0m0.019s
user	0m0.017s
sys	0m0.000s



 Dec 08 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 8 December 2016 at 19:13:23 UTC, Stefan Koch wrote:

 I found the biggest performance bottleneck in newCTFE!

 oldCtfe :

 testStringLength.d testStruct.d testMultipleArrayLiterals.d

 real	0m0.026s
 user	0m0.020s
 sys	0m0.003s


Before Fixing :


 testStringLength.d testStruct.d testMultipleArrayLiterals.d 
 -bc-ctfe

 real	0m0.025s
 user	0m0.020s
 sys	0m0.003s

 After Fixing


 testStringLength.d testStruct.d testMultipleArrayLiterals.d  
 -bc-ctfe

 real	0m0.019s
 user	0m0.017s
 sys	0m0.000s


Please note that the ctfe interpretation, with newCTFE, inside 
the frontend takes 10% of the compilation time
Whereas with the old interpreter it takes 50%



 Dec 08 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


I just wanted to post another performance comparision that does 
not test dmd's memory allocator more then anything else :)


-bc-ctfe
2147385345u
536821761u
4294639619u

real	0m0.114s
user	0m0.110s
sys	0m0.003s

2147385345u
536821761u
4294639619u

real	0m0.921s
user	0m0.843s
sys	0m0.077s

Results are obtained running the following code
uint MakeInitAndSumArray(uint length)
{
     uint result;
     uint[] arr;
     arr.length = length;

     while(length--)
     {
         arr[length] = length;
     }
     foreach(e;arr)
     {
       result += e;
     }

     return result;
}

pragma(msg, MakeInitAndSumArray(ushort.max));
pragma(msg, MakeInitAndSumArray(ushort.max/2));
pragma(msg, MakeInitAndSumArray(ushort.max*2));



 Dec 08 2016



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 12/08/2016 04:24 PM, Stefan Koch wrote:

 Results are obtained running the following code
 uint MakeInitAndSumArray(uint length)
 {
     uint result;
     uint[] arr;
     arr.length = length;

     while(length--)
     {
         arr[length] = length;
     }
     foreach(e;arr)
     {
       result += e;
     }

     return result;
 }

 pragma(msg, MakeInitAndSumArray(ushort.max));
 pragma(msg, MakeInitAndSumArray(ushort.max/2));
 pragma(msg, MakeInitAndSumArray(ushort.max*2));


That's pretty awesome. The new CTFE engine release will soon rival Tesla 
Model 3 in terms of interest raised :o).

In addition to artificial corpora like the above, I suggest comparing 
against real-world code - starting with simpler things such as bitfields 
instantiations.


Andrei



 Dec 08 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 9 December 2016 at 02:10:58 UTC, Andrei Alexandrescu 
wrote:

 On 12/08/2016 04:24 PM, Stefan Koch wrote:

 Results are obtained running the following code
 uint MakeInitAndSumArray(uint length)
 {
     uint result;
     uint[] arr;
     arr.length = length;

     while(length--)
     {
         arr[length] = length;
     }
     foreach(e;arr)
     {
       result += e;
     }

     return result;
 }

 pragma(msg, MakeInitAndSumArray(ushort.max));
 pragma(msg, MakeInitAndSumArray(ushort.max/2));
 pragma(msg, MakeInitAndSumArray(ushort.max*2));


 That's pretty awesome. The new CTFE engine release will soon 
 rival Tesla Model 3 in terms of interest raised :o).

 In addition to artificial corpora like the above, I suggest 
 comparing against real-world code - starting with simpler 
 things such as bitfields instantiations.


 Andrei


That will not work currently.
Since function and methods calls are still "in the works".
They will require another restructuring as well.
Therefore artificial tests are the best I can do to gunge 
performance.



 Dec 08 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 9 December 2016 at 03:06:49 UTC, Stefan Koch wrote:

 On Friday, 9 December 2016 at 02:10:58 UTC, Andrei Alexandrescu 
 wrote:

 That's pretty awesome. The new CTFE engine release will soon 
 rival Tesla Model 3 in terms of interest raised :o).

 In addition to artificial corpora like the above, I suggest 
 comparing against real-world code - starting with simpler 
 things such as bitfields instantiations.


 Andrei


 That will not work currently.
 Since function and methods calls are still "in the works".
 They will require another restructuring as well.
 Therefore artificial tests are the best I can do to gunge 
 performance.


Progress has been made on this front.
However I discovered that some math is translated incorrectly.
This means I have to fix the code-gen again.
So you'll have to wait a bit until this works :).



 Dec 09 2016



prev sibling  parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Wednesday, 30 November 2016 at 13:49:01 UTC, Stefan Koch wrote:

 I just added support for assert in newCTFE.


Cool.


 Nontheless I get a solid 4x performance improvement over the 
 old evaluator when compiled with ldc.


Wow.


 Furthermore struct support regressed.
 It has to be rewritten anyway in order to support method-calls 
 properly.


Keep up!



 Nov 30 2016



prev sibling  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 11/08/2016 02:11 PM, Antonio Corbi wrote:

 Maybe this one is useful for you:

 http://eagain.net/articles/git-for-computer-scientists/


On 11/08/2016 03:01 PM, H. S. Teoh via Digitalmars-d wrote:

 Nothing immediately comes to mind, but thanks to Dr. Google, I found
 this page that's sorta helpful:

 	http://ericsink.com/vcbe/html/directed_acyclic_graphs.html

 And perhaps these:

 	http://eagain.net/articles/git-for-computer-scientists/
 	http://marklodato.github.io/visual-git-guide/index-en.html


Ok, so it looks like each node in the DAG is a commit. I'll definitely 
have to read further. Thanks, both.

Although I have my doubts it would explain all the issues I've hit upon 
with git's CLI. For example: I don't see why annotated tags aren't the 
default. Or why non-annotated ones even exist at all. When I made 
<https://github.com/Abscissa/gen-package-version>, I noticed that it'll 
*only* work if the version tags are "annotated" tags. Because otherwise 
"git describe" doesn't retrieve the tag. Doesn't even matter one bit 
*what* the tag's message is, just that it exists. Why? Who knows! It's git!

So it means that unlike hg users, who can just make their tags with "hg 
tag v1.2.0" and everything's fine, git users can't just say "git tag 
v1.2.0". In git, making a tag like that will work for SOME use-cases but 
not all. Instead, git users have to use the less-obvious "git tag -a 
v1.2.0 -m '...whatever...'" incantation or else "git describe" won't 
freaking work right, therefore gen-package-version won't be able to 
determine the working copy's current tag name.

So then why do the non-annotated tags even exist? Why not just have 
"annotated" tags with *optional* messages? Why even have the "-a" flag 
for "git tag" at all? Why not have "git tag tagname" make a tag with no 
message, and "git tag tagname -m '...'" to make a tag with a message, 
and ditch the whole "-a" nonsense? Because it wouldn't be stupid enough 
that way? I don't get it.

Its things like that. I'd be surprised if that has much to do with git's 
nature as a "dumb" DAG tool. It's just the general good-design principle 
of "The thing you *want to* or *should* do or *expect* 99% of the time 
should be the DEFAULT, not the obscure incantation".



 Or maybe I should write my own article about git being based on a DAG.
 :-P


You should! :)

I've given a lot of thought over the years to the concepts of learning 
and teaching. As I see it, the big key to both is taking seemingly 
complex things, finding a model that makes them simple, and then finding 
a way to communicate that model in a way that preserves its inherent 
simplicity.

The big key there is actually having a simple, well-understood mental 
model, and it sounds like you have that on this, so I'd love to read 
anything you'd have to write about it.



 Nov 09 2016



 next sibling parent reply Dicebot <public dicebot.lv>  writes:


 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: [OT] Re: CTFE Status
References: <gcuuyujyweibqkpgdqiu forum.dlang.org>
 <bdlvsfrnohpffoiofmtt forum.dlang.org> <sjinpbubrbzfhmuxobzu forum.dlang.org>
 <nvgnn8$233l$1 digitalmars.com> <dcciyzoojqgueoqymbfq forum.dlang.org>
 <hbvpgnjsvioiamjerije forum.dlang.org> <mqhyiwiehgxnmnoxmbtd forum.dlang.org>
 <nvsv5v$2k40$1 digitalmars.com>
 <mailman.39.1478624757.3405.digitalmars-d puremagic.com>
 <nvt122$2nba$1 digitalmars.com>
 <mailman.43.1478635777.3405.digitalmars-d puremagic.com>
 <o00rpp$n1a$1 digitalmars.com>
In-Reply-To: <o00rpp$n1a$1 digitalmars.com>

--Q7KlMMvq4dqWtX6BBMWdm3Sa4XtAwWRS7
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 11/10/2016 06:07 AM, Nick Sabalausky wrote:

 On 11/08/2016 02:11 PM, Antonio Corbi wrote:

 Maybe this one is useful for you:

 http://eagain.net/articles/git-for-computer-scientists/


 On 11/08/2016 03:01 PM, H. S. Teoh via Digitalmars-d wrote:

 Nothing immediately comes to mind, but thanks to Dr. Google, I found
 this page that's sorta helpful:

     http://ericsink.com/vcbe/html/directed_acyclic_graphs.html

 And perhaps these:

     http://eagain.net/articles/git-for-computer-scientists/
     http://marklodato.github.io/visual-git-guide/index-en.html


=20
 Ok, so it looks like each node in the DAG is a commit. I'll definitely
 have to read further. Thanks, both.
=20
 Although I have my doubts it would explain all the issues I've hit upon=



 with git's CLI. For example: I don't see why annotated tags aren't the
 default. Or why non-annotated ones even exist at all. When I made
 <https://github.com/Abscissa/gen-package-version>, I noticed that it'll=



 *only* work if the version tags are "annotated" tags. Because otherwise=



 "git describe" doesn't retrieve the tag. Doesn't even matter one bit
 *what* the tag's message is, just that it exists. Why? Who knows! It's =


git!

Studying git object model in more details actually explains that.
Annotated tag is a first class git object (like commit), while plain tag
is simply a reference to existing object (like branch). From usability
PoV difference can be negligible but impact on git internals is huge.

The fact that the default is non-annotated tag is still just plain bad
decision though :)


 Its things like that. I'd be surprised if that has much to do with git'=


s

 nature as a "dumb" DAG tool. It's just the general good-design principl=


e

 of "The thing you *want to* or *should* do or *expect* 99% of the time
 should be the DEFAULT, not the obscure incantation".


I think it is related, but is not necessary consequence. My
understanding is that for a long time command line design was given zero
thoughts on its own - it was directly exposing whatever git does
internally with no usability considerations. Which is why it is so
awkward to use unless you really do know internal object model in great
details.


--Q7KlMMvq4dqWtX6BBMWdm3Sa4XtAwWRS7--



 Nov 10 2016



 next sibling parent  Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 11/10/2016 09:31 AM, Dicebot wrote:

 On 11/10/2016 06:07 AM, Nick Sabalausky wrote:

 Its things like that. I'd be surprised if that has much to do with git's
 nature as a "dumb" DAG tool. It's just the general good-design principle
 of "The thing you *want to* or *should* do or *expect* 99% of the time
 should be the DEFAULT, not the obscure incantation".


 I think it is related, but is not necessary consequence. My
 understanding is that for a long time command line design was given zero
 thoughts on its own - it was directly exposing whatever git does
 internally with no usability considerations. Which is why it is so
 awkward to use unless you really do know internal object model in great
 details.


Yea, see that's the stuff I mean when I say I dislike git's CLI, and 
would really like to fix with a front-end. A few years back, someone 
here compared it to driving a car by leaning under the hood pulling on 
various wires. I like that comparison. Git always did give off that 
kinda feel to me.

Somebody else pointed out gitless. At a quick glance, it looks like it 
has at least some potential. I'll have to take a closer look. Although I 
suspect I'll find a bunch of "oh, hmm, I'd have done that differently" 
just because...well, I tend to be like that ;)



 Nov 10 2016



prev sibling  parent  Jacob Carlborg <doob me.com>  writes:


On 2016-11-10 06:31, Dicebot wrote:


 I think it is related, but is not necessary consequence. My
 understanding is that for a long time command line design was given zero
 thoughts on its own - it was directly exposing whatever git does
 internally with no usability considerations. Which is why it is so
 awkward to use unless you really do know internal object model in great
 details.


It has the porcelain commands now, compared to the more low level 
pluming commands. I guess at one point only the pluming commands existed.

-- 
/Jacob Carlborg



 Nov 12 2016



prev sibling  parent reply Jacob Carlborg <doob me.com>  writes:


On 2016-11-09 20:07, Nick Sabalausky wrote:


 Although I have my doubts it would explain all the issues I've hit upon
 with git's CLI. For example: I don't see why annotated tags aren't the
 default. Or why non-annotated ones even exist at all. When I made
 <https://github.com/Abscissa/gen-package-version>, I noticed that it'll
 *only* work if the version tags are "annotated" tags. Because otherwise
 "git describe" doesn't retrieve the tag. Doesn't even matter one bit
 *what* the tag's message is, just that it exists. Why? Who knows! It's git!


If you add the --tags flag to "describe" it will work.

-- 
/Jacob Carlborg



 Nov 12 2016



  parent  Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 11/12/2016 11:02 AM, Jacob Carlborg wrote:

 On 2016-11-09 20:07, Nick Sabalausky wrote:


 Although I have my doubts it would explain all the issues I've hit upon
 with git's CLI. For example: I don't see why annotated tags aren't the
 default. Or why non-annotated ones even exist at all. When I made
 <https://github.com/Abscissa/gen-package-version>, I noticed that it'll
 *only* work if the version tags are "annotated" tags. Because otherwise
 "git describe" doesn't retrieve the tag. Doesn't even matter one bit
 *what* the tag's message is, just that it exists. Why? Who knows! It's
 git!


 If you add the --tags flag to "describe" it will work.


Ok, I see (sorta). But it does work without --tags if it's an annotated 
tag. Clear as mud.

Yea, see, these are the reasons I consider git's cli to be god-awful.



 Nov 12 2016



prev sibling  parent reply Patrick Schluter <Patrick.Schluter bbox.fr>  writes:


On Tuesday, 8 November 2016 at 16:57:27 UTC, H. S. Teoh wrote:

 On Tue, Nov 08, 2016 at 11:40:31AM -0500, Nick Sabalausky via 
 Digitalmars-d wrote:

 On 11/05/2016 11:48 AM, Marc Schütz wrote:

 On Saturday, 5 November 2016 at 01:21:48 UTC, Stefan Koch 
 wrote:

 
 I recently lost 3 days of work because of my git-skills.


 
 Unless you haven't committed your work yet, almost 
 everything in Git can be undone. Make a copy of your entire 
 project directory (including .git) and then have a look at 
 `git reflog` around the time the disaster happened. It will 
 show you commit IDs that you can check out.


 
 Yea, but unless you're a git-fu master, sometimes figuring out 
 how to fix whatever got messed up can lose you 3 days of work 
 ;)
 
 I really want to make a saner CLI front-end for git, but that 
 would require learning more about git than I really ever want 
 to know :(




If you study carefully the data model of git you will then come 
to the conclusion that its CLI is not that bad.

 The thing about git is that at its core, it's really very 
 simple. Dumb, even.  It's basically a program for managing a 
 directed acyclic graph (DAG).  That's all there is to it. The 
 rest is just frills.

 Trying to rationalize git in terms of traditional version 
 control systems is what usually causes lots of confusion, 
 incomprehension, and frustration.  To truly grok git, you have 
 to just forget about traditional version control concepts, and 
 think purely in terms of DAGs. Once you do, everything falls 
 into place and it all makes sense in its own peculiar way 
 (including all the flaws :-P).


I describe git not as a vcs but as exacly what Linus Torvalds 
described it when he first presented it the Kernel mailing list: 
a directory content state recorder optimized for text files. The 
DAG is already an abstraction above i.e. a mean to that end.



 Nov 08 2016



  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 11/08/2016 02:30 PM, Patrick Schluter wrote:

 I describe git not as a vcs but as exacly what Linus Torvalds described
 it when he first presented it the Kernel mailing list: a directory
 content state recorder optimized for text files.


...that's incapable of comprehending the concept of empty directories, 
contrary to every major file system in use.

I never bought the argument about "b/c it tracks content, not files". If 
that really did explain the limitation, then it wouldn't be capable of 
handling empty *files* either, and yet it does.



 Nov 08 2016



  parent  Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 11/08/2016 05:54 PM, Nick Sabalausky wrote:

 On 11/08/2016 02:30 PM, Patrick Schluter wrote:

 I describe git not as a vcs but as exacly what Linus Torvalds described
 it when he first presented it the Kernel mailing list: a directory
 content state recorder optimized for text files.


 ...that's incapable of comprehending the concept of empty directories,
 contrary to every major file system in use.

 I never bought the argument about "b/c it tracks content, not files". If
 that really did explain the limitation, then it wouldn't be capable of
 handling empty *files* either, and yet it does.


(Pardon the OT)



 Nov 08 2016



prev sibling  parent  Iakh <iaktakh gmail.com>  writes:


On Tuesday, 8 November 2016 at 16:40:31 UTC, Nick Sabalausky 
wrote:

 On 11/05/2016 11:48 AM, Marc Schütz wrote:

 On Saturday, 5 November 2016 at 01:21:48 UTC, Stefan Koch 
 wrote:

 I recently lost 3 days of work because of my git-skills.


 Unless you haven't committed your work yet, almost everything 
 in Git can
 be undone. Make a copy of your entire project directory 
 (including .git)
 and then have a look at `git reflog` around the time the 
 disaster
 happened. It will show you commit IDs that you can check out.


 Yea, but unless you're a git-fu master, sometimes figuring out 
 how to fix whatever got messed up can lose you 3 days of work ;)

 I really want to make a saner CLI front-end for git, but that 
 would require learning more about git than I really ever want 
 to know :(


http://gitless.com/



 Nov 08 2016



prev sibling next sibling parent  deadalnix <deadalnix gmail.com>  writes:


On Saturday, 5 November 2016 at 01:21:48 UTC, Stefan Koch wrote:

 I thought about it some more, and decided that I will replace 
 the old interpreter completely in the long run.
 However in order to get something release before 2017, I have 
 to rely on it being there.


Good thinking.



 Nov 05 2016



prev sibling  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 11/4/16 9:21 PM, Stefan Koch wrote:

 I thought about it some more, and decided that I will replace the old
 interpreter completely in the long run.
 However in order to get something release before 2017, I have to rely on
 it being there.


That's good thinking - leave short term to the short term and long term 
to the long term. As the Romanian proverb goes: "Don't sell the skin of 
the bear before you shoot it." -- Andrei



 Nov 05 2016



 next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 5 November 2016 at 20:57:06 UTC, Andrei Alexandrescu 
wrote:

 On 11/4/16 9:21 PM, Stefan Koch wrote:

 I thought about it some more, and decided that I will replace 
 the old
 interpreter completely in the long run.
 However in order to get something release before 2017, I have 
 to rely on
 it being there.


 That's good thinking - leave short term to the short term and 
 long term to the long term. As the Romanian proverb goes: 
 "Don't sell the skin of the bear before you shoot it." -- Andrei


I would interpret this proverb as referring to sequential 
consistency.

A just a small update today:

I just introduced questionable code that enables the 
interpreter-backed to pass the phobos unitests but will cause the 
llvm backend to fail.

Also, the list of manual bail-outs just grew by a few members.

Currently work on Function-Call support is halted in favor of 
investigating why the bail-outs are necessary.

I begun some work on returning Error-Information for the 
BC-backends.
This is rather difficult due to the high degree of separation 
between the ctfe-engine and the rest of dmd.

A nice weekend to all of you.


P.S.
Phobos and druntime unittests do compile now.
After the ability for returning errors is added all unitests will 
be passed. (Most likely if there are not more bugs inside the 
engine)



 Nov 06 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 6 November 2016 at 15:31:24 UTC, Stefan Koch wrote:

 Phobos and druntime unittests do compile now.
 After the ability for returning errors is added all unitests 
 will be passed. (Most likely if there are not more bugs inside 
 the engine)


I just have just implemented the ability to error out.
And I am still not passing interpret3.d.
That is one hard unit-test.

Currently I pass around 25% of interpret3.d.



 Nov 06 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 6 November 2016 at 22:07:05 UTC, Stefan Koch wrote:

 On Sunday, 6 November 2016 at 15:31:24 UTC, Stefan Koch wrote:

 Phobos and druntime unittests do compile now.
 After the ability for returning errors is added all unitests 
 will be passed. (Most likely if there are not more bugs inside 
 the engine)


 I just have just implemented the ability to error out.
 And I am still not passing interpret3.d.
 That is one hard unit-test.

 Currently I pass around 25% of interpret3.d.


I consider the engine ready to be merged into the feature branch.
There are still a few obscure failures.
But nothing preventing an alpha release.



 Nov 07 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 7 November 2016 at 10:02:37 UTC, Stefan Koch wrote:

 On Sunday, 6 November 2016 at 22:07:05 UTC, Stefan Koch wrote:

 On Sunday, 6 November 2016 at 15:31:24 UTC, Stefan Koch wrote:

 Phobos and druntime unittests do compile now.
 After the ability for returning errors is added all unitests 
 will be passed. (Most likely if there are not more bugs 
 inside the engine)


 I just have just implemented the ability to error out.
 And I am still not passing interpret3.d.
 That is one hard unit-test.

 Currently I pass around 25% of interpret3.d.


 I consider the engine ready to be merged into the feature 
 branch.
 There are still a few obscure failures.
 But nothing preventing an alpha release.


Update I tracked down a bug that prevented a function from 
std.allocator to execute correctly.

Slowly but steadily new ctfe becomes more robust.



 Nov 07 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 7 November 2016 at 14:13:04 UTC, Stefan Koch wrote:

 On Monday, 7 November 2016 at 10:02:37 UTC, Stefan Koch wrote:

 On Sunday, 6 November 2016 at 22:07:05 UTC, Stefan Koch wrote:

 On Sunday, 6 November 2016 at 15:31:24 UTC, Stefan Koch wrote:

 Phobos and druntime unittests do compile now.
 After the ability for returning errors is added all unitests 
 will be passed. (Most likely if there are not more bugs 
 inside the engine)


 I just have just implemented the ability to error out.
 And I am still not passing interpret3.d.
 That is one hard unit-test.

 Currently I pass around 25% of interpret3.d.


 I consider the engine ready to be merged into the feature 
 branch.
 There are still a few obscure failures.
 But nothing preventing an alpha release.


 Update I tracked down a bug that prevented a function from 
 std.allocator to execute correctly.

 Slowly but steadily new ctfe becomes more robust.


Another bug was identified.
There are severe issues with the interpreter back-end and it's 
32bit only legacy.
Fixing these issues will degrade performance of the interpreter 
by roughly 7-15%.

Maybe there is still a way around it, but so far my more 
performance-oriented approaches  are failing to fix the problems.

Errors now know their location, and most of the time assertions 
thrown be the new engine are close enough to the old interpreters 
one.
However my errors have less information in them due to design 
limitations.
It is possible to lift those but only with significant effort.

Slightly OT.
I feel like I hit every bump in the road to a faster interpreter, 
and I know there are a a lot more ahead.

Today I spent 3 hours to find a bug which caused a division by 
zero.
It turned out that 64bit large function arguments where silently 
skipped.
Causing the interpreter to read zero initialized memory instead 
of the actual arguments. However I suspected the division code to 
be at fault and wasted a lot of time of second-guessing correct 
code.

Having these implementation challenges intermixed with design and 
performance considerations can be taxing sometimes.

Without my bytecode-unitests which are ctfeable it would be 
almost impossible to for me to work at the pace I am working.
Simply because I get direct feedback if something goes wrong 
because the compile will abort after less then a second.

These kinds of things are the true strength of ctfe.
CTFE is the one of the greatest development assets one can have.
Because it transforms the compile-debug-edit cycle into a 
compile-edit cycle.



 Nov 07 2016



prev sibling  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 11/05/2016 04:57 PM, Andrei Alexandrescu wrote:

 That's good thinking - leave short term to the short term and long term
 to the long term. As the Romanian proverb goes: "Don't sell the skin of
 the bear before you shoot it." -- Andrei


I really wish Google would take that to heart. They seem to make a habit 
of ripping things out *before* having replacements in place.

I think they just simply love deleting code.



 Nov 08 2016



  parent reply Chris Wright <dhasenan gmail.com>  writes:


On Tue, 08 Nov 2016 11:44:50 -0500, Nick Sabalausky wrote:

 I really wish Google would take that to heart. They seem to make a habit
 of ripping things out *before* having replacements in place.
 
 I think they just simply love deleting code.


I've seen this more internally than externally. The joke was that there 
are two versions of everything: the deprecated one, and the one that 
doesn't work yet.



 Nov 08 2016



  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 11/08/2016 11:40 PM, Chris Wright wrote:

 On Tue, 08 Nov 2016 11:44:50 -0500, Nick Sabalausky wrote:

 I really wish Google would take that to heart. They seem to make a habit
 of ripping things out *before* having replacements in place.

 I think they just simply love deleting code.


 I've seen this more internally than externally. The joke was that there
 are two versions of everything: the deprecated one, and the one that
 doesn't work yet.


It's REALLY poor project management.



 Nov 09 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 10 November 2016 at 04:22:10 UTC, Nick Sabalausky 
wrote:

 On 11/08/2016 11:40 PM, Chris Wright wrote:

 On Tue, 08 Nov 2016 11:44:50 -0500, Nick Sabalausky wrote:

 I really wish Google would take that to heart. They seem to 
 make a habit
 of ripping things out *before* having replacements in place.

 I think they just simply love deleting code.


 I've seen this more internally than externally. The joke was 
 that there
 are two versions of everything: the deprecated one, and the 
 one that
 doesn't work yet.


 It's REALLY poor project management.


Please cut down on the OT guys.

This thread serves as my development log :)
As well as providing a place for discussion of the new engine.

Though that discussion seems to be very quiet at the moment.



 Nov 09 2016



  parent  Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 11/09/2016 11:27 PM, Stefan Koch wrote:

 Please cut down on the OT guys.

 This thread serves as my development log :)
 As well as providing a place for discussion of the new engine.

 Though that discussion seems to be very quiet at the moment.


Although the default web-based front-end for this newsgroup suggests 
otherwise, this *is* a tree-based message system. Uninteresting 
sub-trees can simply be ignored without resorting to content policing.



 Nov 09 2016



prev sibling  parent reply ketmar <ketmar ketmar.no-ip.org>  writes:


On Friday, 4 November 2016 at 10:59:25 UTC, Marc Schütz wrote:

 In this case better make sure to remove the resulting dead code 
 from the old interpreter, otherwise it will become and 
 unmaintainable mess in the long run.


interpreter is used all over the code, and it is a set of 
visitors doing various things. that is, it is not possible to 
remove "dead code" from it, only replace it as a whole. which is 
a great achievement, but completely unpractical. it is way, way 
easier to build the basic VM that runs alongside the old 
interpreter, and call old one if new VM found something it can't 
process yet. this way new VM can be used to speed up some parts, 
but nothing will be broken, and VM can be refined gradually. 
also, it is great for testing: one can call new and old 
interpreters and compare results. as new VM is completely 
independent of old interpreter, the code won't be any messier 
than it is now. ;-)

from a technical POV CTFE interpreter is just a codegen: it 
receiving lowered AST, visit nodes and does some actions. one can 
have many codegens in frontend code, they won't hurt each other. 
;-)



 Nov 04 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 5 November 2016 at 03:01:48 UTC, ketmar wrote:

 On Friday, 4 November 2016 at 10:59:25 UTC, Marc Schütz wrote:

 In this case better make sure to remove the resulting dead 
 code from the old interpreter, otherwise it will become and 
 unmaintainable mess in the long run.


 interpreter is used all over the code, and it is a set of 
 visitors doing various things. that is, it is not possible to 
 remove "dead code" from it, only replace it as a whole.


Yes what ketmar said.

It is looking good, current testsuite-failures are caused by the 
new engines missing checks for ub, and the inability to signal 
those errors.

Figuring out a good mechanism for error-signaling is the next 
challenge.



 Nov 05 2016



prev sibling next sibling parent reply Faux Amis <faux amis.com>  writes:


Any reason for the infinite depth update posting style?
I would have loved to see each update to be a child of the root post 
with its own discussions tree.
Currently, the posts are quite unreadable in tree view (thunderbird).


On 2016-10-31 14:29, Stefan Koch wrote:

 Hi Guys, since I got a few complaints about giving minor status updates
 in the announce group, I am opening this thread.





 Dec 08 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 8 December 2016 at 19:49:47 UTC, Faux Amis wrote:

 Any reason for the infinite depth update posting style?
 I would have loved to see each update to be a child of the root 
 post with its own discussions tree.
 Currently, the posts are quite unreadable in tree view 
 (thunderbird).


 On 2016-10-31 14:29, Stefan Koch wrote:

 Hi Guys, since I got a few complaints about giving minor 
 status updates
 in the announce group, I am opening this thread.




The reason it that I don't use the tree view.
I just answer to myself every time I have an update.
It looks fine on the web-frontend.



 Dec 08 2016



  parent  Faux Amis <faux amis.com>  writes:


On 2016-12-09 00:00, Stefan Koch wrote:

 On Thursday, 8 December 2016 at 19:49:47 UTC, Faux Amis wrote:

 Any reason for the infinite depth update posting style?
 I would have loved to see each update to be a child of the root post
 with its own discussions tree.
 Currently, the posts are quite unreadable in tree view (thunderbird).


 On 2016-10-31 14:29, Stefan Koch wrote:

 Hi Guys, since I got a few complaints about giving minor status updates
 in the announce group, I am opening this thread.




 The reason it that I don't use the tree view.
 I just answer to myself every time I have an update.
 It looks fine on the web-frontend.


Might you consider making use of the tree data structure?
It really has quite nice properties with regard to discussions..
(check the threaded view mode, everybody else seems to use it like that)
Anyway, I kinda missed the updates because I thought it was just a very 
deep discussion until I noticed it was mostly you posting ;)



 Dec 09 2016



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 31 October 2016 at 13:29:01 UTC, Stefan Koch wrote:

 Hi Guys, since I got a few complaints about giving minor status 
 updates in the announce group, I am opening this thread.

 I will start with giving an overview of what works and what 
 does not work.

 Currently the only basic type you can do arithmetic on is int.
 Altough you can compare longs since a few days.

 These are the constructs that will work.

 - foreach on static arrays strings and range-foreach  (those 
 kinds (0 .. 64)).
 - switches (even deeply nested ones)
 - for and while loops
 - ternary expressions (? :)
 - if and else statements (as long as you don't use && and || )
 - lables and gotos
 - arithmetic expressions as well as post and pre increment and 
 decrement

 Constructs that will not work (but are actively worked on)

 - assignment to static array cells
 - long ulong arithmetic.
 - function calls
 - dynamic arrays and slices
 - pointers
 - structs
 - && and ||
 - sliceing

 Constructs that will not work and are futher down the list.

 - classes
 - closures
 - boundschecks
 - asserts

 Please note that there will probably be bugs all over the place.
 So even the working features might not be working completely.


FunctionPointers and plain function calls are currently in 
progress!
Also I made further performance improvements to the 
Bytecode-generator.
As of now it needs around 50 nano-seconds per generated 
instruction.
Generation + Interpretation execution of trivial functions now 
takes LESS then a microsecond on contemporary machines!



 Dec 09 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 10 December 2016 at 07:24:10 UTC, Stefan Koch wrote:

 FunctionPointers and plain function calls are currently in 
 progress!
 Also I made further performance improvements to the 
 Bytecode-generator.
 As of now it needs around 50 nano-seconds per generated 
 instruction.
 Generation + Interpretation execution of trivial functions now 
 takes LESS then a microsecond on contemporary machines!


Function-call support has broken the bailout capability.
I am working on restoring it.



 Dec 10 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 10 December 2016 at 18:51:32 UTC, Stefan Koch wrote:

 On Saturday, 10 December 2016 at 07:24:10 UTC, Stefan Koch 
 wrote:

 FunctionPointers and plain function calls are currently in 
 progress!
 Also I made further performance improvements to the 
 Bytecode-generator.
 As of now it needs around 50 nano-seconds per generated 
 instruction.
 Generation + Interpretation execution of trivial functions now 
 takes LESS then a microsecond on contemporary machines!


 Function-call support has broken the bailout capability.
 I am working on restoring it.


I just fixed the bailout.
certain switch-statements.
And made the creation of bytecode even cheaper.



 Dec 10 2016



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 31 October 2016 at 13:29:01 UTC, Stefan Koch wrote:

 Hi Guys, since I got a few complaints about giving minor status 
 updates in the announce group, I am opening this thread.

 I will start with giving an overview of what works and what 
 does not work.

 Currently the only basic type you can do arithmetic on is int.
 Altough you can compare longs since a few days.

 These are the constructs that will work.

 - foreach on static arrays strings and range-foreach  (those 
 kinds (0 .. 64)).
 - switches (even deeply nested ones)
 - for and while loops
 - ternary expressions (? :)
 - if and else statements (as long as you don't use && and || )
 - lables and gotos
 - arithmetic expressions as well as post and pre increment and 
 decrement

 Constructs that will not work (but are actively worked on)

 - assignment to static array cells
 - long ulong arithmetic.
 - function calls
 - dynamic arrays and slices
 - pointers
 - structs
 - && and ||
 - sliceing

 Constructs that will not work and are futher down the list.

 - classes
 - closures
 - boundschecks
 - asserts

 Please note that there will probably be bugs all over the place.
 So even the working features might not be working completely.


I just fixed a really nasty bug  that prevented the following 
code from working in newCTFE.

int computeFib(int n)
{
     int t = 1;
     int result = 0;

     while(n--)
     {
         result = t - result;
         t = t + result;
     }

    return result;
}

static assert(computeFib(12) == 144);

The line result = t - result is the one triggering the bug.
Because newCTFE broke it down like this
result = t;
result -= result;

This was not found by the compilers testsuite!
I guess we should add it.



 Dec 10 2016



  parent  Basile B. <b2.temp gmx.com>  writes:


On Saturday, 10 December 2016 at 11:03:11 UTC, Stefan Koch wrote:

 This was not found by the compilers testsuite!


Have you already thought to use fuzz testing ?

I'm sure it would find some weird failure cases. The other day 
I've found that `enum a=null+null;` triggers a CTFE ICE. It's on 
the old engine but a tool with a "mean and nasty" approach could 
have found it.



 Dec 10 2016



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 31 October 2016 at 13:29:01 UTC, Stefan Koch wrote:

 Hi Guys, since I got a few complaints about giving minor status 
 updates in the announce group, I am opening this thread.

 I will start with giving an overview of what works and what 
 does not work.

 Currently the only basic type you can do arithmetic on is int.
 Altough you can compare longs since a few days.

 These are the constructs that will work.

 - foreach on static arrays strings and range-foreach  (those 
 kinds (0 .. 64)).
 - switches (even deeply nested ones)
 - for and while loops
 - ternary expressions (? :)
 - if and else statements (as long as you don't use && and || )
 - lables and gotos
 - arithmetic expressions as well as post and pre increment and 
 decrement

 Constructs that will not work (but are actively worked on)

 - assignment to static array cells
 - long ulong arithmetic.
 - function calls
 - dynamic arrays and slices
 - pointers
 - structs
 - && and ||
 - sliceing

 Constructs that will not work and are futher down the list.

 - classes
 - closures
 - boundschecks
 - asserts

 Please note that there will probably be bugs all over the place.
 So even the working features might not be working completely.


Remember those numbers ?

-bc-ctfe
2147385345u
536821761u
4294639619u

real	0m0.114s
user	0m0.110s
sys	0m0.003s

This is how they look like after my performance fixes:

real	0m0.047s
user	0m0.043s
sys	0m0.000s



 Dec 11 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 11 December 2016 at 08:33:49 UTC, Stefan Koch wrote:

 On Monday, 31 October 2016 at 13:29:01 UTC, Stefan Koch wrote:

 Hi Guys, since I got a few complaints about giving minor 
 status updates in the announce group, I am opening this thread.

 I will start with giving an overview of what works and what 
 does not work.

 Currently the only basic type you can do arithmetic on is int.
 Altough you can compare longs since a few days.

 These are the constructs that will work.

 - foreach on static arrays strings and range-foreach  (those 
 kinds (0 .. 64)).
 - switches (even deeply nested ones)
 - for and while loops
 - ternary expressions (? :)
 - if and else statements (as long as you don't use && and || )
 - lables and gotos
 - arithmetic expressions as well as post and pre increment and 
 decrement

 Constructs that will not work (but are actively worked on)

 - assignment to static array cells
 - long ulong arithmetic.
 - function calls
 - dynamic arrays and slices
 - pointers
 - structs
 - && and ||
 - sliceing

 Constructs that will not work and are futher down the list.

 - classes
 - closures
 - boundschecks
 - asserts

 Please note that there will probably be bugs all over the 
 place.
 So even the working features might not be working completely.


 Remember those numbers ?

 -bc-ctfe
 2147385345u
 536821761u
 4294639619u

 real	0m0.114s
 user	0m0.110s
 sys	0m0.003s

 This is how they look like after my performance fixes:

 real	0m0.047s
 user	0m0.043s
 sys	0m0.000s


Disregard That!
I wasn't paying attention
The lower numbers are produced by ldc!
The performance fixes did lower the overall overhead even more 
though.
Which means bytecode generation will not even show up among the 
top 50 functions in dmd,



 Dec 11 2016



  parent reply Anonymouse <asdf asdf.net>  writes:


On Sunday, 11 December 2016 at 08:37:28 UTC, Stefan Koch wrote:

 Disregard That!
 I wasn't paying attention
 The lower numbers are produced by ldc!
 The performance fixes did lower the overall overhead even more 
 though.
 Which means bytecode generation will not even show up among the 
 top 50 functions in dmd,


Would you say it has ended up being more or less (or roughly 
equal) work than you initially expected?

And keep up the good work!



 Dec 11 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 11 December 2016 at 09:05:26 UTC, Anonymouse wrote:

 Would you say it has ended up being more or less (or roughly 
 equal) work than you initially expected?

 And keep up the good work!


I did expect a lot of work.
But with debugging it exceeded my expectations.
Originally I planned for this to over in 3 months.
Now I am going finish the 6th month and it won't be done 
completely.

I am just happy that I could keep the bytecode-overhead so low 
that no one will notice a slow-down when we switch engines.

Work on function-calls is still progressing.



 Dec 11 2016



 next sibling parent  Ethan Watson <gooberman gmail.com>  writes:


On Sunday, 11 December 2016 at 09:13:41 UTC, Stefan Koch wrote:

 Originally I planned for this to over in 3 months.
 Now I am going finish the 6th month and it won't be done 
 completely.


I hear that.

As an anecdote, by Binderoo work has been functional for months; 
but it's only now becoming usable by normal people. And then once 
normal people use it, there's gonna be tons of bugs and edge 
cases that I need to blitz through. But I will have a framework 
that people can use to do Unity-style development of code using D 
as their rapid iteration language at the end of it.

I've been moving more stuff out of templates and in to CTFE in 
anticipation for this work hitting main.



 Dec 11 2016



prev sibling  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 11 December 2016 at 09:13:41 UTC, Stefan Koch wrote:

 On Sunday, 11 December 2016 at 09:05:26 UTC, Anonymouse wrote:

 Would you say it has ended up being more or less (or roughly 
 equal) work than you initially expected?

 And keep up the good work!


 I did expect a lot of work.
 But with debugging it exceeded my expectations.
 Originally I planned for this to over in 3 months.
 Now I am going finish the 6th month and it won't be done 
 completely.

 I am just happy that I could keep the bytecode-overhead so low 
 that no one will notice a slow-down when we switch engines.

 Work on function-calls is still progressing.


I just fixed labled for and foreach!

uint fn1()
{
     uint result;
     L: foreach (i; 0 .. 24)
     {
         if (i != 4)
             continue L;
         result += i;
     }
     return result;
}

static assert(fn1() == 4);

This will work now.
Whereas before it ignored the continue.



 Dec 12 2016



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Hi Guys, I just fixed the LLVM-Backend a little.
It's about 4000 times slower to start up then the interpreter.
And has 1000 microseconds overhead per evaluation.

If you don't want to run a raytracer at compiletime I doubt that 
the llvm backend is the right one for you.

That said, it's a great fit for getting started in compiler 
development.
It can also be used to show potential optimization our own 
peephole optimizer could do.

The llvm backend will currently not run the whole test-suite 
because it fails on ++ (because of a missing load-store cycle I 
suspect)...
The strange thing here is, that it only fails sometimes :)



 Dec 12 2016



 next sibling parent reply Jacob Carlborg <doob me.com>  writes:


On 2016-12-13 08:21, Stefan Koch wrote:

 Hi Guys, I just fixed the LLVM-Backend a little.
 It's about 4000 times slower to start up then the interpreter.
 And has 1000 microseconds overhead per evaluation.

 If you don't want to run a raytracer at compiletime I doubt that the
 llvm backend is the right one for you.


IIRC, the Higgs JavaScript JIT compiler doesn't always use JIT because 
sometimes it faster to interpret because JIT has some overhead. Not sure 
if this is still the case and how it chooses when to interpret or when 
to JIT.

-- 
/Jacob Carlborg



 Dec 12 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 13 December 2016 at 07:53:56 UTC, Jacob Carlborg 
wrote:

 On 2016-12-13 08:21, Stefan Koch wrote:

 Hi Guys, I just fixed the LLVM-Backend a little.
 It's about 4000 times slower to start up then the interpreter.
 And has 1000 microseconds overhead per evaluation.

 If you don't want to run a raytracer at compiletime I doubt 
 that the
 llvm backend is the right one for you.


 IIRC, the Higgs JavaScript JIT compiler doesn't always use JIT 
 because sometimes it faster to interpret because JIT has some 
 overhead. Not sure if this is still the case and how it chooses 
 when to interpret or when to JIT.


Higgs IR is high-level which makes a translation step to a lower 
ir and then x86 unavoidable.

My architecture allows to emit assembly directly (while 
everything is still hot and steamy in the cache).
Therefore JIT will _ALWAYS_ be faster then interpretation.
Given that a executeble-page is already allocated.
Since getting the page from the OS can take a long time.



 Dec 13 2016



prev sibling  parent reply deadalnix <deadalnix gmail.com>  writes:


On Tuesday, 13 December 2016 at 07:21:07 UTC, Stefan Koch wrote:

 Hi Guys, I just fixed the LLVM-Backend a little.
 It's about 4000 times slower to start up then the interpreter.
 And has 1000 microseconds overhead per evaluation.

 If you don't want to run a raytracer at compiletime I doubt 
 that the llvm backend is the right one for you.

 That said, it's a great fit for getting started in compiler 
 development.
 It can also be used to show potential optimization our own 
 peephole optimizer could do.

 The llvm backend will currently not run the whole test-suite 
 because it fails on ++ (because of a missing load-store cycle I 
 suspect)...
 The strange thing here is, that it only fails sometimes :)


Do you use MCJIT or some special sauce made with Orc ?

Anyway, yes, LLVM's JIT is heavy duty, good for long running code 
but probably not so much for CTFE which are mostly one shot.



 Dec 14 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 15 December 2016 at 06:04:12 UTC, deadalnix wrote:

 On Tuesday, 13 December 2016 at 07:21:07 UTC, Stefan Koch wrote:

 Do you use MCJIT or some special sauce made with Orc ?

 Anyway, yes, LLVM's JIT is heavy duty, good for long running 
 code but probably not so much for CTFE which are mostly one 
 shot.


MCJIT, mostly orc is not available with the old set of sdc-llvm 
headers I happen to use.
Orc is too much of a pain to setup if you are not planning on 
rolling with llvm to begin with.
Also LLVM has a but of weirdness to it. Where it'll miscompile 
perfectly valid code.
(I happen to use llvm 3.4)



 Dec 15 2016



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


I made good progress on function calls today.
By deferring them we get both better bytecode-generator 
performance and simpler code.
Since the deferring step seems to work, you can expect calls to 
work by the end of next week.

When function-calls work, I will direct efforts towards correct 
handling of || and &&,
since it is very annoying to work around the lack of them :)

Also start-up times for ctfe are still dropping.
When the old ctfe is completely replaced, (hopefully around 
February)
I can reasonably expect compile-times to be reduced by over 65% 
(given that ctfe was the bottleneck)

Cheers,
Stefan



 Dec 13 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 13 December 2016 at 12:04:31 UTC, Stefan Koch wrote:

 I made good progress on function calls today.
 By deferring them we get both better bytecode-generator 
 performance and simpler code.
 Since the deferring step seems to work, you can expect calls to 
 work by the end of next week.

 When function-calls work, I will direct efforts towards correct 
 handling of || and &&,
 since it is very annoying to work around the lack of them :)

 Also start-up times for ctfe are still dropping.
 When the old ctfe is completely replaced, (hopefully around 
 February)
 I can reasonably expect compile-times to be reduced by over 65% 
 (given that ctfe was the bottleneck)

 Cheers,
 Stefan


String-Switches are supported now.

And just to make a point why I am pissed about templates 
sometimes :
The following paste contains the IR generated by a Phobos 
template which was miscompiled until very recently.

oh ...  pastie does not accept more then 64K :(
Ah pastebin does.
http://pastebin.com/Jfh1NH3m



 Dec 14 2016



  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Wednesday, 14 December 2016 at 08:40:34 UTC, Stefan Koch wrote:

 String-Switches are supported now.


Cool, this is a crucial requirement for pattern matching in 
parser generators, like Pegged and alikes. Is it worth giving 
newCTFE a try with these things now?



 Dec 15 2016



 next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Thursday, 15 December 2016 at 09:21:10 UTC, Nordlöw wrote:

 Cool, this is a crucial requirement for pattern matching in 
 parser generators, like Pegged and alikes. Is it worth giving 
 newCTFE a try with these things now?


BTW, does newCTFE string-switches use O(1) lookups?



 Dec 15 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 15 December 2016 at 09:23:07 UTC, Nordlöw wrote:

 On Thursday, 15 December 2016 at 09:21:10 UTC, Nordlöw wrote:

 Cool, this is a crucial requirement for pattern matching in 
 parser generators, like Pegged and alikes. Is it worth giving 
 newCTFE a try with these things now?


 BTW, does newCTFE string-switches use O(1) lookups?


Nope. Currently it lowers to an if else chain.
Which is O(N), though still faster then calling the runtime :)

I just fixed Assigning static arrays  from ArrayLiterals.
....
And now the phobos-compiletime tests pass!



 Dec 15 2016



 next sibling parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Thursday, 15 December 2016 at 12:06:46 UTC, Stefan Koch wrote:

 I just fixed Assigning static arrays  from ArrayLiterals.
 ....
 And now the phobos-compiletime tests pass!


Wonderful! I just realized that fast string-switch is only 
necessary for parse generators when the generated parse is 
evaluated at compile-time :)



 Dec 15 2016



prev sibling  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Thursday, 15 December 2016 at 12:06:46 UTC, Stefan Koch wrote:

 And now the phobos-compiletime tests pass!


How do you run the phobos compile-time tests?

make unittest does do it all right?



 Dec 15 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 15 December 2016 at 13:38:36 UTC, Nordlöw wrote:

 On Thursday, 15 December 2016 at 12:06:46 UTC, Stefan Koch 
 wrote:

 And now the phobos-compiletime tests pass!


 How do you run the phobos compile-time tests?

 make unittest does do it all right?


yes.
I run make -f posix.mak unittest.



 Dec 15 2016



prev sibling  parent reply Dmitry Olshansky <dmitry.olsh gmail.com>  writes:


On 12/15/16 10:21 AM, Nordlöw wrote:

 On Wednesday, 14 December 2016 at 08:40:34 UTC, Stefan Koch wrote:

 String-Switches are supported now.


 Cool, this is a crucial requirement for pattern matching in parser
 generators, like Pegged and alikes. Is it worth giving newCTFE a try
 with these things now?


Afaik string switches are implemented as a fairly slow hash table. 
Optimal solutions like building a trie would be a nice enhancement.

---
Dmitry Olshansky



 Dec 15 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 15 December 2016 at 23:05:55 UTC, Dmitry Olshansky 
wrote:

 Afaik string switches are implemented as a fairly slow hash 
 table. Optimal solutions like building a trie would be a nice 
 enhancement.

 ---
 Dmitry Olshansky


Tries take up alot of memory for sparse tables.
I would like to avoid them.

Futhermore, CTFE is targeted towards smaller functions with low 
iteration count.
Therefore producing bytecode as fast as possible is priority.

This function from std.uri does now compile and execute correctly.
private enum
{
     URI_Alpha = 1,
     URI_Reserved = 2,
     URI_Mark = 4,
     URI_Digit = 8,

}

immutable ubyte[128] uri_flags =      // indexed by character
     ({
         ubyte[128] uflags;

         // Compile time initialize


         foreach (c; 'A' .. 'Z' + 1)
         {
             uflags[c] |= URI_Alpha;
             uflags[c + 0x20] |= URI_Alpha;   // lowercase letters
         }
         foreach (c; '0' .. '9' + 1) uflags[c] |= URI_Digit;
         foreach (c; ";/?: &=+$,")   uflags[c] |= URI_Reserved;
         foreach (c; "-_.!~*'()")    uflags[c] |= URI_Mark;
         return uflags;
})();

I have to say I was surprised myself :)
It takes around 30 us to execute, which can be improved for sure 
:)



 Dec 15 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 15 December 2016 at 23:37:50 UTC, Stefan Koch wrote:

 On Thursday, 15 December 2016 at 23:05:55 UTC, Dmitry Olshansky 
 wrote:

 Afaik string switches are implemented as a fairly slow hash 
 table. Optimal solutions like building a trie would be a nice 
 enhancement.

 ---
 Dmitry Olshansky


 Tries take up alot of memory for sparse tables.
 I would like to avoid them.

 Futhermore, CTFE is targeted towards smaller functions with low 
 iteration count.
 Therefore producing bytecode as fast as possible is priority.

 This function from std.uri does now compile and execute 
 correctly.
 private enum
 {
     URI_Alpha = 1,
     URI_Reserved = 2,
     URI_Mark = 4,
     URI_Digit = 8,

 }

 immutable ubyte[128] uri_flags =      // indexed by character
     ({
         ubyte[128] uflags;

         // Compile time initialize


         foreach (c; 'A' .. 'Z' + 1)
         {
             uflags[c] |= URI_Alpha;
             uflags[c + 0x20] |= URI_Alpha;   // lowercase 
 letters
         }
         foreach (c; '0' .. '9' + 1) uflags[c] |= URI_Digit;
         foreach (c; ";/?: &=+$,")   uflags[c] |= URI_Reserved;
         foreach (c; "-_.!~*'()")    uflags[c] |= URI_Mark;
         return uflags;
 })();

 I have to say I was surprised myself :)
 It takes around 30 us to execute, which can be improved for 
 sure :)


Hey Guys, I just had a mental breakthrough in regards to handling 
doubly in-directed read-modify-write operations.
such as changing the length of an array through a pointer.
Refernce values will carry a typed pointer to their origin 
thereby allowing them to be treated like normal values most of 
the time, and only receive special treatment on assignment.
(Which is where they have to emit a Store32 Instead of a Set)
As this has design implications, I will resolve this  before 
continuing function-call support.
In fact, I think I will have to delay call support until most of 
the ABI issues are worked out.



 Dec 15 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Hey Guys bad news, Regressions.
The handling of default constructed Struct-Literals inside of 
newCTFE is broken.

I have no idea were this came from all of a sudden.
I was in crunch-mode to get the phobos unittest to compile, I 
don't remember the last 72 hours at all :)
So it might take a while until the regression is resolved.
I apologize for the inconvenience.



 Dec 16 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 16 December 2016 at 09:32:13 UTC, Stefan Koch wrote:

 Hey Guys bad news, Regressions.
 The handling of default constructed Struct-Literals inside of 
 newCTFE is broken.

 I have no idea were this came from all of a sudden.
 I was in crunch-mode to get the phobos unittest to compile, I 
 don't remember the last 72 hours at all :)
 So it might take a while until the regression is resolved.
 I apologize for the inconvenience.


Apparently the regression was introduce in
I confirmed that the regression was introduced in
https://github.com/dlang/dmd/pull/6186/commits/33dbf5aee7d42b1929fd62ef176bd299c4da8ac5

Which suggests that it only worked by chance before :(



 Dec 16 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 16 December 2016 at 10:26:15 UTC, Stefan Koch wrote:

 On Friday, 16 December 2016 at 09:32:13 UTC, Stefan Koch wrote:

 Hey Guys bad news, Regressions.
 The handling of default constructed Struct-Literals inside of 
 newCTFE is broken.

 I have no idea were this came from all of a sudden.
 I was in crunch-mode to get the phobos unittest to compile, I 
 don't remember the last 72 hours at all :)
 So it might take a while until the regression is resolved.
 I apologize for the inconvenience.


 Apparently the regression was introduce in
 I confirmed that the regression was introduced in
 https://github.com/dlang/dmd/pull/6186/commits/33dbf5aee7d42b1929fd62ef176bd299c4da8ac5

 Which suggests that it only worked by chance before :(


It's confirmed.
The test was not thorough enough.
The gist with the current newCTFE compatible code with  was 
updated.
https://gist.github.com/UplinkCoder/89faa06311e417aa93ea99bc92934d3e



 Dec 16 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 16 December 2016 at 10:33:32 UTC, Stefan Koch wrote:

 On Friday, 16 December 2016 at 10:26:15 UTC, Stefan Koch wrote:

 On Friday, 16 December 2016 at 09:32:13 UTC, Stefan Koch wrote:

 Hey Guys bad news, Regressions.
 The handling of default constructed Struct-Literals inside of 
 newCTFE is broken.

 I have no idea were this came from all of a sudden.
 I was in crunch-mode to get the phobos unittest to compile, I 
 don't remember the last 72 hours at all :)
 So it might take a while until the regression is resolved.
 I apologize for the inconvenience.


 Apparently the regression was introduce in
 I confirmed that the regression was introduced in
 https://github.com/dlang/dmd/pull/6186/commits/33dbf5aee7d42b1929fd62ef176bd299c4da8ac5

 Which suggests that it only worked by chance before :(


 It's confirmed.
 The test was not thorough enough.
 The gist with the current newCTFE compatible code with  was 
 updated.
 https://gist.github.com/UplinkCoder/89faa06311e417aa93ea99bc92934d3e


Another bug is going to be fixed soon, causing infinite recursion 
on structs that have pointer of their own type.
This will allow me to remove a few bail-outs and compile more of 
phobos.



 Dec 16 2016



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


I just fixed gotos again.
While researching unusually large performance drops, I found the 
cause for a rather nasty bug. (It was an off-by-one error again 
:))

It turned out that there were circumstances where some gotos 
would be dropped and the corresponding fixup jumps where never 
emitted.

Due to a curious situation in the dmd ast, I had to make jumps 
without addresses into nops.
This was hiding the bug.

BTW supporting NOPS has been a huge pain for some time now.
I will look into fixing my dmd-ast handling to make them 
unnecessary.
Jumps that don't know where they are supposed to go should not be 
ignored but an error instead!



 Dec 16 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 17 December 2016 at 04:14:29 UTC, Stefan Koch wrote:

 I just fixed gotos again.
 While researching unusually large performance drops, I found 
 the cause for a rather nasty bug. (It was an off-by-one error 
 again :))

 It turned out that there were circumstances where some gotos 
 would be dropped and the corresponding fixup jumps where never 
 emitted.

 Due to a curious situation in the dmd ast, I had to make jumps 
 without addresses into nops.
 This was hiding the bug.

 BTW supporting NOPS has been a huge pain for some time now.
 I will look into fixing my dmd-ast handling to make them 
 unnecessary.
 Jumps that don't know where they are supposed to go should not 
 be ignored but an error instead!


I rectified a few more performance problems.
The StringEquals macro put alot of pressure on the bytecode 
generator.
Therefore I implemented it as an instruction.
Interestingly this did not speed up the execution by as much as I 
would have expected,
maybe I need better test-cases ?



 Dec 16 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Miscompiling phobos code is down to 4 functions.
std.range.primitives.{front, empty, back}
and std.traits.defaultParameters.
Note, StructLiterals are still regressed.
I am looking into it.



 Dec 17 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 17 December 2016 at 08:53:07 UTC, Stefan Koch wrote:

 Miscompiling phobos code is down to 4 functions.
 std.range.primitives.{front, empty, back}
 and std.traits.defaultParameters.
 Note, StructLiterals are still regressed.
 I am looking into it.


Fixed the regression.
It was an off-by-one error in the check of the member-count.
causing the last field of a struct to be considered not part of 
the struct :p



 Dec 17 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 17 December 2016 at 09:39:02 UTC, Stefan Koch wrote:

 On Saturday, 17 December 2016 at 08:53:07 UTC, Stefan Koch 
 wrote:

 Miscompiling phobos code is down to 4 functions.
 std.range.primitives.{front, empty, back}
 and std.traits.defaultParameters.
 Note, StructLiterals are still regressed.
 I am looking into it.


 Fixed the regression.
 It was an off-by-one error in the check of the member-count.
 causing the last field of a struct to be considered not part of 
 the struct :p


Many of the forced bailouts could be removed.
Soon newCTFE will compile Phobos without any blacklisted 
functions!



 Dec 19 2016



 next sibling parent  Nicholas Wilson <iamthewilsonator hotmail.com>  writes:


On Monday, 19 December 2016 at 08:26:00 UTC, Stefan Koch wrote:

 On Saturday, 17 December 2016 at 09:39:02 UTC, Stefan Koch 
 wrote:

 On Saturday, 17 December 2016 at 08:53:07 UTC, Stefan Koch 
 wrote:

 Miscompiling phobos code is down to 4 functions.
 std.range.primitives.{front, empty, back}
 and std.traits.defaultParameters.
 Note, StructLiterals are still regressed.
 I am looking into it.


 Fixed the regression.
 It was an off-by-one error in the check of the member-count.
 causing the last field of a struct to be considered not part 
 of the struct :p


 Many of the forced bailouts could be removed.
 Soon newCTFE will compile Phobos without any blacklisted 
 functions!


Excellent! Keep up the great work!



 Dec 19 2016



prev sibling next sibling parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Monday, 19 December 2016 at 08:26:00 UTC, Stefan Koch wrote:

 Soon newCTFE will compile Phobos without any blacklisted 
 functions!


Awesome!



 Dec 19 2016



prev sibling  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Monday, 19 December 2016 at 08:26:00 UTC, Stefan Koch wrote:

 Soon newCTFE will compile Phobos without any blacklisted 
 functions!


Which functions are still blacklisted?



 Dec 19 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 19 December 2016 at 10:11:18 UTC, Nordlöw wrote:

 On Monday, 19 December 2016 at 08:26:00 UTC, Stefan Koch wrote:

 Soon newCTFE will compile Phobos without any blacklisted 
 functions!


 Which functions are still blacklisted?


Check here
https://github.com/UplinkCoder/dmd/blob/newCTFE/src/ctfe_bc.d#L120
currently it's still the range primitives and __lambda2 because 
of ParameterTypeTuple stuff.



 Dec 19 2016



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Hey Guys I wanted to give a quick update.


I recently fixed an interesting bug that happened de-referencing 
null pointers to the heap.
That would cause arrays to be overwritten, when passed as 
arguments.
That bug had me stuck for a few hours.

Getting Function-Calls right will be impossible,
if the ABI issues remain.

Currently it's about the handling of pointers.
My architecture forbids pointers to the stack, therefore every 
time you take the address of something it needs to be pushed onto 
the heap.
And then the heap-value and the stack-value needs to be kept in 
sync with each other.
This is rather complicated, so it will take a while to get right.
At this point it might take another month until function calls 
are working.

--
On a different note.
Thanks for your continued moral support.
It is nice to know that my work is eagerly awaited by people.
As always I am sorry for the long time it takes.
I hope that my reports give a little insight into what issues I 
am facing, and why it is taking so long.



 Dec 20 2016



 next sibling parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 20 December 2016 at 16:53:21 UTC, Stefan Koch wrote:

 At this point it might take another month until function calls 
 are working.


I admire your endurance.



 Dec 20 2016



prev sibling next sibling parent  jmh530 <john.michael.hall gmail.com>  writes:


On Tuesday, 20 December 2016 at 16:53:21 UTC, Stefan Koch wrote:

 On a different note.
 Thanks for your continued moral support.
 It is nice to know that my work is eagerly awaited by people.
 As always I am sorry for the long time it takes.
 I hope that my reports give a little insight into what issues I 
 am facing, and why it is taking so long.


I find it all very interesting. Keep up the good work.



 Dec 20 2016



prev sibling  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 20 December 2016 at 16:53:21 UTC, Stefan Koch wrote:

 I hope that my reports give a little insight into what issues I 
 am facing, and why it is taking so long.


There's a left parenthesis too much on line 3320 in ctfe_bc.d ;)



 Dec 20 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 20 December 2016 at 20:30:43 UTC, Nordlöw wrote:

 On Tuesday, 20 December 2016 at 16:53:21 UTC, Stefan Koch wrote:

 I hope that my reports give a little insight into what issues 
 I am facing, and why it is taking so long.


 There's a left parenthesis too much on line 3320 in ctfe_bc.d ;)


You're right.
I was sleepy when committing that, so I forgot to check if it 
compiles.

It had flags to allow stack pointers for integers to make slice 
handling cheaper in the interpreter, but it turns out that this 
decision hurt performance overall, and now I am removing it.
This unmatched paren was part of the i32Ptr check :)



 Dec 20 2016



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Hey Guys,

Pointers support is coming!

The following code compiles now:

uint* fn(uint v)
{
     return new uint(v + 6);
}


static assert(*fn(5) == 11);



 Dec 22 2016



 next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Thursday, 22 December 2016 at 12:47:12 UTC, Stefan Koch wrote:

 Hey Guys,

 Pointers support is coming!


Awesome!

Are there any D language features working in newCTFE that doesn't 
work on current upstream master?



 Dec 22 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 22 December 2016 at 15:21:08 UTC, Nordlöw wrote:

 On Thursday, 22 December 2016 at 12:47:12 UTC, Stefan Koch 
 wrote:

 Hey Guys,

 Pointers support is coming!


 Awesome!

 Are there any D language features working in newCTFE that 
 doesn't work on current upstream master?


None at the moment, the current work is aimed to provide a fast 
replacement for the engine. It does not aim to add any 
functionality to CTFE.
At least not yet.
I am planning to support closures with multiple contexts at CTFE, 
but that can only be worked on after everything else is working.



 Dec 22 2016



prev sibling  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


I just fixed the "" case for StringLiterals.
However the fix is quite hackish and needs to be revisited soon.
Furthermore I have made a few more preparations for function call 
support.

The reason why I am a bit reculant to enable function calls is 
the vast increase of bug-attack-surface.

Also there has been a slight speed-increase in the interpreter 
due to the removal of stack-pointers which were a left-over from 
the time where the heap was unimplemented.

I may restore the llvm-backend temporarily to verify the 
function-call handling (one I am ready to enable it).



 Dec 22 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 22 December 2016 at 20:34:50 UTC, Stefan Koch wrote:

 I just fixed the "" case for StringLiterals.


This exposed bugs in the code-generator because now 
phobos-unittests with "" are run.
(Which would previously be counted as uncompilable)
Apparently we run into a situation where the interpreter tries to 
fetch instructions that are out-of-range.
This can only be caused by miscompiled code.
I am investigating.



 Dec 23 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 23 December 2016 at 10:25:57 UTC, Stefan Koch wrote:

 On Thursday, 22 December 2016 at 20:34:50 UTC, Stefan Koch 
 wrote:

 I just fixed the "" case for StringLiterals.


 This exposed bugs in the code-generator because now 
 phobos-unittests with "" are run.
 (Which would previously be counted as uncompilable)
 Apparently we run into a situation where the interpreter tries 
 to fetch instructions that are out-of-range.
 This can only be caused by miscompiled code.
 I am investigating.


Seems like that one was triggerd by random garbage being written 
into empty strings :)
Empty string literals are now fixed for good.



 Dec 23 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 23 December 2016 at 11:31:14 UTC, Stefan Koch wrote:

 On Friday, 23 December 2016 at 10:25:57 UTC, Stefan Koch wrote:

 On Thursday, 22 December 2016 at 20:34:50 UTC, Stefan Koch 
 wrote:

 I just fixed the "" case for StringLiterals.


 This exposed bugs in the code-generator because now 
 phobos-unittests with "" are run.
 (Which would previously be counted as uncompilable)
 Apparently we run into a situation where the interpreter tries 
 to fetch instructions that are out-of-range.
 This can only be caused by miscompiled code.
 I am investigating.


 Seems like that one was triggerd by random garbage being 
 written into empty strings :)
 Empty string literals are now fixed for good.


I just made global enums and static immutables visible to newCTFE.
Before they would trigger a bailout since they are not defined in 
the functions stack-frame.

meaning this compiles now:
uint[] arrayLiteralReturn()
{
   return [1,2,3];
}
static immutable alr = arrayLiteralReturn();
static assert(alr == [1,2,3]);



 Dec 23 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


 static immutable alr = arrayLiteralReturn();
 static assert(alr == [1,2,3]);


There were a few problems with that.
Namely some arrays-literals would not have a length.
This is because of literals coercing to slices.
This bug is now fixed, at the expense of registering more types 
then is strictly necessary.
Also it creates null holes in the TypeArrays.
This may might be painful in the future.
But for now, global Array-literals work.



 Dec 25 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 25 December 2016 at 11:27:07 UTC, Stefan Koch wrote:

 static immutable alr = arrayLiteralReturn();
 static assert(alr == [1,2,3]);


 There were a few problems with that.
 Namely some arrays-literals would not have a length.
 This is because of literals coercing to slices.
 This bug is now fixed, at the expense of registering more types 
 then is strictly necessary.
 Also it creates null holes in the TypeArrays.
 This may might be painful in the future.
 But for now, global Array-literals work.


There are sill problems when you try to index those arrays for 
function calls.
This can be fixed by proving a different code-path.
If those arrays are indexed as function-parameters at the top 
level.



 Dec 25 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 25 December 2016 at 12:21:42 UTC, Stefan Koch wrote:

 On Sunday, 25 December 2016 at 11:27:07 UTC, Stefan Koch wrote:

 static immutable alr = arrayLiteralReturn();
 static assert(alr == [1,2,3]);


 There were a few problems with that.
 Namely some arrays-literals would not have a length.
 This is because of literals coercing to slices.
 This bug is now fixed, at the expense of registering more 
 types then is strictly necessary.
 Also it creates null holes in the TypeArrays.
 This may might be painful in the future.
 But for now, global Array-literals work.


 There are sill problems when you try to index those arrays for 
 function calls.
 This can be fixed by proving a different code-path.
 If those arrays are indexed as function-parameters at the top 
 level.


I reverted my ArrayLiteral fixes for now,
they broke the druntime build.



 Dec 25 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 25 December 2016 at 15:26:36 UTC, Stefan Koch wrote:


 I reverted my ArrayLiteral fixes for now,
 they broke the druntime build.


So far by attempts to fix this situation, proved fruitless.
However this getting global static variables to work correctly is 
a high-priority goal, as it is both important for the typical 
use-cases of ctfe and it affects the overall design of the engine.

I'll be taking a short rest during the next days, hopefully I 
will know how to move forward after that.

Also I need to prepare a talk-proposal for next dconf.



 Dec 27 2016



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Some progress on && has been made.
It can now be used within an if.
It is only enabled if debug = andnd is given.



 Dec 24 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 24 December 2016 at 16:10:04 UTC, Stefan Koch wrote:

 Some progress on && has been made.
 It can now be used within an if.
 It is only enabled if debug = andnd is given.


This introduced a bug which is now fixed.
Furthermore returning of 64bit integral values has now been 
enabled.
I just thought about a way to enable slices with less 
"CTFE-runtime" support.
This is a good thing since the CTFE runtime currently is made up 
of a few pretty long macros.
That bloat the bytecode beyond recognition.
As soon as call are supported they can be turned into 
function-calls.
However function calls are pretty expensive.
Therefore it would be good if lightweight slice handling would 
work.



 Dec 24 2016



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Hi Guys,

I just figured out why array constants did not work as function 
arguments.
It's because the array-constant undergoes a cast when used as 
slice, while an array literal can be taken as is.

The currently newCTFE does not really provide the capabilities to 
handle casts.
This is another issue that has to be addressed before 
function-calls come into play.

Within the engine dynamic and static arrays are represented in 
the same structure.
Therefore we don't need to take action on such a cast as long as 
the element types are the same.

While simply ignoring the cast-expression is correct in the case, 
I need to systemize the handling of casts better.



 Dec 29 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 29 December 2016 at 18:21:30 UTC, Stefan Koch wrote:

 Hi Guys,

 I just figured out why array constants did not work as function 
 arguments.
 It's because the array-constant undergoes a cast when used as 
 slice, while an array literal can be taken as is.

 The currently newCTFE does not really provide the capabilities 
 to handle casts.
 This is another issue that has to be addressed before 
 function-calls come into play.

 Within the engine dynamic and static arrays are represented in 
 the same structure.
 Therefore we don't need to take action on such a cast as long 
 as the element types are the same.

 While simply ignoring the cast-expression is correct in the 
 case, I need to systemize the handling of casts better.


This is now partially fixed.

Also all the latest work was merged in the feature branch.

The current timeline is still the same, I suspect that newCTFE 
will be usable in February.



 Dec 30 2016



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


global immutable integer Arrays are now completely functional!

This makes the following code work:
uint echo (uint val)
{
     return val;
}

const(uint) fastLog10(const uint val) pure nothrow  nogc {
     return      (val < 10) ? 0 :
                 (val < 100) ? 1 :
                 (val < 1000) ? 2 :
                 (val < 10000) ? 3 :
                 (val < 100000) ? 4 :
                 (val < 1000000) ? 5 :
                 (val < 10000000) ? 6 :
                 (val < 100000000) ? 7 :
                 (val < 1000000000) ? 8 : 9;
}

/* unique*/ static immutable uint[10] fastPow10tbl = [
         1,
         10,
         100,
         1000,
         10000,
         100000,
         1000000,
         10000000,
         100000000,
         1000000000,
] ;

static assert(fastLog10(fastPow10tbl[2]) == 2);
static assert(fastLog10(fastPow10tbl[3]) == 3);
static assert(fastLog10(1000) == 3);

static assert(echo(fastPow10tbl[1]) == 10);



 Dec 31 2016



  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Saturday, 31 December 2016 at 16:32:42 UTC, Stefan Koch wrote:

 global immutable integer Arrays are now completely functional!


Great work!


 /* unique*/ static immutable uint[10] fastPow10tbl = [


BTW: What role does  unique play here?



 Dec 31 2016



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 31 December 2016 at 17:12:22 UTC, Nordlöw wrote:

 On Saturday, 31 December 2016 at 16:32:42 UTC, Stefan Koch 
 wrote:

 global immutable integer Arrays are now completely functional!


 Great work!


 /* unique*/ static immutable uint[10] fastPow10tbl = [


 BTW: What role does  unique play here?


No role at all. It's a comment.

I have these comments in my tests, to remind myself of certain 
properties.
That, when properly detected, can lead to better generated code.



 Dec 31 2016



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


So guys.

basic function-calls are supported.

Hell YEAH!



 Jan 02 2017



 next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 2 January 2017 at 18:40:44 UTC, Stefan Koch wrote:

 So guys.

 basic function-calls are supported.

 Hell YEAH!


Meaning this will compile :

uint caller(uint n)
{
   return callee(n, 2);
}

uint callee(uint a, uint b)
{
   return a*b;
}

static assert(caller(3) == 6);
static assert(caller(24) == 48);


uint rfn(uint n)
{
   if (n) return n + rfn(n-1);
   return 1;
}

pragma(msg, rfn(2000));



 Jan 02 2017



  parent reply Meta <jared771 gmail.com>  writes:


On Monday, 2 January 2017 at 19:01:43 UTC, Stefan Koch wrote:

 On Monday, 2 January 2017 at 18:40:44 UTC, Stefan Koch wrote:

 So guys.

 basic function-calls are supported.

 Hell YEAH!


 Meaning this will compile :

 uint caller(uint n)
 {
   return callee(n, 2);
 }

 uint callee(uint a, uint b)
 {
   return a*b;
 }

 static assert(caller(3) == 6);
 static assert(caller(24) == 48);


 uint rfn(uint n)
 {
   if (n) return n + rfn(n-1);
   return 1;
 }

 pragma(msg, rfn(2000));


There's something I've been wondering about. NewCTFE is basically 
just another backend, right? Does this mean that you'll have to 
implement various optimizations such as tail-call elision 
independently from what DMD with the default backend currently 
does?



 Jan 02 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 2 January 2017 at 21:52:05 UTC, Meta wrote:

 There's something I've been wondering about. NewCTFE is 
 basically just another backend, right? Does this mean that 
 you'll have to implement various optimizations such as 
 tail-call elision independently from what DMD with the default 
 backend currently does?


Yes, I could borrow analysis passes from dmd however to check if 
the optimization is applicable.

However I do not intend to do so.
CTFE is mostly used for small one-shot things.
There is no point in eliding calls.
Executing the code as soon as possible is often faster then doing 
the optimization.



 Jan 02 2017



prev sibling next sibling parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Monday, 2 January 2017 at 18:40:44 UTC, Stefan Koch wrote:

 Hell YEAH!


Nice work.



 Jan 02 2017



prev sibling  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 2 January 2017 at 18:40:44 UTC, Stefan Koch wrote:

 So guys.

 basic function-calls are supported.

 Hell YEAH!


Disabled for now. Since it pushes some buffers/caches to their 
limit. A more flexible solution is needed.
However the good news is that the ABI seems to work better than 
expected.
Now it's time to deal with this-pointers and method calls.

After remaining issues with nested switches have to be fixed.



 Jan 04 2017



 next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 5 January 2017 at 06:04:00 UTC, Stefan Koch wrote:

 On Monday, 2 January 2017 at 18:40:44 UTC, Stefan Koch wrote:

 So guys.

 basic function-calls are supported.

 Hell YEAH!


 Disabled for now. Since it pushes some buffers/caches to their 
 limit. A more flexible solution is needed.


This is now fixed by simply extending the limits.
Now more failures start to show and more manual bailouts are 
required again.

Although, I expected this to happen it bums me. :(



 Jan 08 2017



prev sibling  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 5 January 2017 at 06:04:00 UTC, Stefan Koch wrote:

 On Monday, 2 January 2017 at 18:40:44 UTC, Stefan Koch wrote:

 So guys.

 basic function-calls are supported.

 Hell YEAH!


 Disabled for now. Since it pushes some buffers/caches to their 
 limit. A more flexible solution is needed.
 However the good news is that the ABI seems to work better than 
 expected.
 Now it's time to deal with this-pointers and method calls.

 After remaining issues with nested switches have to be fixed.


Basic Function call support reenabled.
Phobos unittests pass.



 Jan 16 2017



 next sibling parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Monday, 16 January 2017 at 16:31:25 UTC, Stefan Koch wrote:

 Basic Function call support reenabled.
 Phobos unittests pass.


Nice!



 Jan 16 2017



prev sibling  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 16 January 2017 at 16:31:25 UTC, Stefan Koch wrote:

 On Thursday, 5 January 2017 at 06:04:00 UTC, Stefan Koch wrote:

 On Monday, 2 January 2017 at 18:40:44 UTC, Stefan Koch wrote:

 So guys.

 basic function-calls are supported.

 Hell YEAH!


 Disabled for now. Since it pushes some buffers/caches to their 
 limit. A more flexible solution is needed.
 However the good news is that the ABI seems to work better 
 than expected.
 Now it's time to deal with this-pointers and method calls.

 After remaining issues with nested switches have to be fixed.


 Basic Function call support reenabled.
 Phobos unittests pass.


Just found a bug with recursive functions, which is now fixed.
So time for some perf:
-- t.d
  uint rfn(uint n)
  {
    if (n > 2) return n + rfn(n-1) + rfn(n-2);
    return 1;
  }

pragma(msg, rfn(30));
--

uplink uplink-pc:~/d/dmd$ time src/dmd t.d -c
4674396u

real	0m0.396s
user	0m0.392s
sys	0m0.000s
uplink uplink-pc:~/d/dmd$ time dmd t.d -c
4674396u

real	0m1.993s
user	0m1.872s
sys	0m0.120s



 Jan 16 2017



 next sibling parent  Martin Nowak <code dawg.eu>  writes:


On Tuesday, 17 January 2017 at 06:36:58 UTC, Stefan Koch wrote:

 real	0m1.993s
 user	0m1.872s
 sys	0m0.120s


Just a tip, if you use (maybe install) /bin/time, you also get 
memory usage.

0.00user 0.00system 0:00.00elapsed 85%CPU (0avgtext+0avgdata 
6372maxresident)k
0inputs+0outputs (0major+361minor)pagefaults 0swaps



 Jan 17 2017



prev sibling  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 17 January 2017 at 06:36:58 UTC, Stefan Koch wrote:

 Just found a bug with recursive functions, which is now fixed.
 So time for some perf:


Impressive speedup! What remains? Are you adding more tests to 
the Phobos unittests?



 Jan 17 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 17 January 2017 at 20:06:46 UTC, Nordlöw wrote:

 On Tuesday, 17 January 2017 at 06:36:58 UTC, Stefan Koch wrote:

 Just found a bug with recursive functions, which is now fixed.
 So time for some perf:


 Impressive speedup! What remains? Are you adding more tests to 
 the Phobos unittests?


There are still features to add :

- floating point.
- UTF-de/re-encodung.
- slicing
- concat
- pointer/ref
- member-function calls

But after this is done all the 1.0 goals are reached.



 Jan 17 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 17 January 2017 at 21:47:51 UTC, Stefan Koch wrote:

 On Tuesday, 17 January 2017 at 20:06:46 UTC, Nordlöw wrote:

 On Tuesday, 17 January 2017 at 06:36:58 UTC, Stefan Koch wrote:

 Just found a bug with recursive functions, which is now fixed.
 So time for some perf:


 Impressive speedup! What remains? Are you adding more tests to 
 the Phobos unittests?


 There are still features to add :

 - floating point.
 - UTF-de/re-encodung.
 - slicing
 - concat
 - pointer/ref
 - member-function calls

 But after this is done all the 1.0 goals are reached.


Forget something,
&& and || also needs to be implemented.
As well as various interactions of pointers/arrays/slices and 
structures.



 Jan 17 2017



  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 17 January 2017 at 21:50:42 UTC, Stefan Koch wrote:

 There are still features to add :

 - floating point.
 - UTF-de/re-encodung.
 - slicing
 - concat
 - pointer/ref
 - member-function calls

 But after this is done all the 1.0 goals are reached.


 Forget something,
 && and || also needs to be implemented.
 As well as various interactions of pointers/arrays/slices and 
 structures.


So I reckon the Phobos unittests are quite incomplete, then?



 Jan 17 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 17 January 2017 at 22:06:47 UTC, Nordlöw wrote:

 So I reckon the Phobos unittests are quite incomplete, then?


Not really, I still use a fallback to the old interpreter for the 
unimplemented features.



 Jan 17 2017



  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 17 January 2017 at 22:11:21 UTC, Stefan Koch wrote:

 Not really, I still use a fallback to the old interpreter for 
 the unimplemented features.


Does this mean that the newCTFE branch is usable now? And that I 
get performance improvements for the expression nodes you have 
implemented in newCTFE but not for the others?



 Jan 17 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 17 January 2017 at 22:19:39 UTC, Nordlöw wrote:

 On Tuesday, 17 January 2017 at 22:11:21 UTC, Stefan Koch wrote:

 Not really, I still use a fallback to the old interpreter for 
 the unimplemented features.


 Does this mean that the newCTFE branch is usable now? And that 
 I get performance improvements for the expression nodes you 
 have implemented in newCTFE but not for the others?


Exactly. Modulo bugs of course.
If you have code gives you a different output with my engine then 
with the old one, please open an issue in my github repo.



 Jan 17 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


Hi Guys,

I just enabled the handling for with(Type).
makes the following code work :

import core.time;
import core.sys.linux.time;

  auto _posixClock(ClockType clockType)
     {
             with(ClockType) final switch(clockType)
             {
             case bootTime: return CLOCK_BOOTTIME;
             case coarse: return CLOCK_MONOTONIC_COARSE;
             case normal: return CLOCK_MONOTONIC;
             case precise: return CLOCK_MONOTONIC;
             case processCPUTime: return CLOCK_PROCESS_CPUTIME_ID;
             case raw: return CLOCK_MONOTONIC_RAW;
             case threadCPUTime: return CLOCK_THREAD_CPUTIME_ID;
             case second: assert(0);
             }
}

static assert(_posixClock(ClockType.bootTime) == CLOCK_BOOTTIME);



 Jan 03 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


I have fixed a few bugs in the handling of casts.
And fixed an oversight which would cause Xors to bailout.
However Phobos-unittests are still regressed.

I am now working on Method-Calls or member-function-calls.



 Jan 09 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Druntime and phobos broke, the reason being that,
I did test against an outdated version of druntime/phobos.

I am working on getting them to compile again.
Adding features is stalled until then.



 Jan 20 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 20 January 2017 at 19:08:28 UTC, Stefan Koch wrote:

 Druntime and phobos broke, the reason being that,
 I did test against an outdated version of druntime/phobos.

 I am working on getting them to compile again.
 Adding features is stalled until then.


Phobos and druntime unittests pass again.

this was _NOT_ a proper fix.
only a really crude bailout when we detect that we jump to code 
that does not exist.



 Jan 20 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 21 January 2017 at 05:59:32 UTC, Stefan Koch wrote:

 On Friday, 20 January 2017 at 19:08:28 UTC, Stefan Koch wrote:

 Druntime and phobos broke, the reason being that,
 I did test against an outdated version of druntime/phobos.

 I am working on getting them to compile again.
 Adding features is stalled until then.


 Phobos and druntime unittests pass again.

 this was _NOT_ a proper fix.
 only a really crude bailout when we detect that we jump to code 
 that does not exist.


The cause seems to be fixed now.



 Jan 21 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


I just fixed a bug related to the unsupported floating point.
~master phobos compiles again and runs the unit-tests.

I will probably have to spent the rest of the month with fixing 
bugs that function-call support has uncovered.

Since now newCTFE sees much more code before bailing out.

I have also determined the cause of most of the miscompiled code.
The cause is the incorrect treatment of utf8.
(currently the evaluator really just does (dchar c= (_c & 7f))
which leads to corrupted strings in mixins;
and that has pretty crazy effects.

It was very hard to find this.



 Jan 22 2017



  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Monday, 23 January 2017 at 07:57:35 UTC, Stefan Koch wrote:

 It was very hard to find this.


Good job.



 Jan 23 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 23 January 2017 at 09:06:49 UTC, Nordlöw wrote:

 Good job.


There are more good news

uint fn()
{
   uint x = 7;
   modx(&x);
   return x;
}

void modx(uint* x)
{
     *x = 12;
}

static assert(fn() == 12);

This code does now compile with newCTFE :)



 Jan 23 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


interpret3.d passes!!!

The only remaining issues that cause miscompiled code are 
UTF(8/16/32) encoding and decoding issues.



 Jan 23 2017



 next sibling parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Monday, 23 January 2017 at 17:42:00 UTC, Stefan Koch wrote:

 interpret3.d passes!!!

 The only remaining issues that cause miscompiled code are 
 UTF(8/16/32) encoding and decoding issues.


Wow! Getting close know!



 Jan 23 2017



prev sibling next sibling parent  Dmitry Olshansky <dmitry.olsh gmail.com>  writes:


On 1/23/17 6:42 PM, Stefan Koch wrote:

 interpret3.d passes!!!


Now that's something!


 The only remaining issues that cause miscompiled code are UTF(8/16/32)
 encoding and decoding issues.


---
Dmitry Olshansky



 Jan 23 2017



prev sibling next sibling parent  Daniel =?UTF-8?B?S296w6Fr?= via Digitalmars-d  writes:


V Mon, 23 Jan 2017 17:42:00 +0000
Stefan Koch via Digitalmars-d <digitalmars-d puremagic.com> napsáno:


 interpret3.d passes!!!
 
 The only remaining issues that cause miscompiled code are 
 UTF(8/16/32) encoding and decoding issues.


WOW



 Jan 23 2017



prev sibling  parent reply David Nadlinger <code klickverbot.at>  writes:


On Monday, 23 January 2017 at 17:42:00 UTC, Stefan Koch wrote:

 interpret3.d passes!!!

 The only remaining issues that cause miscompiled code are 
 UTF(8/16/32) encoding and decoding issues.


Is that without bailing out?

  — David



 Jan 23 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 23 January 2017 at 22:09:27 UTC, David Nadlinger wrote:

 On Monday, 23 January 2017 at 17:42:00 UTC, Stefan Koch wrote:

 interpret3.d passes!!!

 The only remaining issues that cause miscompiled code are 
 UTF(8/16/32) encoding and decoding issues.


 Is that without bailing out?

  — David


No I still bail a lot :).
And I will be for some time, until I have written a 
memory-manager and various other runtime-libarary equivalents 
which are required for CTFE.

As well as my own soft-floating point library to ensure proper 
output during cross-compilation.

However this is an important milestone because it can now 
magically speed up a subset of ctfe without breaking things...
modulo bugs not covered by the unittests.



 Jan 23 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


The blacklisted functions are now down to only two.

Those are the bitswap function in druntime.
Because the interpreter does handle all values as 64bit 
internally it tends to miscompile code that uses xor on 32bit 
values.

And the second one is the "to" template from std.conv;
Because of the aforementioned UTF issues.

NOTE: that the blacklist is by nature transitive meaning a 
function that calls a blacklisted function will also not be 
interpreted by newCTFE



 Jan 23 2017



 next sibling parent reply John Colvin <john.loughran.colvin gmail.com>  writes:


On Tuesday, 24 January 2017 at 06:35:56 UTC, Stefan Koch wrote:

 And the second one is the "to" template from std.conv;
 Because of the aforementioned UTF issues.

 NOTE: that the blacklist is by nature transitive meaning a 
 function that calls a blacklisted function will also not be 
 interpreted by newCTFE


`to` is so common that I suspect you'll see a wave of bugs when 
you fix the UTF issues, yes?

Either way, thanks for spending so much time on this, I'm excited 
about these improvements both from a professional and personal 
standpoint.



 Jan 24 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 24 January 2017 at 09:27:52 UTC, John Colvin wrote:

 On Tuesday, 24 January 2017 at 06:35:56 UTC, Stefan Koch wrote:

 And the second one is the "to" template from std.conv;
 Because of the aforementioned UTF issues.

 NOTE: that the blacklist is by nature transitive meaning a 
 function that calls a blacklisted function will also not be 
 interpreted by newCTFE


 `to` is so common that I suspect you'll see a wave of bugs when 
 you fix the UTF issues, yes?

 Either way, thanks for spending so much time on this, I'm 
 excited about these improvements both from a professional and 
 personal standpoint.


NEW CTFE IS GREEN ON 64 BIT!
GREEN!



 Jan 24 2017



 next sibling parent reply Claude <no no.no>  writes:


On Tuesday, 24 January 2017 at 09:52:13 UTC, Stefan Koch wrote:

 NEW CTFE IS GREEN ON 64 BIT!
 GREEN!


I felt first a bit surprised about that "green" thing, but 
somehow if the new-ctfe engine consumes less CPU, it means it 
consumes less power, and therefore less carbone dioxyde is 
released into the atmosphere.
So indeed, we can assert that new-ctfe is eco-friendly!

Good job! :)

I really enjoy reading this thread everyday. It's kind of like a 
Dungeon & Dragon novel where the hero slashes its way through 
monsters, and bugs and features, up to the Holy Grail: the 
New-CTFE Engine implementation!

This is awesome work! Thanks.



 Jan 24 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 24 January 2017 at 10:34:58 UTC, Claude wrote:

 On Tuesday, 24 January 2017 at 09:52:13 UTC, Stefan Koch wrote:

 NEW CTFE IS GREEN ON 64 BIT!
 GREEN!


 I felt first a bit surprised about that "green" thing, but 
 somehow if the new-ctfe engine consumes less CPU, it means it 
 consumes less power, and therefore less carbone dioxyde is 
 released into the atmosphere.
 So indeed, we can assert that new-ctfe is eco-friendly!


Green means it passes all tests on auto-tester.



 Jan 24 2017



 next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 24 January 2017 at 11:01:46 UTC, Stefan Koch wrote:

 Green means it passes all tests on auto-tester.


Wonderful! Time to try out Pegged with newCTFE, then!



 Jan 24 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 24 January 2017 at 12:43:28 UTC, Nordlöw wrote:

 On Tuesday, 24 January 2017 at 11:01:46 UTC, Stefan Koch wrote:

 Green means it passes all tests on auto-tester.


 Wonderful! Time to try out Pegged with newCTFE, then!


There will be NO permanence improvement for pegged.
For the 7th time.

It is bogged down by templates, ctfe is not the problem there.



 Jan 24 2017



  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 24 January 2017 at 14:16:10 UTC, Stefan Koch wrote:

 There will be NO permanence improvement for pegged.
 For the 7th time.

 It is bogged down by templates, ctfe is not the problem there.


Sorry for that. Got it.

What's the essence of this suboptimal performance when using 
templates?

Non-O(1) lookup times?

Mangling?



 Jan 24 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 24 January 2017 at 15:16:29 UTC, Nordlöw wrote:

 On Tuesday, 24 January 2017 at 14:16:10 UTC, Stefan Koch wrote:

 There will be NO permanence improvement for pegged.
 For the 7th time.

 It is bogged down by templates, ctfe is not the problem there.


 Sorry for that. Got it.

 What's the essence of this suboptimal performance when using 
 templates?

 Non-O(1) lookup times?

 Mangling?


The lookup time is one issue but not the most important,
The instanciation of a template is a non-trivial operation within 
the compiler.
Type-deduction ast rewriting expression expansion
all of that takes time and a huge chunk at that.
Also binary code needs to be generated and that code is generated 
for every instance regardless of how similar they are.



 Jan 24 2017



  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 24 January 2017 at 15:33:37 UTC, Stefan Koch wrote:

 The lookup time is one issue but not the most important,
 The instanciation of a template is a non-trivial operation 
 within the compiler.
 Type-deduction ast rewriting expression expansion
 all of that takes time and a huge chunk at that.


Interesting. Is that the major cause of high memory usage in DMD?


 Also binary code needs to be generated and that code is 
 generated for every instance regardless of how similar they are.


Thanks.



 Jan 24 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 24 January 2017 at 15:48:19 UTC, Nordlöw wrote:

 On Tuesday, 24 January 2017 at 15:33:37 UTC, Stefan Koch wrote:

 The lookup time is one issue but not the most important,
 The instanciation of a template is a non-trivial operation 
 within the compiler.
 Type-deduction ast rewriting expression expansion
 all of that takes time and a huge chunk at that.


 Interesting. Is that the major cause of high memory usage in 
 DMD?


 Also binary code needs to be generated and that code is 
 generated for every instance regardless of how similar they 
 are.


 Thanks.


I would have to investigate the specific case.
It's also because the recursive nature of templates that you end 
up with +100_000 instances

It those numbers even small allocations have a huge impact.



 Jan 24 2017



prev sibling  parent  Claude <no no.no>  writes:


On Tuesday, 24 January 2017 at 11:01:46 UTC, Stefan Koch wrote:

 Green means it passes all tests on auto-tester.


Yes, I understood. I was playing with the idea of 
eco-friendliness as well... :)



 Jan 24 2017



prev sibling  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 24 January 2017 at 09:52:13 UTC, Stefan Koch wrote:

 NEW CTFE IS GREEN ON 64 BIT!
 GREEN!



I just fixed a regression found by compiling vibe.d.
We are now even greener!



 Jan 24 2017



prev sibling  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 24 January 2017 at 06:35:56 UTC, Stefan Koch wrote:

 The blacklisted functions are now down to only two.

 Those are the bitswap function in druntime.
 Because the interpreter does handle all values as 64bit 
 internally it tends to miscompile code that uses xor on 32bit 
 values.

 And the second one is the "to" template from std.conv;
 Because of the aforementioned UTF issues.

 NOTE: that the blacklist is by nature transitive meaning a 
 function that calls a blacklisted function will also not be 
 interpreted by newCTFE


The blacklisting of bitswap was removed, because we now bailout 
on bit-level operations on 32bit values.

However I did have to add a unittest-specific bailout for 
fail_compilation/fail14304.d
which requires the ctfe interpeter to detect a semantically 
invalid situation that should have been detected before we ever 
begin ctfe.



 Jan 25 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


newCTFE is green now on all platforms!



 Jan 25 2017



 next sibling parent  deadalnix <deadalnix gmail.com>  writes:


On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch wrote:

 newCTFE is green now on all platforms!


<3



 Jan 25 2017



prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com>  writes:


On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch wrote:

 newCTFE is green now on all platforms!


Congrats.

How much work remains?



 Jan 25 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 25 January 2017 at 15:26:39 UTC, jmh530 wrote:

 On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch 
 wrote:

 newCTFE is green now on all platforms!


 Congrats.

 How much work remains?


Quite a lot.

- Slicing
- Appending
- ||
- String support (implying utf conversions)
- Floating-Point

- classes
- exceptions
- function pointers
- closures

and

proper handling of more complex types
such as arrays of structures



 Jan 25 2017



  parent reply Dmitry Olshansky <dmitry.olsh gmail.com>  writes:


On 1/25/17 4:44 PM, Stefan Koch wrote:

 On Wednesday, 25 January 2017 at 15:26:39 UTC, jmh530 wrote:

 On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch wrote:

 newCTFE is green now on all platforms!


 Congrats.

 How much work remains?


 Quite a lot.

 - Slicing
 - Appending
 - ||
 - String support (implying utf conversions)
 - Floating-Point

 - classes
 - exceptions
 - function pointers
 - closures

 and

 proper handling of more complex types
 such as arrays of structures


Will note that std.regex will requires at least basic support for 
classes (construct and return from a function) and slicing.

---
Dmitry Olshansky



 Jan 29 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 29 January 2017 at 13:21:47 UTC, Dmitry Olshansky 
wrote:

 On 1/25/17 4:44 PM, Stefan Koch wrote:

 On Wednesday, 25 January 2017 at 15:26:39 UTC, jmh530 wrote:

 On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch 
 wrote:

 newCTFE is green now on all platforms!


 Congrats.

 How much work remains?


 Quite a lot.

 - Slicing
 - Appending
 - ||
 - String support (implying utf conversions)
 - Floating-Point

 - classes
 - exceptions
 - function pointers
 - closures

 and

 proper handling of more complex types
 such as arrays of structures


 Will note that std.regex will requires at least basic support 
 for classes (construct and return from a function) and slicing.

 ---
 Dmitry Olshansky


I can support non-polymorphic classes in near future (mid of 
march ?).
Slicing needs to be considered together with concatenation 
esentially it will require a memory-manager.
I still need to figure out if druntime could be reused here.



 Jan 29 2017



prev sibling next sibling parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch wrote:

 newCTFE is green now on all platforms!


Great!



 Jan 25 2017



prev sibling  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch wrote:

 newCTFE is green now on all platforms!


I just found an interesting bug just now.
The following code would cause newCTFE to segfault.

char* initPtr()
{
     return cast(char*) size_t.max;
}

static assert(cast(size_t)initPtr() == size_t.max);

Because it would try to return the char-value at heap-address 
size_t.max;
newCTFE allocates a 16MB heap by default and therefore the 
address was hopelessly out-of-bounds.

The reason this was not detected before is because:
   `cast(char*) size_t.max;`
does not actually produce a cast expression.
But rather an IntegerLiteral with the type char*.

After dealing with dmd for 7 months, I could figure out this bug 
in a day.
If I had run into this earlier I would have been stuck for weeks 
:)



 Jan 25 2017



  parent reply Patrick Schluter <Patrick.Schluter bbox.fr>  writes:


On Thursday, 26 January 2017 at 07:27:13 UTC, Stefan Koch wrote:

 On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch 
 wrote:

 newCTFE is green now on all platforms!


 I just found an interesting bug just now.
 The following code would cause newCTFE to segfault.

 char* initPtr()
 {
     return cast(char*) size_t.max;
 }

 static assert(cast(size_t)initPtr() == size_t.max);

 Because it would try to return the char-value at heap-address 
 size_t.max;
 newCTFE allocates a 16MB heap by default and therefore the 
 address was hopelessly out-of-bounds.

 The reason this was not detected before is because:
   `cast(char*) size_t.max;`
 does not actually produce a cast expression.
 But rather an IntegerLiteral with the type char*.

 After dealing with dmd for 7 months, I could figure out this 
 bug in a day.
 If I had run into this earlier I would have been stuck for 
 weeks :)


I do not understand why it should dereference the pointer? Do you 
mean that the bug was that newCTFE dereferenced but it shouldn't 
or that oldCTFE didn't dereference but it should have?



 Jan 26 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 26 January 2017 at 11:59:48 UTC, Patrick Schluter 
wrote:

 On Thursday, 26 January 2017 at 07:27:13 UTC, Stefan Koch wrote:

 On Wednesday, 25 January 2017 at 12:36:02 UTC, Stefan Koch 
 wrote:
 char* initPtr()
 {
     return cast(char*) size_t.max;
 }

 static assert(cast(size_t)initPtr() == size_t.max);

 Because it would try to return the char-value at heap-address 
 size_t.max;
 newCTFE allocates a 16MB heap by default and therefore the 
 address was hopelessly out-of-bounds.


 I do not understand why it should dereference the pointer? Do 
 you mean that the bug was that newCTFE dereferenced but it 
 shouldn't or that oldCTFE didn't dereference but it should have?


newCTFE should not dereference that pointer.

However since all other pointers need to be dereferenced, this 
bug came totally out of the blue.



 Jan 26 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


Member-Function calls work now!

this code will compile and properly execute with newCTFE:

struct iota_range
{
   int current;
   int end;
   int step;

   uint front()
   {
      return current;
   }

   void popFront()
   {
     current += step;
     // we need this return otherwise we generate invalid code :(
     return ;
   }

   bool empty()
   {
     return current > end;
   }

   this(uint end, uint begin = 0, uint step = 1) pure
   {
     assert(step != 0, "cannot have a step of 0");
     this.step = step;
     this.current = begin;
     this.end = end;
   }
}

auto Iota(int end)
{
   return iota_range(end);
}

uint testThisCall(uint end)
{
   uint result;

   foreach(n;Iota(end))
   {
     result += n;
   }

   return result;
}


static assert(testThisCall(12) == 78);


Neat, huh ?



 Jan 26 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


I just fixed the void initialization fail compilation tests.
By detecting the possibility of returning a void initialized 
field and bailing out.
dmd testsuite does now compile and pass(!) again.

I also fixed an obscure bug related to the recently introduced 
conecpt of heap references.



 Jan 28 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 28 January 2017 at 19:08:42 UTC, Stefan Koch wrote:

 I just fixed the void initialization fail compilation tests.
 By detecting the possibility of returning a void initialized 
 field and bailing out.
 dmd testsuite does now compile and pass(!) again.

 I also fixed an obscure bug related to the recently introduced 
 conecpt of heap references.


Druntime and phobos compile again and pass their respective 
unittests.

I also fixed a little performace bottleneck in the function-call 
code.
Although it is still far from optimal and can still be speed-up 
significantly.

However I would like to get more feature completeness before I am 
fixing anymore perf issues, except when they are jumping in my 
face :)

Also my ctfe engine still requires utf8 support, for 
string-foreach.
Currently there are methods for that in druntime, I hope to 
simply call them at ctfe, rather then re-implement them. If 
someone volunteers I can provide help to make them ctfeable, 
however at the moment my free mental bandwidth is too low to deal 
with utf details.

This approach will require the druntime-source containing the 
utf8 methods to be present and visible at every compiler run that 
uses string foreach.

If you consider this an issue please state why.



 Jan 28 2017



  parent reply pineapple <meapineapple gmail.com>  writes:


On Sunday, 29 January 2017 at 02:17:12 UTC, Stefan Koch wrote:

 Also my ctfe engine still requires utf8 support, for 
 string-foreach.
 Currently there are methods for that in druntime, I hope to 
 simply call them at ctfe, rather then re-implement them. If 
 someone volunteers I can provide help to make them ctfeable, 
 however at the moment my free mental bandwidth is too low to 
 deal with utf details.

 This approach will require the druntime-source containing the 
 utf8 methods to be present and visible at every compiler run 
 that uses string foreach.

 If you consider this an issue please state why.


1:

I think it's good to avoid this if possible, though I'm not 
fiercely opposed

2:

I wrote this for mach:

https://github.com/pineapplemachine/mach.d/tree/master/mach/text/utf

If you want to borrow code or if I can provide an implementation 
meeting different requirements then I am at your disposal.

3:

Can you clarify what string-foreach refers to? Surely not 
`foreach(ch; some_string){}`, which enumerates code units and not 
code points?



 Jan 28 2017



 next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 29 January 2017 at 02:46:16 UTC, pineapple wrote:

 On Sunday, 29 January 2017 at 02:17:12 UTC, Stefan Koch wrote:

 Also my ctfe engine still requires utf8 support, for 
 string-foreach.
 [ ... ]





 Can you clarify what string-foreach refers to? Surely not 
 `foreach(ch; some_string){}`, which enumerates code units and 
 not code points?


Yes exactly that. many times in phobos foreach(dchar ch; 
some_string)
which requires me to encode the utf8 string temporarily into utf32
and then when it is appending to some other string I need to 
reencode it into utf8.



 Jan 28 2017



 next sibling parent  pineapple <meapineapple gmail.com>  writes:


On Sunday, 29 January 2017 at 02:52:51 UTC, Stefan Koch wrote:

 Yes exactly that. many times in phobos foreach(dchar ch; 
 some_string)
 which requires me to encode the utf8 string temporarily into 
 utf32
 and then when it is appending to some other string I need to 
 reencode it into utf8.


Oooh, I never realized foreach acted specially like that. I can't 
decide whether I approve or disapprove.

You'll also have to support UTF-16 for the case `foreach(wchar 
ch; some_string){}`.

As before just let me know if I can help.



 Jan 29 2017



prev sibling  parent reply Dmitry Olshansky <dmitry.olsh gmail.com>  writes:


On 1/29/17 3:52 AM, Stefan Koch wrote:

 On Sunday, 29 January 2017 at 02:46:16 UTC, pineapple wrote:

 On Sunday, 29 January 2017 at 02:17:12 UTC, Stefan Koch wrote:

 Also my ctfe engine still requires utf8 support, for string-foreach.
 [ ... ]





 Can you clarify what string-foreach refers to? Surely not `foreach(ch;
 some_string){}`, which enumerates code units and not code points?


 Yes exactly that. many times in phobos foreach(dchar ch; some_string)
 which requires me to encode the utf8 string temporarily into utf32
 and then when it is appending to some other string I need to reencode it
 into utf8.


Why encode the whole string to utf32? You can lower it to use 
std.utf.decode in a loop.

Essentially this:

foreach(dchar ch; some_string){ ... }

Becomes:

import std.utf;
size_t i=0;
while(i<some_string.length){
	dchar ch = decode(some_string, i);
	...
}

---
Dmitry Olshansky



 Jan 29 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 29 January 2017 at 13:18:11 UTC, Dmitry Olshansky 
wrote:

 Essentially this:

 foreach(dchar ch; some_string){ ... }

 Becomes:

 import std.utf;
 size_t i=0;
 while(i<some_string.length){
 	dchar ch = decode(some_string, i);
 	...
 }

 ---
 Dmitry Olshansky


Won't work right now, since ref is not supported.
Also I really want to minimize dependency and phobos.



 Jan 29 2017



prev sibling  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 29 January 2017 at 02:46:16 UTC, pineapple wrote:


 https://github.com/pineapplemachine/mach.d/tree/master/mach/text/utf

 If you want to borrow code or if I can provide an 
 implementation meeting different requirements then I am at your 
 disposal.


That code is a great help, understanding the issue, thanks!



 Jan 28 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


New bugs incoming:

uint fn(uint a)
{
     final switch(a)
     {
         case 1 : {

             while(a < 20)
//          bool whileCondition;
//            WhileBlockEvalCond :
//            whileCondition = (a < 20);
             {
//                WhileBlockBegin:
                 a++;
                 if (a == 17) break;
/*
                 WhlieBlockEnd:
                     if(!whileCondition)
                        goto WhileBlockAfter
                 // break seems to go here :-/
                 goto WhileBlockEvalCond;
                 // break should go here!
*/
             }
//             WhileBlockAfter:
                 return a;
         }
     }

     return 1;
}

pragma(msg, fn(1)); // should print 17
when executed with with newCTFE this will print 20



 Jan 30 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 30 January 2017 at 15:15:55 UTC, Stefan Koch wrote:

 New bugs incoming:

 uint fn(uint a)
 {
     final switch(a)
     {
         case 1 : {

             while(a < 20)
 //          bool whileCondition;
 //            WhileBlockEvalCond :
 //            whileCondition = (a < 20);
             {
 //                WhileBlockBegin:
                 a++;
                 if (a == 17) break;
 /*
                 WhlieBlockEnd:
                     if(!whileCondition)
                        goto WhileBlockAfter
                 // break seems to go here :-/
                 goto WhileBlockEvalCond;
                 // break should go here!
 */
             }
 //             WhileBlockAfter:
                 return a;
         }
     }

     return 1;
 }

 pragma(msg, fn(1)); // should print 17
 when executed with with newCTFE this will print 20


This bug is fixed now!!
it was a simple oversight were the mechanism I build to deal with 
this was simply not invoked.
Luckily not a failure of the mechanism itself.



 Jan 31 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


newCTFE is now green on all platforms on travis as well.

function pointer support are coming!
with it most of std.algorithm will be supported.



 Jan 31 2017



 next sibling parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 31 January 2017 at 16:21:27 UTC, Stefan Koch wrote:

 newCTFE is now green on all platforms on travis as well.

 function pointer support are coming!
 with it most of std.algorithm will be supported.


Cool!



 Jan 31 2017



prev sibling  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 31 January 2017 at 16:21:27 UTC, Stefan Koch wrote:

 function pointer support is coming!


This is more difficult then I expected,
since the argument handling system was build without considering 
that we could have functions as arguments, therefore when trying 
to evaluate the function-argument it will overwrite the 
data-structures that are supposed to hold the information for the 
function we currently processing the arguments for.

As you can imagine this bug took me two days to find :)

maybe this can be fixed by handling the arguments before handling 
the function itself.



 Feb 01 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 1 February 2017 at 11:53:09 UTC, Stefan Koch wrote:

 On Tuesday, 31 January 2017 at 16:21:27 UTC, Stefan Koch wrote:

 function pointer support is coming!


 This is more difficult then I expected,
 since the argument handling system was build without 
 considering that we could have functions as arguments, 
 therefore when trying to evaluate the function-argument it will 
 overwrite the data-structures that are supposed to hold the 
 information for the function we currently processing the 
 arguments for.

 maybe this can be fixed by handling the arguments before 
 handling the function itself.


I have not fixed the bug with top-level function arguments yet,
However I have enabled function-pointer support for non-toplevel 
use
such as in :
int[] filterBy(int[] arr , bool function(uint) fn)
{
     int[] result;
     uint resultLength;

     result.length = arr.length;
     foreach(i;0 .. arr.length)
     {
         auto e = arr[i];
         bool r = true;
         r = fn(e);
         if(r)
         {
             result[resultLength++] = e;
         }
     }

    int[] filterResult;
    filterResult.length = resultLength;

    foreach(i; 0 .. resultLength)
    {
      filterResult[i] = result[i];
    }

   return filterResult;
}


bool isDiv2(uint e)
{
   bool result_;
   result_ = (e % 2 == 0);
   return result_;
}

bool isNotDiv2(uint e)
{
   bool result_;
   result_ = (e % 2 != 0);
   return result_;
}

int[] run(int[] arr, bool div2)
{
   return filterBy(arr, div2 ? &isDiv2 : &isNotDiv2);
}


static assert(run([3,4,5], true) == [4]);
static assert(run([3,4,5], false) == [3,5]);

//static assert(filterBy([3,4,5], &isDiv2) == [4]);
// top-level function arguments do currently not work with newCTFE

Enjoy!



 Feb 03 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


std.conv.to is no longer blacklisted.
since I now bail on every string foreach.
As soon as ctfeable utf-conversion functions are in place, 
newCTFE can be used for fast low-memory overhead comptiletime 
parsing.



 Feb 04 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 4 February 2017 at 10:27:58 UTC, Stefan Koch wrote:

 std.conv.to is no longer blacklisted.
 since I now bail on every string foreach.
 As soon as ctfeable utf-conversion functions are in place, 
 newCTFE can be used for fast low-memory overhead comptiletime 
 parsing.


Also I just enabled &&.
It's still experimental but it seems to work sofar.



 Feb 04 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 4 February 2017 at 11:32:52 UTC, Stefan Koch wrote:

 On Saturday, 4 February 2017 at 10:27:58 UTC, Stefan Koch wrote:

 std.conv.to is no longer blacklisted.
 since I now bail on every string foreach.
 As soon as ctfeable utf-conversion functions are in place, 
 newCTFE can be used for fast low-memory overhead comptiletime 
 parsing.


 Also I just enabled &&.
 It's still experimental but it seems to work sofar.


experiment failed.
Not necessarily because AndAnd itself is wrong but rather because 
something that is now not bailed out on, puts us in a bad state.
I will disable && for the time being and continue working on the 
function argument problem.

I can present you a nice snippet that just compiled with newCTFE



 Feb 04 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 4 February 2017 at 16:30:30 UTC, Stefan Koch wrote:

 I can present you a nice snippet that just compiled with newCTFE


int[] makeAndInitArray(int until)
{
   int[] result;
   result.length = until;

   foreach(i; 0 .. until)
   {
     result[i] = i;
   }

   return result;
}


int[] filter(alias filterFn)(int[] arr)
{
   int[] result;
   result.length = arr.length;

   uint resultLength;
   foreach(i;0 .. arr.length)
   {
     auto e = arr[i];
     if (filterFn(e))
     {
       result[resultLength++] = e;
     }
   }

   result.length = resultLength;
   // I cannot remember implementing it but shrinking seems to work
   return result;
}
enum arr_500_000 = makeAndInitArray(500_000);

static assert(arr_500_000.filter!(e => !(e % 3)));
static assert([1,2,3,4,5,6,7,8,9,10,11,12,13].filter!(e => !(e % 
3)) == [3,6,9,12]);


newCTFE takes a quarter of the time, and only half of the memory.
It would go even faster if we did not create the enum arr_500_000 
but instead do that in a function-local variable.

because then the array does not allocate 500_000 expression nodes.



 Feb 04 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


Function pointers and other advanced features interfere with the 
ability to cache bytecode I just never noticed because the 
bytecode caching feature was disabled a long time.
Since I do not know how much of it is bitrot I removed it 
alltogether.
This feature will return and be more performant, when the outside 
function-arguments are getting fixed.
When this is done we can also remove a few special cases for the 
interpreter backend which is always good.



 Feb 08 2017



  parent reply Kagamin <spam here.lot>  writes:


Do I understand it right that union at compile time takes space 
equal to the sum of sizes of its members instead of the size of 
the largest member?



 Feb 08 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 8 February 2017 at 14:01:33 UTC, Kagamin wrote:

 Do I understand it right that union at compile time takes space 
 equal to the sum of sizes of its members instead of the size of 
 the largest member?


generally it takes as much space as the largest member.
Why do you ask ?
And in what context ?



 Feb 08 2017



  parent  Kagamin <spam here.lot>  writes:


On Wednesday, 8 February 2017 at 14:03:13 UTC, Stefan Koch wrote:

 generally it takes as much space as the largest member.
 Why do you ask ?
 And in what context ?


Asking about memory consumption.
I thought it allowed access to overlapped fields, now I see it 
doesn't.



 Feb 08 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


I just re-enabled << and >> for 32bit values, as they should not 
behave differently when applied to mixtures of 32bit and 64bit 
values.

Furthermore I fixed a few performance issues in the interaction 
between the interpreter and the bytecode-gen.

I also took the first steps for utf8/32 support.

32bit linux now fails on travis and the autotester.
I cannot reproduce this on my local setup therefore I deduce that 
it is a regression in the dmd version that is used on those 
platforms.



 Feb 09 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


I just encountered a familiar problem again ...

uint fn(uint a)
{
     final switch(a)
     {
         case 1 :
             do
             {
                 a++;
                 if (a == 17) break;
             } while(a < 20);
                 return a;

         case 2 : return 1;
     }
}

static assert(fn(1) == 17); // fails because the return is 20
static assert(fn(2) == 1);

It turns out I forgot use the facility I implemented to handle 
this on DoStatements as well.
This is now fixed.



 Feb 10 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


I am currently working on && again,
So far I hit a curious situation when handling
(b1 && b2 && b3)
everything works fine except that b3 will be evaluated 3 times.
It's a puzzler :)



 Feb 13 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 13 February 2017 at 12:26:14 UTC, Stefan Koch wrote:

 I am currently working on && again,
 So far I hit a curious situation when handling
 (b1 && b2 && b3)
 everything works fine except that b3 will be evaluated 3 times.
 It's a puzzler :)


I just reverted every change of last week I should have a clean 
building branch after that.
Tomorrow I will rewrite the removed functionality more cautiously.



 Feb 13 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 13 February 2017 at 21:09:40 UTC, Stefan Koch wrote:

 On Monday, 13 February 2017 at 12:26:14 UTC, Stefan Koch wrote:

 I am currently working on && again,
 So far I hit a curious situation when handling
 (b1 && b2 && b3)
 everything works fine except that b3 will be evaluated 3 times.
 It's a puzzler :)


 I just reverted every change of last week I should have a clean 
 building branch after that.
 Tomorrow I will rewrite the removed functionality more 
 cautiously.


I have now rebuild the new structures partially.
It turns out that I was missing a few bailouts.
However those should have been necessary before as well, so I am 
still puzzled.



 Feb 14 2017



  parent  Jacob Carlborg <doob me.com>  writes:


On 2017-02-14 10:51, Stefan Koch wrote:


 I have now rebuild the new structures partially.
 It turns out that I was missing a few bailouts.
 However those should have been necessary before as well, so I am still
 puzzled.


301 posts (302 with this one). Time for a new thread?

-- 
/Jacob Carlborg



 Feb 14 2017



prev sibling  parent  ketmar <ketmar ketmar.no-ip.org>  writes:


Jacob Carlborg wrote:

301 posts (302 with this one). Time for a new thread?


please, no! i just downloaded 4500+ posts only to get the root of this 
thread! don't turn it into useless effort!



 Feb 14 2017