www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - CTFE Status 2

reply Stefan Koch <uplink.coder googlemail.com> writes:
Hi Guys, due to the old CTFE status thread getting to page 30, I 
am now starting a new one.

First let me summerize which features are currently working:
In order of date, the latest features come first.

- fixed continue and break for DoWhileStatements
- non-toplevel Function Pointers as arguments
- basic function pointers
--- phobos, druntime & dmd unit-tests pass ----
- Method Calls (now range foreach will work)
- 1-level Pointers
- recursive function calls
- fixed continue and break for ForStatements (includes foreach 
which is lowered)
- basic Function Calls
- static immutable globals
- fixed continue & break for WhileStatemens
- 1d array literals
- ternary expressions ? :
- struct literals.
- basic struct support.
- Slice and Array boundschecks
- 1d array and slice indexing
- do while statements
- break and continue support
- gotos and labels
- foreach over arrays
- for statements
- integer math
.... (and I few a forgot to list)

unsupported features are
- anything with strings (due to missing unicode handling)
- slicing & concat (due to missing memeory-manager)
- floating point math
- classes (due to missing vtbl and ref support)
- exceptions (due to missing stack-unwinding support and 
side-band channels)
- more complex structs i.e. with struct-members (due to 
incomplete type-handling)
-----------

Currently I am having trouble caused by a bug in dmds inliner 
that only happens on when dmd is compiled as a 32bit executable 
until I have  isolated / fixed this development is slowed down.

--
I hope this thread is informative and will continue to be that 
way.

Cheers,
Stefan (aka UplinkCoder)
Feb 16 2017
next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 Cheers,
 Stefan (aka UplinkCoder)


Stupid question: is your online alias a reference to the game 
Uplink?
Feb 16 2017
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:19:14 UTC, Jack Stouffer 
wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 Cheers,
 Stefan (aka UplinkCoder)


 Stupid question: is your online alias a reference to the game 
 Uplink?


Yes it is.
Uplink Hackers Elite is a great game.
Feb 16 2017
prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Feb 16, 2017 at 09:05:51PM +0000, Stefan Koch via Digitalmars-d wrote:

 Hi Guys, due to the old CTFE status thread getting to page 30, I am now
 starting a new one.
 
 First let me summerize which features are currently working:
 In order of date, the latest features come first.
 
 - fixed continue and break for DoWhileStatements
 - non-toplevel Function Pointers as arguments
 - basic function pointers
 --- phobos, druntime & dmd unit-tests pass ----


[...]

This is great!  The fact that you got it to the point unittests pass is
awesome.  Can't wait for the new CTFE engine to be released!

Do you have any estimate as to when it will be mergeable into master?


T

-- 
There are three kinds of people in the world: those who can count, and those
who can't.
Feb 16 2017
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 23:16:34 UTC, H. S. Teoh wrote:

 On Thu, Feb 16, 2017 at 09:05:51PM +0000, Stefan Koch via 
 Digitalmars-d wrote:

 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.
 
 First let me summerize which features are currently working: 
 In order of date, the latest features come first.
 
 - fixed continue and break for DoWhileStatements
 - non-toplevel Function Pointers as arguments
 - basic function pointers
 --- phobos, druntime & dmd unit-tests pass ----


 [...]

 This is great!  The fact that you got it to the point unittests 
 pass is awesome.  Can't wait for the new CTFE engine to be 
 released!

 Do you have any estimate as to when it will be mergeable into 
 master?


 T


I estimate that newCTFE 1.0 as outlined in the vision document 
will be ready around April.

This estimate is a lower bound as it assumes that things go 
smoothly, and the past 6 months have shown that the CTFE-work 
does not go smoothly most of the time.
As an upper bound:
I have the personal goal to get it merged before DConf.

At this point most parts of the actual engine are done, the 
remaining work mostly involves either making the runtime ctfeable 
or building runtime-feature support into dmd.
These things are outside of my domain knowledge and therefore 
take much more time then they should.
Feb 16 2017
prev sibling next sibling parent Andrea Fontana <nospam example.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [...]
 I hope this thread is informative and will continue to be that 
 way.

 Cheers,
 Stefan (aka UplinkCoder)


Yes, it is!
Feb 17 2017
prev sibling next sibling parent Lurker <lurker gmail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 ...
 --
 I hope this thread is informative and will continue to be that 
 way.

 Cheers,
 Stefan (aka UplinkCoder)


Thanks Stefan for your work, and for documenting it. Very 
exciting to read the progress! I wish more people were as devoted 
as you are! Keep up the good work!
Feb 17 2017
prev sibling next sibling parent reply Temtaime <temtaime gmail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 First let me summerize which features are currently working:
 In order of date, the latest features come first.

 - fixed continue and break for DoWhileStatements
 - non-toplevel Function Pointers as arguments
 - basic function pointers
 --- phobos, druntime & dmd unit-tests pass ----
 - Method Calls (now range foreach will work)
 - 1-level Pointers
 - recursive function calls
 - fixed continue and break for ForStatements (includes foreach 
 which is lowered)
 - basic Function Calls
 - static immutable globals
 - fixed continue & break for WhileStatemens
 - 1d array literals
 - ternary expressions ? :
 - struct literals.
 - basic struct support.
 - Slice and Array boundschecks
 - 1d array and slice indexing
 - do while statements
 - break and continue support
 - gotos and labels
 - foreach over arrays
 - for statements
 - integer math
 .... (and I few a forgot to list)

 unsupported features are
 - anything with strings (due to missing unicode handling)
 - slicing & concat (due to missing memeory-manager)
 - floating point math
 - classes (due to missing vtbl and ref support)
 - exceptions (due to missing stack-unwinding support and 
 side-band channels)
 - more complex structs i.e. with struct-members (due to 
 incomplete type-handling)
 -----------

 Currently I am having trouble caused by a bug in dmds inliner 
 that only happens on when dmd is compiled as a 32bit executable 
 until I have  isolated / fixed this development is slowed down.

 --
 I hope this thread is informative and will continue to be that 
 way.

 Cheers,
 Stefan (aka UplinkCoder)


Just get LDC.
Make it use JIT.
And you'll get all the features working.
Writing slow interpreter is ... wasting efforts.
Feb 17 2017
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 17 February 2017 at 19:44:10 UTC, Temtaime wrote:

 Just get LDC.
 Make it use JIT.
 And you'll get all the features working.
 Writing slow interpreter is ... wasting efforts.


For your information LLVM takes about 5 milliseconds to start up, 
it also takes a lot of time to generate code even when completely 
unoptimized.
For usual one-shot CTFE functions this leads to a _HUGE_ 
pessimisation of performance.
For the quite common usecase of returning a literal out of 
function templates newCTFE takes UNDER a MICROsecond.
There is no way LLVM could even come close no matter how many 
caching layers one adds.
Feb 17 2017
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 2/17/17 8:44 PM, Temtaime wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 I hope this thread is informative and will continue to be that way.

 Cheers,
 Stefan (aka UplinkCoder)


 Just get LDC.
 Make it use JIT.
 And you'll get all the features working.
 Writing slow interpreter is ... wasting efforts.


This is a common misconception. LLVM as AoT optimizing compiler is 
great, but for the JIT compiler it's actually far too slow at codegen to 
be a sensible choice.

---
Dmitry Olshansky
Feb 17 2017
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Friday, 17 February 2017 at 21:01:33 UTC, Dmitry Olshansky 
wrote:

 This is a common misconception. LLVM as AoT optimizing compiler 
 is great, but for the JIT compiler it's actually far too slow 
 at codegen to be a sensible choice.


I don't know how big of a misconception it is. The Dropbox effort 
on Pyston (LLVM JIT for Python) just recently shut down and 
nobody expressed a lot of surprise in the HN comments.
Feb 17 2017
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 2/17/17 10:37 PM, jmh530 wrote:

 On Friday, 17 February 2017 at 21:01:33 UTC, Dmitry Olshansky wrote:

 This is a common misconception. LLVM as AoT optimizing compiler is
 great, but for the JIT compiler it's actually far too slow at codegen
 to be a sensible choice.


 I don't know how big of a misconception it is. The Dropbox effort on
 Pyston (LLVM JIT for Python) just recently shut down and nobody
 expressed a lot of surprise in the HN comments.


It might be that HN just in general are pessimistic ;)
w.r.t Pyston I've seen it coming since the very first blog post that 
described their approach to the problem. However I've seen lots of folks 
being quite enthusiastic about that effort.

----
Dmitry Olshansky
Feb 17 2017
prev sibling next sibling parent reply Satoshi <satoshi rikarin.org> writes:
Will be possible to make precompiled headers and distribute it 
with static library for example?
Feb 18 2017
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 18 February 2017 at 21:24:18 UTC, Satoshi wrote:

 Will be possible to make precompiled headers and distribute it 
 with static library for example?


anything is possible with ctfe and mixins and templates.
certainly header generation as well, you can write your own 
system for it and do it as a build step, in case the built-in 
header generator is inadequate.
Feb 19 2017
prev sibling next sibling parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]


Thank you for your continued work on this. I heavily rely on D's 
CTFE functionality and I try to read all your updates on it.
Feb 18 2017
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 18 February 2017 at 22:40:44 UTC, Moritz Maxeiner 
wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]


 Thank you for your continued work on this. I heavily rely on 
 D's CTFE functionality and I try to read all your updates on it.


Hello Moritz,
D's ctfe functionality is almost complete.
This thread is not about ctfe as it is currently implemented.
I am working on/writing about a faster re-implementation of ctfe.

When my work is finished nothing will change functionality wise, 
it will just be much more efficient.
Feb 18 2017
parent Moritz Maxeiner <moritz ucworks.org> writes:
On Sunday, 19 February 2017 at 01:52:07 UTC, Stefan Koch wrote:

 On Saturday, 18 February 2017 at 22:40:44 UTC, Moritz Maxeiner 
 wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 Hi Guys, due to the old CTFE status thread getting to page 
 30, I am now starting a new one.

 [...]


 Thank you for your continued work on this. I heavily rely on 
 D's CTFE functionality and I try to read all your updates on 
 it.


 Hello Moritz,
 D's ctfe functionality is almost complete.
 This thread is not about ctfe as it is currently implemented.
 I am working on/writing about a faster re-implementation of 
 ctfe.

 When my work is finished nothing will change functionality 
 wise, it will just be much more efficient.


Hello Stefan,
my apologies if I wasn't clear: I'm aware that this isn't about 
adding anything new in terms of functionality, what I intended to 
imply with "heavily rely on" was that I use enough of it for it 
to be noticeable in compile time / memory consumption, which is 
why I'm very happy about potential speedup and/or memory 
consumption reduction in CTFE. One public example of that was 
llvm-d <2.0.
Feb 18 2017
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:


 Currently I am having trouble caused by a bug in dmds inliner 
 that only happens on when dmd is compiled as a 32bit executable 
 until I have  isolated / fixed this development is slowed down.


I am slowly narrowing down on the source  of this bug.
Until it is fixed builds of newCTFE might not behave correctly, 
if they have been complied with -inline!
Feb 20 2017
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 21 February 2017 at 01:57:21 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:


 Currently I am having trouble caused by a bug in dmds inliner 
 that only happens on when dmd is compiled as a 32bit 
 executable until I have  isolated / fixed this development is 
 slowed down.


 I am slowly narrowing down on the source  of this bug.
 Until it is fixed builds of newCTFE might not behave correctly, 
 if they have been complied with -inline!


The culprit that introduced the bug seems to be :
  
https://github.com/dlang/dmd/commit/cd01efb4dd5e32277cb156c3cc2b451bdcb68b52

however more testing is needed before I can be sure.
Feb 21 2017
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:


 Currently I am having trouble caused by a bug in dmds inliner 
 that only happens on when dmd is compiled as a 32bit executable 
 until I have  isolated / fixed this development is slowed down.



Unitl the auto-tester is updated is updated workarounds have to 
suffice.
In this case the workaround is padding a structure such that it's 
size in no longer 32.
Feb 24 2017
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi Guys,

ref variable handing is coming soon!
(only int/uint for now)...

Also I have suspicions that there is a bug in my ctfe_evaluator 
in which the first default arguments can overridden with the this 
pointer :)

I will test "this" tomorrow :)

Also I submitted my talk proposal a few hours ago;
Of course it's about newCTFE.

Cheers,
Stefan
Feb 25 2017
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [...]


Simple ref calls work now.
Meaning the following code will compile :)

uint sum(uint[] arr)
{
   uint sum;
   foreach(uint i;0 .. cast(uint)arr.length)
   {
     addToSum(sum, arr[i]);
   }
   return sum;
}
void addToSum(ref uint sum, uint element)
{
   sum = sum + element; // works now as well
   return ;
}

static assert([1,2,3,4,5].sum == 15);
Feb 27 2017
parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 27 February 2017 at 23:48:13 UTC, Stefan Koch wrote:

 Simple ref calls work now.
 Meaning the following code will compile :)


Great work.
Feb 28 2017
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi Guys,

The implementation of ref parameters by itself was successful.
However they hey have one bug, which is a consquence of newCTFE 
not touching dmd's state.
They cannot modify an outer parameter. (where outer means that 
the parameter does not originate from a ctfe-evaluation further 
up the call stack)

uint add8ret3(ref uint a) {a += 8; return 3;}

pragma (msg, (uint outer) { outer += add8ret3(a); return 
outer;}(1)); // will print 4

pragma (msg, (uint outer) { uint inner = outer; inner += 
add8ret3(inner); return inner; }(1)); // will print 12 as 
expected.

There are two options I could take:
The first one is to disallow having refs of outer parameters 
taken.
(and bailing out in this case)
The second option is to implement the logic necessary to modify 
these expression nodes.

Both options require me to detect if we have an outer ref 
parameter. but that should be easy to do.

Cheers,
Stefan
Mar 01 2017
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 2 March 2017 at 03:09:56 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]





 uint add8ret3(ref uint a) {a += 8; return 3;}

 pragma (msg, (uint outer) { outer += add8ret3(a); return 
 outer;}(1)); // will print 4


was supposed to say:
pragma (msg, (uint outer) { outer += add8ret3(outer); return 
outer;}(1)); // will print 4
Mar 01 2017
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 2 March 2017 at 03:09:56 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 Hi Guys,

 The implementation of ref parameters by itself was successful.
 However they hey have one bug, which is a consquence of newCTFE 
 not touching dmd's state.
 They cannot modify an outer parameter. (where outer means that 
 the parameter does not originate from a ctfe-evaluation further 
 up the call stack)

 Cheers,
 Stefan



It turn out that this was not the problem.
It was rather old code in the byte-code evaluator that dated from 
a time where parameters were not considered stack values.
causing that code to copy the parameters into a temporary.
When I removed the distinction between parameters and normal 
stack values I deleted the code that kept track of the parameters 
that were copied into temporaries.

Since I forgot to remove the creation of those temporaries.
We ended up with the situation described above.
Mar 01 2017
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Time for an update.

I am currently working on integrating 64bit values into codegen 
API.
However, a backend may not have native 64bit registers or 
arithmetic (the x86/arm architectures come to mind)
For those a common fallback is to be implemented such that not 
every architecture has to implement their 64bit arithmetic on 
their own.

Also work is underway to finally support slicing, which is 
crucial to using phobos algorithms.
Mar 10 2017
next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Mar 10, 2017 at 11:32:14AM +0000, Stefan Koch via Digitalmars-d wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


 
 Time for an update.
 
 I am currently working on integrating 64bit values into codegen API.
 However, a backend may not have native 64bit registers or arithmetic
 (the x86/arm architectures come to mind) For those a common fallback
 is to be implemented such that not every architecture has to implement
 their 64bit arithmetic on their own.


Makes sense to me.



 Also work is underway to finally support slicing, which is crucial to
 using phobos algorithms.


This is awesome news!  Glad to hear there's steady progress being made
on the new CTFE engine.  Can't wait for it to be done and merged into
master!  There are so many awesome things I wanna do with CTFE that
currently would be unacceptably slow.  Can't wait to take D compile-time
capabilities to a whole 'nother level!


T

-- 
"A one-question geek test. If you get the joke, you're a geek: Seen on a
California license plate on a VW Beetle: 'FEATURE'..." -- Joshua D. Wachs -
Natural Intelligence, Inc.
Mar 10 2017
prev sibling next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 10 March 2017 at 11:32:14 UTC, Stefan Koch wrote:

 Also work is underway to finally support slicing, which is 
 crucial to using phobos algorithms.


Incredible diligence.
Mar 10 2017
prev sibling parent reply Kagamin <spam here.lot> writes:
On Friday, 10 March 2017 at 11:32:14 UTC, Stefan Koch wrote:

 I am currently working on integrating 64bit values into codegen 
 API.


How can dmd generate 64-bit code if backend doesn't support 64bit 
values?
Mar 13 2017
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 13 March 2017 at 11:53:59 UTC, Kagamin wrote:

 On Friday, 10 March 2017 at 11:32:14 UTC, Stefan Koch wrote:

 I am currently working on integrating 64bit values into 
 codegen API.


 How can dmd generate 64-bit code if backend doesn't support 
 64bit values?


I was referring to my newCTFE Bytecode-codegen.
It is separate from dmd's backend.

Just for completeness sake, it is perfectly possible to generate 
x86_64 code without supporting 64bit values, you will just have 
alot of zeros in the upper bits :)

dmd does support 64bit values of course.
Mar 13 2017
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Basic Slicing support is in;
The following code will now compile with the newCTFE_slicing 
branch:

static immutable uint[] OneToTen = [1,2,3,4,5,6,7,8,9,10];

const(uint[]) SliceOf1to10(uint lwr, uint upr) {
     return OneToTen[lwr .. upr];
}

static assert(SliceOf1to10(2,8) == [3u, 4u, 5u, 6u, 7u, 8u]);

Contrary to the old interpreter no copies are made.
newCTFE slices reference memory just as runtime slices do.

Cheers, Stefan;
Mar 11 2017
next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Saturday, 11 March 2017 at 14:39:54 UTC, Stefan Koch wrote:

 Basic Slicing support is in;
 The following code will now compile with the newCTFE_slicing 
 branch:


I guess this should greatly benefit compile-time parsing, right?
Mar 11 2017
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 11 March 2017 at 17:38:04 UTC, Nordlöw wrote:

 On Saturday, 11 March 2017 at 14:39:54 UTC, Stefan Koch wrote:

 Basic Slicing support is in;
 The following code will now compile with the newCTFE_slicing 
 branch:


 I guess this should greatly benefit compile-time parsing, right?


As soon as the UTF8 stuff works properly, yes.
Slicing is really just syntactic sugar :)
Mar 11 2017
prev sibling next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Mar 11, 2017 at 02:39:54PM +0000, Stefan Koch via Digitalmars-d wrote:
[...]

 Contrary to the old interpreter no copies are made.
 newCTFE slices reference memory just as runtime slices do.


[...]

Awesome!  This alone ought to greatly expand the scope of what's
feasible in CTFE.  Now I really can't wait to see this merged into
master. :-)


T

-- 
English has the lovely word "defenestrate", meaning "to execute by throwing
someone out a window", or more recently "to remove Windows from a computer and
replace it with something useful". :-) -- John Cowan
Mar 11 2017
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 11 March 2017 at 14:39:54 UTC, Stefan Koch wrote:

 [ ... Slice Support ... ]


Hi Guys,
Since Slice support required an ABI there were a few bugs.
Interestingly those bugs where there for a very long time :)
The Type-handling I chose uses  an index into a specific 
type-array to represent types.
When this index is 0 we consider the type invalid.
However I had an off by one bug in the check, causing the first 
TypeInstance of static arrays to be considered invalid.
As an Invalid type the array has the size 0.
Which in turn causes the Allocation for that array to allocate 
zero bytes for it.
This zero allocation returns a valid pointer to the current top 
of the heap.
(just without reserving any memory).
Then when we slice the array the slice-descriptor has to go onto 
the heap.
And it overwrites the array-descriptor which was allocated with 
zero size.
causing it to point a to a bogus address which is equivalent to 
the array length.

My Sunday was ruined before it began :)
Mar 12 2017
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]


Hey Stefan, you have been working on this for many months now: is 
someone funding your work?  If not, you should put up a bitcoin 
address or something, I know I'd contribute.
Mar 13 2017
next sibling parent Meta <jared771 gmail.com> writes:
On Monday, 13 March 2017 at 15:49:35 UTC, Joakim wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]


 Hey Stefan, you have been working on this for many months now: 
 is someone funding your work?  If not, you should put up a 
 bitcoin address or something, I know I'd contribute.


There's Patreon, but they don't support bitcoin currently.
Mar 13 2017
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 13 March 2017 at 15:49:35 UTC, Joakim wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]


 Hey Stefan, you have been working on this for many months now: 
 is someone funding your work?  If not, you should put up a 
 bitcoin address or something, I know I'd contribute.


Hi Joakim,

My work does get sponsored by the D Language Foundation.
Therefore any Donations best be directed to them.

If someone wants to support my work specifically it's best to 
contact me personally.

I have been thinking about setting up a Patreon account and 
decided it was too much trouble; considering that there is not a 
large number of people who even know my of work.

Cheers, Stefan
Mar 13 2017
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


I just fixed a bug in ref support;
It should now work with slices.

I also made it _slightly_ easier to make ABI changes for slice, 
since I expect the structure to change as soon as concat support 
is added.
(e.g. slices will get a refcount, capacity and an origin ptr)
Mar 13 2017
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 13 March 2017 at 19:40:30 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 I just fixed a bug in ref support;
 It should now work with slices.

 I also made it _slightly_ easier to make ABI changes for slice, 
 since I expect the structure to change as soon as concat 
 support is added.
 (e.g. slices will get a refcount, capacity and an origin ptr)


Bad news.
Array expansion via assignment to length regressed.
Fixing this is surprisingly time intensive.
.... I am just not seeing where it's going wrong.

It seems to use completely bogus offsets ... causing it to read 
from uninitialized memory.
Mar 14 2017
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Mar 14, 2017 at 03:26:44PM +0000, Stefan Koch via Digitalmars-d wrote:
[...]

 Bad news.
 Array expansion via assignment to length regressed.
 Fixing this is surprisingly time intensive.
 .... I am just not seeing where it's going wrong.
 
 It seems to use completely bogus offsets ... causing it to read from
 uninitialized memory.


Sounds like there's a pointer bug / stack overflow / buffer overflow
somewhere.  Just my gut feeling from having faced similar bugs in my
career. Unfortunately, these kinds of bugs are usually very difficult to
trace, because the root cause can be very far away from where the
symptoms show up, and can come from completely unrelated code.

One way that sometimes works (but not always) is to try to shuffle the
stack by moving functions / local variables around to see if the
symptoms change. That may yield some clues as to the nature of the
problem.  But that's just a shot in the dark... generally these kinds of
bugs are very hard to trace.

Or maybe carefully step through the code starting from the length
assignment in a debugger and see if any of the variables seem to have
strange values. Sometimes the code immediately following is fine
(inserting printf's of the buffer may indicate correct values) but it's
something that happens afterwards that screws it up.

Or, possibly, the state is already messed up before the length
assignment... in which case it would be far more difficult to trace. :-(


T

-- 
If lightning were to ever strike an orchestra, it'd always hit the conductor
first.
Mar 14 2017
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 14 March 2017 at 16:42:01 UTC, H. S. Teoh wrote:

 [ ... ]


  H.S.Teoh
Thanks for your advice.
Since I am running my own bytecode I am quite sure that there is 
no memory corruption going on.
It has to do with the recent ABI changes.

--
I just made strings use the same representation as every other 
slice,
Meaning soon we can use std.utf.decode for correct string-foreach 
:)
After that it is not a big step to supporting string concat.
Mar 14 2017
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 3/14/17 4:26 PM, Stefan Koch wrote:

 On Monday, 13 March 2017 at 19:40:30 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


 I just fixed a bug in ref support;
 It should now work with slices.

 I also made it _slightly_ easier to make ABI changes for slice, since
 I expect the structure to change as soon as concat support is added.
 (e.g. slices will get a refcount, capacity and an origin ptr)


 Bad news.
 Array expansion via assignment to length regressed.
 Fixing this is surprisingly time intensive.
 .... I am just not seeing where it's going wrong.

 It seems to use completely bogus offsets ... causing it to read from
 uninitialized memory.


Time to try valgrind ? Last time I checked it worked just fine with D.

---
Dmitry Olshansky
Mar 14 2017
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 14 March 2017 at 21:53:29 UTC, Dmitry Olshansky wrote:

 On 3/14/17 4:26 PM, Stefan Koch wrote:

 On Monday, 13 March 2017 at 19:40:30 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 I just fixed a bug in ref support;
 It should now work with slices.

 I also made it _slightly_ easier to make ABI changes for 
 slice, since
 I expect the structure to change as soon as concat support is 
 added.
 (e.g. slices will get a refcount, capacity and an origin ptr)


 Bad news.
 Array expansion via assignment to length regressed.
 Fixing this is surprisingly time intensive.
 .... I am just not seeing where it's going wrong.

 It seems to use completely bogus offsets ... causing it to 
 read from
 uninitialized memory.


 Time to try valgrind ? Last time I checked it worked just fine 
 with D.

 ---
 Dmitry Olshansky


When I talk about uninitialized memory I mean memory inside the 
VM, which is untouched.
The Problem is a problem with bytecode generation, not the 
execution.
The code executes as it should, but the generated code is invalid.
Mar 14 2017
prev sibling parent Moritz Maxeiner <moritz ucworks.org> writes:
On Tuesday, 14 March 2017 at 21:53:29 UTC, Dmitry Olshansky wrote:

 [...]


 Time to try valgrind ? Last time I checked it worked just fine 
 with D.


Valgrind 3.12.0, at least, works perfectly with dmd&ldc.
One just has to remember to suppress the (static) memory leak in 
druntime[1]

[1] https://github.com/dlang/druntime/pull/1557



 Mar 14 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


String Slicing is in!
The following code will now compile with newCTFE.

string slice(string s, uint lwr, uint upr)
{
     return s[lwr .. upr];
}

static assert(slice("Hello World", 6, 11) == "World");



 Mar 15 2017



  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Wednesday, 15 March 2017 at 13:49:43 UTC, Stefan Koch wrote:

 String Slicing is in!


Great! What remains?



 Mar 15 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 15 March 2017 at 15:49:59 UTC, Nordlöw wrote:

 On Wednesday, 15 March 2017 at 13:49:43 UTC, Stefan Koch wrote:

 String Slicing is in!


 Great! What remains?


Unsupported Features include :

  - Unicode/Auto-decoding support.
NOTE: foreach(char c; string) { ... } will work since it does not 
auto-decode.

  - Floating point.
  - Classes
  - unions
  - structs containing other structs.
  - arrays of structs
  - arrays of arrays
  - || and &&
  - ref return (though I doubt it's needed)
  - references of 64bit values

And maybe other I cannot think of right now.



 Mar 15 2017



 next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 15 March 2017 at 15:59:46 UTC, Stefan Koch wrote:

 On Wednesday, 15 March 2017 at 15:49:59 UTC, Nordlöw wrote:

 On Wednesday, 15 March 2017 at 13:49:43 UTC, Stefan Koch wrote:

 String Slicing is in!


 Great! What remains?


 Unsupported Features include :

  - Unicode/Auto-decoding support.
 NOTE: foreach(char c; string) { ... } will work since it does 
 not auto-decode.

  - Floating point.
  - Classes
  - unions
  - structs containing other structs.
  - arrays of structs
  - arrays of arrays
  - || and &&
  - ref return (though I doubt it's needed)
  - references of 64bit values

 And maybe other I cannot think of right now.


slice expansion.
and string-foreach caused dmd to miscomplie itself ... *sigh*

So no string-foreach for now;



 Mar 15 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 15 March 2017 at 16:11:44 UTC, Stefan Koch wrote:

 and string-foreach caused dmd to miscomplie itself ... *sigh*


Guess what ?
Things work so much better when you allocate space for the zero 
terminator :)
The segfaults are mysteriously gone :D



 Mar 15 2017



prev sibling next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Wednesday, 15 March 2017 at 15:59:46 UTC, Stefan Koch wrote:

 Unsupported Features include :

  - Unicode/Auto-decoding support.
 NOTE: foreach(char c; string) { ... } will work since it does 
 not auto-decode.

  - Floating point.
  - Classes
  - unions
  - structs containing other structs.
  - arrays of structs
  - arrays of arrays
  - || and &&
  - ref return (though I doubt it's needed)
  - references of 64bit values

 And maybe other I cannot think of right now.


How long do you estimate this will take?

Can/Will some of your work on newCTFE-branch be merged onto 
master before _all_ these things have been fixed?



 Mar 15 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 15 March 2017 at 19:00:21 UTC, Nordlöw wrote:

 On Wednesday, 15 March 2017 at 15:59:46 UTC, Stefan Koch wrote:

 Unsupported Features include :

  - Unicode/Auto-decoding support.
 NOTE: foreach(char c; string) { ... } will work since it does 
 not auto-decode.

  - Floating point.
  - Classes
  - unions
  - structs containing other structs.
  - arrays of structs
  - arrays of arrays
  - || and &&
  - ref return (though I doubt it's needed)
  - references of 64bit values

 And maybe other I cannot think of right now.


 How long do you estimate this will take?

 Can/Will some of your work on newCTFE-branch be merged onto 
 master before _all_ these things have been fixed?


Yes the plan is to forgo Classes Execptions and FP for newCTFE 
1.0.



 Mar 15 2017



prev sibling  parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Wed, Mar 15, 2017 at 03:59:46PM +0000, Stefan Koch via Digitalmars-d wrote:
[...]

 Unsupported Features include :


[...]

  - Floating point.


[...]

  - unions


[...]

What are the chances / what's the expected timeframe of unions being
implemented?

Support for unions in CTFE is a MAJOR milestone in making std.math
CTFE-able, which IMO will take D compile-time capabilities to a whole
new level, because it will greatly expand the scope of what's computable
at compile-time in terms of floating-point constants, lookup tables,
etc..  Imagine, for example, a precomputed table of values
transcendental functions with some given resolution, for fast runtime
lookups.

(Of course, this also requires floating-point support in CTFE. But that
should be relatively easy(?). As long as both the host and target
architectures support the same set of IEEE floating-point types, which
is probably the case for our currently-supported platforms.)


T

-- 
People say I'm indecisive, but I'm not sure about that. -- YHL, CONLANG



 Mar 15 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 15 March 2017 at 18:52:16 UTC, H. S. Teoh wrote:

 On Wed, Mar 15, 2017 at 03:59:46PM +0000, Stefan Koch via 
 Digitalmars-d wrote: [...]

 Unsupported Features include :


 [...]

  - Floating point.


 [...]

  - unions


 [...]

 What are the chances / what's the expected timeframe of unions 
 being implemented?

 Support for unions in CTFE is a MAJOR milestone in making 
 std.math CTFE-able, which IMO will take D compile-time 
 capabilities to a whole new level, because it will greatly 
 expand the scope of what's computable at compile-time in terms 
 of floating-point constants, lookup tables, etc..  Imagine, for 
 example, a precomputed table of values transcendental functions 
 with some given resolution, for fast runtime lookups.

 (Of course, this also requires floating-point support in CTFE. 
 But that should be relatively easy(?). As long as both the host 
 and target architectures support the same set of IEEE 
 floating-point types, which is probably the case for our 
 currently-supported platforms.)


 T


I am not sure.
The CTFE-ABI is different from the ABI at runtime.
So more adventurous uses of unions are likely to be surprising.
As for floating point, I have yet to find a solution that will 
work for more numerically inclined people.
dmds constant folder apparently does some funky things in that 
domain as well.
I do not anticipate to have any of the fp stuff working before 
dconf.

Currently Unicode correct string-handling is more important.
After that comes class support.
FP is the very last thing on my list and will like take a lot of 
time.



 Mar 15 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi Guys I am proud to say that newCTFE with string-foreach 
enabled, is now green!

As soon as array-length assignment (and therefore expansion) is 
working again;
You can start writing fast compile-time parsers.

Albeit it will be bit a bit awkward without && and ||. :o)

Today I feel like I have reached a milestone.

A good day to all of you,

Cheers Stefan.



 Mar 16 2017



 next sibling parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org>  writes:


Am 16.03.2017 um 15:48 schrieb Stefan Koch:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


 Hi Guys I am proud to say that newCTFE with string-foreach enabled, is
 now green!

 As soon as array-length assignment (and therefore expansion) is working
 again;
 You can start writing fast compile-time parsers.

 Albeit it will be bit a bit awkward without && and ||. :o)

 Today I feel like I have reached a milestone.

 A good day to all of you,

 Cheers Stefan.


Congratulations! It's really starting to get exciting. You have no idea 
how much I look forward to see diet-ng run on the new engine ;-)



 Mar 16 2017



  parent  "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Thu, Mar 16, 2017 at 04:18:35PM +0100, Snke Ludwig via Digitalmars-d wrote:

 Am 16.03.2017 um 15:48 schrieb Stefan Koch:


[...]

 Hi Guys I am proud to say that newCTFE with string-foreach enabled,
 is now green!
 
 As soon as array-length assignment (and therefore expansion) is
 working again; You can start writing fast compile-time parsers.
 
 Albeit it will be bit a bit awkward without && and ||. :o)
 
 Today I feel like I have reached a milestone.
 
 A good day to all of you,
 
 Cheers Stefan.


 
 Congratulations! It's really starting to get exciting. You have no
 idea how much I look forward to see diet-ng run on the new engine ;-)


And I'm so excited to be able to soon write complicated compile-time
parsers and parser generators without drastic reduction in compilation
time or drastic increase in compiler memory requirements.


T

-- 
Fact is stranger than fiction.



 Mar 16 2017



prev sibling  parent reply Kagamin <spam here.lot>  writes:


On Thursday, 16 March 2017 at 14:48:22 UTC, Stefan Koch wrote:

 As soon as array-length assignment (and therefore expansion) is 
 working again;
 You can start writing fast compile-time parsers.


Though for AST one would need unions, like
struct Node
{
   int type;
   union
   {
     StringLiteral str;
     NumberLiteral num;
     ExpressionNode expr;
   }
}



 Mar 17 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 17 March 2017 at 12:55:22 UTC, Kagamin wrote:

 On Thursday, 16 March 2017 at 14:48:22 UTC, Stefan Koch wrote:

 As soon as array-length assignment (and therefore expansion) 
 is working again;
 You can start writing fast compile-time parsers.


 Though for AST one would need unions, like
 struct Node
 {
   int type;
   union
   {
     StringLiteral str;
     NumberLiteral num;
     ExpressionNode expr;
   }
 }


You might as well replace union with struct here.



 Mar 17 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Small update I fixed the handling of $ in slices.
[1,2,3,4][$-2 .. $] == [3,4]
will now work in newCTFE



 Mar 17 2017



  parent  "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Sat, Mar 18, 2017 at 03:29:15AM +0000, Stefan Koch via Digitalmars-d wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


 
 Small update I fixed the handling of $ in slices.
 [1,2,3,4][$-2 .. $] == [3,4]
 will now work in newCTFE


Awesome!  It's great to see the new CTFE engine slowly but surely taking
shape.  Really looking forward to see the final product.


T

-- 
The irony is that Bill Gates claims to be making a stable operating system and
Linus Torvalds claims to be trying to take over the world. -- Anonymous



 Mar 17 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


I just restored Array/Slice-Expansion.
I also introduced concat!

This will now compile with newCTFE!

static immutable uint[] OneToTen = [1,2,3,4,5,6,7,8,9,10];

const(uint[]) SliceOf1to10(uint lwr, uint upr)
{
     return OneToTen[lwr .. upr];
}

const(uint)[] testConcat()
{
     return SliceOf1to10(0,4) ~ SliceOf1to10(7,9);
}

static assert(testConcat == [1,2,3,4,8,9]);



 Mar 19 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Oh darn, function pointers regressed!
I did not notice because the corresponding tests were commented 
out.

I am working to fix it ASAP.



 Mar 20 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 20 March 2017 at 11:48:56 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 Oh darn, function pointers regressed!
 I did not notice because the corresponding tests were commented 
 out.

 I am working to fix it ASAP.


It's not function pointers.
It's slice extension.

It just happened that my function pointer tests relied on slice 
extension working.



 Mar 20 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 20 March 2017 at 12:06:57 UTC, Stefan Koch wrote:

 On Monday, 20 March 2017 at 11:48:56 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 Oh darn, function pointers regressed!
 I did not notice because the corresponding tests were 
 commented out.

 I am working to fix it ASAP.


 It's not function pointers.
 It's slice extension.

 It just happened that my function pointer tests relied on slice 
 extension working.


Actually it's just local variables of slices :)
Doged a nasty bullet there.



 Mar 20 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi I fixed the local slice issue.
It turned out that there was special code in place to deal with 
this case;
But that I forgot to change that to the new slice ABI.

The moral of this story is;
ABI changes are hard to do if you have an ad-hoc approach to it.
However now I made ABI-dependeant values more visible.
And it is unlikely that further slice-ABI changes will cause 
problems.

Cheers,
Stefan



 Mar 21 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


I just fixed a bug in switches, where the fall-trough case would 
incorrectly jump after the switch.

The reason this bug occurred is that none of my tests did cover 
the fall-trough case.
The code that handles switches converts them into a big if-else 
chain because jump-tables are usually more expensive for small 
switches.



 Mar 22 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Alright guys.

I am just fixing the newCTFE LLVM backend such that I can have 
better guesses as to what performance a JIT could archive.

Due to the ABI changes the llvm-backend a little optimization 
potential.

It'll be a while until the llvm backend is on par again.
Because functions and such need to be dealt with quite 
differently.
Also the llvm backend recives less testing since I cannot run it 
at compile time(Yet :))

Cheers,
Stefan
--
sorry for the short message I am busy fixing codegen bugs as 
always.



 Mar 31 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 31 March 2017 at 14:05:00 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 Alright guys.

 I am just fixing the newCTFE LLVM backend such that I can have 
 better guesses as to what performance a JIT could archive.

 Due to the ABI changes the llvm-backend a little optimization 
 potential.

 It'll be a while until the llvm backend is on par again.
 Because functions and such need to be dealt with quite 
 differently.
 Also the llvm backend recives less testing since I cannot run 
 it at compile time(Yet :))

 Cheers,
 Stefan
 --
 sorry for the short message I am busy fixing codegen bugs as 
 always.


oh, yeah ... If you want to checkout the llvm backend fethc the 
newCTFE_LLVMBackend branch from 
https://github.com/UplinkCoder/dmd.

the posix.mak is hardcoded to use libLLVM-3.5.
However you should be able to use any version newer then 3.3.



 Mar 31 2017



  parent reply Inquie <Inquie data.com>  writes:


On Friday, 31 March 2017 at 14:16:15 UTC, Stefan Koch wrote:

 On Friday, 31 March 2017 at 14:05:00 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 Alright guys.

 I am just fixing the newCTFE LLVM backend such that I can have 
 better guesses as to what performance a JIT could archive.

 Due to the ABI changes the llvm-backend a little optimization 
 potential.

 It'll be a while until the llvm backend is on par again.
 Because functions and such need to be dealt with quite 
 differently.
 Also the llvm backend recives less testing since I cannot run 
 it at compile time(Yet :))

 Cheers,
 Stefan
 --
 sorry for the short message I am busy fixing codegen bugs as 
 always.


 oh, yeah ... If you want to checkout the llvm backend fethc the 
 newCTFE_LLVMBackend branch from 
 https://github.com/UplinkCoder/dmd.

 the posix.mak is hardcoded to use libLLVM-3.5.
 However you should be able to use any version newer then 3.3.



How far off until newCTFE is usable to compile the majority of 
templates out there? I have some new code that doesn't do 
anything real complex but seems quite slow for no apparent reason 
and would like to try the newCTFE if it has a good chance of 
working.



 Apr 01 2017



 next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 1 April 2017 at 17:06:14 UTC, Inquie wrote:

 How far off until newCTFE is usable to compile the majority of 
 templates out there? I have some new code that doesn't do 
 anything real complex but seems quite slow for no apparent 
 reason and would like to try the newCTFE if it has a good 
 chance of working.


Quite far.
Templates are usually not slow because of CTFE.
But because of O(n^2) ((O(n!) actully in some cases) nature of 
recursive templates.

I am going to fix this after CTFE, (if the D Langauge Foundation 
or a third-party can provide sponsorship for that).



 Apr 01 2017



prev sibling  parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Sat, Apr 01, 2017 at 05:06:14PM +0000, Inquie via Digitalmars-d wrote:
[...]

 How far off until newCTFE is usable to compile the majority of
 templates out there?


CTFE and templates are two separate things. You may want to read this
(draft) article to get a better understanding of how they fit together:

	https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time


T

-- 
Computers shouldn't beep through the keyhole.



 Apr 01 2017



 next sibling parent  Nicholas Wilson <iamthewilsonator hotmail.com>  writes:


On Sunday, 2 April 2017 at 04:34:34 UTC, H. S. Teoh wrote:

 On Sat, Apr 01, 2017 at 05:06:14PM +0000, Inquie via 
 Digitalmars-d wrote: [...]

 How far off until newCTFE is usable to compile the majority of 
 templates out there?


 CTFE and templates are two separate things. You may want to 
 read this (draft) article to get a better understanding of how 
 they fit together:

 	https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time


 T


some drive-by review
---
struct Box!float
{
     int data;
}

Box!float floatBox;
---

Is not what you meant.



 Apr 02 2017



prev sibling next sibling parent  data pulverizer <data.pulverizer gmail.com>  writes:


On Sunday, 2 April 2017 at 04:34:34 UTC, H. S. Teoh wrote:

 On Sat, Apr 01, 2017 at 05:06:14PM +0000, Inquie via 
 Digitalmars-d wrote: [...]

 How far off until newCTFE is usable to compile the majority of 
 templates out there?


 CTFE and templates are two separate things. You may want to 
 read this (draft) article to get a better understanding of how 
 they fit together:

 	https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time


 T


Just read your draft article, very illuminating. Thank you. There 
was a section "Interleaved AST manipulation and CTFE" which you 
mention is "one of the keystones of meta-programming in D". I 
think there should be a separate article on this topic as a way 
of popularising D's idioms. When can we expect this? :-)



 Apr 05 2017



prev sibling next sibling parent reply Yuxuan Shui <yshuiv7 gmail.com>  writes:


On Sunday, 2 April 2017 at 04:34:34 UTC, H. S. Teoh wrote:

 On Sat, Apr 01, 2017 at 05:06:14PM +0000, Inquie via 
 Digitalmars-d wrote: [...]

 How far off until newCTFE is usable to compile the majority of 
 templates out there?


 CTFE and templates are two separate things. You may want to 
 read this (draft) article to get a better understanding of how 
 they fit together:

 	https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time


 T


CTFE and template expansion might be more tangled than you 
thought.

For example, you do have access to CTFE during template 
expansion: 
http://forum.dlang.org/thread/yaekhryalyxyooaiuakj forum.dlang.org



 Apr 05 2017



  parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Wed, Apr 05, 2017 at 11:20:28AM +0000, Yuxuan Shui via Digitalmars-d wrote:

 On Sunday, 2 April 2017 at 04:34:34 UTC, H. S. Teoh wrote:

 On Sat, Apr 01, 2017 at 05:06:14PM +0000, Inquie via Digitalmars-d
 wrote: [...]

 How far off until newCTFE is usable to compile the majority of
 templates out there?


 
 CTFE and templates are two separate things. You may want to read
 this (draft) article to get a better understanding of how they fit
 together:
 
 	https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time
 
 
 T


 
 CTFE and template expansion might be more tangled than you thought.
 
 For example, you do have access to CTFE during template expansion:
 http://forum.dlang.org/thread/yaekhryalyxyooaiuakj forum.dlang.org


Did you read the entire article?

There is an entire section dedicated to interleaving of CTFE and
templates.  And no, you still cannot run CTFE on the same part of the
AST that is being template-expanded. But you *can* run CTFE on a subtree
that has already been fully expanded.

And no, the forum post you linked to has nothing to do with CTFE. The
so-called "static foreach" is unrolled at AST expansion time, and is not
run through CTFE at all (unless later on you call the expanded
function at "compile-time"). And is() expressions are also not CTFE,
they are also evaluated at AST expansion time.

Read the entire article first. ;-)


T

-- 
Guns don't kill people. Bullets do.



 Apr 05 2017



  parent reply Yuxuan Shui <yshuiv7 gmail.com>  writes:


On Wednesday, 5 April 2017 at 16:06:39 UTC, H. S. Teoh wrote:

 On Wed, Apr 05, 2017 at 11:20:28AM +0000, Yuxuan Shui via 
 Digitalmars-d wrote:

 [...]


 Did you read the entire article?

 There is an entire section dedicated to interleaving of CTFE 
 and templates.  And no, you still cannot run CTFE on the same 
 part of the AST that is being template-expanded. But you *can* 
 run CTFE on a subtree that has already been fully expanded.

 And no, the forum post you linked to has nothing to do with 
 CTFE. The
 so-called "static foreach" is unrolled at AST expansion time, 
 and is not
 run through CTFE at all (unless later on you call the expanded
 function at "compile-time"). And is() expressions are also not 
 CTFE,
 they are also evaluated at AST expansion time.

 Read the entire article first. ;-)


 T


I was talking about the use of R.front, R.drop in the template.



 Apr 05 2017



  parent  "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Wed, Apr 05, 2017 at 08:08:32PM +0000, Yuxuan Shui via Digitalmars-d wrote:

 On Wednesday, 5 April 2017 at 16:06:39 UTC, H. S. Teoh wrote:

 On Wed, Apr 05, 2017 at 11:20:28AM +0000, Yuxuan Shui via Digitalmars-d
 wrote:

 [...]


 
 Did you read the entire article?
 
 There is an entire section dedicated to interleaving of CTFE and
 templates.  And no, you still cannot run CTFE on the same part of
 the AST that is being template-expanded. But you *can* run CTFE on a
 subtree that has already been fully expanded.
 
 And no, the forum post you linked to has nothing to do with CTFE.
 The so-called "static foreach" is unrolled at AST expansion time,
 and is not run through CTFE at all (unless later on you call the
 expanded function at "compile-time"). And is() expressions are also
 not CTFE, they are also evaluated at AST expansion time.
 
 Read the entire article first. ;-)
 
 
 T


 
 I was talking about the use of R.front, R.drop in the template.


Yes, that's the interleaving I was talking about. In your code example,
R is an alias to something outside the template itself, so as long as
whatever it aliases can be "finalized" in its AST, it can be handed to
the CTFE engine for evaluation.  This is really just a more fancy
instance of the more common `enum X = f(Y);` idiom, but it's really the
same thing.  The compiler tries to use CTFE every time it sees a value
that's needed at "compile-time"; `enum` is the usual way to trigger
this, but a template expansion that depends on the value, as you have
here, is also where the same thing happens.

What you *cannot* have, though, is if R is in the process of being
AST-expanded.  E.g., if .front itself requires expandRange() in its
definition, then it won't work, because then the compiler can't finalize
the AST of .front and CTFE won't be able to run it.


T

-- 
Nobody is perfect.  I am Nobody. -- pepoluan, GKC forum



 Apr 05 2017



prev sibling  parent  Dragos Carp <dragoscarp gmail.com>  writes:


On Sunday, 2 April 2017 at 04:34:34 UTC, H. S. Teoh wrote:

 On Sat, Apr 01, 2017 at 05:06:14PM +0000, Inquie via 
 Digitalmars-d wrote: [...]

 How far off until newCTFE is usable to compile the majority of 
 templates out there?


 CTFE and templates are two separate things. You may want to 
 read this (draft) article to get a better understanding of how 
 they fit together:

 	https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time


 T


Very nice and informative article! Thanks!

Still I miss some comments about mixins and template mixins.



 Apr 05 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi Guys,
I wanted to give a short status update again.
No new features;
But a few things have changed under the hood.
I am currently cleaning up code to get ready for a first 
publicized preview build.

After || und && are implemented (which currently still give me an 
unreasonable amount of trouble), the preview will go life :)

Cheers,
Stefan



 Apr 06 2017



  parent reply crimaniak <crimaniak gmail.com>  writes:


Hi!

Is it hard to implement some way to access symbol from UDA 
constructor of symbol?

```
class uda
{
     this()
     {
          // Iterate by Foo members
         foreach (member; __traits(allMembers, __SYMBOL__))
             ...
     }
}


 uda
struct Foo
{

}

```



 Apr 08 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 8 April 2017 at 08:50:27 UTC, crimaniak wrote:

 Hi!

 Is it hard to implement some way to access symbol from UDA 
 constructor of symbol?

 ```
 class uda
 {
     this()
     {
          // Iterate by Foo members
         foreach (member; __traits(allMembers, __SYMBOL__))
             ...
     }
 }


  uda
 struct Foo
 {

 }

 ```


This is a question for D.Learn.
please post there.
Also the question is to terse, I do not know what you are asking.



 Apr 08 2017



 next sibling parent reply rikki cattermole <rikki cattermole.co.nz>  writes:


On 08/04/2017 10:14 AM, Stefan Koch wrote:

 On Saturday, 8 April 2017 at 08:50:27 UTC, crimaniak wrote:

 Hi!

 Is it hard to implement some way to access symbol from UDA constructor
 of symbol?

 ```
 class uda
 {
     this()
     {
          // Iterate by Foo members
         foreach (member; __traits(allMembers, __SYMBOL__))
             ...
     }
 }


  uda
 struct Foo
 {

 }

 ```


 This is a question for D.Learn.
 please post there.
 Also the question is to terse, I do not know what you are asking.


Nope, not valid for D.learn as it is not currently possible.



 Apr 08 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 8 April 2017 at 09:19:03 UTC, rikki cattermole wrote:

 Nope, not valid for D.learn as it is not currently possible.


perfectly valid in D.learn.
You could have given the answer that is it not possible there.



 Apr 08 2017



prev sibling  parent reply crimaniak <crimaniak gmail.com>  writes:


On Saturday, 8 April 2017 at 09:14:17 UTC, Stefan Koch wrote:

 This is a question for D.Learn.
 please post there.


Yes, I did it already and know, it is not possible for now: 
https://forum.dlang.org/post/crkxlbtfhsplxfilakrk forum.dlang.org

This is exact reason why I asked, how hard _to implement_ it. I 
asked you, because UDF constructor executed in CTFE, and you can 
give the most competent answer, because you are working on it.


 Also the question is to terse, I do not know what you are 
 asking.


   How hard to implement something like special keyword 
__SYMBOL__, which will be alias for Foo (or string with fully 
qualified name of Foo) in example in my initial letter, in the 
scope of udf constructor.



 Apr 08 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 8 April 2017 at 19:03:52 UTC, crimaniak wrote:

 On Saturday, 8 April 2017 at 09:14:17 UTC, Stefan Koch wrote:

 This is a question for D.Learn.
 please post there.


 Yes, I did it already and know, it is not possible for now: 
 https://forum.dlang.org/post/crkxlbtfhsplxfilakrk forum.dlang.org

 This is exact reason why I asked, how hard _to implement_ it. I 
 asked you, because UDF constructor executed in CTFE, and you 
 can give the most competent answer, because you are working on 
 it.


 Also the question is to terse, I do not know what you are 
 asking.


   How hard to implement something like special keyword 
 __SYMBOL__, which will be alias for Foo (or string with fully 
 qualified name of Foo) in example in my initial letter, in the 
 scope of udf constructor.


This is not related to the ctfe subsystem.

Therefore take my answer with a grain of salt.
I assume that what you want will quite expensive in terms of 
compiler performance.
Also it would require a language change.
A facility as you want it would need to modify the constructor 
behind the scenes, either by adding a hidden template parameter 
to the constructor.
I general I would be weary about adding this facility.
Though right now I cannot see any wired corner-cases, I think 
that you are going to run into trouble with this.
Particularly when forward-references are involved.



 Apr 08 2017



  parent  crimaniak <crimaniak gmail.com>  writes:


On Sunday, 9 April 2017 at 00:25:45 UTC, Stefan Koch wrote:
...

 I assume that what you want will quite expensive in terms of 
 compiler performance.


...
Thanks for your answer!



 Apr 09 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi Guys :)

I am currently fixing a bug involving complex members of structs
(where complex means (slice, struct, array or pointer))
I did expect them to be broken ...
but not to be _that_ broken :)

struct S
{
     uint[] slice;
}

uint fn()
{
   S s;
   s.slice.length = 12;
   return cast(uint)s.slice.length;
}

static assert(fn() == 12);
This code will not work because s.slice has no elementType :)
(which does not mean it has the s.slice[0] has the type void)
newCTFE literally looses the type information somewhere.

And people wonder why I don't like mondays :)



 Apr 10 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 10 April 2017 at 20:49:58 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 Hi Guys :)

 I am currently fixing a bug involving complex members of structs
 (where complex means (slice, struct, array or pointer))
 I did expect them to be broken ...
 but not to be _that_ broken :)

 struct S
 {
     uint[] slice;
 }

 uint fn()
 {
   S s;
   s.slice.length = 12;
   return cast(uint)s.slice.length;
 }

 static assert(fn() == 12);
 This code will not work because s.slice has no elementType :)
 (which does not mean it has the s.slice[0] has the type void)
 newCTFE literally looses the type information somewhere.

 And people wonder why I don't like mondays :)


I found out that slice was never allocated :)
This is an orthogonal problem, but it's fixed now.

The problem from the above post still remains.
And I still don't know why it happens.



 Apr 11 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


I just found more states we get into, that should be impossible 
to ever get into.
I am stumped.
Baffled.
And seriously befuddled!



 Apr 12 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 12 April 2017 at 09:19:39 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 I just found more states we get into, that should be impossible 
 to ever get into.
 I am stumped.
 Baffled.
 And seriously befuddled!


So .. this is partially because we assume the stack to be zeroed 
if we have not written to it yet. It is zero-initialized after 
all, however If we are returning from a function that wrote to 
the stack and then we are calling another function, that function 
will see the state the previous function left there...

which just means ... we have to zero our temporaries and locals 
on function entery.
implementing this however breaks incremental code-generation.

Awwwww ....



 Apr 12 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


I just fixed the static assert((null ~ null) is null);
Hence I can now enable string-concat!



 Apr 12 2017



  parent  "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Wed, Apr 12, 2017 at 01:18:13PM +0000, Stefan Koch via Digitalmars-d wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


 
 I just fixed the static assert((null ~ null) is null);
 Hence I can now enable string-concat!


[...]

Good news!


T

-- 
Век живи - век учись. А дураком помрёшь.



 Apr 12 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Comma expressions should now work.



 Apr 12 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi I want to share another story.
I was pretty happy to have recursive function calls working.
So happy in fact that I overlooked that they were actually 
generated twice.
Let me illustrate what happend.

Suppose we have the following module :
void fn(uint rcount)
{
   uint ctr = rcount;

   while (rcount--)
     ctr += fn(rcount);

   return ctr;
}

pragma(msg, fn(26));

the compiler hits the pragma(msg) and goes on to evaluate fn(26)
newCTFE receives the functionBody as an ast-node.
it starts processing (function 1)
and hits the fn(rcount)

it checks if it has the code for fn already.
this check returns false since fn has not been completely 
generated yet.

when this check returns it will write the FunctionBody in it's 
todo list.
it wires up the call to the entry in the todo list. (function 2)
it then hits the end of fn  (function 1)and saves it in its 
code-cache.

now it processes the TODO list
it finds that it has to process fn.
it starts processing fn (function 2)
it hits the call.
This time it does find an entry in the code cache for fn (funcion 
1)
it wires of the call
and returns.

The generate pseudo-code looks like :

... fn_0 (...) {
  ...
  call (fn_1, ...)
  ...
}
... fn_1 (...) {
   ...
   call (fn_0, ...)
   ...
}

It was very surprised then I saw this :)

Cheers,
Stefan



 Apr 14 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 { ... }


Wonderful news!
Most of the Byteocode macros are gone!
meaning less templates and faster bytecode generartion!



 Apr 14 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]


The llvm backend is back in a fully working state.
It's about 2times slower in my then my interpreter ;)



 Apr 15 2017



  parent reply Moritz Maxeiner <moritz ucworks.org>  writes:


On Saturday, 15 April 2017 at 10:10:54 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 Hi Guys, due to the old CTFE status thread getting to page 30, 
 I am now starting a new one.

 [...]


 The llvm backend is back in a fully working state.
 It's about 2times slower in my then my interpreter ;)


Huh. In all cases, or only in trivial ones? Because I would have 
expected the overhead of jitting to become less relevant the more 
complex stuff you interpret vs jit.



 Apr 15 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Saturday, 15 April 2017 at 10:30:57 UTC, Moritz Maxeiner wrote:

 On Saturday, 15 April 2017 at 10:10:54 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 Hi Guys, due to the old CTFE status thread getting to page 
 30, I am now starting a new one.

 [...]


 The llvm backend is back in a fully working state.
 It's about 2times slower in my then my interpreter ;)


 Huh. In all cases, or only in trivial ones? Because I would 
 have expected the overhead of jitting to become less relevant 
 the more complex stuff you interpret vs jit.


It's an average number.
on tests the represent the usual usage of ctfe.



 Apr 15 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi Guys,

I just fixed default initialization of structs.
So now a larger portion of code will be compiled and executed by 
newCTFE.

my_struct MyStruct;
will now work, before it would trigger a bailout.
NOTE: this will create bogus results if the struct does contain 
complex initializers i.e. anything other then integers.

Complex type support will come after dconf.



 Apr 16 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi Guys,

As you already probably know some work has been done in the past 
week to get an x86 jit rolling.

It is designed to produce very simple code with _any_ 
optimization at all.

Since optimization introduces heavy complexity down the road, 
even if at first it looks very affordable. My opinion is : "_any_ 
optimization too much."

This stance should make it possible to get some _really_ shiny 
performance numbers for dconf.

Cheers,
Stefan



 Apr 26 2017



 next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Thu, Apr 27, 2017 at 02:15:30AM +0000, Stefan Koch via Digitalmars-d wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


 Hi Guys,
 
 As you already probably know some work has been done in the past week
 to get an x86 jit rolling.
 
 It is designed to produce very simple code with _any_ optimization at
 all.
 
 Since optimization introduces heavy complexity down the road, even if
 at first it looks very affordable. My opinion is : "_any_ optimization
 too much."
 
 This stance should make it possible to get some _really_ shiny
 performance numbers for dconf.


[...]

Is it possible at all to use any of the backend (in particular what
parts of the optimizer that are pertinent), or is the API not conducive
for this?


T

-- 
It always amuses me that Windows has a Safe Mode during bootup. Does that mean
that Windows is normally unsafe?



 Apr 26 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 27 April 2017 at 03:33:03 UTC, H. S. Teoh wrote:

 Is it possible at all to use any of the backend (in particular 
 what parts of the optimizer that are pertinent), or is the API 
 not conducive for this?


 T


It is of course possible to use dmds backend but not very 
desirable, dmds backend works on an expression-tree, which would 
be expensive to build from the linear IR newCTFE uses.
Dmds backend is also very hard to debug for anyone who is not 
Walter.

CTFE is the common case will be fastest if executed without any 
optimizer interfering.
modern x86 chips done a very fine job indeed executing crappy 
code fast.
Therefore making it possible to get away with very simple and 
fast codegen.
(Where fast means code-generation speed rather then code 
execution speed).



 Apr 26 2017



prev sibling  parent reply Dmitry Olshansky <dmitry.olsh gmail.com>  writes:


On 4/27/17 4:15 AM, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


 Hi Guys,

 As you already probably know some work has been done in the past week to
 get an x86 jit rolling.

 It is designed to produce very simple code with _any_ optimization at all.

 Since optimization introduces heavy complexity down the road, even if at
 first it looks very affordable. My opinion is : "_any_ optimization too
 much."


There is also trade-off of spending too much time doing an optimization.
That being said simple peep-hole optimizations may be well worth the effort.


 This stance should make it possible to get some _really_ shiny
 performance numbers for dconf.

 Cheers,
 Stefan





 Apr 27 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 27 April 2017 at 08:51:17 UTC, Dmitry Olshansky 
wrote:

 On 4/27/17 4:15 AM, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 Hi Guys,

 As you already probably know some work has been done in the 
 past week to
 get an x86 jit rolling.

 It is designed to produce very simple code with _any_ 
 optimization at all.

 Since optimization introduces heavy complexity down the road, 
 even if at
 first it looks very affordable. My opinion is : "_any_ 
 optimization too
 much."


 There is also trade-off of spending too much time doing an 
 optimization.
 That being said simple peep-hole optimizations may be well 
 worth the effort.


 This stance should make it possible to get some _really_ shiny
 performance numbers for dconf.

 Cheers,
 Stefan




I should probably clarify;  I made a typo.
I was meaning to write "without _any_ optimization at all."
Peep-holing would be worth it for wanting to get the last drop of 
performance;
However in the specific case of newCTFE, the crappiest JIT will 
already be much faster then an optimized interpreter would be.

Small peephole optimization quickly turns into and endless source 
of bugs.



 Apr 27 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


After a little of exploration of the JIT, I have now determined 
that a simple risc architecture is still the best.
(codegen for scaled loads is hard :p)

I am now back to fixing non-compiling code,
such as :

struct S
{
     uint[] slice;
}

uint fn()
{
   S s;
   s.slice.length = 12;
   return cast(uint)s.slice.length;
}

static assert(fn() == 12);

This simple test does not compile because;
ahm well ...
Somewhere along the road we loose the type of s.slice and we 
cannot tell where to get .length from.



 Apr 28 2017



 next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Friday, 28 April 2017 at 08:47:43 UTC, Stefan Koch wrote:

 After a little of exploration of the JIT, I have now determined 
 that a simple risc architecture is still the best.
 (codegen for scaled loads is hard :p)


Do you mean no Jit?



 Apr 28 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 28 April 2017 at 13:03:42 UTC, Nordlöw wrote:

 On Friday, 28 April 2017 at 08:47:43 UTC, Stefan Koch wrote:

 After a little of exploration of the JIT, I have now 
 determined that a simple risc architecture is still the best.
 (codegen for scaled loads is hard :p)


 Do you mean no Jit?


Of course there will be a JIT.

But currently I am fixing busy bugs in the generated IR.
So the implementation of jit will have to wait a little.



 Apr 28 2017



  parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Friday, 28 April 2017 at 13:13:16 UTC, Stefan Koch wrote:

 Do you mean no Jit?


 Of course there will be a JIT.


Ah, I misunderstood you formulation.


 But currently I am fixing busy bugs in the generated IR.
 So the implementation of jit will have to wait a little.


Ok. Thanks.



 Apr 28 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 28 April 2017 at 08:47:43 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 After a little of exploration of the JIT, I have now determined 
 that a simple risc architecture is still the best.
 (codegen for scaled loads is hard :p)

 I am now back to fixing non-compiling code,
 such as :

 struct S
 {
     uint[] slice;
 }

 uint fn()
 {
   S s;
   s.slice.length = 12;
   return cast(uint)s.slice.length;
 }

 static assert(fn() == 12);

 This simple test does not compile because;
 ahm well ...
 Somewhere along the road we loose the type of s.slice and we 
 cannot tell where to get .length from.


I fixed this just now!
ABI's are hard :)



 Apr 29 2017



prev sibling  parent  Kagamin <spam here.lot>  writes:


An article about LLVM jit: 
https://webkit.org/blog/5852/introducing-the-b3-jit-compiler/



 May 03 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi Guys, I just implemented sliceAssigment.

meaning the following code will now compile:

uint[] assignSlice(uint from, uint to, uint[] stuff)
{
     uint[] slice;
     slice.length = to + 4;
     foreach (uint i; 0 .. to + 4)
     {
         slice[i] = i + 1;
     }

     slice[from .. to] = stuff;
     return slice;
}

static assert(assignSlice(1, 4, [9, 8, 7]) == [1, 9, 8, 7, 5, 6, 
7, 8]);



 Apr 28 2017



 next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 28 April 2017 at 17:53:04 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 Hi Guys, I just implemented sliceAssigment.

 meaning the following code will now compile:

 uint[] assignSlice(uint from, uint to, uint[] stuff)
 {
     uint[] slice;
     slice.length = to + 4;
     foreach (uint i; 0 .. to + 4)
     {
         slice[i] = i + 1;
     }

     slice[from .. to] = stuff;
     return slice;
 }

 static assert(assignSlice(1, 4, [9, 8, 7]) == [1, 9, 8, 7, 5, 
 6, 7, 8]);


as always ... I spoke too soon :(
while running test I forgot to specify -bc-ctfe ...

Although I use the same code for slicing ... it seems it 
misbehaves in the usecase.



 Apr 28 2017



prev sibling  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 28 April 2017 at 17:53:04 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 Hi Guys, I just implemented sliceAssigment.

 meaning the following code will now compile:

 uint[] assignSlice(uint from, uint to, uint[] stuff)
 {
     uint[] slice;
     slice.length = to + 4;
     foreach (uint i; 0 .. to + 4)
     {
         slice[i] = i + 1;
     }

     slice[from .. to] = stuff;
     return slice;
 }

 static assert(assignSlice(1, 4, [9, 8, 7]) == [1, 9, 8, 7, 5, 
 6, 7, 8]);


FIXED!



 Apr 29 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Big news!
The first step to include debug info has been done.

Yes this means you will be able to step through ctfe code while 
the compiler executes it.



 Apr 30 2017



 next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Sun, Apr 30, 2017 at 01:26:09PM +0000, Stefan Koch via Digitalmars-d wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


 
 Big news!
 The first step to include debug info has been done.
 
 Yes this means you will be able to step through ctfe code while the
 compiler executes it.


Wow! Will that be accessible to users in the end?  That could be a
totally awesome way of debugging CTFE code!


T

-- 
Life begins when you can spend your spare time programming instead of watching
television. -- Cal Keegan



 Apr 30 2017



 next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 30 April 2017 at 19:52:27 UTC, H. S. Teoh wrote:

 On Sun, Apr 30, 2017 at 01:26:09PM +0000, Stefan Koch via 
 Digitalmars-d wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 
 Big news!
 The first step to include debug info has been done.
 
 Yes this means you will be able to step through ctfe code 
 while the compiler executes it.


 Wow! Will that be accessible to users in the end?  That could 
 be a totally awesome way of debugging CTFE code!


 T


Yes the plan is to make it accessible for the advanced user.
probably with a really bad ui, though (since I am awful at UI 
code).



 May 01 2017



  parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Mon, May 01, 2017 at 06:23:08PM +0000, Stefan Koch via Digitalmars-d wrote:

 On Sunday, 30 April 2017 at 19:52:27 UTC, H. S. Teoh wrote:

 On Sun, Apr 30, 2017 at 01:26:09PM +0000, Stefan Koch via Digitalmars-d
 wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


 
 Big news!  The first step to include debug info has been done.
 
 Yes this means you will be able to step through ctfe code while
 the compiler executes it.


 
 Wow! Will that be accessible to users in the end?  That could be a
 totally awesome way of debugging CTFE code!
 
 
 T


 
 Yes the plan is to make it accessible for the advanced user.  probably
 with a really bad ui, though (since I am awful at UI code).


I'm not sure about providing a debugger UI inside the compiler itself...
it's certainly possible, and could lead to interesting new ways of using
a compiler, but I was thinking more along the lines of providing the
necessary hooks so that you could attach an external debugger to the
CTFE engine.

But if the debugger UI is simple enough, perhaps having it built into
the compiler may not be a bad thing. It would also avoid potential
trouble caused by some platforms not having debuggers capable of
plugging into the compiler in that way.  But still, I can see people
demanding IDE integration for this eventually... :-O


T

-- 
Obviously, some things aren't very obvious.



 May 01 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 1 May 2017 at 19:06:24 UTC, H. S. Teoh wrote:

 On Mon, May 01, 2017 at 06:23:08PM +0000, Stefan Koch via 
 Digitalmars-d wrote:

 [...]


 I'm not sure about providing a debugger UI inside the compiler 
 itself... it's certainly possible, and could lead to 
 interesting new ways of using a compiler, but I was thinking 
 more along the lines of providing the necessary hooks so that 
 you could attach an external debugger to the CTFE engine.

 But if the debugger UI is simple enough, perhaps having it 
 built into the compiler may not be a bad thing. It would also 
 avoid potential trouble caused by some platforms not having 
 debuggers capable of plugging into the compiler in that way.  
 But still, I can see people demanding IDE integration for this 
 eventually... :-O


 T


I intended for the debugging functionality to be exposed via a 
udp socket listening on localhost.
Such that a debug-ui does not have to deal with ipc difficulties.
I am strictly against building a debugger into dmd.



 May 02 2017



 next sibling parent  "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Tue, May 02, 2017 at 09:55:56AM +0000, Stefan Koch via Digitalmars-d wrote:
[...]

 I intended for the debugging functionality to be exposed via a udp
 socket listening on localhost.
 Such that a debug-ui does not have to deal with ipc difficulties.
 I am strictly against building a debugger into dmd.


Nice, that's a good approach.


T

-- 
Bomb technician: If I'm running, try to keep up.



 May 02 2017



prev sibling  parent reply Moritz Maxeiner <moritz ucworks.org>  writes:


On Tuesday, 2 May 2017 at 09:55:56 UTC, Stefan Koch wrote:

 [...]

 I intended for the debugging functionality to be exposed via a 
 udp socket listening on localhost.
 Such that a debug-ui does not have to deal with ipc 
 difficulties.


Hm, rationale for UDP over TCP here? I would assume one wouldn't 
want debugging info to be delivered out of order (or not at all); 
I mean, I guess it would be ok for localhost only (though one is 
then depending on implementation specifics vs. protocol 
semantics), but *if* you use sockets, you will eventually get 
people who use that over the network (and then the fun begins). 
Using TCP would just remove any potential future headache from 
the equation.



 May 02 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 2 May 2017 at 22:08:31 UTC, Moritz Maxeiner wrote:

 On Tuesday, 2 May 2017 at 09:55:56 UTC, Stefan Koch wrote:

 [...]

 I intended for the debugging functionality to be exposed via a 
 udp socket listening on localhost.
 Such that a debug-ui does not have to deal with ipc 
 difficulties.


 Hm, rationale for UDP over TCP here? I would assume one 
 wouldn't want debugging info to be delivered out of order (or 
 not at all); I mean, I guess it would be ok for localhost only 
 (though one is then depending on implementation specifics vs. 
 protocol semantics), but *if* you use sockets, you will 
 eventually get people who use that over the network (and then 
 the fun begins). Using TCP would just remove any potential 
 future headache from the equation.


I think any ordering should be done explicitly at the debugging 
protocol level.
for example when sending sourceline messages the order is given 
by the line number and ordering can be done by the application.
It's the same for breakpoint setting or for breakpoint trigger 
notification.
As for lost packages, we want to respond intelligently as well.
And maybe reduce the amount of data in the package, to make 
arrival of future packages more likely.



 May 02 2017



 next sibling parent reply Adrian Matoga <dlang.spam matoga.info>  writes:


On Wednesday, 3 May 2017 at 04:22:00 UTC, Stefan Koch wrote:

 On Tuesday, 2 May 2017 at 22:08:31 UTC, Moritz Maxeiner wrote:

 [...]
 Using TCP would just remove any potential future headache from 
 the equation.


 I think any ordering should be done explicitly at the debugging 
 protocol level.
 for example when sending sourceline messages the order is given 
 by the line number and ordering can be done by the application.
 It's the same for breakpoint setting or for breakpoint trigger 
 notification.
 As for lost packages, we want to respond intelligently as well.
 And maybe reduce the amount of data in the package, to make 
 arrival of future packages more likely.


So you're going to reinvent TCP in your debugging protocol?



 May 02 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 3 May 2017 at 06:10:22 UTC, Adrian Matoga wrote:

 So you're going to reinvent TCP in your debugging protocol?



No. there is no need for a full blown recovery mechanism.

For the typical usecase a lossless orderd connection can be 
assumed.

And most things are not order dependent



 May 03 2017



 next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Wednesday, 3 May 2017 at 07:35:56 UTC, Stefan Koch wrote:

 On Wednesday, 3 May 2017 at 06:10:22 UTC, Adrian Matoga wrote:

 So you're going to reinvent TCP in your debugging protocol?



 No. there is no need for a full blown recovery mechanism.

 For the typical usecase a lossless orderd connection can be 
 assumed.

 And most things are not order dependent


What about packet boundaries?



 May 03 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 3 May 2017 at 08:23:54 UTC, Nordlöw wrote:

 On Wednesday, 3 May 2017 at 07:35:56 UTC, Stefan Koch wrote:

 On Wednesday, 3 May 2017 at 06:10:22 UTC, Adrian Matoga wrote:

 So you're going to reinvent TCP in your debugging protocol?



 No. there is no need for a full blown recovery mechanism.

 For the typical usecase a lossless orderd connection can be 
 assumed.

 And most things are not order dependent


 What about packet boundaries?


We send source line by line.
Packets should be under 1k in most cases.



 May 03 2017



prev sibling next sibling parent  Moritz Maxeiner <moritz ucworks.org>  writes:


On Wednesday, 3 May 2017 at 07:35:56 UTC, Stefan Koch wrote:

 For the typical usecase a lossless orderd connection can be 
 assumed.


- udp is not connection oriented, i.e. there is no connection
- udp is not lossless and pretending it is means setting yourself 
up for a headache down the road
- udp datagrams are not guaranteed to arrive in the order they 
were sent and pretending they do is also setting yourself up for 
a headache down the road

What you've described so far is a classic, textbook use case of 
tcp.



 May 03 2017



prev sibling  parent  Minty Fresh <mint fresh.com>  writes:


On Wednesday, 3 May 2017 at 07:35:56 UTC, Stefan Koch wrote:

 On Wednesday, 3 May 2017 at 06:10:22 UTC, Adrian Matoga wrote:

 So you're going to reinvent TCP in your debugging protocol?



 No. there is no need for a full blown recovery mechanism.

 For the typical usecase a lossless orderd connection can be 
 assumed.

 And most things are not order dependent


The debugger isn't a massive, real-time system that needs to 
service thousands of clients and squeeze as much performance out 
of the network as possible. The overhead incurred by TCP is 
essentially not worth considering for something that's going to 
be run over localhost 90%, and service just the one client.

Reinventing the wheel adds a far bigger overhead: Maintaining 
your new protocol.
TCP implementations are readily available. They're well 
maintained, well documented, and are essentially guaranteed to 
work across platforms.
Implementing a new protocol just adds an extra point of breakage 
for little to no gain. It also incurs the cost of the associated 
development time, and - down the line - any time spent fixing or 
iterating. Not to mention that tests need to be written, 
documentation needs to be put in place.

A classic case of premature optimization.



 May 03 2017



prev sibling  parent  Moritz Maxeiner <moritz ucworks.org>  writes:


On Wednesday, 3 May 2017 at 04:22:00 UTC, Stefan Koch wrote:

 [...]

 I think any ordering should be done explicitly at the debugging 
 protocol level.
 for example when sending sourceline messages the order is given 
 by the line number and ordering can be done by the application.
 It's the same for breakpoint setting or for breakpoint trigger 
 notification.


Why? If I were to write a client for the debugging protocol, I 
wouldn't want to write protocol ordering logic (and essentially 
reimplement part of tcp). I would want to react to protocol 
messages as they arrive.


 As for lost packages, we want to respond intelligently as well.


The only way I know of to respond intelligently to lost packages 
using udp - if you care about the information in them (which we 
do in this use case) - is to implement a retransmit mechanism; 
i.e. you would be reimplementing another part of tcp.


 And maybe reduce the amount of data in the package, to make 
 arrival of future packages more likely.


You assume a causation between udp datagram size and delivery 
probability, which - however likely - is an implementation detail.



 May 03 2017



prev sibling  parent  Adrian Matoga <dlang.spam matoga.info>  writes:


On Sunday, 30 April 2017 at 19:52:27 UTC, H. S. Teoh wrote:

 On Sun, Apr 30, 2017 at 01:26:09PM +0000, Stefan Koch via 
 Digitalmars-d wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 
 Big news!
 The first step to include debug info has been done.
 
 Yes this means you will be able to step through ctfe code 
 while the compiler executes it.


 Wow! Will that be accessible to users in the end?  That could 
 be a totally awesome way of debugging CTFE code!


I used to think the same, but with each new line of code I write 
that is to be executed in CT, I become more convinced that 
there's no need to expose such debugging interface to the end 
user. Why? Because the end user expects that CTFE either gives 
exactly the same results as run-time execution or that it stops 
with an explicit error message on something that is not designed 
to be executed in CT.  Any other problem found during CTFE 
execution must be 100% reproducible in run time or it's an ICE.
Any CTFE debugging should be only for compiler maintainers, and 
the user shouldn't worry that CTFE could mess up something.



 May 03 2017



prev sibling  parent  Swoorup Joshi <swoorupjoshi gmail.com>  writes:


On Sunday, 30 April 2017 at 13:26:09 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 Big news!
 The first step to include debug info has been done.

 Yes this means you will be able to step through ctfe code while 
 the compiler executes it.


This should have been kept secret before C++ steals it and does 
not give credit to D. :D



 May 03 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Thanks to Daniel Murphy's input; '&&' works now in my 
experimental version.
I hope to get it to pass the auto-tester soon!

This is a big step :)

Thanks Daniel.



 May 10 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi Guys,

Outer function arguments are now supperted.

meaning this code will now work:

int[] filterBy(int[] arr , bool function(uint) fn)
{
     int[] result = [];
     uint resultLength;

     result.length = arr.length;
     foreach(i;0 .. arr.length)
     {
         auto e = arr[i];
         bool r = true;
         r = fn(e);
         if(r)
         {
             result[resultLength++] = e;
         }
     }

    int[] filterResult;
    filterResult.length = resultLength;

    foreach(i; 0 .. resultLength)
    {
      filterResult[i] = result[i];
    }

   return filterResult;
}

bool isDiv2(uint e)
{
   bool result;
   result = (e % 2 == 0);
   return result;
}


static assert(filterBy([3,4,5], &isDiv2) == [4]);

before this would behaved very strangely ;)
because isDiv would have been executed instead filterBy.
And since after bytecode compilation there is no type checking 
anymore the arrayPtr would have been interpreter as an integer.
(which is always 4byte aligend)
that would have caused the isDiv to return 1;
which would have been interpreted as address.
and whatever was there would have been treated as array 
descriptor.
resulting in mostly the [] return value.

...
anyway.
I am happy this is fixed now.



 May 12 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Friday, 12 May 2017 at 11:21:56 UTC, Stefan Koch wrote:


 ...
 anyway.
 I am happy this is fixed now.


Now I am less happy.
The fallout of this fix causes code in std.ascii to miscompile.
Apperantly we don't make sure our function list is cleared before 
finalization.



 May 12 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


So I have fixed a few cases of outer function evaluation.
Unfortunately this exposed some hard to track down bugs in how 
expressions are handled.
The JIT and Debugger features are on ice, until those bugs are 
eliminated.

Sorry about the delay.



 May 16 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


So ...
I just encountered more ABI issues; related to slices which are 
part of structures.
struct R
{
   uint[] s1;
   uint[] s2;
}

like this.

R returnSlices(int[] s1, int[] s2)
{
     return R(s1[], s2[]);
}
static assert(returnSlices([1,2,3], [4,5,6,7]) == 
R([1,2,3][4.5.6.7])); // works


R returnSlicedSlices(int[] s1, int[] s2)
{
     return R(s1[], s2[1 .. $-1]);
}
static assert(returnSlicedSlices([1,2,3], [4,5,6,7]) == 
R([1,2,3],[5,6])); // fails
// returns R([1,2,3],null); at the moment

The reason ABI issues.
Where exactly ? No Idea.



 May 16 2017



 next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 16 May 2017 at 13:44:27 UTC, Stefan Koch wrote:

 [ ... ]
 The reason ABI issues.
 Where exactly ? No Idea.


Not just abi issues ...
There are more fundamental problems where we sometimes forget to 
allocate space, for locals (of composite types).



 May 17 2017



prev sibling  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 16 May 2017 at 13:44:27 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 So ...
 I just encountered more ABI issues; related to slices which are 
 part of structures.
 struct R
 {
   uint[] s1;
   uint[] s2;
 }

 like this.

 R returnSlices(int[] s1, int[] s2)
 {
     return R(s1[], s2[]);
 }
 static assert(returnSlices([1,2,3], [4,5,6,7]) == 
 R([1,2,3][4.5.6.7])); // works


 R returnSlicedSlices(int[] s1, int[] s2)
 {
     return R(s1[], s2[1 .. $-1]);
 }
 static assert(returnSlicedSlices([1,2,3], [4,5,6,7]) == 
 R([1,2,3],[5,6])); // fails
 // returns R([1,2,3],null); at the moment

 The reason ABI issues.
 Where exactly ? No Idea.


Huh apparently I fixed this issue.
But I cannot rememberer when or how I did it.
This test will now magically work.
This concerns me.



 May 21 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 21 May 2017 at 14:49:32 UTC, Stefan Koch wrote:

 Huh apparently I fixed this issue.
 But I cannot rememberer when or how I did it.
 This test will now magically work.
 This concerns me.


I finally figured out what was going on.
I forgot to activate the -bc-ctfe switch ....
A few days of work into the wrong direction *sigh*



 May 23 2017



  parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com>  writes:


On Tuesday, 23 May 2017 at 17:26:01 UTC, Stefan Koch wrote:

 On Sunday, 21 May 2017 at 14:49:32 UTC, Stefan Koch wrote:

 Huh apparently I fixed this issue.
 But I cannot rememberer when or how I did it.
 This test will now magically work.
 This concerns me.


 I finally figured out what was going on.
 I forgot to activate the -bc-ctfe switch ....
 A few days of work into the wrong direction *sigh*


Seems like you need to make -bc-ctfe the default in your dmd 
branch :P



 May 23 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 23 May 2017 at 17:46:23 UTC, Petar Kirov [ZombineDev] 
wrote:

 Seems like you need to make -bc-ctfe the default in your dmd 
 branch :P


That is what I usually do, however I re-enabled the switch in 
order to test which error messages the current Interpreter throws 
out.



 May 23 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


So I just verified that the following code runs fine with newCTFE.

int[2] aaa2(bool b1, bool b2, bool b3)
{
   int x = 0;
   if (b1 && ++x && b2 && x++ && b3 && x++)
   {
     return [x, 1];
   }
   else
   {
     return [x, 0];
   }
}

static assert(aaa2(1, 0, 1) == [1, 0]);
static assert(aaa2(1, 1, 1) == [3, 1]);
static assert(aaa2(0, 0, 1) == [0, 0]);



 May 20 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 ...


|| works also fine now ... on it's own.
As soon as || and && are mixed wired things happen.
I am happy that the combinations I tried do at least expose the 
issue.
This is one of the things that you can totally overlook if the 
test-case does not trigger it.

int[2] aaa2(bool b1, bool b2, bool b3, bool b4)
{
   int x = 0;
   if (b1 && ++x && b2 && x++ && b3 && (b4 || x++))
   {
     return [x, 1];
   }
   else
   {
     return [x, 0];
   }
}

static assert(aaa2(1, 0, 1, 0) == [1, 0]);
//static assert(aaa2(1, 1, 1, 0) == [3, 1]); // argh
static assert(aaa2(1, 1, 1, 1) == [2, 1]);
static assert(aaa2(0, 0, 1, 0) == [0, 0]);


int[2] ooo2(bool b1, bool b2, bool b3, bool b4)
{
   int x = 0;
   if (b1 || x++ || b2 || !x++ || b3 || (b4 && x++))
   {
     return [x, 1];
   }
   else
   {
     return [x, 0];
   }
}

static assert(ooo2(1, 0, 1, 0) == [0, 1]);
static assert(ooo2(0, 1, 1, 0) == [1, 1]);
static assert(ooo2(0, 0, 1, 0) == [2, 1]);
static assert(ooo2(0, 0, 0, 0) == [2, 0]);
//static assert(ooo2(0, 0, 0, 1) == [3, 1]); // oh god ...



 May 20 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi Guys,

I just fixed the sliceAssignment!
now overlapping assignments are correctly detected.

I also re-enabled a bailout on struct-member operation which are 
not simple integers.
Which means I chose to ignore some ABI issues for now ....

With that the dmd unittests are green again.

Phobos still fails an I am investigating.



 May 28 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Sunday, 28 May 2017 at 22:26:18 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 Hi Guys,

 I just fixed the sliceAssignment!
 now overlapping assignments are correctly detected.

 I also re-enabled a bailout on struct-member operation which 
 are not simple integers.
 Which means I chose to ignore some ABI issues for now ....

 With that the dmd unittests are green again.

 Phobos still fails an I am investigating.


The failing phobos test works now.
However due to the corrected expression handling, I now encounter 
cases, which I overlooked before.
Thus we now fail the dmd testsuite.



 Jun 03 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ .. ]


Hi Guys,

I am still working on the buggy Slice-Assignment.
Just now I fixed a bug where an [Bytecode-Assert]
would flip it's the condition ... and trigger whenever it was 
true instead of false.
Assertions are one of things which cannot be tested at 
compiletime since they need to interact with dmd.... bummer.

Originally I intended for floating-point support to be 
implemented in this and the following month. Looks like I am not 
gonna make it.

My main system is waiting for replacement parts.
And as you know my laptop cannot link dmd when a gui is active... 
so that's not great either.

Cheers,
Stefan



 Jun 05 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 5 June 2017 at 20:51:03 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ .. ]


 Hi Guys,

 I am still working on the buggy Slice-Assignment.
 Just now I fixed a bug where an [Bytecode-Assert]
 would flip it's the condition ... and trigger whenever it was 
 true instead of false.
 Assertions are one of things which cannot be tested at 
 compiletime since they need to interact with dmd.... bummer.

 Originally I intended for floating-point support to be 
 implemented in this and the following month. Looks like I am 
 not gonna make it.

 My main system is waiting for replacement parts.
 And as you know my laptop cannot link dmd when a gui is 
 active... so that's not great either.

 Cheers,
 Stefan


I solved the slice assignment case.
Now here is a subtle bug for you.

As you may know, newCTFE has it's own type-system in order to 
help make the IR simpler.
Later on typed version of the ir gets lowered though until only 
int32 expressions remain.
When processing Slice expression I lowered to early though, and 
returned an i32 to be interpreted as pointer to a slice 
descriptor.
However in the slice-assignment code I wanted to the the 
elmementType form the SliceExp, which came back as 
BCType(invalid).
Then I took it the size of it, which came back as 0. (an invalid 
size for an invalid type).
And multiplied the index and length, with it to move to the 
correct point in the slice-buffer.

That comes back as target[0 .. 0] = source[0 .. 0].
So no modification occurs.

Time to find this: roughly 2 weeks.

Why did it take so long ?
Because I thought the ABI was at fault; And spent a lot of time 
staring on memory dumps :(



 Jun 05 2017



  parent reply jmh530 <john.michael.hall gmail.com>  writes:


On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote:

 Time to find this: roughly 2 weeks.


Damn. That's some commitment.



 Jun 05 2017



 next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 6 June 2017 at 02:03:46 UTC, jmh530 wrote:

 On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote:

 Time to find this: roughly 2 weeks.


 Damn. That's some commitment.


There is no other way, really.
These things need to be fixed.



 Jun 05 2017



  parent  Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 6 June 2017 at 04:11:33 UTC, Stefan Koch wrote:

 On Tuesday, 6 June 2017 at 02:03:46 UTC, jmh530 wrote:

 On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote:

 Time to find this: roughly 2 weeks.


 Damn. That's some commitment.


 There is no other way, really.
 These things need to be fixed.


Great work. Keep up.



 Jun 06 2017



prev sibling  parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Tue, Jun 06, 2017 at 02:03:46AM +0000, jmh530 via Digitalmars-d wrote:

 On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote:

 
 Time to find this: roughly 2 weeks.
 


 
 Damn. That's some commitment.


2 weeks is not bad for subtle bugs in complex code like this one. In my
day job I've seen bugs that took 2 *months* to figure out. One of them
involved a rare race condition that can only be reproduced under very
specific circumstances, and it took a long time and a lot of guesswork
before a coworker and myself discovered the exact combination that
triggered the bug, thereby leading to the subtle problem in a piece of
code that looked perfectly innocuous before then.


T

-- 
A mathematician is a device for turning coffee into theorems. -- P. Erdos



 Jun 06 2017



 next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 6 June 2017 at 16:39:07 UTC, H. S. Teoh wrote:

 On Tue, Jun 06, 2017 at 02:03:46AM +0000, jmh530 via 
 Digitalmars-d wrote:

 On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote:

 
 Time to find this: roughly 2 weeks.
 


 
 Damn. That's some commitment.


 2 weeks is not bad for subtle bugs in complex code like this 
 one. In my day job I've seen bugs that took 2 *months* to 
 figure out. One of them involved a rare race condition that can 
 only be reproduced under very specific circumstances, and it 
 took a long time and a lot of guesswork before a coworker and 
 myself discovered the exact combination that triggered the bug, 
 thereby leading to the subtle problem in a piece of code that 
 looked perfectly innocuous before then.


 T


Wow, 2 Months.
And I always feel slow, when a bug takes more then a week.
Luckily my architecture is designed to be completely 
deterministic and reproducibly.
So things like race conditions cannot hit me.
... Thank god for that.



 Jun 06 2017



prev sibling  parent reply Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 6/6/17 12:39 PM, H. S. Teoh via Digitalmars-d wrote:

 On Tue, Jun 06, 2017 at 02:03:46AM +0000, jmh530 via Digitalmars-d wrote:

 On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote:

 Time to find this: roughly 2 weeks.


 Damn. That's some commitment.


 2 weeks is not bad for subtle bugs in complex code like this one. In my
 day job I've seen bugs that took 2 *months* to figure out. One of them
 involved a rare race condition that can only be reproduced under very
 specific circumstances, and it took a long time and a lot of guesswork
 before a coworker and myself discovered the exact combination that
 triggered the bug, thereby leading to the subtle problem in a piece of
 code that looked perfectly innocuous before then.


Oh, I've had those before. I had a race condition that reproduced 
*randomly* and usually took about 2 weeks to happen, and that's by 
pounding it non-stop. The result was deadlock. Any debugging after the 
fact resulted in no clues.

Only way I solved it was to print out state as it was going, so I could 
see what happened when the state went bad. I think it took at least 2 
cycles to find it.

This kind of stuff makes you appreciate how important avoiding race 
conditions and memory corruption is.

-Steve



 Jun 06 2017



  parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Tue, Jun 06, 2017 at 02:23:59PM -0400, Steven Schveighoffer via
Digitalmars-d wrote:

 On 6/6/17 12:39 PM, H. S. Teoh via Digitalmars-d wrote:

 On Tue, Jun 06, 2017 at 02:03:46AM +0000, jmh530 via Digitalmars-d wrote:

 On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote:

 
 Time to find this: roughly 2 weeks.
 


 
 Damn. That's some commitment.


 
 2 weeks is not bad for subtle bugs in complex code like this one. In
 my day job I've seen bugs that took 2 *months* to figure out. One of
 them involved a rare race condition that can only be reproduced
 under very specific circumstances, and it took a long time and a lot
 of guesswork before a coworker and myself discovered the exact
 combination that triggered the bug, thereby leading to the subtle
 problem in a piece of code that looked perfectly innocuous before
 then.


 
 Oh, I've had those before. I had a race condition that reproduced
 *randomly* and usually took about 2 weeks to happen, and that's by
 pounding it non-stop. The result was deadlock. Any debugging after the
 fact resulted in no clues.
 
 Only way I solved it was to print out state as it was going, so I
 could see what happened when the state went bad. I think it took at
 least 2 cycles to find it.
 
 This kind of stuff makes you appreciate how important avoiding race
 conditions and memory corruption is.


[...]

Yeah, race conditions and memory corruption / pointer bugs are the worst
to track down.  Since the codebase I deal with is in C, there are plenty
of opportunities for slip-ups that lead to pointer bugs.  And the worst
of them are dangling pointers... you write to them, and there's no SEGV
because they point to valid memory, but that memory has been allocated
to something else now.  By the time the corruption manifests itself,
you're already long, long past the original buggy code, usually in some
completely-innocent code that you can stare at for weeks or months and
not find a single flaw.  Meanwhile the original bug randomly corrupts
different things depending on who gets the memory pointed to by the bad
pointer, making it almost impossible to reproduce. Even after you
reproduce it, you've no idea how to trace it to the original cause,
because you're long past where it happened.  And it's almost impossible
to narrow it down, because reducing the test case may make the bad
pointer corrupt something else that you don't see, so you don't know if
the bug is still happening or not.

Things like this make you *really* appreciate D features like bounds
checking and the oft-maligned but life-saving GC.


T

-- 
Error: Keyboard not attached. Press F1 to continue. -- Yoon Ha Lee, CONLANG



 Jun 06 2017



 next sibling parent  Paolo Invernizzi <paolo.invernizzi gmail.com>  writes:


On Tuesday, 6 June 2017 at 18:51:58 UTC, H. S. Teoh wrote:

 Things like this make you *really* appreciate D features <snip> 
 and the oft-maligned but life-saving GC.


+1000! GC is an opportunity, not a burden! :-P

/Paolo



 Jun 06 2017



prev sibling  parent reply Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 6/6/17 2:51 PM, H. S. Teoh via Digitalmars-d wrote:

 On Tue, Jun 06, 2017 at 02:23:59PM -0400, Steven Schveighoffer via
Digitalmars-d wrote:

 On 6/6/17 12:39 PM, H. S. Teoh via Digitalmars-d wrote:

 On Tue, Jun 06, 2017 at 02:03:46AM +0000, jmh530 via Digitalmars-d wrote:

 On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote:

 Time to find this: roughly 2 weeks.


 Damn. That's some commitment.


 2 weeks is not bad for subtle bugs in complex code like this one. In
 my day job I've seen bugs that took 2 *months* to figure out. One of
 them involved a rare race condition that can only be reproduced
 under very specific circumstances, and it took a long time and a lot
 of guesswork before a coworker and myself discovered the exact
 combination that triggered the bug, thereby leading to the subtle
 problem in a piece of code that looked perfectly innocuous before
 then.


 Oh, I've had those before. I had a race condition that reproduced
 *randomly* and usually took about 2 weeks to happen, and that's by
 pounding it non-stop. The result was deadlock. Any debugging after the
 fact resulted in no clues.

 Only way I solved it was to print out state as it was going, so I
 could see what happened when the state went bad. I think it took at
 least 2 cycles to find it.

 This kind of stuff makes you appreciate how important avoiding race
 conditions and memory corruption is.


 [...]

 Yeah, race conditions and memory corruption / pointer bugs are the worst
 to track down.  Since the codebase I deal with is in C, there are plenty
 of opportunities for slip-ups that lead to pointer bugs.  And the worst
 of them are dangling pointers... you write to them, and there's no SEGV
 because they point to valid memory, but that memory has been allocated
 to something else now.  By the time the corruption manifests itself,
 you're already long, long past the original buggy code, usually in some
 completely-innocent code that you can stare at for weeks or months and
 not find a single flaw.  Meanwhile the original bug randomly corrupts
 different things depending on who gets the memory pointed to by the bad
 pointer, making it almost impossible to reproduce. Even after you
 reproduce it, you've no idea how to trace it to the original cause,
 because you're long past where it happened.  And it's almost impossible
 to narrow it down, because reducing the test case may make the bad
 pointer corrupt something else that you don't see, so you don't know if
 the bug is still happening or not.

 Things like this make you *really* appreciate D features like bounds
 checking and the oft-maligned but life-saving GC.


Yep, there were memory errors too. We used a proprietary tool that was 
like valgrind (this was before valgrind existed) called purify to find 
those. I think most of them were either double-freeing/deleting 
something (usually in a destructor that was called more than once -- 
always set your members that you deleted to null), or freeing new'd 
memory/deleting malloc'd memory. Thought I had everything, and then the 
hang. At first we thought it was a memory issue not caught by purify, 
but then we found it eventually later as I described.

I can characterize memory corruption bugs as errors that occur randomly 
and can manifest in any kind of behavior. Even more nasty is sometimes 
they happen in code that you didn't even touch, because it was *always* 
happening, but just didn't cause a bug until you changed memory 
organization around slightly. Race conditions are also generally random 
but usually manifest the same way. Both are nasty to find and debug.

I don't miss those days :)

-Steve



 Jun 06 2017



  parent  "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Tue, Jun 06, 2017 at 03:35:11PM -0400, Steven Schveighoffer via
Digitalmars-d wrote:
[...]

 I can characterize memory corruption bugs as errors that occur
 randomly and can manifest in any kind of behavior. Even more nasty is
 sometimes they happen in code that you didn't even touch, because it
 was *always* happening, but just didn't cause a bug until you changed
 memory organization around slightly.


[...]

Yep, that's exactly the kind of bug I'm talking about.  The kind that
can appear/disappear depending on the order you link your object files,
or declare dummy variables in an unrelated function, or whether you
compile with debugging symbols (because that changes the memory layout
of your program enough to make the symptoms disappear). The latter is
the worst variant of its kind, because it means you're up Bug Creek
without any debugging symbol paddles to help you. (And yes, I've
actually had to deal with that before. It was not pretty.)


T

-- 
"If you're arguing, you're losing." -- Mike Thomas



 Jun 06 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi there,

I just pulled another all nighter.
I found a bug in the code that was supposed to adjust the values 
of || and &&.
As will as a mixup in the error messages for overlapping 
slice-assignment.
Both are fixed.

I am aware that overlapping slice assignment check is not yet 
good enough.
This is on my short-term todo list.
But in the even shorter-term, there are a couple hours sleep 
waiting to be claimed.

Night guys,
Stefan



 Jun 08 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Slice Assignment bugs fixed!
With that we are green again.

I am going to improve concat a little such that it computes the 
buffer lengths if it can.
And allocs the needed amount upfront.

The alternative is lazy concat which is much more work. With alot 
of potential for bugs.

|| and && still have me puzzledw :)



 Jun 13 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


newCTFE specific tests are now included in my version of the 
test-suite.
No more silent breakage. (Atleast for anything covered by the 
test.)
For this to work I introduced a __trait which returns true if 
newCTFE bailed-out on something, and false otherwise.

So far it seems to work quite well.



 Jun 19 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


limited Support for 32bit floating point ops [+, -, *, /, %] has 
just landed in newCTFE.

float fmadd(float a, float b, float c)
{
     return b + a * c;
}

pragma(msg, fmadd(6.7, 8.9, 1.3)/* == 17.61f*/);
pragma(msg, fmadd(6.7, 8.9, -1.3)/* == 19.00f*/);



 Jun 20 2017



 next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 20 June 2017 at 12:07:00 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 limited Support for 32bit floating point ops [+, -, *, /, %] 
 has just landed in newCTFE.

 float fmadd(float a, float b, float c)
 {
     return b + a * c;
 }

 pragma(msg, fmadd(6.7, 8.9, 1.3)/* == 17.61f*/);
 pragma(msg, fmadd(6.7, 8.9, -1.3)/* == 19.00f*/);


These will pass with newCTFE:
static assert(fmadd(0x1.acccccp+2, 0x1.166666p+3, 0x1.4cccccp+0) 
== 0x1.168f5cp+4);
static assert(fmadd(0x1.acccccp+2, 0x1.166666p+3, -0x1.4cccccp+0) 
== -0x1.47a8p-7);

Proving that newCTFE's floating-point-math works the same as 
runtime-floating-point math.
Which is not surprising since the interpreter uses same code as 
the runtime version.
At-least if you bootstrap dmd with itself.



 Jun 20 2017



prev sibling next sibling parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 20 June 2017 at 12:07:00 UTC, Stefan Koch wrote:

 limited Support for 32bit floating point ops [+, -, *, /, %] 
 has just landed in newCTFE.


Nice!



 Jun 20 2017



prev sibling next sibling parent  "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Tue, Jun 20, 2017 at 12:07:00PM +0000, Stefan Koch via Digitalmars-d wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


 
 limited Support for 32bit floating point ops [+, -, *, /, %] has just
 landed in newCTFE.


Awesome!!!!!



 float fmadd(float a, float b, float c)
 {
     return b + a * c;
 }
 
 pragma(msg, fmadd(6.7, 8.9, 1.3)/* == 17.61f*/);
 pragma(msg, fmadd(6.7, 8.9, -1.3)/* == 19.00f*/);


Awesome stuff.  Can't wait to get my hands on the newCTFE engine for my
math code!


T

-- 
Written on the window of a clothing store: No shirt, no shoes, no service.



 Jun 20 2017



prev sibling  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 20 June 2017 at 12:07:00 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 limited Support for 32bit floating point ops [+, -, *, /, %] 
 has just landed in newCTFE.

 float fmadd(float a, float b, float c)
 {
     return b + a * c;
 }

 pragma(msg, fmadd(6.7, 8.9, 1.3)/* == 17.61f*/);
 pragma(msg, fmadd(6.7, 8.9, -1.3)/* == 19.00f*/);


Cheer Guys!
64bit floating point is in!

Hit me with brittle numeric code please!
At the same time 64bit integer support was expanded.
Such that we can now return long and ulong values.

Unfortunately this also broke the phobos unitttests since now 
more of is attempted to be evaluated.



 Jun 20 2017



 next sibling parent reply David Nadlinger <code klickverbot.at>  writes:


On Tuesday, 20 June 2017 at 17:35:28 UTC, Stefan Koch wrote:

 Hit me with brittle numeric code please!
 […]

 Unfortunately this also broke the phobos unitttests since now 
 more of is attempted to be evaluated.


Just making sure that the Phobos unit tests pass at compile time 
(with 64 bit reals, if that is what you support) should be a good 
start.

  — David



 Jun 20 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 20 June 2017 at 18:58:36 UTC, David Nadlinger wrote:

 On Tuesday, 20 June 2017 at 17:35:28 UTC, Stefan Koch wrote:

 Hit me with brittle numeric code please!
 […]

 Unfortunately this also broke the phobos unitttests since now 
 more of is attempted to be evaluated.


 Just making sure that the Phobos unit tests pass at compile 
 time (with 64 bit reals, if that is what you support) should be 
 a good start.

  — David


Fixed.



 Jun 20 2017



  parent reply David Nadlinger <code klickverbot.at>  writes:


On Tuesday, 20 June 2017 at 19:01:06 UTC, Stefan Koch wrote:

 On Tuesday, 20 June 2017 at 18:58:36 UTC, David Nadlinger wrote:

 On Tuesday, 20 June 2017 at 17:35:28 UTC, Stefan Koch wrote:

 Hit me with brittle numeric code please!
 […]

 Unfortunately this also broke the phobos unitttests since now 
 more of is attempted to be evaluated.


 Just making sure that the Phobos unit tests pass at compile 
 time (with 64 bit reals, if that is what you support) should 
 be a good start.

  — David


 Fixed.


But how much of the std.math code are you actually executing with 
newCTFE? What I meant is that if the std.math{,special} 
implementations of the various mathematical functions work, there 
shouldn't be any egregious issues. I'm not sure how much of it is 
usually run during CTFE, though.

  — David



 Jun 20 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 20 June 2017 at 19:37:47 UTC, David Nadlinger wrote:

 But how much of the std.math code are you actually executing 
 with newCTFE? What I meant is that if the std.math{,special} 
 implementations of the various mathematical functions work, 
 there shouldn't be any egregious issues. I'm not sure how much 
 of it is usually run during CTFE, though.

  — David


I just checked.
I execute exactly 0% of it.
It mostly bails out on || and &&.



 Jun 20 2017



prev sibling  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Tuesday, 20 June 2017 at 17:35:28 UTC, Stefan Koch wrote:

 At the same time 64bit integer support was expanded.
 Such that we can now return long and ulong values.

 Unfortunately this also broke the phobos unitttests since now 
 more of is attempted to be evaluated.


The reason this broke was because we do not handle default 
arguments in calls correctly.
I have added a bailout such that we will not try to call 
functions with default arguments.

with that we should pass phobos uinttests again.



 Jun 20 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


I just figured out howto fix the defaultArg situation.
We just have to check at or slightly before we emit the Call 
instruction,
if we have enough arguments for the function.
If we don't we pull the missing ones from the default arguments.
This does have the problem that we can generate the same default 
once per call.
However we would have the same problem if a user typed the 
default argument manually.

Therefore I argue we should emit the code rather then demanding 
something clever from the backends.

Clever is usually the opposite of smart :) (In those kinds of 
things)



 Jun 20 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


I finally came around and implemented 64bit as a hacky extension 
to the interpreter.

ulong[5] testArrayAssignL()
{
     typeof(return) arr = 12;
     arr[3] = 4;
     arr[0] = 1;
     return arr;
}

static immutable arrL = testArrayAssignL();
pragma(msg, arrL); // outputs: [1LU, 12LU, 12LU, 4LU, 12LU]

This should naturally extend to arrays of floats or doubles as 
well.



 Jun 25 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi There,

I just wrote the first parts of the ctfe debugger.
Ctfe-Breakpoints are working :)

It looks like we will be able to have greatly enhanced debugging 
experience soon.
(okay soonish ... since there are still a few tricky details to 
work out.)



 Jun 28 2017



  parent  "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Wed, Jun 28, 2017 at 04:19:02PM +0000, Stefan Koch via Digitalmars-d wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


 
 Hi There,
 
 I just wrote the first parts of the ctfe debugger.
 Ctfe-Breakpoints are working :)
 
 It looks like we will be able to have greatly enhanced debugging
 experience soon.
 (okay soonish ... since there are still a few tricky details to work out.)


Awesome!


T

-- 
There are four kinds of lies: lies, damn lies, and statistics.



 Jun 28 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


I just "discovered" a bug in my debugging system.
Which caused me to debug the wrong function for about 5 hours.
Since I really could not see why the codegen would produce such 
vastly different code.
(Almost as if it were generating code for a different function :/ 
)

The takeaway: Better Debuggers only safe you time if they are 
actually showing you correct information.

If your debugger is buggy then your are in a world of hurts.
I had my share of pain for today.

The good thing is however, that the floating point stuff is going 
smooth so far.
The x87 can be persuaded to almost obey the ieee standard if you 
issue a write after every operation.

The better option would be to just use sse2 all the way. But 
whatever.



 Jun 29 2017



  parent  Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 6/29/17 4:24 AM, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


 
 I just "discovered" a bug in my debugging system.
 Which caused me to debug the wrong function for about 5 hours.
 Since I really could not see why the codegen would produce such vastly 
 different code.
 (Almost as if it were generating code for a different function :/ )
 
 The takeaway: Better Debuggers only safe you time if they are actually 
 showing you correct information.
 
 If your debugger is buggy then your are in a world of hurts.
 I had my share of pain for today.


The same is true for compilers that are buggy. Debugging code is 
horrible when the compiled code isn't doing what the source says. I've 
seen my share of that as well...


 The good thing is however, that the floating point stuff is going smooth 
 so far.
 The x87 can be persuaded to almost obey the ieee standard if you issue a 
 write after every operation.
 
 The better option would be to just use sse2 all the way. But whatever.


I'm looking very much forward to newCTFE. Keep up the good work!

-Steve



 Jun 29 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


I ran in trouble with floating point.
This will probably take a week.
The debugger will take probably a month (without gui).



 Jul 01 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi Guys,

I am very pleased indeed about the following code now running in 
ctfe.

int maxFloat()
{
   float prev = 0;
   float fOne = 1;
   foreach(int i; 1 .. int.max)
   {
     if (i == prev)
       return i-1;
     else
       prev += fOne;
   }
   return int.max;
}

static assert(maxFloat() == 16777216);



 Jul 02 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Monday, 3 July 2017 at 06:15:44 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]


This code works now as well ;)

int maxFloat()
{
   float prev = 1;
   foreach(int i; 0 .. int.max)
   {
     if (i == prev++)
       return i;
   }
   return int.max;
}

static assert(maxFloat() == 16777216);



 Jul 03 2017



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hi Guys,

It's been a while since my last post.
I have now prepared the interpreter to support
float <=> double casts
as well as float <=> long
double <=> long
(int*) <=> (float*) // reinterpret
(long*) <=> (double*) //reinterpret

However is is not yet hooked up to newCTFE.
As I need to think about the interface.

Either I can go with a Cast and BitCast function
(which are prone to strange behavior, if they used with the 
lowered IR)
Or
Add the more explicit
F32ToF64, F64To32,
F32ToSI, SIToF32, UIToF32,
F64ToSI, SIToF64, UIToF64,

Which adds 8 calls to the IR-API and looks unsightly.



 Jul 07 2017



prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Hey Guys I just fixed an ABI issue concerning slices and structs.

This issue took 2.5 weeks to find and fix.

So let me describe what was going on.
As you may know newCTFE defines it's own ABI (since it is a 
platform in it's own right)
Slices are generally represented as 16Byte SliceDescriptors
of the form
struct SliceDesc
{
   uint Base; // T* really. but always 4 bytes regardless
   uint Length;
   uint Capcity; // unused for now.
   uint ExtraFlags // unused will be important for array-casts
}

Whereas A struct is always stored __packed__ and aligned to 4 
byte boundaries

struct S // size: 20
{
   ubyte m1; // offset: 0
   ulong m2; // offset: 4
   ubyte m3; // offset: 12
   ubyte m4; // offset: 16
}

Now Slices inside of structs are stored by injecting the 
corresponding sliceDescriptor
Meaning:
AS { uint[] a; uint aLen; } => {  uint[4] aDesc; uint aLen }
However the code that is repsonsible for converting newCTFE-ABI 
values to DMD-expressions was still assuming this data-layout
AS { uint[] a; uint aLen; } => { (uint[4])* aDescPtr; uint aLen }
which caused it to do an extra derefence and and actually 
interpreting the sliceBase as a descriptor pointer.

Which of course produced complete garbage.

The reason why this took me so long to fix is because I suspected 
that the conversion was using the new convention and the 
byte-code generator was at fault.
And in my attemptes to fix the bytecode generation I introduced 
even more ABI bugs.

long story short:
This is now fixed.
and
ABI bugs are HELL!

Cheers,
Stefan



 Jul 12 2017



 next sibling parent reply Kagamin <spam here.lot>  writes:


On Wednesday, 12 July 2017 at 07:58:30 UTC, Stefan Koch wrote:

 Whereas A struct is always stored __packed__ and aligned to 4 
 byte boundaries

 struct S // size: 20
 {
   ubyte m1; // offset: 0
   ulong m2; // offset: 4
   ubyte m3; // offset: 12
   ubyte m4; // offset: 16
 }


Do you plan to keep it like that? This won't work on non-x86 
architectures that don't allow unaligned memory access.



 Jul 12 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 12 July 2017 at 09:46:39 UTC, Kagamin wrote:

 On Wednesday, 12 July 2017 at 07:58:30 UTC, Stefan Koch wrote:

 Whereas A struct is always stored __packed__ and aligned to 4 
 byte boundaries

 struct S // size: 20
 {
   ubyte m1; // offset: 0
   ulong m2; // offset: 4
   ubyte m3; // offset: 12
   ubyte m4; // offset: 16
 }


 Do you plan to keep it like that? This won't work on non-x86 
 architectures that don't allow unaligned memory access.


For the time being, yes I plan to keep it.
It is 4 byte aligned which is plenty of alignment.

For architectures which do not allow unaligned access we would 
split the ulong into hi and lo parts.
64bit arithmetic can generally be assumed to take a slower path 
in newCTFE.



 Jul 12 2017



prev sibling  parent reply Martin Tschierschke <mt smartdolphin.de>  writes:


On Wednesday, 12 July 2017 at 07:58:30 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
 long story short:
 This is now fixed.
 and
 ABI bugs are HELL!

 Cheers,
 Stefan


As being a newbie, could you please point to a post describing 
what you are doing in more general terms. Answering the question, 
what will the new CTFE bring to us all and when should we start 
preparing a big celebration for the time it gets part of DMD?



 Jul 12 2017



  parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 12 July 2017 at 09:57:21 UTC, Martin Tschierschke 
wrote:

 On Wednesday, 12 July 2017 at 07:58:30 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:
 [ ... ]
 long story short:
 This is now fixed.
 and
 ABI bugs are HELL!

 Cheers,
 Stefan


 As being a newbie, could you please point to a post describing 
 what you are doing in more general terms. Answering the 
 question, what will the new CTFE bring to us all and when 
 should we start preparing a big celebration for the time it 
 gets part of DMD?


Here is a blog-port describing the project.
http://dlang.org/blog/2017/04/10/the-new-ctfe-engine/

newCTFE will reduce the time taken for ctfe by roughly 10x
And, more importantly it reduces the memory-usage.
Instead of a few gigabyte it will not even allocate more then a 
megabyte.
(Currently it is hard capped to a maximum of 64K stack space and 
16M heap).



 Jul 12 2017



prev sibling  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:

 [ ... ]


Another ABI issue just pooped on my Wednesday.

this time it manifests while doing concatenation inside a template
the test failing is (test/runnable/template8.d)

This is again hard to debug because I have to piece together the 
actual source-code which is being executed.

templates are a pain in the butt!



 Jul 12 2017



  parent reply watcher <watcher aol.com>  writes:


On Wednesday, 12 July 2017 at 18:57:12 UTC, Stefan Koch wrote:

 On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch 
 wrote:

 [ ... ]


 Another ABI issue just pooped on my Wednesday.

 this time it manifests while doing concatenation inside a 
 template
 the test failing is (test/runnable/template8.d)

 This is again hard to debug because I have to piece together 
 the actual source-code which is being executed.

 templates are a pain in the butt!


stop advertising yourself and polluting this thread.
seek help - no down-payments in Germany!!



 Jul 12 2017



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 12 July 2017 at 21:05:28 UTC, watcher wrote:

 stop advertising yourself and polluting this thread.
 seek help - no down-payments in Germany!!


I am not sure what you mean ...
I assume it's a joke ?

If you are indeed offering help, you can reproduce this by 
checking out
https://github.com/UplinkCoder/dmd/tree/newCTFE_on_master
and compiling tests/runnable/template8.d



 Jul 12 2017



  parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Thu, Jul 13, 2017 at 01:23:53AM +0000, Stefan Koch via Digitalmars-d wrote:

 On Wednesday, 12 July 2017 at 21:05:28 UTC, watcher wrote:

 
 stop advertising yourself and polluting this thread.
 seek help - no down-payments in Germany!!


 
 I am not sure what you mean ...
 I assume it's a joke ?


Or a troll.  Just ignore it.


T

-- 
Debian GNU/Linux: Cray on your desktop.



 Jul 12 2017



  parent  Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 7/12/17 9:20 PM, H. S. Teoh via Digitalmars-d wrote:

 On Thu, Jul 13, 2017 at 01:23:53AM +0000, Stefan Koch via Digitalmars-d wrote:

 On Wednesday, 12 July 2017 at 21:05:28 UTC, watcher wrote:

 stop advertising yourself and polluting this thread.
 seek help - no down-payments in Germany!!


 I am not sure what you mean ...
 I assume it's a joke ?


 
 Or a troll.  Just ignore it.


I kind of like Stefan's response though :)

Carry on, Stefan!

Though here is a suggestion -- start a new thread every month. Every 
time you post a response on this thread that's not in response to 
something, it jumps the entire history back up to the top of the list.

-Steve



 Jul 13 2017