www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DMD backend generation future with the new AI race processor

reply Hipreme <msnmancini hotmail.com> writes:
Hello guys.
I've been following some of the tech tendencies, and there is 
just like a bunch of hints that looks like Microsoft is looking 
to develop their own processors based on ARM64. And I would 
guess, more enterprises could come to that since there is:

1. The new AI race happening, with NVidia stocks sky rocketing, 
and there's almost no competitor to it.
2. Apple has moved already to ARM64 and developing their own 
things.
3. There is a high possibility that also on game industry things 
move to that way too

Currently, although we have LDC, I would expect things to start 
becoming harsh to DMD. I'm a little concerned because this looks 
like a big tendency of the future and I saw no movement on that. 
I'm not requesting for anyone to develop that specially because I 
myself can't, but I would like to at least bring to people what 
is happening.

I think that if that work is not done in DMD, or people do not 
pretend to work on that and if those rumors really become true, I 
would guess that there is a high possibility of people dropping 
DMD for using LDC or of DMD project being merged with LDC, which 
could also break one of the biggest advantages of using D - the 
fast compilation
Mar 10
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Brad looked into doing an ARM backend for DMD many years ago, but gave it up as 
too much work.

I'd love to do a competitive one, but it would take a while.
Mar 10
next sibling parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 11/03/2024 8:42 AM, Walter Bright wrote:

 Brad looked into doing an ARM backend for DMD many years ago, but gave 
 it up as too much work.
 
 I'd love to do a competitive one, but it would take a while.


For reference here is an old C++ version that was modified for ARM by 
Iain/Brad.

https://github.com/ibuclaw/dmd/tree/dmd-cxx-arm

This would be an ideal project to throw a student at long term (PhD maybe?).

If we don't, OSX support for dmd is dead, and we might as well kill it 
off now.
Mar 10
prev sibling parent reply Brad Roberts <braddr puremagic.com> writes:
The issues I had with doing the arm DMD backend were primarily the 
difficulty of deciphering and penetrating the dmd backend, not the arm 
parts.  The other big issue I had was me and my work patterns.  I got 
enough done that I knew it could be done.  That's an inflection point 
for me where it's likely I'll drop many projects, particularly the 
exploratory ones.  I was also busy with a more than full time job and 
had to prioritize it over massive side projects.

Combine that with the timing, and it just didn't make sense to continue 
the work.  This was about 10 years ago and LDC and GDC were becoming 
very usable, which brought along arm backends that were far more mature 
and already very usable.

If anyone wants to pick up the work, it's in the arm branch of my dmd 
fork.  Chances are it'd take a week or three to bring it back up to 
where I left it, meaning the barely capable of doing any code gen.  A 
LOT has changed over the years, but the touch points with the backend 
code haven't evolved all that much compared to the front end).  This 
would have been before the code base was all D and before switching to 
build.d -- neither of which are actually a big problem, just gives a bit 
of what era we're talking about.

As to being competitive, that's unrealistic for the same reason that 
dmd's backend isn't competitive with ldc and gdc.  It can reach the 
level of competent and usable (which _is_ a useful level of 
functionality), but there's just no way to complete with the legions of 
engineers that work on those optimizers and backend code generators.

My 2 cents,
Brad

On 3/10/2024 12:42 PM, Walter Bright via Digitalmars-d wrote:

 Brad looked into doing an ARM backend for DMD many years ago, but gave 
 it up as too much work.
 
 I'd love to do a competitive one, but it would take a while.
Mar 10
next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 11/03/2024 11:55 AM, Brad Roberts wrote:

 The issues I had with doing the arm DMD backend were primarily the 
 difficulty of deciphering and penetrating the dmd backend, not the arm 
 parts.  The other big issue I had was me and my work patterns.  I got 
 enough done that I knew it could be done.  That's an inflection point 
 for me where it's likely I'll drop many projects, particularly the 
 exploratory ones.  I was also busy with a more than full time job and 
 had to prioritize it over massive side projects.


I can't even find leafs or entry points to begin an understanding.

So +1 on the indecipherable aspect to it.


 As to being competitive, that's unrealistic for the same reason that 
 dmd's backend isn't competitive with ldc and gdc.  It can reach the 
 level of competent and usable (which _is_ a useful level of 
 functionality), but there's just no way to complete with the legions of 
 engineers that work on those optimizers and backend code generators.


Walter has recently been arguing against  restrict, and previously 
argued against atomics being intrinsics with me.

Even with Walter working full time on it for a few years, that backend 
will never match what LLVM 19 can do without things like that.

It is unfortunate.
Mar 10
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/10/2024 5:40 PM, Richard (Rikki) Andrew Cattermole wrote:

 Even with Walter working full time on it for a few years, that backend will 
 never match what LLVM 19 can do without things like that.


I regularly have people telling me what I can't do :-)

For example, people in this forum used to tell me that DMD didn't do Data Flow 
Analysis and DFA had been invented in the 2000s.

Never mind that Datalight C circa 1985 was the very first DOS compiler to do
DFA.

I was told the backend couldn't do loop unrolling which was a special magical 
thing that only super advanced engineers could write, which it now does.

Then I was told the backend couldn't do SROA, which it does now.

Then I was told the backend couldn't do SIMD, which it does now.

And so on.

It just takes time to do these things, and I'm pretty booked. The DMD backend
is 
doing remarkably well considering how little time I spend on it.

Just recently the imminent demise of it was predicted because it wasn't 
generating IBT (Indirect Branch Tracking) code. It generates IBT now.
Mar 11
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 12/03/2024 4:47 PM, Walter Bright wrote:

 On 3/10/2024 5:40 PM, Richard (Rikki) Andrew Cattermole wrote:

 Even with Walter working full time on it for a few years, that backend 
 will never match what LLVM 19 can do without things like that.


 
 I regularly have people telling me what I can't do :-)
 
 For example, people in this forum used to tell me that DMD didn't do 
 Data Flow Analysis and DFA had been invented in the 2000s.
 
 Never mind that Datalight C circa 1985 was the very first DOS compiler 
 to do DFA.
 
 I was told the backend couldn't do loop unrolling which was a special 
 magical thing that only super advanced engineers could write, which it 
 now does.
 
 Then I was told the backend couldn't do SROA, which it does now.
 
 Then I was told the backend couldn't do SIMD, which it does now.
 
 And so on.
 
 It just takes time to do these things, and I'm pretty booked. The DMD 
 backend is doing remarkably well considering how little time I spend on it.
 
 Just recently the imminent demise of it was predicted because it wasn't 
 generating IBT (Indirect Branch Tracking) code. It generates IBT now.


It's not that I wouldn't love to see you do it, its just that you have 
25 years worth of work to do if you want to get it up to state of the art.

That is an incredible amount of work for a team to do, let alone one person.

The most beneficial thing now would probably be refactoring it into 
something the average compiler developer can recognize.

That way others can improve it whilst you work on other things ;)

I would absolutely love to see that backend discussed being used on 
r/compilers and r/ProgrammingLanguages. It could fill a very niche role 
in the programming language development communities that would be ideal 
to getting D used there.

It could quite easily make QBE look like a toy, it's just a shame it 
can't without some massive refactoring and documentation efforts.
Mar 11
parent reply Walter Bright <newshound2 digitalmars.com> writes:
I'd love to see it, too.

It's actually a fairly straightforward design. The complexity comes from the 
complexity of the x86 instruction set. There's a zillion special cases.

Other people have converted it before. There used to be a 68000 code generator 
written by others.
Mar 11
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 12/03/2024 7:53 PM, Walter Bright wrote:

 I'd love to see it, too.
 
 It's actually a fairly straightforward design. The complexity comes from 
 the complexity of the x86 instruction set. There's a zillion special cases.
 
 Other people have converted it before. There used to be a 68000 code 
 generator written by others.


Yeah I'm aware of m68k.

I looked into the possibility of running D on my Macintosh classic and I 
found an old message about you on UseNet for it!

That was quite a joy lol.
Mar 11
prev sibling next sibling parent reply Ben Jones <fake fake.fake> writes:
On Sunday, 10 March 2024 at 22:55:32 UTC, Brad Roberts wrote:

 The issues I had with doing the arm DMD backend were primarily 
 the difficulty of deciphering and penetrating the dmd backend, 
 not the arm parts.  The other big issue I had was me and my 
 work patterns.  I got enough done that I knew it could be done. 
  That's an inflection point for me where it's likely I'll drop 
 many projects, particularly the exploratory ones.  I was also 
 busy with a more than full time job and had to prioritize it 
 over massive side projects.


I'm happy to help with this project.  I also don't think we'll 
hit LDC performance, but there's benefits to being self hosting 
on ARM platforms without LDC/GDC.
Mar 11
next sibling parent electricface <electricface qq.com> writes:
On Monday, 11 March 2024 at 20:32:44 UTC, Ben Jones wrote:

 On Sunday, 10 March 2024 at 22:55:32 UTC, Brad Roberts wrote:

 The issues I had with doing the arm DMD backend were primarily 
 the difficulty of deciphering and penetrating the dmd backend, 
 not the arm parts.  The other big issue I had was me and my 
 work patterns.  I got enough done that I knew it could be 
 done.  That's an inflection point for me where it's likely 
 I'll drop many projects, particularly the exploratory ones.  I 
 was also busy with a more than full time job and had to 
 prioritize it over massive side projects.


 I'm happy to help with this project.  I also don't think we'll 
 hit LDC performance, but there's benefits to being self hosting 
 on ARM platforms without LDC/GDC.


Maybe we could consider adding a backend for the Go language 
toolchain. I think there are many similarities between the D and 
Go languages, such as simplicity, readability, and speed.
Mar 11
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/11/2024 1:32 PM, Ben Jones wrote:

 I'm happy to help with this project.  I also don't think we'll hit LDC 
 performance, but there's benefits to being self hosting on ARM platforms
without 
 LDC/GDC.


That would be great if you were to pick up the flag on that!
Mar 11
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/10/2024 3:55 PM, Brad Roberts wrote:

 As to being competitive, that's unrealistic for the same reason that dmd's 
 backend isn't competitive with ldc and gdc.  It can reach the level of
competent 
 and usable (which _is_ a useful level of functionality), but there's just no
way 
 to complete with the legions of engineers that work on those optimizers and 
 backend code generators.


I meant competitive in terms of being equivalent to the x86 code generator.
Mar 11
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Nice to see you here, Brad!
Mar 11