www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - The future of Int128 support in D language

reply Siarhei Siamashka <siarhei.siamashka gmail.com> writes:
Hello,

Will it ever turn into a native integer type nicely supported by 
the language? What about 128-bit integer literals? Also unsigned 
128-bit type? Implicit conversions between `int` and `Int128`? Is 
interoperability with the other parts of Phobos planned? I mean, 
it would be great if things like this worked:

```D
import std.int128, std.stdio, std.conv;

void main() {
   Int128 x;
   x = 123;
   writefln!"x = %d"(x);
   writeln(x.to!int);
}
```

I'm posting this because I tried to experiment with Int128 in 
https://github.com/ssvb/speedy-int128 and see that performance 
parity with Clang can be easily achieved for LDC after applying a 
few minor tweaks.

What's the best way to get some inline LLVM IR optimizations for 
Int128 included in the next release of LDC? So that it works fast 
out of the box. The struct alignment is causing some troubles 
though.
Jan 07 2023
next sibling parent reply Johan <j j.nl> writes:
On Sunday, 8 January 2023 at 03:11:15 UTC, Siarhei Siamashka 
wrote:

 I'm posting this because I tried to experiment with Int128 in 
 https://github.com/ssvb/speedy-int128 and see that performance 
 parity with Clang can be easily achieved for LDC after applying 
 a few minor tweaks.


Nice.


 What's the best way to get some inline LLVM IR optimizations 
 for Int128 included in the next release of LDC? So that it 
 works fast out of the box. The struct alignment is causing some 
 troubles though.


Submit PRs to LDC's Phobos.
https://github.com/ldc-developers/phobos
We use "version (LDC)", see e.g. 
https://github.com/ldc-developers/phobos/blob/63090119d71f9795744618c7386b7d6f29454a15/std/math/traits.d#L614-L629

What's the alignment issue?

cheers,
   Johan
Jan 08 2023
parent reply Johan <j j.nl> writes:
On Sunday, 8 January 2023 at 11:51:19 UTC, Johan wrote:

 Submit PRs to LDC's Phobos.
 https://github.com/ldc-developers/phobos


Or probably LDC's druntime is better:
https://github.com/ldc-developers/ldc/blob/master/runtime/druntime/src/core/int128.d
Jan 08 2023
parent reply Siarhei Siamashka <siarhei.siamashka gmail.com> writes:
On Sunday, 8 January 2023 at 11:53:52 UTC, Johan wrote:

 On Sunday, 8 January 2023 at 11:51:19 UTC, Johan wrote:

 Submit PRs to LDC's Phobos.
 https://github.com/ldc-developers/phobos


 Or probably LDC's druntime is better:
 https://github.com/ldc-developers/ldc/blob/master/runtime/druntime/src/core/int128.d


Thanks! That's exactly what I wanted to know.


 What's the alignment issue?


I can see that it's already workarounded here: 
https://github.com/ldc-developers/ldc/blob/498b0c79a6030a48085d384fa872ec6c5c815348/runtime/druntime/src/core/int128.d#L24-L25

Still I don't like where this is going. Because each compiler 
wants to configure its own funny mutually incompatible platform 
specific alignments for this struct and this seems to be getting 
out of hand: https://github.com/dlang/dmd/pull/14768
Jan 08 2023
parent reply Johan <j j.nl> writes:
On Sunday, 8 January 2023 at 17:25:29 UTC, Siarhei Siamashka 
wrote:

 On Sunday, 8 January 2023 at 11:53:52 UTC, Johan wrote:

 On Sunday, 8 January 2023 at 11:51:19 UTC, Johan wrote:

 Submit PRs to LDC's Phobos.
 https://github.com/ldc-developers/phobos


 Or probably LDC's druntime is better:
 https://github.com/ldc-developers/ldc/blob/master/runtime/druntime/src/core/int128.d


 Thanks! That's exactly what I wanted to know.


I like that you splitted the implementation into a separate file, 
that's a good idea.

GDC also can benefit of a specialized implementation. Probably 
good to ask Iain of how best to approach these specializations 
(i.e. what files, how to forward to the specialized 
implementation from the upstream druntime file, ...)

-Johan
Jan 08 2023
parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Sunday, 8 January 2023 at 17:36:07 UTC, Johan wrote:

 I like that you splitted the implementation into a separate 
 file, that's a good idea.

 GDC also can benefit of a specialized implementation. Probably 
 good to ask Iain of how best to approach these specializations 
 (i.e. what files, how to forward to the specialized 
 implementation from the upstream druntime file, ...)

 -Johan


Actually, I had implemented core.int128 as compiler-recognized 
intrinsics that do the necessary conversion to/from native on the 
fly.

I ran into some problems when testing this on SPARC though (to 
name one non-x86 target that I use for sanity checking new 
code-gen features).  Never found the time to get to the bottom of 
what was causing it to crash, so left it as a TBD patch set.

I suspect I am going wrong somewhere with all the concatenated 
`*cast(cent*)&c` and `*cast(core.int128.Cent*)&c` going on all 
around the place when you chain these functions together with 
UFCS.  Maybe should investigate using a `union` instead, but not 
enough bandwidth to do that at the moment.
Jan 09 2023
next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On Monday, 9 January 2023 at 16:51:45 UTC, Iain Buclaw wrote:

 On Sunday, 8 January 2023 at 17:36:07 UTC, Johan wrote:

 I like that you splitted the implementation into a separate 
 file, that's a good idea.

 GDC also can benefit of a specialized implementation. Probably 
 good to ask Iain of how best to approach these specializations 
 (i.e. what files, how to forward to the specialized 
 implementation from the upstream druntime file, ...)

 -Johan


 Actually, I had implemented core.int128 as compiler-recognized 
 intrinsics that do the necessary conversion to/from native on 
 the fly.


FAOD, on x86_64 these intrinsics resulted in identical assembly 
to  C/C++ __int128.
Jan 09 2023
prev sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Monday, 9 January 2023 at 16:51:45 UTC, Iain Buclaw wrote:

 On Sunday, 8 January 2023 at 17:36:07 UTC, Johan wrote:

 [...]


 Actually, I had implemented core.int128 as compiler-recognized 
 intrinsics that do the necessary conversion to/from native on 
 the fly.

 I ran into some problems when testing this on SPARC though (to 
 name one non-x86 target that I use for sanity checking new 
 code-gen features).  Never found the time to get to the bottom 
 of what was causing it to crash, so left it as a TBD patch set.

 I suspect I am going wrong somewhere with all the concatenated 
 `*cast(cent*)&c` and `*cast(core.int128.Cent*)&c` going on all 
 around the place when you chain these functions together with 
 UFCS.  Maybe should investigate using a `union` instead, but 
 not enough bandwidth to do that at the moment.


SPARC is big endian. Probably somewhere an intermediate value 
that is truncated to 64 bits which passes in LE but not in BE.
Talking completely out of my a.s. :-)
Jan 09 2023
parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On Monday, 9 January 2023 at 18:10:43 UTC, Patrick Schluter wrote:

 On Monday, 9 January 2023 at 16:51:45 UTC, Iain Buclaw wrote:

 On Sunday, 8 January 2023 at 17:36:07 UTC, Johan wrote:

 [...]


 Actually, I had implemented core.int128 as compiler-recognized 
 intrinsics that do the necessary conversion to/from native on 
 the fly.

 I ran into some problems when testing this on SPARC though (to 
 name one non-x86 target that I use for sanity checking new 
 code-gen features).  Never found the time to get to the bottom 
 of what was causing it to crash, so left it as a TBD patch set.

 I suspect I am going wrong somewhere with all the concatenated 
 `*cast(cent*)&c` and `*cast(core.int128.Cent*)&c` going on all 
 around the place when you chain these functions together with 
 UFCS.  Maybe should investigate using a `union` instead, but 
 not enough bandwidth to do that at the moment.


 SPARC is big endian. Probably somewhere an intermediate value 
 that is truncated to 64 bits which passes in LE but not in BE.
 Talking completely out of my a.s. :-)


Endianess is already covered.

https://github.com/dlang/dmd/blob/ee1acbb62dc4a65e72d7bf27e61761fb34d497e2/druntime/src/core/int128.d#L24-L36

At some point the backend dies on receiving invalid code, or 
something it cannot reasonably expand/lower.  So I must be doing 
something wrong. :-)
Jan 09 2023
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 8 January 2023 at 03:11:15 UTC, Siarhei Siamashka 
wrote:

 What's the best way to get some inline LLVM IR optimizations 
 for Int128 included in the next release of LDC? So that it 
 works fast out of the box. The struct alignment is causing some 
 troubles though.


I'm going to be the party pooper but this is completely and 100% 
useless work. It's not a dunk on you and your work, more on all 
the decision that preceeds it that lead to you even attempting to 
do this.

D has cent/ucent, and what's required is actually support that. 
At some point, someone decided nonono, what we need instead is a 
retarded library implementation that is obviously going to be 
terrible, and that will then lead people to be like, hey we can 
improve this with intrinsics, specific for each compiler and 
plateform, and now we have this thread, where well meaning people 
are putting in work to solve problem we shouldn't be having in 
the first place.

All modern backends understand the concept of an int128, so it's 
literally half an hour of work to support this in LDC/GDC. The 
fact Walter chose to have its own backend to maintain is all well 
and good, but it is not my problem as a user, nor it is the 
problem of most people here. Just forward cent/ucent to 
core.int128 in there and generate horrible codegen for all that 
matter if it cannot support int128.

And just in case you wonder who I am to assert this forcefully: 
I'm the guy who made it so that LLVM generate half decent codegen 
for int128 to begin with. I'm very familiar with the problem, in 
fact, as frightening as it may seems, I may even be one of the 
world finest expert on the matter (a very scary prospect indeed). 
Don't believe me? https://reviews.llvm.org/D29872
Jan 09 2023
next sibling parent max haughton <maxhaton gmail.com> writes:
On Monday, 9 January 2023 at 17:26:58 UTC, deadalnix wrote:

 On Sunday, 8 January 2023 at 03:11:15 UTC, Siarhei Siamashka 
 wrote:

 What's the best way to get some inline LLVM IR optimizations 
 for Int128 included in the next release of LDC? So that it 
 works fast out of the box. The struct alignment is causing 
 some troubles though.


 I'm going to be the party pooper but this is completely and 
 100% useless work. It's not a dunk on you and your work, more 
 on all the decision that preceeds it that lead to you even 
 attempting to do this.

 D has cent/ucent, and what's required is actually support that. 
 At some point, someone decided nonono, what we need instead is 
 a retarded library implementation that is obviously going to be 
 terrible, and that will then lead people to be like, hey we can 
 improve this with intrinsics, specific for each compiler and 
 plateform, and now we have this thread, where well meaning 
 people are putting in work to solve problem we shouldn't be 
 having in the first place.

 All modern backends understand the concept of an int128, so 
 it's literally half an hour of work to support this in LDC/GDC. 
 The fact Walter chose to have its own backend to maintain is 
 all well and good, but it is not my problem as a user, nor it 
 is the problem of most people here. Just forward cent/ucent to 
 core.int128 in there and generate horrible codegen for all that 
 matter if it cannot support int128.

 And just in case you wonder who I am to assert this forcefully: 
 I'm the guy who made it so that LLVM generate half decent 
 codegen for int128 to begin with. I'm very familiar with the 
 problem, in fact, as frightening as it may seems, I may even be 
 one of the world finest expert on the matter (a very scary 
 prospect indeed). Don't believe me? 
 https://reviews.llvm.org/D29872


I vote for this. Consider also that integers in D have VRP and 
implicit conversions and so on so 128 bit types not having this 
is (although not the end of the world[1]) not good.

[1] I say this, because 128 bit ints are not used often in the 
grand scheme of things *but* when you do need them they are 
typically quite densely used in that code (e.g. crypto), which is 
why they aren't library types with magic intrinsic in other 
languages either.
Jan 09 2023
prev sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Monday, 9 January 2023 at 17:26:58 UTC, deadalnix wrote:

 I'm going to be the party pooper but this is completely and 
 100% useless work. It's not a dunk on you and your work, more 
 on all the decision that preceeds it that lead to you even 
 attempting to do this.

 [...]


You could help us revive this PR then. ;-)

https://github.com/dlang/dmd/pull/6743

Most of it would just be slowly adding the explicit construction 
to the code base as a "refactor" before introducing the 
double-int type proper.

Another attempt at salvaging the original work

https://github.com/ibuclaw/dmd/tree/ctinteger
Jan 09 2023
parent deadalnix <deadalnix gmail.com> writes:
On Monday, 9 January 2023 at 21:04:06 UTC, Iain Buclaw wrote:

 On Monday, 9 January 2023 at 17:26:58 UTC, deadalnix wrote:

 I'm going to be the party pooper but this is completely and 
 100% useless work. It's not a dunk on you and your work, more 
 on all the decision that preceeds it that lead to you even 
 attempting to do this.

 [...]


 You could help us revive this PR then. ;-)

 https://github.com/dlang/dmd/pull/6743

 Most of it would just be slowly adding the explicit 
 construction to the code base as a "refactor" before 
 introducing the double-int type proper.

 Another attempt at salvaging the original work

 https://github.com/ibuclaw/dmd/tree/ctinteger


It seems to me like simple refactors like 
https://github.com/dlang/dmd/commit/b4930f9e52b8f4dd18d61b1a6a47dc90e1d651ae
could go in right away.
Jan 09 2023