www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Warning, ABI breakage from 2.074 to 2.075

reply Basile B. <b2.temp gmx.com> writes:
Static libraries that are
- compiled with dmd 2.074 (maybe previous versions too)
- call format() in their API

will be responsible for strange errors when used by programs 
compiled with dmd 2.075. People will see their software raising a 
FormatException (orphan argument) for no reason. When the static 
library will be recompiled with dmd 2.075 the errors will 
disappear.

I haven't investigated much since there's been a non trivial 
amount of change in std.format() during the this spring. I don't 
know what to do with this facts except to report them here.
May 25
parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 25 May 2017 at 10:42:44 UTC, Basile B. wrote:
 Static libraries that are
 - compiled with dmd 2.074 (maybe previous versions too)
 - call format() in their API

 will be responsible for strange errors when used by programs 
 compiled with dmd 2.075. People will see their software raising 
 a FormatException (orphan argument) for no reason. When the 
 static library will be recompiled with dmd 2.075 the errors 
 will disappear.

 I haven't investigated much since there's been a non trivial 
 amount of change in std.format() during the this spring. I 
 don't know what to do with this facts except to report them 
 here.
Why is this unexpected? D has never committed to ABI stability across compiler versions or the different D compilers.
May 25
next sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thursday, May 25, 2017 13:23:57 Joakim via Digitalmars-d wrote:
 On Thursday, 25 May 2017 at 10:42:44 UTC, Basile B. wrote:
 Static libraries that are
 - compiled with dmd 2.074 (maybe previous versions too)
 - call format() in their API

 will be responsible for strange errors when used by programs
 compiled with dmd 2.075. People will see their software raising
 a FormatException (orphan argument) for no reason. When the
 static library will be recompiled with dmd 2.075 the errors
 will disappear.

 I haven't investigated much since there's been a non trivial
 amount of change in std.format() during the this spring. I
 don't know what to do with this facts except to report them
 here.
Why is this unexpected? D has never committed to ABI stability across compiler versions or the different D compilers.
Yeah, if you're relying on ABI compatibility across releases, then you're likely to have some weird problems. It may actually work often enough that you can get away with it a good chunk of the time (I don't know), but we certainly don't promised ABI compatibility, and we do break it some percentage of the time. Heck, we frequently do stuff like templatize a function that wasn't templatized before (generally to make it more generic), which breaks all code that was taking it's address to use as a function pointer (but doesn't break most uses of the function). We do try to avoid superfluous breakage, and _most_ code should be fine from release to release, but corner cases do get broken on a regular basis and that's even when you recompile all your code, not even trying to use newly compiled code with code compiled with an older compiler. The sitation is _far_ better than it once was, but we're definitely not ABI stable, and the nature of D actually makes it pretty hard to be. - Jonathan M Davis
May 25
parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Thursday, 25 May 2017 at 14:36:43 UTC, Jonathan M Davis wrote:
 On Thursday, May 25, 2017 13:23:57 Joakim via Digitalmars-d 
 wrote:
 On Thursday, 25 May 2017 at 10:42:44 UTC, Basile B. wrote:
 Static libraries that are
 - compiled with dmd 2.074 (maybe previous versions too)
 - call format() in their API

 will be responsible for strange errors when used by programs 
 compiled with dmd 2.075. People will see their software 
 raising a FormatException (orphan argument) for no reason. 
 When the static library will be recompiled with dmd 2.075 
 the errors will disappear.

 I haven't investigated much since there's been a non trivial 
 amount of change in std.format() during the this spring. I 
 don't know what to do with this facts except to report them 
 here.
Why is this unexpected? D has never committed to ABI stability across compiler versions or the different D compilers.
Yeah, if you're relying on ABI compatibility across releases, then you're likely to have some weird problems. It may actually work often enough that you can get away with it a good chunk of the time (I don't know), but we certainly don't promised ABI compatibility, and we do break it some percentage of the time. Heck, we frequently do stuff like templatize a function that wasn't templatized before (generally to make it more generic), which breaks all code that was taking it's address to use as a function pointer (but doesn't break most uses of the function). We do try to avoid superfluous breakage, and _most_ code should be fine from release to release, but corner cases do get broken on a regular basis and that's even when you recompile all your code, not even trying to use newly compiled code with code compiled with an older compiler. The sitation is _far_ better than it once was, but we're definitely not ABI stable, and the nature of D actually makes it pretty hard to be.
Could someone please explain why people talk always of ABI compatibilty while what is described would imo better classified as API compatibilty. ABI is the way parameter are passed to functions i.e. which parameters go to which register and is defined by the platform, while the API is the set of signature of a library. I find it a bit disturbing that at each release the ABI would change, while that is generally something which is (should) be extremely stable. The only time I have been confronted with an ABI (C) change within a platform was in the transition from Solaris 9 to Solaris 10 where the way structs were returned from functions changed. TL;DR is there a confusion between ABI and API or does the calling conventions change at every release?
May 25
next sibling parent Joakim <dlang joakim.fea.st> writes:
On Thursday, 25 May 2017 at 16:16:19 UTC, Patrick Schluter wrote:
 On Thursday, 25 May 2017 at 14:36:43 UTC, Jonathan M Davis 
 wrote:
[...]
Could someone please explain why people talk always of ABI compatibilty while what is described would imo better classified as API compatibilty. ABI is the way parameter are passed to functions i.e. which parameters go to which register and is defined by the platform, while the API is the set of signature of a library. I find it a bit disturbing that at each release the ABI would change, while that is generally something which is (should) be extremely stable. The only time I have been confronted with an ABI (C) change within a platform was in the transition from Solaris 9 to Solaris 10 where the way structs were returned from functions changed. TL;DR is there a confusion between ABI and API or does the calling conventions change at every release?
The calling convention is only one aspect of an ABI: https://dlang.org/spec/abi.html For example, David notes in that Debian thread that every time attributes are added to druntime functions, their mangling changes, ie the ABI changes.
May 25
prev sibling parent Basile B. <b2.temp gmx.com> writes:
On Thursday, 25 May 2017 at 16:16:19 UTC, Patrick Schluter wrote:
 Could someone please explain why people talk always of ABI 
 compatibilty while what is described would imo better 
 classified as API compatibilty.
Here the function mangle is involved. Mangles/Names are part of the ABI in D specs: https://dlang.org/spec/abi.html#type_mangling, although you're right, generally speaking the ABI is rather what you described. What's strange is that with a different mangle, the previous function that stand in the static library could still be linked but with different results. Usually this is not the case when the mangle changes. That's why this breakage is particularly "remarkable" i find.
May 25
prev sibling parent reply Jason King via Digitalmars-d <digitalmars-d puremagic.com> writes:
That=E2=80=99s a fairly important requirement if it=E2=80=99s supposed to b=
e a systems
programming language, less so for application focused stuff.  I would hope
it=E2=80=99s at least an eventual goal even if it=E2=80=99s not quite the c=
ase today.


On May 25, 2017 at 8:26:04 AM, Joakim via Digitalmars-d (
digitalmars-d puremagic.com) wrote:

On Thursday, 25 May 2017 at 10:42:44 UTC, Basile B. wrote:
 Static libraries that are
 - compiled with dmd 2.074 (maybe previous versions too)
 - call format() in their API

 will be responsible for strange errors when used by programs
 compiled with dmd 2.075. People will see their software raising
 a FormatException (orphan argument) for no reason. When the
 static library will be recompiled with dmd 2.075 the errors
 will disappear.

 I haven't investigated much since there's been a non trivial
 amount of change in std.format() during the this spring. I
 don't know what to do with this facts except to report them
 here.
Why is this unexpected? D has never committed to ABI stability across compiler versions or the different D compilers.
May 25
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 25/05/2017 4:02 PM, Jason King via Digitalmars-d wrote:
 That’s a fairly important requirement if it’s supposed to be a systems 
 programming language, less so for application focused stuff.  I would 
 hope it’s at least an eventual goal even if it’s not quite the case today.
You would be fighting the existing companies using D who are all saying "break our stuff". So highly unlikely that its going to happen.
May 25
prev sibling next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Thursday, 25 May 2017 at 15:02:00 UTC, Jason King wrote:
 That’s a fairly important requirement if it’s supposed to be a 
 systems programming language, less so for application focused 
 stuff.  I would hope it’s at least an eventual goal even if 
 it’s not quite the case today.
The reason we don't have ABI compatibility is the same reason neither Rust or Go does, it's a lot of work for a minority of users and it stops the language from progressing. Maybe D will have it eventually due to pressure from large D using companies, but I highly doubt it.
May 25
parent reply Jason King via Digitalmars-d <digitalmars-d puremagic.com> writes:
Yes it is a lot of work, which I strongly suspect is a big reason why C
still reigns supreme at the systems level =E2=80=94 because it does have a =
stable
ABI which solves a lot of headaches from a systems point of view (obviously
momentum and history are also very big reasons).

If that=E2=80=99s the direction D wants to go in, there=E2=80=99s nothing w=
rong with that,
but it needs to be setting the correct expectations for users.  Not having
a stable ABI is perfectly fine for application level stuff, but it can be
rather (in some cases extremely) problematic for systems level stuff--that
needs to be understood both by the users and the people working on D (and I
haven=E2=80=99t really seen much recognition of it).

On May 25, 2017 at 10:25:59 AM, Jack Stouffer via Digitalmars-d (
digitalmars-d puremagic.com) wrote:

On Thursday, 25 May 2017 at 15:02:00 UTC, Jason King wrote:
 That=E2=80=99s a fairly important requirement if it=E2=80=99s supposed to=
be a
 systems programming language, less so for application focused
 stuff. I would hope it=E2=80=99s at least an eventual goal even if
 it=E2=80=99s not quite the case today.
The reason we don't have ABI compatibility is the same reason neither Rust or Go does, it's a lot of work for a minority of users and it stops the language from progressing. Maybe D will have it eventually due to pressure from large D using companies, but I highly doubt it.
May 25
next sibling parent Paulo Pinto <pjmlp progtools.org> writes:
On Thursday, 25 May 2017 at 15:36:38 UTC, Jason King wrote:
 Yes it is a lot of work, which I strongly suspect is a big 
 reason why C still reigns supreme at the systems level — 
 because it does have a stable ABI which solves a lot of 
 headaches from a systems point of view (obviously momentum and 
 history are also very big reasons).
That is a common misconception. C only has a stable ABI on operating systems written in C, because the C ABI is actually the OS ABI. In operating systems not written in C, like all the mainframe OSes before C got widespread out of UNIX and still in use nowadays (IBM i, z/OS, ClearPath), real time OSes written in Ada and quite a few other examples, the "C ABI" only has a meaning inside the POSIX emulation environment. In fact, during the 80 and 90's it was common not being able to link object files from different C compilers on home OSes that were actually mostly written in Assembly. -- Paulo
May 25
prev sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 25 May 2017 at 15:36:38 UTC, Jason King wrote:
 Yes it is a lot of work, which I strongly suspect is a big 
 reason why C still reigns supreme at the systems level — 
 because it does have a stable ABI which solves a lot of 
 headaches from a systems point of view (obviously momentum and 
 history are also very big reasons).

 If that’s the direction D wants to go in, there’s nothing wrong 
 with that,
 but it needs to be setting the correct expectations for users.  
 Not having
 a stable ABI is perfectly fine for application level stuff, but 
 it can be
 rather (in some cases extremely) problematic for systems level 
 stuff--that
 needs to be understood both by the users and the people working 
 on D (and I
 haven’t really seen much recognition of it).

 On May 25, 2017 at 10:25:59 AM, Jack Stouffer via Digitalmars-d 
 (
 digitalmars-d puremagic.com) wrote:

 On Thursday, 25 May 2017 at 15:02:00 UTC, Jason King wrote:
 That’s a fairly important requirement if it’s supposed to be a 
 systems programming language, less so for application focused 
 stuff. I would hope it’s at least an eventual goal even if 
 it’s not quite the case today.
The reason we don't have ABI compatibility is the same reason neither Rust or Go does, it's a lot of work for a minority of users and it stops the language from progressing. Maybe D will have it eventually due to pressure from large D using companies, but I highly doubt it.
There was a long thread last month about getting dmd into Debian, that discussed the ABI stability issue among others: https://forum.dlang.org/thread/hhefnnighbowonxsnbdy forum.dlang.org ABI stability is not promised, not now or anytime soon, not just from D but many languages, as Jack said. It just doesn't make sense.
May 25
parent reply Jason King via Digitalmars-d <digitalmars-d puremagic.com> writes:
And how many of those are claiming to be a systems programming language?

I have no problems with an unstable ABI, what I have a problem is with
claiming to be a systems programming language AND not having a stable ABI.
You realistically cannot have both =E2=80=94 it seems like D is trying to h=
ave it=E2=80=99s
cake and eat it too and I=E2=80=99m just pointing out that it=E2=80=99s goi=
ng to lead to
sadness.  If there are no plans to ever have a stable ABI, that=E2=80=99s f=
ine (may
even be good for the long term usage of the language), just drop the whole
systems programming language bit and focus more on application level, but
I=E2=80=99ve not really seen any recognition of that.

On May 25, 2017 at 11:41:52 AM, Joakim via Digitalmars-d (
digitalmars-d puremagic.com) wrote:

On Thursday, 25 May 2017 at 15:36:38 UTC, Jason King wrote:
 Yes it is a lot of work, which I strongly suspect is a big
 reason why C still reigns supreme at the systems level =E2=80=94
 because it does have a stable ABI which solves a lot of
 headaches from a systems point of view (obviously momentum and
 history are also very big reasons).

 If that=E2=80=99s the direction D wants to go in, there=E2=80=99s nothing=
wrong
 with that,
 but it needs to be setting the correct expectations for users.
 Not having
 a stable ABI is perfectly fine for application level stuff, but
 it can be
 rather (in some cases extremely) problematic for systems level
 stuff--that
 needs to be understood both by the users and the people working
 on D (and I
 haven=E2=80=99t really seen much recognition of it).

 On May 25, 2017 at 10:25:59 AM, Jack Stouffer via Digitalmars-d
 (
 digitalmars-d puremagic.com) wrote:

 On Thursday, 25 May 2017 at 15:02:00 UTC, Jason King wrote:
 That=E2=80=99s a fairly important requirement if it=E2=80=99s supposed t=
o be a
 systems programming language, less so for application focused
 stuff. I would hope it=E2=80=99s at least an eventual goal even if
 it=E2=80=99s not quite the case today.
The reason we don't have ABI compatibility is the same reason neither Rust or Go does, it's a lot of work for a minority of users and it stops the language from progressing. Maybe D will have it eventually due to pressure from large D using companies, but I highly doubt it.
There was a long thread last month about getting dmd into Debian, that discussed the ABI stability issue among others: https://forum.dlang.org/thread/hhefnnighbowonxsnbdy forum.dlang.org ABI stability is not promised, not now or anytime soon, not just from D but many languages, as Jack said. It just doesn't make sense.
May 25
next sibling parent reply Paulo Pinto <pjmlp progtools.org> writes:
On Thursday, 25 May 2017 at 17:04:10 UTC, Jason King wrote:
 And how many of those are claiming to be a systems programming 
 language?

 I have no problems with an unstable ABI, what I have a problem 
 is with
 claiming to be a systems programming language AND not having a 
 stable ABI.
 You realistically cannot have both — it seems like D is trying 
 to have it’s
 cake and eat it too and I’m just pointing out that it’s going 
 to lead to
 sadness.  If there are no plans to ever have a stable ABI, 
 that’s fine (may
 even be good for the long term usage of the language), just 
 drop the whole
 systems programming language bit and focus more on application 
 level, but
 I’ve not really seen any recognition of that.
From that point of view, C++ and Ada aren't system programming languages.
May 25
parent evilrat <evilrat666 gmail.com> writes:
On Thursday, 25 May 2017 at 17:10:01 UTC, Paulo Pinto wrote:
 On Thursday, 25 May 2017 at 17:04:10 UTC, Jason King wrote:
 And how many of those are claiming to be a systems programming 
 language?

 I have no problems with an unstable ABI, what I have a problem 
 is with
 claiming to be a systems programming language AND not having a 
 stable ABI.
 You realistically cannot have both — it seems like D is trying 
 to have it’s
 cake and eat it too and I’m just pointing out that it’s going 
 to lead to
 sadness.  If there are no plans to ever have a stable ABI, 
 that’s fine (may
 even be good for the long term usage of the language), just 
 drop the whole
 systems programming language bit and focus more on application 
 level, but
 I’ve not really seen any recognition of that.
From that point of view, C++ and Ada aren't system programming languages.
Exactly. Maybe I have holes in my memory, but I can't remember anything on "final C++ ABI standard". In fact I can even recall something like this - "by not doing so(ABI freeze, I mean) we are able to stay in touch with newer hardware"
May 25
prev sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 05/25/2017 01:04 PM, Jason King via Digitalmars-d wrote:
 
 I have no problems with an unstable ABI, what I have a problem is with
 claiming to be a systems programming language AND not having a stable ABI.
 You realistically cannot have both
Why?
May 25
parent reply Jason King via Digitalmars-d <digitalmars-d puremagic.com> writes:
On May 26, 2017 at 12:11:09 AM, Nick Sabalausky (Abscissa) via
Digitalmars-d (digitalmars-d puremagic.com) wrote:

On 05/25/2017 01:04 PM, Jason King via Digitalmars-d wrote:
 I have no problems with an unstable ABI, what I have a problem is with
 claiming to be a systems programming language AND not having a stable ABI=
.
 You realistically cannot have both
Why? Unless it=E2=80=99s a completely self contained system, if you are operatin= g at the systems level, you are going to be providing the foundations for other people=E2=80=99s code to build on (vs. application level stuff where you=E2= =80=99re already at the top). Just imagine if every time you applied Windows updates or applied a security fix to your OS of choice you had to rebuild every piece of software you have, and if providing it to others, provide corresponding versions of your code with every update and fix that was released with your OS =E2=80=94 not because anything of yours changed, but because just becaus= e you wanted to fix a problem with the underlying system. Trying to build something on top of an unstable ABI is building your foundations on sand. All I=E2=80=99m saying is if no attention is going to be paid to this (it d= oesn=E2=80=99t mean you can=E2=80=99t change the ABI, but it needs to be managed it better= than =E2=80=98whoops!=E2=80=99), just stop claiming the systems bit and stay up = stack where this isn=E2=80=99t a problem.
May 26
parent Ola Fosheim Grostad <ola.fosheim.grostad gmail.com> writes:
On Friday, 26 May 2017 at 13:23:20 UTC, Jason King wrote:
 wanted to fix a problem with the underlying system.   Trying to 
 build
 something on top of an unstable ABI is building your 
 foundations on sand.

 All I’m saying is if no attention is going to be paid to this 
 (it doesn’t mean you can’t change the ABI, but it needs to be 
 managed it better than ‘whoops!’), just stop claiming the 
 systems bit and stay up stack where this isn’t a problem.
There is some truth to this as BeOS used C++, and ABI was a concern, but it really depends on the context. D has a too big runtime and too many runtime dependent features to be classified as a low level language anyway, though...
May 26
prev sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/25/17 11:02 AM, Jason King via Digitalmars-d wrote:
 That’s a fairly important requirement if it’s supposed to be a systems
 programming language, less so for application focused stuff.  I would
 hope it’s at least an eventual goal even if it’s not quite the case today.
Two large reasons why D is not focused on ABI compatibility: 1. Most of Phobos is templates. Template signatures can change easily from one version to the next, even if the template didn't change. C++ has the same issue. 2. D is generally statically compiled, not with shared libs. There is some progress on this front, but really, until the shared library support is mainstream, then having binary compatibility between releases, and therefore ABI compatibility, is not critical -- you have to recompile anyway. -Steve
May 25