www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - What does 'inline' mean?

reply Manu <turkeyman gmail.com> writes:
Inline has been bugging me forever, it's usually not what I want. The spec
says this-or-that, but I think we should take a step back, ignore what's
written there, look at the problem space, determine the set of things that
we want, and then make sure they're expressed appropriately.

I think a first part of the conversation to understand, is that since D
doesn't really have first-class `inline` (just a pragma, assumed to be
low-level compiler control), I think most people bring their conceptual
definition over from C/C++, and that definition is a little odd (although
it is immensely useful), but it's not like what D does.

In C/C++, inline says that a function will be emit to the binary only when
it is called, and the function is marked with internal linkage (it is not
visible to the linker from the symbol table)
By this definition; what inline REALLY means is that the function is not
placed in the binary where it is defined, it is placed in the binary where
it is CALLED, and each CU that calls an inline function receives their own
copy of the function. From here; optimisers will typically inline the call
if they determine it's an advantage to do so.

Another take on inline, and perhaps a more natural take (if your mind is
not poisoned by other native languages), is that the function is not
actually emit to an object anywhere, it is rather wired directly inline
into the AST instead of 'called'. Essentially a form of AST macro.

I reach for inline in C/C++ for various different reasons at different
times, and I'd like it if we were able to express each of them:

1. I only want the function to be present in the CALLING binary. I do not
want an inline function present in the local binary where it was defined
(unless it was called internally). I do not want a linker to see the inline
function symbols and be able to link to them externally. [This is about
linkage and controlling the binary or distribution environment]

2. I am unhappy that the optimiser chose to not inline a function call, and
I want to override that judgement. [This is about micro-optimisation]

3. I want to treat the function like an AST macro; I want the function
inserted at the callsite, and I want to have total confidence in this
mechanic. [This is about articulate mechanical control over code-gen; ie, I
know necessary facts about the execution context/callstack that I expect to
maintain]

I think these are the 3 broad categories of behaviour I have ever wanted
control over.
Personally speaking, I am perfectly happy with C/C++'s choice to conflate 1
& 2 into a thing called `inline`, and that's what I want the word 'inline'
to do at an absolute minimum.
The 3rd thing I would term something like `force inline` or perhaps
pragma(inline, force).

D's inline today doesn't do any of those things. It doesn't implement a
mechanic that I have ever wanted or known a use for.

Are there non-theoretical use cases I've missed that people have
encountered?
Jun 07
next sibling parent reply evilrat <evilrat666 gmail.com> writes:
On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:
 I think a first part of the conversation to understand, is that 
 since D doesn't really have first-class `inline` (just a 
 pragma, assumed to be low-level compiler control), I think most 
 people bring their conceptual definition over from C/C++, and 
 that definition is a little odd (although it is immensely 
 useful), but it's not like what D does.

 In C/C++, inline says that a function will be emit to the 
 binary only when
 it is called, and the function is marked with internal linkage 
 (it is not
 visible to the linker from the symbol table)
 By this definition; what inline REALLY means is that the 
 function is not
 placed in the binary where it is defined, it is placed in the 
 binary where
 it is CALLED, and each CU that calls an inline function 
 receives their own
 copy of the function. From here; optimisers will typically 
 inline the call
 if they determine it's an advantage to do so.
Sorry for sticking in my nose, but in result inline in C++ simply tells the linker "this symbol could be present multiple times, just pick whichever one of them and done with it" so it won't complain about multiple symbols anymore. And that's why you have to slap inline to functions that have body in header in order to build. But I'm not very certain if that's all about it, not using C++ that much since then, and never to that truly deep extent where any loose bit can destroy performance, not beyond fixing the client's code and some crappy pet projects.
Jun 08
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/8/2020 1:42 AM, evilrat wrote:
 Sorry for sticking in my nose, but in result inline in C++ simply tells the 
 linker "this symbol could be present multiple times, just pick whichever one
of 
 them and done with it" so it won't complain about multiple symbols anymore.
 And that's why you have to slap inline to functions that have body in header
in 
 order to build.
You're right that the linker expected to combine multiple copies into one. A more advanced linker will remove unreferenced functions.
Jun 08
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/8/20 4:42 AM, evilrat wrote:
 On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:
 I think a first part of the conversation to understand, is that since 
 D doesn't really have first-class `inline` (just a pragma, assumed to 
 be low-level compiler control), I think most people bring their 
 conceptual definition over from C/C++, and that definition is a little 
 odd (although it is immensely useful), but it's not like what D does.

 In C/C++, inline says that a function will be emit to the binary only 
 when
 it is called, and the function is marked with internal linkage (it is not
 visible to the linker from the symbol table)
 By this definition; what inline REALLY means is that the function is not
 placed in the binary where it is defined, it is placed in the binary 
 where
 it is CALLED, and each CU that calls an inline function receives their 
 own
 copy of the function. From here; optimisers will typically inline the 
 call
 if they determine it's an advantage to do so.
Sorry for sticking in my nose, but in result inline in C++ simply tells the linker "this symbol could be present multiple times, just pick whichever one of them and done with it" so it won't complain about multiple symbols anymore. And that's why you have to slap inline to functions that have body in header in order to build. But I'm not very certain if that's all about it, not using C++ that much since then, and never to that truly deep extent where any loose bit can destroy performance, not beyond fixing the client's code and some crappy pet projects.
This was not true the last time I found "inline abuse". At a previous company, there was a type where a function was inline or not depending on whether you defined a macro. And when you defined that macro, the implementation was DIFFERENT (and different in a way that calling the non-inline version would be a bug). Which meant that if the linker was free to choose whichever implementation it wanted, it would defeat the purpose of the inline mechanism (BTW, this was the first time I ever experienced an inline function, and had to research what it meant). I think Manu's description is accurate, but I also thought it actually inlined the function call as well, even without an optimizer. Hard to tell when the end result isn't obvious. -Steve
Jun 08
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Tue, Jun 9, 2020 at 1:00 AM Steven Schveighoffer via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/8/20 4:42 AM, evilrat wrote:
 On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:
 I think a first part of the conversation to understand, is that since
 D doesn't really have first-class `inline` (just a pragma, assumed to
 be low-level compiler control), I think most people bring their
 conceptual definition over from C/C++, and that definition is a little
 odd (although it is immensely useful), but it's not like what D does.

 In C/C++, inline says that a function will be emit to the binary only
 when
 it is called, and the function is marked with internal linkage (it is
not
 visible to the linker from the symbol table)
 By this definition; what inline REALLY means is that the function is not
 placed in the binary where it is defined, it is placed in the binary
 where
 it is CALLED, and each CU that calls an inline function receives their
 own
 copy of the function. From here; optimisers will typically inline the
 call
 if they determine it's an advantage to do so.
Sorry for sticking in my nose, but in result inline in C++ simply tells the linker "this symbol could be present multiple times, just pick whichever one of them and done with it" so it won't complain about multiple symbols anymore. And that's why you have to slap inline to functions that have body in header in order to build. But I'm not very certain if that's all about it, not using C++ that much since then, and never to that truly deep extent where any loose bit can destroy performance, not beyond fixing the client's code and some crappy pet projects.
This was not true the last time I found "inline abuse". At a previous company, there was a type where a function was inline or not depending on whether you defined a macro. And when you defined that macro, the implementation was DIFFERENT (and different in a way that calling the non-inline version would be a bug). Which meant that if the linker was free to choose whichever implementation it wanted, it would defeat the purpose of the inline mechanism (BTW, this was the first time I ever experienced an inline function, and had to research what it meant). I think Manu's description is accurate, but I also thought it actually inlined the function call as well, even without an optimizer. Hard to tell when the end result isn't obvious.
What's funny is, in most cases, whether the function is ACTUALLY inlined is not really interesting in 2020. What inline allows is control over the binary environment as I describe. I read it these days as "inline to the calling CU" rather than "inline to the calling function". There are cases where inline is really important, and I do want an error if it fails; for instance, if you have a leaf function (does not allocate any stack memory), it's only possible to make calls from that function where the callee is inlined... and if inlining fails, your caller will lose its no-stack-frame requirement. I've had this come up numerous times, and in those cases, a really-strong-does-make-compile-error inline would be useful, but C++ doesn't have anything like that.
Jun 08
next sibling parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Tuesday, 9 June 2020 at 00:36:18 UTC, Manu wrote:

 What's funny is, in most cases, whether the function is 
 ACTUALLY inlined is
 not really interesting in 2020.
 What inline allows is control over the binary environment as I 
 describe. I
 read it these days as "inline to the calling CU" rather than 
 "inline to the
 calling function".

 There are cases where inline is really important, and I do want 
 an error if it fails; for instance, if you have a leaf function 
 (does not allocate any stack memory), it's only possible to 
 make calls from that function where the callee is inlined... 
 and if inlining fails, your caller will lose its no-stack-frame 
 requirement. I've had this come up numerous times, and in those 
 cases, a really-strong-does-make-compile-error inline would be 
 useful, but C++ doesn't have anything like that.
Maybe it's a case where a clear disambiguation is in order? E.g. make a new pragma(local); ...which would instruct the compiler to do what you're describing. Or, perhaps, expand the range of options for the existing pragma(inline), from the current bool to an enum of behaviors.
Jun 08
parent reply Manu <turkeyman gmail.com> writes:
On Tue, Jun 9, 2020 at 11:30 AM Stanislav Blinov via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 9 June 2020 at 00:36:18 UTC, Manu wrote:

 What's funny is, in most cases, whether the function is
 ACTUALLY inlined is
 not really interesting in 2020.
 What inline allows is control over the binary environment as I
 describe. I
 read it these days as "inline to the calling CU" rather than
 "inline to the
 calling function".

 There are cases where inline is really important, and I do want
 an error if it fails; for instance, if you have a leaf function
 (does not allocate any stack memory), it's only possible to
 make calls from that function where the callee is inlined...
 and if inlining fails, your caller will lose its no-stack-frame
 requirement. I've had this come up numerous times, and in those
 cases, a really-strong-does-make-compile-error inline would be
 useful, but C++ doesn't have anything like that.
Maybe it's a case where a clear disambiguation is in order? E.g. make a new pragma(local); ...which would instruct the compiler to do what you're describing. Or, perhaps, expand the range of options for the existing pragma(inline), from the current bool to an enum of behaviors.
I've suggested that before, and I think that's what I'd encourage: pragma(inline, never) do not inline pragma(inline, true) like C/C++, emit to calling CU, hint preference the optimiser (if it is capable of receiving hints) pragma(inline, force) same as true, but error when it fails
Jun 08
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 09/06/2020 1:40 PM, Manu wrote:
 I've suggested that before, and I think that's what I'd encourage:
    pragma(inline, never) do not inline
    pragma(inline, true) like C/C++, emit to calling CU, hint preference 
 the optimiser (if it is capable of receiving hints)
    pragma(inline, force) same as true, but error when it fails
+1
Jun 08
prev sibling next sibling parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Tuesday, 9 June 2020 at 00:36:18 UTC, Manu wrote:
 On Tue, Jun 9, 2020 at 1:00 AM Steven Schveighoffer via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

[...]
What's funny is, in most cases, whether the function is ACTUALLY inlined is not really interesting in 2020. What inline allows is control over the binary environment as I describe. I read it these days as "inline to the calling CU" rather than "inline to the calling function". [...]
Even when forcing inlining with __attribute__((always_inline)) ? (in gcc).
Jun 08
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 9 June 2020 at 00:36:18 UTC, Manu wrote:
 for instance, if you have a leaf function (does not allocate 
 any stack memory), it's only possible to make calls from that 
 function where the callee is inlined... and if inlining fails, 
 your caller will lose its no-stack-frame requirement.
Out of interest: What does implementing a function without a stack frame enable?
Jun 09
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 9 June 2020 at 10:19:44 UTC, Dukc wrote:
 On Tuesday, 9 June 2020 at 00:36:18 UTC, Manu wrote:
 for instance, if you have a leaf function (does not allocate 
 any stack memory), it's only possible to make calls from that 
 function where the callee is inlined... and if inlining fails, 
 your caller will lose its no-stack-frame requirement.
Out of interest: What does implementing a function without a stack frame enable?
Better predictability of memory access patterns for example. Knowing that you can't have a stack overflow.
Jun 09
prev sibling parent Manu <turkeyman gmail.com> writes:
On Tue, Jun 9, 2020 at 8:20 PM Dukc via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 9 June 2020 at 00:36:18 UTC, Manu wrote:
 for instance, if you have a leaf function (does not allocate
 any stack memory), it's only possible to make calls from that
 function where the callee is inlined... and if inlining fails,
 your caller will lose its no-stack-frame requirement.
Out of interest: What does implementing a function without a stack frame enable?
Leaf functions have a massive performance advantage, and in certain situations, functions can run in a zero-ram or zero-stack environment; micro controllers, within scheduler implementations while performing operations like context switching, come constructs of hardware exception handling, etc. The most common case is maintaining a leaf function's leaf-ness.
Jun 09
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/20 10:59 AM, Steven Schveighoffer wrote:
 I think Manu's description is accurate
Most of Manu's description is flat wrong or confuses behavior of specific implementations with standard guarantees.
Jun 12
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/12/20 9:29 AM, Andrei Alexandrescu wrote:
 On 6/8/20 10:59 AM, Steven Schveighoffer wrote:
 I think Manu's description is accurate
Most of Manu's description is flat wrong or confuses behavior of specific implementations with standard guarantees.
Yeah, I read the inline documentation for C++, and it's quite different from what Manu said, and from my experience (this was a long time ago). I think the code I had dealt with that used inline was not correct. -Steve
Jun 12
prev sibling next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:
 Inline has been bugging me forever, it's usually not what I 
 want. The spec says this-or-that, but I think we should take a 
 step back, ignore what's written there, look at the problem 
 space, determine the set of things that we want, and then make 
 sure they're expressed appropriately.

 I think a first part of the conversation to understand, is that 
 since D doesn't really have first-class `inline` (just a 
 pragma, assumed to be low-level compiler control), I think most 
 people bring their conceptual definition over from C/C++, and 
 that definition is a little odd (although it is immensely 
 useful), but it's not like what D does.

 In C/C++, inline says that a function will be emit to the 
 binary only when
 it is called, and the function is marked with internal linkage 
 (it is not
 visible to the linker from the symbol table)
 By this definition; what inline REALLY means is that the 
 function is not
 placed in the binary where it is defined, it is placed in the 
 binary where
 it is CALLED, and each CU that calls an inline function 
 receives their own
 copy of the function. From here; optimisers will typically 
 inline the call
 if they determine it's an advantage to do so.

 Another take on inline, and perhaps a more natural take (if 
 your mind is not poisoned by other native languages), is that 
 the function is not actually emit to an object anywhere, it is 
 rather wired directly inline into the AST instead of 'called'. 
 Essentially a form of AST macro.
No. I rather see "inline" as a hint for the backend. DMD is peculiar with its way of inlining.
 I reach for inline in C/C++ for various different reasons at 
 different times, and I'd like it if we were able to express 
 each of them:

 1. I only want the function to be present in the CALLING 
 binary. I do not want an inline function present in the local 
 binary where it was defined (unless it was called internally). 
 I do not want a linker to see the inline function symbols and 
 be able to link to them externally. [This is about linkage and 
 controlling the binary or distribution environment]
what if the function address is took in a delegate ? It still needs to be there, in the object matching to the CU where it is declared, otherwise there will be surprises, e.g &func in a CU and &func in another will have different addresses.
Jun 08
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 8 June 2020 at 08:45:02 UTC, Basile B. wrote:

 what if the function address is took in a delegate ?
 It still needs to be there, in the object matching to the CU 
 where it is declared, otherwise there will be surprises, e.g 
 &func in a CU and &func in another will have different 
 addresses.
I have _never_ compared addresses of inline functions from different CUs. How would I even do that? by storing the function pointer in a global which is visible from multiple translation units? Is that a valid concern? Will _any_ project out there break because of that?
Jun 08
next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Monday, 8 June 2020 at 08:47:36 UTC, Stefan Koch wrote:
 On Monday, 8 June 2020 at 08:45:02 UTC, Basile B. wrote:

 what if the function address is took in a delegate ?
 It still needs to be there, in the object matching to the CU 
 where it is declared, otherwise there will be surprises, e.g 
 &func in a CU and &func in another will have different 
 addresses.
I have _never_ compared addresses of inline functions from different CUs. How would I even do that? by storing the function pointer in a global which is visible from multiple translation units? Is that a valid concern? Will _any_ project out there break because of that?
It's a detail. I just meant that in case where the address of function that is marked for inlining is took then it must still be emitted in the object matching to the declaration unit.
Jun 08
next sibling parent Basile B. <b2.temp gmx.com> writes:
On Monday, 8 June 2020 at 09:08:33 UTC, Basile B. wrote:
 On Monday, 8 June 2020 at 08:47:36 UTC, Stefan Koch wrote:
 On Monday, 8 June 2020 at 08:45:02 UTC, Basile B. wrote:

 [...]
I have _never_ compared addresses of inline functions from different CUs. How would I even do that? by storing the function pointer in a global which is visible from multiple translation units? Is that a valid concern? Will _any_ project out there break because of that?
It's a detail. I just meant that in case where the address of function that is marked for inlining is took then it must still be emitted in the object matching to the declaration unit.
but of course inlined where it is used as-is.
Jun 08
prev sibling parent Manu <turkeyman gmail.com> writes:
On Mon, Jun 8, 2020 at 7:10 PM Basile B. via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Monday, 8 June 2020 at 08:47:36 UTC, Stefan Koch wrote:
 On Monday, 8 June 2020 at 08:45:02 UTC, Basile B. wrote:

 what if the function address is took in a delegate ?
 It still needs to be there, in the object matching to the CU
 where it is declared, otherwise there will be surprises, e.g
 &func in a CU and &func in another will have different
 addresses.
I have _never_ compared addresses of inline functions from different CUs. How would I even do that? by storing the function pointer in a global which is visible from multiple translation units? Is that a valid concern? Will _any_ project out there break because of that?
It's a detail. I just meant that in case where the address of function that is marked for inlining is took then it must still be emitted in the object matching to the declaration unit.
It should be emit to the CU where the address is taken.
Jun 08
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Monday, 8 June 2020 at 08:47:36 UTC, Stefan Koch wrote:
 Is that a valid concern?
 Will _any_ project out there break because of that?
It's easy to imagine. If one has a very simple mathematical function taking, say int and float and returning a float, and that function is called in a very hot loop, performance can drop noticeably if inlining suddently fails. Especially if the mathematical function calls another function instead of using the compiler primitives directly.
Jun 08
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 8 June 2020 at 11:53:11 UTC, Dukc wrote:
 On Monday, 8 June 2020 at 08:47:36 UTC, Stefan Koch wrote:
 Is that a valid concern?
 Will _any_ project out there break because of that?
It's easy to imagine. If one has a very simple mathematical function taking, say int and float and returning a float, and that function is called in a very hot loop, performance can drop noticeably if inlining suddently fails. Especially if the mathematical function calls another function instead of using the compiler primitives directly.
I didn't ask about not inlinng. I asked about the address of a function changing. From object file to object file
Jun 08
prev sibling parent Manu <turkeyman gmail.com> writes:
On Mon, Jun 8, 2020 at 6:50 PM Basile B. via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:
 Inline has been bugging me forever, it's usually not what I
 want. The spec says this-or-that, but I think we should take a
 step back, ignore what's written there, look at the problem
 space, determine the set of things that we want, and then make
 sure they're expressed appropriately.

 I think a first part of the conversation to understand, is that
 since D doesn't really have first-class `inline` (just a
 pragma, assumed to be low-level compiler control), I think most
 people bring their conceptual definition over from C/C++, and
 that definition is a little odd (although it is immensely
 useful), but it's not like what D does.

 In C/C++, inline says that a function will be emit to the
 binary only when
 it is called, and the function is marked with internal linkage
 (it is not
 visible to the linker from the symbol table)
 By this definition; what inline REALLY means is that the
 function is not
 placed in the binary where it is defined, it is placed in the
 binary where
 it is CALLED, and each CU that calls an inline function
 receives their own
 copy of the function. From here; optimisers will typically
 inline the call
 if they determine it's an advantage to do so.

 Another take on inline, and perhaps a more natural take (if
 your mind is not poisoned by other native languages), is that
 the function is not actually emit to an object anywhere, it is
 rather wired directly inline into the AST instead of 'called'.
 Essentially a form of AST macro.
No. I rather see "inline" as a hint for the backend. DMD is peculiar with its way of inlining.
 I reach for inline in C/C++ for various different reasons at
 different times, and I'd like it if we were able to express
 each of them:

 1. I only want the function to be present in the CALLING
 binary. I do not want an inline function present in the local
 binary where it was defined (unless it was called internally).
 I do not want a linker to see the inline function symbols and
 be able to link to them externally. [This is about linkage and
 controlling the binary or distribution environment]
what if the function address is took in a delegate ? It still needs to be there, in the object matching to the CU where it is declared, otherwise there will be surprises, e.g &func in a CU and &func in another will have different addresses.
Right. This is fine and normal.
Jun 08
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/7/2020 11:14 PM, Manu wrote:
 I think a first part of the conversation to understand, is that since D
doesn't 
 really have first-class `inline` (just a pragma, assumed to be low-level 
 compiler control), I think most people bring their conceptual definition over 
 from C/C++, and that definition is a little odd (although it is immensely 
 useful), but it's not like what D does.
C/C++ inline has always been a hint to the compiler, not a command.
 In C/C++, inline says that a function will be emit to the binary only when it
is 
 called, and the function is marked with internal linkage (it is not visible to 
 the linker from the symbol table)
 By this definition; what inline REALLY means is that the function is not
placed 
 in the binary where it is defined, it is placed in the binary where it is 
 CALLED, and each CU that calls an inline function receives their own copy of
the 
 function.
Why does it matter where it is emitted? Why would you want multiple copies of the same function in the binary?
 Another take on inline, and perhaps a more natural take (if your mind is not 
 poisoned by other native languages), is that the function is not actually emit 
 to an object anywhere, it is rather wired directly inline into the AST instead 
 of 'called'. Essentially a form of AST macro.
The problem with this is what is inlined and what isn't is rather fluid, i.e. it varies depending on circumstances and which compiler you use. For example, if you recursively call a function, it's going to have to give up on inlining it. Changes in the compiler can expand or contract inlining opportunities. Having it inline or issue an error is disaster for compiling existing code without constantly breaking it.
 I reach for inline in C/C++ for various different reasons at different times, 
 and I'd like it if we were able to express each of them:
 
 1. I only want the function to be present in the CALLING binary. I do not want 
 an inline function present in the local binary where it was defined (unless it 
 was called internally). I do not want a linker to see the inline function 
 symbols and be able to link to them externally. [This is about linkage and 
 controlling the binary or distribution environment]
Why? What is the problem with the emission of one copy where it was defined?
 2. I am unhappy that the optimiser chose to not inline a function call, and I 
 want to override that judgement. [This is about micro-optimisation]
It's not always possible to inline a function.
 3. I want to treat the function like an AST macro; I want the function
inserted 
 at the callsite, and I want to have total confidence in this mechanic. [This
is 
 about articulate mechanical control over code-gen; ie, I know necessary facts 
 about the execution context/callstack that I expect to maintain]
The PR I have on this makes it an informational warning. You can choose to be notified if inlining fails.
 Are there non-theoretical use cases I've missed that people have encountered?
At its root, inlining is an optimization, like deciding which variables go into registers.
Jun 08
next sibling parent reply Jan =?UTF-8?B?SMO2bmln?= <hrominium gmail.com> writes:
On Monday, 8 June 2020 at 10:19:16 UTC, Walter Bright wrote:
 Why does it matter where it is emitted? Why would you want 
 multiple copies of the same function in the binary?
Performance in HPC. In C++, consider an []operator. There would be a lot of function calls inside a kernel (some function with lot of loops, one billion iterations of the inner most loop easily). If then I have some kind of stencil or any array accesses, calling a function each time a top of resolving the current pointer would be very costly.
Jun 08
next sibling parent John Colvin <john.loughran.colvin gmail.com> writes:
On Monday, 8 June 2020 at 12:35:01 UTC, Jan Hönig wrote:
 On Monday, 8 June 2020 at 10:19:16 UTC, Walter Bright wrote:
 Why does it matter where it is emitted? Why would you want 
 multiple copies of the same function in the binary?
Performance in HPC. In C++, consider an []operator. There would be a lot of function calls inside a kernel (some function with lot of loops, one billion iterations of the inner most loop easily). If then I have some kind of stencil or any array accesses, calling a function each time a top of resolving the current pointer would be very costly.
What does this have to do with whether a symbol is emitted? Emitting a symbol doesn't mean you didn't get inlining.
Jun 08
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/8/2020 5:35 AM, Jan Hönig wrote:
 In C++, consider an []operator. There would be a lot of function calls inside
a 
 kernel (some function with lot of loops, one billion iterations of the inner 
 most loop easily). If then I have some kind of stencil or any array accesses, 
 calling a function each time a top of resolving the current pointer would be 
 very costly.
I infer what you're talking about is functions that are "strongly connected" should be located near each other in memory so they'll both be in the cache at the same time. The best way to achieve this is by runtime profiling, and using the profiling data to group together strongly connected function in the executable. The Digital Mars C/C++ compiler would do this, and it was a nice optimization. Trying to do it by hand isn't likely to be very effective.
Jun 09
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/7/2020 11:14 PM, Manu wrote:
 I think a first part of the conversation to understand, is that since D
doesn't
 really have first-class `inline` (just a pragma, assumed to be low-level
 compiler control), I think most people bring their conceptual definition
over
 from C/C++, and that definition is a little odd (although it is
immensely
 useful), but it's not like what D does.
C/C++ inline has always been a hint to the compiler, not a command.
It's not a hint at all. It's a mechanical tool; it marks symbols with internal linkage, and it also doesn't emit them if it's never referenced. The compiler may not choose to ignore that behaviour, it's absolutely necessary, and very important.
 In C/C++, inline says that a function will be emit to the binary only
 when it is
 called, and the function is marked with internal linkage (it is not
visible to
 the linker from the symbol table)
 By this definition; what inline REALLY means is that the function is not
placed
 in the binary where it is defined, it is placed in the binary where it
is
 CALLED, and each CU that calls an inline function receives their own
copy of the
 function.
Why does it matter where it is emitted? Why would you want multiple copies of the same function in the binary?
I want zero copies if it's never called. That's very important. I also want copies to appear locally when it is referenced; inline functions should NOT require that you link something to get the code... that's not inline at all.
 Another take on inline, and perhaps a more natural take (if your mind is
 not
 poisoned by other native languages), is that the function is not
actually emit
 to an object anywhere, it is rather wired directly inline into the AST
instead
 of 'called'. Essentially a form of AST macro.
The problem with this is what is inlined and what isn't is rather fluid, i.e. it varies depending on circumstances and which compiler you use. For example, if you recursively call a function, it's going to have to give up on inlining it. Changes in the compiler can expand or contract inlining opportunities. Having it inline or issue an error is disaster for compiling existing code without constantly breaking it.
I understand why this particular take is more complicated; and as such I wouldn't suggest we do it. I'm just offering it as one possible take on the concept. I think to make this form work, it must be handled in the frontend; essentially an AST macro. Naturally, it would fail on recursive calls, because that would be a recursive expansion. case could potentially be this.
 I reach for inline in C/C++ for various different reasons at different
 times,
 and I'd like it if we were able to express each of them:

 1. I only want the function to be present in the CALLING binary. I do
not want
 an inline function present in the local binary where it was defined
(unless it
 was called internally). I do not want a linker to see the inline
function
 symbols and be able to link to them externally. [This is about linkage
and
 controlling the binary or distribution environment]
Why? What is the problem with the emission of one copy where it was defined?
That's the antithesis of inline. If I wanted that, I wouldn't mark it inline. I don't want a binary full of code that shouldn't be there. It's very important to be able to control what code is in your binaries. If it's not referenced, it doesn't exist.
 2. I am unhappy that the optimiser chose to not inline a function call,
 and I
 want to override that judgement. [This is about micro-optimisation]
It's not always possible to inline a function.
 3. I want to treat the function like an AST macro; I want the function
 inserted
 at the callsite, and I want to have total confidence in this mechanic.
[This is
 about articulate mechanical control over code-gen; ie, I know necessary
facts
 about the execution context/callstack that I expect to maintain]
The PR I have on this makes it an informational warning. You can choose to be notified if inlining fails.
That's not sufficient though for all use cases. This is a different kind of inline (I think it's 'force inline'). want a sea of warnings to apply to cases of 1/2.
 Are there non-theoretical use cases I've missed that people have
 encountered?

 At its root, inlining is an optimization, like deciding which variables go
 into
 registers.
No, actually... it's not. It's not an 'optimisation' in any case except Low level control of code generation is important in native languages; that's why we're here.
Jun 08
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Monday, 8 June 2020 at 14:09:04 UTC, Manu wrote:
 [snip]
 It's not a hint at all. It's a mechanical tool; it marks 
 symbols with internal linkage, and it also doesn't emit them if 
 it's never referenced. The compiler may not choose to ignore 
 that behaviour, it's absolutely necessary, and very important.
[snip]
Perhaps it would be helpful to split the discussion into inline as a linkage attribute vs. inline as an tool for inline expansion? D has no linkage attribute that has the same behavior as inline or extern inline, and you believe that is important.
Jun 08
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/8/2020 7:09 AM, Manu wrote:
 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d 
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
 
     On 6/7/2020 11:14 PM, Manu wrote:
      > I think a first part of the conversation to understand, is that since D
     doesn't
      > really have first-class `inline` (just a pragma, assumed to be low-level
      > compiler control), I think most people bring their conceptual definition
     over
      > from C/C++, and that definition is a little odd (although it is
immensely
      > useful), but it's not like what D does.
 
     C/C++ inline has always been a hint to the compiler, not a command.
 
 
 It's not a hint at all. It's a mechanical tool; it marks symbols with internal 
 linkage, and it also doesn't emit them if it's never referenced.
 The compiler may not choose to ignore that behaviour,
The C/C++ inline semantics revolve around the mechanics of .h files because it doesn't have modules. These reasons are irrelevant for D.
 it's absolutely necessary, and very important.
For .h files, sure. Why for D, though?
     Why does it matter where it is emitted? Why would you want multiple copies
of
     the same function in the binary?
 I want zero copies if it's never called. That's very important.
Why are 0 or N copies fine, but 1 is not?
 I also want copies to appear locally when it is referenced; inline functions 
 should NOT require that you link something to get the code... that's not
inline 
 at all.
Why? What problem are you solving?
     Why? What is the problem with the emission of one copy where it was
defined?
That's the antithesis of inline. If I wanted that, I wouldn't mark it inline. I don't want a binary full of code that shouldn't be there. It's very important to be able to control what code is in your binaries.
I know I'm being boring, but why is it important? Also, check out -gc-sections: https://gcc.gnu.org/onlinedocs/gnat_ugn/Compilation-options.html Which is a general solution, not a kludge.
 If it's not referenced, it doesn't exist.
Executables (on virtual memory systems) are not loaded into memory and then run. They are memory-mapped into memory, and then pages are read off of disk on demand. Unmapped code consumes neither memory nor resources.
     The PR I have on this makes it an informational warning. You can choose to
be
     notified if inlining fails.
That's not sufficient though for all use cases. This is a different kind of inline (I think it's 'force inline').
The default, and pragma(inline,true) are sufficient for all use cases except which ones?

 sea of warnings to apply to cases of 1/2.
You won't get a sea of warnings unless you put pragma(inline,true) on a sea of functions that can't be inlined.

Why?
   At its root, inlining is an optimization, like deciding which variables go
into
   registers.
No, actually... it's not. It's not an 'optimisation' in any case except maaaaybe
Inlining is 100% about optimization.
 Low level control of code generation is important in native languages; that's 
 why we're here.
Optimizing things that don't matter is wasting your valuable time. Optimizing things that are more effectively and thoroughly done with the linker (-gc-sections) - it's like chipping wood with a hatchet rather than a woodchipper.
Jun 09
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 9 June 2020 at 09:29:47 UTC, Walter Bright wrote:
 On 6/8/2020 7:09 AM, Manu wrote:
 No, actually... it's not. It's not an 'optimisation' in any 

 output and code generation.
Inlining is 100% about optimization.
We are not talking about inlining. The whole idea behind this thread is to explain that, c++ use of "inline" does not actually mean that a function is to be inlined. What we are talking about is a non-hacky way to say, I want a copy of this function in my object file, regardless of the module it was defined in. You _can_ get the same effect by doing this: --- string I_need_this_function()(string x) { return "x: " ~ x; } --- But that introduces a template which comes with another set of problems. Such that you don't get type-checking of the body if it's not used.
Jun 09
prev sibling next sibling parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Tuesday, 9 June 2020 at 09:29:47 UTC, Walter Bright wrote:
 On 6/8/2020 7:09 AM, Manu wrote:
 [...]
The C/C++ inline semantics revolve around the mechanics of .h files because it doesn't have modules. These reasons are irrelevant for D.
 [...]
For .h files, sure. Why for D, though?
 [...]
Why are 0 or N copies fine, but 1 is not?
 [...]
Why? What problem are you solving?
 [...]
I know I'm being boring, but why is it important? Also, check out -gc-sections: https://gcc.gnu.org/onlinedocs/gnat_ugn/Compilation-options.html Which is a general solution, not a kludge.
 [...]
Executables (on virtual memory systems) are not loaded into memory and then run. They are memory-mapped into memory, and then pages are read off of disk on demand. Unmapped code consumes neither memory nor resources.
 [...]
The default, and pragma(inline,true) are sufficient for all use cases except which ones?
 [...]
You won't get a sea of warnings unless you put pragma(inline,true) on a sea of functions that can't be inlined.
 [...]
Why?
 [...]
Inlining is 100% about optimization.
You forgot the case where it is about pessimization :-) The thing with inlining is that it often looks good in benchmarks but not that good in whole applications because of the impact it can have on code size. In my modest experience, the worse offenders where always the people that insisted very much on forcing inlining. I had once a colleague that had put inlines everywhere in his lib without realizing that nesting inlined functions has a multiplying effect. I calculated the equivalent in line of codes that were inlined and it was around 6000, for a library that had only 13000 lines to begin with. The effect on the instruction cache was huge.
 [...]
Optimizing things that don't matter is wasting your valuable time. Optimizing things that are more effectively and thoroughly done with the linker (-gc-sections) - it's like chipping wood with a hatchet rather than a woodchipper.
Jun 09
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Tue, Jun 9, 2020 at 7:30 PM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/8/2020 7:09 AM, Manu wrote:
 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>>
wrote:
     On 6/7/2020 11:14 PM, Manu wrote:
      > I think a first part of the conversation to understand, is that
since D
     doesn't
      > really have first-class `inline` (just a pragma, assumed to be
low-level
      > compiler control), I think most people bring their conceptual
definition
     over
      > from C/C++, and that definition is a little odd (although it is
immensely
      > useful), but it's not like what D does.

     C/C++ inline has always been a hint to the compiler, not a command.


 It's not a hint at all. It's a mechanical tool; it marks symbols with
internal
 linkage, and it also doesn't emit them if it's never referenced.
 The compiler may not choose to ignore that behaviour,
The C/C++ inline semantics revolve around the mechanics of .h files because it doesn't have modules. These reasons are irrelevant for D.
That's completely incorrect. It's 100% as relevant for D as it is for C++ for exactly the same reasons. You'll need to support your claim.
 it's absolutely necessary, and very important.

 For .h files, sure. Why for D, though?
Because in this category of use case, inlining is a concept related to native languages with binary linkage, and not really anything to do with the language specifically. Calling an inline function from a foreign module does not require that I link that module's compiled code, because the inline function is emit locally to the calling CU. This is just as relevant in D as it is in C++, although the technical manifestations are slightly different; in C++, a function defined in a header emits a hard-copy of the function in each CU, and link errors because multiple defined symbol. In D, you end up with no copy of the function anywhere, and link errors because undefined symbol. `Inline` addresses that same issue in both cases the same way. There are a whole lot of reasons this comes up in binary ecosystems. Build systems and dev tooling is a really complex topic, which tends to take years and years of iteration for millions-loc software, and there are frequently awkward requirements for various reasons. We need to fit in with existing expectations. It must be painless to plug into existing native code ecosystems; we have invested heavily in extern C and C++, but compatibility with those ecosystems also involves integrating into complex existing legacy build, link, and distribution environments.
     Why does it matter where it is emitted? Why would you want multiple
 copies of
     the same function in the binary?
 I want zero copies if it's never called. That's very important.
Why are 0 or N copies fine, but 1 is not?
No, that's not what I've said; I expect exactly N copies of x() for N CU's which reference x(). And they should appropriately be marked with internal linkage. Any other result is just 'weird', and while it might be workable, it's just asking for trouble. (1 redundant copy in the owning CU despite being un-referenced MIGHT be link-stripped if the surrounding tooling all works as we hope... but it also might not, as I have demonstrated on multiple occasions) There's just no reason to invite this problem... and no advantage.
 I also want copies to appear locally when it is referenced; inline
 functions
 should NOT require that you link something to get the code... that's not
inline
 at all.
Why? What problem are you solving?
Literally inline function calling. No-link libs are a really common and extremely useful thing.
     Why? What is the problem with the emission of one copy where it was
defined?
 That's the antithesis of inline. If I wanted that, I wouldn't mark it
inline.
 I don't want a binary full of code that shouldn't be there. It's very
important
 to be able to control what code is in your binaries.
I know I'm being boring, but why is it important? Also, check out -gc-sections: https://gcc.gnu.org/onlinedocs/gnat_ugn/Compilation-options.html Which is a general solution, not a kludge.
I guess my key issue is that I have complained before because I have experienced multiple counts of the link stripping not working like you say. There is no reason to invite that problem; we can trivially eliminate the problem at the source. I don't care if it's fixable, I don't want the problem to exist. NOBODY wants to be tasked to solve awkward build ecosystem issues... we already have a working build ecosystem, and this D thing is making it harder than it already is. That's a really bad thing, and I would entertain excuses for this if not for the fact that it's trivially avoidable. We do not need to waste anybodies time this way; we won't win any friends by wasting their time with problems they HATE to have. The secondary issue is, I want my binaries to contain what I put in there, and not what I don't. Rogue symbols that I specified shouldn't exist only bloat the binary, invite possibility of link collision, and raise the probability of the issues I mentioned above.
     The PR I have on this makes it an informational warning. You can
choose to be
     notified if inlining fails.
That's not sufficient though for all use cases. This is a different kind
of
 inline (I think it's 'force inline').
The default, and pragma(inline,true) are sufficient for all use cases except which ones?
Recursive calls or taking the address of functions (and probably other situations) are incompatible with a hard-error based inline. CU-inline is the overwhelmingly common case. Absolutely-must-call-site-inline is extremely rare by contrast, but very important in the rare instance it's necessary. I suggest, the default should model the common case, and the rare niche case can be the 3rd 'force' state by explicit request.

 want a
 sea of warnings to apply to cases of 1/2.
You won't get a sea of warnings unless you put pragma(inline,true) on a sea of functions that can't be inlined.
There are classes of modules where EVERY function is inline. No-link libs are a very common and useful tool.

 cases.

 Why?
territory, which you need to manually validate, and then have no way to detect when the conditions or context changes. along the way...
   At its root, inlining is an optimization, like deciding which
variables go into
   registers.
No, actually... it's not. It's not an 'optimisation' in any case except
maaaaybe

Inlining is 100% about optimization.
No. I feel like I couldn't make this case clearer... It's got almost nothing to do with optimisation, it's about codegen control. In the very rare event that I disagree with an optimisers heuristic, it's nice to have an override hint, but that's like a 0.1% use case. Inline provides mechanical control over codegen, and this is a native systems language. We must have that control. GDC/LDC have it, but it really should be specced to have uniform behaviour across D compilers. We don't have macro's like C++ does to wrangle cases where implementations differ as easily.
 Low level control of code generation is important in native languages;
 that's
 why we're here.
Optimizing things that don't matter is wasting your valuable time. Optimizing things that are more effectively and thoroughly done with the linker (-gc-sections) - it's like chipping wood with a hatchet rather than a woodchipper.
This isn't about optimisation, it's about controlling the output of the compiler. Taking that control away and forcing us to try and reproduce that expected functionality with external tooling within a deeply complex build ecosystem is wasting our valuable time.
Jun 09
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/9/2020 7:21 AM, Manu wrote:
     The C/C++ inline semantics revolve around the mechanics of .h files
because it
     doesn't have modules. These reasons are irrelevant for D.
 
 
 That's completely incorrect. It's 100% as relevant for D as it is for C++ for 
 exactly the same reasons.
 You'll need to support your claim.
Um, because of the way #include works, C++ sees the same inline function definition over and over again, and has no way of knowing if any other files see the same definition or not, or if they have the same implementation or not. The semantics are carefully crafted around the reality that there may be multiple copies of the inline function definition in compilation units that have no clue of the existence of each other. The internal linkage thing is there to support more primitive linkers that have no support for COMDAT sections. This is completely different from a module system. Also, D relies heavily on linker support for COMDATs.
 Because in this category of use case, inlining is a concept related to native 
 languages with binary linkage, and not really anything to do with the language 
 specifically.
Given all the verbiage about inlines in the C++ spec, it does have to do with the language specifically.
 Any other result is just 'weird', and while it might be workable, it's just 
 asking for trouble. (1 redundant copy in the owning CU despite being 
 un-referenced MIGHT be link-stripped if the surrounding tooling all works as
we 
 hope... but it also might not, as I have demonstrated on multiple occasions)
 There's just no reason to invite this problem... and no advantage.
 
      > I also want copies to appear locally when it is referenced; inline
functions
      > should NOT require that you link something to get the code... that's not
     inline
      > at all.
 
     Why? What problem are you solving?
 
 
 Literally inline function calling. No-link libs are a really common and 
 extremely useful thing.
At last! You say why! We have that in D with template functions. You get N instantiations. But they all have COMDAT linkage, which means only 1 winds up in the executable instead of N.
     I know I'm being boring, but why is it important? Also, check out
-gc-sections:
 
     https://gcc.gnu.org/onlinedocs/gnat_ugn/Compilation-options.html
 
     Which is a general solution, not a kludge.
 
 
 I guess my key issue is that I have complained before because I have
experienced 
 multiple counts of the link stripping not working like you say. There is no 
 reason to invite that problem; we can trivially eliminate the problem at the 
 source. I don't care if it's fixable, I don't want the problem to exist.
NOBODY 
 wants to be tasked to solve awkward build ecosystem issues... we already have
a 
 working build ecosystem, and this D thing is making it harder than it already 
 is. That's a really bad thing, and I would entertain excuses for this if not
for 
 the fact that it's trivially avoidable. We do not need to waste anybodies time 
 this way; we won't win any friends by wasting their time with problems they
HATE 
 to have.
If gc-sections isn't working, perhaps there's something else going on. Did you check with the linker support people? gc-sections has been around at least 20 years with gcc. If it fundamentally didn't work somebody would have fixed it by now. It also may not work with your C++ compiler because it is NOT emitting the functions each into its own section. DMD *does* put each and every function into its own section, so that gc-sections will work. I.e. C++ has workarounds for bugs it its own implementation, and those workarounds are not features.
 The secondary issue is, I want my binaries to contain what I put in there, and 
 not what I don't. Rogue symbols that I specified shouldn't exist only bloat
the 
 binary, invite possibility of link collision, and raise the probability of the 
 issues I mentioned above.
But you're happy with (and require) N useless redundant copies of inline functions.
 I suggest, the default should model the common case, and the rare niche case
can 
 be the 3rd 'force' state by explicit request.
The upcoming case is pragma(inline,true) (in my PR) is that the compiler attempts to inline them regardless of the -inline switch, and lists an informational warning (-wi) if they can't be inlined.
 This isn't about optimisation, it's about controlling the output of the 
 compiler. Taking that control away and forcing us to try and reproduce that 
 expected functionality with external tooling within a deeply complex build 
 ecosystem is wasting our valuable time.
C++'s inline semantics are rooted in accommodation for: 1. .h files rather than modules 2. object file formats that don't support COMDATs 3. linkers that don't support COMDATs It's a collection of hacks, not features. D is free'd from those constraints.
Jun 09
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jun 09, 2020 at 04:48:17PM -0700, Walter Bright via Digitalmars-d wrote:
 On 6/9/2020 7:21 AM, Manu wrote:
[...]
   Why? What problem are you solving?
Literally inline function calling. No-link libs are a really common and extremelyuseful thing.
At last! You say why! We have that in D with template functions. You get N instantiations. But they all have COMDAT linkage, which means only 1 winds up in the executable instead of N.
[...] Yeah, I was about to say, judging from what Manu has written several times in this thread, his concept of 'inline' bears striking resemblance to a template function: - The template function is not instantiated in its defining module (unless something else instantiates it there); - A template function is instantiated in the invoking module, just like how Manu described his 'inline' functions; - A template function is more likely to be actually inlined (recently I've been adding () to some of my functions in order to prod the compiler to inline them). Manu has even stated that whether or not the 'inline' function is actually inlined isn't even that important anymore; given that, I'd say template functions are what he's looking for all this time! T -- Why ask rhetorical questions? -- JC
Jun 09
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 10 June 2020 at 00:02:09 UTC, H. S. Teoh wrote:
 On Tue, Jun 09, 2020 at 04:48:17PM -0700, Walter Bright via 
 Digitalmars-d wrote:
 [...]
[...]
 [...]
[...] Yeah, I was about to say, judging from what Manu has written several times in this thread, his concept of 'inline' bears striking resemblance to a template function: [...]
Yes making the function a function template will do that. And Manu is fully aware of it. It's a nasty hack. And in certain cases it screws with overload-resolution.
Jun 09
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/9/20 8:20 PM, Stefan Koch wrote:
 On Wednesday, 10 June 2020 at 00:02:09 UTC, H. S. Teoh wrote:
 On Tue, Jun 09, 2020 at 04:48:17PM -0700, Walter Bright via 
 Digitalmars-d wrote:
 [...]
[...]
 [...]
[...] Yeah, I was about to say, judging from what Manu has written several times in this thread, his concept of 'inline' bears striking resemblance to a template function: [...]
Yes making the function a function template will do that. And Manu is fully aware of it. It's a nasty hack. And in certain cases it screws with overload-resolution.
It's not exactly the same. If the compiler thinks an imported module has instantiated it, it will not include it in the object file (unless you use -allinst). -Steve
Jun 10
parent reply kinke <noone nowhere.com> writes:
On Wednesday, 10 June 2020 at 13:55:40 UTC, Steven Schveighoffer 
wrote:
 On 6/9/20 8:20 PM, Stefan Koch wrote:
 Yes making the function a function template will do that.
 And Manu is fully aware of it.
 
 It's a nasty hack.
 And in certain cases it screws with overload-resolution.
It's not exactly the same. If the compiler thinks an imported module has instantiated it, it will not include it in the object file (unless you use -allinst).
Nope, -allinst doesn't make sure all templates are instantiated in each object file. I'll link again to https://github.com/ldc-developers/ldc/pull/3422#issuecomment-625945508.
Jun 10
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/10/20 2:54 PM, kinke wrote:
 On Wednesday, 10 June 2020 at 13:55:40 UTC, Steven Schveighoffer wrote:
 On 6/9/20 8:20 PM, Stefan Koch wrote:
 Yes making the function a function template will do that.
 And Manu is fully aware of it.

 It's a nasty hack.
 And in certain cases it screws with overload-resolution.
It's not exactly the same. If the compiler thinks an imported module has instantiated it, it will not include it in the object file (unless you use -allinst).
Nope, -allinst doesn't make sure all templates are instantiated in each object file. I'll link again to https://github.com/ldc-developers/ldc/pull/3422#issuecomment-625945508.
I can't tell from that discussion what you mean. My understanding with -allinst is that the compiler is supposed to instantiate all templates even if it thinks a dependent library has already done it. But I'm happy to defer to you as an actual developer of the compiler, I'm not. I have had experience as a user with the compiler making wrong decisions about whether it should emit code for a template instance or not. In any case, my point is that just making it a template isn't the equivalent as has been suggested. -Steve
Jun 10
parent reply kinke <noone nowhere.com> writes:
On Wednesday, 10 June 2020 at 19:20:55 UTC, Steven Schveighoffer 
wrote:
 My understanding with -allinst is that the compiler is supposed 
 to instantiate all templates even if it thinks a dependent 
 library has already done it.

 But I'm happy to defer to you as an actual developer of the 
 compiler, I'm not. I have had experience as a user with the 
 compiler making wrong decisions about whether it should emit 
 code for a template instance or not.

 In any case, my point is that just making it a template isn't 
 the equivalent as has been suggested.
You're spot-on, except for the fact that a template instance is emitted into at most 1 object file per compiler command-line. So if you compile a static lib with 100 modules in one go, and each module instantiates some common template, it's emitted into exactly one object file when using -allinst (and possibly -unittest too, I don't remember exactly), and *might* be emitted into a (single) object file without -allinst (because as you said, if the compiler sees another non-compiled imported module instantiating it, the compiler elides its codegen and depends on that external definition later when linking). TLDR: Templates, with the current emission scheme, can definitely not be used to emit a function into every referencing CU.
Jun 10
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/10/2020 1:10 PM, kinke wrote:
 TLDR: Templates, with the current emission scheme, can definitely not be used
to 
 emit a function into every referencing CU.
Yes, but in Manu's case the idea is to not need to link with a library. It's hard to see how the compiler would know that a template was instantiated in a library - meaning it'll emit an instance for at least one of the modules given on the command line. That should resolve the particular issue Manu was having.
Jun 11
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 11 June 2020 at 10:05:14 UTC, Walter Bright wrote:
 On 6/10/2020 1:10 PM, kinke wrote:
 TLDR: Templates, with the current emission scheme, can 
 definitely not be used to emit a function into every 
 referencing CU.
Yes, but in Manu's case the idea is to not need to link with a library. It's hard to see how the compiler would know that a template was instantiated in a library - meaning it'll emit an instance for at least one of the modules given on the command line. That should resolve the particular issue Manu was having.
The Idea is to be able to have a "header-only" library where importing is enough to use it. Which will only (transitively) draw in the functionality used. While it is possible with nullary templates it is dissatisfying, dissatisfying enough to cause a busy person like Manu to write a post about it.
Jun 11
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/11/20 6:05 AM, Walter Bright wrote:
 On 6/10/2020 1:10 PM, kinke wrote:
 TLDR: Templates, with the current emission scheme, can definitely not 
 be used to emit a function into every referencing CU.
Yes, but in Manu's case the idea is to not need to link with a library. It's hard to see how the compiler would know that a template was instantiated in a library - meaning it'll emit an instance for at least one of the modules given on the command line. That should resolve the particular issue Manu was having.
An example: --- b.d: module b; int foo(T)() { return 1; } enum x = foo!int(); --- a.d: import b; void main() { auto = foo!int(); } ---
 dmd a.d
Undefined symbols for architecture x86_64: "__D1b__T3fooTiZQhFNaNbNiNfZi", referenced from: __Dmain in a.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) Error: linker exited with status 1 comment out the enum and it works. -Steve
Jun 11
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/11/2020 8:16 AM, Steven Schveighoffer wrote:
 comment out the enum and it works.
Sounds good to me. Don't do that in the "header" file and it'll work.
Jun 11
prev sibling parent reply kinke <noone nowhere.com> writes:
On Thursday, 11 June 2020 at 10:05:14 UTC, Walter Bright wrote:
 On 6/10/2020 1:10 PM, kinke wrote:
 TLDR: Templates, with the current emission scheme, can 
 definitely not be used to emit a function into every 
 referencing CU.
Yes, but in Manu's case the idea is to not need to link with a library.
I know, just saying that the workaround suggested by Stefan ('just make it a dummy template') doesn't work in general - it *can* work, but only when using -allinst in combination with compiling a single object file per compiler invocation, basically excluding all static libraries compiled in one cmdline (dub default, right?). In the linked LDC PR, I'm experimenting with a different template emission scheme, emitting each referenced template stuff into each referencing CU/object file, using linkonce_odr linkage. As a side effect, that would enable the template workaround (or open the door for some other syntactic way of emitting regular non-templated functions the same way), without -allinst or having to take care of how the compiler is invoked. My interest in this wasn't triggered by wanting to enforce particular functions to be emitted into each referencing CU, but by a) the current emission scheme requiring LTO in order to inline suited little templated functions when compiling static libs in one go, b) being able to use linkonce_odr instead of weak_odr linkage for template stuff, meaning potentially less work for the linker, and potentially quite a bit less work for the optimizer. Compiling an optimized dub (a single object file) with LDC using -linkonce-templates reduces the total compile+optimize times by roughly 25%, simply because LLVM inlines most little template stuff and can then discard the unused linkonce_odr functions early in the process, without uselessly optimizing them to death (as it currently cannot discard it, as other libraries/objects might depend on some template instances as mentioned earlier).
Jun 11
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/11/2020 8:42 AM, kinke wrote:
 My interest in this wasn't triggered by wanting to enforce particular
functions 
 to be emitted into each referencing CU, but by
 a) the current emission scheme requiring LTO in order to inline suited little 
 templated functions when compiling static libs in one go,
 b) being able to use linkonce_odr instead of weak_odr linkage for template 
 stuff, meaning potentially less work for the linker, and potentially quite a
bit 
 less work for the optimizer. Compiling an optimized dub (a single object file) 
 with LDC using -linkonce-templates reduces the total compile+optimize times by 
 roughly 25%, simply because LLVM inlines most little template stuff and can
then 
 discard the unused linkonce_odr functions early in the process, without 
 uselessly optimizing them to death (as it currently cannot discard it, as
other 
 libraries/objects might depend on some template instances as mentioned
earlier).
A library consists of two parts: 1. the "header" file 2. the binary to link with, which may be absent Take some care in what goes in 1, what goes in 2, and you'll be fine. For example, consider core.checkedint. The functions in it are all templates so that they can be used with -betterC which doesn't link with druntime. It's a classic "header only" library. It works fine.
Jun 11
next sibling parent Manu <turkeyman gmail.com> writes:
On Fri, Jun 12, 2020 at 8:20 AM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/11/2020 8:42 AM, kinke wrote:
 My interest in this wasn't triggered by wanting to enforce particular
functions
 to be emitted into each referencing CU, but by
 a) the current emission scheme requiring LTO in order to inline suited
little
 templated functions when compiling static libs in one go,
 b) being able to use linkonce_odr instead of weak_odr linkage for
template
 stuff, meaning potentially less work for the linker, and potentially
quite a bit
 less work for the optimizer. Compiling an optimized dub (a single object
file)
 with LDC using -linkonce-templates reduces the total compile+optimize
times by
 roughly 25%, simply because LLVM inlines most little template stuff and
can then
 discard the unused linkonce_odr functions early in the process, without
 uselessly optimizing them to death (as it currently cannot discard it,
as other
 libraries/objects might depend on some template instances as mentioned
earlier). A library consists of two parts: 1. the "header" file 2. the binary to link with, which may be absent Take some care in what goes in 1, what goes in 2, and you'll be fine. For example, consider core.checkedint. The functions in it are all templates so that they can be used with -betterC which doesn't link with druntime. It's a classic "header only" library. It works fine.
We're not talking about templates.
Jun 11
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On Fri, Jun 12, 2020 at 11:51 AM Manu <turkeyman gmail.com> wrote:

 On Fri, Jun 12, 2020 at 8:20 AM Walter Bright via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:

 On 6/11/2020 8:42 AM, kinke wrote:
 My interest in this wasn't triggered by wanting to enforce particular
functions
 to be emitted into each referencing CU, but by
 a) the current emission scheme requiring LTO in order to inline suited
little
 templated functions when compiling static libs in one go,
 b) being able to use linkonce_odr instead of weak_odr linkage for
template
 stuff, meaning potentially less work for the linker, and potentially
quite a bit
 less work for the optimizer. Compiling an optimized dub (a single
object file)
 with LDC using -linkonce-templates reduces the total compile+optimize
times by
 roughly 25%, simply because LLVM inlines most little template stuff and
can then
 discard the unused linkonce_odr functions early in the process, without
 uselessly optimizing them to death (as it currently cannot discard it,
as other
 libraries/objects might depend on some template instances as mentioned
earlier). A library consists of two parts: 1. the "header" file 2. the binary to link with, which may be absent Take some care in what goes in 1, what goes in 2, and you'll be fine. For example, consider core.checkedint. The functions in it are all templates so that they can be used with -betterC which doesn't link with druntime. It's a classic "header only" library. It works fine.
We're not talking about templates.
This is another one of those cases is really bizarre resistance to what seems like a blindingly obvious thing. Can you please explain why you feel opposed to this, and why our existing solution, which is 'weird' by all accounts, and satisfies ZERO of the goals assigned to inline is preferable? The existing implementation is inline is a COMPLETELY useless thing. If you can't see where I'm coming from in this thread, then I suggest we remove it completely. It's existence solves zero problems, and does actual damage to D by existing, because people try it and find that it's broken, and then they're confused and/or lose a little confidence in D. I mean that seriously; if you deny there's a problem and refuse to fix this, then I seriously encourage you to deprecate and remove pragma(inline) from the language. It's worse than nothing.
Jun 11
next sibling parent reply Avrina <avrina12309412342 gmail.com> writes:
On Friday, 12 June 2020 at 01:55:26 UTC, Manu wrote:
 Can you please explain why you feel opposed to this, and why 
 our existing
 solution, which is 'weird' by all accounts, and satisfies ZERO 
 of the goals
 assigned to inline is preferable?
You won't get a direct answer to this. You can't have a debate if he never presents an argument. Or rather in business, you don't have to deal with a problem if you don't acknowledge there is a problem, management 101.
Jun 11
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/11/20 11:01 PM, Avrina wrote:
 On Friday, 12 June 2020 at 01:55:26 UTC, Manu wrote:
 Can you please explain why you feel opposed to this, and why our existing
 solution, which is 'weird' by all accounts, and satisfies ZERO of the 
 goals
 assigned to inline is preferable?
You won't get a direct answer to this. You can't have a debate if he never presents an argument. Or rather in business, you don't have to deal with a problem if you don't acknowledge there is a problem, management 101.
It seems to me the problem lies with the way the problem is formulated.
Jun 12
parent reply Avrina <avrina12309412342 gmail.com> writes:
On Friday, 12 June 2020 at 15:37:16 UTC, Andrei Alexandrescu 
wrote:
 On 6/11/20 11:01 PM, Avrina wrote:
 On Friday, 12 June 2020 at 01:55:26 UTC, Manu wrote:
 Can you please explain why you feel opposed to this, and why 
 our existing
 solution, which is 'weird' by all accounts, and satisfies 
 ZERO of the goals
 assigned to inline is preferable?
You won't get a direct answer to this. You can't have a debate if he never presents an argument. Or rather in business, you don't have to deal with a problem if you don't acknowledge there is a problem, management 101.
It seems to me the problem lies with the way the problem is formulated.
You managed to figure it out :), at least better than Walter. When talking about inline, it seems even you and Walter seem to be focused on the "inline" part. Yes an empty template does what Manu wants, the problem is that templates have their own nasty side effects. int foo(int) { return 1; } bool foo(bool) { return true; } int bar()(int) { return 1; } bool bar()(bool) { return true; } pragma(msg, __traits(getOverloads, __traits(parent, foo), "foo")); // tuple of (foo, foo) pragma(msg, __traits(getOverloads, __traits(parent, bar), "bar")); // empty tuple () It's relevant enough to be a feature that has already existed for quite some time. All Manu wants is to have that same functionality without the template. Maybe a bit different of an implementation but it does almost the same thing. There were also a few other examples of problems provided that were meant with unhelpful "workarounds" such as this: https://forum.dlang.org/post/rbua3s$1s5m$1 digitalmars.com Yes, just don't ever use enums and/or waste your time trying to figure out what is causing the link error and try to figure out a way to just remove it. So helpful.
Jun 12
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/12/20 4:13 PM, Avrina wrote:
 On Friday, 12 June 2020 at 15:37:16 UTC, Andrei Alexandrescu wrote:
 On 6/11/20 11:01 PM, Avrina wrote:
 On Friday, 12 June 2020 at 01:55:26 UTC, Manu wrote:
 Can you please explain why you feel opposed to this, and why our 
 existing
 solution, which is 'weird' by all accounts, and satisfies ZERO of 
 the goals
 assigned to inline is preferable?
You won't get a direct answer to this. You can't have a debate if he never presents an argument. Or rather in business, you don't have to deal with a problem if you don't acknowledge there is a problem, management 101.
It seems to me the problem lies with the way the problem is formulated.
You managed to figure it out :), at least better than Walter. When talking about inline, it seems even you and Walter seem to be focused on the "inline" part. Yes an empty template does what Manu wants, the problem is that templates have their own nasty side effects.    int  foo(int)  { return 1; }    bool foo(bool) { return true; }    int  bar()(int)  { return 1; }    bool bar()(bool) { return true; }    pragma(msg, __traits(getOverloads, __traits(parent, foo), "foo")); // tuple of (foo, foo)    pragma(msg, __traits(getOverloads, __traits(parent, bar), "bar")); // empty tuple () It's relevant enough to be a feature that has already existed for quite some time. All Manu wants is to have that same functionality without the template.
Thanks, that's edifying. That makes sense - have pragma(inline, true) do what a template would, without actually using template instantiation.
 Maybe a bit different of an implementation but it does almost 
 the same thing.
This is important. Again it's been a repeating pattern that a valid request comes with a barrage of red herring details and suggestions for implementation that are often incomplete, unnecessarily complex, difficult to implement, or just plain wrong.
 There were also a few other examples of problems 
 provided that were meant with unhelpful "workarounds" such as this: 
 https://forum.dlang.org/post/rbua3s$1s5m$1 digitalmars.com Yes, just 
 don't ever use enums and/or waste your time trying to figure out what is 
 causing the link error and try to figure out a way to just remove it. So 
 helpful.
I confess I don't understand why the example doesn't work. It should. And the "doctor, it hurts when I do that - then don't do that" back-and-forth is not helping.
Jun 12
prev sibling parent Manu <turkeyman gmail.com> writes:
On Sat, Jun 13, 2020 at 6:15 AM Avrina via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Friday, 12 June 2020 at 15:37:16 UTC, Andrei Alexandrescu
 wrote:
 On 6/11/20 11:01 PM, Avrina wrote:
 On Friday, 12 June 2020 at 01:55:26 UTC, Manu wrote:
 Can you please explain why you feel opposed to this, and why
 our existing
 solution, which is 'weird' by all accounts, and satisfies
 ZERO of the goals
 assigned to inline is preferable?
You won't get a direct answer to this. You can't have a debate if he never presents an argument. Or rather in business, you don't have to deal with a problem if you don't acknowledge there is a problem, management 101.
It seems to me the problem lies with the way the problem is formulated.
You managed to figure it out :), at least better than Walter. When talking about inline, it seems even you and Walter seem to be focused on the "inline" part. Yes an empty template does what Manu wants, the problem is that templates have their own nasty side effects. int foo(int) { return 1; } bool foo(bool) { return true; } int bar()(int) { return 1; } bool bar()(bool) { return true; } pragma(msg, __traits(getOverloads, __traits(parent, foo), "foo")); // tuple of (foo, foo) pragma(msg, __traits(getOverloads, __traits(parent, bar), "bar")); // empty tuple ()
Right, this is but one of an endless sea of edge cases extending from the fact that templates are not functions, they are templates... they may become functions when they are instantiated, but that requires instantiation. Basically the only way that a nullary template function is like a normal function is a direct call: `foo()`, almost anything else you do has sufficient semantic differences that break interactions with generic code. Eg: void foo() {} void bar()() {} // this works (this is the only thing that works): foo(); bar(); // this doesn't: auto x = &foo; auto y = &bar; // this doesn't: static assert(is(typeof(foo) == function)); static assert(is(typeof(bar) == function)); // this doesn't: pragma(msg, __traits(getOverloads, __traits(parent, foo), "foo")); // (foo) pragma(msg, __traits(getOverloads, __traits(parent, bar), "bar")); // () // etc... Dummy templates are not a cool workaround. Sometimes functions are just meant to be functions. What the template hack does show us though, is that DMD is already perfectly capable of implementing `inline`, it's probably a 1-line fix.
Jun 12
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/11/20 9:55 PM, Manu wrote:
 This is another one of those cases is really bizarre resistance to what 
 seems like a blindingly obvious thing.
After perusing this entire thread, I must confess I'm unclear of what the request is. I'm sure it's something reasonable, and that the misunderstanding is mine. Can someone summarize what the matter is and what the motivation behind it is? Also, starting from the pragma definition: https://dlang.org/spec/pragma.html#inline. It's very thin! There's definitely a need to improve it. It would need at least enough detail to describe what the linkage of inline function is.
Jun 12
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/11/2020 6:55 PM, Manu wrote:
     We're not talking about templates. 
 
 This is another one of those cases is really bizarre resistance to what seems 
 like a blindingly obvious thing.
 Can you please explain why you feel opposed to this, and why our existing 
 solution, which is 'weird' by all accounts, and satisfies ZERO of the goals 
 assigned to inline is preferable?
Specifying solutions without explaining what the problem is means I will never understand what you want. For example, I had to keep saying why? why? why? to you over several posts before you finally said what you wanted was a header only library. I show you how you can do that with existing D, along with an existing, working solution. Then you say you're not talking about templates. Hence I still don't know what problem(s) you are having, as I have no idea why templates are no good for you.
 The existing implementation is inline is a COMPLETELY useless thing.
I use it, and it's not useless. I suggest coming up with a complete list of what problems you are trying to solve, not a list of proposed solutions. BTW, the pragma(inline) came out of a conversation you & I had at a DConf some years ago, and we talked about inline and not inlining. I don't recall you ever mentioning issues with linkage, multiple instantiations, etc. I also have problems understanding what you want when things are distributed piecemeal over a large number of n.g. posts. I suggest again: 1. putting each problem into a bugzilla issue 2. keeping a text file of URLs to those issues 3. cut & paste that list as necessary and where necessary N.g. postings are utterly disorganized and ephemeral. Using targeted bugzilla issues is a far better way of organizing specific topics for change.
Jun 12
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Sat, Jun 13, 2020 at 11:31 AM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/11/2020 6:55 PM, Manu wrote:
     We're not talking about templates.

 This is another one of those cases is really bizarre resistance to what
seems
 like a blindingly obvious thing.
 Can you please explain why you feel opposed to this, and why our
existing
 solution, which is 'weird' by all accounts, and satisfies ZERO of the
goals
 assigned to inline is preferable?
Specifying solutions without explaining what the problem is means I will never understand what you want. For example, I had to keep saying why? why? why? to you over several posts before you finally said what you wanted was a header only library.
I felt like my OP was really clear. I identify 3 use cases for inline, and then commented on their level of importance. I've answered your "why"s so many times. 'inline') judgement (C++ calls this `__forceinline` among various others, completely non-standard, but useful) can't (C++ doesn't have this at all. It's useful for preserving features of the calling context) I show you how you can do that with existing D, along with an existing,
 working
 solution.
Templates are not a solution, they're a weak workaround which I'm completely aware of. This thread exists because we don't want that workaround. Templates are not functions, and inserting a template into any generic code where a function is expected leads to a sea of edge cases. Then you say you're not talking about templates.

Because we're not talking about templates. Functions are _functions_, they
have a lot of semantic differences from templates.

Hence I still don't know what problem(s) you are having, as I have no idea
 why
 templates are no good for you.
We don't satisfy any of the 3 key goals in my OP. After reading the various responses in this thread, I could rewrite that post to be clearer, but I really didn't feel it was that unclear in the first place.
 The existing implementation is inline is a COMPLETELY useless thing.
I use it, and it's not useless.
I'm not aware that I've ever wanted inline for any reason other than those 3 points I described. I invited others to add their own use cases if I was missing some. You're welcome to add a use case showing how you'd like to use inline... but it doesn't implement any of my use cases today. I suggest coming up with a complete list of what problems you are trying to
 solve, not a list of proposed solutions.
I have, that's the OP. BTW, the pragma(inline) came out of a conversation you & I had at a DConf
 some
 years ago, and we talked about inline and not inlining. I don't recall you
 ever
 mentioning issues with linkage, multiple instantiations, etc.
Notably, that is a case that C/C++ has absolutely no way to express and it was very interesting to me at the time to have a way to express it. I think the mistake I made at the time was that I held an underlying assumption about what inline does do, and presumed that to be a given. We never discussed what inline does (emit to the calling CU), and I certainly never said that was incorrect functionality and that we should change that behaviour. We didn't discuss it, I presumed it was a given, fundamental to the concept of inline, and never gave it a second thought. It never occurred to me that mechanic could come into question. When I did realise that mechanic was broken though, I mentioned it at a later time, and it's never been fixed. any incarnation of inline must have to be useful. I also have problems understanding what you want when things are
 distributed
 piecemeal over a large number of n.g. posts. I suggest again:

 1. putting each problem into a bugzilla issue
 2. keeping a text file of URLs to those issues
 3. cut & paste that list as necessary and where necessary

 N.g. postings are utterly disorganized and ephemeral. Using targeted
 bugzilla
 issues is a far better way of organizing specific topics for change.
I agree, my bugzilla issues on inline are confused. I'll straighten them out.
Jun 12
parent welkam <wwwelkam gmail.com> writes:
On Saturday, 13 June 2020 at 01:58:55 UTC, Manu wrote:
 On Sat, Jun 13, 2020 at 11:31 AM Walter Bright via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:
 For example, I had to keep saying why? why? why?
I've answered your "why"s so many times. <...> judgement
Oh cmon, Manu. This does not answer why. This is what statement that has "why?" before it. WHAT: Users want to add a hint to override the optimisers judgement WHY: Inlining has a big effect on programs performance and compiler`s heuristics based inlining algorithm doesn't always make the right decision. Manually going through the code base and marking select functions with a tag that tells inliner whether to inline that function or not can improve performance by several percent. In areas where high performance is critical and you have gain a lot by extracting even 1% of performance a tag that controls inliner is absolutely necessary. There fore D needs a way to control which function gets inlined and which is not otherwise D wont be even considered for writing code in high performance computing applications. Now do that for other 2. P.s. Remember (X) and (Y)?
Jun 13
prev sibling parent Manu <turkeyman gmail.com> writes:
On Sat, Jun 13, 2020 at 11:58 AM Manu <turkeyman gmail.com> wrote:

 On Sat, Jun 13, 2020 at 11:31 AM Walter Bright via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:

 I also have problems understanding what you want when things are
 distributed
piecemeal over a large number of n.g. posts. I suggest again:
 1. putting each problem into a bugzilla issue
 2. keeping a text file of URLs to those issues
 3. cut & paste that list as necessary and where necessary

 N.g. postings are utterly disorganized and ephemeral. Using targeted
 bugzilla
 issues is a far better way of organizing specific topics for change.
I agree, my bugzilla issues on inline are confused. I'll straighten them out.
Let's get this right first: https://issues.dlang.org/show_bug.cgi?id=20925 they both depend on that as a baseline requirement.
Jun 12
prev sibling parent Avrina <avrina12309412342 gmail.com> writes:
On Tuesday, 9 June 2020 at 09:29:47 UTC, Walter Bright wrote:
 On 6/8/2020 7:09 AM, Manu wrote:
 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d 
 It's not a hint at all. It's a mechanical tool; it marks 
 symbols with internal linkage, and it also doesn't emit them 
 if it's never referenced.
 The compiler may not choose to ignore that behaviour,
The C/C++ inline semantics revolve around the mechanics of .h files because it doesn't have modules. These reasons are irrelevant for D.
 it's absolutely necessary, and very important.
For .h files, sure. Why for D, though?
The same is true for .di files. There's no option for the compiler to tell the compiler to include the source of a function for the header files in D. So there's no chance for it to inline. I know .di files aren't used much, but they are still supported nonetheless. Note: This is different than the argument Manu is making, but it is as relevant for D as it is for C++ for the compiler to actually be able to inline the source.
Jun 09
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/20 10:09 AM, Manu wrote:
 It's not a hint at all. It's a mechanical tool; it marks symbols with 
 internal linkage, and it also doesn't emit them if it's never referenced.
 The compiler may not choose to ignore that behaviour, it's absolutely 
 necessary, and very important.
Manu, your understanding of inlining is very wrong. Maybe you refer to C's __inline__ (with which I'm not familiar)?
Jun 12
parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Friday, 12 June 2020 at 13:27:23 UTC, Andrei Alexandrescu 
wrote:
 On 6/8/20 10:09 AM, Manu wrote:
 It's not a hint at all. It's a mechanical tool; it marks 
 symbols with internal linkage, and it also doesn't emit them 
 if it's never referenced.
 The compiler may not choose to ignore that behaviour, it's 
 absolutely necessary, and very important.
Manu, your understanding of inlining is very wrong. Maybe you refer to C's __inline__ (with which I'm not familiar)?
Since C99 it's as you've described above. Before that it was implementation defined as it was not part of the language but compiler dependent. The behaviour was very different between gcc 3 and gcc 4, after that it was as you described above. I came up with a complex scheme of macros to be able to handle inlining efficiently, and portably for our project that was (and still is) pure C. The issue with inlining for a project like ours, is that we have a set of libraries and binaries that a linked against these libraries. Some of the apps have to be linked to static versions of the libs, other to .so version of them, this is 3 variants, optimized for deployment, debug with a lot of diagnistics and profiling. This requires that all inline functions have their code emitted exactly once in exactly one library, regardless if the caller inlined it or not. Not as easy as people think.
Jun 12
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/12/20 9:57 AM, Patrick Schluter wrote:
 On Friday, 12 June 2020 at 13:27:23 UTC, Andrei Alexandrescu wrote:
 On 6/8/20 10:09 AM, Manu wrote:
 It's not a hint at all. It's a mechanical tool; it marks symbols with 
 internal linkage, and it also doesn't emit them if it's never 
 referenced.
 The compiler may not choose to ignore that behaviour, it's absolutely 
 necessary, and very important.
Manu, your understanding of inlining is very wrong. Maybe you refer to C's __inline__ (with which I'm not familiar)?
Since C99 it's as you've described above. Before that it was implementation defined as it was not part of the language but compiler dependent. The behaviour was very different between gcc 3 and gcc 4, after that it was as you described above. I came up with a complex scheme of macros to be able to handle inlining efficiently, and portably for our project that was (and still is) pure C. The issue with inlining for a project like ours, is that we have a set of libraries and binaries that a linked against these libraries. Some of the apps have to be linked to static versions of the libs, other to .so version of them, this is 3 variants, optimized for deployment, debug with a lot of diagnistics and profiling. This requires that all inline functions have their code emitted exactly once in exactly one library, regardless if the caller inlined it or not. Not as easy as people think.
Yes, such batteries of macros are the norm in large projects. I'm familiar with those in the HHVM project: https://github.com/facebook/hhvm/blob/a0ca1ffa9a3d690ad57feadf27031886f8eeb2f7/hphp/util/portability.h
Jun 12
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jun 09, 2020 at 12:09:04AM +1000, Manu via Digitalmars-d wrote:
[...]
    I don't want a binary full of code that shouldn't be there. It's
    very important to be able to control what code is in your binaries.
    If it's not referenced, it doesn't exist.
[...] Could you just use LTO for this? LDC's LTO, for example, lets the linker discard unreferenced symbols. T -- A mathematician is a device for turning coffee into theorems. -- P. Erdos
Jun 08
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Monday, 8 June 2020 at 14:45:46 UTC, H. S. Teoh wrote:

 Could you just use LTO for this?  LDC's LTO, for example, lets 
 the linker discard unreferenced symbols.
LTO is a tool that attempts to solve a problem that does not need to exist. This could be said about linkers in general.
Jun 08
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/8/2020 7:54 AM, Stanislav Blinov wrote:
 On Monday, 8 June 2020 at 14:45:46 UTC, H. S. Teoh wrote:
 
 Could you just use LTO for this?  LDC's LTO, for example, lets the linker 
 discard unreferenced symbols.
LTO is a tool that attempts to solve a problem that does not need to exist. This could be said about linkers in general.
In the separate compilation model, compilers know nothing about what other compilation units may or may not call. This is why elision of unreferenced symbols belongs in the linker.
Jun 09
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Tuesday, 9 June 2020 at 10:26:45 UTC, Walter Bright wrote:
 On 6/8/2020 7:54 AM, Stanislav Blinov wrote:
 On Monday, 8 June 2020 at 14:45:46 UTC, H. S. Teoh wrote:
 
 Could you just use LTO for this?  LDC's LTO, for example, 
 lets the linker discard unreferenced symbols.
LTO is a tool that attempts to solve a problem that does not need to exist. This could be said about linkers in general.
In the separate compilation model, compilers know nothing about what other compilation units may or may not call. This is why elision of unreferenced symbols belongs in the linker.
That is no reason for another [instance of the same] compiler not to have access to symbol table, and some form of source representation, at compile time, obviating the need for (much of) linker's work. LTO does what compilers ought to be doing.
Jun 09
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/9/2020 4:30 AM, Stanislav Blinov wrote:
 That is no reason for another [instance of the same] compiler not to have
access 
 to symbol table,
Since the linker has this information already, there's no reason to not do the job in the linker.
Jun 09
next sibling parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Tuesday, 9 June 2020 at 21:26:38 UTC, Walter Bright wrote:
 On 6/9/2020 4:30 AM, Stanislav Blinov wrote:
 That is no reason for another [instance of the same] compiler 
 not to have access to symbol table,
Since the linker has this information already, there's no reason to not do the job in the linker.
So, "linkers have to do it because linkers have to do it". Let's just stick to outdated tech forever.
Jun 09
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/9/2020 2:54 PM, Stanislav Blinov wrote:
 So, "linkers have to do it because linkers have to do it". Let's just stick to 
 outdated tech forever.
I have seriously considered building the linker into DMD, but don't have the resources to do it. If you want to do it, feel free to.
Jun 09
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On Wed, Jun 10, 2020 at 7:30 AM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/9/2020 4:30 AM, Stanislav Blinov wrote:
 That is no reason for another [instance of the same] compiler not to
have access
 to symbol table,
Since the linker has this information already, there's no reason to not do the job in the linker.
Redundant work, undesired object bloat, and various potential link issues/errors.
Jun 09
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/9/2020 4:31 PM, Manu wrote:
 Redundant work,
It isn't redundant work to do it in the linker. It is redundant to do half of the linker's job in the compiler, throw away the other half of it, and re-do it in the linker.
 undesired object bloat,
Not a problem since we moved on from floppy disks.
 and various potential link issues/errors.
I suppose that fits in with the common notion that the linker is black magic full of trolls and dragons. It isn't, it's a boringly simple program. Although it is true that probably only 1 in 100 programmers can explain what a linker does. (Most of the complexity of linkers is not inherent, it is the result of file formats designed by hamsters and workarounds for compiler and loader bugs. You have to pity the linker developer - there's no glamor, nobody understands what they do, nobody praises them, they just dump on them with "what does 'undefined symbol mean' - must be a bug in the linker" questions.)
Jun 09
next sibling parent reply Avrina <avrina12309412342 gmail.com> writes:
On Wednesday, 10 June 2020 at 01:48:20 UTC, Walter Bright wrote:
 On 6/9/2020 4:31 PM, Manu wrote:
 Redundant work,
It isn't redundant work to do it in the linker. It is redundant to do half of the linker's job in the compiler, throw away the other half of it, and re-do it in the linker.
 undesired object bloat,
Not a problem since we moved on from floppy disks.
People complain about Visual Studio's download size, most of the download is library files. I imagine it is also a problem for high performance computing as well. So, yes still a problem.
 and various potential link issues/errors.
I suppose that fits in with the common notion that the linker is black magic full of trolls and dragons. It isn't, it's a boringly simple program. Although it is true that probably only 1 in 100 programmers can explain what a linker does. (Most of the complexity of linkers is not inherent, it is the result of file formats designed by hamsters and workarounds for compiler and loader bugs. You have to pity the linker developer - there's no glamor, nobody understands what they do, nobody praises them, they just dump on them with "what does 'undefined symbol mean' - must be a bug in the linker" questions.)
I think maybe new programmers think that, ones that come from languages like python or otherwise where they don't have to deal with the abomination known as a linker. That's the thing, the way the workflow is setup the problem usually isn't with the linker which makes it really annoying to try and find where the problem actually is. Especially if it's a compiler bug :).
Jun 10
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 10 June 2020 at 18:03:21 UTC, Avrina wrote:
 [snip]

 I imagine it is also a problem for high performance computing 
 as well. So, yes still a problem.
 [snip]
For HPC, it would only be an issue during development (where you can probably do a lot of the work on your own machine), rather than during production.
Jun 10
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/10/2020 11:03 AM, Avrina wrote:
 I think maybe new programmers think that,
I've seen near total linker puzzlement from people with long experience with C and C++. I don't know how this happens, just that it is commonplace.
 ones that come from languages like 
 python or otherwise where they don't have to deal with the abomination known
as 
 a linker.
"Abomination" is a misunderstanding of what a linker is and does.
 That's the  thing, the way the workflow is setup the problem usually 
 isn't with the linker which makes it really annoying to try and find where the 
 problem actually is.
Understanding the simple process of linking is key to quickly resolving whether a problem is linker or compiler. I wrote a "how linkers work" chapter for Kevlin Henney's book "97 Things Every Programmer Should Know": https://www.amazon.com/Things-Every-Programmer-Should-Know/dp/0596809484/ No, I don't get any payments from that book. I was talking to Kevlin once grousing about nobody understanding linkers, and he invited me to write a chapter :-)
Jun 10
parent reply matheus <matheus gmail.com> writes:
On Wednesday, 10 June 2020 at 22:29:59 UTC, Walter Bright wrote:
 ...
 I wrote a "how linkers work" chapter for Kevlin Henney's book 
 "97 Things Every Programmer Should Know":

 https://www.amazon.com/Things-Every-Programmer-Should-Know/dp/0596809484/

 No, I don't get any payments from that book. I was talking to 
 Kevlin once grousing about nobody understanding linkers, and he 
 invited me to write a chapter :-)
Interesting: https://github.com/97-things/97-things-every-programmer-should-know/blob/master/en/thing_53/README.md Matheus.
Jun 10
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/10/2020 3:47 PM, matheus wrote:
 Interesting: 
 https://github.com/97-things/97-things-every-programmer-should-know/blob/master/e
/thing_53/README.md 
Haha, I didn't know it was free online! Thanks for the link, and enjoy!
Jun 10
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 10 June 2020 at 01:48:20 UTC, Walter Bright wrote:
 On 6/9/2020 4:31 PM, Manu wrote:
 [...]
It isn't redundant work to do it in the linker. It is redundant to do half of the linker's job in the compiler, throw away the other half of it, and re-do it in the linker. [...]
we have a project that compiles with "-allinst -allinst -allinst". To avoid such errors. (the repetition comes from dub appending to the dflags)
Jun 10
prev sibling parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Wednesday, 10 June 2020 at 01:48:20 UTC, Walter Bright wrote:
 On 6/9/2020 4:31 PM, Manu wrote:
 Redundant work,
It isn't redundant work to do it in the linker. It is redundant to do half of the linker's job in the compiler, throw away the other half of it, and re-do it in the linker.
It is redundant to do the job in the compiler, throw *that* away and then shrug it off with "meh, the linker will take care of it". Which is precisely what compilers are doing. Which is precisely what you suggested to be doing in the issue report. ...And of course, the linker doesn't take care of it, which is why LTO came to be - to solve a problem that doesn't need to exist. And DMD does not need a built-in linker.
Jun 10
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/10/2020 12:04 PM, Stanislav Blinov wrote:
 It is redundant to do the job in the compiler, throw *that* away and then
shrug 
 it off with "meh, the linker will take care of it". Which is precisely what 
 compilers are doing.
All I can say is you're misunderstanding what linkers do.
Jun 10
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, June 8, 2020 8:09:04 AM MDT Manu via Digitalmars-d wrote:
 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d <
 C/C++ inline has always been a hint to the compiler, not a command.
It's not a hint at all. It's a mechanical tool; it marks symbols with internal linkage, and it also doesn't emit them if it's never referenced. The compiler may not choose to ignore that behaviour, it's absolutely necessary, and very important.
It is my understanding that in C++, inline is a hint to the compiler with regards to whether a particular function call is actually inlined. There are other effects that inline has which aren't just a hint, but the behavior of actually inlining the function is a hint, not a command. Of course, in D, it's even weirder, because it's been "give me an error if the compiler fails to inline it," and I don't know that it even functions as a hint at that point as opposed to just a way to find out that the compiler didn't do what you wanted. - Jonathan M Davis
Jun 08
parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 8 June 2020 at 23:19:55 UTC, Jonathan M Davis wrote:
 On Monday, June 8, 2020 8:09:04 AM MDT Manu via Digitalmars-d 
 wrote:
 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d 
 <
 C/C++ inline has always been a hint to the compiler, not a 
 command.
It's not a hint at all. It's a mechanical tool; it marks symbols with internal linkage, and it also doesn't emit them if it's never referenced. The compiler may not choose to ignore that behaviour, it's absolutely necessary, and very important.
It is my understanding that in C++, inline is a hint to the compiler with regards to whether a particular function call is actually inlined.
That's a common misconception, and one that exists due to that being its original intended purpose. But nowawadays? Nope: https://en.cppreference.com/w/cpp/language/inline
Jun 09
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/9/2020 4:31 AM, Atila Neves wrote:
 On Monday, 8 June 2020 at 23:19:55 UTC, Jonathan M Davis wrote:
 On Monday, June 8, 2020 8:09:04 AM MDT Manu via Digitalmars-d wrote:
 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d <
 C/C++ inline has always been a hint to the compiler, not a > command.
It's not a hint at all. It's a mechanical tool; it marks symbols with internal linkage, and it also doesn't emit them if it's never referenced. The compiler may not choose to ignore that behaviour, it's absolutely necessary, and very important.
It is my understanding that in C++, inline is a hint to the compiler with regards to whether a particular function call is actually inlined.
That's a common misconception, and one that exists due to that being its original intended purpose. But nowawadays? Nope: https://en.cppreference.com/w/cpp/language/inline
"Since this meaning of the keyword inline is non-binding, compilers are free to use inline substitution for any function that's not marked inline, and are free to generate function calls to any function marked inline. Those optimization choices do not change the rules regarding multiple definitions and shared statics listed above." I.e. it is a hint.
Jun 09
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Wed, Jun 10, 2020 at 7:15 AM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/9/2020 4:31 AM, Atila Neves wrote:
 On Monday, 8 June 2020 at 23:19:55 UTC, Jonathan M Davis wrote:
 On Monday, June 8, 2020 8:09:04 AM MDT Manu via Digitalmars-d wrote:
 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d <
 C/C++ inline has always been a hint to the compiler, not a > command.
It's not a hint at all. It's a mechanical tool; it marks symbols with internal linkage, and it also doesn't emit them if it's never
referenced. The
 compiler may not choose to ignore that behaviour, it's absolutely
necessary,
 and very important.
It is my understanding that in C++, inline is a hint to the compiler
with
 regards to whether a particular function call is actually inlined.
That's a common misconception, and one that exists due to that being its original intended purpose. But nowawadays? Nope: https://en.cppreference.com/w/cpp/language/inline
"Since this meaning of the keyword inline is non-binding, compilers are free to use inline substitution for any function that's not marked inline, and are free to generate function calls to any function marked inline. Those optimization choices do not change the rules regarding multiple definitions and shared statics listed above." I.e. it is a hint.
It's like you skipped over ALL OF THE OTHER TEXT, where it details numerous precise behavioural requirements :/
Jun 09
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/9/2020 4:30 PM, Manu wrote:
 It's like you skipped over ALL OF THE OTHER TEXT, where it details numerous 
 precise behavioural requirements :/
As I replied elsewhere to you, those other requirements are there to deal with .h files, obsolete linkers, and ancient object file formats.
 "Because the meaning of the keyword inline for functions came to mean 
"multiple definitions are permitted" rather than "inlining is preferred", that meaning was extended to variables." It's a hack for the the same problem - .h files, etc. There's no reason for that in D.
Jun 09
prev sibling parent Manu <turkeyman gmail.com> writes:
On Wed, Jun 10, 2020 at 9:30 AM Manu <turkeyman gmail.com> wrote:

 On Wed, Jun 10, 2020 at 7:15 AM Walter Bright via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:

 On 6/9/2020 4:31 AM, Atila Neves wrote:
 On Monday, 8 June 2020 at 23:19:55 UTC, Jonathan M Davis wrote:
 On Monday, June 8, 2020 8:09:04 AM MDT Manu via Digitalmars-d wrote:
 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d <
 C/C++ inline has always been a hint to the compiler, not a >
command.
 It's not a hint at all. It's a mechanical tool; it marks symbols with
 internal linkage, and it also doesn't emit them if it's never
referenced. The
 compiler may not choose to ignore that behaviour, it's absolutely
necessary,
 and very important.
It is my understanding that in C++, inline is a hint to the compiler
with
 regards to whether a particular function call is actually inlined.
That's a common misconception, and one that exists due to that being
its
 original intended purpose. But nowawadays? Nope:

 https://en.cppreference.com/w/cpp/language/inline
"Since this meaning of the keyword inline is non-binding, compilers are free to use inline substitution for any function that's not marked inline, and are free to generate function calls to any function marked inline. Those optimization choices do not change the rules regarding multiple definitions and shared statics listed above." I.e. it is a hint.
It's like you skipped over ALL OF THE OTHER TEXT, where it details numerous precise behavioural requirements :/
The the sentence that immediately follows your quote is: "Because the meaning of the keyword inline for functions came to mean "multiple definitions are permitted" rather than "inlining is preferred", that meaning was extended to variables."
Jun 09
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Tue, Jun 9, 2020 at 12:46 AM H. S. Teoh via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Tue, Jun 09, 2020 at 12:09:04AM +1000, Manu via Digitalmars-d wrote:
 [...]
    I don't want a binary full of code that shouldn't be there. It's
    very important to be able to control what code is in your binaries.
    If it's not referenced, it doesn't exist.
[...] Could you just use LTO for this? LDC's LTO, for example, lets the linker discard unreferenced symbols.
I think the compiler should specify behaviour that's universally compatible, and doesn't depend on special tooling to integrate with existing workflows.
Jun 08
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Tue, Jun 9, 2020 at 9:20 AM Jonathan M Davis via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Monday, June 8, 2020 8:09:04 AM MDT Manu via Digitalmars-d wrote:
 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d <
 C/C++ inline has always been a hint to the compiler, not a command.
It's not a hint at all. It's a mechanical tool; it marks symbols with internal linkage, and it also doesn't emit them if it's never referenced. The compiler may not choose to ignore that behaviour, it's absolutely necessary, and very important.
It is my understanding that in C++, inline is a hint to the compiler with regards to whether a particular function call is actually inlined. There are other effects that inline has which aren't just a hint, but the behavior of actually inlining the function is a hint, not a command.
I'm not sure it's even a hint. That behaviour is basically irrelevant. The 'other effects' you describe are what inline does in C++, and we need that too.
Jun 08
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/8/2020 3:19 AM, Walter Bright wrote:
 The PR I have on this makes it an informational warning. You can choose to be 
 notified if inlining fails.
https://github.com/dlang/dmd/pull/11236
Jun 10
prev sibling next sibling parent reply kinke <kinke gmx.net> writes:
On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:
 D's inline today doesn't do any of those things. It doesn't 
 implement a mechanic that I have ever wanted or known a use for.
We've had this discussion a while back. As of the latest LDC beta, `pragma(inline, true)` functions are now properly emitted into each referencing CU, and will be inlined in most cases (`alwaysinline` LLVM function attribute), even at -O0. In modules other than the owning one, the function 'copy' is emitted as `available_externally`, meaning that it's *only* available for inlining at the IR level, it will never make it to the assembler and object file. In its owning module, the function is emitted as a regular function, as a fallback for non-inlined cases (and for when people take its address etc.). Our opinions diverge wrt. whether that's a problem - to me it's clearly no big deal, as the function is a) most likely small, and b) subject to linker stripping if unreferenced. Wrt. control of which CUs contain which functions, that's totally out of hand anyway due to the way templates are emitted.
Jun 08
parent reply Manu <turkeyman gmail.com> writes:
On Mon, Jun 8, 2020 at 8:30 PM kinke via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:
 D's inline today doesn't do any of those things. It doesn't
 implement a mechanic that I have ever wanted or known a use for.
We've had this discussion a while back. As of the latest LDC beta, `pragma(inline, true)` functions are now properly emitted into each referencing CU, and will be inlined in most cases (`alwaysinline` LLVM function attribute), even at -O0.
Yes, and thank you; it's always nice that we can fix broken things in LDC. But I think for inline it's actually quite important that the spec is useful, and that all compilers do the same thing; otherwise DMD experiences link errors that LDC properly avoids. In modules other than the owning one, the function 'copy' is
 emitted as `available_externally`, meaning that it's *only*
 available for inlining at the IR level, it will never make it to
 the assembler and object file.
Yeah I always forget the proper names for all the linkage flags that symbols can have. This approach is fine from a linkage point of view (there shouldn't be errors), but I'd really like to not see the symbol in the owning module if it's never called locally. It's a shame to have a ton of noise in a binary when it's unnecessary. Most D software is full of bloaty junk, but in the cases where I care about this, it's actually a very tightly controlled binary ecosystem, which binds between multiple languages, and only exposes necessary stuff at the ABI boundary. I've also seen cases where the function is not referenced in the owning module, but it does not get stripped by the linker for whatever reason, and then if the inline function has a call to an extern symbol that you don't link, you'll get link errors, even though there's no calls. By emitting it even when it's never been referenced, we're just inviting link errors and inconsistent behaviour between different linkers. We can trivially avoid that risk. In its owning module, the function is emitted as a regular
 function, as a fallback for non-inlined cases (and for when
 people take its address etc.). Our opinions diverge wrt. whether
 that's a problem - to me it's clearly no big deal, as the
 function is a) most likely small, and b) subject to linker
 stripping if unreferenced.
a) most likely small, but still not nothing; the symbol table is public ABI material, and in some projects I've worked on, the symbol table is carefully curated. b) linker stripped is not reliable. We are unnecessarily inviting issues in some cases, and there's just no reason for that. If we're confident that link stripping is 100% reliable when a symbol is not referenced, then I have no complaint here. Can you show what case a hard-symbol in the owning CU solves? Non-inlined cases will still find it locally if it has internal linkage (or whatever that link flag is called). I think it's the same flag that `static` (or `inline`) in C++ specifies right? Wrt. control of which CUs contain which functions, that's totally
 out of hand anyway due to the way templates are emitted.
I'm not sure what you mean. Templates work correctly; template instances are only generated and emit to the calling CU.
Jun 08
next sibling parent kinke <kinke gmx.net> writes:
On Monday, 8 June 2020 at 14:22:50 UTC, Manu wrote:
 Templates work correctly; template instances are only generated 
 and emit to the calling CU.
Nope - they *might* be emitted into the instantiating module, but very likely aren't. Just try compiling a static lib with multiple modules instantiating equivalent templates. See https://github.com/ldc-developers/ldc/pull/3422#issuecomment-625945508 for an example.
Jun 08
prev sibling parent reply kinke <noone nowhere.com> writes:
On Monday, 8 June 2020 at 14:22:50 UTC, Manu wrote:
 On Mon, Jun 8, 2020 at 8:30 PM kinke via Digitalmars-d <
 In its owning module, the function is emitted as a regular
 function, as a fallback for non-inlined cases (and for when 
 people take its address etc.). Our opinions diverge wrt. 
 whether that's a problem - to me it's clearly no big deal, as 
 the function is a) most likely small, and b) subject to linker 
 stripping if unreferenced.
a) most likely small, but still not nothing; the symbol table is public ABI material, and in some projects I've worked on, the symbol table is carefully curated. b) linker stripped is not reliable. We are unnecessarily inviting issues in some cases, and there's just no reason for that. If we're confident that link stripping is 100% reliable when a symbol is not referenced, then I have no complaint here. Can you show what case a hard-symbol in the owning CU solves? Non-inlined cases will still find it locally if it has internal linkage (or whatever that link flag is called). I think it's the same flag that `static` (or `inline`) in C++ specifies right?
Internal linkage (C(++) static), AFAIK, means that you may end up with multiple identical functions in the final linked binary. Some linkers may be able to still fold them, such as the MS linker and lld with /OPT:ICF (identical COMDAT folding). Linkonce_odr linkage (C++ templates and `inline` functions) on the other hand means that you end up with either 0 or 1 function in the final binary, even when multiple object files define it; AFAIK, without having to rely on /OPT:REF or --gc-sections etc. At least for LLVM, linkonce_odr additionally means that the definition is discarded early if unused and might not make it to the object file at all. Weak_odr linkage (current D templates) is similar to linkonce_odr but not 'officially' discardable (if unreferenced) for the final binary (and thus always making it to the object file). This can be overridden by /OPT:REF and --gc-sections etc., the success of which also depends on symbol visibility and/or --export-dynamic etc.
Jun 11
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Fri, Jun 12, 2020 at 3:40 AM kinke via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Monday, 8 June 2020 at 14:22:50 UTC, Manu wrote:
 On Mon, Jun 8, 2020 at 8:30 PM kinke via Digitalmars-d <
 In its owning module, the function is emitted as a regular
 function, as a fallback for non-inlined cases (and for when
 people take its address etc.). Our opinions diverge wrt.
 whether that's a problem - to me it's clearly no big deal, as
 the function is a) most likely small, and b) subject to linker
 stripping if unreferenced.
a) most likely small, but still not nothing; the symbol table is public ABI material, and in some projects I've worked on, the symbol table is carefully curated. b) linker stripped is not reliable. We are unnecessarily inviting issues in some cases, and there's just no reason for that. If we're confident that link stripping is 100% reliable when a symbol is not referenced, then I have no complaint here. Can you show what case a hard-symbol in the owning CU solves? Non-inlined cases will still find it locally if it has internal linkage (or whatever that link flag is called). I think it's the same flag that `static` (or `inline`) in C++ specifies right?
Internal linkage (C(++) static), AFAIK, means that you may end up with multiple identical functions in the final linked binary. Some linkers may be able to still fold them, such as the MS linker and lld with /OPT:ICF (identical COMDAT folding).
Actually, I might be wrong about what the link flag is called. I'm fairly sure C++ uses the link flag that LLVM calls "choose one". It's in the C++ spec that all inlines collapse to the same one. We should be using the same flag. Linkonce_odr linkage (C++ templates and `inline` functions) on
 the other hand means that you end up with either 0 or 1 function
 in the final binary, even when multiple object files define it;
 AFAIK, without having to rely on /OPT:REF or --gc-sections etc.
 At least for LLVM, linkonce_odr additionally means that the
 definition is discarded early if unused and might not make it to
 the object file at all.

 Weak_odr linkage (current D templates) is similar to linkonce_odr
 but not 'officially' discardable (if unreferenced) for the final
 binary (and thus always making it to the object file). This can
 be overridden by /OPT:REF and --gc-sections etc., the success of
 which also depends on symbol visibility and/or --export-dynamic
 etc.
Jun 11
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/11/2020 6:51 PM, Manu wrote:
 I'm fairly sure C++ uses the link flag that LLVM calls "choose one".
 It's in the C++ spec that all inlines collapse to the same one. We should be 
 using the same flag.
D totally relies on that behavior.
Jun 12
parent reply Manu <turkeyman gmail.com> writes:
On Sat, Jun 13, 2020 at 11:00 AM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/11/2020 6:51 PM, Manu wrote:
 I'm fairly sure C++ uses the link flag that LLVM calls "choose one".
 It's in the C++ spec that all inlines collapse to the same one. We
should be
 using the same flag.
D totally relies on that behavior.
So emit the function to the CU where it's called and we're done here!
Jun 12
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/12/2020 6:16 PM, Manu wrote:
 On Sat, Jun 13, 2020 at 11:00 AM Walter Bright via Digitalmars-d 
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
 
     On 6/11/2020 6:51 PM, Manu wrote:
      > I'm fairly sure C++ uses the link flag that LLVM calls "choose one".
      > It's in the C++ spec that all inlines collapse to the same one. We
should be
      > using the same flag.
 
     D totally relies on that behavior.
 
 
   So emit the function to the CU where it's called and we're done here!
Next bug report: D takes too long to compile because it's recompiling code it has already compiled over and over! Yes, this has been a bug report in the past. We fixed it. Besides, I replied to Andrei in another message how you can resolve this - just put the 'header' file on the command line to dmd.
Jun 13
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/11/2020 10:39 AM, kinke wrote:
 Internal linkage (C(++) static), AFAIK, means that you may end up with
multiple 
 identical functions in the final linked binary. Some linkers may be able to 
 still fold them, such as the MS linker and lld with /OPT:ICF (identical COMDAT 
 folding).
 
 Linkonce_odr linkage (C++ templates and `inline` functions) on the other hand 
 means that you end up with either 0 or 1 function in the final binary, even
when 
 multiple object files define it; AFAIK, without having to rely on /OPT:REF or 
 --gc-sections etc. At least for LLVM, linkonce_odr additionally means that the 
 definition is discarded early if unused and might not make it to the object
file 
 at all.
 
 Weak_odr linkage (current D templates) is similar to linkonce_odr but not 
 'officially' discardable (if unreferenced) for the final binary (and thus
always 
 making it to the object file). This can be overridden by /OPT:REF and 
 --gc-sections etc., the success of which also depends on symbol visibility 
 and/or --export-dynamic etc.
gc-sections isn't about merging multiple definitions, it's about discarding unreferenced sections.
Jun 12
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/7/2020 11:14 PM, Manu wrote:
 [...]
Please provide URLs to the bugzilla issue(s) you have on this topic. As I've mentioned before, it's easier if you keep a text file of the bugzilla issues that are important to you, so you can copy/paste them where appropriate. Otherwise, both of our times are wasted with "did you file a bugzilla issue for this?"
Jun 08
prev sibling next sibling parent Dukc <ajieskola gmail.com> writes:
On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:
 1. I only want the function to be present in the CALLING 
 binary. I do not want an inline function present in the local 
 binary where it was defined (unless it was called internally). 
 I do not want a linker to see the inline function symbols and 
 be able to link to them externally. [This is about linkage and 
 controlling the binary or distribution environment]
I think this should be controlled by the visibility attributes, not `pragma(inline)`. If a function is `private`, there's no reason why the linker should see it at all. If it's `public`, it should be seen when linking the internal object files together, but not when linking a precompiled binary. `export` should always be visible.
 2. I am unhappy that the optimiser chose to not inline a 
 function call, and I want to override that judgement. [This is 
 about micro-optimisation]
This is what `pragma(inline)` should be for. For some extent at least, it also is. If you compile a simple example with `ldc2 -O1`, it'll inline the function if you use the pragma but not otherwise. `ldc2 -O2` will inline it regardless and DMD will never do it IIRC, at least not without the `-inline` switch. I agree that it should.
 3. I want to treat the function like an AST macro; I want the 
 function inserted at the callsite, and I want to have total 
 confidence in this mechanic. [This is about articulate 
 mechanical control over code-gen; ie, I know necessary facts 
 about the execution context/callstack that I expect to maintain]
Sounds like a case for mixins, either type of them. But it'd be better if the compiler would only ignore `pragma(inline, true)` if either linking externally or avoiding recursion. There's just no reason to ignore it otherwise. This is an implementation problem, not a spec problem IMO.
Jun 08
prev sibling next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/8/20 2:14 AM, Manu wrote:
 3. I want to treat the function like an AST macro; I want the function 
 inserted at the callsite, and I want to have total confidence in this 
 mechanic. [This is about articulate mechanical control over code-gen; 
 ie, I know necessary facts about the execution context/callstack that I 
 expect to maintain]
This is my main use case. Not because I want AST macros (not sure I'd agree that AST macro is the right description), but because I write a lot of wrapping functions which are like "parameter modifiers". iopipe is full of them for instance. Being able to inject a few statement before/after a call without having to dispatch to a separate function first is very useful for a wrapper. It's what the optimizer should do anyway, but making it part of the API makes a declaration that the function is a wrapper and should be treated that way. Unfortunately, D does not implement this feature in a way that I prefer: https://issues.dlang.org/show_bug.cgi?id=15671 -Steve
Jun 08
prev sibling next sibling parent 9il <ilyayaroshenko gmail.com> writes:
On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:
 1. I only want the function to be present in the CALLING 
 binary. I do not want an inline function present in the local 
 binary where it was defined (unless it was called internally). 
 I do not want a linker to see the inline function symbols and 
 be able to link to them externally. [This is about linkage and 
 controlling the binary or distribution environment]
Yes!
 2. I am unhappy that the optimiser chose to not inline a 
 function call, and I want to override that judgement. [This is 
 about micro-optimisation]
Yes!
 3. I want to treat the function like an AST macro; I want the 
 function inserted at the callsite, and I want to have total 
 confidence in this mechanic. [This is about articulate 
 mechanical control over code-gen; ie, I know necessary facts 
 about the execution context/callstack that I expect to maintain]
Yes!!!
 I think these are the 3 broad categories of behaviour I have 
 ever wanted
 control over.
The same for me. I have the same experience. Moreover, non-AST inlining has the worst optimization abilities comparing with AST. Even if a function is inlined it is often inlined badly ignoring some optimization attributes, local SIMD and FMA instructions, and better loop unrolling patterns (better doesn't mean larger). AST-like inlining is a critical and killer feature.
Jun 08
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/20 2:14 AM, Manu wrote:
 In C/C++, inline says that a function will be emit to the binary only 
 when it is called, and the function is marked with internal linkage (it 
 is not visible to the linker from the symbol table)
By my recollection this is not the case for C++, at all. * "inline" does NOT change a function's linkage in C++. You may have inline functions with internal linkage (static inline) or (default) external linkage. This is important because e.g. defining a static variable in an extern inline function will have the same address in all calls to the function. * "inline" in C++ is entirely advisory, meaning a conforming implementation is entirely free to simply ignore it, save it for duplicate symbols (which are ignored for inline functions and cause errors for non-inline functions). * In fact, all major C++ implementations routinely ignore "inline" in favor of their own heuristics, which are usually better. However, they are not always better, and that in turn has generated demands for an "I really mean inline" directive. Compiler implementers responded to that demand with a trickle of additional nonstandard language features: - MSVC has __inline, __forceinline, and __declspec(noinline): https://docs.microsoft.com/en-us/cpp/cpp/inline-functions-cpp?view=vs-2019. As a funny aside, __inline and __forceinline sometimes do not inline, and __declspec(noinline) functions are sometimes inlined. - g++ and clang have __inline__, __attribute__((always_inline)), and __attribute__ ((noinline)): https://stackoverflow.com/questions/8381293/how-do-i-force-gcc-t -inline-a-function. Needless to say, __inline__ and __attribute__((always_inline)) do not always inline, and __attribute__ ((noinline)) does not always prevent inlining. If the above is correct, a D language feature dedicated to inlining should do the following: * Always emit the function body in the .di file if ever asked to generate it. * Never complain about duplicate symbols if an inline function has duplicate definitions. Then compilers can decide on specific inlining strategies using the language feature as a hint. D does ALL of the above already for templates, so this is not a difficult feature to implement. In fact we use this technique (e.g. in druntime) by spelling inline as "()". Consider: int func(int) { ... body ... } Let's make this inline: int func()(int) { ... body ... } Done. In D there's one additional implication of body availability - the function is eligible for CTFE. I think any adjustment to the extant inline pragma needs to make this a top-level consideration.
Jun 12
next sibling parent reply Johan <j j.nl> writes:
On Friday, 12 June 2020 at 13:22:35 UTC, Andrei Alexandrescu 
wrote:
 On 6/8/20 2:14 AM, Manu wrote:
 In C/C++, inline says that a function will be emit to the 
 binary only when it is called, and the function is marked with 
 internal linkage (it is not visible to the linker from the 
 symbol table)
By my recollection this is not the case for C++, at all. * "inline" does NOT change a function's linkage in C++. You may have inline functions with internal linkage (static inline) or (default) external linkage. This is important because e.g. defining a static variable in an extern inline function will have the same address in all calls to the function.
I believe Manu tried to explain that `inline` in C++ really only affects how the linker must treat the symbol, and it is best to remember that it does nothing at all concerning "inlining" (putting function body inside another). `inline` is really a misnomer. The "linkage" effect of `inline` is required for C++ to work with #include files. There is some room for confusion because, depending on who is talking, the word "linkage" can have more nuances than just internal/external. I personally take the broad stance of "linkage" including anything the linker can do with the symbol. Without `inline`, you would get multiple definition linker errors for functions defined in a header file when linking together two translation units that both included that header file. Instead of normal function linkage that forbids multiple definition, `inline` changes linkage to merge multiple definitions into one. In addition, emission of the function must happen in any translation unit that references it (calling or address taken), and thus that translation unit must also define it (in contrast to just declaring it). I do not think `inline` forbids emission if the function is not referenced. Note that C++17 added `inline` for variables too, because without it you run into the same problem of multiple definitions in e.g. header-only libraries. Obviously that has nothing to do with "inlining" the callee's body; it is purely a linker directive to merge the symbols which is exactly the functionality it provides for functions. In D, this is actually not a problem: multiple definitions of symbols are merged already. You'll never see a multiple definition error with D (also not with extern(C) functions). The wikipedia article seems fairly complete on all the intricacies of `inline`: https://en.wikipedia.org/wiki/Inline_function And the cppreference text is also pretty clear https://en.cppreference.com/w/cpp/language/inline . -Johan
Jun 12
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/12/20 2:08 PM, Johan wrote:
 On Friday, 12 June 2020 at 13:22:35 UTC, Andrei Alexandrescu wrote:
 On 6/8/20 2:14 AM, Manu wrote:
 In C/C++, inline says that a function will be emit to the binary only 
 when it is called, and the function is marked with internal linkage 
 (it is not visible to the linker from the symbol table)
By my recollection this is not the case for C++, at all. * "inline" does NOT change a function's linkage in C++. You may have inline functions with internal linkage (static inline) or (default) external linkage. This is important because e.g. defining a static variable in an extern inline function will have the same address in all calls to the function.
I believe Manu tried to explain that `inline` in C++ really only affects how the linker must treat the symbol, and it is best to remember that it does nothing at all concerning "inlining" (putting function body inside another).
Thank you. That's not at all what he wrote. AT ALL. It's what I wrote. I'm emphasizing this because it has been a recurring problem: a legitimate problem with an explanation lost in translation.
 `inline` is really a misnomer.
 The "linkage" effect of `inline` is required for C++ to work with 
 #include files. There is some room for confusion because, depending on 
 who is talking, the word "linkage" can have more nuances than just 
 internal/external.
It helps to use terminology that is well defined and standardized. That's where simply using the meaning defined and consistently used in the C++ standard document can be of great help.
 I personally take the broad stance of "linkage" 
 including anything the linker can do with the symbol.
Makes sense. It's really nice to just use http://eel.is/c++draft/basic.link, which ensures we all understand one another.
 Without `inline`, you would get multiple definition linker errors for 
 functions defined in a header file when linking together two translation 
 units that both included that header file. Instead of normal function 
 linkage that forbids multiple definition, `inline` changes linkage to 
 merge multiple definitions into one.
Not sure about that part - if linkage was static by means of using the "static" keyword, multiple definitions may not be merged. (I may be wrong, please correct me.) Consider: static inline int fun() { static int x; return ++x; } In C++, each translation unit containing a definition of fun() will have a distinct address for x. I don't see how the bodies of those functions can be merged.
 In addition, emission of the function must happen in any translation 
 unit that references it (calling or address taken), and thus that 
 translation unit must also define it (in contrast to just declaring it). 
 I do not think `inline` forbids emission if the function is not referenced.
I also think that's the case. (Also not what was claimed.)
Jun 12
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:
 Not sure about that part - if linkage was static by means of using the
"static" 
 keyword, multiple definitions may not be merged. (I may be wrong, please
correct 
 me.) Consider:
 
 static inline int fun() {
      static int x;
      return ++x;
 }
 
 In C++, each translation unit containing a definition of fun() will have a 
 distinct address for x. I don't see how the bodies of those functions can be 
 merged.
They are not merged in D, for the simple reason that ModuleA.fun() and ModuleB.fun() will have different (mangled) names presented to the linker. (Of course, if you mark them extern(C) or extern(C++) the mangling will be the same and so they will be merged.) As I've mentioned previously, C++ has all kinds of contortions trying to make inline work in a universe with: 1. .h file inclusion 2. no modules 3. no COMDATs D just defines these problems out of existence.
Jun 12
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/12/20 8:54 PM, Walter Bright wrote:
 On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:
 Not sure about that part - if linkage was static by means of using the 
 "static" keyword, multiple definitions may not be merged. (I may be 
 wrong, please correct me.) Consider:

 static inline int fun() {
      static int x;
      return ++x;
 }

 In C++, each translation unit containing a definition of fun() will 
 have a distinct address for x. I don't see how the bodies of those 
 functions can be merged.
They are not merged in D, for the simple reason that ModuleA.fun() and ModuleB.fun() will have different (mangled) names presented to the linker.
For D the question is if they are merged if the function is defined in a .di file and imported in two other modules.
Jun 12
next sibling parent Manu <turkeyman gmail.com> writes:
On Sat, Jun 13, 2020 at 1:30 PM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/12/20 8:54 PM, Walter Bright wrote:
 On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:
 Not sure about that part - if linkage was static by means of using the
 "static" keyword, multiple definitions may not be merged. (I may be
 wrong, please correct me.) Consider:

 static inline int fun() {
      static int x;
      return ++x;
 }

 In C++, each translation unit containing a definition of fun() will
 have a distinct address for x. I don't see how the bodies of those
 functions can be merged.
They are not merged in D, for the simple reason that ModuleA.fun() and ModuleB.fun() will have different (mangled) names presented to the
linker. For D the question is if they are merged if the function is defined in a .di file and imported in two other modules.
They are not 'merged', they just don't exist. The problem I've repeated many times for D is that it doesn't emit the function ANYWHERE, and as such, you get a "undefined symbol" error. This is different than C++ where you would have gotten a "multiply defined symbol" error, but it's exactly the same problem for the exact same reason. It just manifests differently because C++ has .h files which naturally duplicates the code into each CU and D doesn't.
Jun 12
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Sat, Jun 13, 2020 at 1:39 PM Manu <turkeyman gmail.com> wrote:

 On Sat, Jun 13, 2020 at 1:30 PM Andrei Alexandrescu via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:

 On 6/12/20 8:54 PM, Walter Bright wrote:
 On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:
 Not sure about that part - if linkage was static by means of using the
 "static" keyword, multiple definitions may not be merged. (I may be
 wrong, please correct me.) Consider:

 static inline int fun() {
      static int x;
      return ++x;
 }

 In C++, each translation unit containing a definition of fun() will
 have a distinct address for x. I don't see how the bodies of those
 functions can be merged.
They are not merged in D, for the simple reason that ModuleA.fun() and ModuleB.fun() will have different (mangled) names presented to the
linker. For D the question is if they are merged if the function is defined in a .di file and imported in two other modules.
They are not 'merged', they just don't exist. The problem I've repeated many times for D is that it doesn't emit the function ANYWHERE, and as such, you get a "undefined symbol" error. This is different than C++ where you would have gotten a "multiply defined symbol" error, but it's exactly the same problem for the exact same reason. It just manifests differently because C++ has .h files which naturally duplicates the code into each CU and D doesn't.
And where I say "it's the same problem", perhaps it's better to say "it's the same issue"; about the requirement to emit inline code to each referencing CU, and what the linkonce link flags are designed to address.
Jun 12
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/12/2020 8:25 PM, Andrei Alexandrescu wrote:
 On 6/12/20 8:54 PM, Walter Bright wrote:
 On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:
 Not sure about that part - if linkage was static by means of using the 
 "static" keyword, multiple definitions may not be merged. (I may be wrong, 
 please correct me.) Consider:

 static inline int fun() {
      static int x;
      return ++x;
 }

 In C++, each translation unit containing a definition of fun() will have a 
 distinct address for x. I don't see how the bodies of those functions can be 
 merged.
They are not merged in D, for the simple reason that ModuleA.fun() and ModuleB.fun() will have different (mangled) names presented to the linker.
For D the question is if they are merged if the function is defined in a .di file and imported in two other modules.
If the di file is mentioned on the command line to the compiler, yes (1) instance of it appears in the executable. Otherwise, (0) instances of it appear in the executable. There are never 2 or more instances in the executable.
Jun 12
parent reply Manu <turkeyman gmail.com> writes:
On Sat, Jun 13, 2020 at 2:05 PM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/12/2020 8:25 PM, Andrei Alexandrescu wrote:
 On 6/12/20 8:54 PM, Walter Bright wrote:
 On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:
 Not sure about that part - if linkage was static by means of using the
 "static" keyword, multiple definitions may not be merged. (I may be
wrong,
 please correct me.) Consider:

 static inline int fun() {
      static int x;
      return ++x;
 }

 In C++, each translation unit containing a definition of fun() will
have a
 distinct address for x. I don't see how the bodies of those functions
can be
 merged.
They are not merged in D, for the simple reason that ModuleA.fun() and ModuleB.fun() will have different (mangled) names presented to the
linker.
 For D the question is if they are merged if the function is defined in a
.di
 file and imported in two other modules.
If the di file is mentioned on the command line to the compiler
It's not, that's literally the point of a .di file.
 , yes (1)
 instance of it appears in the executable. Otherwise, (0) instances of it
 appear
 in the executable. There are never 2 or more instances in the executable.
Exactly. And this is not a useful design.
Jun 12
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/12/2020 9:08 PM, Manu wrote:
     If the di file is mentioned on the command line to the compiler
 
 It's not, that's literally the point of a .di file.
No, it isn't. A .di file is more of a convention than a feature. It's a module and does not get special treatment by the compiler.
     , yes (1)
     instance of it appears in the executable. Otherwise, (0) instances of it
appear
     in the executable. There are never 2 or more instances in the executable.
 
 Exactly. And this is not a useful design.
I hate to say it, but these sorts of replies are completely useless to resolving your issues. You omitted the *why*. Why can't you put it on the command line? May I suggest a paradigm shift? A .d (or .di) file is not a .h file and does not work like .h files other than in a very superficial way. (In fact, .h files do not work like .h files. The C++ compiler has no idea what a .h file is. The characteristics you've imputed to it do not exist.) It's a module. It's fundamentally different. The .h file rules you are used to do not apply. You'd never put a .h file on the command line to a C++ compiler. Agreed. But those well-traveled notions don't apply to modules.
Jun 13
parent reply Johannes Pfau <nospam example.com> writes:
Am Sat, 13 Jun 2020 01:33:34 -0700 schrieb Walter Bright:

 On 6/12/2020 9:08 PM, Manu wrote:
     If the di file is mentioned on the command line to the compiler
 
 It's not, that's literally the point of a .di file.
No, it isn't. A .di file is more of a convention than a feature. It's a module and does not get special treatment by the compiler.
     , yes (1)
     instance of it appears in the executable. Otherwise, (0) instances
     of it appear in the executable. There are never 2 or more instances
     in the executable.
 
 Exactly. And this is not a useful design.
I hate to say it, but these sorts of replies are completely useless to resolving your issues. You omitted the *why*. Why can't you put it on the command line?
a.di: void foo() {} b.d: import a; c.d: import a; void main() {foo();} dmd -c b.d dmd -c c.d dmd b.o c.o => undefined reference to `_D1a3fooFZv' dmd -c a.di b.d -ofb.o dmd -c a.di c.d -ofc.o dmd b.o c.o => undefined reference to `_D1a3fooFZv' mv a.di a.d dmd -c a.d b.d -ofb.o dmd -c a.d c.d -ofc.o dmd b.o c.o => multiple definition of `_D1a12__ModuleInfoZ' dmd -c a.d b.d -ofb.o -betterC dmd -c a.d c.d -ofc.o -betterC dmd b.o c.o => OK BTW: If you do dmd -c a.di you get no object file output. So .di files are treated differently I think it's interesting that DMD seems to emit some (all?) normal functions as weak. Not sure if LDC and GDC do the same thing. Would also be interesting to see how all this interacts with staic & shared libraries, though I'm optimistic that it just works. So basically all that's missing for Manu's inline case would be to emit pragma(inlne) functions from non-root modules. Probably a 1-line change. -- Johannes
Jun 13
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/13/20 7:31 AM, Johannes Pfau wrote:
 Am Sat, 13 Jun 2020 01:33:34 -0700 schrieb Walter Bright:
 
 On 6/12/2020 9:08 PM, Manu wrote:
      If the di file is mentioned on the command line to the compiler

 It's not, that's literally the point of a .di file.
No, it isn't. A .di file is more of a convention than a feature. It's a module and does not get special treatment by the compiler.
      , yes (1)
      instance of it appears in the executable. Otherwise, (0) instances
      of it appear in the executable. There are never 2 or more instances
      in the executable.

 Exactly. And this is not a useful design.
I hate to say it, but these sorts of replies are completely useless to resolving your issues. You omitted the *why*. Why can't you put it on the command line?
a.di: void foo() {} b.d: import a; c.d: import a; void main() {foo();} dmd -c b.d dmd -c c.d dmd b.o c.o => undefined reference to `_D1a3fooFZv' dmd -c a.di b.d -ofb.o dmd -c a.di c.d -ofc.o dmd b.o c.o => undefined reference to `_D1a3fooFZv' mv a.di a.d dmd -c a.d b.d -ofb.o dmd -c a.d c.d -ofc.o dmd b.o c.o => multiple definition of `_D1a12__ModuleInfoZ' dmd -c a.d b.d -ofb.o -betterC dmd -c a.d c.d -ofc.o -betterC dmd b.o c.o => OK BTW: If you do dmd -c a.di you get no object file output. So .di files are treated differently I think it's interesting that DMD seems to emit some (all?) normal functions as weak. Not sure if LDC and GDC do the same thing. Would also be interesting to see how all this interacts with staic & shared libraries, though I'm optimistic that it just works. So basically all that's missing for Manu's inline case would be to emit pragma(inlne) functions from non-root modules. Probably a 1-line change.
Should that go in bugzilla?
Jun 13
parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On Saturday, 13 June 2020 at 18:24:18 UTC, Andrei Alexandrescu 
wrote:
 On 6/13/20 7:31 AM, Johannes Pfau wrote:
 
 So basically all that's missing for Manu's inline case would 
 be to emit
 pragma(inlne) functions from non-root modules. Probably a 
 1-line change.
Should that go in bugzilla?
I don't see why there should not be a bugzilla for it. Manu, pardon my skipping the 11 pages of replies between the first and last posts, so if it's already been said, please ignore. I think trying to impose the C/C++ notion of inline as a linkage modifier into D is at odds with modules. I'm going to make a potentially disagreeable statement and say that there is no external or internal linkage in D. Rather there's language linkage, e.g: `extern(C++)`; and then there's module linkage, where a given declaration resolves to the same symbol across all TUs that import its residing module, but otherwise not strictly visible outside that. To address your laments, it may be wise to pay attention to the direction that `inline` is heading in C++ modules. http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1498r1.html#inline http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p1779r3.html Notably the sentence: "Within interface units of modules, we suggest converging on an interperation more firmly rooted in the semantics: an inline entity (function definition or variable initialization) is semantically incorporated into (or inlined into) the interface of a module." Which I interpret as, given: module mod; pragma(inline, true) void fun() { ... } Any module that imports `mod` (called non-root), will have the `fun` emitted into the module as well. This would be implemented using the support of weak (ELF) or selectany (COFF), so the linker will throw away all duplicated definitions. However, to satisfy your "emit to the binary only when it is called" constraint, it will also have to be discard-able at compile-time. But I'm not sure whether that really a desirable feature to have. At the very least, it should only extend to non-root modules, so there's always one copy in the root module it was declared, to avoid linker errors.
Jun 21
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/13/2020 4:31 AM, Johannes Pfau wrote:
 BTW: If you do dmd -c a.di you get no object file output. So .di files
 are treated differently
Yeah, you're right. Only in one spot, however, the .obj file is omitted. You can always rename it to .d
 I think it's interesting that DMD seems to emit some (all?) normal
 functions as weak.
It emits them all as COMDATs. This is so multiple definitions can be combined.
 So basically all that's missing for Manu's inline case would be to emit
 pragma(inlne) functions from non-root modules. Probably a 1-line change.
It's simply unnecessary for what Manu wants to achieve (header only libraries).
Jun 13
parent Manu <turkeyman gmail.com> writes:
On Sun, Jun 14, 2020 at 5:55 AM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/13/2020 4:31 AM, Johannes Pfau wrote:
 BTW: If you do dmd -c a.di you get no object file output. So .di files
 are treated differently
Yeah, you're right. Only in one spot, however, the .obj file is omitted. You can always rename it to .d
By definition; we're talking about libraries here. Inline's are only relevant when calling into foreign code. I don't molest the source tree of my libraries. There's a high probability the directory doesn't even have write permissions.
Jun 13
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/13/20 12:08 AM, Manu wrote:
 On Sat, Jun 13, 2020 at 2:05 PM Walter Bright via Digitalmars-d 
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
 
     On 6/12/2020 8:25 PM, Andrei Alexandrescu wrote:
      > On 6/12/20 8:54 PM, Walter Bright wrote:
      >> On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:
      >>> Not sure about that part - if linkage was static by means of
     using the
      >>> "static" keyword, multiple definitions may not be merged. (I
     may be wrong,
      >>> please correct me.) Consider:
      >>>
      >>> static inline int fun() {
      >>>      static int x;
      >>>      return ++x;
      >>> }
      >>>
      >>> In C++, each translation unit containing a definition of fun()
     will have a
      >>> distinct address for x. I don't see how the bodies of those
     functions can be
      >>> merged.
      >>
      >> They are not merged in D, for the simple reason that
     ModuleA.fun() and
      >> ModuleB.fun() will have different (mangled) names presented to
     the linker.
      >
      > For D the question is if they are merged if the function is
     defined in a .di
      > file and imported in two other modules.
 
     If the di file is mentioned on the command line to the compiler
 
 
 It's not, that's literally the point of a .di file.
 
     , yes (1)
     instance of it appears in the executable. Otherwise, (0) instances
     of it appear
     in the executable. There are never 2 or more instances in the
     executable.
 
 
 Exactly. And this is not a useful design.
All I can say is if that and/or other wrinkles prevent header-only libraries definition and use, yes, not a useful design. So Manu, is the top-level goal "I want header-only libraries in D"?
Jun 13
parent Manu <turkeyman gmail.com> writes:
On Sun, Jun 14, 2020 at 4:20 AM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/13/20 12:08 AM, Manu wrote:
 On Sat, Jun 13, 2020 at 2:05 PM Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>>
wrote:
     On 6/12/2020 8:25 PM, Andrei Alexandrescu wrote:
      > On 6/12/20 8:54 PM, Walter Bright wrote:
      >> On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:
      >>> Not sure about that part - if linkage was static by means of
     using the
      >>> "static" keyword, multiple definitions may not be merged. (I
     may be wrong,
      >>> please correct me.) Consider:
      >>>
      >>> static inline int fun() {
      >>>      static int x;
      >>>      return ++x;
      >>> }
      >>>
      >>> In C++, each translation unit containing a definition of fun()
     will have a
      >>> distinct address for x. I don't see how the bodies of those
     functions can be
      >>> merged.
      >>
      >> They are not merged in D, for the simple reason that
     ModuleA.fun() and
      >> ModuleB.fun() will have different (mangled) names presented to
     the linker.
      >
      > For D the question is if they are merged if the function is
     defined in a .di
      > file and imported in two other modules.

     If the di file is mentioned on the command line to the compiler


 It's not, that's literally the point of a .di file.

     , yes (1)
     instance of it appears in the executable. Otherwise, (0) instances
     of it appear
     in the executable. There are never 2 or more instances in the
     executable.


 Exactly. And this is not a useful design.
All I can say is if that and/or other wrinkles prevent header-only libraries definition and use, yes, not a useful design. So Manu, is the top-level goal "I want header-only libraries in D"?
It is one among multiple use cases in that category. It's essentially the That is one useful case. It's not limited to that though, it comes up in various situations. For instance, I do a lot with DLL's... it's typical to load a DLL, and then bind to a single function that creates an instance of some object that the DLL provides. From there, further calls into the library are via the object vtable.
From this point, there are a few design pressures that lead to inline
functions: 1. v-calls are slow, and trivial operations (ie, getters/setters) should just be inlines where it makes sense. Cross-module calls are not desirable. 2. DLL's ABI is essentially part of the API, and changes to the ABI are breaking changes to the lib. - Skilled design is to have fewer virtual functions that 'do stuff', with flexibility in how it's called and lots of soft-extensibility option arguments. * Enhancement to a lib can be made by adding a new item to an enum, rather than breaking the ABI. - This broad style of function is inconvenient for users, and typical use cases often wrap these work functions up in convenience inline calls, to automate a complex calling scheme. 3. Such libraries are installed by the package manager in some location, along with their 'headers', which root path is automatically added in a build script somewhere. - It is not the job of the application's build environment to determine what functions it calls from a lib and then cherry-pick source files from the lib and include them into the applications build... that creates compile-time couplings which are awkward at best, and naturally break-down over time. It self-defeats the use of DLL's. A build system might not even know how to identify the paths to a particular library's random source files, and it shouldn't be the client's responsibility to understand a lib's source tree. It's hard to know what source files you need to include and build... I probably don't call every function in the lib; I don't want to include the source for a DLL into my application; that's self-defeating. TL;DR, right now, I care about DLL's, and inline functions are an essential part of DLL-based libraries. It doesn't end there though either; there are so many cases where a reference to a lib should not create a 'hard' coupling between projects. I have often had situations involving 'optional' functionality; if a call is never made, then the dependency beneath that call isn't inferred on the client. An inline with no references naturally doesn't pull the dependency reference into the binary unless it's actually called. You could argue for stripping to solve that case, but that's a much more brittle and unsatisfying solution.
Jun 13
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On Sat, Jun 13, 2020 at 10:20 AM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/12/20 2:08 PM, Johan wrote:
 On Friday, 12 June 2020 at 13:22:35 UTC, Andrei Alexandrescu wrote:
 On 6/8/20 2:14 AM, Manu wrote:
 In C/C++, inline says that a function will be emit to the binary only
 when it is called, and the function is marked with internal linkage
 (it is not visible to the linker from the symbol table)
By my recollection this is not the case for C++, at all. * "inline" does NOT change a function's linkage in C++. You may have inline functions with internal linkage (static inline) or (default) external linkage. This is important because e.g. defining a static variable in an extern inline function will have the same address in all calls to the function.
I believe Manu tried to explain that `inline` in C++ really only affects how the linker must treat the symbol, and it is best to remember that it does nothing at all concerning "inlining" (putting function body inside another).
Thank you. That's not at all what he wrote. AT ALL. It's what I wrote. I'm emphasizing this because it has been a recurring problem: a legitimate problem with an explanation lost in translation.
You're joking right? """ 1. *I only want the function to be present in the CALLING binary*. I do not want an inline function present in the local binary where it was defined (unless it was called internally). I do not want a linker to see the inline function symbols and be able to link to them externally. [*This is about linkage and controlling the binary or distribution environment*] """ There is absolutely nowhere in that text where I describe that a function should be emit inline to the calling function. I couldn't possibly be clearer about its placement into the calling CU. I genuinely don't have any idea how I could have been misunderstood in this thread. It seems most readers did not misunderstand me.
 `inline` is really a misnomer.
 The "linkage" effect of `inline` is required for C++ to work with
 #include files. There is some room for confusion because, depending on
 who is talking, the word "linkage" can have more nuances than just
 internal/external.
It helps to use terminology that is well defined and standardized. That's where simply using the meaning defined and consistently used in the C++ standard document can be of great help.
We *are* using standard terminology. `inline` means something very specific in C; it means deduplication of symbols, and that's what we're talking about, for years.
 I personally take the broad stance of "linkage"
 including anything the linker can do with the symbol.
Makes sense. It's really nice to just use http://eel.is/c++draft/basic.link, which ensures we all understand one another.
There's no text in there that describes the link flags with standard terminology. I have never heard standard terminology for them. Can you show a reference for standard linker terminology which we can refer to in the future? LLVM is the only reference I know: https://llvm.org/docs/LangRef.html#linkage-types `internal` or `linkonce` are possible choices here, `linkonce` would match C++.
 Without `inline`, you would get multiple definition linker errors for
 functions defined in a header file when linking together two translation
 units that both included that header file. Instead of normal function
 linkage that forbids multiple definition, `inline` changes linkage to
 merge multiple definitions into one.
Not sure about that part - if linkage was static by means of using the "static" keyword, multiple definitions may not be merged. (I may be wrong, please correct me.) Consider: static inline int fun() { static int x; return ++x; } In C++, each translation unit containing a definition of fun() will have a distinct address for x. I don't see how the bodies of those functions can be merged.
`static inline` is strictly internal. They will not be merged. Each CU will have its own copy of the function, and it will not deduplicate. We should be able use `private` the same way in D.
 In addition, emission of the function must happen in any translation
 unit that references it (calling or address taken), and thus that
 translation unit must also define it (in contrast to just declaring it).
 I do not think `inline` forbids emission if the function is not
referenced. I also think that's the case. (Also not what was claimed.)
have possibly been clearer about this. I've been saying it over and over and over again for years. The only difference you could perceive is that I suggest it should not emit it if it's never referenced... to be clear; I was not commenting on what C++ does, I was commenting on what I'd like. It would be nice to not generate and emit the code and bloat the binary if there are no calls. It's not necessary, because any external references will have their own copy anyway.
Jun 12
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/12/20 9:13 PM, Manu wrote:
 On Sat, Jun 13, 2020 at 10:20 AM Andrei Alexandrescu via Digitalmars-d 
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
 
     On 6/12/20 2:08 PM, Johan wrote:
      > On Friday, 12 June 2020 at 13:22:35 UTC, Andrei Alexandrescu wrote:
      >> On 6/8/20 2:14 AM, Manu wrote:
      >>> In C/C++, inline says that a function will be emit to the
     binary only
      >>> when it is called, and the function is marked with internal
     linkage
      >>> (it is not visible to the linker from the symbol table)
      >>
      >> By my recollection this is not the case for C++, at all.
      >>
      >> * "inline" does NOT change a function's linkage in C++. You may
     have
      >> inline functions with internal linkage (static inline) or (default)
      >> external linkage. This is important because e.g. defining a static
      >> variable in an extern inline function will have the same address in
      >> all calls to the function.
      >
      > I believe Manu tried to explain that `inline` in C++ really only
     affects
      > how the linker must treat the symbol, and it is best to remember
     that it
      > does nothing at all concerning "inlining" (putting function body
     inside
      > another).
 
     Thank you. That's not at all what he wrote. AT ALL. It's what I wrote.
 
     I'm emphasizing this because it has been a recurring problem: a
     legitimate problem with an explanation lost in translation.
 
 
 You're joking right?
Well, again I tried to get communication going and again I failed. So... again I will bow out, with apologies.
Jun 12
parent Manu <turkeyman gmail.com> writes:
On Sat, Jun 13, 2020 at 1:35 PM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/12/20 9:13 PM, Manu wrote:
 On Sat, Jun 13, 2020 at 10:20 AM Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>>
wrote:
     On 6/12/20 2:08 PM, Johan wrote:
      > On Friday, 12 June 2020 at 13:22:35 UTC, Andrei Alexandrescu
wrote:
      >> On 6/8/20 2:14 AM, Manu wrote:
      >>> In C/C++, inline says that a function will be emit to the
     binary only
      >>> when it is called, and the function is marked with internal
     linkage
      >>> (it is not visible to the linker from the symbol table)
      >>
      >> By my recollection this is not the case for C++, at all.
      >>
      >> * "inline" does NOT change a function's linkage in C++. You may
     have
      >> inline functions with internal linkage (static inline) or
(default)
      >> external linkage. This is important because e.g. defining a
static
      >> variable in an extern inline function will have the same address
in
      >> all calls to the function.
      >
      > I believe Manu tried to explain that `inline` in C++ really only
     affects
      > how the linker must treat the symbol, and it is best to remember
     that it
      > does nothing at all concerning "inlining" (putting function body
     inside
      > another).

     Thank you. That's not at all what he wrote. AT ALL. It's what I
wrote.
     I'm emphasizing this because it has been a recurring problem: a
     legitimate problem with an explanation lost in translation.


 You're joking right?
Well, again I tried to get communication going and again I failed. So... again I will bow out, with apologies.
No, I appreciate your effort to improve the conversation. What I don't appreciate is suggesting that "That's not at all what [he] wrote. AT ALL." when I really feel like it is exactly what I wrote. And if it was confused, I apologise, but there are plenty of people who were able to read it and understand precisely what I was saying. Anyway...
Jun 12
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On Fri, Jun 12, 2020 at 11:25 PM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/8/20 2:14 AM, Manu wrote:
 In C/C++, inline says that a function will be emit to the binary only
 when it is called, and the function is marked with internal linkage (it
 is not visible to the linker from the symbol table)
By my recollection this is not the case for C++, at all. * "inline" does NOT change a function's linkage in C++. You may have inline functions with internal linkage (static inline) or (default) external linkage. This is important because e.g. defining a static variable in an extern inline function will have the same address in all calls to the function.
It absolutely changes the linkage. I believe it uses what LLVM calls 'ChooseOne' in its code generator, I don't know about 'standard' linker terminology, if such a thing exists. It's clearly in the spec too: """ 1. There may be more than one definition <https://en.cppreference.com/w/cpp/language/definition#One_Definition_Rule> of an inline function or variable in the program as long as each definition *appears in a different translation unit* and (for non-static inline functions and variables) all definitions are identical. For example, an inline function or an inline variable may be defined in a header file that is #include'd in multiple source files. """ If not for that link flag, you receive a "multiply declared symbol" error. If the above is correct, a D language feature dedicated to inlining
 should do the following:

 * Always emit the function body in the .di file if ever asked to
 generate it.
Your sentence is unclear, but I think we agree. I like how I describe this better: "The function must be emit to the _calling_ CU" * Never complain about duplicate symbols if an inline function has
 duplicate definitions.
And as I have been saying: "It must have the appropriate linker flag" (again, I think LLVM calls it "ChooseOne") Then compilers can decide on specific inlining strategies using the
 language feature as a hint.
useful to have SOME WAY, to 'force inline' and receive an error if it failed. performed in the front-end as some sort of AST macro, then we can have complete confidence it happened, and error if not. This is a DIFFERENT THING than 'inline' generally, and I would describe it as "force" inline, and suggested `pragma(inline, force)`. D does ALL of the above already for templates, so this is not a
 difficult feature to implement. In fact we use this technique (e.g. in
 druntime) by spelling inline as "()". Consider:

 int func(int) { ... body ... }

 Let's make this inline:

 int func()(int) { ... body ... }

 Done.
Except for the sea of edge cases when you supply `func` to generic code that was expecting a function. This is a sloppy and unsatisfying work-around. In D there's one additional implication of body availability - the
 function is eligible for CTFE. I think any adjustment to the extant
 inline pragma needs to make this a top-level consideration.
I think this will work naturally with no changes today. If a function definition is present in a di file, you can CTFE call it, but (unless we fix inline) you can not call it at runtime (because no code will be emit anywhere).
Jun 12
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/12/20 8:52 PM, Manu wrote:
 On Fri, Jun 12, 2020 at 11:25 PM Andrei Alexandrescu via Digitalmars-d 
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
 
     On 6/8/20 2:14 AM, Manu wrote:
      > In C/C++, inline says that a function will be emit to the binary
     only
      > when it is called, and the function is marked with internal
     linkage (it
      > is not visible to the linker from the symbol table)
 
     By my recollection this is not the case for C++, at all.
 
     * "inline" does NOT change a function's linkage in C++. You may have
     inline functions with internal linkage (static inline) or (default)
     external linkage. This is important because e.g. defining a static
     variable in an extern inline function will have the same address in all
     calls to the function.
 
 
 It absolutely changes the linkage.
No.
 I believe it uses what LLVM calls 'ChooseOne' in its code generator, I 
 don't know about 'standard' linker terminology, if such a thing exists.
It does: http://eel.is/c++draft/basic.link
 It's clearly in the spec too:
 """
 
  1. There may be more than one definition
     <https://en.cppreference.com/w/cpp/language/definition#One_Definition_Rule> of
     an inline function or variable in the program as long as each
     definition *appears in a different translation unit* and (for
     non-static inline functions and variables) all definitions are
     identical. For example, an inline function or an inline variable may
     be defined in a header file that is #include'd in multiple source files.
 
 """
The quote does not even contain the word "linkage". I'm insisting on this because it happens so often. We need to use the terms with the same meaning, otherwise we get bogged down in silly side quests "it doesn't change linkage" - "oh but it does" and there is no progress.
Jun 12
parent Manu <turkeyman gmail.com> writes:
On Sat, Jun 13, 2020 at 1:25 PM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/12/20 8:52 PM, Manu wrote:
 On Fri, Jun 12, 2020 at 11:25 PM Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>>
wrote:
     On 6/8/20 2:14 AM, Manu wrote:
      > In C/C++, inline says that a function will be emit to the binary
     only
      > when it is called, and the function is marked with internal
     linkage (it
      > is not visible to the linker from the symbol table)

     By my recollection this is not the case for C++, at all.

     * "inline" does NOT change a function's linkage in C++. You may have
     inline functions with internal linkage (static inline) or (default)
     external linkage. This is important because e.g. defining a static
     variable in an extern inline function will have the same address in
all
     calls to the function.


 It absolutely changes the linkage.
No.
Dump a binary with and without `inline`; look at the link flags... they are different.
 I believe it uses what LLVM calls 'ChooseOne' in its code generator, I
 don't know about 'standard' linker terminology, if such a thing exists.
It does: http://eel.is/c++draft/basic.link
 It's clearly in the spec too:
 """

  1. There may be more than one definition
     <
https://en.cppreference.com/w/cpp/language/definition#One_Definition_Rule
 of
     an inline function or variable in the program as long as each
     definition *appears in a different translation unit* and (for
     non-static inline functions and variables) all definitions are
     identical. For example, an inline function or an inline variable may
     be defined in a header file that is #include'd in multiple source
files.
 """
The quote does not even contain the word "linkage". I'm insisting on this because it happens so often. We need to use the terms with the same meaning, otherwise we get bogged down in silly side quests "it doesn't change linkage" - "oh but it does" and there is no progress.
That's because this is a C++ spec, and C++ is not a linker. The spec just specifies the required semantics, and the compiler implements them. The compiler happens to implement them appropriately for the linking ecosystem (by using appropriate link flags), because that's how native code works... you build modules and then link them with a linker which is outside the domain of the language spec. The whole point is that different languages can interact with each other successfully in 'linker space'. Unless we want to have a self-contained language island, we can't ignore that a linker will link our code (together with other code), and the compilers output has to work properly in that environment. Like C++, we don't need to spec details about link flags, but the compiler implementation still needs to implement them in those terms, cus that's how the ecosystem has been designed to work. I imagine any changes made here would have a similar expression in the D spec as what you read in the C++ spec... and the practical reality is the implementation will apply the appropriate link flags to the symbols.
Jun 12
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/12/20 8:52 PM, Manu wrote:

 to have SOME WAY, to 'force inline' and receive an error if it failed.
I recall a couple of compilers (TopSpeed, does anyone remember?) had such a feature. The warnings listing the arbitrary functions that failed whatever heuristics was utterly useless. No C++ compiler implements it today, and I don't think any should.
Jun 12
next sibling parent Manu <turkeyman gmail.com> writes:
On Sat, Jun 13, 2020 at 1:25 PM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 6/12/20 8:52 PM, Manu wrote:

 to have SOME WAY, to 'force inline' and receive an error if it failed.
I recall a couple of compilers (TopSpeed, does anyone remember?) had such a feature. The warnings listing the arbitrary functions that failed whatever heuristics was utterly useless. No C++ compiler implements it today, and I don't think any should.
I feel like I clearly agreed here too; I gave 3 cases which are distinct those cases. agree C++ shouldn't try and jam this idea into `inline` because that has a confused history, and a fairly well defined present. That doesn't mean that it's not a useful tool though, and one that I've wanted lots of times... but as I've made clear, I see this as a distinct use case, and should be explicit and distinct. common use cases.
Jun 12
prev sibling parent Johannes Pfau <nospam example.com> writes:
Am Fri, 12 Jun 2020 23:24:32 -0400 schrieb Andrei Alexandrescu:

 On 6/12/20 8:52 PM, Manu wrote:

 to have SOME WAY, to 'force inline' and receive an error if it failed.
I recall a couple of compilers (TopSpeed, does anyone remember?) had such a feature. The warnings listing the arbitrary functions that failed whatever heuristics was utterly useless. No C++ compiler implements it today, and I don't think any should.
Doesn't GCC's always_inline do exactly that? https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html Generally, functions are not inlined unless optimization is specified. For functions declared inline, this attribute inlines the function independent of any restrictions that otherwise apply to inlining. Failure to inline such a function is diagnosed as an error. Note that if such a function is called indirectly the compiler may or may not inline it depending on optimization level and a failure to inline an indirect call may or may not be diagnosed. -- Johannes
Jun 13