www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 7854] New: Non-C attributes allowed on extern(C) function parameters

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854

           Summary: Non-C attributes allowed on extern(C) function
                    parameters
           Product: D
           Version: unspecified
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: DMD
        AssignedTo: nobody puremagic.com
        ReportedBy: jmdavisProg gmx.com


--- Comment #0 from Jonathan M Davis <jmdavisProg gmx.com> 2012-04-07 17:32:19
PDT ---
This compiles:

extern(C) void func(ref int a,
                    out int b,
                    in int* c,
                    scope int* d,
                    immutable int* e);

void main()
{
}

which seems pretty atrocious to me. ref, out, in, scope, and immutable do not
exist in C. Other than the fact that D tends to be lax with attributes on
functions (though not generally parameters), I don't know why these would work.

The fact that pure and nothrow work on an extern(C) function is useful, because
it then becomes possible to use them in other pure and nothrow functions, but
that doesn't affect how parameters are passed at all, just how D allows you to
call the function. A similar argument might be made for scope (and therefore
in), but the same cannot be said for ref, out, or immutable. They _do_ affect
the parameters themeselves, not just how the function is called.

Assuming that ref and out really translate to just pointers underneath as
opposed to D doing some extra stuff with them, then translating ref and out to
pointers when generating the code would work, but I'm not sure that that's the
case, and I'd argue that it still makes no sense to allow type modifiers on
extern(C) function parameters which don't exist in C. They're _C_, not _D_. C
has pointers. The function parameters should reflect that. Allowing ref/out on
them buys us little to nothing (assuming that it even works properly) and means
that instead of giving a C function signature, a D function signature is being
given which must be translated to a C function signature.

immutable could just be translated to const, but I'd argue that there's no
point in allowing it, for the same reason - it's not C, and extern(C) functions
are supposed to be C functions.

It's bad enough to allow pure and nothrow on extern(C) functions, but we pretty
much _have_ to in order to avoid a bunch of casting and other such ugliness for
pure/nothrow D functions to use them (though at least with nothrow, you can
catch(Exception) - pure has no such way out). But those have _zero_ affect on
what the actual C function looks like, because they're just additional
attributes that tell the D compiler something about the function, whereas the
ones in the example above affect the actual, C signature.

Is the fact that _any_ such attributes are allowed on extern(C) functions on
purpose? The pages on interfacing with C code don't talk about them, and it
looks like a definite bug to me if they're allowed. I'd be _very_ concerned
about them doing weird things when you actually try and use the extern(C)
function, and I don't like the idea on general principle.

So, either D-specific attributes should be disallowed on extern(C) function
parameters, or they should be explicitly specificied in the spec (including
what they translate to) - and I'd argue that the former is more desirable.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 07 2012
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854


Don <clugdbug yahoo.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |clugdbug yahoo.com.au


--- Comment #1 from Don <clugdbug yahoo.com.au> 2012-04-08 23:09:26 PDT ---
This is part of the general spec bug that extern(C) is almost completely
undocumented. Eg, what does this do?

extern(C) void func(lazy int x);

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 08 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854


Steven Schveighoffer <schveiguy yahoo.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |schveiguy yahoo.com
          Component|DMD                         |websites


--- Comment #2 from Steven Schveighoffer <schveiguy yahoo.com> 2012-04-12
11:55:47 PDT ---
extern(C) does not have any effect on parameters.  Note that you can easily
implement extern(C) functions in D (in fact the runtime heavily relies on
this).  AFAICT, it basically is just a way to treat the symbol as demangled.

Also note that because C treats fixed-sized array parameters as pointers, and D
treats them as values, the following idiom has emerged, which would not be
possible if ref wasn't allowed:

extern(C) int pipe(ref int[2] fds);

Changing to websites, since this is really a spec issue.  The compiler is
implemented properly IMO.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854


Don <clugdbug yahoo.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |spec
          Component|websites                    |DMD


--- Comment #3 from Don <clugdbug yahoo.com.au> 2012-04-12 12:56:33 PDT ---
(In reply to comment #2)

 extern(C) does not have any effect on parameters. 


Not so. It most definitely does! Doesn't make much difference on Linux, but on
Windows they are quite different. Also variadic functions are defined in the
spec to be different for extern(C) vs extern(D), on all platforms.



 Note that you can easily
 implement extern(C) functions in D (in fact the runtime heavily relies on
 this).


I know, and it worries me. The question is, is that just
implementation-specific behaviour?


 Also note that because C treats fixed-sized array parameters as pointers, and D
 treats them as values, the following idiom has emerged, which would not be
 possible if ref wasn't allowed:
 
 extern(C) int pipe(ref int[2] fds);


Yeah, and that's nice. But the odd thing is, that is actually implementable in
C. Whereas other things, such as lazy, might not be. The key issue is, should
extern(C) allow signatures that *cannot* be implemented in C?


 Changing to websites, since this is really a spec issue.  The compiler is
 implemented properly IMO.


I agree it is probably a spec issue. Spec issues are normally treated with DMD
+ spec keyword. It is part of the DMD download.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854



--- Comment #4 from Steven Schveighoffer <schveiguy yahoo.com> 2012-04-12
14:20:09 PDT ---
(In reply to comment #3)

 
 I agree it is probably a spec issue. Spec issues are normally treated with DMD
 + spec keyword. It is part of the DMD download.


From description in bugzilla for websites component:

"Problems with the contents of www.digitalmars.com and
www.d-programming-language.org, including the language specification"

If what you say is true, we should probably update the component description.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854



--- Comment #5 from Steven Schveighoffer <schveiguy yahoo.com> 2012-04-12
14:28:18 PDT ---
(In reply to comment #3)

 (In reply to comment #2)

 extern(C) does not have any effect on parameters. 


 
 Not so. It most definitely does! Doesn't make much difference on Linux, but on
 Windows they are quite different. Also variadic functions are defined in the
 spec to be different for extern(C) vs extern(D), on all platforms.


I'm not exactly talking about binding or calling convention, I'm more talking
about types.  To me, the two are orthogonal.

And I don't see why lazy would be any different for extern(C) than it is for D,
all the hard work is done by the caller.  It's just that the type is really
transformed into a delegate.

variadic functions of course are different and defined differently, no argument
there.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854



--- Comment #6 from Jonathan M Davis <jmdavisProg gmx.com> 2012-04-12 23:07:41
PDT ---
< I'm not exactly talking about binding or calling convention, I'm more talking
< about types.  To me, the two are orthogonal.

Whereas I would argue that since you're declaring a C function, it should be a
_C_ function and therefore not include features which C doesn't have. The only
reason that I think that permitting pure and nothrow on C functions makes any
sense is out of pure necessity.


 extern(C) int pipe(ref int[2] fds);


I would expect this to simply be

extern(C) int pipe(int* fds);

because what C does is pass a pointer, not a fixed-size array.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854


Maxim Fomin <maxim maxim-fomin.ru> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |maxim maxim-fomin.ru


--- Comment #7 from Maxim Fomin <maxim maxim-fomin.ru> 2012-04-14 22:29:04 PDT
---
(In reply to comment #3)

 (In reply to comment #2)

 extern(C) does not have any effect on parameters. 


 
 Not so. It most definitely does! Doesn't make much difference on Linux, but on
 Windows they are quite different. Also variadic functions are defined in the
 spec to be different for extern(C) vs extern(D), on all platforms.
 


And how does it affect? 
In most cases what I found about externs in D, C++, C they all are about
linkage.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 14 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854



--- Comment #8 from Maxim Fomin <maxim maxim-fomin.ru> 2012-04-15 01:15:46 PDT
---
When extern(C) appears it boils down to:

1) either it is used with purely C features:
extern (C) int *array_func (int *array, size_t size);
This is heavily used when interfacing D code with C libraries and vice versa.

2) or it is used with non-C features:
extern (C) int[] array_func (int[] array);
Assumptions here are following:
- array_func is in different object file which was made (possibly) by different
implementation of which current source code is compiled (it may be even not D);
- such another implemenation mangles names differently, to overcome this
obstacle there is need to remove any mangling;
- but such implementation handles at least arrays (as in the example) equally
with current implementation. As new features would be introduced in
declaration, more features should be treated equally.

Although compiler (and programmer) can distinguish among 1) and 2) the problem
is worsen by the fact that much depends on what will be happening after
compilation and compiler cannot solve such things. 

Shortly speaking, the fact that dmd allows such thing is a loophole to link D
code with something that couldn't be used directly but has something common
with current implementation of D and is worth using. It is an option, and it is
a tradeoff between possible benefits and bugs. I do not know about any benefits
which it gives now, but it may change in the future. The first idea is that
different D compilers may mangle names differently, but ABI page at dlang.org
states that all implementations should conform to one mangling standard.
However, it is not official standard and nobody can enforce to follow it. 
Speaking about possible bugs it is clear that they are obvious (in majority
cases the code with 2) even wouldn't contain hidden bug but would not work).
So, code containing something like 2) means that it was written by somebody 
who either knows what he is he doing, or is looking for problems.

Possible solutions:
1) forbid non-C features in extern(C) as it was suggested. D code could be
linked only with C, D (suggesting that every D implementation follows abi page,
otherwise D community would have big problem) and any other language which can
be interfaced in C way.
2) leave the situation as it is. The only problem would come from code written
by people who experince difficulties in distinguishing D and C
3) forbid non-C features in extern(C) as it was suggested but introduce
directive that instructs compiler not to mangle function names.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 15 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854



--- Comment #9 from Steven Schveighoffer <schveiguy yahoo.com> 2012-04-16
05:43:17 PDT ---
(In reply to comment #6)

 I'm not exactly talking about binding or calling convention, I'm more talking
 about types.  To me, the two are orthogonal.


 
 Whereas I would argue that since you're declaring a C function, it should be a
 _C_ function and therefore not include features which C doesn't have. The only
 reason that I think that permitting pure and nothrow on C functions makes any
 sense is out of pure necessity.


No.  An extern(C) function does not mean it's a C function.  It just means it
has C linkage.  See here: http://dlang.org/attribute.html#linkage

extern(C) has nothing to do with parameters, only calling conventions.

What would you expect here?

extern(C) void foo(long);

to accept 32-bit or 64-bit integer?  The answer is, 64 bit.  The confusion that
would abound if you had to use C's types and keywords whenever declaring an
extern(C) function would be not worth the trouble.

The one exception, as Don pointed out, is variadic arguments, but that is
covered here: http://dlang.org/function.html#variadic


 extern(C) int pipe(ref int[2] fds);


 
 I would expect this to simply be
 
 extern(C) int pipe(int* fds);
 
 because what C does is pass a pointer, not a fixed-size array.


This does not say anything about the size.  Even though it's a 'hint' in C,
it's enforceable in D.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 16 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854



--- Comment #10 from Don <clugdbug yahoo.com.au> 2012-04-17 02:03:47 PDT ---
(In reply to comment #9)

 (In reply to comment #6)

 I'm not exactly talking about binding or calling convention, I'm more talking
 about types.  To me, the two are orthogonal.


 
 Whereas I would argue that since you're declaring a C function, it should be a
 _C_ function and therefore not include features which C doesn't have. The only
 reason that I think that permitting pure and nothrow on C functions makes any
 sense is out of pure necessity.


 
 No.  An extern(C) function does not mean it's a C function.  It just means it
 has C linkage.  See here: http://dlang.org/attribute.html#linkage
 
 extern(C) has nothing to do with parameters, only calling conventions.


I don't understand that statement. Do you mean 'parameters, only name mangling"
? If so, that that isn't true.
On 32-bit x86 Windows,

extern(D) void foo(int x)

passes x in the EAX register.

extern(C) void foo(int x)

passes x on the stack.
It's not just a name mangling issue.

The confusion comes because there are some druntime functions which use C name
mangling but the extern(D) calling convention! This is not true of all
extern(C) functions.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 17 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854



--- Comment #11 from Steven Schveighoffer <schveiguy yahoo.com> 2012-04-18
05:21:34 PDT ---
(In reply to comment #10)

 (In reply to comment #9)

 No.  An extern(C) function does not mean it's a C function.  It just means it
 has C linkage.  See here: http://dlang.org/attribute.html#linkage
 
 extern(C) has nothing to do with parameters, only calling conventions.


 
 I don't understand that statement. Do you mean 'parameters, only name mangling"
 ? If so, that that isn't true.


No, I mean the extern(C) designates linkage and calling convention, it doesn't
affect the parameters or qualifiers for those parameters.  It also doesn't mean
C is the language used for implementation.

In other words, if the language implementing the function
implements/understands immutable, then why should it be disallowed to pass
immutable to an extern(C) function?  Same goes for ref, scope, in, out.

If something is passed in a register vs on the stack, that's part of the
calling convention, no?  It doesn't actually affect the parameter type or its
properties.  That's what I meant for 'nothing to do with parameters'.  Maybe
that was a misleading statement...


 The confusion comes because there are some druntime functions which use C name
 mangling but the extern(D) calling convention! This is not true of all
 extern(C) functions.


Wait, how can a function marked extern(C) not use C calling convention?  Are
these special-cased compiler functions?  If that's the case, I don't see how
those exceptions should affect the rules of extern(C), it doesn't affect
anything outside the compiler/runtime.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 18 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854



--- Comment #12 from Don <clugdbug yahoo.com.au> 2012-04-18 10:59:56 PDT ---
(In reply to comment #11)

 (In reply to comment #10)

 (In reply to comment #9)

 No.  An extern(C) function does not mean it's a C function.  It just means it
 has C linkage.  See here: http://dlang.org/attribute.html#linkage
 
 extern(C) has nothing to do with parameters, only calling conventions.


 
 I don't understand that statement. Do you mean 'parameters, only name mangling"
 ? If so, that that isn't true.


 
 No, I mean the extern(C) designates linkage and calling convention, it doesn't
 affect the parameters or qualifiers for those parameters.  It also doesn't mean
 C is the language used for implementation.


But C doesn't HAVE a calling convention for non-C types!

extern(C) void foo( void delegate() dg)
DOES NOT MAKE SENSE. It's not C, it's something else.

With one exception: if extern(C) means only "mangle as a C function" then it is
well defined for everything (because C doesn't mangle types). But I think
everybody expects it to include calling convention as well.

There are really two options:
(1) disallow non-C parameters in extern(C) functions
(2) extend the definition of extern(C) to specify what happens with non-C
parameters.


 The confusion comes because there are some druntime functions which use C name
 mangling but the extern(D) calling convention! This is not true of all
 extern(C) functions.


 
 Wait, how can a function marked extern(C) not use C calling convention?  Are
 these special-cased compiler functions?  


Yes. In those special cases, extern(C) just means C name mangling.
I think that's an indicator of how vague the spec is.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 18 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854



--- Comment #13 from Steven Schveighoffer <schveiguy yahoo.com> 2012-04-18
11:23:14 PDT ---
(In reply to comment #12)

 
 But C doesn't HAVE a calling convention for non-C types!
 
 extern(C) void foo( void delegate() dg)
 DOES NOT MAKE SENSE. It's not C, it's something else.


I would expect that in this case, whatever rules are established for passing
custom types of size dg.sizeof would apply.  I would never have guessed that
calling convention concerned itself with types or how to pass specific types. 
I thought it had to do only with where the arguments are placed, what order
they are passed, etc.

But does C even *have* a defined calling convention?  I mean even on the same
platform (Microsoft CRT) you can use any of 6 calling conventions
http://msdn.microsoft.com/en-us/library/984x0h58%28v=vs.80%29.aspx

I also note that none of those specify anything about how types affect the
calls.


 There are really two options:
 (1) disallow non-C parameters in extern(C) functions
 (2) extend the definition of extern(C) to specify what happens with non-C
 parameters.


I think option 1 is too damaging to current code.  Option 2 is fine, as long as
it doesn't get into describing the actual calling convention (which is platform
specific).  e.g. say ref parameters are passed as if the address of the
parameter was passed.


 Yes. In those special cases, extern(C) just means C name mangling.
 I think that's an indicator of how vague the spec is.


That sounds unreasonably confusing.  What benefit do we have for specially
treating those functions?  I'd rather see a pragma(nomangle), or have the
functions actually use C calling convention.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 18 2012
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7854



--- Comment #14 from Don <clugdbug yahoo.com.au> 2012-04-18 21:29:37 PDT ---
(In reply to comment #13)

 (In reply to comment #12)

 
 But C doesn't HAVE a calling convention for non-C types!
 
 extern(C) void foo( void delegate() dg)
 DOES NOT MAKE SENSE. It's not C, it's something else.


 
 I would expect that in this case, whatever rules are established for passing
 custom types of size dg.sizeof would apply.


There's no such rule. Currently DMD casts a delegate to a long (for 32 bits) or
a cent (for 64 bits) when passing it as a parameter. If it passed it as a
struct, the convention would be different.
So there are a couple of reasonable choices, but they aren't documented.


 But does C even *have* a defined calling convention?  I mean even on the same
 platform (Microsoft CRT) you can use any of 6 calling conventions
 http://msdn.microsoft.com/en-us/library/984x0h58%28v=vs.80%29.aspx


Yes, it's defined, though it is OS dependent. It's consistent among all Windows
compilers (excluding gcc, which refuses to cooperate).

Note that (for example) the Windows calling convention doesn't support varargs.
So in the Windows API everything used extern(Windows) except for a single
function which used extern(C).
There is however no defined calling convention for C++. 



 There are really two options:
 (1) disallow non-C parameters in extern(C) functions
 (2) extend the definition of extern(C) to specify what happens with non-C
 parameters.


 
 I think option 1 is too damaging to current code.  Option 2 is fine, as long as
 it doesn't get into describing the actual calling convention (which is platform
 specific).  e.g. say ref parameters are passed as if the address of the
 parameter was passed.


You _always_ need to know what it is doing. There may need to be a
platform-specific description. It can describe things in terms of the C
convention (eg, "delegates are passed as a struct" or "array slices are passed
as a unsigned long long".


 Yes. In those special cases, extern(C) just means C name mangling.
 I think that's an indicator of how vague the spec is.


 
 That sounds unreasonably confusing.  What benefit do we have for specially
 treating those functions?  I'd rather see a pragma(nomangle), or have the
 functions actually use C calling convention.


I think it's just a hack.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 18 2012