www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - extern(C) in druntime

reply Steven Schveighoffer <schveiguy yahoo.com> writes:
Opinion from those in the know:

When extern(C) is used in druntime, my understanding is that the reason 
is twofold:

1. Because the compiler requires certain symbols for hooks.
2. To get around some requirements that "druntime knows better", such as 
purity for things like memory allocation that touch global state.

Is this true? There was a heated debate on a bug report [1] about 
extern(C) parameter definitions allowing D features, such as ref. Of 
course, C has no knowledge of ref, but it works like passing a pointer.

Now, if the above is true, I would propose that we migrate druntime to 
not use extern(C), but instead use pragma(mangle). The reason is simple 
-- extern(C) should behave like C. But there are some things that aren't 
the same. The one being debated heatedly was passing a fixed size array. 
An example:

C:
int pipe(int fd[2]);

Note that in C, static arrays are passed *by reference*. If we copy this 
to d:

extern(C) int pipe(int[2] fd);

This currently means pass fd *by value*. I think to do otherwise would 
be confusing. So the correct (IMO) thing to do is:

extern(C) int pipe(ref int[2] fd);

However, this doesn't sit right with many people who expect extern(C) to 
mean "call this function as C would call it".

But we have also as I said above, "hijacked" extern(C) to take advantage 
of the non-mangling features. In those cases, all those functions are 
not meant to be called from C, only from D. And they are all implemented 
in D. I don't want to limit those functions to only things that C supports.

Would it make sense to change to use pragma(mangle)? or is there some 
other reason why we use extern(C) in druntime? I'm asking the old 
veterans who may know some of the history, i.e. Walter, Sean, etc.

If we go this route, we can put more restrictions on actual extern(C) 
functions, and require the proper marking of functions. For example, 
passing a static array without ref should really be an error.

-Steve

[1] https://issues.dlang.org/show_bug.cgi?id=8887
Nov 10 2014
next sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
Also, eliminating compile time dependencies between namespaces. 
Everything can import core.*, but nothing outside rt is allowed 
to import rt modules and nothing outside gc is allowed to import 
gc modules. This allows us to theoretically choose the compiler 
runtime and GC at link time and also allows us not to put this 
code in the import tree when distributing druntime.  In short, 
extern (C) is often used because you can't forward declare extern 
(D) functions.  It doesn't mean anything about the language used 
by the function.
Nov 10 2014
parent reply "Dicebot" <public dicebot.lv> writes:
???

$ cat a.d
module a;

pragma(mangle, "foo")
extern(D) void foo()
{
     import std.stdio;
     writeln("in extern foo");
}

$ cat b.d
module b;

pragma(mangle, "foo")
extern void foo();

void main()
{
     foo();
}

$ ./a
in extern foo
Nov 10 2014
parent reply "Dicebot" <public dicebot.lv> writes:
This was an answer to "you can't forward declare extern
(D) functions"
Nov 10 2014
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/10/14 10:40 AM, Dicebot wrote:
 This was an answer to "you can't forward declare extern
 (D) functions"
I think Sean's point is that is another historical reason why we used extern(C), back when pragma(mangle) didn't exist. -Steve
Nov 10 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 10 November 2014 at 15:52:56 UTC, Steven Schveighoffer 
wrote:
 On 11/10/14 10:40 AM, Dicebot wrote:
 This was an answer to "you can't forward declare extern
 (D) functions"
I think Sean's point is that is another historical reason why we used extern(C), back when pragma(mangle) didn't exist.
Yep. Since some D types don't exist in C and so have to be passed differently by convention (static arrays), I think an argument could be made that we should assume that anything behind an extern (C) interface is in fact C code. But that's not how it's treated today, where extern (C) is simply another calling convention.
Nov 10 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 10 November 2014 at 16:28:11 UTC, Sean Kelly wrote:
 I think Sean's point is that is another historical reason why 
 we used extern(C), back when pragma(mangle) didn't exist.
Yep. Since some D types don't exist in C and so have to be passed differently by convention (static arrays), I think an argument could be made that we should assume that anything behind an extern (C) interface is in fact C code. But that's not how it's treated today, where extern (C) is simply another calling convention.
This was exactly what I was proposing in https://issues.dlang.org/show_bug.cgi?id=8887 (with possibly making any differences compile-time error initially for soft transition) Right now question is, however, are there any legitimate uses of `extern(C)` in druntime or those all can be replaced with `pragma(mangle, "XXX") extern(D)`?
Nov 10 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 10 November 2014 at 16:59:47 UTC, Dicebot wrote:
 Right now question is, however, are there any legitimate uses 
 of `extern(C)` in druntime or those all can be replaced with 
 `pragma(mangle, "XXX") extern(D)`?
I don't think there's any reason to use extern (C) in druntime any more.
Nov 10 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 10 November 2014 at 17:48:04 UTC, Sean Kelly wrote:
 On Monday, 10 November 2014 at 16:59:47 UTC, Dicebot wrote:
 Right now question is, however, are there any legitimate uses 
 of `extern(C)` in druntime or those all can be replaced with 
 `pragma(mangle, "XXX") extern(D)`?
I don't think there's any reason to use extern (C) in druntime any more.
I guess that nails it, will make a PR
Nov 10 2014
parent reply "Logan Capaldo" <logancapaldo gmail.com> writes:
On Monday, 10 November 2014 at 19:44:05 UTC, Dicebot wrote:
 On Monday, 10 November 2014 at 17:48:04 UTC, Sean Kelly wrote:
 On Monday, 10 November 2014 at 16:59:47 UTC, Dicebot wrote:
 Right now question is, however, are there any legitimate uses 
 of `extern(C)` in druntime or those all can be replaced with 
 `pragma(mangle, "XXX") extern(D)`?
I don't think there's any reason to use extern (C) in druntime any more.
I guess that nails it, will make a PR
What about things that are "meant" to be called from C? E.g. `rt_init`/`rt_term`?
Nov 10 2014
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/10/14 5:15 PM, Logan Capaldo wrote:
 On Monday, 10 November 2014 at 19:44:05 UTC, Dicebot wrote:
 On Monday, 10 November 2014 at 17:48:04 UTC, Sean Kelly wrote:
 On Monday, 10 November 2014 at 16:59:47 UTC, Dicebot wrote:
 Right now question is, however, are there any legitimate uses of
 `extern(C)` in druntime or those all can be replaced with
 `pragma(mangle, "XXX") extern(D)`?
I don't think there's any reason to use extern (C) in druntime any more.
I guess that nails it, will make a PR
What about things that are "meant" to be called from C? E.g. `rt_init`/`rt_term`?
Those should remain extern(C). Of course, those have no "D only" features, so they can easily remain extern(C) :) -Steve
Nov 10 2014
parent reply "Logan Capaldo" <logancapaldo gmail.com> writes:
On Monday, 10 November 2014 at 23:02:54 UTC, Steven Schveighoffer 
wrote:
 On 11/10/14 5:15 PM, Logan Capaldo wrote:
 On Monday, 10 November 2014 at 19:44:05 UTC, Dicebot wrote:
 On Monday, 10 November 2014 at 17:48:04 UTC, Sean Kelly wrote:
 On Monday, 10 November 2014 at 16:59:47 UTC, Dicebot wrote:
 Right now question is, however, are there any legitimate 
 uses of
 `extern(C)` in druntime or those all can be replaced with
 `pragma(mangle, "XXX") extern(D)`?
[snip]
 What about things that are "meant" to be called from C? E.g.
 `rt_init`/`rt_term`?
Those should remain extern(C). Of course, those have no "D only" features, so they can easily remain extern(C) :)
So just to be clear, there are _some_ legitimate uses of extern (C) in druntime, yes? rt_init/rt_term, rt_loadLibrary, thread_init(? think this one can be bootstrapped from D code), ...?
Nov 10 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 10 November 2014 at 23:08:55 UTC, Logan Capaldo wrote:
 So just to be clear, there are _some_ legitimate uses of extern 
 (C) in druntime, yes? rt_init/rt_term, rt_loadLibrary, 
 thread_init(? think this one can be bootstrapped from D code), 
 ...?
Yes. There are a few functions meant to be callable from C code: rt_init, rt_term, thread_attachThis, thread_detachThis, etc. These could either be exposed as wrappers on top of extern (D) functions or left as-is.
Nov 10 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 10 November 2014 at 23:22:00 UTC, Sean Kelly wrote:
 On Monday, 10 November 2014 at 23:08:55 UTC, Logan Capaldo 
 wrote:
 So just to be clear, there are _some_ legitimate uses of 
 extern (C) in druntime, yes? rt_init/rt_term, rt_loadLibrary, 
 thread_init(? think this one can be bootstrapped from D code), 
 ...?
Yes. There are a few functions meant to be callable from C code: rt_init, rt_term, thread_attachThis, thread_detachThis, etc. These could either be exposed as wrappers on top of extern (D) functions or left as-is.
Well this is exactly what makes somewhat uneasy about making the change myself - it isn't immediately obvious which functions are legitimately extern(C) and which have it only for mangling / forward declaration.
Nov 11 2014
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/11/14 11:01 AM, Dicebot wrote:
 On Monday, 10 November 2014 at 23:22:00 UTC, Sean Kelly wrote:
 On Monday, 10 November 2014 at 23:08:55 UTC, Logan Capaldo wrote:
 So just to be clear, there are _some_ legitimate uses of extern (C)
 in druntime, yes? rt_init/rt_term, rt_loadLibrary, thread_init(?
 think this one can be bootstrapped from D code), ...?
Yes. There are a few functions meant to be callable from C code: rt_init, rt_term, thread_attachThis, thread_detachThis, etc. These could either be exposed as wrappers on top of extern (D) functions or left as-is.
Well this is exactly what makes somewhat uneasy about making the change myself - it isn't immediately obvious which functions are legitimately extern(C) and which have it only for mangling / forward declaration.
I think make the PR, and we can debate which ones shouldn't be changed. I would also at least mark these ones that shouldn't be changed as "intended to be called from C/C++" with a comment to prevent accidental changes in the future. -Steve
Nov 11 2014
prev sibling parent Martin Nowak <code+news.digitalmars dawg.eu> writes:
On 11/10/2014 03:48 PM, Steven Schveighoffer wrote:
 Opinion from those in the know:

 When extern(C) is used in druntime, my understanding is that the reason
 is twofold:

 1. Because the compiler requires certain symbols for hooks.
 2. To get around some requirements that "druntime knows better", such as
 purity for things like memory allocation that touch global state.
Sometimes that might be required, but those "hacks" should be on the implementation not the call site. I made a template to bind extern(D) function and I'd like to see it replacing most of the extern(C) bindings. It allows typesafe and attribute-correct bindings and things break if they aren't kept in sync. https://github.com/D-Programming-Language/druntime/search?q=externDFunc
Nov 14 2014