www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Philosophy of how OS API imports are laid out in druntime

reply Walter Bright <newshound2 digitalmars.com> writes:
This is an important debate going on here:

https://github.com/D-Programming-Language/druntime/pull/732

It has a wide impact, and so I'm bringing it up here so everyone can
participate.
Mar 03 2014
next sibling parent reply "Craig Dillabaugh" <cdillaba cg.scs.carleton.ca> writes:
On Tuesday, 4 March 2014 at 00:09:47 UTC, Walter Bright wrote:
 This is an important debate going on here:

 https://github.com/D-Programming-Language/druntime/pull/732

 It has a wide impact, and so I'm bringing it up here so 
 everyone can participate.

You mean the debate about copyright notices :o)
Mar 04 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/4/2014 7:53 AM, Craig Dillabaugh wrote:
 On Tuesday, 4 March 2014 at 00:09:47 UTC, Walter Bright wrote:
 This is an important debate going on here:

 https://github.com/D-Programming-Language/druntime/pull/732

 It has a wide impact, and so I'm bringing it up here so everyone can
participate.

You mean the debate about copyright notices :o)

No, see Sean's post.
Mar 04 2014
prev sibling next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Tuesday, 4 March 2014 at 00:09:47 UTC, Walter Bright wrote:
 This is an important debate going on here:

 https://github.com/D-Programming-Language/druntime/pull/732

 It has a wide impact, and so I'm bringing it up here so 
 everyone can participate.

I think the current debate is more a result of miscommunication than disagreement. Or at least I hope it is, as I simply don't understand your perspective here. For those who haven't seen it yet, the subject of the discussion is where the D version of C header files currently in core.stdc and core.sys.* should live and how they should be exposed to users. I came up with the current design and so I obviously favor the status quo, and I stand by my original reasons for the design (explained in the thread).
Mar 04 2014
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
On Tuesday, 4 March 2014 at 00:09:47 UTC, Walter Bright wrote:
 This is an important debate going on here:

 https://github.com/D-Programming-Language/druntime/pull/732

 It has a wide impact, and so I'm bringing it up here so 
 everyone can participate.

So, if we combine your, Sean's and yebblies' proposals, we will have 1) ports folder with implementation subfolders for each os and their translated headers - this works better with your "copy what I need" use case - just copy your os's folder and you can work, you can't copy what you need if headers for the same os are spread across many folders. 2) public module core.stdc will conditionally import ports.os.* for platform-specific definitions compliant with C standard 3) public module core.sys.windows - it may depend on stdc and thus import ports.windows.* 4) public module core.sys.posix - for everything else, which corresponds to headers seen in unixes and conditionally imports ports.os.*, it may have a folder for windows for things like unistd if those didn't fall under stdc. 5) we probably shouldn't have cocoa and gtk?
Mar 05 2014
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
On Wednesday, 5 March 2014 at 10:37:03 UTC, Kagamin wrote:
 2) public module core.stdc will conditionally import ports.os.* 
 for platform-specific definitions compliant with C standard

different c libraries.
Mar 05 2014
prev sibling next sibling parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Tue, 04 Mar 2014 00:09:46 -0000, Walter Bright  
<newshound2 digitalmars.com> wrote:

 This is an important debate going on here:

 https://github.com/D-Programming-Language/druntime/pull/732

 It has a wide impact, and so I'm bringing it up here so everyone can  
 participate.

The disagreement here seems to boil down to two competing goals. 1. Walter wants the C include to map directly to a D import. 2. Sean wants to be able to ensure he does not import and use a platform specific function/definiton in a cross platform application. Is that about right? To clarify some points.. Walter are you asking for ALL includes even #include <windows.h> to map? OR, are you asking for only those headers thought to be "cross platform" headers to map? For example, <sys/ioctl.h> is not a windows header and would not be considered "cross platform". I have the following include folders in Visual Studio 2008: C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\include - posix, c[++] std library headers C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\atlmfc\include - ATL/MFC headers C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Include - windows specific headers would you only expect to map headers from the first of those, and not the rest? That first folder contains 184 files which correlate to posix, and c[++] standard library headers. Sean to achieve your ends you are currently importing core.sys.posix.* right? So, any modules added to core.* would not affect you? I presume the modules in core.sys.* are cut down versions of the headers in core.sys.linux etc with any the platform specific definitions removed, yes? So, if we currently have the following layout: [the root folders] core\stdc core\sync core\sys [the platform specific tree] core\sys\freebsd core\sys\freebsd\sys core\sys\linux core\sys\linux\sys core\sys\osx core\sys\osx\mach core\sys\windows [the posix tree] core\sys\posix core\sys\posix\arpa core\sys\posix\net core\sys\posix\netinet core\sys\posix\sys Note; I think that "sys" folder in core is unnecessary and may be causing some confusion. Why not have a "c" folder to separate the C modules from other core components. I mean, why isn't stdc in sys? So, anyway, could we not simply add modules as Walter described to core.* and core.sys.* to map to the specific platform and header for the build system? Likewise we would want to map core.* to core.stdc.* where appropriate. It seems this will satisfy Walter without impacting Sean.. other than being loads of "unnecessary" modules from your perspective. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Mar 05 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/5/2014 9:16 AM, Regan Heath wrote:
 1. Walter wants the C include to map directly to a D import.
 2. Sean wants to be able to ensure he does not import and use a platform
 specific function/definiton in a cross platform application.

 Is that about right?

Yes.
 To clarify some points..

  Walter are you asking for ALL includes even #include <windows.h> to map?

Yes. Of course, if you're on linux and attempt to import core.windows, you should expect a compilation failure, just as you would in C with: #include <windows.h>
 For example, <sys/ioctl.h> is not a windows header and would not be considered
 "cross platform".

I'd expect: import core.sys.ioctl; to work on linux systems and fail to compile on windows systems. Just as one would expect in C with: #include <sys/ioctl.h>
 I have the following include folders in Visual Studio 2008:
 C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\include        - posix,
 c[++] std library headers
 C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\atlmfc\include - ATL/MFC
 headers
 C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Include          - windows
 specific headers

 would you only expect to map headers from the first of those, and not the rest?

The first, yes, ATL and MFC are not usable with D, and windows, yes.
 That first folder contains 184 files which correlate to posix, and c[++]
 standard library headers.

We wouldn't need the posix or C++ ones.
 It seems this will satisfy Walter without impacting Sean.. other than being
 loads of "unnecessary" modules from your perspective.

Any install on Linux should be able to delete any freebsd.d, osx, windows files with impunity, etc.
Mar 06 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/6/2014 2:32 PM, Vladimir Panteleev wrote:
 First of all, even considering the idea of putting everything in one package,
it
 should not be "core". There are many D-specific things in "core" right now.
It's
 probably best to keep OS includes and D stuff differently.

Sure. The package name isn't important, it's what follows that is.
 Second, I think that the different ways with how C and D approach headers (list
 of include paths vs. module system and package hierarchy) warrants considering
 another solution. For example, at least on Windows, the compiler consults at
 least two include directories: the C runtime includes (for declarations that
are
 part of C and whatever part of POSIX emulation is included), and the Windows
 SDK. Sometimes, components migrate from one project to another (e.g. DirectShow
 was moved out of the DirectX SDK and into the Windows Platform SDK). Thus I
 think it would be preferable to keep things organized in packages rather than
 flattening everything together.

What I want to see is a strong correspondence between what a C coder would write when #including system files, and the corresponding D import. What I object to are: 1. Trying to reinvent, improve, refactor, fix, clean up, etc., the OS api. This is not in D's charter. We have no resources to do it properly anyway. 2. Trying to split C's header files into separate posix / nonposix modules. As in (1), this is beyond the scope of what D is trying to do. 3. Pushing versioning into user code, as in: version (linux) import os.linux.whatever; else version (FreeBSD) import os.freebsd.whatever; ... else static assert(0); as opposed to: import os.whatever; I understand the sentiment that this will signal to the user that he's using non-portable imports, but I don't think it's worth it. Portability is what Phobos is for, not OS api interfaces. 4. Attempts to eliminate code duplication in OS api interface modules. A desire to do this comes up repeatedly. I can expand on this if anyone remains unconvinced :-)
Mar 06 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/6/2014 3:51 PM, Iain Buclaw wrote:
 On 6 March 2014 22:53, Walter Bright <newshound2 digitalmars.com> wrote:
 1. Trying to reinvent, improve, refactor, fix, clean up, etc., the OS api.
 This is not in D's charter. We have no resources to do it properly anyway.

I don't think that is what druntime does in terms of it's C interfacing modules.

Usually not. But proposals to do such come up now and then - I just want to head these off before anyone invests effort in it.
 What it does do is make it more logical for
 people to find what they are looking for, or if they want to make
 changes, which areas to look at.

Eh, "more logical" is a trap. I'd prefer a simple 1:1 correspondence between C and D for OS api's. Then there's nothing clever about it, no decisions to make, and the user familiar with C will know where it is.
 2. Trying to split C's header files into separate posix / nonposix modules.
 As in (1), this is beyond the scope of what D is trying to do.

I think the word posix seems to be the cesspool that's repeatedly brought up here to fan the flames here. What about the linux/freebsd/osx/windows packages?

Eliminate the linux/freebsd/osx/windows package inventions. I don't see in C code: #include <sys/linux/sys/foo.h> so why should we do that? Again, what I propose is really simple from a user point of view - a 1:1 mapping between what he'd type in for C vs what he'd type in for D.
 3. Pushing versioning into user code, as in:

      version (linux) import os.linux.whatever;
      else version (FreeBSD) import os.freebsd.whatever;
      ...
      else static assert(0);

 as opposed to:

      import os.whatever;

 I understand the sentiment that this will signal to the user that he's using
 non-portable imports, but I don't think it's worth it. Portability is what
 Phobos is for, not OS api interfaces.

This is something that needs to be worked on, but I'm not convinced a flat hierachy will solve anything.

What it will solve is when the user reads the OS api documentation, and it says: #include <sys/ioctl.h> then the user knows that in D he has to do: import whatever.sys.ioctl; What the user should not have to do is: version (linux) import whatever.linux.sys.ioctl; else version (FreeBSD) import whatever.freebsd.sys.ioctl; else version (OSX) import whatever.osx.sys.ioctl; ... else static assert(0); nor: import whatever.posix.sys.ioctl; version (linux) import whatever.linux.sys.ioctl; else version (FreeBSD) import whatever.freebsd.sys.ioctl; else version (OSX) import whatever.osx.sys.ioctl; ... else static assert(0);
Mar 06 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/6/2014 5:10 PM, Sean Kelly wrote:
 However, as much as I like that having the Posix headers strongly
 encourages the user to write portable code, I can see the idea
 doesn't have a lot of traction around here.  I beleive Iain's
 proposal is to phase them out and do exactly what you suggest,

Thanks, Sean, I appreciate the graciousness of your reply.
 only to make "whatever" "core.foo" rather than simply "core".
 Which works for me as well.

I don't have any axe to grind about what the "whatever" expands to, I'm cool with either.
Mar 06 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/7/2014 12:00 AM, Iain Buclaw wrote:
 I assume the C headers will remain strictly under core.stdc?  So it
 might make sense to transition core.sys -> core.stdc.sys?

I think core.stdc for the Standard C headers can remain the same.
Mar 07 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/7/2014 1:03 AM, Iain Buclaw wrote:
 On 7 March 2014 08:34, Walter Bright <newshound2 digitalmars.com> wrote:
 On 3/7/2014 12:00 AM, Iain Buclaw wrote:
 I assume the C headers will remain strictly under core.stdc?  So it
 might make sense to transition core.sys -> core.stdc.sys?

I think core.stdc for the Standard C headers can remain the same.

OK - now that has been sorted. What's your opinion on where ports of standard C and system C headers should go?

Standard C => core.stdc Operating system API = > core? core.sys? core.os? I don't have strong feelings about that.
Mar 07 2014
parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Iain Buclaw"  wrote in message 
news:mailman.27.1394213489.25740.digitalmars-d puremagic.com...

 Arch, arch, arch! Platform ports are pretty much covered by Walter and
 Sean's agreement. :)

Arch, arch, arch! https://www.youtube.com/watch?v=5-LIqdjqHts
Mar 07 2014
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 5 March 2014 17:16, Regan Heath <regan netmail.co.nz> wrote:
 On Tue, 04 Mar 2014 00:09:46 -0000, Walter Bright
 <newshound2 digitalmars.com> wrote:

 This is an important debate going on here:

 https://github.com/D-Programming-Language/druntime/pull/732

 It has a wide impact, and so I'm bringing it up here so everyone can
 participate.

The disagreement here seems to boil down to two competing goals. 1. Walter wants the C include to map directly to a D import. 2. Sean wants to be able to ensure he does not import and use a platform specific function/definiton in a cross platform application. Is that about right?

3. Iain wants to be able to ensure ports of druntime (ARM, MIPS, SPARC, etc...) are conveniently - as in not complex - split up without introducing a new namespace. :o)
Mar 05 2014
prev sibling next sibling parent "MattCoder" <somekindofmonster email.com.br> writes:
On Tuesday, 4 March 2014 at 20:50:40 UTC, Sean Kelly wrote:
 On Tuesday, 4 March 2014 at 00:09:47 UTC, Walter Bright wrote:
 This is an important debate going on here:

 https://github.com/D-Programming-Language/druntime/pull/732

 It has a wide impact, and so I'm bringing it up here so 
 everyone can participate.

I think the current debate is more a result of miscommunication than disagreement. Or at least I hope it is, as I simply don't understand your perspective here. For those who haven't seen it yet, the subject of the discussion is where the D version of C header files currently in core.stdc and core.sys.* should live and how they should be exposed to users. I came up with the current design and so I obviously favor the status quo, and I stand by my original reasons for the design (explained in the thread).

I agree with "complexmath" POV too. PS: By the way, look at the issue number (11666). Well 666 so of course it would generate some disagreement. :) Matheus.
Mar 05 2014
prev sibling next sibling parent "Regan Heath" <regan netmail.co.nz> writes:
On Wed, 05 Mar 2014 17:55:27 -0000, Iain Buclaw <ibuclaw gdcproject.org>  
wrote:

 On 5 March 2014 17:16, Regan Heath <regan netmail.co.nz> wrote:
 On Tue, 04 Mar 2014 00:09:46 -0000, Walter Bright
 <newshound2 digitalmars.com> wrote:

 This is an important debate going on here:

 https://github.com/D-Programming-Language/druntime/pull/732

 It has a wide impact, and so I'm bringing it up here so everyone can
 participate.

The disagreement here seems to boil down to two competing goals. 1. Walter wants the C include to map directly to a D import. 2. Sean wants to be able to ensure he does not import and use a platform specific function/definiton in a cross platform application. Is that about right?

3. Iain wants to be able to ensure ports of druntime (ARM, MIPS, SPARC, etc...) are conveniently - as in not complex - split up without introducing a new namespace. :o)

Sorry. Missed that requirement :) I like your last idea re transitioning away from core.sys.posix.* by using version(Posix). Presumably, in the case of modules which contain POSIX and non-POSIX definition we would wrap those in version blocks also. I think if we add the mapping modules as Walter suggested then to split the runtime for a specific platform (which GCC requires?) then you would copy the modules in core.* and core.sys.* and then the core.sys.<platform>.*. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Mar 06 2014
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
On Wednesday, 5 March 2014 at 17:16:34 UTC, Regan Heath wrote:
 It seems this will satisfy Walter without impacting Sean..

As I understand, the idea is that Sean get little trying to fix posix standard: the only way to check if the code works on some platform is to compile and test it on that platform and posix standard doesn't change that. So various platforms ended up adding new functions to posix headers. Having straightforward translations of headers takes less thinking and probably helps migrate from C and doesn't change posix compliance of the headers - it remains conventional.
Mar 06 2014
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
It can be a module pragma:

pragma(restrictImportTo,"core.sys.posix.ucontext")
module ports.linux.ucontext;
Mar 06 2014
prev sibling next sibling parent "Regan Heath" <regan netmail.co.nz> writes:
On Thu, 06 Mar 2014 11:17:36 -0000, Kagamin <spam here.lot> wrote:

 On Wednesday, 5 March 2014 at 17:16:34 UTC, Regan Heath wrote:
 It seems this will satisfy Walter without impacting Sean..

As I understand, the idea is that Sean get little trying to fix posix standard: the only way to check if the code works on some platform is to compile and test it on that platform and posix standard doesn't change that. So various platforms ended up adding new functions to posix headers. Having straightforward translations of headers takes less thinking and probably helps migrate from C and doesn't change posix compliance of the headers - it remains conventional.

Sure. The core.sys.<platform>.* modules are/will be straight translations. Walter wants additional core.* and core.sys.* modules which map to core.sys.<platform>.* as appropriate. Sean wants/uses core.sys.posix.* modules, which are maintained by someone and only contain posix definitions for all platforms. Iain wants to be able to split a single platform easily from the rest; taking core.* core.sys.* and core.sys.<platform>.*. Right? R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Mar 06 2014
prev sibling next sibling parent "Regan Heath" <regan netmail.co.nz> writes:
On Thu, 06 Mar 2014 11:40:55 -0000, Kagamin <spam here.lot> wrote:

 It can be a module pragma:

 pragma(restrictImportTo,"core.sys.posix.ucontext")
 module ports.linux.ucontext;

Good idea, then the platform specific modules can only define the platform specific things. But, it means when maintaining them you inherently have to consider posix, making the burden a little higher. Which runs counter to Walter's goal I think. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Mar 06 2014
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 6 March 2014 14:37, Regan Heath <regan netmail.co.nz> wrote:
 On Thu, 06 Mar 2014 11:17:36 -0000, Kagamin <spam here.lot> wrote:

 On Wednesday, 5 March 2014 at 17:16:34 UTC, Regan Heath wrote:
 It seems this will satisfy Walter without impacting Sean..

As I understand, the idea is that Sean get little trying to fix posix standard: the only way to check if the code works on some platform is to compile and test it on that platform and posix standard doesn't change that. So various platforms ended up adding new functions to posix headers. Having straightforward translations of headers takes less thinking and probably helps migrate from C and doesn't change posix compliance of the headers - it remains conventional.

Sure. The core.sys.<platform>.* modules are/will be straight translations. Walter wants additional core.* and core.sys.* modules which map to core.sys.<platform>.* as appropriate. Sean wants/uses core.sys.posix.* modules, which are maintained by someone and only contain posix definitions for all platforms. Iain wants to be able to split a single platform easily from the rest; taking core.* core.sys.* and core.sys.<platform>.*. Right?

Correct on my part. :)
Mar 06 2014
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Thursday, 6 March 2014 at 21:35:34 UTC, Walter Bright wrote:
 To clarify some points..

  Walter are you asking for ALL includes even #include 
 <windows.h> to map?

Yes. Of course, if you're on linux and attempt to import core.windows, you should expect a compilation failure, just as you would in C with: #include <windows.h>

I'm not sure about this. First of all, even considering the idea of putting everything in one package, it should not be "core". There are many D-specific things in "core" right now. It's probably best to keep OS includes and D stuff differently. Consider, for example, the D module "core.time". POSIX has the header <time.h>. You have a collision right there. Second, I think that the different ways with how C and D approach headers (list of include paths vs. module system and package hierarchy) warrants considering another solution. For example, at least on Windows, the compiler consults at least two include directories: the C runtime includes (for declarations that are part of C and whatever part of POSIX emulation is included), and the Windows SDK. Sometimes, components migrate from one project to another (e.g. DirectShow was moved out of the DirectX SDK and into the Windows Platform SDK). Thus I think it would be preferable to keep things organized in packages rather than flattening everything together.
Mar 06 2014
prev sibling next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
The conversation began with Iain talking about how to improve
things from a maintenance perspective and was kind of sidetracked
by my and Walter's discussion.  I agree that something should be
done to address Iain's issue and think his suggestions sound
pretty good.  But concerning the rest...

Druntime has always been intended to contain the runtime code,
public interfaces to that runtime, plus core functionality that
could be considered essential or intrinsic to the language, a bit
like java.lang.

Having C headers there at all has always been more a matter of
necessity than design, since Druntime needs them for its own use.
   To my knowledge, it's always been a goal to minimize or at least
gently discourage direct use of C routines.  So the declarations
are there when needed, but tucked away in identifiable packages
so their use in an application can be easily determined via grep.
   This also serves an an indicator of where D might be lacking
from a functional perspective, as I don't think we want to
provide a standard library where users feel the need to lean on
another language's standard library.  Thus using libc as a
resource for the implementation of Druntime is more an artifact
of the age of the language than by design, though I'll grant that
it's nearly impossible to get away from using at least kernel
calls in many cases (I think we could absolutely do away with all
use of standard C routines).

My primary concern with Walter's proposal is that it violates the
principal I outlined above by placing the C headers front and
center in core.  It would effectively turn Druntime into a C
interface library, which is vastly different than its original
design goals.  If this is the intent, I propose creating a new
project on github for this purpose.  Otherwise, perhaps some
location beneath the root core package could be chosen for these
to live?  Perhaps just publish everything to core.sys, since that
package already exists?  I admit to completely not understanding
the claim that declarations are hard to find.  But then I
designed the current layout so it clearly makes sense to me.

As far as maintenance is concerned, while I do appreciate that it
becomes untenable to hold the declarations for every platform in
a single module as we have now, say, in core.stdc, I question the
claim that direct conversion of a platform's header files is
somehow less work than cherry-picking declarations according to
some standard.  As I've said before, I believe that automatic
conversion of headers may be considered a copyright violation (by
my reading of the license discussion on the Boost website), and
so manual conversion is our only option.  Given this, and having
all of the work creating and maintaining core.stdc and
core.sys.posix until a few years ago, I would have lost my mind
if I'd had to convert entire headers as-is rather than
cherry-picking declarations.  These headers are so big and so
filled with preprocessor macros and such that replicating the
entirety of what they contained in D code would have been a
nightmare.  But perhaps someone has a good answer for this that I
missed.  I certainly have no desire to maintain these headers in
any manner.  The current approach was simply the one I found that
required the least work.  If there's a way to do it with even
less then I'm all for it.

So I apologize for my part in derailing Iain's discussion.  I
suspect that there's a straightforward solution that's simply
been overlooked from Walter and me talking at cross-purposes.
Mar 06 2014
prev sibling next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 6 March 2014 at 22:53:40 UTC, Walter Bright wrote:
 4. Attempts to eliminate code duplication in OS api interface 
 modules. A desire to do this comes up repeatedly. I can expand 
 on this if anyone remains unconvinced :-)

I 100% agree with this. Trying to avoid code duplication here leads to madness.
Mar 06 2014
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 6 March 2014 22:53, Walter Bright <newshound2 digitalmars.com> wrote:
 On 3/6/2014 2:32 PM, Vladimir Panteleev wrote:
 First of all, even considering the idea of putting everything in one
 package, it
 should not be "core". There are many D-specific things in "core" right
 now. It's
 probably best to keep OS includes and D stuff differently.

Sure. The package name isn't important, it's what follows that is.
 Second, I think that the different ways with how C and D approach headers
 (list
 of include paths vs. module system and package hierarchy) warrants
 considering
 another solution. For example, at least on Windows, the compiler consults
 at
 least two include directories: the C runtime includes (for declarations
 that are
 part of C and whatever part of POSIX emulation is included), and the
 Windows
 SDK. Sometimes, components migrate from one project to another (e.g.
 DirectShow
 was moved out of the DirectX SDK and into the Windows Platform SDK). Thus
 I
 think it would be preferable to keep things organized in packages rather
 than
 flattening everything together.

What I want to see is a strong correspondence between what a C coder would write when #including system files, and the corresponding D import. What I object to are: 1. Trying to reinvent, improve, refactor, fix, clean up, etc., the OS api. This is not in D's charter. We have no resources to do it properly anyway.

I don't think that is what druntime does in terms of it's C interfacing modules. What it does do is make it more logical for people to find what they are looking for, or if they want to make changes, which areas to look at.
 2. Trying to split C's header files into separate posix / nonposix modules.
 As in (1), this is beyond the scope of what D is trying to do.

I think the word posix seems to be the cesspool that's repeatedly brought up here to fan the flames here. What about the linux/freebsd/osx/windows packages?
 3. Pushing versioning into user code, as in:

     version (linux) import os.linux.whatever;
     else version (FreeBSD) import os.freebsd.whatever;
     ...
     else static assert(0);

 as opposed to:

     import os.whatever;

 I understand the sentiment that this will signal to the user that he's using
 non-portable imports, but I don't think it's worth it. Portability is what
 Phobos is for, not OS api interfaces.

This is something that needs to be worked on, but I'm not convinced a flat hierachy will solve anything. Regards Iain
Mar 06 2014
prev sibling next sibling parent "Mike" <none none.com> writes:
On Tuesday, 4 March 2014 at 00:09:47 UTC, Walter Bright wrote:
 This is an important debate going on here:

 https://github.com/D-Programming-Language/druntime/pull/732

 It has a wide impact, and so I'm bringing it up here so 
 everyone can participate.

I originally posted issue 11666 after reading Iain's post here [1], as I am trying to port the D Runtime to a bare-metal platform and couldn't see the abstractions in the D Runtime. If this statement about Iain's stance is correct ("Iain wants to be able to split a single platform easily from the rest") then I am with Iain. However, I see from the whole posix/C debate, that D is having somewhat of an identity crisis. And I think if D could decide what it wants to be, the solution may present itself. *Question* Does D's implementation require an OS, or does the OS's implementation require a language like D? I refer you to a suggestion by Manu here [2]. "I prefer the idea already mentioned of making a more well-defined separation of D-core (static stuff), and then the rest as a higher layer." I think there should be 3 different abstractions in the D Runtime: 1. Language features. 2. OS features built on those core language features. 3. Language features built on those OS features Finally Phobos can be built on top of 3. Each one of those abstractions would require a platform's folder/package. How those folders/packages should be organized can be further debated, but without first making these abstractions, the discussion becomes too convoluted. My opinions: * D should not require C * D should not do things the C way * Some features of D should not require an OS destroy(me); Mike [1] http://forum.dlang.org/post/mailman.1526.1333966829.4860.digitalmars-d puremagic.com [2] http://forum.dlang.org/post/mailman.18.1392171489.6445.digitalmars-d puremagic.com
Mar 06 2014
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 6 March 2014 23:46, Sean Kelly <sean invisibleduck.org> wrote:
 On Thursday, 6 March 2014 at 22:53:40 UTC, Walter Bright wrote:
 4. Attempts to eliminate code duplication in OS api interface modules. A
 desire to do this comes up repeatedly. I can expand on this if anyone
 remains unconvinced :-)

I 100% agree with this. Trying to avoid code duplication here leads to madness.

Indeed, I don't recall a single port-related pull in the last several months that tried to do any refactoring. I think the general consensus among the current mantainers is that explicit versioning is good, vague/fallback else is bad. However on the flip side, the more ports you add into one single file, the more you descend back into the madness you were initially trying to avoid. I hope code duplication in many files rather than one file is in the interest of everyone. :)
Mar 06 2014
prev sibling next sibling parent "Mike" <none none.com> writes:
On Thursday, 6 March 2014 at 23:56:35 UTC, Iain Buclaw wrote:
 I hope code duplication in many files rather than one file is 
 in the
 interest of everyone. :)

The only people it doesn't benefit are the ones maintaining multiple ports :) I agree with Walter that "attempts to eliminate code duplication in OS api interface modules" is misguided. Software often has common implementations simply by coincidence, not because there is an inherent relationship between them. Mike
Mar 06 2014
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
--001a11c136aa50100904f3f928ee
Content-Type: text/plain; charset=ISO-8859-1

On Mar 7, 2014 12:10 AM, "Mike" <none none.com> wrote:
 On Thursday, 6 March 2014 at 23:56:35 UTC, Iain Buclaw wrote:
 I hope code duplication in many files rather than one file is in the
 interest of everyone. :)

The only people it doesn't benefit are the ones maintaining multiple


I disagree. It makes my work more organised, and it removes the trap of deduping without the hassle. Regards -- Iain Buclaw *(p < e ? p++ : p) = (c & 0x0f) + '0'; --001a11c136aa50100904f3f928ee Content-Type: text/html; charset=ISO-8859-1 <p><br> On Mar 7, 2014 12:10 AM, &quot;Mike&quot; &lt;<a href="mailto:none none.com">none none.com</a>&gt; wrote:<br> &gt;<br> &gt; On Thursday, 6 March 2014 at 23:56:35 UTC, Iain Buclaw wrote:<br> &gt;&gt;<br> &gt;&gt; I hope code duplication in many files rather than one file is in the<br> &gt;&gt; interest of everyone. :)<br> &gt;<br> &gt;<br> &gt; The only people it doesn&#39;t benefit are the ones maintaining multiple ports :)<br> &gt;</p> <p>I disagree. It makes my work more organised, and it removes the trap of deduping without the hassle.<br></p> <p>Regards<br> -- <br> Iain Buclaw</p> <p>*(p &lt; e ? p++ : p) = (c &amp; 0x0f) + &#39;0&#39;;</p> --001a11c136aa50100904f3f928ee--
Mar 06 2014
prev sibling next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 6 March 2014 at 23:56:35 UTC, Iain Buclaw wrote:
 Indeed, I don't recall a single port-related pull in the last  
 several
 months that tried to do any refactoring.  I think the general
 consensus among the current mantainers is that explicit 
 versioning is
 good, vague/fallback else is bad.  However on the flip side, 
 the more ports you add into one single file, the more you 
 descend back into the madness you were initially trying to 
 avoid.

 I hope code duplication in many files rather than one file is 
 in the interest of everyone. :)

Yep. I think I mentioned in the github thread that while versioning implementations within a single module worked for me, it was because I was only targeting 3 platforms. With the number of targets GDC and LLDC have, I think breaking things up as you suggest is a preferable option.
Mar 06 2014
prev sibling next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Friday, 7 March 2014 at 00:53:18 UTC, Walter Bright wrote:
 What it will solve is when the user reads the OS api 
 documentation, and it says:

     #include <sys/ioctl.h>

 then the user knows that in D he has to do:

     import whatever.sys.ioctl;

 What the user should not have to do is:

     version (linux) import whatever.linux.sys.ioctl;
     else version (FreeBSD) import whatever.freebsd.sys.ioctl;
     else version (OSX) import whatever.osx.sys.ioctl;
     ...
     else static assert(0);

 nor:

     import whatever.posix.sys.ioctl;
     version (linux) import whatever.linux.sys.ioctl;
     else version (FreeBSD) import whatever.freebsd.sys.ioctl;
     else version (OSX) import whatever.osx.sys.ioctl;
     ...
     else static assert(0);

Toe be fair, I expect the user would almost never be importing platform-specific headers. Importing the posix headers is all that should be needed in the common case. Or if you need something platform-specific, then don't import the posix header because doing so is redundant--the platform header publicly imports it anyway. However, as much as I like that having the Posix headers strongly encourages the user to write portable code, I can see the idea doesn't have a lot of traction around here. I beleive Iain's proposal is to phase them out and do exactly what you suggest, only to make "whatever" "core.foo" rather than simply "core". Which works for me as well.
Mar 06 2014
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 7 March 2014 01:24, Walter Bright <newshound2 digitalmars.com> wrote:
 On 3/6/2014 5:10 PM, Sean Kelly wrote:
 However, as much as I like that having the Posix headers strongly
 encourages the user to write portable code, I can see the idea
 doesn't have a lot of traction around here.  I beleive Iain's
 proposal is to phase them out and do exactly what you suggest,

Thanks, Sean, I appreciate the graciousness of your reply.
 only to make "whatever" "core.foo" rather than simply "core".
 Which works for me as well.

I don't have any axe to grind about what the "whatever" expands to, I'm cool with either.

I assume the C headers will remain strictly under core.stdc? So it might make sense to transition core.sys -> core.stdc.sys?
Mar 07 2014
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 7 March 2014 08:34, Walter Bright <newshound2 digitalmars.com> wrote:
 On 3/7/2014 12:00 AM, Iain Buclaw wrote:
 I assume the C headers will remain strictly under core.stdc?  So it
 might make sense to transition core.sys -> core.stdc.sys?

I think core.stdc for the Standard C headers can remain the same.

OK - now that has been sorted. What's your opinion on where ports of standard C and system C headers should go?
Mar 07 2014
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
ports of C headers stand for translations of headers from 
different C libraries: glibc, snn, msvcrt, mingwrt, bionic and 
should reside in their respective folders like os api interface 
modules, and stdc will import them conditionally.
Mar 07 2014
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 7 March 2014 09:34, Walter Bright <newshound2 digitalmars.com> wrote:
 On 3/7/2014 1:03 AM, Iain Buclaw wrote:
 On 7 March 2014 08:34, Walter Bright <newshound2 digitalmars.com> wrote:
 On 3/7/2014 12:00 AM, Iain Buclaw wrote:
 I assume the C headers will remain strictly under core.stdc?  So it
 might make sense to transition core.sys -> core.stdc.sys?

I think core.stdc for the Standard C headers can remain the same.

OK - now that has been sorted. What's your opinion on where ports of standard C and system C headers should go?

Standard C => core.stdc Operating system API = > core? core.sys? core.os? I don't have strong feelings about that.

Arch ports, not platform ports.
Mar 07 2014
prev sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 7 March 2014 11:25, Kagamin <spam here.lot> wrote:
 ports of C headers stand for translations of headers from different C
 libraries: glibc, snn, msvcrt, mingwrt, bionic and should reside in their
 respective folders like os api interface modules, and stdc will import them
 conditionally.

Arch, arch, arch! Platform ports are pretty much covered by Walter and Sean's agreement. :)
Mar 07 2014