www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.internals - druntime: proposed changes in code organization

reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
Hi!

I propose to discuss a possibility of reorganizing code by this 
way:

For example, druntime/src/core/sys/posix/setjmp.d file contains 
few version branches of libc and OSes versions. It can be 
reorganized as:

druntime/impl/arm/core/sys/posix/-no-file-
druntime/impl/x86/core/sys/posix/-no-file-
druntime/impl/netbsd/core/sys/posix/setjmp.d
druntime/impl/openbsd/core/sys/posix/setjmp.d
druntime/impl/linux/core/sys/posix/-no-file-
druntime/impl/glibc/core/sys/posix/setjmp.d
druntime/impl/default/core/sys/posix/setjmp.d

(druntime/impl contains directories what named like most 
frequently used versions names)

Platform will be switched on tag-based by build script without 
"branches of branches" inside of code. I.e., if we build for a 
wide available Debian Linux for x86 it would be 3 tags: 
linux+glibc+x86, and for module core/sys/posix/setjmp.d these 
directories will be used:

druntime/impl/x86/core/sys/posix/-no-file-
druntime/impl/glibc/core/sys/posix/setjmp.d <- this will be used
druntime/impl/linux/core/sys/posix/-no-file-

(Obviously, except default, only one file must be found - 
otherwise this is error condition)

You should remember: not all modules are so complex as setjmp.d, 
most files will be in default directory (and many will not be in 
druntime/impl hierarchy at all). And, also, druntime is a special 
sort of software and this approach will help to maintain 
duplicated nature of druntime's bindings.

This approach will force us to avoid difficult branching schemas 
and to separate common logic and OS- or arch- related stuff

It opens ability to import modules with foreign data structures 
on non-native OSes: it will be possible to remove module "guards" 
like "version (Linux):" at start of modules and your Windows app 
will be able to import Linux data structures

This approach also helps to add new platforms (most important for 
me)
Nov 13 2023
next sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Tuesday, 14 November 2023 at 00:33:34 UTC, Denis Feklushkin 
wrote:

 Hi!

 I propose to discuss a possibility of reorganizing code by this 
 way:


Be warned, this has a very long history and ulti ended up nowhere.

https://issues.dlang.org/show_bug.cgi?id=11666

https://github.com/dlang/druntime/pull/1255
https://github.com/dlang/druntime/pull/744
https://github.com/dlang/druntime/pull/741
https://github.com/dlang/druntime/pull/731
Nov 13 2023
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Tuesday, 14 November 2023 at 06:04:27 UTC, Iain Buclaw wrote:

 On Tuesday, 14 November 2023 at 00:33:34 UTC, Denis Feklushkin 
 wrote:

 Hi!

 I propose to discuss a possibility of reorganizing code by 
 this way:




Forgot to add to the potential benefits list:

Proposed approach eliminates tons of these identical lines:
```
version (AArch64) version = ARM_Any;
version (ARM)     version = ARM_Any;
[...]
version (X86)     version = X86_Any;
version (X86_64)  version = X86_Any;

version (OSX)
     version = Darwin;
else version (iOS)
     version = Darwin;
else version (TVOS)
     version = Darwin;
else version (WatchOS)
     version = Darwin;
```


 Be warned, this has a very long history and ulti ended up 
 nowhere.


Yes! And I suggested something similar too - 
https://forum.dlang.org/thread/rwmqulighwecqyxqbnkt forum.dlang.org

This all suggests that something needs to be done with the 
existing approach. For example, for more than 10 years we haven't 
seen the expected creation of "lightweight druntime" for 
embedded. No one was able to solve such static branches puzzle 
gracefully.

By the way, proposed here "tags+directories" solution is the full 
equivalent of currently used "version branching"? (Can anyone 
refute this proposition?)
Nov 14 2023
parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Tuesday, 14 November 2023 at 11:48:00 UTC, Denis Feklushkin 
wrote:

Proof of concept: https://github.com/dlang/dmd/pull/15822
Nov 16 2023
prev sibling parent reply IGotD- <nise nise.com> writes:
On Tuesday, 14 November 2023 at 00:33:34 UTC, Denis Feklushkin 
wrote:

 Hi!

 I propose to discuss a possibility of reorganizing code by this 
 way:

 For example, druntime/src/core/sys/posix/setjmp.d file contains 
 few version branches of libc and OSes versions. It can be 
 reorganized as:

 druntime/impl/arm/core/sys/posix/-no-file-
 druntime/impl/x86/core/sys/posix/-no-file-
 druntime/impl/netbsd/core/sys/posix/setjmp.d
 druntime/impl/openbsd/core/sys/posix/setjmp.d
 druntime/impl/linux/core/sys/posix/-no-file-
 druntime/impl/glibc/core/sys/posix/setjmp.d
 druntime/impl/default/core/sys/posix/setjmp.d


I suggested this a while ago that druntime should be refactored 
to files instead of the version hell like #ifdef hell in C. 
However, in order this to properly work was that import should be 
able to use version strings in the filename path.

Also, one question in the example. Why two files

druntime/impl/arm/core/sys/posix/-no-file-
druntime/impl/x86/core/sys/posix/-no-file-

as I assume these are stubs. Isn't a stub no OS at all?

Apart from that. I strongly support having stubbed versions in 
druntime, also with more fine granularity which would be great 
for embedded systems. Some systems have file systems but no 
process support for example.


 It opens ability to import modules with foreign data structures 
 on non-native OSes: it will be possible to remove module 
 "guards" like "version (Linux):" at start of modules and your 
 Windows app will be able to import Linux data structures

 This approach also helps to add new platforms (most important 
 for me)


First thing is that we need version strings in imports. This has 
been suggested for years but nothing has happened. If we add 
this, then modularizing druntime will fall more naturally (and 
perhaps their own projects too).
Nov 16 2023
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Thursday, 16 November 2023 at 15:13:44 UTC, IGotD- wrote:


 For example, druntime/src/core/sys/posix/setjmp.d file 
 contains few version branches of libc and OSes versions. It 
 can be reorganized as:

 druntime/impl/arm/core/sys/posix/-no-file-
 druntime/impl/x86/core/sys/posix/-no-file-
 druntime/impl/netbsd/core/sys/posix/setjmp.d
 druntime/impl/openbsd/core/sys/posix/setjmp.d
 druntime/impl/linux/core/sys/posix/-no-file-
 druntime/impl/glibc/core/sys/posix/setjmp.d
 druntime/impl/default/core/sys/posix/setjmp.d


 I suggested this a while ago that druntime should be refactored 
 to files instead of the version hell like #ifdef hell in C. 
 However, in order this to properly work was that import should 
 be able to use version strings in the filename path.


If I understand you, this is ~same approach as I implementing


 Also, one question in the example. Why two files

 druntime/impl/arm/core/sys/posix/-no-file-
 druntime/impl/x86/core/sys/posix/-no-file-

 as I assume these are stubs. Isn't a stub no OS at all?


No, there is only for illustration what this dirs isn't contain 
file setjmp.d. These directories can contain another files, of 
course

If you ask about "no-OS environment" supporting, then, in my 
opinion, at the druntime level it is not make sense to discuss 
about this: if you implement needed calls then druntime will work 
in this environment. Briefly, "No-OS" - is just a kind of OS type.


 Apart from that. I strongly support having stubbed versions in 
 druntime, also with more fine granularity which would be great 
 for embedded systems. Some systems have file systems but no 
 process support for example.


I believe that such  granularity will formed naturally if the 
code is reorganized in the proposed way

And I think processes (threads?) are necessary part of the D 
language. Even on MCUs: the fact is that threads are involved 
into the garbage collection process
Nov 16 2023
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Thursday, 16 November 2023 at 17:13:37 UTC, Denis Feklushkin 
wrote:

I made update of the proposal description text, 
https://github.com/dlang/dmd/pull/15822
Nov 19 2023
parent reply IGotD- <nise nise.com> writes:
On Sunday, 19 November 2023 at 19:53:53 UTC, Denis Feklushkin 
wrote:

 On Thursday, 16 November 2023 at 17:13:37 UTC, Denis Feklushkin 
 wrote:

 I made update of the proposal description text, 
 https://github.com/dlang/dmd/pull/15822


Not sure If I'm going to reply here or on github.

First, I think that you need to more precisely explain what a 
"tag" is. What is the underlying data type and how does it differ 
from a version identifier. I don't understand what a tag is by 
reading the PR.

The PR doesn't mention anything about the generic part of the 
implementation. Where should the generic part be?

Also how the source code hierarchy, how would it look like?

Let's take really small example like setjmp.d and show how it 
should be implemented, both inside the file and where the files 
should located.
Nov 20 2023
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Monday, 20 November 2023 at 13:39:21 UTC, IGotD- wrote:

 On Sunday, 19 November 2023 at 19:53:53 UTC, Denis Feklushkin 
 wrote:

 On Thursday, 16 November 2023 at 17:13:37 UTC, Denis 
 Feklushkin wrote:

 I made update of the proposal description text, 
 https://github.com/dlang/dmd/pull/15822


 Not sure If I'm going to reply here or on github.


Doesn't matter. Sorry for a response delay, I was away


 First, I think that you need to more precisely explain what a 
 "tag" is.


Okay, I'll write here and when everything is discussed I’ll add 
it to the PR description


 What is the underlying data type and how does it differ from a 
 version identifier. I don't understand what a tag is by reading 
 the PR.


Proposed tags are similar to a D's versions, but I decided to 
separate its name because tags become not corresponding to a 
built-in D versions, at least.

For example, most likely, tags with names like 
“posix_compatible_threads” will appear, because in the future we 
will understand that some OSes use Posix interace API to provide 
threads functionality but while not being a Posix system. The 
same goes for working with files, etc.

I also suggest to write sets of tags for each target in this form:
```
x86_64,posix,osx
```
and not in:
```
x86_64-posix-osx
```
to prevent attempts to combine proposed druntime tags with 
compiler target triplets


 The PR doesn't mention anything about the generic part of the 
 implementation. Where should the generic part be?


I don’t understand what is meant by "generic part" here


 Also how the source code hierarchy, how would it look like?


All tags-driven code will be placed into druntime/config/ 
directory

Inside of config/ each tag provides it's own files hierarchy 
corresponding to main druntime modules hierarchy.

For example, file:
```
druntime/config/posix/core/sync/event.d
```
will be used as module "core.sync.event" if "posix" tag choised


 Let's take really small example like setjmp.d and show how it 
 should be implemented, both inside the file and where the files 
 should located.


For now, all *.d files involved into discussed PR is a such 
examples
Nov 30 2023
next sibling parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Friday, 1 December 2023 at 05:29:31 UTC, Denis Feklushkin 
wrote:


 What is the underlying data type and how does it differ from a 
 version identifier. I don't understand what a tag is by 
 reading the PR.


 Proposed tags are similar to a D's versions, but I decided to 
 separate its name because tags become not corresponding to a 
 built-in D versions, at least.


Most importantly, new name need to avoid confusion. If we call 
them "version", we will inevitably begin to confuse it with D 
versions.

"Tags" (in various uses in the different software) are a similar 
scheme. Therefore, I think the name fits well.
Nov 30 2023
prev sibling parent reply IGotD- <nise nise.com> writes:
On Friday, 1 December 2023 at 05:29:31 UTC, Denis Feklushkin 
wrote:

 First, I think that you need to more precisely explain what a 
 "tag" is.


 Okay, I'll write here and when everything is discussed I’ll add 
 it to the PR description


 What is the underlying data type and how does it differ from a 
 version identifier. I don't understand what a tag is by 
 reading the PR.


 Proposed tags are similar to a D's versions, but I decided to 
 separate its name because tags become not corresponding to a 
 built-in D versions, at least.

 For example, most likely, tags with names like 
 “posix_compatible_threads” will appear, because in the future 
 we will understand that some OSes use Posix interace API to 
 provide threads functionality but while not being a Posix 
 system. The same goes for working with files, etc.

 I also suggest to write sets of tags for each target in this 
 form:
 ```
 x86_64,posix,osx
 ```
 and not in:
 ```
 x86_64-posix-osx
 ```
 to prevent attempts to combine proposed druntime tags with 
 compiler target triplets


Thank you, now I have a better understanding what a tag is. 
Correct me if I'm wrong so these tags are like "feature defines" 
similarly to a boatload of C defines that you must define for 
certain C-libraries. Will these tags eventually become version 
identifiers inside the code? Also one question if the tags are 
visible inside the D code or just outside the code in the build 
system?


 The PR doesn't mention anything about the generic part of the 
 implementation. Where should the generic part be?


 I don’t understand what is meant by "generic part" here


We have OS/API specific code in druntime but also generic code 
that can be shared by all OSes. What I was looking for is an 
explanation where this generic code should located. Will it be 
like it is right now and you break out the OS specific code and 
put it in druntime/config/? For example we can make the code in 
druntime/src/core/thread/osthread.d (which is absolutely 
horrible) and start to define what is OS independent, leave it 
inside osthread.d and then put OS dependent stuff in the new 
files?
Dec 01 2023
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Friday, 1 December 2023 at 12:53:55 UTC, IGotD- wrote:


 Thank you, now I have a better understanding what a tag is. 
 Correct me if I'm wrong so these tags are like "feature 
 defines" similarly to a boatload of C defines that you must 
 define for certain C-libraries.


No, if I correctly understand this "C features defines" concept


 Will these tags eventually become version identifiers inside 
 the code?


No


 Also one question if the tags are visible inside the D code or 
 just outside the code in the build system?


Tags are visible only for druntime build system


 The PR doesn't mention anything about the generic part of the 
 implementation. Where should the generic part be?


 I don’t understand what is meant by "generic part" here


 We have OS/API specific code in druntime but also generic code 
 that can be shared by all OSes. What I was looking for is an 
 explanation where this generic code should located. Will it be 
 like it is right now and you break out the OS specific code and 
 put it in druntime/config/?


Yes, this is correct

Good example is event.d - proposed PR splits it into 3 files: 
Posix and Windows implementations and also shared between OSes 
module druntime/src/core/internal/event_tests.d, containing 
mandatory unittests which are common for all Event implementations


 For example we can make the code in 
 druntime/src/core/thread/osthread.d (which is absolutely 
 horrible) and start to define what is OS independent, leave it 
 inside osthread.d and then put OS dependent stuff in the new 
 files?


Specifically for osthread.d:

At first step I propose some small refactoring for osthread.d: it 
is need to move code shared for all OSes into 
"core.thread.threadbase" module. As I remember, community wanted 
to do this for a long time but there was no reason to begin this 
process.

Then we can move osthread.d into confing/ and also split it into 
Windows and non-Windows versions

All this can be done gradually. We don't even forced to split 
osthread.d at first stages if we afraid to broke something.
Dec 02 2023
next sibling parent reply IGotD- <nise nise.com> writes:
On Saturday, 2 December 2023 at 08:10:57 UTC, Denis Feklushkin 
wrote:

 Also one question if the tags are visible inside the D code or 
 just outside the code in the build system?


 Tags are visible only for druntime build system


The tags, how are they injected into the build system? Keep in 
mind that we have several types of build systems in the D 
project. I think DMD uses traditional make files. LDC uses Cmake.

Are they going to be a separate file where you just define 
variables for build system?
Dec 02 2023
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Saturday, 2 December 2023 at 12:30:31 UTC, IGotD- wrote:

 On Saturday, 2 December 2023 at 08:10:57 UTC, Denis Feklushkin 
 wrote:

 Also one question if the tags are visible inside the D code 
 or just outside the code in the build system?


 Tags are visible only for druntime build system


 The tags, how are they injected into the build system?


Tags defined in the build script. For DMD, druntime/posix.mak:
```make

ifeq ($(MODEL), 64)
	ARCH=x86_64
else
	ARCH=x86
endif

ifeq ($(OS), Win_64)

	TAGS:=windows,default,$(ARCH)
else
	TAGS:=$(ARCH),posix,$(OS),default
endif
```

In the future it will be trivial to add environment variable or 
option for manual setting tags and dir(s) to expand ability of 
builds for various platforms


 Keep in mind that we have several types of build systems in the 
 D project. I think DMD uses traditional make files. LDC uses 
 Cmake.


Yes. (+I use Meson for druntime builds for experimental embedded 
targets)

ldc/runtime/CMakeLists.txt contains:
```make

file(GLOB_RECURSE DRUNTIME_D_BIONIC  
${RUNTIME_DIR}/src/core/sys/bionic/*.d)
file(GLOB_RECURSE DRUNTIME_D_DARWIN  
${RUNTIME_DIR}/src/core/sys/darwin/*.d)
file(GLOB_RECURSE DRUNTIME_D_DRAGONFLYBSD 
${RUNTIME_DIR}/src/core/sys/dragonflybsd/*.d)
```

I don't know CMake deeply, but it looks like it is not too 
difficult to adopt proposed changes into LDC build facility


 Are they going to be a separate file where you just define 
 variables for build system?


Most likely, this will not be done due to the fact that all build 
systems are too different (Although I just found similar idea in 
my comment "TODO: implement tags table")
Dec 02 2023
parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Saturday, 2 December 2023 at 14:02:07 UTC, Denis Feklushkin 
wrote:

Everything is almost ready: 
https://github.com/dlang/dmd/pull/15822

Also, PR description changed and expanded

There is only one small blocker: 
https://github.com/dlang/dmd/pull/15880

I'm really looking forward to your feedback!
Dec 04 2023
prev sibling parent reply Johan <j j.nl> writes:
On Saturday, 2 December 2023 at 08:10:57 UTC, Denis Feklushkin 
wrote:

 On Friday, 1 December 2023 at 12:53:55 UTC, IGotD- wrote:


 Thank you, now I have a better understanding what a tag is. 
 Correct me if I'm wrong so these tags are like "feature 
 defines" similarly to a boatload of C defines that you must 
 define for certain C-libraries.


 No, if I correctly understand this "C features defines" concept


 Will these tags eventually become version identifiers inside 
 the code?


 No


 Also one question if the tags are visible inside the D code or 
 just outside the code in the build system?


 Tags are visible only for druntime build system


How is this going to work for cross-compilation with _one_ 
compiler druntime? LDC is a cross-compiler and uses the same 
source files for any target. A single LDC installation must be 
able to compile to any target platform, i.e. with a single 
druntime installation. (target-dependent copying of files by the 
compiler build system does not fulfill that requirement)
To me, it sounds like your changes make separate druntime 
installations per target. This is not acceptable for LDC. But 
perhaps I misunderstood your changes.

-Johan
Dec 13 2023
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Wednesday, 13 December 2023 at 17:43:06 UTC, Johan wrote:

 On Saturday, 2 December 2023 at 08:10:57 UTC, Denis Feklushkin 
 wrote:

 On Friday, 1 December 2023 at 12:53:55 UTC, IGotD- wrote:


 Thank you, now I have a better understanding what a tag is. 
 Correct me if I'm wrong so these tags are like "feature 
 defines" similarly to a boatload of C defines that you must 
 define for certain C-libraries.


 No, if I correctly understand this "C features defines" concept


 Will these tags eventually become version identifiers inside 
 the code?


 No


 Also one question if the tags are visible inside the D code 
 or just outside the code in the build system?


 Tags are visible only for druntime build system


 How is this going to work for cross-compilation with _one_ 
 compiler druntime? LDC is a cross-compiler and uses the same 
 source files for any target.


With ldc2 we can switch druntime by using custom .conf file 
("--conf=" option).
I successfully use this for my custom druntime builds


 A single LDC installation must be able to compile to any target 
 platform, i.e. with a single druntime installation. 
 (target-dependent copying of files by the compiler build system 
 does not fulfill that requirement)
 To me, it sounds like your changes make separate druntime 
 installations per target.


Yes, you are right

More precise: same as we already(?) provide separate druntime 
binaries for each target (i.e., Linux, Windows, etc) I propose to 
provide a separate include dir for each of these druntime binary


 This is not acceptable for LDC.


It seems to me, for proposed druntime switching, syntax of 
/etc/ldc2.conf already contains all necessary things?
Dec 16 2023
parent reply Johan <j j.nl> writes:
On Saturday, 16 December 2023 at 13:04:27 UTC, Denis Feklushkin 
wrote:

 On Wednesday, 13 December 2023 at 17:43:06 UTC, Johan wrote:

 This is not acceptable for LDC.


 It seems to me, for proposed druntime switching, syntax of 
 /etc/ldc2.conf already contains all necessary things?


I remain my stance that this is unacceptable. You are forcing a 
big change on a system that already works, without a real benefit.
Yes, it is possible to work around the problems introduced by you 
by writing .conf entries for every platform, but that is extra 
burden on package maintainers that they currently don't need to 
do.

Simply use `version` in source files, and all is good (like now). 
Separate files do not require completely separate source trees. 
You can have a file with something like:
```
version (A) {
   public import foo_A;
} else version (B) {
   public import foo_B;
...
```

-Johan
Dec 16 2023
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Saturday, 16 December 2023 at 18:09:57 UTC, Johan wrote:

 On Saturday, 16 December 2023 at 13:04:27 UTC, Denis Feklushkin 
 wrote:

 On Wednesday, 13 December 2023 at 17:43:06 UTC, Johan wrote:

 This is not acceptable for LDC.


 It seems to me, for proposed druntime switching, syntax of 
 /etc/ldc2.conf already contains all necessary things?


 I remain my stance that this is unacceptable. You are forcing a 
 big change on a system that already works, without a real 
 benefit.


Make sure what you read benefits list:
https://github.com/dlang/dmd/pull/15822#issue-1996484133

Also it constains considerations about why this approahch will 
not work ("Another options to solve problem is more painful"):


 Simply use `version` in source files, and all is good (like 
 now). Separate files do not require completely separate source 
 trees. You can have a file with something like:
 ```
 version (A) {
   public import foo_A;
 } else version (B) {
   public import foo_B;
 ...
Dec 16 2023
parent reply Johan <j j.nl> writes:
On Saturday, 16 December 2023 at 18:44:00 UTC, Denis Feklushkin 
wrote:

 On Saturday, 16 December 2023 at 18:09:57 UTC, Johan wrote:

 On Saturday, 16 December 2023 at 13:04:27 UTC, Denis 
 Feklushkin wrote:

 On Wednesday, 13 December 2023 at 17:43:06 UTC, Johan wrote:

 This is not acceptable for LDC.


 It seems to me, for proposed druntime switching, syntax of 
 /etc/ldc2.conf already contains all necessary things?


 I remain my stance that this is unacceptable. You are forcing 
 a big change on a system that already works, without a real 
 benefit.


 Make sure what you read benefits list:
 https://github.com/dlang/dmd/pull/15822#issue-1996484133


Much of your list are not benefits, just removing things from the 
source code that you find "ugly" I guess. For example, removing 
`version (Linux):` is not a benefit; it's just developer taste, 
with no benefit for the user.

I understand that for you currently the situation is not nice. 
But it is not a good solution to make life good for you, and make 
it worse for the status quo. A custom druntime or non-standard OS 
is not a very common case.

-Johan
Dec 16 2023
next sibling parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Saturday, 16 December 2023 at 18:59:50 UTC, Johan wrote:


 Make sure what you read benefits list:
 https://github.com/dlang/dmd/pull/15822#issue-1996484133


 Much of your list are not benefits, just removing things from 
 the source code that you find "ugly" I guess. For example, 
 removing `version (Linux):` is not a benefit; it's just 
 developer taste, with no benefit for the user.


This is actually one (little) benefit, because currently you 
cannot write a program for Windows that will use some druntime 
Linux structures for parsing something like net traffic, core 
dumps, etc. You will need to duplicate Linux (or Posix) 
structures for that.

This is a meaningless block that has formed historically because 
it was easier and faster to implement by this way


 I understand that for you currently the situation is not nice. 
 But it is not a good solution to make life good for you, and 
 make it worse for the status quo. A custom druntime or 
 non-standard OS is not a very common case.


(At first, I don't see what currently druntime is distributed as 
ready for cross-compilation bundle. I.e., ldc2 archieve isn't 
contains druntime binary for Windows, Linux and *BSDs 
simultaneously, right? In this case we should not change anything 
particularly much in the config)

As for me, we don't know common this case or not. "Non-standard 
OS" includes thousands variations of hardware+software from 
realtime embedded software to modern gaming consoles support or 
so on.

But it will be simpler to be always at second place (after 
invincible C and C++) on D support implementation timeline in the 
event of the appearance of some new (revolutionary, of course) 
devices or OS inventions. So I think I guess that the impact of 
such a changes will be great.

ldc2.conf will obtain 3-6 sections (i.e. about ~50 lines total) 
for common platforms like Linux, Windows and *BSD. Is that too 
expensive?
Dec 16 2023
parent reply Johan <j j.nl> writes:
On Saturday, 16 December 2023 at 19:42:43 UTC, Denis Feklushkin 
wrote:

 On Saturday, 16 December 2023 at 18:59:50 UTC, Johan wrote:


 Make sure what you read benefits list:
 https://github.com/dlang/dmd/pull/15822#issue-1996484133


 Much of your list are not benefits, just removing things from 
 the source code that you find "ugly" I guess. For example, 
 removing `version (Linux):` is not a benefit; it's just 
 developer taste, with no benefit for the user.


 This is actually one (little) benefit, because currently you 
 cannot write a program for Windows that will use some druntime 
 Linux structures for parsing something like net traffic, core 
 dumps, etc. You will need to duplicate Linux (or Posix) 
 structures for that.

 This is a meaningless block that has formed historically 
 because it was easier and faster to implement by this way


If it's meaningless, it can be removed today without your big 
file change. Can it?


 I understand that for you currently the situation is not nice. 
 But it is not a good solution to make life good for you, and 
 make it worse for the status quo. A custom druntime or 
 non-standard OS is not a very common case.


 (At first, I don't see what currently druntime is distributed 
 as ready for cross-compilation bundle. I.e., ldc2 archieve 
 isn't contains druntime binary for Windows, Linux and *BSDs 
 simultaneously, right? In this case we should not change 
 anything particularly much in the config)


For cross-compilation, you don't need druntime _binaries_. The 
binaries are only needed for linking. But if linking with 
druntime/phobos is needed, then LDC has a specific tool to make 
it easier for users to build druntime+phobos for a different 
target platform. And then you just specify the location of those 
binaries on the cmdline. Yes, this can be simplified for 
continued use by editing ldc2.conf, but it is not required to do 
so; nor is it needed to have a separate druntime source tree 
stored somewhere else.


 As for me, we don't know common this case or not. "Non-standard 
 OS" includes thousands variations of hardware+software from 
 realtime embedded software to modern gaming consoles support or 
 so on.

 But it will be simpler to be always at second place (after 
 invincible C and C++) on D support implementation timeline in 
 the event of the appearance of some new (revolutionary, of 
 course) devices or OS inventions. So I think I guess that the 
 impact of such a changes will be great.


You make it sound as if it is currently impossible to implement 
druntime for linux, Apple, Windows, Solaris, OpenBSD, NetBSD, 
FreeBSD, DragonFly, Android, ... x86, x86_64, arm32, aarch64, 
riscv, ... Yet, the implementation for all these exists.


 ldc2.conf will obtain 3-6 sections (i.e. about ~50 lines total) 
 for common platforms like Linux, Windows and *BSD. Is that too 
 expensive?


Please show us the PR where you fix LDC, Dub, and GDC for the 
problems you are introducing.

Thanks,
   Johan
Dec 17 2023
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Sunday, 17 December 2023 at 09:37:33 UTC, Johan wrote:


 This is a meaningless block that has formed historically 
 because it was easier and faster to implement by this way


 If it's meaningless, it can be removed today without your big 
 file change.


*Tiny file change


 (At first, I don't see what currently druntime is distributed 
 as ready for cross-compilation bundle. I.e., ldc2 archieve 
 isn't contains druntime binary for Windows, Linux and *BSDs 
 simultaneously, right? In this case we should not change 
 anything particularly much in the config)


 For cross-compilation, you don't need druntime _binaries_.


Only in dynamic linking mode, yes


 The binaries are only needed for linking. But if linking with 
 druntime/phobos is needed, then LDC has a specific tool to make 
 it easier for users to build druntime+phobos for a different 
 target platform. And then you just specify the location of 
 those binaries on the cmdline. Yes, this can be simplified for 
 continued use by editing ldc2.conf, but it is not required to 
 do so; nor is it needed to have a separate druntime source tree 
 stored somewhere else.


For example, no one is surprised by the different Linux kernel 
headers packages, coresponding to a different kernel binaries, 
but derrived from one kernel source tree

Also, we can generate and provide import/ dirs for a few 
platforms (what you mentioned below) simultaneously if it will be 
decided as more convient


 As for me, we don't know common this case or not. 
 "Non-standard OS" includes thousands variations of 
 hardware+software from realtime embedded software to modern 
 gaming consoles support or so on.

 But it will be simpler to be always at second place (after 
 invincible C and C++) on D support implementation timeline in 
 the event of the appearance of some new (revolutionary, of 
 course) devices or OS inventions. So I think I guess that the 
 impact of such a changes will be great.


 You make it sound as if it is currently impossible to implement 
 druntime for linux, Apple, Windows, Solaris, OpenBSD, NetBSD, 
 FreeBSD, DragonFly, Android, ... x86, x86_64, arm32, aarch64, 
 riscv, ... Yet, the implementation for all these exists.


Sorry, but this is obvious demagoguery - matrix of 3 RTOSes, 3 
CPU architectures and 3 LIBCs easily give us 27 platforms in this 
terms


 ldc2.conf will obtain 3-6 sections (i.e. about ~50 lines 
 total) for common platforms like Linux, Windows and *BSD. Is 
 that too expensive?


 Please show us the PR where you fix LDC, Dub,


What's wrong with DUB?


 and GDC for the problems you are introducing.


Sorry, but now your turn to propose solution other than these 3 
listed in the previous message
Dec 17 2023
parent reply Johan <j j.nl> writes:
On Sunday, 17 December 2023 at 11:00:02 UTC, Denis Feklushkin 
wrote:

 On Sunday, 17 December 2023 at 09:37:33 UTC, Johan wrote:

 Please show us the PR where you fix LDC, Dub,


 What's wrong with DUB?


 and GDC for the problems you are introducing.


 Sorry, but now your turn to propose solution other than these 3 
 listed in the previous message


Yes indeed. And there will be more breakage. Cross-compilation on 
godbolt.org will also break.

-Johan
Dec 17 2023
parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Sunday, 17 December 2023 at 12:13:57 UTC, Johan wrote:


 Yes indeed. And there will be more breakage. Cross-compilation 
 on godbolt.org will also break.


Nothing will be broken - 
https://github.com/dlang/dmd/pull/15887#issuecomment-1890961483

2 All:

Recently latest issue that was preventing tests from passing on 
low-end MCU-style platforms has been resolved (Bugzilla 15504)

I suggest us think again about what we are losing now

This is a whole new World that is hidden from us (i.e., from 
developers who use Windows-Linux-Mac) only because devs have to 
go through hell to get a LED blinking on D. We lose good synergy 
effect. Newcomers with a different expierence will pass through 
our low-level code to implement D support for a very different 
platforms. Community will be able to very quickly implement D 
support of the mostest inventions such as virtual reality helmets 
(if is this still considered as innovation), specialised 
computing systems (for which now is only C and Fortran) and for 
all modern stuff. Also, to write on D for Sega Genesis (Megadrive 
2) or for Playdate - this is just cool!
Jan 31
prev sibling parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Saturday, 16 December 2023 at 18:59:50 UTC, Johan wrote:


 But it is not a good solution to make life good for you, and 
 make it worse for the status quo.


I want to explain more here, so that it does not seem like an 
inability to accept another point of view

My reasoning are as follows:

It is need to switch source files (partially) by some way. It's 
possible, apparently, by 3 convient ways:

1. By adding public import into modules (as you proposed in 
message above, and I answered why this will not work well)
2. By adding script witch switches SRCS and COPY lists (my 
solution)
3. By patching compilers: adding some CLI switches that will 
force all compilers(!) to switch modules directories or something 
like. It seems to me, no one will like such approach

So, only one realistic solution is possible

And, after all, if proposed approach will not work, i.e., if time 
shows that this is terrible to maintain all theese modules 
sources in config/, then we can use a trivial script to return 
everything to its ~original state. (Even better because the 
script will not forget to add static assert "fuse" for 
non-supported version(..))
Dec 16 2023