www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Why don't we use the CRT functions under Windows?

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
While working with the dmd codebase 
(https://github.com/dlang/dmd/pull/12560) I noticed that it 
systematically shuns C runtime calls to functions such as unlink, rename 
etc. Instead the code uses native Windows API calls (DeleteFile, MoveFile).

Spoke to Walter about it, he said the reason is mostly historical and 
for the most part forgotten and probably obsolete - at least in the 
past, Windows implementations of CRT was poorly done, had subtle 
differences, and bugs.

Is that still the case? If not, we could reduce the complexity of source 
code considerably by unifying version(Posix) and version(Windows) code.
May 22 2021
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
I haven't confirmed but I think the CRT functions work in ANSI 
instead of Unicode (unless you use the w prefixed ones... and at 
that point you're branching anyway so you haven't gained 
anything).

Probably should try it with a utf-8 name and confirm it works or 
doesn't.
May 22 2021
prev sibling next sibling parent reply IGotD- <nise nise.com> writes:
On Saturday, 22 May 2021 at 16:25:08 UTC, Andrei Alexandrescu 
wrote:

 While working with the dmd codebase 
 (https://github.com/dlang/dmd/pull/12560) I noticed that it 
 systematically shuns C runtime calls to functions such as 
 unlink, rename etc. Instead the code uses native Windows API 
 calls (DeleteFile, MoveFile).

 Spoke to Walter about it, he said the reason is mostly 
 historical and for the most part forgotten and probably 
 obsolete - at least in the past, Windows implementations of CRT 
 was poorly done, had subtle differences, and bugs.

 Is that still the case? If not, we could reduce the complexity 
 of source code considerably by unifying version(Posix) and 
 version(Windows) code.


Instead of trying to unify the code, OS dependent implementations 
should be moved to separate files. Also, should D rely on the C 
library? I would say the D should go the other way and rely on 
the C library a little as possible.

Another thing I have noticed that languages that use the 
C-library just creates a native copy of the interface (which just 
add another layer), instead of defining an own cross platform API
May 22 2021
next sibling parent reply Ola Fosheim Grostad <ola.fosheim.grostad gmail.com> writes:
On Saturday, 22 May 2021 at 19:08:14 UTC, IGotD- wrote:

 Another thing I have noticed that languages that use the 
 C-library just creates a native copy of the interface (which 
 just add another layer), instead of defining an own cross 
 platform API


It allows them to claim that they are based on Posix which is a 
standard, which looks good on paper... But for a systems 
development language it is a bit underwhelming as modern file 
systems have many more features...
May 22 2021
parent reply IGotD- <nise nise.com> writes:
On Saturday, 22 May 2021 at 20:05:46 UTC, Ola Fosheim Grostad 
wrote:

 It allows them to claim that they are based on Posix which is a 
 standard, which looks good on paper... But for a systems 
 development language it is a bit underwhelming as modern file 
 systems have many more features...


C library API is an ISO C standard, POSIX is another standard 
which Windows doesn't natively support.

By today's standard the C library API is ancient and doesn't 
provide an asynchronous API for example. Win32 and POSIX (late to 
the game) have both an asynchronous APIs.
May 22 2021
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 22 May 2021 at 20:18:58 UTC, IGotD- wrote:

 By today's standard the C library API is ancient and doesn't 
 provide an asynchronous API for example. Win32 and POSIX (late 
 to the game) have both an asynchronous APIs.


The Windows overlapped io api is actually very nice too. In fact, 
I overall prefer the Windows functions to most the posix and c 
ones.
May 22 2021
prev sibling parent Ola Fosheim Grostad <ola.fosheim.grostad gmail.com> writes:
On Saturday, 22 May 2021 at 20:18:58 UTC, IGotD- wrote:

 C library API is an ISO C standard, POSIX is another standard 
 which Windows doesn't natively support.


Alright, my mistake then. I thought they had a POSIX layer.. My 
windows machine has Windows XP :-D.


 By today's standard the C library API is ancient and doesn't 
 provide an asynchronous API for example. Win32 and POSIX (late 
 to the game) have both an asynchronous APIs.


It would be cool to have something directly built on Linux 
syscalls... Not sure if it is more useful, but sound cool :-D
May 22 2021
prev sibling parent Kagamin <spam here.lot> writes:
On Saturday, 22 May 2021 at 19:08:14 UTC, IGotD- wrote:

 Instead of trying to unify the code, OS dependent 
 implementations should be moved to separate files. Also, should 
 D rely on the C library? I would say the D should go the other 
 way and rely on the C library a little as possible.


This, even on posix the only useful stuff in libc is malloc and 
getaddrinfo. Try anything mildly useful like pipes, processes, 
threads, atomics, mutexes, sockets, async io, and you find 
yourself writing a PAL anyway.
May 24 2021
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/22/21 12:25 PM, Andrei Alexandrescu wrote:

 While working with the dmd codebase 
 (https://github.com/dlang/dmd/pull/12560) I noticed that it 
 systematically shuns C runtime calls to functions such as unlink, rename 
 etc. Instead the code uses native Windows API calls (DeleteFile, MoveFile).
 
 Spoke to Walter about it, he said the reason is mostly historical and 
 for the most part forgotten and probably obsolete - at least in the 
 past, Windows implementations of CRT was poorly done, had subtle 
 differences, and bugs.
 
 Is that still the case? If not, we could reduce the complexity of source 
 code considerably by unifying version(Posix) and version(Windows) code.
 


It so happens that the CRT functions for something like unlink and 
rename *are* the system calls on Unix. On windows, they are wrappers. I 
know in at least some cases Windows implements minimal features for 
their C interface, while providing full featured support with their SDK 
calls. I think druntime/phobos should focus on the minimal required 
libraries to work on the OS. Especially for things that require i/o 
handles, the C library abstraction is poor on Windows.

But this is my opinion for our standard library. For DMD though? I don't 
really care if it uses/depends on CRT. Ideally, it should have an 
abstraction layer for these things, and use whatever works.

-Steve
May 22 2021
next sibling parent IGotD- <nise nise.com> writes:
On Sunday, 23 May 2021 at 00:13:51 UTC, Steven Schveighoffer 
wrote:

 But this is my opinion for our standard library. For DMD 
 though? I don't really care if it uses/depends on CRT. Ideally, 
 it should have an abstraction layer for these things, and use 
 whatever works.


Support for the C standard library and specific OS support isn't 
mutually exclusive. D runtime/phobos can also support CRT as an 
OS option with limited functionality. This would be equivalent to 
OS "any" in Nim, that supports whatever is provided by the CRT.

For Linux and Windows I would just go for the raw system API. The 
C library also imposes limitations and extra overhead. For 
example maximum allowed open files. The file operations also 
might do buffering in the CRT and the question if we want 
buffering there as well.
May 22 2021
prev sibling parent Ola Fosheim Grostad <ola.fosheim.grostad gmail.com> writes:
On Sunday, 23 May 2021 at 00:13:51 UTC, Steven Schveighoffer 
wrote:

 It so happens that the CRT functions for something like unlink 
 and rename *are* the system calls on Unix. On windows, they are


They are wrappers on unix too, but the cost is low as the context 
switch overhead is so large that it makes the wrapper cost 
neglible. On mac there is no ABI guarantees if you dont use the 
wrappers IIRC.
May 22 2021
prev sibling parent Tony <tonytdominguez aol.com> writes:
On Saturday, 22 May 2021 at 16:25:08 UTC, Andrei Alexandrescu 
wrote:

 While working with the dmd codebase 
 (https://github.com/dlang/dmd/pull/12560) I noticed that it 
 systematically shuns C runtime calls to functions such as 
 unlink, rename etc. Instead the code uses native Windows API 
 calls (DeleteFile, MoveFile).

 Spoke to Walter about it, he said the reason is mostly 
 historical and for the most part forgotten and probably 
 obsolete - at least in the past, Windows implementations of CRT 
 was poorly done, had subtle differences, and bugs.

 Is that still the case? If not, we could reduce the complexity 
 of source code considerably by unifying version(Posix) and 
 version(Windows) code.


Wikipedia has a small section on the C Runtime DLL which talks 
about the changing history and the present use of two DLLs, one 
of which has name changes in sync with VC++ versions:
https://en.wikipedia.org/wiki/Microsoft_Windows_library_files#MSVCRT.DLL,_MSVCP*.DLL_and_CRTDLL.DLL


_______________________________________________________________________
MSVCRT.DLL, MSVCP*.DLL and CRTDLL.DLL

MSVCRT.DLL is the C standard library for the Visual C++ (MSVC) 
compiler from version 4.2 to 6.0. It provides programs compiled 
by these versions of MSVC with most of the standard C library 
functions. These include string manipulation, memory allocation, 
C-style input/output calls, and others. MSVCP*.DLL is the 
corresponding C++ library.

It has shipped with Windows versions since Windows 95 OSR2.5 for 
use by other Windows components; earlier versions shipped with 
the CRTDLL.DLL library instead. In older versions of Windows, 
programs which linked against MSVCRT.DLL were expected to install 
a compatible copy in the System32 folder, but this contributed to 
DLL Hell because many installers failed to check the library 
version against the installed version before replacing it.

Versions of MSVC before 4.0 and from 7.0 to 13.0 used differently 
named DLLs for each version (MSVCR20.DLL, MSVCR70.DLL, 
MSVCR71.DLL, MSVCP110.DLL, etc.). Applications are required to 
install the appropriate version,[16] and Microsoft offers Visual 
C++ Redistributable packages for this purpose, though Windows 
typically comes with one version already installed.

With Version 14.0, most of the C/C++ runtime was moved into a new 
DLL, UCRTBASE.DLL. However, C/C++ programs using UCRTBASE.DLL are 
forced to link against another new DLL, the VCRuntime, whose name 
continues to change with each version of MSVC (e.g. 
VCRUNTIME140.DLL).

Source code for runtime libraries is included in Visual C++[17] 
for reference and debugging (e.g. in C:\Program Files\Microsoft 
Visual Studio 11.0\VC\crt\src).

This runtime library is used by programs written in Visual C++ 
and a few other compilers (e.g. MinGW). Some compilers have their 
own runtime libraries.
May 22 2021