www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Read to stdout doesn't trigger correct action

reply IGotD- <nise nise.com> writes:
I've done some adaptations to druntime for another C library that 
isn't currently supported. Obtaining the FILE* structure of the 
clib is done via a function call rather than global variables. 
However this function call is never triggered when issuing a 
writeln function call. The FILE* structure is a pointer that 
points to a completely wrong location.

Reading core.stdc.stdio.stdin and core.stdc.stdio.stdout 
explicitly will trigger the function call and the correct 
addresses can be read.

The function writeln seems to obtain the std FILE* structures is 
" property ref File makeGlobal(StdFileHandle _iob)()" in 
std.stdio.d.


// Undocumented but public because the std* handles are aliasing 
it.
 property ref File makeGlobal(StdFileHandle _iob)()
{
     __gshared File.Impl impl;
     __gshared File result;

     // Use an inline spinlock to make sure the initializer is 
only run once.
     // We assume there will be at most uint.max / 2 threads 
trying to initialize
     // `handle` at once and steal the high bit to indicate that 
the globals have
     // been initialized.
     static shared uint spinlock;
     import core.atomic : atomicLoad, atomicOp, MemoryOrder;
     if (atomicLoad!(MemoryOrder.acq)(spinlock) <= uint.max / 2)
     {
         for (;;)
         {
             if (atomicLoad!(MemoryOrder.acq)(spinlock) > uint.max 
/ 2)
                 break;
             if (atomicOp!"+="(spinlock, 1) == 1)
             {
                 with (StdFileHandle)
                     assert(_iob == stdin || _iob == stdout || 
_iob == stderr);
                 impl.handle = mixin(_iob);
                 result._p = &impl;
                 atomicOp!"+="(spinlock, uint.max / 2);
                 break;
             }
             atomicOp!"-="(spinlock, 1);
         }
     }
     return result;
}


This seems do some atomic operation preventing the D File class 
for stdio not to be initialized several times. I'm not quite sure 
if this is global or per thread but I guess it is for the entire 
process. For some reason the std File classes are never 
initialized at all. Another problem is that the function that is 
called to obtain the clib stdin/out use a structure that is lazy 
initialized per thread, so it must be called at least a first 
time for each thread in order to get the correct stdin/out. 
Removing the atomic operations so that the File initialization is 
done every time, then it works.

Question is if "File makeGlobal(StdFileHandle _iob)()" is correct 
when it comes to ensure compatibility among all the clib versions 
out there. Not too seldom are clib global variables really 
functions, like errno is often a function rather than a variable. 
The internal code the the clib (Newlib) does not have this 
"optimization" but always get stdin/out using this function call.
Jun 22 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/22/20 10:05 AM, IGotD- wrote:


 This seems do some atomic operation preventing the D File class for 
 stdio not to be initialized several times. I'm not quite sure if this is 
 global or per thread but I guess it is for the entire process. For some 
 reason the std File classes are never initialized at all. Another 
 problem is that the function that is called to obtain the clib stdin/out 
 use a structure that is lazy initialized per thread, so it must be 
 called at least a first time for each thread in order to get the correct 
 stdin/out. Removing the atomic operations so that the File 
 initialization is done every time, then it works.


The point of the makeGlobal call is to avoid dependency on static ctors 
(which is how it was done before this implementation), to allow usage of 
stdio without druntime.

The __gshared designation is equivalent to C's globals, so it is shared 
among all threads (though the type is still as if it were thread-local).


 Question is if "File makeGlobal(StdFileHandle _iob)()" is correct when 
 it comes to ensure compatibility among all the clib versions out there. 
 Not too seldom are clib global variables really functions, like errno is 
 often a function rather than a variable. The internal code the the clib 
 (Newlib) does not have this "optimization" but always get stdin/out 
 using this function call.


I'm sure if there is a clib that doesn't work with this, it is a bug 
with druntime, and should be addressed. I don't know enough about the 
exact functionality to be able to write such a bug report, but you 
probably should if it's not working for you.

-Steve
Jun 22 2020
parent IGotD- <nise nise.com> writes:
On Monday, 22 June 2020 at 14:27:18 UTC, Steven Schveighoffer 
wrote:

 I'm sure if there is a clib that doesn't work with this, it is 
 a bug with druntime, and should be addressed. I don't know 
 enough about the exact functionality to be able to write such a 
 bug report, but you probably should if it's not working for you.

 -Steve


I don't really have a good solution for this and it seems to work 
for existing supported C libraries. One solution would be to move 
the makeGlobal implementation to become OS/C-library specific and 
each platform would have a suitable implementation but that kind 
of contradicts having as little platform specific code possible.

Another thing that is a bit sketchy here is the usage of raw 
atomics as spinlocks as it would have side effects in case of 
locking (CPU could spin for several milliseconds if the locker 
CPU is preempted and does something else). I guess in this 
particular case it is used because it would be race creating a 
mutex as well because we want lazy initialization. While I kind 
of like lazy initialization myself it kind creates situations 
like these. I don't quite follow the exact reason behind avoiding 
static ctors, lazy initialization is nice for many things but 
ctors have their place as well. Using C stdio without druntime 
would read the OS specific FILE* directly, or what am I missing 
here?
Jun 22 2020