www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Cannot get thread ID with Thread.getThis() in specific callback functions

reply Haruki Shigemori <rayerd.wiz gmail.com> writes:
Hi.
I cannot get thread ID with Thread.getThis() in specific callback
functions on Windows.
What is the cause of this problem?

import win32.windows;
import win32.mmsystem;
import std.stdio;
import core.thread;

extern (Windows)
void waveInProc(in HWAVEIN handle, in uint message, in DWORD instance,
in DWORD param1, in DWORD param2)
{
        auto tid = Thread.getThis();
        assert(tid); // tid is null
}

void main()
{
        WAVEFORMATEX formatEx;
        with (formatEx)
        {
                wFormatTag = WAVE_FORMAT_PCM;
                nChannels = 1;
                nSamplesPerSec = 44100;
                wBitsPerSample = 16;
                nBlockAlign = cast(ushort)(wBitsPerSample * nChannels / 8);
                nAvgBytesPerSec = nSamplesPerSec * nBlockAlign;
        }

        HWAVEIN handle;
        waveInOpen(&handle, WAVE_MAPPER, cast(WAVEFORMATEX*)&formatEx,
cast(DWORD)&waveInProc, cast(DWORD)null, CALLBACK_FUNCTION);

        uint bufferSize = cast(uint)(formatEx.nAvgBytesPerSec *
1/+second+/);
        WAVEHDR* hdr = new WAVEHDR;
        hdr.lpData = cast(LPSTR)new ushort[bufferSize];
        hdr.dwBufferLength = bufferSize;
        hdr.dwFlags = 0;
        hdr.dwLoops = 0;

        waveInPrepareHeader(handle, hdr, WAVEHDR.sizeof);
        waveInAddBuffer(handle, hdr, WAVEHDR.sizeof);
        waveInStart(handle);

        readln();
}
Dec 05 2010
parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Mon, 06 Dec 2010 06:38:40 +0300, Haruki Shigemori  
<rayerd.wiz gmail.com> wrote:

 Hi.
 I cannot get thread ID with Thread.getThis() in specific callback
 functions on Windows.
 What is the cause of this problem?

 import win32.windows;
 import win32.mmsystem;
 import std.stdio;
 import core.thread;

 extern (Windows)
 void waveInProc(in HWAVEIN handle, in uint message, in DWORD instance,
 in DWORD param1, in DWORD param2)
 {
         auto tid = Thread.getThis();
         assert(tid); // tid is null
 }

 void main()
 {
         WAVEFORMATEX formatEx;
         with (formatEx)
         {
                 wFormatTag = WAVE_FORMAT_PCM;
                 nChannels = 1;
                 nSamplesPerSec = 44100;
                 wBitsPerSample = 16;
                 nBlockAlign = cast(ushort)(wBitsPerSample * nChannels /  
 8);
                 nAvgBytesPerSec = nSamplesPerSec * nBlockAlign;
         }

         HWAVEIN handle;
         waveInOpen(&handle, WAVE_MAPPER, cast(WAVEFORMATEX*)&formatEx,
 cast(DWORD)&waveInProc, cast(DWORD)null, CALLBACK_FUNCTION);

         uint bufferSize = cast(uint)(formatEx.nAvgBytesPerSec *
 1/+second+/);
         WAVEHDR* hdr = new WAVEHDR;
         hdr.lpData = cast(LPSTR)new ushort[bufferSize];
         hdr.dwBufferLength = bufferSize;
         hdr.dwFlags = 0;
         hdr.dwLoops = 0;

         waveInPrepareHeader(handle, hdr, WAVEHDR.sizeof);
         waveInAddBuffer(handle, hdr, WAVEHDR.sizeof);
         waveInStart(handle);

         readln();
 }

Because D runtime doesn't know about the newly created thread, and static constructors weren't called. Try calling the following in your callbacks: if (Thread.getThis() is null) thread_attachThis(); // if (dont_know_about_this_thread) initialize_it();
Dec 05 2010
parent reply Haruki Shigemori <rayerd.wiz gmail.com> writes:
(2010/12/06 16:59), Denis Koroskin wrote:
 Because D runtime doesn't know about the newly created thread, and
 static constructors weren't called. Try calling the following in your
 callbacks:

 if (Thread.getThis() is null) thread_attachThis(); // if
 (dont_know_about_this_thread) initialize_it();

Wow... Is this D's design? I want phobos or druntime to take care of this problem. Is it difficult?
Dec 06 2010
parent reply Mike Parker <aldacron gmail.com> writes:
On 12/7/2010 2:10 AM, Haruki Shigemori wrote:
 (2010/12/06 16:59), Denis Koroskin wrote:
 Because D runtime doesn't know about the newly created thread, and
 static constructors weren't called. Try calling the following in your
 callbacks:

 if (Thread.getThis() is null) thread_attachThis(); // if
 (dont_know_about_this_thread) initialize_it();

Wow... Is this D's design? I want phobos or druntime to take care of this problem. Is it difficult?

The problem is that the thread is created outside of druntime, internal to the Windows API. How is druntime to know about it unless you tell it?
Dec 06 2010
parent reply Max Samukha <spambox d-coding.com> writes:
On 12/06/2010 10:47 PM, Mike Parker wrote:
 On 12/7/2010 2:10 AM, Haruki Shigemori wrote:
 (2010/12/06 16:59), Denis Koroskin wrote:
 Because D runtime doesn't know about the newly created thread, and
 static constructors weren't called. Try calling the following in your
 callbacks:

 if (Thread.getThis() is null) thread_attachThis(); // if
 (dont_know_about_this_thread) initialize_it();

Wow... Is this D's design? I want phobos or druntime to take care of this problem. Is it difficult?

The problem is that the thread is created outside of druntime, internal to the Windows API. How is druntime to know about it unless you tell it?

This could be done by installing a kernel driver that would use PsSetCreateThreadNotifyRoutine (if I remember the name correctly) to register a callback that would use an IPC mechanism to notify druntime about new threads created by the system. Chances that this approach will be adopted by druntime are less than zero.
Dec 07 2010
parent reply Haruki Shigemori <rayerd.wiz gmail.com> writes:
(2010/12/07 20:02), Max Samukha wrote:
 On 12/06/2010 10:47 PM, Mike Parker wrote:
 On 12/7/2010 2:10 AM, Haruki Shigemori wrote:
 (2010/12/06 16:59), Denis Koroskin wrote:
 Because D runtime doesn't know about the newly created thread, and
 static constructors weren't called. Try calling the following in your
 callbacks:

 if (Thread.getThis() is null) thread_attachThis(); // if
 (dont_know_about_this_thread) initialize_it();

Wow... Is this D's design? I want phobos or druntime to take care of this problem. Is it difficult?

The problem is that the thread is created outside of druntime, internal to the Windows API. How is druntime to know about it unless you tell it?

This could be done by installing a kernel driver that would use PsSetCreateThreadNotifyRoutine (if I remember the name correctly) to register a callback that would use an IPC mechanism to notify druntime about new threads created by the system. Chances that this approach will be adopted by druntime are less than zero.

As you said, it seems difficult to run away from this problem. But, this problem causes the phenomenon that is hard to reappear. import win32.windows; import win32.mmsystem; import std.stdio; import core.thread; extern (Windows) void waveInProc(in HWAVEIN handle, in uint message, in DWORD instance, in DWORD param1, in DWORD param2) { int[] a; int[] b = a.dup; // Access violation occurs in druntime rarely. //auto tid = Thread.getThis(); //assert(tid); // assertion failure } void main() { foreach (_; 0..1000) // many times { WAVEFORMATEX formatEx; with (formatEx) { wFormatTag = WAVE_FORMAT_PCM; nChannels = 1; nSamplesPerSec = 44100; wBitsPerSample = 16; nBlockAlign = cast(ushort)(wBitsPerSample * nChannels / 8); nAvgBytesPerSec = nSamplesPerSec * nBlockAlign; } HWAVEIN handle; waveInOpen(&handle, WAVE_MAPPER, cast(WAVEFORMATEX*)&formatEx, cast(DWORD)&waveInProc, cast(DWORD)null, CALLBACK_FUNCTION); uint bufferSize = cast(uint)(formatEx.nAvgBytesPerSec * 1/+second+/); WAVEHDR* hdr = new WAVEHDR; hdr.lpData = cast(LPSTR)new ushort[bufferSize]; hdr.dwBufferLength = bufferSize; hdr.dwFlags = 0; hdr.dwLoops = 0; waveInPrepareHeader(handle, hdr, WAVEHDR.sizeof); waveInAddBuffer(handle, hdr, WAVEHDR.sizeof); waveInStart(handle); waveInClose(handle); } } Because the following "t" is null in core.thread.getThis rarely. Why? I think that we should react to this problem. core.thread.d(1214): t = thread_findByAddr( GetCurrentThreadId() );
Dec 09 2010
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
A DLL gets notified in its entrypoint DllMain about new threads. When it 
receives such a notification, it mainly does the following:

             if( !thread_findByAddr( GetCurrentThreadId() ) )
                 thread_attachThis();
             if( !_moduleinfo_tlsdtors_i ) // avoid duplicate calls
                 _moduleTlsCtor();

You might want to try inserting this snippet into your callback.

Rainer

Haruki Shigemori wrote:
 (2010/12/07 20:02), Max Samukha wrote:
 On 12/06/2010 10:47 PM, Mike Parker wrote:
 On 12/7/2010 2:10 AM, Haruki Shigemori wrote:
 (2010/12/06 16:59), Denis Koroskin wrote:
 Because D runtime doesn't know about the newly created thread, and
 static constructors weren't called. Try calling the following in your
 callbacks:

 if (Thread.getThis() is null) thread_attachThis(); // if
 (dont_know_about_this_thread) initialize_it();

Wow... Is this D's design? I want phobos or druntime to take care of this problem. Is it difficult?

The problem is that the thread is created outside of druntime, internal to the Windows API. How is druntime to know about it unless you tell it?

This could be done by installing a kernel driver that would use PsSetCreateThreadNotifyRoutine (if I remember the name correctly) to register a callback that would use an IPC mechanism to notify druntime about new threads created by the system. Chances that this approach will be adopted by druntime are less than zero.

As you said, it seems difficult to run away from this problem. But, this problem causes the phenomenon that is hard to reappear. import win32.windows; import win32.mmsystem; import std.stdio; import core.thread; extern (Windows) void waveInProc(in HWAVEIN handle, in uint message, in DWORD instance, in DWORD param1, in DWORD param2) { int[] a; int[] b = a.dup; // Access violation occurs in druntime rarely. //auto tid = Thread.getThis(); //assert(tid); // assertion failure } void main() { foreach (_; 0..1000) // many times { WAVEFORMATEX formatEx; with (formatEx) { wFormatTag = WAVE_FORMAT_PCM; nChannels = 1; nSamplesPerSec = 44100; wBitsPerSample = 16; nBlockAlign = cast(ushort)(wBitsPerSample * nChannels / 8); nAvgBytesPerSec = nSamplesPerSec * nBlockAlign; } HWAVEIN handle; waveInOpen(&handle, WAVE_MAPPER, cast(WAVEFORMATEX*)&formatEx, cast(DWORD)&waveInProc, cast(DWORD)null, CALLBACK_FUNCTION); uint bufferSize = cast(uint)(formatEx.nAvgBytesPerSec * 1/+second+/); WAVEHDR* hdr = new WAVEHDR; hdr.lpData = cast(LPSTR)new ushort[bufferSize]; hdr.dwBufferLength = bufferSize; hdr.dwFlags = 0; hdr.dwLoops = 0; waveInPrepareHeader(handle, hdr, WAVEHDR.sizeof); waveInAddBuffer(handle, hdr, WAVEHDR.sizeof); waveInStart(handle); waveInClose(handle); } } Because the following "t" is null in core.thread.getThis rarely. Why? I think that we should react to this problem. core.thread.d(1214): t = thread_findByAddr( GetCurrentThreadId() );

Dec 09 2010
parent reply Haruki Shigemori <rayerd.wiz gmail.com> writes:
(2010/12/10 5:14), Rainer Schuetze wrote:
 A DLL gets notified in its entrypoint DllMain about new threads. When it
 receives such a notification, it mainly does the following:

 if( !thread_findByAddr( GetCurrentThreadId() ) )
 thread_attachThis();
 if( !_moduleinfo_tlsdtors_i ) // avoid duplicate calls
 _moduleTlsCtor();

 You might want to try inserting this snippet into your callback.

 Rainer

Thank you! $ type a.d import win32.windows; import win32.mmsystem; import std.stdio; import core.thread; extern (Windows) void waveInProc(in HWAVEIN handle, in uint message, in DWORD instance, in DWORD param1, in DWORD param2) { if( !thread_findByAddr( GetCurrentThreadId() ) ) thread_attachThis(); if( !_moduleinfo_tlsdtors_i ) // avoid duplicate calls _moduleTlsCtor(); int[] a; int[] b = a.dup; } ... $ bud a /Ldetailedmap -g -full ... $ ddbg a -> r ... Unhandled Exception: EXCEPTION_ACCESS_VIOLATION(0xc0000005) at gc.gcx.Gcx.__invariant src\gc\gcx.d:1553 (0x0041e507) thread(6136) ->q Ummm...?
Dec 09 2010
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
I tried your code with the additions, and it works (dmd-2.050 on XP/32).

If you are on some 64-bit system: I've seen threads without any TLS set 
up at all. If you check the disassembly at the crash location, and there 
is a read from FS:[0x2c] that results in 0, then you are probably 
hitting this problem.

It is rather delicate to modify the windows TLS data structures, so an 
option might be to not touch any TLS in the waveInProc (including any 
memory allocations), but to just set an event to notify another thread 
that has been created with "new Thread".

Rainer

Haruki Shigemori wrote:
 (2010/12/10 5:14), Rainer Schuetze wrote:
 A DLL gets notified in its entrypoint DllMain about new threads. When it
 receives such a notification, it mainly does the following:

 if( !thread_findByAddr( GetCurrentThreadId() ) )
 thread_attachThis();
 if( !_moduleinfo_tlsdtors_i ) // avoid duplicate calls
 _moduleTlsCtor();

 You might want to try inserting this snippet into your callback.

 Rainer

Thank you! $ type a.d import win32.windows; import win32.mmsystem; import std.stdio; import core.thread; extern (Windows) void waveInProc(in HWAVEIN handle, in uint message, in DWORD instance, in DWORD param1, in DWORD param2) { if( !thread_findByAddr( GetCurrentThreadId() ) ) thread_attachThis(); if( !_moduleinfo_tlsdtors_i ) // avoid duplicate calls _moduleTlsCtor(); int[] a; int[] b = a.dup; } .... $ bud a /Ldetailedmap -g -full .... $ ddbg a -> r .... Unhandled Exception: EXCEPTION_ACCESS_VIOLATION(0xc0000005) at gc.gcx.Gcx.__invariant src\gc\gcx.d:1553 (0x0041e507) thread(6136) ->q Ummm...?

Dec 10 2010
parent reply Haruki Shigemori <rayerd.wiz gmail.com> writes:
Hello.

(2010/12/10 17:18), Rainer Schuetze wrote:
 I tried your code with the additions, and it works (dmd-2.050 on XP/32).

Oh, Sorry. It didn't work on Vista/64 (dmd-r795, druntime-r444 and Phobos-r2217).
 If you are on some 64-bit system: I've seen threads without any TLS set
 up at all. If you check the disassembly at the crash location, and there
 is a read from FS:[0x2c] that results in 0, then you are probably
 hitting this problem.

I don't know the assembly...
 It is rather delicate to modify the windows TLS data structures, so an
 option might be to not touch any TLS in the waveInProc (including any
 memory allocations), but to just set an event to notify another thread
 that has been created with "new Thread".

I understood that any memory allocations were not possible in these functions. Though I wanted to use std.concurrency.send, but it is impossible. Thank you.
Dec 10 2010
parent Haruki Shigemori <rayerd.wiz gmail.com> writes:
 (2010/12/10 17:18), Rainer Schuetze wrote:
 It is rather delicate to modify the windows TLS data structures, so an
 option might be to not touch any TLS in the waveInProc (including any
 memory allocations), but to just set an event to notify another thread
 that has been created with "new Thread".

I understood that any memory allocations were not possible in these functions. Though I wanted to use std.concurrency.send, but it is impossible. Thank you.

Cannot we use some memory allocations such as "assert" and "new Exception"?
Dec 13 2010