www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ldc - TLS for Android

reply "Joakim" <joakim airpost.net> writes:
So I've been looking into implementing TLS for Android/x86, 
rummaging through old TLS git commits for dmd and ldc to see what 
to do.  It appears that Walter implemented TLS on OS X more than 
four years ago by packing thread-local variables into special 
segments and then unpacking them in druntime, which uses 
pthread_(get|set)specific on OS X nowadays:

http://www.drdobbs.com/architecture-and-design/implementing-thread-local-storage-on-os/228701185
https://github.com/D-Programming-Language/druntime/blob/master/src/rt/sections_osx.d#L106

Since Android also provides these pthread functions for TLS, 
seems like a similar approach is called for.

I notice that ldc never used this approach, depending on llvm's 
built-in TLS support instead:

https://github.com/ldc-developers/ldc/commit/4d7a6eda234bc8d12703cc577c09c2ca50ac6bda#diff-19

It seems that this also meant that TLS wasn't garbage-collected 
on OSX, until David added it a little more than a year ago:

https://github.com/ldc-developers/druntime/blob/ldc/src/ldc/osx_tls.c

I can copy what dmd is doing on OS X Mach-O with ELF, but it's 
not going to be easily transferable to ldc, which will be 
necessary for Android/ARM.

Do you have any advice on how to pull this off with ldc?  Should 
I be going the dmd route and packing the TLS myself?  Does llvm 
provide good support for this?

Or is there some other llvm TLS shortcut I can use?  I tried to 
see if llvm just has some thread-local implementation that 
automatically uses pthread_setspecific, but didn't find anything.
Mar 07 2014
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2014-03-08 01:55, Joakim wrote:
 So I've been looking into implementing TLS for Android/x86, rummaging
 through old TLS git commits for dmd and ldc to see what to do.  It
 appears that Walter implemented TLS on OS X more than four years ago by
 packing thread-local variables into special segments and then unpacking
 them in druntime, which uses pthread_(get|set)specific on OS X nowadays:

 http://www.drdobbs.com/architecture-and-design/implementing-thread-local-storage-on-os/228701185

 https://github.com/D-Programming-Language/druntime/blob/master/src/rt/sections_osx.d#L106


 Since Android also provides these pthread functions for TLS, seems like
 a similar approach is called for.

 I notice that ldc never used this approach, depending on llvm's built-in
 TLS support instead:

 https://github.com/ldc-developers/ldc/commit/4d7a6eda234bc8d12703cc577c09c2ca50ac6bda#diff-19
Yes. DMD started to implemented support for TLS on OS X before 10.7 which is the first version of OS X to natively support TLS. LDC doesn't support older versions of OS X than 10.7 since it uses native TLS. -- /Jacob Carlborg
Mar 08 2014
prev sibling parent reply David Nadlinger <code klickverbot.at> writes:
On 03/08/2014 01:55 AM, Joakim wrote:
 Do you have any advice on how to pull this off with ldc?  Should I be
 going the dmd route and packing the TLS myself?  Does llvm provide good
 support for this?

 Or is there some other llvm TLS shortcut I can use?  I tried to see if
 llvm just has some thread-local implementation that automatically uses
 pthread_setspecific, but didn't find anything.
LLVM does support putting variables into custom sections, and you can more or less get away with the DMD bracketing approach (see e.g. the new ModuleInfo discovery functionality I implemented for Linux, which is the same as DMD's druntime uses). However, there is a catch: Due to what I can only imagine is a bug, LLVM does not support emitting a symbol both into a custom section and with weak linkage. Thus, you might be in for a round of LLVM hacking either way, even though it will likely involve much less when going the DMD route. However, there is a third options which might be worth investigating, namely re-implementing at least parts of the necessary runtime linker features in druntime and continuing to use the same scheme as on GNU Linux/x86. This depends on %gs not being used in another way, etc. though. David
Mar 08 2014
next sibling parent reply "Joakim" <joakim airpost.net> writes:
On Saturday, 8 March 2014 at 14:25:43 UTC, David Nadlinger wrote:
 LLVM does support putting variables into custom sections, and 
 you can more or less get away with the DMD bracketing approach 
 (see e.g. the new ModuleInfo discovery functionality I 
 implemented for Linux, which is the same as DMD's druntime 
 uses).
You're talking about findDataSection and friends? https://github.com/ldc-developers/druntime/blob/ldc/src/rt/sections_ldc.d#L115
 However, there is a catch: Due to what I can only imagine is a 
 bug, LLVM does not support emitting a symbol both into a custom 
 section and with weak linkage. Thus, you might be in for a 
 round of LLVM hacking either way, even though it will likely 
 involve much less when going the DMD route.
Hmm, I guess this is why you don't use the bracketing approach anywhere? What will be much less when going the DMD route?
 However, there is a third options which might be worth 
 investigating, namely re-implementing at least parts of the 
 necessary runtime linker features in druntime and continuing to 
 use the same scheme as on GNU Linux/x86. This depends on %gs 
 not being used in another way, etc. though.
I tried to reuse the existing dl_iterate_phdr approach on Android, but then I noticed that the dl_phdr_info struct defined in bionic doesn't include the dlpi_tls_modid and dlpi_tls_data members. However, now that you mention it, maybe those aren't strictly necessary, as long as I'm not worried about shared libraries. I'll look into it further. As for reimplementing the runtime linker, in a sense that's what's being done with dmd/druntime for OS X, where it implements it's own ___tls_get_addr using pthread_setspecific. I'll have to do the same for Android, as bionic doesn't have a __tls_get_addr.
Mar 08 2014
parent reply David Nadlinger <code klickverbot.at> writes:
On Sat, Mar 8, 2014 at 7:16 PM, Joakim <joakim airpost.net> wrote:
 On Saturday, 8 March 2014 at 14:25:43 UTC, David Nadlinger wrote:
 LLVM does support putting variables into custom sections, and you can more
 or less get away with the DMD bracketing approach (see e.g. the new
 ModuleInfo discovery functionality I implemented for Linux, which is the
 same as DMD's druntime uses).
You're talking about findDataSection and friends? https://github.com/ldc-developers/druntime/blob/ldc/src/rt/sections_ldc.d#L115
Not quite. I was referring to https://github.com/ldc-developers/druntime/blob/ldc-merge-2.064/src/rt/sections_linux.d (_d_dso_registry, ...) and the associated compiler-side implementation, https://github.com/ldc-developers/ldc/blob/5b14a5e5c4f292024afd8e5f520e837035942003/gen/module.cpp#L396.
 However, there is a catch: Due to what I can only imagine is a bug, LLVM
 does not support emitting a symbol both into a custom section and with weak
 linkage. Thus, you might be in for a round of LLVM hacking either way, even
 though it will likely involve much less when going the DMD route.
Hmm, I guess this is why you don't use the bracketing approach anywhere? What will be much less when going the DMD route?
Actually, we didn't use the special section approach at all until very recently (i.e. Martin's shared library changes in 2.064). And I meant that you would probably get away with less LLVM hacking when just changing the way LDC emits TLS globals/accesses than when implementing "emulated" TLS on the LLVM backend side.
 As for reimplementing the runtime linker, in a sense that's what's being
 done with dmd/druntime for OS X, where it implements it's own
 ___tls_get_addr using pthread_setspecific.  I'll have to do the same for
 Android, as bionic doesn't have a __tls_get_addr.
Well, yes and no. I was specifically referring to keeping the normal TLS infrastructure (i.e. %gs-based addressing on Linux/x86) in place and just replacing the part that Glibc does (but Bionic doesn't) with a piece of code in druntime. __tls_get_addr isn't necessarily used on x86. David
Mar 08 2014
parent reply "Joakim" <joakim airpost.net> writes:
On Saturday, 8 March 2014 at 22:44:16 UTC, David Nadlinger wrote:
 On Sat, Mar 8, 2014 at 7:16 PM, Joakim <joakim airpost.net> 
 wrote:
 On Saturday, 8 March 2014 at 14:25:43 UTC, David Nadlinger 
 wrote:
 LLVM does support putting variables into custom sections, and 
 you can more
 or less get away with the DMD bracketing approach (see e.g. 
 the new
 ModuleInfo discovery functionality I implemented for Linux, 
 which is the
 same as DMD's druntime uses).
You're talking about findDataSection and friends? https://github.com/ldc-developers/druntime/blob/ldc/src/rt/sections_ldc.d#L115
Not quite. I was referring to https://github.com/ldc-developers/druntime/blob/ldc-merge-2.064/src/rt/sections_linux.d (_d_dso_registry, ...) and the associated compiler-side implementation, https://github.com/ldc-developers/ldc/blob/5b14a5e5c4f292024afd8e5f520e837035942003/gen/module.cpp#L396.
Okay, I started looking around the master branch and didn't find what you were talking about. No wonder, it's in the merge-2.064 branch. I'll look at what you did there.
 However, there is a catch: Due to what I can only imagine is 
 a bug, LLVM
 does not support emitting a symbol both into a custom section 
 and with weak
 linkage. Thus, you might be in for a round of LLVM hacking 
 either way, even
 though it will likely involve much less when going the DMD 
 route.
Hmm, I guess this is why you don't use the bracketing approach anywhere? What will be much less when going the DMD route?
Actually, we didn't use the special section approach at all until very recently (i.e. Martin's shared library changes in 2.064). And I meant that you would probably get away with less LLVM hacking when just changing the way LDC emits TLS globals/accesses than when implementing "emulated" TLS on the LLVM backend side.
Well, the special section approach still isn't in the master branch, hence my confusion. Okay, I wasn't clear that you were comparing the dmd route to having llvm generate the right pthread calls for Android.
 As for reimplementing the runtime linker, in a sense that's 
 what's being
 done with dmd/druntime for OS X, where it implements it's own
 ___tls_get_addr using pthread_setspecific.  I'll have to do 
 the same for
 Android, as bionic doesn't have a __tls_get_addr.
Well, yes and no. I was specifically referring to keeping the normal TLS infrastructure (i.e. %gs-based addressing on Linux/x86) in place and just replacing the part that Glibc does (but Bionic doesn't) with a piece of code in druntime. __tls_get_addr isn't necessarily used on x86.
While Android/X86 TLS does use the %gs register (https://github.com/android/platform_bionic/blob/master/libc/priva e/__get_tls.h#L45), that's not portable and I'd like to try Android/ARM after this, so I'll stick with the pthread_(get|set)specific calls to wrap it: https://github.com/android/platform_bionic/blob/master/libc/bionic/pthread_key.cpp
Mar 08 2014
parent reply "Joakim" <joakim airpost.net> writes:
On Sunday, 9 March 2014 at 05:38:07 UTC, Joakim wrote:
 On Saturday, 8 March 2014 at 22:44:16 UTC, David Nadlinger 
 wrote:
 Well, yes and no. I was specifically referring to keeping the 
 normal
 TLS infrastructure (i.e. %gs-based addressing on Linux/x86) in 
 place
 and just replacing the part that Glibc does (but Bionic 
 doesn't) with
 a piece of code in druntime. __tls_get_addr isn't necessarily 
 used on
 x86.
While Android/X86 TLS does use the %gs register (https://github.com/android/platform_bionic/blob/master/libc/priva e/__get_tls.h#L45), that's not portable and I'd like to try Android/ARM after this, so I'll stick with the pthread_(get|set)specific calls to wrap it: https://github.com/android/platform_bionic/blob/master/libc/bionic/pthread_key.cpp
You mention "replacing the part that Glibc does (but Bionic doesn't) with a piece of code in druntime." Just to be clear, you're referring to accessing TLS variables using an offset into the initialization image, which is what ___tls_get_addr from druntime does in Walter's packed TLS approach, right? If not, I'm not sure exactly what you're referring to. With all this TLS stuff split up between the compiler, linker, and runtime linker, often undocumented or poorly documenented in the latter two cases, it's been confusing to follow the TLS code path to see what's happening.
Mar 08 2014
parent reply "David Nadlinger" <code klickverbot.at> writes:
On 9 Mar 2014, at 8:36, Joakim wrote:
 You mention "replacing the part that Glibc does (but Bionic doesn't) 
 with a piece of code in druntime."  Just to be clear, you're referring 
 to accessing TLS variables using an offset into the initialization 
 image, which is what ___tls_get_addr from druntime does in Walter's 
 packed TLS approach, right?  If not, I'm not sure exactly what you're 
 referring to.  With all this TLS stuff split up between the compiler, 
 linker, and runtime linker, often undocumented or poorly documenented 
 in the latter two cases, it's been confusing to follow the TLS code 
 path to see what's happening.
There are several possible ABIs for thread-local storage. For the sake of this argument, let's assume that our particular system works like the Linux/x86 implementation or Walter's OS X approach in that the TLS storage area is simply a flat block of memory where the individual variables reside at some offset. Then, there is still the question of how the application knows a) the base address of the block and b) the offset of the variable of interest. In Walter's OS X implementation, both is taken care of by __tls_get_addr, which expects a pointer into the section where the TLS initialization data is stored. On e.g. Linux/x86_64, however, the base address is stored in %fs, and the offset is provided by special linker relocations (which essentially evaluate to the offset of a given symbol from the beginning of the initialization image). No extra function calls are inserted by the compiler here to access TLS data, and the (C) runtime is not directly involved for the accesses. For an overview of the different models, see http://www.akkadia.org/drepper/tls.pdf (which is the most comprehensive document I could find, in spite of what you might think about the author). But regardless of what model is chosen, there is still the issue of actually setting up a copy of the data for each thread during initialization. This is what I was referring to when I mentioned "replacing the part that Glibc does (but Bionic doesn't) with a piece of code in druntime". So, if %gs works as expected on Android and the linker supports the necessary relocations, then it might be an option to simply use the existing TLS implementation in LLVM and simply provide the missing bits in druntime. On the other hand, if you choose to go with an entirely different TLS scheme (such as the DMD OS X implementation), you need to figure out how to change the codegen to emit the extra function calls to your __tls_get_addr analog, etc. Looking at llvm/lib/Target/ARM/ARMISelLowering.cpp, there might actually be a working implementation for this in LLVM already (which I didn't realize before), so this route would not necessarily be more complex than going with a different scheme. You'd probably just need to provide the __tls_get_addr implementation in druntime and figure out how LLVM emits the TLS image resp. how to get its base address. Hope this helps, David
Mar 09 2014
parent reply "Joakim" <joakim airpost.net> writes:
On Sunday, 9 March 2014 at 16:12:19 UTC, David Nadlinger wrote:
 On 9 Mar 2014, at 8:36, Joakim wrote:
 You mention "replacing the part that Glibc does (but Bionic 
 doesn't) with a piece of code in druntime."  Just to be clear, 
 you're referring to accessing TLS variables using an offset 
 into the initialization image, which is what ___tls_get_addr 
 from druntime does in Walter's packed TLS approach, right?  If 
 not, I'm not sure exactly what you're referring to.  With all 
 this TLS stuff split up between the compiler, linker, and 
 runtime linker, often undocumented or poorly documenented in 
 the latter two cases, it's been confusing to follow the TLS 
 code path to see what's happening.
There are several possible ABIs for thread-local storage. For the sake of this argument, let's assume that our particular system works like the Linux/x86 implementation or Walter's OS X approach in that the TLS storage area is simply a flat block of memory where the individual variables reside at some offset. Then, there is still the question of how the application knows a) the base address of the block and b) the offset of the variable of interest. In Walter's OS X implementation, both is taken care of by __tls_get_addr, which expects a pointer into the section where the TLS initialization data is stored. On e.g. Linux/x86_64, however, the base address is stored in %fs, and the offset is provided by special linker relocations (which essentially evaluate to the offset of a given symbol from the beginning of the initialization image). No extra function calls are inserted by the compiler here to access TLS data, and the (C) runtime is not directly involved for the accesses. For an overview of the different models, see http://www.akkadia.org/drepper/tls.pdf (which is the most comprehensive document I could find, in spite of what you might think about the author).
Yeah, I've had that pdf loaded in my browser for the last couple months, skimmed some of it initially and I've been slowly going through it in more detail. I tried simply loading a binary built using bracketed sections and the linker's current TLS relocations, ie no extra function calls, in Android/x86 and I got some other random data in the resulting TLS initialization image. I think this is because bionic stores the pthread_setspecific-created void* pointers in the normal TLS area, so you can't just use the TLS relocations that dmd and the gold linker generate for linux/x86 on Android/x86, ie using the %gs register directly. I have no opinion on the author, should I? ;)
 But regardless of what model is chosen, there is still the 
 issue of actually setting up a copy of the data for each thread 
 during initialization. This is what I was referring to when I 
 mentioned "replacing the part that Glibc does (but Bionic 
 doesn't) with a piece of code in druntime".
I was finally able to access a proper initialization image created by dmd in druntime on Android/x86 a couple hours back, by using dl_phdr_info similarly to what is done on linux now.
 So, if %gs works as expected on Android and the linker supports 
 the necessary relocations, then it might be an option to simply 
 use the existing TLS implementation in LLVM and simply provide 
 the missing bits in druntime. On the other hand, if you choose 
 to go with an entirely different TLS scheme (such as the DMD OS 
 X implementation), you need to figure out how to change the 
 codegen to emit the extra function calls to your __tls_get_addr 
 analog, etc. Looking at 
 llvm/lib/Target/ARM/ARMISelLowering.cpp, there might actually 
 be a working implementation for this in LLVM already (which I 
 didn't realize before), so this route would not necessarily be 
 more complex than going with a different scheme. You'd probably 
 just need to provide the __tls_get_addr implementation in 
 druntime and figure out how LLVM emits the TLS image resp. how 
 to get its base address.
I think this is the best route, with the advantage that if my ___tls_get_addr uses pthread_(get|set)specific, it will likely just work on ARM too. I thought I'd have to get ldc to generate slightly different IR to do this, but it'd be great if llvm already does this. I had briefly looked at X86ISelLowering.cpp but not the ARM one, I'll see what it does.
 Hope this helps,
 David
Yeah, I think we're on the same page, thanks for the explanation. I've just been learning about TLS recently, so I wasn't sure before.
Mar 09 2014
parent reply "Joakim" <joakim airpost.net> writes:
On Sunday, 9 March 2014 at 18:23:00 UTC, Joakim wrote:
 On Sunday, 9 March 2014 at 16:12:19 UTC, David Nadlinger wrote:
 So, if %gs works as expected on Android and the linker 
 supports the necessary relocations, then it might be an option 
 to simply use the existing TLS implementation in LLVM and 
 simply provide the missing bits in druntime. On the other 
 hand, if you choose to go with an entirely different TLS 
 scheme (such as the DMD OS X implementation), you need to 
 figure out how to change the codegen to emit the extra 
 function calls to your __tls_get_addr analog, etc. Looking at 
 llvm/lib/Target/ARM/ARMISelLowering.cpp, there might actually 
 be a working implementation for this in LLVM already (which I 
 didn't realize before), so this route would not necessarily be 
 more complex than going with a different scheme. You'd 
 probably just need to provide the __tls_get_addr 
 implementation in druntime and figure out how LLVM emits the 
 TLS image resp. how to get its base address.
I think this is the best route, with the advantage that if my ___tls_get_addr uses pthread_(get|set)specific, it will likely just work on ARM too. I thought I'd have to get ldc to generate slightly different IR to do this, but it'd be great if llvm already does this. I had briefly looked at X86ISelLowering.cpp but not the ARM one, I'll see what it does.
Alright, I looked into the ARM and X86 assembly lowering source and it appears that those __tls_get_addr calls are simply the ones put in for the dynamic thread models. I tried hijacking those ___tls_get_addr calls by compiling all code as PIC, which forces a dynamic thread model in llvm that puts in the __tls_get_addr function calls, and then building as a shared library, which causes the gold linker to disable any linker optimizations that remove those calls. However, the resulting shared library would not run because there are still a few TLS relocations from the GOT for the dynamic linker to execute and the Android dynamic linker doesn't do those TLS relocations. So that was a deadend, looks like it's back to the packed TLS approach and having ldc generate IR that calls my __tls_get_addr manually.
Mar 17 2014
parent reply "Joakim" <joakim airpost.net> writes:
On Monday, 17 March 2014 at 10:25:22 UTC, Joakim wrote:
 So that was a deadend, looks like it's back to the packed TLS 
 approach and having ldc generate IR that calls my 
 __tls_get_addr manually.
Since packed TLS looks like the way this needs to be done, any chance one of the ldc developers might be able to toss this off? This is the first time I've ever tinkered with a compiler, so it will very likely take me longer than it would take one of you. Right now, I'm looking at hacking dmd to do this, as that seems like the fastest route to get something working, but obviously ldc will need it too for Android/ARM and the dmd patch is not going to be reusable for ldc. If not, not a big deal, I'm sure I'll get something working eventually.
Mar 20 2014
parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
Any TLS progress out there in LDC-land?

To pass thread/fiber unittests on iOS, I put in temporary workaround
using pthread_get/setspecific directly for the two threadlocals
(Thread.sm_this and Fiber.sm_this).  Now I can pass 74 of 85
druntime/phobos unittests on iOS.

If nobody is working on the emulated TLS for LDC, I will give it a try.
Nothing to lose.
-- 
Dan
Mar 27 2014
next sibling parent reply "Joakim" <joakim airpost.net> writes:
On Thursday, 27 March 2014 at 16:01:31 UTC, Dan Olson wrote:
 Any TLS progress out there in LDC-land?
I've been familiarizing myself with the relevant dmd backend source, but haven't tried anything yet.
 To pass thread/fiber unittests on iOS, I put in temporary 
 workaround
 using pthread_get/setspecific directly for the two threadlocals
 (Thread.sm_this and Fiber.sm_this).  Now I can pass 74 of 85
 druntime/phobos unittests on iOS.
I thought about doing the same, but didn't bother since I was able to get all of druntime's unit tests to pass by using Android's limited and flaky TLS support, left over from the linux kernel.
 If nobody is working on the emulated TLS for LDC, I will give 
 it a try.
 Nothing to lose.
Whatever I do to implement packed TLS in the dmd backend is not going to work for ldc anyway, so nothing stopping you from making your own effort. You will have to patch llvm also, if the weak symbols bug David pointed out is still around in llvm 3.5. Let us know what approach you take.
Mar 27 2014
parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
"Joakim" <joakim airpost.net> writes:

 On Thursday, 27 March 2014 at 16:01:31 UTC, Dan Olson wrote:

 If nobody is working on the emulated TLS for LDC, I will give it a
 try.
 Nothing to lose.
Whatever I do to implement packed TLS in the dmd backend is not going to work for ldc anyway, so nothing stopping you from making your own effort. You will have to patch llvm also, if the weak symbols bug David pointed out is still around in llvm 3.5. Let us know what approach you take.
The approach I started with was to make LLVM do the work. I read through all of the comments in this thread and decided this might be the most fun. ARMISelLowering.cpp has TLS disabled for all but ELF targets. I commented out an assertion blocking other targets to see what would happen for iOS (Mach-O). To my suprise, found that Mach-O tls sections are generated (__thread_vars, __thread_data, .tbss) and populated with the D thread local vars. The load/store instructions were treating TLS vars like global data though. So I looked at the Mach-O X86 version and saw what it is trying to do. LLVM coding is still a mystery to me, but managed after many hours today to hack together something that would turn this D code module tlsd; int a; void test() { a += 4; // access a } into this: movw r0, :lower16:(__D4tlsd1ai-(LPC4_0+4)) movt r0, :upper16:(__D4tlsd1ai-(LPC4_0+4)) LPC4_0: add r0, pc blx ___tls_get_addr ldr r1, [r0] adds r1, #4 str r1, [r0] ... .tbss __D4tlsd1ai$tlv$init, 4, 2 .section __DATA,__thread_vars,thread_local_variables .globl __D4tlsd1ai __D4tlsd1ai: .long __tlv_bootstrap .long 0 .long __D4tlsd1ai$tlv$init The following link helped explain what is going on with the __thread_vars data layout. http://www.opensource.apple.com/source/dyld/dyld-210.2.3/src/threadLocalVariables.c Mach-O dyln replaces tlv_bootstrap (thunk) with tlv_get_addr in the TLVDescriptor (__thread_vars). My LLVM hack for now is just doing a direct call to __tls_get_addr instead of indirect to tlv_get_addr. For proof of concept (one thread only), I have __tls_get_addr hard wired as follows: extern (C) { struct TLVDescriptor { void* function(TLVDescriptor*) thunk; uint key; uint offset; } //void* tlv_get_addr(TLVDescriptor* d) //void* __tls_get_addr(void* ptr) void* __tls_get_addr(TLVDescriptor* tlvd) { __gshared static ubyte data[512]; printf("__tls_get_addr %p \n", tlvd); printf("thunk %p, key %u, offset %u\n", tlvd.thunk, tlvd.key, tlvd.offset); return data.ptr + tlvd.offset; } void _tlv_bootstrap() { assert(false, "Should not get here"); } } It looks promising. Next step is to add in some realistic runtime support. Not sure if I will base it on dmd's sections-osx or the Apple dyld. Probably a hybrid. Eventually will need some help getting the LLVM changes clean instead of my hack job. Now that I've gone down this path a bit, I am beginning to wonder if changing LLVM to support iOS thread locals will have issues. Would LLVM want changes that affect Darwin/Mach-O (Apple's turf)? I suppose they could be optional. -- Dan
Mar 30 2014
next sibling parent reply "Joakim" <joakim airpost.net> writes:
On Sunday, 30 March 2014 at 08:22:15 UTC, Dan Olson wrote:
 "Joakim" <joakim airpost.net> writes:

 On Thursday, 27 March 2014 at 16:01:31 UTC, Dan Olson wrote:

 If nobody is working on the emulated TLS for LDC, I will give 
 it a
 try.
 Nothing to lose.
Whatever I do to implement packed TLS in the dmd backend is not going to work for ldc anyway, so nothing stopping you from making your own effort. You will have to patch llvm also, if the weak symbols bug David pointed out is still around in llvm 3.5. Let us know what approach you take.
The approach I started with was to make LLVM do the work. I read through all of the comments in this thread and decided this might be the most fun. ARMISelLowering.cpp has TLS disabled for all but ELF targets. I commented out an assertion blocking other targets to see what would happen for iOS (Mach-O). To my suprise, found that Mach-O tls sections are generated (__thread_vars, __thread_data, .tbss) and populated with the D thread local vars.
Nice find, I guess it helps that they have a desktop OS that does it differently.
 The load/store instructions were treating TLS vars like global 
 data
 though.  So I looked at the Mach-O X86 version and saw what it 
 is trying
 to do.  LLVM coding is still a mystery to me, but managed after 
 many
 hours today to hack together something that would turn this D 
 code

 module tlsd;
 int a;

 void test()
 {
   a += 4;   // access a
 }

 into this:

 	movw	r0, :lower16:(__D4tlsd1ai-(LPC4_0+4))
 	movt	r0, :upper16:(__D4tlsd1ai-(LPC4_0+4))
 LPC4_0:
 	add	r0, pc
 	blx	___tls_get_addr
 	ldr	r1, [r0]
 	adds	r1, #4
 	str	r1, [r0]

 ...


 .tbss __D4tlsd1ai$tlv$init, 4, 2

 	.section	__DATA,__thread_vars,thread_local_variables
 	.globl	__D4tlsd1ai
 __D4tlsd1ai:
 	.long	__tlv_bootstrap
 	.long	0
 	.long	__D4tlsd1ai$tlv$init


 The following link helped explain what is going on with the
 __thread_vars data layout.

 http://www.opensource.apple.com/source/dyld/dyld-210.2.3/src/threadLocalVariables.c

 Mach-O dyln replaces tlv_bootstrap (thunk) with tlv_get_addr in 
 the
 TLVDescriptor (__thread_vars).  My LLVM hack for now is just 
 doing a
 direct call to __tls_get_addr instead of indirect to 
 tlv_get_addr.  For
 proof of concept (one thread only), I have __tls_get_addr hard 
 wired as
 follows:

 extern (C)
 {
     struct TLVDescriptor
     {
 	void*  function(TLVDescriptor*) thunk;
 	uint	key;
 	uint	offset;
     }

     //void* tlv_get_addr(TLVDescriptor* d)
     //void* __tls_get_addr(void* ptr)
     void* __tls_get_addr(TLVDescriptor* tlvd)
     {
         __gshared static ubyte data[512];

         printf("__tls_get_addr %p \n", tlvd);
         printf("thunk %p, key %u, offset %u\n",
                tlvd.thunk, tlvd.key, tlvd.offset);
         return data.ptr + tlvd.offset;
     }

     void _tlv_bootstrap()
     {
         assert(false, "Should not get here");
     }
 }

 It looks promising.  Next step is to add in some realistic 
 runtime
 support.  Not sure if I will base it on dmd's sections-osx or 
 the Apple
 dyld.  Probably a hybrid.
Have you experimented with seeing which of that TLV stuff from OS X that iOS actually supports? The iOS dyld could be pretty different. We don't know since they don't release the source for the iOS core like they do for OS X, ie is tlv_get_addr even available in the iOS dyld and does it execute other possible TLS relocations? Only way to find out is to try it, or somehow inspect their iOS binaries. ;) Their source does show an ARM assembly implementation of tlv_get_address but it's commented out: http://www.opensource.apple.com/source/dyld/dyld-210.2.3/src/threadLocalHelpers.s I wonder if it'd be easier to pack your own Mach-O sections rather than figuring out how to access all their sections and reimplementing their TLV functions, assuming they're not available. You might even be able to do it as an llvm patch since the relevant lib/MC/ files where llvm packs the TLS data into Mach-O sections seem pretty straightforward.
 Eventually will need some help getting the LLVM changes clean 
 instead of
 my hack job.

 Now that I've gone down this path a bit, I am beginning to 
 wonder if
 changing LLVM to support iOS thread locals will have issues.  
 Would LLVM
 want changes that affect Darwin/Mach-O (Apple's turf)?  I 
 suppose they
 could be optional.
I've never submitted anything to llvm, so not really based on anything than speculation, but I doubt they would accept such a patch, doesn't mean we can't use it though. ;)
Mar 30 2014
next sibling parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
"Joakim" <joakim airpost.net> writes:

 Have you experimented with seeing which of that TLV stuff from OS X
 that iOS actually supports?  The iOS dyld could be pretty different.
 We don't know since they don't release the source for the iOS core
 like they do for OS X, ie is tlv_get_addr even available in the iOS
 dyld and does it execute other possible TLS relocations?  Only way to
 find out is to try it, or somehow inspect their iOS binaries. ;) Their
 source does show an ARM assembly implementation of tlv_get_address but
 it's commented out:
 http://www.opensource.apple.com/source/dyld/dyld-210.2.3/src/threadLocalHelpers.s
I did try it in an iOS app. The function _tlv_bootstrap is unresolved when I link in Xcode using the current iPhoneSDK. That is why I had to provide a stub. Pretty sure tlv functions are not available.
 I wonder if it'd be easier to pack your own Mach-O sections rather
 than figuring out how to access all their sections and reimplementing
 their TLV functions, assuming they're not available.  You might even
 be able to do it as an llvm patch since the relevant lib/MC/ files
 where llvm packs the TLS data into Mach-O sections seem pretty
 straightforward.
I think we can use their sections and it did not take long to figure out. Here is what an example link map has for one of my test apps: 0x0004E22C 0x00000084 __DATA __thread_vars 0x0004E2B0 0x0000000C __DATA __thread_data 0x0004E2BC 0x00000024 __DATA __thread_bss The _thread_vars section has a TVLDescriptors for each thread local. It is used for caching the pthread_get/set key and has the variable offset into the thread local chunk of memory that can be initialized by copying _thread_data and _thread_bss (or just zerofill it).
 I've never submitted anything to llvm, so not really based on anything
 than speculation, but I doubt they would accept such a patch, doesn't
 mean we can't use it though. ;)
Another thing, Apple might consider the tlv functions and thread local sections a reserved API. A long way off from submitting anything to App Store. With the way things change, tlv may show up in a near future sdk, then this just becomes a bridge. -- Dan
Mar 30 2014
parent "Joakim" <joakim airpost.net> writes:
On Sunday, 30 March 2014 at 15:24:53 UTC, Dan Olson wrote:
 "Joakim" <joakim airpost.net> writes:
 I think we can use their sections and it did not take long to 
 figure
 out.  Here is what an example link map has for one of my test 
 apps:

 0x0004E22C	0x00000084	__DATA	__thread_vars
 0x0004E2B0	0x0000000C	__DATA	__thread_data
 0x0004E2BC	0x00000024	__DATA	__thread_bss

 The _thread_vars section has a TVLDescriptors for each thread 
 local.  It
 is used for caching the pthread_get/set key and has the 
 variable offset
 into the thread local chunk of memory that can be initialized 
 by copying
 _thread_data and _thread_bss (or just zerofill it).
---snip---
 A long way off from submitting anything to App Store.  With the 
 way
 things change, tlv may show up in a near future sdk, then this 
 just
 becomes a bridge.
Hmm, you and Jacob are probably right, it may be better to just follow what they do. On Sunday, 30 March 2014 at 15:34:08 UTC, Dan Olson wrote:
 Jacob Carlborg <doob me.com> writes:

 I would follow the native TLS implementation in OS X, i.e. 
 using
 "tlv_get_addr", as close as possible. In theory it should be 
 possible
 to move the code from threadLocalVariables.c and 
 threadLocalHelpers.s
 directly in to druntime.

 Hopefully that would mean the same code for generating TLS 
 access
 could be used both on OS X and iOS.
Do think we can just drop the dyld code into druntime? It should work with perhaps some modifications, but I am not familiar with the Apple opensource license. I should read it. It is BSD-like right?
I think the APSL is more similar to the CDDL, which was Sun's license for OpenSolaris and much of their open-source contributions, and requires that source is provided for APS-licensed files. I think you could always add an APS-licensed file to druntime and the licenses would not clash, but that would make druntime not completely boost-licensed anymore, as the APSL has additional requirements than the minimal boost license. It's probably best to just reimplement the necessary functions yourself.
 Would still
 need to hook in the garbage collector so it scans the thread 
 local
 memory.  I'll have to try it tonight.
David did this for the TLV code on OS X a year back, should be pretty straightforward to do something similar to what he did. On Sunday, 30 March 2014 at 15:44:52 UTC, Dan Olson wrote:
 "Joakim" <joakim airpost.net> writes:

 I wonder if it'd be easier to pack your own Mach-O sections 
 rather
 than figuring out how to access all their sections and 
 reimplementing
 their TLV functions, assuming they're not available.  You 
 might even
 be able to do it as an llvm patch since the relevant lib/MC/ 
 files
 where llvm packs the TLS data into Mach-O sections seem pretty
 straightforward.
Thinking about this some more. It probably makes sense to have an optional approach that can be used on any target that does not have native TLS. This current approach for iOS will only work for Mach-O. I wonder if the LLVM folks are working toward a generic TLS without OS support.
Doesn't look like it, plus it'll need to be specialized for each object format, like Mach, ELF, or COFF, anyway. After looking at the relevant llvm source for packing sections to see how it was working for you with Mach, I wonder if I won't be able to patch some of the existing llvm files for packing TLS data into ELF and get the TLS variables packed easily that way. I'll try that approach at some point.
Mar 30 2014
prev sibling parent Dan Olson <zans.is.for.cans yahoo.com> writes:
"Joakim" <joakim airpost.net> writes:

 I wonder if it'd be easier to pack your own Mach-O sections rather
 than figuring out how to access all their sections and reimplementing
 their TLV functions, assuming they're not available.  You might even
 be able to do it as an llvm patch since the relevant lib/MC/ files
 where llvm packs the TLS data into Mach-O sections seem pretty
 straightforward.
Thinking about this some more. It probably makes sense to have an optional approach that can be used on any target that does not have native TLS. This current approach for iOS will only work for Mach-O. I wonder if the LLVM folks are working toward a generic TLS without OS support. -- Dan
Mar 30 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-03-30 10:22, Dan Olson wrote:
 "Joakim" <joakim airpost.net> writes:

 On Thursday, 27 March 2014 at 16:01:31 UTC, Dan Olson wrote:

 If nobody is working on the emulated TLS for LDC, I will give it a
 try.
 Nothing to lose.
Whatever I do to implement packed TLS in the dmd backend is not going to work for ldc anyway, so nothing stopping you from making your own effort. You will have to patch llvm also, if the weak symbols bug David pointed out is still around in llvm 3.5. Let us know what approach you take.
The approach I started with was to make LLVM do the work. I read through all of the comments in this thread and decided this might be the most fun. ARMISelLowering.cpp has TLS disabled for all but ELF targets. I commented out an assertion blocking other targets to see what would happen for iOS (Mach-O). To my suprise, found that Mach-O tls sections are generated (__thread_vars, __thread_data, .tbss) and populated with the D thread local vars. The load/store instructions were treating TLS vars like global data though. So I looked at the Mach-O X86 version and saw what it is trying to do. LLVM coding is still a mystery to me, but managed after many hours today to hack together something that would turn this D code module tlsd; int a; void test() { a += 4; // access a } into this: movw r0, :lower16:(__D4tlsd1ai-(LPC4_0+4)) movt r0, :upper16:(__D4tlsd1ai-(LPC4_0+4)) LPC4_0: add r0, pc blx ___tls_get_addr ldr r1, [r0] adds r1, #4 str r1, [r0] ... .tbss __D4tlsd1ai$tlv$init, 4, 2 .section __DATA,__thread_vars,thread_local_variables .globl __D4tlsd1ai __D4tlsd1ai: .long __tlv_bootstrap .long 0 .long __D4tlsd1ai$tlv$init The following link helped explain what is going on with the __thread_vars data layout. http://www.opensource.apple.com/source/dyld/dyld-210.2.3/src/threadLocalVariables.c Mach-O dyln replaces tlv_bootstrap (thunk) with tlv_get_addr in the TLVDescriptor (__thread_vars). My LLVM hack for now is just doing a direct call to __tls_get_addr instead of indirect to tlv_get_addr. For proof of concept (one thread only), I have __tls_get_addr hard wired as follows: extern (C) { struct TLVDescriptor { void* function(TLVDescriptor*) thunk; uint key; uint offset; } //void* tlv_get_addr(TLVDescriptor* d) //void* __tls_get_addr(void* ptr) void* __tls_get_addr(TLVDescriptor* tlvd) { __gshared static ubyte data[512]; printf("__tls_get_addr %p \n", tlvd); printf("thunk %p, key %u, offset %u\n", tlvd.thunk, tlvd.key, tlvd.offset); return data.ptr + tlvd.offset; } void _tlv_bootstrap() { assert(false, "Should not get here"); } } It looks promising. Next step is to add in some realistic runtime support. Not sure if I will base it on dmd's sections-osx or the Apple dyld. Probably a hybrid.
I would follow the native TLS implementation in OS X, i.e. using "tlv_get_addr", as close as possible. In theory it should be possible to move the code from threadLocalVariables.c and threadLocalHelpers.s directly in to druntime. Hopefully that would mean the same code for generating TLS access could be used both on OS X and iOS. -- /Jacob Carlborg
Mar 30 2014
parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
Jacob Carlborg <doob me.com> writes:

 I would follow the native TLS implementation in OS X, i.e. using
 "tlv_get_addr", as close as possible. In theory it should be possible
 to move the code from threadLocalVariables.c and threadLocalHelpers.s
 directly in to druntime.

 Hopefully that would mean the same code for generating TLS access
 could be used both on OS X and iOS.
Do think we can just drop the dyld code into druntime? It should work with perhaps some modifications, but I am not familiar with the Apple opensource license. I should read it. It is BSD-like right? Would still need to hook in the garbage collector so it scans the thread local memory. I'll have to try it tonight. -- Dan
Mar 30 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 30/03/14 17:34, Dan Olson wrote:

 Do think we can just drop the dyld code into druntime?
Yes, with minor modifications. The TLS related code in dyld is pretty much self contained. I don't see dyld using any functionality that isn't available to a regular application.
 It should work with perhaps some modifications, but I am not familiar with the
Apple
 opensource license. I should read it. It is BSD-like right?
The license is a completely different issue. The safest would be to re-implement the code. One can document the existing code and some other can do the implementation. Regardless of the license, you can still give a try to see if the technical parts work.
 Would still need to hook in the garbage collector so it scans the thread local
 memory.  I'll have to try it tonight.
You'll just need to add a call to druntime in one of the functions in the dyld TLS code. Have a look at: https://github.com/D-Programming-Language/druntime/blob/master/src/rt/sections_osx.d -- /Jacob Carlborg
Mar 30 2014
next sibling parent reply "David Nadlinger" <code klickverbot.at> writes:
On 31 Mar 2014, at 8:25, Jacob Carlborg wrote:
 You'll just need to add a call to druntime in one of the functions in 
 the dyld TLS code. Have a look at:

 https://github.com/D-Programming-Language/druntime/blob/master/src/rt/sections_osx.d
More specifically, for the DMD TLS emulation implementation, this is done in the initTLSRanges() function, which forwards to getTLSBlock(). IIRC, initTLSRanges() is only called for new threads. For the main thread, the TLS ranges is included in the GC ranges detected in initSections(). For LDC on OS X, which makes use of the 10.7+ system-level TLS implementation, the place where this is handled is https://github.com/ldc-developers/druntime/blob/a08f158618eb5d06c42bd4746b782312e937f6b3/src/rt/ ections_ldc.d#L296. _d_dyld_getTLSRange uses an undocumented dyld API function (dyld_enumerate_tlv_storage) to get the actual TLS memory range on the current thread: https://github.com/ldc-developers/druntime/blob/a08f158618eb5d06c42bd4746b782312e937f6b3/src/ldc/osx_tls.c. David
Mar 31 2014
next sibling parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
"David Nadlinger" <code klickverbot.at> writes:

 On 31 Mar 2014, at 8:25, Jacob Carlborg wrote:
 You'll just need to add a call to druntime in one of the functions
 in the dyld TLS code. Have a look at:

 https://github.com/D-Programming-Language/druntime/blob/master/src/rt/sections_osx.d
More specifically, for the DMD TLS emulation implementation, this is done in the initTLSRanges() function, which forwards to getTLSBlock(). IIRC, initTLSRanges() is only called for new threads. For the main thread, the TLS ranges is included in the GC ranges detected in initSections(). For LDC on OS X, which makes use of the 10.7+ system-level TLS implementation, the place where this is handled is https://github.com/ldc-developers/druntime/blob/a08f158618eb5d06c42bd4746b782312e937f6b3/src/rt/ ections_ldc.d#L296. _d_dyld_getTLSRange uses an undocumented dyld API function (dyld_enumerate_tlv_storage) to get the actual TLS memory range on the current thread: https://github.com/ldc-developers/druntime/blob/a08f158618eb5d06c42bd4746b782312e937f6b3/src/ldc/osx_tls.c. David
I had disabled initTLSRanges for iOS since dyld_enumerate_tlv_storage is a stub for x86 (see http://www.opensource.apple.com/source/dyld/dyld-210.2.3/src/threadLocalVariables.c). Now that I have tweaked threadLocalVariables.c, dyld_enumerate_tlv_storage should now work on iOS. I will have to reenble initTLSRanges and see what happens. -- Dan
Mar 31 2014
parent Dan Olson <zans.is.for.cans yahoo.com> writes:
Dan Olson <zans.is.for.cans yahoo.com> writes:

 "David Nadlinger" <code klickverbot.at> writes:

 On 31 Mar 2014, at 8:25, Jacob Carlborg wrote:
 You'll just need to add a call to druntime in one of the functions
 in the dyld TLS code. Have a look at:

 https://github.com/D-Programming-Language/druntime/blob/master/src/rt/sections_osx.d
More specifically, for the DMD TLS emulation implementation, this is done in the initTLSRanges() function, which forwards to getTLSBlock(). IIRC, initTLSRanges() is only called for new threads. For the main thread, the TLS ranges is included in the GC ranges detected in initSections(). For LDC on OS X, which makes use of the 10.7+ system-level TLS implementation, the place where this is handled is https://github.com/ldc-developers/druntime/blob/a08f158618eb5d06c42bd4746b782312e937f6b3/src/rt/ ections_ldc.d#L296. _d_dyld_getTLSRange uses an undocumented dyld API function (dyld_enumerate_tlv_storage) to get the actual TLS memory range on the current thread: https://github.com/ldc-developers/druntime/blob/a08f158618eb5d06c42bd4746b782312e937f6b3/src/ldc/osx_tls.c. David
I had disabled initTLSRanges for iOS since dyld_enumerate_tlv_storage is a stub for x86 (see http://www.opensource.apple.com/source/dyld/dyld-210.2.3/src/threadLocalVariables.c).
I meant it is a stub for ARM.
 Now that I have tweaked threadLocalVariables.c,
 dyld_enumerate_tlv_storage should now work on iOS. I will have to
 reenble initTLSRanges and see what happens.
I did reenable and it works. I can tell because the std.datetime unittest uses enough memory that it causes a GC. When I first rebuild everything with TLS enabled and plugged in support from threadLocalVariables.c (but without initTLSRanges enabled), the std.datetime unittest started crashing. The datetime unittest tests have a fair number of thread locals. Then I reenabled David's initTLSRanges() for iOS, and std.datetime unittest went back to passing. -- Dan
Apr 01 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-03-31 16:05, David Nadlinger wrote:

 For LDC on OS X, which makes use of the 10.7+ system-level TLS
 implementation, the place where this is handled is
 https://github.com/ldc-developers/druntime/blob/a08f158618eb5d06c42bd4746b782312e937f6b3/src/rt/sections_ldc.d#L296.
 _d_dyld_getTLSRange uses an undocumented dyld API function
 (dyld_enumerate_tlv_storage) to get the actual TLS  memory range on the
 current thread:
 https://github.com/ldc-developers/druntime/blob/a08f158618eb5d06c42bd4746b782312e937f6b3/src/ldc/osx_tls.c.
"dyld_enumerate_tlv_storage" should probably be replaced with a function that is publicly available, at some point. -- /Jacob Carlborg
Mar 31 2014
parent reply "David Nadlinger" <code klickverbot.at> writes:
On 31 Mar 2014, at 19:39, Jacob Carlborg wrote:
 "dyld_enumerate_tlv_storage" should probably be replaced with a 
 function that is publicly available, at some point.
If you find such a function, please let me know (or, better, submit a pull request). Maybe it is possible to reimplement dyld_enumerate_tlv_storage using public APIs, but back then I didn't spend too much time on investigating that. David
Mar 31 2014
parent Jacob Carlborg <doob me.com> writes:
On 2014-03-31 19:42, David Nadlinger wrote:

 If you find such a function, please let me know (or, better, submit a
 pull request).
Hmm, it might be a bit more complicated than I first thought. I might have a look at it some time.
 Maybe it is possible to reimplement dyld_enumerate_tlv_storage using
 public APIs, but back then I didn't spend too much time on investigating
 that.
Fair enough. -- /Jacob Carlborg
Mar 31 2014
prev sibling parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
Jacob Carlborg <doob me.com> writes:

 Regardless of the license, you can still give a try to see if the
 technical parts work.
I have tried and success. I added threadLocalHelpers.s and threadLocalVariables.c, modified to enable for arm, then had to sprinkle in some missing types from dl_priv.h. Then put in a call to tlv_initializer(). The proof is that thread locals get proper initial values when accessed through tlv_get_addr(). For example, a thread local double is being initialized to nan. Still having LLVM emit a __tls_get_addr, so to try this out, I changed my test __tls_get_addr() implementation to forward to tlv_get_addr() in threadLocalHelpers.s. extern (C) void* __tls_get_addr(TLVDescriptor* tlvd) { __gshared static ubyte data[512]; printf("__tls_get_addr %p \n", tlvd); printf("thunk %p, key %u, offset %u\n", tlvd.thunk, tlvd.key, tlvd.offset); // tlv_initializer() will change thunk to tlv_get_addr if (tlvd.thunk is &tlv_get_addr) { puts("calling real tlv_get_addr instead"); return tlv_get_addr(tlvd); } // tlv not initialized yet, return my fake thread local data. return data.tlvd + tlvd.offset; }
Mar 31 2014
next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On 31 Mar 2014, at 17:23, Dan Olson wrote:
 I added threadLocalHelpers.s and threadLocalVariables.c, modified to
 enable for arm, then had to sprinkle in some missing types from
 dl_priv.h.  Then put in a call to tlv_initializer().

 The proof is that thread locals get proper initial values when accessed
 through tlv_get_addr().  For example, a thread local double is being
 initialized to nan.
Nice! David
Mar 31 2014
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2014-03-31 17:23, Dan Olson wrote:

 I have tried and success.

 I added threadLocalHelpers.s and threadLocalVariables.c, modified to
 enable for arm, then had to sprinkle in some missing types from
 dl_priv.h.  Then put in a call to tlv_initializer().

 The proof is that thread locals get proper initial values when accessed
 through tlv_get_addr().  For example, a thread local double is being
 initialized to nan.
Awesome :) -- /Jacob Carlborg
Mar 31 2014
prev sibling parent "David Nadlinger" <code klickverbot.at> writes:
On 27 Mar 2014, at 17:01, Dan Olson wrote:
 If nobody is working on the emulated TLS for LDC, I will give it a try.
 Nothing to lose.
Would be great – I don't think anybody else is working on this right now. David
Mar 27 2014
prev sibling parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
David Nadlinger <code klickverbot.at> writes:

 On 03/08/2014 01:55 AM, Joakim wrote:
 Do you have any advice on how to pull this off with ldc?  Should I be
 going the dmd route and packing the TLS myself?  Does llvm provide good
 support for this?

 Or is there some other llvm TLS shortcut I can use?  I tried to see if
 llvm just has some thread-local implementation that automatically uses
 pthread_setspecific, but didn't find anything.
LLVM does support putting variables into custom sections, and you can more or less get away with the DMD bracketing approach (see e.g. the new ModuleInfo discovery functionality I implemented for Linux, which is the same as DMD's druntime uses). However, there is a catch: Due to what I can only imagine is a bug, LLVM does not support emitting a symbol both into a custom section and with weak linkage. Thus, you might be in for a round of LLVM hacking either way, even though it will likely involve much less when going the DMD route. However, there is a third options which might be worth investigating, namely re-implementing at least parts of the necessary runtime linker features in druntime and continuing to use the same scheme as on GNU Linux/x86. This depends on %gs not being used in another way, etc. though. David
While on the subject of TLS, that is probably the most needed language feature to allow threading to work reliably on iOS. So hoping the solution will work on iOS too! Another topic - I was looking at adding fiber_switchContext support for arm in threadasm.S, and noticed GDC's version has an arm implementation. Is it ok to use portions of GDC source in LDC? -- Dan
Mar 08 2014
next sibling parent reply David Nadlinger <code klickverbot.at> writes:
On Sat, Mar 8, 2014 at 8:11 PM, Dan Olson <zans.is.for.cans yahoo.com> wrote:
 Another topic - I was looking at adding fiber_switchContext support for
 arm in threadasm.S, and noticed GDC's version has an arm
 implementation.  Is it ok to use portions of GDC source in LDC?
If their code is Boost-licensed (general druntime/Phobos license), yes. David
Mar 08 2014
parent Dan Olson <zans.is.for.cans yahoo.com> writes:
David Nadlinger <code klickverbot.at> writes:

 On Sat, Mar 8, 2014 at 8:11 PM, Dan Olson <zans.is.for.cans yahoo.com> wrote:
 Another topic - I was looking at adding fiber_switchContext support for
 arm in threadasm.S, and noticed GDC's version has an arm
 implementation.  Is it ok to use portions of GDC source in LDC?
If their code is Boost-licensed (general druntime/Phobos license), yes. David
Yes, that file is Boost - good!
Mar 08 2014
prev sibling parent reply "Joakim" <joakim airpost.net> writes:
On Saturday, 8 March 2014 at 19:11:52 UTC, Dan Olson wrote:
 While on the subject of TLS, that is probably the most needed 
 language
 feature to allow threading to work reliably on iOS.  So hoping 
 the
 solution will work on iOS too!
I wondered earlier why you weren't just using Walter's packed TLS approach and now I see why, ldc doesn't use it. Looks like Apple hasn't ported the TLV functions which ldc uses to iOS yet either, so you're out of luck there too. I guess you'll have to port Walter's approach to ldc to get TLS working on iOS: https://github.com/D-Programming-Language/dmd/blob/master/src/backend/machobj.c#L1673 Either that or get llvm to emit the right pthread calls, like I mentioned earlier.
Mar 08 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 2014-03-09 07:04, Joakim wrote:

 I wondered earlier why you weren't just using Walter's packed TLS
 approach and now I see why, ldc doesn't use it.  Looks like Apple hasn't
 ported the TLV functions which ldc uses to iOS yet either, so you're out
 of luck there too.  I guess you'll have to port Walter's approach to ldc
 to get TLS working on iOS:
I think it would be possible to implement the missing TLV functions our self in druntime. Hopefully this would allow to use the same TLS approach both on OS X and on iOS. -- /Jacob Carlborg
Mar 09 2014
parent reply "Joakim" <joakim airpost.net> writes:
On Sunday, 9 March 2014 at 09:55:33 UTC, Jacob Carlborg wrote:
 On 2014-03-09 07:04, Joakim wrote:

 I wondered earlier why you weren't just using Walter's packed 
 TLS
 approach and now I see why, ldc doesn't use it.  Looks like 
 Apple hasn't
 ported the TLV functions which ldc uses to iOS yet either, so 
 you're out
 of luck there too.  I guess you'll have to port Walter's 
 approach to ldc
 to get TLS working on iOS:
I think it would be possible to implement the missing TLV functions our self in druntime. Hopefully this would allow to use the same TLS approach both on OS X and on iOS.
OK, I assumed OS support was necessary, maybe not. On Saturday, 8 March 2014 at 18:16:58 UTC, Joakim wrote:
 On Saturday, 8 March 2014 at 14:25:43 UTC, David Nadlinger 
 wrote:
 However, there is a third options which might be worth 
 investigating, namely re-implementing at least parts of the 
 necessary runtime linker features in druntime and continuing 
 to use the same scheme as on GNU Linux/x86. This depends on 
 %gs not being used in another way, etc. though.
I tried to reuse the existing dl_iterate_phdr approach on Android, but then I noticed that the dl_phdr_info struct defined in bionic doesn't include the dlpi_tls_modid and dlpi_tls_data members. However, now that you mention it, maybe those aren't strictly necessary, as long as I'm not worried about shared libraries. I'll look into it further.
Speaking of OS support, I just tried this and I was able to access the TLS initialization image using dl_phdr_info on Android/x86. Those dlpi_tls_* members are not necessary, though I'm guessing dlpi_tls_modid would be for shared library support. Now I just have to figure out some way to have the TLS relocations access the initialization image, presumably the way Walter does it for dmd/OSX.
Mar 09 2014
parent Jacob Carlborg <doob me.com> writes:
On 2014-03-09 11:11, Joakim wrote:

 OK, I assumed OS support was necessary, maybe not.
Well, yes. In this case the OS support comes in the form of the dynamic linker. We can do the same as the dynamic linker does in druntime. I don't know if it helps but the dynamic linker on OS X has code for tlv_get_addr for ARM, but it's disabled. -- /Jacob Carlborg
Mar 09 2014