www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - OS X 10.7 (Lion) breaks DMD-compiled executables due to ASLR (?)

reply David Nadlinger <see klickverbot.at> writes:
I have been observing crashes in semi-random places around my D 
applications (Segmentation fault/Bus errors) on the recently released OS 
X 10.7, dubbed Lion. They miraculously disappeared every time I ran the 
executables from GDB, and as I found out (thanks wm4 for the hint), this 
was due to GDB disabling ASLR by default. And indeed, if I »set 
disable-aslr off«, the crashes also happened within the debugger.

This is not totally surprising as Lion is the first OS X release to 
include full ASLR for both 32 bit and 64 bit applications, but I have 
not been able to track down what exactly goes on.

Any guesses what could go wrong here without additional details (I'm not 
at all sure where to look right now)? Maybe something related to the OS 
X-specific things like TLS handling?

David
Jul 23 2011
next sibling parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 24/07/2011 06:24, David Nadlinger wrote:

 I have been observing crashes in semi-random places around my D
 applications (Segmentation fault/Bus errors) on the recently released OS
 X 10.7, dubbed Lion. They miraculously disappeared every time I ran the
 executables from GDB, and as I found out (thanks wm4 for the hint), this
 was due to GDB disabling ASLR by default. And indeed, if I »set
 disable-aslr off«, the crashes also happened within the debugger.

 This is not totally surprising as Lion is the first OS X release to
 include full ASLR for both 32 bit and 64 bit applications, but I have
 not been able to track down what exactly goes on.

 Any guesses what could go wrong here without additional details (I'm not
 at all sure where to look right now)? Maybe something related to the OS
 X-specific things like TLS handling?

 David


I've not run into any issues with this yet, if I do I'll see if I can 
shed some more light on the situation. Off the top of my head, my guess 
would be that the runtime is doing something funky that depends on ASLR 
not being there, don't know what without investigating though.

-- 
Robert
http://octarineparrot.com/
Jul 24 2011
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 24.07.2011 14:21, schrieb Robert Clipsham:

 On 24/07/2011 06:24, David Nadlinger wrote:

 I have been observing crashes in semi-random places around my D
 applications (Segmentation fault/Bus errors) on the recently released OS
 X 10.7, dubbed Lion. They miraculously disappeared every time I ran the
 executables from GDB, and as I found out (thanks wm4 for the hint), this
 was due to GDB disabling ASLR by default. And indeed, if I »set
 disable-aslr off«, the crashes also happened within the debugger.

 This is not totally surprising as Lion is the first OS X release to
 include full ASLR for both 32 bit and 64 bit applications, but I have
 not been able to track down what exactly goes on.

 Any guesses what could go wrong here without additional details (I'm not
 at all sure where to look right now)? Maybe something related to the OS
 X-specific things like TLS handling?

 David


 I've not run into any issues with this yet, if I do I'll see if I can
 shed some more light on the situation. Off the top of my head, my guess
 would be that the runtime is doing something funky that depends on ASLR
 not being there, don't know what without investigating though.


Linux and Windows have been doing this for years (since Kernel 2.6.12 / 
Windows Vista). Are the relevant parts of the runtime so different on 
OSX? (And has this ever broken D executables/was DMD or the runtime 
changed to fix it?)

Cheers,
- Daniel
Jul 24 2011
parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 24/07/2011 13:55, Daniel Gibson wrote:

 Linux and Windows have been doing this for years (since Kernel 2.6.12 /
 Windows Vista). Are the relevant parts of the runtime so different on
 OSX? (And has this ever broken D executables/was DMD or the runtime
 changed to fix it?)


I was under the impression they'd gone above and beyond what 
Windows/Linux did in the way of ASLR? Perhaps not.

There are definitely parts of the runtime that could be affected by it 
that differ on OS X compared to linux/windows (TLS is the first thing 
that comes to mind), whether that would be affected at all by it though 
I have no idea.


 Cheers,
 - Daniel


-- 
Robert
http://octarineparrot.com/
Jul 24 2011
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 24.07.2011 15:56, schrieb Robert Clipsham:

 On 24/07/2011 13:55, Daniel Gibson wrote:

 Linux and Windows have been doing this for years (since Kernel 2.6.12 /
 Windows Vista). Are the relevant parts of the runtime so different on
 OSX? (And has this ever broken D executables/was DMD or the runtime
 changed to fix it?)


 I was under the impression they'd gone above and beyond what
 Windows/Linux did in the way of ASLR? Perhaps not.


Maybe, I don't know the details of each implementation.
Also, according to Wikipedia, OSX did ASLR before (which I didn't know), 
but with Lion it has "improved", so it's likely that it does stuff 
differently than Windows and Linux now.

Cheers,
- Daniel
Jul 24 2011
parent David Nadlinger <see klickverbot.at> writes:
On 7/24/11 4:42 PM, Daniel Gibson wrote:

 Am 24.07.2011 15:56, schrieb Robert Clipsham:

 I was under the impression they'd gone above and beyond what
 Windows/Linux did in the way of ASLR? Perhaps not.


 Maybe, I don't know the details of each implementation.
 Also, according to Wikipedia, OSX did ASLR before (which I didn't know),
 but with Lion it has "improved", so it's likely that it does stuff
 differently than Windows and Linux now.


Lion ironically is the first version to include ASLR also for 32-bit 
applications. As DMD is still 32-bit only on OS X, it simply never was a 
possible issue before.

David
Jul 24 2011
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-24 07:24, David Nadlinger wrote:

 I have been observing crashes in semi-random places around my D
 applications (Segmentation fault/Bus errors) on the recently released OS
 X 10.7, dubbed Lion. They miraculously disappeared every time I ran the
 executables from GDB, and as I found out (thanks wm4 for the hint), this
 was due to GDB disabling ASLR by default. And indeed, if I »set
 disable-aslr off«, the crashes also happened within the debugger.

 This is not totally surprising as Lion is the first OS X release to
 include full ASLR for both 32 bit and 64 bit applications, but I have
 not been able to track down what exactly goes on.

 Any guesses what could go wrong here without additional details (I'm not
 at all sure where to look right now)? Maybe something related to the OS
 X-specific things like TLS handling?

 David


Is this with a DMD compiled on Lion or a pre-built DMD (compiled on Snow 
Leopard) ?

-- 
/Jacob Carlborg
Jul 24 2011
parent David Nadlinger <see klickverbot.at> writes:
On 7/24/11 3:03 PM, Jacob Carlborg wrote:

 Is this with a DMD compiled on Lion or a pre-built DMD (compiled on Snow
 Leopard) ?


With a DMD built on Snow Leopard. Building DMD with a LLVM backend 
(LLVM-GCC or Clang) currently leads to a strange zero dereference crash 
in el_picvar(), and I haven't tried using gcc-4.2 with the GCC backend yet.

David
Jul 24 2011
prev sibling next sibling parent reply KennyTM~ <kennytm gmail.com> writes:
On Jul 24, 11 13:24, David Nadlinger wrote:

 I have been observing crashes in semi-random places around my D
 applications (Segmentation fault/Bus errors) on the recently released OS
 X 10.7, dubbed Lion. They miraculously disappeared every time I ran the
 executables from GDB, and as I found out (thanks wm4 for the hint), this
 was due to GDB disabling ASLR by default. And indeed, if I »set
 disable-aslr off«, the crashes also happened within the debugger.

 This is not totally surprising as Lion is the first OS X release to
 include full ASLR for both 32 bit and 64 bit applications, but I have
 not been able to track down what exactly goes on.

 Any guesses what could go wrong here without additional details (I'm not
 at all sure where to look right now)? Maybe something related to the OS
 X-specific things like TLS handling?

 David


Seems to be due to throwing exceptions, not TLS.

http://d.puremagic.com/issues/show_bug.cgi?id=6376
Jul 24 2011
parent reply David Nadlinger <see klickverbot.at> writes:
On 7/25/11 12:48 AM, KennyTM~ wrote:

 On Jul 24, 11 13:24, David Nadlinger wrote:

 I have been observing crashes in semi-random places around my D
 applications (Segmentation fault/Bus errors) on the recently released OS
 X 10.7, dubbed Lion. They miraculously disappeared every time I ran the
 executables from GDB, and as I found out (thanks wm4 for the hint), this
 was due to GDB disabling ASLR by default. And indeed, if I »set
 disable-aslr off«, the crashes also happened within the debugger.

 This is not totally surprising as Lion is the first OS X release to
 include full ASLR for both 32 bit and 64 bit applications, but I have
 not been able to track down what exactly goes on.

 Any guesses what could go wrong here without additional details (I'm not
 at all sure where to look right now)? Maybe something related to the OS
 X-specific things like TLS handling?

 David


 Seems to be due to throwing exceptions, not TLS.

 http://d.puremagic.com/issues/show_bug.cgi?id=6376


Yeah, can confirm that, all my issues seem to be somehow related to 
exceptions to (even though in non-obvious ways sometimes).

David
Jul 24 2011
parent KennyTM~ <kennytm gmail.com> writes:
On Jul 25, 11 07:26, David Nadlinger wrote:

 On 7/25/11 12:48 AM, KennyTM~ wrote:

 On Jul 24, 11 13:24, David Nadlinger wrote:

 I have been observing crashes in semi-random places around my D
 applications (Segmentation fault/Bus errors) on the recently released OS
 X 10.7, dubbed Lion. They miraculously disappeared every time I ran the
 executables from GDB, and as I found out (thanks wm4 for the hint), this
 was due to GDB disabling ASLR by default. And indeed, if I »set
 disable-aslr off«, the crashes also happened within the debugger.

 This is not totally surprising as Lion is the first OS X release to
 include full ASLR for both 32 bit and 64 bit applications, but I have
 not been able to track down what exactly goes on.

 Any guesses what could go wrong here without additional details (I'm not
 at all sure where to look right now)? Maybe something related to the OS
 X-specific things like TLS handling?

 David


 Seems to be due to throwing exceptions, not TLS.

 http://d.puremagic.com/issues/show_bug.cgi?id=6376


 Yeah, can confirm that, all my issues seem to be somehow related to
 exceptions to (even though in non-obvious ways sometimes).

 David


This fixes the issue for me:

https://github.com/D-Programming-Language/druntime/pull/42
Jul 25 2011
prev sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
On Jul 23, 2011, at 10:24 PM, David Nadlinger wrote:


 I have been observing crashes in semi-random places around my D =


applications (Segmentation fault/Bus errors) on the recently released OS =
X 10.7, dubbed Lion. They miraculously disappeared every time I ran the =
executables from GDB, and as I found out (thanks wm4 for the hint), this =
was due to GDB disabling ASLR by default. And indeed, if I =BBset =
disable-aslr off=AB, the crashes also happened within the debugger.

=20
 This is not totally surprising as Lion is the first OS X release to =


include full ASLR for both 32 bit and 64 bit applications, but I have =
not been able to track down what exactly goes on.

=20
 Any guesses what could go wrong here without additional details (I'm =


not at all sure where to look right now)? Maybe something related to the =
OS X-specific things like TLS handling?

I ran into this issue but didn't have time to track it down at the time. =
 I'd guess that it's the same old object file generation issue that =
arises with every OSX release (where the way DMD marks data blocks for =
exception handling, TLS, etc, breaks for one reason or another), and =
that ASLR is simply the latest cause.  I couldn't suggest a fix =
though... I don't know how ASLR works well enough.=
Jul 24 2011
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2011-07-25 05:07:32 +0000, Sean Kelly <sean invisibleduck.org> said:


 I ran into this issue but didn't have time to track it down at the time.
  I'd guess that it's the same old object file generation issue that
 arises with every OSX release (where the way DMD marks data blocks for
 exception handling, TLS, etc, breaks for one reason or another), and
 that ASLR is simply the latest cause.  I couldn't suggest a fix
 though... I don't know how ASLR works well enough.


The way all those blocks are identified by the runtime is a hack. It 
works, but it's too fragile. What needs to be done is to use the proper 
API for getting data segments -- namely "getsectdatafromheader" or 
"getsectdatafromheader_64", or some equivalent -- instead of depending 
on some dummy segments always being ordered in a specific way by the 
linker and the loader.

There was a discussion about that on the phobos mailing list back in 
november 2010 (see "Showstopper bug: Hello world fails on OSX!").

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Jul 25 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-25 13:29, Michel Fortin wrote:

 On 2011-07-25 05:07:32 +0000, Sean Kelly <sean invisibleduck.org> said:


 I ran into this issue but didn't have time to track it down at the time.
 I'd guess that it's the same old object file generation issue that
 arises with every OSX release (where the way DMD marks data blocks for
 exception handling, TLS, etc, breaks for one reason or another), and
 that ASLR is simply the latest cause. I couldn't suggest a fix
 though... I don't know how ASLR works well enough.


 The way all those blocks are identified by the runtime is a hack. It
 works, but it's too fragile. What needs to be done is to use the proper
 API for getting data segments -- namely "getsectdatafromheader" or
 "getsectdatafromheader_64", or some equivalent -- instead of depending
 on some dummy segments always being ordered in a specific way by the
 linker and the loader.

 There was a discussion about that on the phobos mailing list back in
 november 2010 (see "Showstopper bug: Hello world fails on OSX!").


I created a patch back then but it was never applied. The patch removed 
those "hacks" and uses "getsectdatafromheader" instead, but it never 
fixed the "Hello world fails on OSX!" issue completely.

http://d.puremagic.com/issues/show_bug.cgi?id=4854

-- 
/Jacob Carlborg
Jul 25 2011
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/25/2011 5:57 AM, Jacob Carlborg wrote:

 I created a patch back then but it was never applied. The patch removed those
 "hacks" and uses "getsectdatafromheader" instead, but it never fixed the "Hello
 world fails on OSX!" issue completely.

 http://d.puremagic.com/issues/show_bug.cgi?id=4854



Why didn't it work?
Jul 25 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-25 20:49, Walter Bright wrote:

 On 7/25/2011 5:57 AM, Jacob Carlborg wrote:

 I created a patch back then but it was never applied. The patch
 removed those
 "hacks" and uses "getsectdatafromheader" instead, but it never fixed
 the "Hello
 world fails on OSX!" issue completely.

 http://d.puremagic.com/issues/show_bug.cgi?id=4854



 Why didn't it work?


As you can see in my last post in that issue I didn't get the original 
error, which was in "__tls_get_addr". But instead I got a different 
error in "strncmp":

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
0x97655133 in strncmp ()
(gdb) bt


Previous frame inner to this frame (gdb could not unwind past this frame)
(gdb)

Either I messed something up or the solution we thought of couldn't fix 
the problem. In either case it might good to apply anyway, it woks on 
Snow Leopard.

I guess now that we have github it should be pull request.

-- 
/Jacob Carlborg
Jul 25 2011
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/25/2011 12:00 PM, Jacob Carlborg wrote:

 Either I messed something up or the solution we thought of couldn't fix the
 problem. In either case it might good to apply anyway, it woks on Snow Leopard.


I'm pretty unsure about applying fixes that we don't know why they work or just 
what is wrong with them.
Jul 25 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-25 21:06, Walter Bright wrote:

 On 7/25/2011 12:00 PM, Jacob Carlborg wrote:

 Either I messed something up or the solution we thought of couldn't
 fix the
 problem. In either case it might good to apply anyway, it woks on Snow
 Leopard.


 I'm pretty unsure about applying fixes that we don't know why they work
 or just what is wrong with them.


What I meant was that I think this is a good change to do regardless if 
this bug did exist or not.

-- 
/Jacob Carlborg
Jul 25 2011
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/25/2011 11:28 PM, Jacob Carlborg wrote:

 On 2011-07-25 21:06, Walter Bright wrote:

 On 7/25/2011 12:00 PM, Jacob Carlborg wrote:

 Either I messed something up or the solution we thought of couldn't
 fix the
 problem. In either case it might good to apply anyway, it woks on Snow
 Leopard.


 I'm pretty unsure about applying fixes that we don't know why they work
 or just what is wrong with them.


 What I meant was that I think this is a good change to do regardless if this
bug
 did exist or not.


Ok.
Jul 26 2011