www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 9243] New: Order locals to prevent buffer overflow attacks

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9243

           Summary: Order locals to prevent buffer overflow attacks
           Product: D
           Version: D2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: DMD
        AssignedTo: nobody puremagic.com
        ReportedBy: bugzilla digitalmars.com



17:37:34 PST ---
The idea is to order the locals so that local static arrays are at higher
addresses than the pointer and integral locals. This prevents array overflows
from being able to rewrite those pointers (and integers that may be used as
array indices). This, of course, won't stop overwriting the return address, but
this enhancement has zero cost, so why not?

Yes, array overflow checking is done and would prevent buffer overflow attacks,
but this feature can be turned off for performance reasons.

Another idea is to use the return value as the "canary". Upon function entry,
copy the return value to a location on the other side of the arrays. Upon exit,
compare that with the original return value. If they differ, halt.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Dec 29 2012
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9243


bearophile_hugs eml.cc changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bearophile_hugs eml.cc





 but this enhancement has zero cost, so why not?


Advanced compilers do two things:
- If in a function you have nested scopes that define variables, two successive
scopes are sometimes allowed to re-use the same stack space inside the same
stack frame. I think this can "shuffle" (change) the position of variables in
the stack frame.
- I think they sometimes use profile-driven optimization to tell what are the
hottest stack variables used in a stack frame. This information is probably
later used to set the position of the variables in the stack frame to put the
hottest closer, or on the contrary to let them go different CPU cache lines, to
avoid a slower modify-and-read access patterns.

So maybe it's better to use this idea only in debug builds.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Dec 29 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9243




18:30:18 PST ---




 but this enhancement has zero cost, so why not?


 
 Advanced compilers do two things:
 - If in a function you have nested scopes that define variables, two successive
 scopes are sometimes allowed to re-use the same stack space inside the same
 stack frame. I think this can "shuffle" (change) the position of variables in
 the stack frame.


Dmd already does this using graph coloring. I guess dmd is advanced? :-)



 - I think they sometimes use profile-driven optimization to tell what are the
 hottest stack variables used in a stack frame. This information is probably
 later used to set the position of the variables in the stack frame to put the
 hottest closer, or on the contrary to let them go different CPU cache lines, to
 avoid a slower modify-and-read access patterns.


Dmd doesn't do this one, although it could based on the same weighting
algorithm used to prioritize register assignments.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Dec 29 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9243







 Dmd already does this using graph coloring. I guess dmd is advanced? :-)


Despite in some cases DMD compiles in a sub-optimal way compared to LDC/GDC,
given the complexity of D language DMD is surely an advanced compiler.



 Dmd doesn't do this one, although it could based on the same weighting
 algorithm used to prioritize register assignments.


The point of my comments isn't about what dmd does or doesn't. The point is
that some compilers perform optimizations that go against the safety feature
you propose here. So I have suggested that it's better to activate this safety
feature only when the programmer wants it, and let the compiler do things its
way otherwise.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Dec 29 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9243




21:59:11 PST ---



 

 Dmd already does this using graph coloring. I guess dmd is advanced? :-)


 
 Despite in some cases DMD compiles in a sub-optimal way compared to LDC/GDC,
 given the complexity of D language DMD is surely an advanced compiler.


The complexity of D has nothing to do with the advanced'ness of the optimizer
and back end. Dmc (which shares a back end with dmd) has had an advanced data
flow analysis back end since around 1985. One of the things data flow analysis
provides is a weighted usage graph of variables, which is used for register
allocation and also for sharing stack space.

I think it was last week that LLVM announced its latest version is using
coloring information for stack layout - about 20 years behind dmc/dmd.

Please check what dmd does before defaulting to assuming it does not.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Dec 29 2012
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9243







 The complexity of D has nothing to do with the advanced'ness of the optimizer
 and back end. Dmc (which shares a back end with dmd) has had an advanced data
 flow analysis back end since around 1985. One of the things data flow analysis
 provides is a weighted usage graph of variables, which is used for register
 allocation and also for sharing stack space.


There are several different things that can be named "compiler", but for me and
normal programmers a "compiler" isn't just a back-end, it's the sum of
front-end plus a back-end. From Wikipedia
(http://en.wikipedia.org/wiki/Compiler ):


The name "compiler" is primarily used for programs that translate source code
from a high-level programming language to a lower level language (e.g.,
assembly language or machine code).<




 I think it was last week that LLVM announced its latest version is using
 coloring information for stack layout - about 20 years behind dmc/dmd.


Life is complex, reading the text of the LLVM changelog is not enough to say X
is better than Y. So you should write some benchmarks and compare between LLVM
and Dmc the performance, both in compilation speed and in the number of spilled
registers.



 Please check what dmd does before defaulting to assuming it does not.


I am doing my best.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Dec 30 2012