www.digitalmars.com         C & C++   DMDScript  

D - Dernel: Protected mode far pointers in D?

reply "=?iso-8859-1?Q?Robert_M._M=FCnch?=" <robert.muench robertmuench.de> writes:
Hi, I found this quote:

	A disadvantage of all the mainstream IA-32 compilers is that they assume  
a flat 32-bit address space, with CS, DS, ES and SS each having the same  
base address. Because they don't use far (48-bit seg16:ofs32) pointers  
they make it a lot harder to write code for a segmented OS. Programs on a  
segmented OS written in C would have to rely on a lot of assembly code and  
would be a lot less efficient than programs written for a flat OS.  
However, at the time of writing, Watcom are still promising to release  
their OpenWatcom compiler, which will supposedly support far pointers in  
32-bit protected mode.

Can the D compiler use protected mode far pointers? Is it an issue to care  
about at all?

-- 
Robert M. Münch
Management & IT Freelancer
http://www.robertmuench.de
Mar 02 2004
next sibling parent reply "Jeroen van Bemmel" <someone somewhere.com> writes:
 Can the D compiler use protected mode far pointers? Is it an issue to care
 about at all?

How many "segmented OSs" can you list?
Mar 03 2004
parent reply "=?iso-8859-1?Q?Robert_M._M=FCnch?=" <robert.muench robertmuench.de> writes:
On Wed, 3 Mar 2004 21:06:25 +0100, Jeroen van Bemmel  
<someone somewhere.com> wrote:

 How many "segmented OSs" can you list?

Hmm... I don't know. I tried google but didn't found anything. However, I'm not that a big fan of x86 (this is mostly due being an old Amigian with 68x assembler know-how) but x86 is pretty advanced these days and using segmentation seems to be a valuable option. -- Robert M. Münch Management & IT Freelancer http://www.robertmuench.de
Mar 05 2004
parent reply "Walter" <walter digitalmars.com> writes:
"Robert M. Münch" <robert.muench robertmuench.de> wrote in message
news:opr4d2mbseheztw6 news.digitalmars.com...
 On Wed, 3 Mar 2004 21:06:25 +0100, Jeroen van Bemmel
 <someone somewhere.com> wrote:

 How many "segmented OSs" can you list?

Hmm... I don't know. I tried google but didn't found anything. However, I'm not that a big fan of x86 (this is mostly due being an old Amigian with 68x assembler know-how) but x86 is pretty advanced these days and using segmentation seems to be a valuable option.

It would be a differentiating feature! But it'd be crippled by the lack of tools for writing segmented 32 bit code. Instead, how about these features: 1) The x86 CPU's have 4 privilege levels. These are unused by windows, and I think linux too, which runs everything at the max privilege level (or they only use 2 of the 4 levels, I forgot the details). This is why security is so difficult on windows/linux, as everything is essentially root. Using the 4 levels enables you to create a *hardware* sandbox for running, for example, internet apps. This is inherently much more robust than software sandboxes such as Java's vm or .net. 2) Expose the full functionality of the virtual paging hardware to application programming. This will enable better/faster garbage collectors to be written. For example, the gc could read the 'dirty' bit on a page to see if it needs to be scanned. In fact, gc should be part of the os core functionality, so that all gc'd languages can have access to it.
Mar 05 2004
next sibling parent "=?iso-8859-1?Q?Robert_M._M=FCnch?=" <robert.muench robertmuench.de> writes:
On Fri, 5 Mar 2004 18:52:17 -0800, Walter <walter digitalmars.com> wrote:

 It would be a differentiating feature! But it'd be crippled by the lack  
 of tools for writing segmented 32 bit code.

Hi, yes right, that's the impression I get too. IMO for user applications it doesn't make any sense to muck around with this x86 feature. But using such features only on the kernel level and making it transparent to applications might be an option. I'm going to investigate this further. \justification Why to care anyhow? As I undestand current approaches, he problem is that there is no 4GB of memory available to user applications. That's mostly because the kernel needs to map in the address space as well. Hence the well known 1GB/3GB or 2GB/2GB slice. So, getting as much memory as possible in the fastest way is something that applications will benefit from. /justification
 1) The x86 CPU's have 4 privilege levels.

Yes, that's a nice feature being available in hardware.
 These are unused by windows, and I think linux too,which runs everything  
 at the max privilege level (or they only use 2 of the 4 levels, I forgot  
 the details).

Aha, I didn't knew this. Interesting... I expected that this feature is used.
 This is why security is
 so difficult on windows/linux,
 as everything is essentially root.

Do these use the x86 protection features for segments or are those checks done within a some kernel internal code?
 Using the 4 levels enables you to create a *hardware* sandbox for  
 running, for
 example, internet apps. This is inherently much more robust than software
 sandboxes such as Java's vm or .net.

Yes, good point.
 2) Expose the full functionality of the virtual paging hardware to
 application programming. This will enable better/faster garbage  
 collectors to be written. For example, the gc could read the 'dirty' bit  
 on a page to see if it needs to be scanned. In fact, gc should be part  
 of the os core
 functionality, so that all gc'd languages can have access to it.

The latter option is the one I'm interested in. With the OSs I know, you now have several layers of memory-management, starting from the virtual paging hardware, thru the OS, up to application level. My idea is to get the GC as much down to the metal as possible and than provide very simple, fast and lean interface to it. Even C/C++ code can have an implementation of memory functions using the GC than. Thanks for the ideas and the information. -- Robert M. Münch Management & IT Freelancer http://www.robertmuench.de
Mar 06 2004
prev sibling parent Scott Wood <scott buserror.net> writes:
On Fri, 5 Mar 2004 18:52:17 -0800, Walter <walter digitalmars.com> wrote:
 1) The x86 CPU's have 4 privilege levels. These are unused by
 windows, and I think linux too, which runs everything at the max
 privilege level (or they only use 2 of the 4 levels, I forgot the
 details).

They use two levels (which is all that paging provides on x86), 0 and 3.
 This is why security is so difficult on windows/linux, as
 everything is essentially root. Using the 4 levels enables you to
 create a *hardware* sandbox for running, for example, internet
 apps. 

You can do that with two levels as well. The problem on Linux is not that everything's root (as that's not the case), but that the security model it inherited from Unix is rather primitive, causing a lot of things to have more privilege than they need. I'm not familiar enough with Windows to comment on its security model (for those versions that have one (i.e. NT and derivatives)).
 2) Expose the full functionality of the virtual paging hardware to
 application programming. This will enable better/faster garbage collectors
 to be written. For example, the gc could read the 'dirty' bit on a page to
 see if it needs to be scanned. 

That'd be nice. The ability to request arbitrary copy-on-write regions (as opposed to only being able to use it with fork() and inter-process mmap()) would be great as well, both for doing GC in the background and for non-GC-related things.
 In fact, gc should be part of the os core
 functionality, so that all gc'd languages can have access to it.

Though there should be a way of hooking language-specific GC code into that, which can know more about how the data is laid out. -Scott
Mar 06 2004
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Robert M. Münch" <robert.muench robertmuench.de> wrote in message
news:opr39xslc8heztw6 news.digitalmars.com...
 Hi, I found this quote:

 A disadvantage of all the mainstream IA-32 compilers is that they assume
 a flat 32-bit address space, with CS, DS, ES and SS each having the same
 base address. Because they don't use far (48-bit seg16:ofs32) pointers
 they make it a lot harder to write code for a segmented OS. Programs on a
 segmented OS written in C would have to rely on a lot of assembly code and
 would be a lot less efficient than programs written for a flat OS.
 However, at the time of writing, Watcom are still promising to release
 their OpenWatcom compiler, which will supposedly support far pointers in
 32-bit protected mode.

 Can the D compiler use protected mode far pointers?

No. I know all about protected mode far pointers in the 16 bit compiler and the grief it causes.
 Is it an issue to care
 about at all?

No. <g> Nobody has ever asked for this capability for 32 bit code generation for 15 years. Writing one would imply one needed a bigger memory space for a program than 2 gigs. But these days, if you need that, you'd be better off programming for the new 64 bit chips.
Mar 04 2004
parent reply "=?iso-8859-1?Q?Robert_M._M=FCnch?=" <robert.muench robertmuench.de> writes:
On Thu, 4 Mar 2004 10:54:10 -0800, Walter <walter digitalmars.com> wrote:

 No. I know all about protected mode far pointers in the 16 bit compiler  
 and the grief it causes.

Hi, I can remember some parts too.
 No. <g> Nobody has ever asked for this capability for 32 bit code  
 generation for 15 years. Writing one would imply one needed a bigger  
 memory space for a program than 2 gigs.

Which happens these days quite fast... at least in enterprise applications.
 But these days, if you need that, you'd be better off
 programming for the new 64 bit chips.

Are these than supported by the D compiler? ;-) -- Robert M. Münch Management & IT Freelancer http://www.robertmuench.de
Mar 05 2004
next sibling parent Scott Wood <scott buserror.net> writes:
On Fri, 05 Mar 2004 13:43:15 +0100, Robert M. Münch
<robert.muench robertmuench.de> wrote:
 On Thu, 4 Mar 2004 10:54:10 -0800, Walter <walter digitalmars.com> wrote:
 No. <g> Nobody has ever asked for this capability for 32 bit code  
 generation for 15 years. Writing one would imply one needed a bigger  
 memory space for a program than 2 gigs.

Which happens these days quite fast... at least in enterprise applications.

Segmentation won't help there, though. The result of a base+offset calculation is still a 32-bit virtual address. 32-bit segmentation (beyond basic fixed values for CS and DS/ES/SS) is mainly useful for the transparent relocation/growth of pointers, but it's really not worth it IMHO. Even with compiler support, manipulating segments (and storing the larger pointers) is going to cause significant overhead and lots of programmer headaches. -Scott
Mar 05 2004
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Robert M. Münch" <robert.muench robertmuench.de> wrote in message
news:opr4d2ydf0heztw6 news.digitalmars.com...
 But these days, if you need that, you'd be better off
 programming for the new 64 bit chips.

Are these than supported by the D compiler? ;-)

Not yet, I don't even have a 64 bit machine. But that's where things are going.
Mar 05 2004
parent Mark T <Mark_member pathlink.com> writes:
In article <c2be3f$ul1$2 digitaldaemon.com>, Walter says...
"Robert M. Münch" <robert.muench robertmuench.de> wrote in message
news:opr4d2ydf0heztw6 news.digitalmars.com...
 But these days, if you need that, you'd be better off
 programming for the new 64 bit chips.

Are these than supported by the D compiler? ;-)

Not yet, I don't even have a 64 bit machine. But that's where things are going.

not in the embedded world, we just moved to 32 bits :)
Mar 06 2004