www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - 64 Bit D Kernel

reply Brian Madden <untwisted gmail.com> writes:
Just a heads up to anyone interested, there is work currently being done 
on a 64 bit D kernel.  I know a few had expressed interest in seeing the 
source when we made some progress, so I'm posting what we have thus far. 
  Right now we're in the middle of making sure interrupts work so that 
we can move on to in-kernel memory management.  Our checklist / roadmap 
can be found here:

http://wiki.pittgeeks.org/index.php?title=PGOS#TODO

and the source tree can be found here (there are branches in 
pgos/PaGanOS/branch, but they're all defunct for now):

http://www.pittgeeks.org/pgos/PaGanOS/trunk/src

As of right now our code is sort of all over the place, and fairly 
undocumented, but we're mostly groping in the dark right now.  If anyone 
has questions or comments, feel free to voice them.

-Brian
Oct 21 2007
next sibling parent reply doob <doobnet gmail.com> writes:
I suggest using Tango as the runtime lib or maybe it's possible to have support
for both Tango and Phobos.
Oct 22 2007
parent Alexander Panek <a.panek brainsware.org> writes:
doob wrote:
 I suggest using Tango as the runtime lib or maybe it's possible to have
support for both Tango and Phobos.

I doubt this is going to be part of the agenda for quite some time. (Apart from the fact that it has nothing to do with the operating system.)
Oct 22 2007
prev sibling parent reply Alexander Panek <a.panek brainsware.org> writes:
Brian Madden wrote:
 As of right now our code is sort of all over the place, and fairly 
 undocumented, but we're mostly groping in the dark right now.  If anyone 
 has questions or comments, feel free to voice them.

I hope there'll be some more documentation, soon. I don't know how you guys communicate during development, but e.g. a public mailing list/newsgroup/forum would be tremendously cool to track down your design/implementation decisions and such (doesn't really make sense if you're sitting next to each other, communicating vocally, of course). Ad comments..mhm.. I got a few questions, if you don't mind :) - Did you guys try to make the build process work with DSSS entirely? - Where there approaches to get rid of assembly and/or C? - How's the memory footprint (what's the minimum amount of memory you have to give your virtual machine so that Grub still loads the kernel)? - ..important question: Why 64 bit? (just interested in your decisions, as you might have guessed :) ) - Why exokernel? Why not micro? .. I think that's it for now. :) Best regards, Alex P.S.: Keep up the good work!
Oct 22 2007
parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Alexander Panek" <a.panek brainsware.org> wrote in message 
news:ffk3sd$2is7$1 digitalmars.com...
 Brian Madden wrote:
 As of right now our code is sort of all over the place, and fairly 
 undocumented, but we're mostly groping in the dark right now.  If anyone 
 has questions or comments, feel free to voice them.

I hope there'll be some more documentation, soon. I don't know how you guys communicate during development, but e.g. a public mailing list/newsgroup/forum would be tremendously cool to track down your design/implementation decisions and such (doesn't really make sense if you're sitting next to each other, communicating vocally, of course).

Usually we communicate with our mouths. Texts, IMing, and vulgar hand signals are our other main channels of communication. ;)
 Ad comments..mhm.. I got a few questions, if you don't mind :)

 - Did you guys try to make the build process work with DSSS entirely?

We've got a pretty involved build process, and not many files yet. I don't think DSSS would be of much use at least until a later stage.
 - Where there approaches to get rid of assembly and/or C?

There are no C files left. I think the only (non-inline) assembly left is boot.S, which is called by GRUB and sets up the stack, initial GDT, and initial page table and makes the jump to 64-bit mode. After that, it's all D with some inline assembly here and there (thank you GDC's extended ASM syntax). As for _how_ we god rid of C, I don't think there really was much, besides "translate this C file to D!"
 - How's the memory footprint (what's the minimum amount of memory you have 
 to give your virtual machine so that Grub still loads the kernel)?

Haven't tried, but the image is no more than 512KB or so. No idea if that makes an impact.
 - ..important question: Why 64 bit? (just interested in your decisions, as 
 you might have guessed :) )

ITS AWESUM. x86-64 is probably going to be around for a while, and it doesn't really make sense to develop for a platform which is kind of going out of style. All the new OSes are 64-bit compatible. x64 also simplifies and removes some legacy crap (mostly segmentation and things related to it) that we would have had to have dealt with in 32-bit. I have no idea if we plan on moving this to any other architecture.
 - Why exokernel? Why not micro?

I defer to James for that argument :S
 .. I think that's it for now. :)

 Best regards,
 Alex

 P.S.: Keep up the good work! 

Oct 23 2007
parent reply Alexander Panek <a.panek brainsware.org> writes:
Jarrett Billingsley wrote:
 "Alexander Panek" <a.panek brainsware.org> wrote in message 
 news:ffk3sd$2is7$1 digitalmars.com...
 Brian Madden wrote:
 As of right now our code is sort of all over the place, and fairly 
 undocumented, but we're mostly groping in the dark right now.  If anyone 
 has questions or comments, feel free to voice them.

guys communicate during development, but e.g. a public mailing list/newsgroup/forum would be tremendously cool to track down your design/implementation decisions and such (doesn't really make sense if you're sitting next to each other, communicating vocally, of course).

Usually we communicate with our mouths. Texts, IMing, and vulgar hand signals are our other main channels of communication. ;)

That's great! Be sure to keep a Stik around.
 Ad comments..mhm.. I got a few questions, if you don't mind :)

 - Did you guys try to make the build process work with DSSS entirely?

We've got a pretty involved build process, and not many files yet. I don't think DSSS would be of much use at least until a later stage.

As of now, you seem to use a Makefile; building with DSSS would be more interesting in terms of "proof of concept", etc. I tend to try to keep a project sticking to its languages'(or language's) tools, which would be DSSS in case of D, respectively Make in case of C or assembly (though you can do that one file with DSSS, too :P).
 - Where there approaches to get rid of assembly and/or C?

There are no C files left. I think the only (non-inline) assembly left is boot.S, which is called by GRUB and sets up the stack, initial GDT, and initial page table and makes the jump to 64-bit mode. After that, it's all D with some inline assembly here and there (thank you GDC's extended ASM syntax).

Right..you jump into the cold water (64bit) right there, missed that.
 I have no idea if we plan on moving this to any other architecture.

Hehe, maybe you should at least /plan/ to, so you have a code base that allows it....
 - Why exokernel? Why not micro?

I defer to James for that argument :S

...but apparently James knows more about this in particular. :) Personally, I am planning on splitting up my kernel into a completely platform dependant (the nucleus), and a mostly independant (the actual kernel) part. Basically, the nucleus implements a specific interface for all sorts of memory operations (allocating, sharing, moving, as well as paging and other rather platform specific [from an implementation point of view] things) and maybe additionally the base of IPC, as long as the platform has some support for it, that might speed up things. The rest should be implemented "generic" enough to keep the code base platform independent. If this independency isn't feasable, this particular piece of code should be put into a module, rather than "polluting" the kernel package(s). Best regards, Alex
Oct 23 2007
parent Brian Madden <untwisted gmail.com> writes:
Alexander Panek wrote:
 Jarrett Billingsley wrote:
 "Alexander Panek" <a.panek brainsware.org> wrote in message 
 news:ffk3sd$2is7$1 digitalmars.com...
 Brian Madden wrote:
 As of right now our code is sort of all over the place, and fairly 
 undocumented, but we're mostly groping in the dark right now.  If 
 anyone has questions or comments, feel free to voice them.

you guys communicate during development, but e.g. a public mailing list/newsgroup/forum would be tremendously cool to track down your design/implementation decisions and such (doesn't really make sense if you're sitting next to each other, communicating vocally, of course).

Usually we communicate with our mouths. Texts, IMing, and vulgar hand signals are our other main channels of communication. ;)

That's great! Be sure to keep a Stik around.
 Ad comments..mhm.. I got a few questions, if you don't mind :)

 - Did you guys try to make the build process work with DSSS entirely?

We've got a pretty involved build process, and not many files yet. I don't think DSSS would be of much use at least until a later stage.

As of now, you seem to use a Makefile; building with DSSS would be more interesting in terms of "proof of concept", etc. I tend to try to keep a project sticking to its languages'(or language's) tools, which would be DSSS in case of D, respectively Make in case of C or assembly (though you can do that one file with DSSS, too :P).
 - Where there approaches to get rid of assembly and/or C?

There are no C files left. I think the only (non-inline) assembly left is boot.S, which is called by GRUB and sets up the stack, initial GDT, and initial page table and makes the jump to 64-bit mode. After that, it's all D with some inline assembly here and there (thank you GDC's extended ASM syntax).

Right..you jump into the cold water (64bit) right there, missed that.
 I have no idea if we plan on moving this to any other architecture.

Hehe, maybe you should at least /plan/ to, so you have a code base that allows it....
 - Why exokernel? Why not micro?


 I defer to James for that argument :S

...but apparently James knows more about this in particular. :)

The short answer to this is that we really like the exokernel idea, and in prior research has been shown to significantly increase the speed of applications. The long answer is that we're really in to the idea of using/writing pluggable libOSes. It'd have a huge (positive) impact on research, as well as system performance. Since libOSes are found in userland rather than in the kernel there is less trapping (less context switching), and more stability (even if the libOS dies it doesn't mean the system dies). The kernel itself is very small and only performs protection, not abstraction. This protection, not abstraction idea gives developers a wider range of access possibilities. Since we're not abstracting, someone could write code that is highly optimized for the hardware using direct access, or they could write code that makes system calls to libOSes that handle the hardware access for them. James wrote up a little manifesto that you can read here: http://wiki.pittgeeks.org/index.php?title=Why_an_exokernel%3F that explains everything with a bit more depth. :)
 
 Personally, I am planning on splitting up my kernel into a completely 
 platform dependant (the nucleus), and a mostly independant (the actual 
 kernel) part. Basically, the nucleus implements a specific interface for 
 all sorts of memory operations (allocating, sharing, moving, as well as 
 paging and other rather platform specific [from an implementation point 
 of view] things) and maybe additionally the base of IPC, as long as the 
 platform has some support for it, that might speed up things.
 
 The rest should be implemented "generic" enough to keep the code base 
 platform independent. If this independency isn't feasable, this 
 particular piece of code should be put into a module, rather than 
 "polluting" the kernel package(s).

Thats not a bad idea, and I'm pretty sure that without really talking about it, we had intended to do this already. The boot code we're working with right now is x86_64 specific, but once we start on the kernel's protection mechanisms we'll keep things generic enough that it can be shoved on top of different boot code and just work.
 
 Best regards,
 Alex

Oct 25 2007