www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - A different kind of Walter? :-)

reply John Reimer <terminal.node gmail.com> writes:
See this:

http://www.beosmax.org/wiki/index.php/WalterOS

Seems Haiku, the long developed BeOS clone, is going to be a
Walter.  But where is D? :D

-JJR
Apr 12 2007
next sibling parent reply Davidl <Davidl 126.com> writes:
i would rather call it BrightOS

 See this:

 http://www.beosmax.org/wiki/index.php/WalterOS

 Seems Haiku, the long developed BeOS clone, is going to be a
 Walter.  But where is D? :D

 -JJR

Apr 12 2007
next sibling parent reply John Reimer <terminal.node gmail.com> writes:
Yeah, true.  Regardless, I'd love to see a D-based OS someday. :-)

-JJR
Apr 12 2007
next sibling parent reply Dan <murpsoft hotmail.com> writes:
Alexander Panek Wrote:
 On Fri, 13 Apr 2007 05:31:49 +0000 (UTC)
 John Reimer <terminal.node gmail.com> wrote:
 
 Yeah, true.  Regardless, I'd love to see a D-based OS someday. :-)

Just.. give me some time (and pay me, then it'll take less time, haha! :P). Eventually hakware (from #d) made interrupts working, and added quite some features upon my codebase - even a working garbage collector is present at kernel-level, IIRC. :D I'm sure going to spend some time getting a proper codebase for further development done, and write some documentation about how to actually get to the point of being able to write an operating system with D. I think that's a major weak point of D, as it claims to be a systems language, but there's no actual system written in it from scratch, neither documentation on how to achieve that.

Alex, please try to contact me. I'm interested in assisting with this project, and am well able to contribute.
Apr 17 2007
parent reply Alexander Panek <a.panek brainsware.org> writes:
Dan wrote:
 Alex, please try to contact me.  I'm interested in assisting with this
project, and am well able to contribute.

Do you use IRC? If so, hakware would be glad to have assistance, I bet. (#xana Freenode) I'm not actively developing the OS at the moment, as I have my finals at school now.. sucks up all my time, nerves and such. But I will continue developing when I'm finished with school, which is about the 5th of June. Best regards, Alex
Apr 17 2007
parent reply Dan <murpsoft hotmail.com> writes:
Alexander Panek Wrote:

 Dan wrote:
 Alex, please try to contact me.  I'm interested in assisting with this
project, and am well able to contribute.

Do you use IRC? If so, hakware would be glad to have assistance, I bet. (#xana Freenode) I'm not actively developing the OS at the moment, as I have my finals at school now.. sucks up all my time, nerves and such. But I will continue developing when I'm finished with school, which is about the 5th of June. Best regards, Alex

Oh my... IRC? I haven't used that for almost 12 years. Like Windows 3.11. : ) That said, what url is "Freenode", and/or how else can I contact this "hakware"? : p I feel like an old man, and I'm only 23. : p
Apr 17 2007
next sibling parent reply Sean Kelly <sean f4.ca> writes:
Dan wrote:
 
 That said, what url is "Freenode", and/or how else can I contact this
"hakware"?

If you're using Firefox, this link should work: irc://chat.freenode.net Then /join #D. Sean
Apr 17 2007
parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Sean Kelly wrote:
 Dan wrote:
 That said, what url is "Freenode", and/or how else can I contact this 
 "hakware"?

If you're using Firefox, this link should work: irc://chat.freenode.net Then /join #D. Sean

Alert ------------------ Firefox doesn't know how to open this address, because the protocol (irc) isn't associated with any program. Darn. I was all set to be impressed by Firefox's versatility. Apparently it doesn't handle irc internally, but just opens up an external client for you. And it seems that's not set up by default -- at least on Windows. --bb
Apr 17 2007
next sibling parent James Dennett <jdennett acm.org> writes:
Bill Baxter wrote:
 Sean Kelly wrote:
 Dan wrote:
 That said, what url is "Freenode", and/or how else can I contact this
 "hakware"?

If you're using Firefox, this link should work: irc://chat.freenode.net Then /join #D. Sean

Alert ------------------ Firefox doesn't know how to open this address, because the protocol (irc) isn't associated with any program. Darn. I was all set to be impressed by Firefox's versatility. Apparently it doesn't handle irc internally, but just opens up an external client for you. And it seems that's not set up by default -- at least on Windows.

It's an option for Firefox to handle it with an integrated IRC client. It's not a great client though; I think it's still viewed as experimental, but I may be out of date. -- James
Apr 17 2007
prev sibling next sibling parent reply Mike Parker <aldacron71 yahoo.com> writes:
Bill Baxter wrote:

 ------------------
 Firefox doesn't know how to open this address, because the protocol 
 (irc) isn't associated with any program.
 
 Darn.  I was all set to be impressed by Firefox's versatility. 
 Apparently it doesn't handle irc internally, but just opens up an 
 external client for you.  And it seems that's not set up by default -- 
 at least on Windows.

You can install ChatZilla via the Firefox/Mozilla extension system: https://addons.mozilla.org/en-US/firefox/addon/16
Apr 17 2007
parent Bill Baxter <dnewsgroup billbaxter.com> writes:
Mike Parker wrote:
 Bill Baxter wrote:
 
 ------------------
 Firefox doesn't know how to open this address, because the protocol 
 (irc) isn't associated with any program.

 Darn.  I was all set to be impressed by Firefox's versatility. 
 Apparently it doesn't handle irc internally, but just opens up an 
 external client for you.  And it seems that's not set up by default -- 
 at least on Windows.

You can install ChatZilla via the Firefox/Mozilla extension system: https://addons.mozilla.org/en-US/firefox/addon/16

Neat. Thanks. --bb
Apr 17 2007
prev sibling parent Sean Kelly <sean f4.ca> writes:
Bill Baxter wrote:
 Sean Kelly wrote:
 Dan wrote:
 That said, what url is "Freenode", and/or how else can I contact this 
 "hakware"?

If you're using Firefox, this link should work: irc://chat.freenode.net Then /join #D. Sean

Alert ------------------ Firefox doesn't know how to open this address, because the protocol (irc) isn't associated with any program. Darn. I was all set to be impressed by Firefox's versatility. Apparently it doesn't handle irc internally, but just opens up an external client for you. And it seems that's not set up by default -- at least on Windows.

Huh. I'm pretty sure that used to work. Or maybe that's from when I used Mozilla. In any case, ChatZilla (the client I was thinking of) is (now?) a separate download for Firefox. Sean
Apr 17 2007
prev sibling parent Pragma <ericanderton yahoo.removeme.com> writes:
Dan wrote:
 Alexander Panek Wrote:
 
 Dan wrote:
 Alex, please try to contact me.  I'm interested in assisting with this
project, and am well able to contribute.

(#xana Freenode) I'm not actively developing the OS at the moment, as I have my finals at school now.. sucks up all my time, nerves and such. But I will continue developing when I'm finished with school, which is about the 5th of June. Best regards, Alex

Oh my... IRC? I haven't used that for almost 12 years. Like Windows 3.11. : ) That said, what url is "Freenode", and/or how else can I contact this "hakware"? : p I feel like an old man, and I'm only 23. : p

Dont' feel bad. When I first logged into freenode it was only a year or two ago. Being in an interactive text environment reminded me of time (mis)spent on local bulletin boards a long time ago. I could almost swear I heard my old modem connecting... -- - EricAnderton at yahoo
Apr 17 2007
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Alexander Panek wrote:
 I'm sure going to spend some time getting a proper codebase for further
 development done, and write some documentation about how to actually
 get to the point of being able to write an operating system with D. I
 think that's a major weak point of D, as it claims to be a systems
 language, but there's no actual system written in it from scratch,
 neither documentation on how to achieve that.

Andrei suggested that the source code for Minix, which is fairly small, could be transliterated from C almost directly into D. This would neatly resolve the issue, and provide a starting point for anyone wanting to take it further.
Apr 17 2007
next sibling parent reply Alexander Panek <a.panek brainsware.org> writes:
Walter Bright wrote:
 Alexander Panek wrote:
 I'm sure going to spend some time getting a proper codebase for further
 development done, and write some documentation about how to actually
 get to the point of being able to write an operating system with D. I
 think that's a major weak point of D, as it claims to be a systems
 language, but there's no actual system written in it from scratch,
 neither documentation on how to achieve that.

Andrei suggested that the source code for Minix, which is fairly small, could be transliterated from C almost directly into D. This would neatly resolve the issue, and provide a starting point for anyone wanting to take it further.

True. On the other hand, a kernel written purely in D is something that really would (and will, arr) be cool! I just find it less exciting to code "C with D", so to speak. I'll make sure to announce the project, once a few milestones are finished, on the newsgroups.
Apr 17 2007
next sibling parent reply "David B. Held" <dheld codelogicconsulting.com> writes:
Alexander Panek wrote:
 [...]
 It might look like a "hackish" (yes, I call anything that's not
 properly designed and implemented - with pedantic attention for each
 detail - hackish..no offense intended, that just applies to my own
 code) kernel would has its benefits in the long-term, but I think a
 proper research to see what features of D can ease the pain of
 operating system development will gain more knowledge and will
 hopefully help me invent A Good Thing (tm).
 [...]

If you guys really are young college students and can afford to sink time into something like this, I say go for it 100%. It really doesn't matter whether it amounts to anything, because you will learn a ton from it and it will be a fun project. That's just the kind of thing to do at your age. At my age, you have to do things that matter (read: pay the bills), which is to say that you simply cannot afford to do something like write an OS kernel, and I really envy you. If you're really good, you can make a name for yourself with it, even if the kernel is a commercial failure, which is worth plenty to your career. That being said, I think it would be *particularly* clever of you to use your start-from-scratch kernel to explore new ideas in multi-processing and massive multithreading, but this would imply having decent multi-CPU hardware to play with. Most of the OS support we have is decades old technology hacked together to keep up with the hardware, but maybe you have an opportunity to do it right from the start. That is, to create a kernel where multiprocessing is as fundamental as memory management or process scheduling. We try to write lock-free algorithms using just one special assembly instruction, and look how far we have to bend over backwards to make it work. If an MP-friendly kernel gave us more primitives than CAS, because it was designed that way from the start (meaning, the process scheduler was hacked up in a new way), I suspect that you could make MP a lot easier on users. Just some stuff to think about... Dave
Apr 18 2007
next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
David B. Held wrote:
 That being said, I think it would be *particularly* clever of you to use 
 your start-from-scratch kernel to explore new ideas in multi-processing 
 and massive multithreading, but this would imply having decent multi-CPU 
 hardware to play with.

I'd also like to see better operating system support for garbage collection - either making gc an operating system service, or providing hooks to the virtual memory subsystem so the gc can tell when pages are dirty.
Apr 19 2007
next sibling parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Alexander Panek wrote:
 On Thu, 19 Apr 2007 01:07:22 -0700
 Walter Bright <newshound1 digitalmars.com> wrote:
 
 David B. Held wrote:
 That being said, I think it would be *particularly* clever of you
 to use your start-from-scratch kernel to explore new ideas in
 multi-processing and massive multithreading, but this would imply
 having decent multi-CPU hardware to play with.

collection - either making gc an operating system service, or providing hooks to the virtual memory subsystem so the gc can tell when pages are dirty.

Well, something along the lines of that is what I was thinking of. I'm not so sure how to make the GC an operating system service, though..?

http://www-cs.canisius.edu/~hertzm/bc.html has a nice idea on support the OS could give to garbage collection: they modified Linux to notify the process when it was about to get some of its memory swapped out. Then the process could perform a quick GC (focusing on freeing up pages of memory) and tell the OS which pages to unmap or swap out. The GC freeing pages means the GC can tell the OS memory manager that certain pages can be thrown out without saving them to disk first (no disk writes is faster than disk writes :) ). If it can't avoid swapping pages out, the GC checks the page about to be swapped out for pointers and marks the pointed-to objects as referenced by a swapped-out page, then tells the OS to put that page back at the head of the swap-out queue (since it has just been touched by the GC, it was removed from that queue). In this way, their GC also avoids touching swapped-out pages except in rare cases, so most GC cycles shouldn't cause a single page fault. Pretty cool stuff. About making it a system service: I'm not sure how much help that would be, though I guess it could cut down on privilege level switches (caused by warnings about swapping and/or system calls to nominate pages for swapping) if the kernel could perform something like the above GC itself.
Apr 19 2007
parent reply Sean Kelly <sean f4.ca> writes:
Frits van Bommel wrote:
 Alexander Panek wrote:
 On Thu, 19 Apr 2007 01:07:22 -0700
 Walter Bright <newshound1 digitalmars.com> wrote:

 David B. Held wrote:
 That being said, I think it would be *particularly* clever of you
 to use your start-from-scratch kernel to explore new ideas in
 multi-processing and massive multithreading, but this would imply
 having decent multi-CPU hardware to play with.

collection - either making gc an operating system service, or providing hooks to the virtual memory subsystem so the gc can tell when pages are dirty.

Well, something along the lines of that is what I was thinking of. I'm not so sure how to make the GC an operating system service, though..?

http://www-cs.canisius.edu/~hertzm/bc.html has a nice idea on support the OS could give to garbage collection: they modified Linux to notify the process when it was about to get some of its memory swapped out. Then the process could perform a quick GC (focusing on freeing up pages of memory) and tell the OS which pages to unmap or swap out. The GC freeing pages means the GC can tell the OS memory manager that certain pages can be thrown out without saving them to disk first (no disk writes is faster than disk writes :) ).

I can think of at least two papers on this topic for anyone interested. One is "Reducing Garbage Collector Cache Missed" by Hans Boehm, and the other is "Garbage Collection Without Paging" by Hertz, Feng, and Berger. Both should be available with a bit of googling. Sean
Apr 19 2007
parent Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Sean Kelly wrote:
 Frits van Bommel wrote:
 http://www-cs.canisius.edu/~hertzm/bc.html has a nice idea on support 
 the OS could give to garbage collection: they modified Linux to notify 
 the process when it was about to get some of its memory swapped out. 
 Then the process could perform a quick GC (focusing on freeing up 
 pages of memory) and tell the OS which pages to unmap or swap out. The 
 GC freeing pages means the GC can tell the OS memory manager that 
 certain pages can be thrown out without saving them to disk first (no 
 disk writes is faster than disk writes :) ).

I can think of at least two papers on this topic for anyone interested. One is "Reducing Garbage Collector Cache Missed" by Hans Boehm, and the other is "Garbage Collection Without Paging" by Hertz, Feng, and Berger. Both should be available with a bit of googling.

The second one is actually available at the link I posted, but thanks for the first one, I don't think I read that one yet.
Apr 19 2007
prev sibling next sibling parent reply Sean Kelly <sean f4.ca> writes:
Walter Bright wrote:
 David B. Held wrote:
 That being said, I think it would be *particularly* clever of you to 
 use your start-from-scratch kernel to explore new ideas in 
 multi-processing and massive multithreading, but this would imply 
 having decent multi-CPU hardware to play with.

I'd also like to see better operating system support for garbage collection - either making gc an operating system service, or providing hooks to the virtual memory subsystem so the gc can tell when pages are dirty.

Same here. That the OS is aware of context switches and such should allow for some clever methods of garbage collection. I'm actually a bit surprised that MS didn't do anything with GC support for Vista, given their push with CLI. Sean
Apr 19 2007
parent reply Dave <Dave_member pathlink.com> writes:
Sean Kelly wrote:
 Walter Bright wrote:
 David B. Held wrote:
 That being said, I think it would be *particularly* clever of you to 
 use your start-from-scratch kernel to explore new ideas in 
 multi-processing and massive multithreading, but this would imply 
 having decent multi-CPU hardware to play with.

I'd also like to see better operating system support for garbage collection - either making gc an operating system service, or providing hooks to the virtual memory subsystem so the gc can tell when pages are dirty.

Same here. That the OS is aware of context switches and such should allow for some clever methods of garbage collection. I'm actually a bit surprised that MS didn't do anything with GC support for Vista, given their push with CLI.

Not only that, but I've never even read/heard of any plans to add a GC to any main-stream OS's. Seems to me there's got to be a reason for this (not having an OS implement a GC). I wonder what that would be, since now-a-days catering to application development is such a vital thing for any commercial OS?
 
 Sean

Apr 20 2007
parent reply Dan <murpsoft hotmail.com> writes:
Dave Wrote:
 Seems to me there's got to be a reason for this (not having an OS implement a
GC). I wonder what 
 that would be, since now-a-days catering to application development is such a
vital thing for any 
 commercial OS?

Well, people are developing them. The reason they aren't included by default is a mystery to me, then most OS developers have left the entire process this gigantic undocumented undescribable mess that explains why most new OS starts just give up. I think it also explains why currently widely-used OS's still don't implement any additional features. It's difficult, and they don't have to. That said, I've been architecting a ring -1 program that multiplexes hardware resources without any drivers, HAL or OS underneath it; taking the exokernel principle to the farthest extreme. My exokernel isn't even a kernel, it's just a gatekeeper. I was trying to think of a good name for it: Maat, Aker, St. Peter, Gatekeeper, Guardian, BlackSphere, or Garmies were the first that came to mind.
Apr 20 2007
next sibling parent reply Chris Nicholson-Sauls <ibisbasenji gmail.com> writes:
Dan wrote:
 Dave Wrote:
 Seems to me there's got to be a reason for this (not having an OS implement a
GC). I wonder what 
 that would be, since now-a-days catering to application development is such a
vital thing for any 
 commercial OS?

Well, people are developing them. The reason they aren't included by default is a mystery to me, then most OS developers have left the entire process this gigantic undocumented undescribable mess that explains why most new OS starts just give up. I think it also explains why currently widely-used OS's still don't implement any additional features. It's difficult, and they don't have to. That said, I've been architecting a ring -1 program that multiplexes hardware resources without any drivers, HAL or OS underneath it; taking the exokernel principle to the farthest extreme. My exokernel isn't even a kernel, it's just a gatekeeper. I was trying to think of a good name for it: Maat, Aker, St. Peter, Gatekeeper, Guardian, BlackSphere, or Garmies were the first that came to mind.

Going the route of those first couple ideas (naming after psychopomps) there's the possibilities Yinepu/Anpu and Wepwawet (which literally means he who opens the way). Actually I think that's a perfect theme for selecting a name for something like that. Sounds like an interesting project indeed. -- Chris Nicholson-Sauls
Apr 20 2007
parent reply Dan <murpsoft hotmail.com> writes:
Chris Nicholson-Sauls Wrote:
 Dan wrote:
 Maat, Aker, St. Peter, Gatekeeper, Guardian, BlackSphere, or Garmies were the
first that came to mind.

Going the route of those first couple ideas (naming after psychopomps) there's the possibilities Yinepu/Anpu and Wepwawet (which literally means he who opens the way). Actually I think that's a perfect theme for selecting a name for something like that. Sounds like an interesting project indeed. -- Chris Nicholson-Sauls

I had to look psychopomps up, thanks Chris, I learned something new today. I initially took 'he who opens the way' in a buddhist sense, which would be cool, but Wepwawet was actually more scout-for-the-military sort of God. Check the mythology for Nandi, that's not too bad either. So, if I named it after Mr Pete, they catholics would probably hang me. If I name it 'Garmies', the Mormons will. If I name it 'Maat' I would probably get away with it; Maat is the Egyptian god of justice, who's feather would be weighed against one's heart upon death; and who ancient judges and even gods would seek approval. It's also unique enough that I don't think there'll be a name collision. So Maat it will be? Now to keep working on Walnut. : p
Apr 20 2007
parent Chris Nicholson-Sauls <ibisbasenji gmail.com> writes:
Dan wrote:
 Chris Nicholson-Sauls Wrote:
 Dan wrote:
 Maat, Aker, St. Peter, Gatekeeper, Guardian, BlackSphere, or Garmies were the
first that came to mind.

possibilities Yinepu/Anpu and Wepwawet (which literally means he who opens the way). Actually I think that's a perfect theme for selecting a name for something like that. Sounds like an interesting project indeed. -- Chris Nicholson-Sauls

I had to look psychopomps up, thanks Chris, I learned something new today. I initially took 'he who opens the way' in a buddhist sense, which would be cool, but Wepwawet was actually more scout-for-the-military sort of God. Check the mythology for Nandi, that's not too bad either. So, if I named it after Mr Pete, they catholics would probably hang me. If I name it 'Garmies', the Mormons will. If I name it 'Maat' I would probably get away with it; Maat is the Egyptian god of justice, who's feather would be weighed against one's heart upon death; and who ancient judges and even gods would seek approval. It's also unique enough that I don't think there'll be a name collision. So Maat it will be? Now to keep working on Walnut. : p

I just don't like anything else with a feather for its obvious logo. I've got a side project named Wren that intends to use that... ;) Then again, Apache beat us both to it, so who cares. Okay, shutting up now. (Du'a Netjer.) -- Chris Nicholson-Sauls
Apr 20 2007
prev sibling parent reply torhu <fake address.dude> writes:
Dan wrote:
 My exokernel isn't even a kernel, it's just a gatekeeper.  I was trying to
think of a good name for it:
 
 Maat, Aker, St. Peter, Gatekeeper, Guardian, BlackSphere, or Garmies were the
first that came to mind.

I think you forgot Zuul. :)
Apr 21 2007
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
torhu wrote:
 Dan wrote:
 My exokernel isn't even a kernel, it's just a gatekeeper.  I was
 trying to think of a good name for it:

 Maat, Aker, St. Peter, Gatekeeper, Guardian, BlackSphere, or Garmies
 were the first that came to mind.

I think you forgot Zuul. :)

There is no kernel, only Zuul. And if you remap StdOut to StdIn and vice-versa, you get total protonic reversal. That's bad. -- Daniel -- int getRandomNumber() { return 4; // chosen by fair dice roll. // guaranteed to be random. } http://xkcd.com/ v2sw5+8Yhw5ln4+5pr6OFPma8u6+7Lw4Tm6+7l6+7D i28a2Xs3MSr2e4/6+7t4TNSMb6HTOp5en5g6RAHCP http://hackerkey.com/
Apr 21 2007
parent Brad Anderson <brad dsource.org> writes:
Daniel Keep wrote:
 And if you remap StdOut to StdIn and vice-versa, you get total protonic
 reversal.  That's bad.

Right. That's bad. Okay. All right. Important safety tip. Thanks, Egon.
Apr 21 2007
prev sibling parent reply Lionello Lunesu <lio lunesu.remove.com> writes:
Walter Bright wrote:
 David B. Held wrote:
 That being said, I think it would be *particularly* clever of you to 
 use your start-from-scratch kernel to explore new ideas in 
 multi-processing and massive multithreading, but this would imply 
 having decent multi-CPU hardware to play with.

I'd also like to see better operating system support for garbage collection - either making gc an operating system service, or providing hooks to the virtual memory subsystem so the gc can tell when pages are dirty.

I wish there were a function like VirtualRealloc that would grow a memory block without EVER having to physically move the data. (That doesn't exist in Win32, does it?) L.
Apr 19 2007
parent reply Sean Kelly <sean f4.ca> writes:
Lionello Lunesu wrote:
 Walter Bright wrote:
 David B. Held wrote:
 That being said, I think it would be *particularly* clever of you to 
 use your start-from-scratch kernel to explore new ideas in 
 multi-processing and massive multithreading, but this would imply 
 having decent multi-CPU hardware to play with.

I'd also like to see better operating system support for garbage collection - either making gc an operating system service, or providing hooks to the virtual memory subsystem so the gc can tell when pages are dirty.

I wish there were a function like VirtualRealloc that would grow a memory block without EVER having to physically move the data. (That doesn't exist in Win32, does it?)

I don't think so. If you have some data whose logical address range is bounded by other data, I don't think it would be possible for it to grow without being moved. Sean
Apr 20 2007
next sibling parent reply "David B. Held" <dheld codelogicconsulting.com> writes:
Sean Kelly wrote:
 Lionello Lunesu wrote:
 [...]
 I wish there were a function like VirtualRealloc that would grow a 
 memory block without EVER having to physically move the data.

 (That doesn't exist in Win32, does it?)

I don't think so. If you have some data whose logical address range is bounded by other data, I don't think it would be possible for it to grow without being moved.

Well, there's at least two solutions to this. One is to use infinite memory, in which case you can allocate at addresses that are infinitely far apart, allowing for unlimited growth in data structures. The second is to use quantum superposition to allow storage of distinct data at the same location. The trick is getting the state to collapse in a way that makes all the data recoverable, which is nothing more than an engineering detail, really. We should get an intern to work on these projects over the summer. Dave
Apr 20 2007
parent Sean Kelly <sean f4.ca> writes:
David B. Held wrote:
 Sean Kelly wrote:
 Lionello Lunesu wrote:
 [...]
 I wish there were a function like VirtualRealloc that would grow a 
 memory block without EVER having to physically move the data.

 (That doesn't exist in Win32, does it?)

I don't think so. If you have some data whose logical address range is bounded by other data, I don't think it would be possible for it to grow without being moved.

Well, there's at least two solutions to this. One is to use infinite memory, in which case you can allocate at addresses that are infinitely far apart, allowing for unlimited growth in data structures. The second is to use quantum superposition to allow storage of distinct data at the same location. The trick is getting the state to collapse in a way that makes all the data recoverable, which is nothing more than an engineering detail, really. We should get an intern to work on these projects over the summer.

And Google's "summer of code" has already stopped taking applications, too. Darn it! Sean
Apr 20 2007
prev sibling parent reply "Lionello Lunesu" <lionello lunesu.remove.com> writes:
"Sean Kelly" <sean f4.ca> wrote in message 
news:f09ouq$2ium$1 digitalmars.com...
 Lionello Lunesu wrote:
 Walter Bright wrote:
 David B. Held wrote:
 That being said, I think it would be *particularly* clever of you to 
 use your start-from-scratch kernel to explore new ideas in 
 multi-processing and massive multithreading, but this would imply 
 having decent multi-CPU hardware to play with.

I'd also like to see better operating system support for garbage collection - either making gc an operating system service, or providing hooks to the virtual memory subsystem so the gc can tell when pages are dirty.

I wish there were a function like VirtualRealloc that would grow a memory block without EVER having to physically move the data. (That doesn't exist in Win32, does it?)

I don't think so. If you have some data whose logical address range is bounded by other data, I don't think it would be possible for it to grow without being moved.

I don't care if there address changes, just as long as there's no data being copied physically. I mean, a block of continuous memory might already be all-over-the-place in physical memory. That's what I want: a VirtualRealloc that returns the address of a new virtual memory block, but which is using the same physical memory.. 'Free' array resizes, surely this is worth something! L.
Apr 20 2007
parent reply Sean Kelly <sean f4.ca> writes:
Lionello Lunesu wrote:
 "Sean Kelly" <sean f4.ca> wrote in message 
 news:f09ouq$2ium$1 digitalmars.com...
 Lionello Lunesu wrote:
 Walter Bright wrote:
 David B. Held wrote:
 That being said, I think it would be *particularly* clever of you to 
 use your start-from-scratch kernel to explore new ideas in 
 multi-processing and massive multithreading, but this would imply 
 having decent multi-CPU hardware to play with.

collection - either making gc an operating system service, or providing hooks to the virtual memory subsystem so the gc can tell when pages are dirty.

block without EVER having to physically move the data. (That doesn't exist in Win32, does it?)

bounded by other data, I don't think it would be possible for it to grow without being moved.

I don't care if there address changes, just as long as there's no data being copied physically. I mean, a block of continuous memory might already be all-over-the-place in physical memory. That's what I want: a VirtualRealloc that returns the address of a new virtual memory block, but which is using the same physical memory.. 'Free' array resizes, surely this is worth something!

Hm, assuming there is nothing else of the page(s) but the data to move, it should be possible to do so without explicit copying. I have no idea is VirtualAlloc can do this, but it's definitely possible. Sean
Apr 20 2007
parent reply Dan <murpsoft hotmail.com> writes:
Sean Kelly Wrote:
 Lionello Lunesu wrote:
 I don't care if there address changes, just as long as there's no data being 
 copied physically. I mean, a block of continuous memory might already be 
 all-over-the-place in physical memory. That's what I want: a VirtualRealloc 
 that returns the address of a new virtual memory block, but which is using 
 the same physical memory.. 'Free' array resizes, surely this is worth 
 something!

Hm, assuming there is nothing else of the page(s) but the data to move, it should be possible to do so without explicit copying. I have no idea is VirtualAlloc can do this, but it's definitely possible.

Oh my! On the x86, this sort of implementation would slaughter your performance, as you'd get massive numbers of cache and page misses. You could theoretically implement a system on the x86 that would virtualize memory locations arbitrarily, but.... ewww... I see two effective ways to realloc memory in a space too small for it: 1) move the data you are trying to reallocate to a larger space. 2) move the data that is confining it; which requires updating pointers and such to that data, which you can't realistically track on the x86.
Apr 20 2007
next sibling parent reply Sean Kelly <sean f4.ca> writes:
Dan wrote:
 Sean Kelly Wrote:
 Lionello Lunesu wrote:
 I don't care if there address changes, just as long as there's no data being 
 copied physically. I mean, a block of continuous memory might already be 
 all-over-the-place in physical memory. That's what I want: a VirtualRealloc 
 that returns the address of a new virtual memory block, but which is using 
 the same physical memory.. 'Free' array resizes, surely this is worth 
 something!

it should be possible to do so without explicit copying. I have no idea is VirtualAlloc can do this, but it's definitely possible.

Oh my! On the x86, this sort of implementation would slaughter your performance, as you'd get massive numbers of cache and page misses. You could theoretically implement a system on the x86 that would virtualize memory locations arbitrarily, but.... ewww...

Yeah. I was thinking that the way to do this would be in the virtual memory manager. When the pages are moved the page table is updated, etc. Sean
Apr 20 2007
parent reply Dan <murpsoft hotmail.com> writes:
Sean Kelly Wrote:

 Dan wrote:
 Sean Kelly Wrote:
 Lionello Lunesu wrote:
 I don't care if there address changes, just as long as there's no data being 
 copied physically. I mean, a block of continuous memory might already be 
 all-over-the-place in physical memory. That's what I want: a VirtualRealloc 
 that returns the address of a new virtual memory block, but which is using 
 the same physical memory.. 'Free' array resizes, surely this is worth 
 something!

it should be possible to do so without explicit copying. I have no idea is VirtualAlloc can do this, but it's definitely possible.

Oh my! On the x86, this sort of implementation would slaughter your performance, as you'd get massive numbers of cache and page misses. You could theoretically implement a system on the x86 that would virtualize memory locations arbitrarily, but.... ewww...

Yeah. I was thinking that the way to do this would be in the virtual memory manager. When the pages are moved the page table is updated, etc. Sean

That works Sean, if your arrays are only ever split if they're bigger than a page; in which case letting the OS split them and 'handle it' is the best way. You *can* just declare a 64kb array and it'll handle it well enough. If you're talking about arbitrarily splitting arrays whenever they happen to collide, and hoping the paging system will handle it, you'll be grabbing 4kb segments for each misplaced segment. What's almost as bad, is that you'll have to have additional code tracking what segments belong to what virtual array.
Apr 20 2007
parent Sean Kelly <sean f4.ca> writes:
Dan wrote:
 Sean Kelly Wrote:
 
 Dan wrote:
 Sean Kelly Wrote:
 Lionello Lunesu wrote:
 I don't care if there address changes, just as long as there's no data being 
 copied physically. I mean, a block of continuous memory might already be 
 all-over-the-place in physical memory. That's what I want: a VirtualRealloc 
 that returns the address of a new virtual memory block, but which is using 
 the same physical memory.. 'Free' array resizes, surely this is worth 
 something!

it should be possible to do so without explicit copying. I have no idea is VirtualAlloc can do this, but it's definitely possible.

On the x86, this sort of implementation would slaughter your performance, as you'd get massive numbers of cache and page misses. You could theoretically implement a system on the x86 that would virtualize memory locations arbitrarily, but.... ewww...

memory manager. When the pages are moved the page table is updated, etc. Sean

That works Sean, if your arrays are only ever split if they're bigger than a page; in which case letting the OS split them and 'handle it' is the best way. You *can* just declare a 64kb array and it'll handle it well enough. If you're talking about arbitrarily splitting arrays whenever they happen to collide, and hoping the paging system will handle it, you'll be grabbing 4kb segments for each misplaced segment. What's almost as bad, is that you'll have to have additional code tracking what segments belong to what virtual array.

This is basically how the DMD garbage collector already works, so it could be done almost "for free" if there were VMM calls to handle it. Sean
Apr 21 2007
prev sibling parent reply "Lionello Lunesu" <lionello lunesu.remove.com> writes:
"Dan" <murpsoft hotmail.com> wrote in message 
news:f0b16i$2k5p$1 digitalmars.com...
 Sean Kelly Wrote:
 Lionello Lunesu wrote:
 I don't care if there address changes, just as long as there's no data 
 being
 copied physically. I mean, a block of continuous memory might already 
 be
 all-over-the-place in physical memory. That's what I want: a 
 VirtualRealloc
 that returns the address of a new virtual memory block, but which is 
 using
 the same physical memory.. 'Free' array resizes, surely this is worth
 something!

Hm, assuming there is nothing else of the page(s) but the data to move, it should be possible to do so without explicit copying. I have no idea is VirtualAlloc can do this, but it's definitely possible.

Oh my! On the x86, this sort of implementation would slaughter your performance, as you'd get massive numbers of cache and page misses.

Is a cache using virtual addresses or physical addresses?
 You could theoretically implement a system on the x86 that would 
 virtualize memory locations arbitrarily, but.... ewww...

Isn't this what's happening already? L.
Apr 21 2007
parent reply Dan <murpsoft hotmail.com> writes:
Lionello Lunesu Wrote:

 
 "Dan" <murpsoft hotmail.com> wrote in message 
 news:f0b16i$2k5p$1 digitalmars.com...
 Sean Kelly Wrote:
 Lionello Lunesu wrote:
 I don't care if there address changes, just as long as there's no data 
 being
 copied physically. I mean, a block of continuous memory might already 
 be
 all-over-the-place in physical memory. That's what I want: a 
 VirtualRealloc
 that returns the address of a new virtual memory block, but which is 
 using
 the same physical memory.. 'Free' array resizes, surely this is worth
 something!

Hm, assuming there is nothing else of the page(s) but the data to move, it should be possible to do so without explicit copying. I have no idea is VirtualAlloc can do this, but it's definitely possible.

Oh my! On the x86, this sort of implementation would slaughter your performance, as you'd get massive numbers of cache and page misses.

Is a cache using virtual addresses or physical addresses?

The cache loads in _blocks_ of physical memory, which can be specified using a virtual address or a physical address regardless; "virtual memory" just means you're using a translated address.
 You could theoretically implement a system on the x86 that would 
 virtualize memory locations arbitrarily, but.... ewww...

Isn't this what's happening already?

Not arbitrarily sized ones. If you have a buffer like so: [slice0,slice1,slice2] = virtual address table [minemineminemine][7Mb][minemineminemine][24b][minemine] = data buffer Iterate through your virtual array, and you're going to get a hit on the virtual address table, then on slice0, then on slice1 which will also load the 24b segment and probably slice2 for you. So you get 3 requests to memory, which if they're not already in the cache, will cause a cache miss. If they're not in memory (swap), they'll cause a page miss. If you force that memory to stay in one place, it'll all load in 1 shot instead of 3. If you're curious, a cache miss costs several dozen cycles; while a page miss will cost you thousands. Additionally, your arbitrary virtual address table takes up alot of space if someone has alot of small dynamic arrays. Worst case, your virtual table can be double the size of the data itself. Sincerely, Dan
Apr 21 2007
parent reply Dan <murpsoft hotmail.com> writes:
Dan Wrote:

 Lionello Lunesu Wrote:
 Isn't this what's happening already?

Not arbitrarily sized ones. If you have a buffer like so: [slice0,slice1,slice2] = virtual address table [minemineminemine][7Mb][minemineminemine][24b][minemine] = data buffer Iterate through your virtual array, and you're going to get a hit on the virtual address table, then on slice0, then on slice1 which will also load the 24b segment and probably slice2 for you.

On another note, this is done by the x86 system already for memory pages (4kb fixed block size). If you need a 32kb array, you can take 8 pages from anywhere in memory and call it yours, and iterate through the array and it'll translate the address for you in the hardware via the "virtual memory" process. : )
Apr 21 2007
parent reply "Lionello Lunesu" <lionello lunesu.remove.com> writes:
"Dan" <murpsoft hotmail.com> wrote in message 
news:f0di8h$q2e$1 digitalmars.com...
 Dan Wrote:

 Lionello Lunesu Wrote:
 Isn't this what's happening already?

Not arbitrarily sized ones. If you have a buffer like so: [slice0,slice1,slice2] = virtual address table [minemineminemine][7Mb][minemineminemine][24b][minemine] = data buffer Iterate through your virtual array, and you're going to get a hit on the virtual address table, then on slice0, then on slice1 which will also load the 24b segment and probably slice2 for you.

On another note, this is done by the x86 system already for memory pages (4kb fixed block size). If you need a 32kb array, you can take 8 pages from anywhere in memory and call it yours, and iterate through the array and it'll translate the address for you in the hardware via the "virtual memory" process.

Well, that was *exactly* what I was getting at! OK, granted, it's only done in 4KB increments, but that's when a regular array resize (copying the contents over to a new place) would actually start harming performance. L.
Apr 22 2007
parent 0ffh <spam frankhirsch.net> writes:
Lionello Lunesu wrote:
 Well, that was *exactly* what I was getting at! OK, granted, it's only done 
 in 4KB increments, but that's when a regular array resize (copying the 
 contents over to a new place) would actually start harming performance.

That sounds enticing! If you could put garbage collection into the kernel - even with tile-sized granularity - that would make many things easier. And finally put the idle thread to good use... :-))) Regards, Frank
Apr 22 2007
prev sibling next sibling parent reply Dan <murpsoft hotmail.com> writes:
Alexander Panek Wrote:
 Now those are some ideas and encouragement. Thanks! :)

Indeed. Lately a big thing in kernel development world is to move everything possible out of the kernel. Why? The kernel is supposed to provide one thing and one thing only: securely allowing multiple programs to run. To that end, the best kernel is the smallest one that is proveably secure and can efficiently use other programs to do hardware management stuff (read:drivers) So when I think of a kernel, I write an exokernel that controls the system by means of controlling the: 1) memory 2) ports 3) thread/process switcher attached to timer interrupt If you control memory, you control what agents can be loaded where and when. If an agent cannot get memory for a code block, they obviously cannot run it. Just my view. As I said, I'm interested in helping. Unfortunately, I've been pulling overtime the last few so I haven't had a chance to try getting on freenode.net again.
Apr 19 2007
parent reply 0ffh <spam frankhirsch.net> writes:
Dan wrote:
 Lately a big thing in kernel development world is to move everything possible
out of the kernel.

I don't know how lately should be defined in that context, but when I did the research for my diploma thesis (design and implementation of a RTOS-Kernel on a Fujitsu microcontroller) in 2001 there where already numerous papers to be found on the exokernel subject. It was a fascinating idea then, and still am. I sure like the approach.
 Just my view.  As I said, I'm interested in helping.

Maybe I'd also be able to throw in a few hours of work... :) Happy hacking, 0ffh
Apr 19 2007
parent Dan <murpsoft hotmail.com> writes:
0ffh Wrote:
 RTOS-Kernel on a Fujitsu microcontroller) in 2001 there where already
 numerous papers to be found on the exokernel subject.
 It was a fascinating idea then, and still....

Yeah, the MIT boys back then did a paper on it, and make Xok, which IMHO demonstrated the concept but otherwise sucked because they used the wrong strategies to 'import' code. When you have an exokernel, you need to be able to delegate gc, thread/process switching (cpu driver), paging and gc (memory driver), port drivers, dma and such out. Doing that efficiently IMHO requires using shared memory and priveleges rather than strictly RPC. I think I'll probably spend some time on this code this weekend; if I *ever* get a URL.
Apr 19 2007
prev sibling parent Sean Kelly <sean f4.ca> writes:
David B. Held wrote:
 
 That being said, I think it would be *particularly* clever of you to use 
 your start-from-scratch kernel to explore new ideas in multi-processing 
 and massive multithreading, but this would imply having decent multi-CPU 
 hardware to play with.  Most of the OS support we have is decades old 
 technology hacked together to keep up with the hardware, but maybe you 
 have an opportunity to do it right from the start.  That is, to create a 
 kernel where multiprocessing is as fundamental as memory management or 
 process scheduling.  We try to write lock-free algorithms using just one 
 special assembly instruction, and look how far we have to bend over 
 backwards to make it work.  If an MP-friendly kernel gave us more 
 primitives than CAS, because it was designed that way from the start 
 (meaning, the process scheduler was hacked up in a new way), I suspect 
 that you could make MP a lot easier on users.  Just some stuff to think 
 about...

Great idea :-) BeOS had very good multithreaded performance, but it didn't offer a new way to write multithreaded code. If such a beast existed, I'd definitely be interested in playing with it. Sean
Apr 19 2007
prev sibling next sibling parent Georg Wrede <georg nospam.org> writes:
Alexander Panek wrote:
 Walter Bright wrote:
 
 Alexander Panek wrote:

 I'm sure going to spend some time getting a proper codebase for further
 development done, and write some documentation about how to actually
 get to the point of being able to write an operating system with D. I
 think that's a major weak point of D, as it claims to be a systems
 language, but there's no actual system written in it from scratch,
 neither documentation on how to achieve that.

Andrei suggested that the source code for Minix, which is fairly small, could be transliterated from C almost directly into D. This would neatly resolve the issue, and provide a starting point for anyone wanting to take it further.

True. On the other hand, a kernel written purely in D is something that really would (and will, arr) be cool! I just find it less exciting to code "C with D", so to speak. I'll make sure to announce the project, once a few milestones are finished, on the newsgroups.

Linus T. started the same way. He had a Minix running on his machine, and piece by piece replaced stuff there, until no original code was left. (And then he /really/ started writing Linux.) We should do the same thing. Figuratively speaking. So, do the thing in Cish D, and only when it gets bootable, start rewriting it in Real D (tm).
Apr 25 2007
prev sibling parent reply Georg Wrede <georg nospam.org> writes:
Brad Roberts wrote:
 Alexander Panek wrote:
 
 Walter Bright wrote:

 Alexander Panek wrote:

 I'm sure going to spend some time getting a proper codebase for further
 development done, and write some documentation about how to actually
 get to the point of being able to write an operating system with D. I
 think that's a major weak point of D, as it claims to be a systems
 language, but there's no actual system written in it from scratch,
 neither documentation on how to achieve that.

Andrei suggested that the source code for Minix, which is fairly small, could be transliterated from C almost directly into D. This would neatly resolve the issue, and provide a starting point for anyone wanting to take it further.

True. On the other hand, a kernel written purely in D is something that really would (and will, arr) be cool! I just find it less exciting to code "C with D", so to speak. I'll make sure to announce the project, once a few milestones are finished, on the newsgroups.

Pardon the curmudgeon in me, but aside from being educational and being able to say 'see, someone's done it', what is to be achieved from inventing yet another kernel?

The advertising value is simply enormous. A lot of car makers (the smaller, the likelier) have always had some top models that give the brand prestige. Those cars were sold in very small numbers, and they made losses with them. But that was on purpose, because the prestige value sold much more "regular cars". I've seen it with motorcycles, cameras, watches, liquor, perfumes, and in service marketing! In the 80's many companies had doubts about Turbo Pascal. But when Borland demonstrated that it was not only possible but downright straightforward to write a TSR with it, the acceptance was huge. It became a Serious Programming Language. Oh, and speaking of cameras, somebody buying a camera wants it to have all the bells and whistles, even when they never end up using them. But they /have to be there/.
  Anything beyond a toy kernel is an _enormous_ effort.

We don't have to beat Linux. All we need is to have a valid counterargument when the [yourLanguageHere]weenies start saying D is not for systems work. Besides, the tinier the merrier. A small (e.g. Minix like) actually running OS (even if totally toy) is a good vehicle for trying out all sorts of things, without having to buy Annotated Linux Kernel Source Code ($200) or even worse, having to read it through only to test your little kernel heap algorithm. --- For the less-than-midle-aged of us: a TSR (Terminate and Say Resident) was a killer technology of the time. Microcomputers (as "PCs" were called, emphasizing their laughable lack of memory and horsepower -- as opposed to Minicomputers, Computers and Mainframes) were single-user single-tasking, which effectively meant that you can't start a calculator while you're in the text editor, or make notes or look at your calendar while in e-mail (yes, we had e-mail at the time, in Europe too, before the WWW, or even before the Internet, or Usenet became available). A TSR let you switch between two tasks, usually with some special key combination (like pressing both shifts or Ctrl-shift), and that was implemened with a wedge into the keyboard interrupt routine. Yes, you could have several TSRs running and switch between all of them -- if you were savvy. There was even a "multitasking" app, called Software Carousel, which let you run arbitrary programs while the others were suspended. But that came much later. But the notion of TSR was common with the CP/M operating system (that the world used until PC-DOS (and later MS-DOS) usurped them) way before Bill got into the OS-game.
Apr 25 2007
parent BCS <BCS pathlink.com> writes:
Georg Wrede wrote:
 Brad Roberts wrote:
 
  Anything beyond a toy kernel is an _enormous_ effort.

We don't have to beat Linux. All we need is to have a valid counterargument when the [yourLanguageHere]weenies start saying D is not for systems work. Besides, the tinier the merrier. A small (e.g. Minix like) actually running OS (even if totally toy) is a good vehicle for trying out all sorts of things, without having to buy Annotated Linux Kernel Source Code ($200) or even worse, having to read it through only to test your little kernel heap algorithm.

Besides, there is all sorts of fun to be had in the "really really small OS" department. I've been toying with the idea of how to architect a micro-kernel with only about 5 systems calls (remap memory, create process, kill process, switch to process, jump to on interrupt, fixed size IPC). If I wasn't already +3x booked for time, I would be figuring out how to wright it, in D.
Apr 25 2007
prev sibling next sibling parent Brad Roberts <braddr puremagic.com> writes:
Alexander Panek wrote:
 Walter Bright wrote:
 Alexander Panek wrote:
 I'm sure going to spend some time getting a proper codebase for further
 development done, and write some documentation about how to actually
 get to the point of being able to write an operating system with D. I
 think that's a major weak point of D, as it claims to be a systems
 language, but there's no actual system written in it from scratch,
 neither documentation on how to achieve that.

Andrei suggested that the source code for Minix, which is fairly small, could be transliterated from C almost directly into D. This would neatly resolve the issue, and provide a starting point for anyone wanting to take it further.

True. On the other hand, a kernel written purely in D is something that really would (and will, arr) be cool! I just find it less exciting to code "C with D", so to speak. I'll make sure to announce the project, once a few milestones are finished, on the newsgroups.

Pardon the curmudgeon in me, but aside from being educational and being able to say 'see, someone's done it', what is to be achieved from inventing yet another kernel? Anything beyond a toy kernel is an _enormous_ effort. I don't mean to say "don't do it". But I'm genuinely curious what the goals are. There's so many more things that would have a higher long term benefit to the development community as a whole that it feels like a waste unless it really is just a 'I want to learn more about kernels' sort of project. Later, Brad
Apr 17 2007
prev sibling parent reply Dan <murpsoft hotmail.com> writes:
I went to freenode and nobody was in the #D channel.  : (

Any chance someone could point me to the source?  I really derive my feelings
about a project by how sexy the source looks.
Apr 18 2007
next sibling parent Lars Ivar Igesund <larsivar igesund.net> writes:
Dan wrote:

 
 I went to freenode and nobody was in the #D channel.  : (
 
 Any chance someone could point me to the source?  I really derive my
 feelings about a project by how sexy the source looks.

You sure you went to the right place? There haven't been less than 60 there for a long time, 74 just now. I am logged on to irc.freenode.org:8001 -- Lars Ivar Igesund blog at http://larsivi.net DSource, #d.tango & #D: larsivi Dancing the Tango
Apr 18 2007
prev sibling parent Sean Kelly <sean f4.ca> writes:
Dan wrote:
 I went to freenode and nobody was in the #D channel.  : (

That's weird. There are normally like 40 people in there. Sean
Apr 18 2007
prev sibling next sibling parent reply 0ffh <spam frankhirsch.net> writes:
Davidl wrote:
 i would rather call it BrightOS

You can find a lot of Brights here: http://www.the-brights.net/ :-))))
Apr 13 2007
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

0ffh wrote:
 Davidl wrote:
 i would rather call it BrightOS

You can find a lot of Brights here: http://www.the-brights.net/ :-))))

Eh, never heard of those guys. My lineage is replete with nerds. Consider my grandfather's favorite picture of himself, with his telescope.
Apr 13 2007
parent reply Georg Wrede <georg nospam.org> writes:
Walter Bright wrote:
 0ffh wrote:
 Davidl wrote:

 i would rather call it BrightOS

You can find a lot of Brights here: http://www.the-brights.net/

Eh, never heard of those guys. My lineage is replete with nerds. Consider my grandfather's favorite picture of himself, with his telescope.

Well, consider yourself lucky! My grandfather's _non_-favorite picture was at a Baccarat table, in Wiesbaden or Monte Carlo (nobody seems to remember which), where he spent every holiday for 40 years, meticulously (and unwillingly) losing all of his fortune. When he died, it was the maid who owned even the furniture. A bit of a surprise to my Dad and his brother. And yes, my father is and my grandfather was an incurable nerd. Maybe at the casino tables the women pretended not to see this. My Dad bought himself an Osbourne-1 at 50, without even a hint of a serious reason or excuse for buying it. Twenty-five years ago this was unheard of, especially when you could buy a sub-compact car for the same price. Last year he gave it to me. Alongside with the Atari Portfolio, whic is now so seriously outdated it's unusable. But hey, it runs MS-DOS. Contrast that with my HP LX-95 pocket-size PC-compatible, a late 80's design (also Dad's surplus, since he bought the LX-200), which essentially was an entire 8080 IBM PC with 1MB mem and PC-cards as "hard drives". Even today it beats the pants off of smart phones, communicators and palms, when it comes to sheer raw utility. The calendar, the spreadsheet, the database and the word processor are just unbelievably intuitive while providing advanced features. Connect the thing to a modem, and you can edit your /etc/passwd like you were there. And the bundled RPN (+arith!) calculator let's you do off-the-hip math that nobody with an ordinary calculator could. And solving and graphs! Sometimes I feel that the world is not advancing. And computer keyboards are still laid out for the illiterate, with Caps Lock where Control should be. If we nerds were as aggressive as the neuro normals, we'd take over the world in two weeks. Oh well, let's just say I'm lucky I got born late enough to enjoy computers for most of my life!
Apr 13 2007
next sibling parent Don Clugston <dac nospam.com.au> writes:
Georg Wrede wrote:
 Sometimes I feel that the world is not advancing. And computer keyboards 
 are still laid out for the illiterate, with Caps Lock where Control 
 should be. If we nerds were as aggressive as the neuro normals, we'd 
 take over the world in two weeks.

The fact that QWERTY keyboards are overwhelmingly dominant shows that sometimes the worst possible technology can win. One of the major design rules was that you can type the word "typewriter" using only the top row of letters. (And basically the same layout is used in non-English speaking countries, where 'typewriter' isn't even a word). It's hard to imagine a worse design goal.
Apr 13 2007
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Georg Wrede wrote:
 Walter Bright wrote:
 0ffh wrote:
 Davidl wrote:

 i would rather call it BrightOS

You can find a lot of Brights here: http://www.the-brights.net/

Eh, never heard of those guys. My lineage is replete with nerds. Consider my grandfather's favorite picture of himself, with his telescope.

Well, consider yourself lucky!

I do. The only unlucky part is he died before I was born, I never got to meet him. I'd give a lot just to be able to spend an afternoon with him. When I was in high school, nerd was a perjorative term. But I've grown out of all that nonsense, and am happy to be what I am.
 My grandfather's _non_-favorite picture was at a Baccarat table, in 
 Wiesbaden or Monte Carlo (nobody seems to remember which), where he 
 spent every holiday for 40 years, meticulously (and unwillingly) losing 
 all of his fortune. When he died, it was the maid who owned even the 
 furniture. A bit of a surprise to my Dad and his brother.

Everyone has feet of clay.
 And yes, my father is and my grandfather was an incurable nerd. Maybe at 
 the casino tables the women pretended not to see this.
 My Dad bought 
 himself an Osbourne-1 at 50, without even a hint of a serious reason or 
 excuse for buying it. Twenty-five years ago this was unheard of, 
 especially when you could buy a sub-compact car for the same price. Last 
 year he gave it to me. Alongside with the Atari Portfolio, whic is now 
 so seriously outdated it's unusable. But hey, it runs MS-DOS.
 
 Contrast that with my HP LX-95 pocket-size PC-compatible, a late 80's 
 design (also Dad's surplus, since he bought the LX-200), which 
 essentially was an entire 8080 IBM PC with 1MB mem and PC-cards as "hard 
 drives". Even today it beats the pants off of smart phones, 
 communicators and palms, when it comes to sheer raw utility. The 
 calendar, the spreadsheet, the database and the word processor are just 
 unbelievably intuitive while providing advanced features. Connect the 
 thing to a modem, and you can edit your /etc/passwd like you were there. 
 And the bundled RPN (+arith!) calculator let's you do off-the-hip math 
 that nobody with an ordinary calculator could. And solving and graphs!

I still have my original IBM PC. With an 8080 chip!
 Sometimes I feel that the world is not advancing. And computer keyboards 
 are still laid out for the illiterate, with Caps Lock where Control 
 should be. If we nerds were as aggressive as the neuro normals, we'd 
 take over the world in two weeks.
 
 Oh well, let's just say I'm lucky I got born late enough to enjoy 
 computers for most of my life!

If I was born in the 1800's, I'd be happily immersed in designing locomotives. In the 1930's, I'd be building high performance aircraft. Every decade has its cool nerdly stuff to work on!
Apr 13 2007
next sibling parent reply "David B. Held" <dheld codelogicconsulting.com> writes:
Walter Bright wrote:
 [...]
 When I was in high school, nerd was a perjorative term. But I've grown 
 out of all that nonsense, and am happy to be what I am.
 [...]

When you were in high school, "nerds" weren't driving around in BMWs with millions in stock options... Dave
Apr 13 2007
parent Walter Bright <newshound1 digitalmars.com> writes:
David B. Held wrote:
 Walter Bright wrote:
 [...]
 When I was in high school, nerd was a perjorative term. But I've grown 
 out of all that nonsense, and am happy to be what I am.
 [...]

When you were in high school, "nerds" weren't driving around in BMWs with millions in stock options...

True, the high school loser nerd who becomes a billionaire stereotype didn't exist yet. But then again, my grandfather did all right.
Apr 14 2007
prev sibling parent 0ffh <spam frankhirsch.net> writes:
Walter Bright wrote:
 Georg Wrede wrote:
 I still have my original IBM PC. With an 8080 chip!

Sniff... never had one of those (yet my dad does), but I had an XT-Clone with a NEC V30 CPU going an amazing 8MHz - with "Turbo" key depressed naturally, as it used to break all hardcoded delay-loops... still before that I had a C64 (bought 1986). As did many others, I suppose. Regards, Frank
Apr 14 2007
prev sibling parent John Reimer <terminal.node gmail.com> writes:
On Tue, 17 Apr 2007 17:26:59 -0700, Walter Bright wrote:

 Alexander Panek wrote:
 I'm sure going to spend some time getting a proper codebase for further
 development done, and write some documentation about how to actually
 get to the point of being able to write an operating system with D. I
 think that's a major weak point of D, as it claims to be a systems
 language, but there's no actual system written in it from scratch,
 neither documentation on how to achieve that.

Andrei suggested that the source code for Minix, which is fairly small, could be transliterated from C almost directly into D. This would neatly resolve the issue, and provide a starting point for anyone wanting to take it further.

Minix3? That would be a great idea! I was thinking how translating Minix3 would be a good start for a D kernel. -JJR
Apr 17 2007
prev sibling next sibling parent Alexander Panek <alexander.panek brainsware.org> writes:
On Fri, 13 Apr 2007 05:31:49 +0000 (UTC)
John Reimer <terminal.node gmail.com> wrote:

 Yeah, true.  Regardless, I'd love to see a D-based OS someday. :-)

Just.. give me some time (and pay me, then it'll take less time, haha! :P). Eventually hakware (from #d) made interrupts working, and added quite some features upon my codebase - even a working garbage collector is present at kernel-level, IIRC. :D I'm sure going to spend some time getting a proper codebase for further development done, and write some documentation about how to actually get to the point of being able to write an operating system with D. I think that's a major weak point of D, as it claims to be a systems language, but there's no actual system written in it from scratch, neither documentation on how to achieve that. Kind regards, Alex
Apr 17 2007
prev sibling next sibling parent Alexander Panek <alexander.panek brainsware.org> writes:
On Tue, 17 Apr 2007 18:59:04 -0700
Brad Roberts <braddr puremagic.com> wrote:

 [...]

being able to say 'see, someone's done it', what is to be achieved from inventing yet another kernel? Anything beyond a toy kernel is an _enormous_ effort.

It's meant to be a research project, which I have started with C, anyways, out of personal interest. System design & development is my future occupation, hopefully, so this is educational at least for me. :)
 I don't mean to say "don't do it".  But I'm genuinely curious what
 the goals are.  There's so many more things that would have a higher
 long term benefit to the development community as a whole that it
 feels like a waste unless it really is just a 'I want to learn more
 about kernels' sort of project.

It might look like a "hackish" (yes, I call anything that's not properly designed and implemented - with pedantic attention for each detail - hackish..no offense intended, that just applies to my own code) kernel would has its benefits in the long-term, but I think a proper research to see what features of D can ease the pain of operating system development will gain more knowledge and will hopefully help me invent A Good Thing (tm). I'm more the innovation-at-all-cost guy.. so.. that might be the reason. Still, there's quite a broad code base already regarding the initialization of an x86 CPU, and according to hakware, interrupts are now working properly. This means, that things like memory management (paging, and such) is not far. I hope to bring in some benefits for D with the memory management, too, so it can colaborate with the GC. Don't know how far that's optimizable, but it's worth some research effort, imho. Anyways, it seems like quite a few people in the D community seem to be interested in kernel development. So if one project evolved enough to have overcome the beginning obstacles (mostly CPU weirdness-issues..), I bet it's possible to form a team with a bit more momentum. Best regards, Alex
Apr 18 2007
prev sibling next sibling parent Alexander Panek <alexander.panek brainsware.org> writes:
On Wed, 18 Apr 2007 04:06:40 +0000 (UTC)
John Reimer <terminal.node gmail.com> wrote:

 On Tue, 17 Apr 2007 17:26:59 -0700, Walter Bright wrote:
 
 Alexander Panek wrote:
 I'm sure going to spend some time getting a proper codebase for
 further development done, and write some documentation about how
 to actually get to the point of being able to write an operating
 system with D. I think that's a major weak point of D, as it
 claims to be a systems language, but there's no actual system
 written in it from scratch, neither documentation on how to
 achieve that.

Andrei suggested that the source code for Minix, which is fairly small, could be transliterated from C almost directly into D. This would neatly resolve the issue, and provide a starting point for anyone wanting to take it further.

Minix3? That would be a great idea! I was thinking how translating Minix3 would be a good start for a D kernel. -JJR

Well I think, when there's enough interest and a few people who'd like to help, we (read as: the D community) could start doing so, anyways. It'd at least help understanding the basic principles of operating system development. :)
Apr 18 2007
prev sibling next sibling parent Alexander Panek <alexander.panek brainsware.org> writes:
On Thu, 19 Apr 2007 01:07:22 -0700
Walter Bright <newshound1 digitalmars.com> wrote:

 David B. Held wrote:
 That being said, I think it would be *particularly* clever of you
 to use your start-from-scratch kernel to explore new ideas in
 multi-processing and massive multithreading, but this would imply
 having decent multi-CPU hardware to play with.

I'd also like to see better operating system support for garbage collection - either making gc an operating system service, or providing hooks to the virtual memory subsystem so the gc can tell when pages are dirty.

Well, something along the lines of that is what I was thinking of. I'm not so sure how to make the GC an operating system service, though..?
Apr 19 2007
prev sibling parent Alexander Panek <alexander.panek brainsware.org> writes:
On Wed, 18 Apr 2007 22:33:18 -0700
"David B. Held" <dheld codelogicconsulting.com> wrote:

 Alexander Panek wrote:
 [...]
 It might look like a "hackish" (yes, I call anything that's not
 properly designed and implemented - with pedantic attention for each
 detail - hackish..no offense intended, that just applies to my own
 code) kernel would has its benefits in the long-term, but I think a
 proper research to see what features of D can ease the pain of
 operating system development will gain more knowledge and will
 hopefully help me invent A Good Thing (tm).
 [...]

If you guys really are young college students and can afford to sink time into something like this, I say go for it 100%. It really doesn't matter whether it amounts to anything, because you will learn a ton from it and it will be a fun project. That's just the kind of thing to do at your age. At my age, you have to do things that matter (read: pay the bills), which is to say that you simply cannot afford to do something like write an OS kernel, and I really envy you. If you're really good, you can make a name for yourself with it, even if the kernel is a commercial failure, which is worth plenty to your career. That being said, I think it would be *particularly* clever of you to use your start-from-scratch kernel to explore new ideas in multi-processing and massive multithreading, but this would imply having decent multi-CPU hardware to play with. Most of the OS support we have is decades old technology hacked together to keep up with the hardware, but maybe you have an opportunity to do it right from the start. That is, to create a kernel where multiprocessing is as fundamental as memory management or process scheduling. We try to write lock-free algorithms using just one special assembly instruction, and look how far we have to bend over backwards to make it work. If an MP-friendly kernel gave us more primitives than CAS, because it was designed that way from the start (meaning, the process scheduler was hacked up in a new way), I suspect that you could make MP a lot easier on users. Just some stuff to think about... Dave

Now those are some ideas and encouragement. Thanks! :)
Apr 19 2007