www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - OpenMW, an open morrowind reimplementation

reply Nicolay Korslund <korslund gmail.com> writes:
Hi guys.

I'd like to announce a project I've been working on for some time - an open
source reimplementation of Morrowind. I've called it OpenMW, see
http://openmw.snaptoad.com

OpenMW is written almost completely in D, with some minor parts in C++
to interface with libraries. I'm using Ogre for graphics, and there's already
a discussion going (with screenshots) at
http://www.ogre3d.org/phpBB2/viewtopic.php?p=288886

My hope is that this project will make D more visible in the game dev
community, and prove that D has merit in that field.

One specific thing is that it connects D code with Ogre and other C++
libraries. Even though this is done in a way that is very specific to
openmw, it show that it can be done without much problem. (In fact it
isn't hard at all, the interface is not a complete low-level wrapper but
a small set of "medium-level" functions using extern(C). It will be even
easier with D2.0-style C++ linkage.)

For more info and shots, check http://openmw.snaptoad.com . Note
that it's still in a very early stage of development though.

Nico
Jun 06 2008
next sibling parent reply "Saaa" <empty needmail.com> writes:
Interesting! :)

Two questions:
Why not SDL?
How fast is monster?
Jun 06 2008
parent reply Nicolay Korslund <korslund gmail.com> writes:
Saaa Wrote:

 Interesting! :)
 
 Two questions:
 Why not SDL?

Do you mean for graphics? Or for input? Or for sound? There's three different reasons ;-)
 How fast is monster?

It's still under development, so it hasn't been optimized much yet (and I haven't really speed-tested it.) There are some other features I want to do first, but if someone wants to use monster for a project then I can probably start optimizing earlier. But here's no reason it can't become as fast as any other byte-code compiled language out there though (like Java, Lua, etc.) Nico
Jun 06 2008
parent reply "Saaa" <empty needmail.com> writes:
 Why not SDL?

Do you mean for graphics? Or for input? Or for sound? There's three different reasons ;-)

Would you give them all then? : )
 How fast is monster?

It's still under development, so it hasn't been optimized much yet (and I haven't really speed-tested it.) There are some other features I want to do first, but if someone wants to use monster for a project then I can probably start optimizing earlier. But here's no reason it can't become as fast as any other byte-code compiled language out there though (like Java, Lua, etc.)

First off, I do not know much about byte-code compilation but I'd reckon the optimizations in java are probably not something you can replicate on your own. When I think about it, its more important that the (user created) injected code (using the monster library) won't hurt the injected program, at least for use in games. How do you cope with this? (ie. infinite loop)
Jun 06 2008
parent reply Nicolay Korslund <korslund gmail.com> writes:
Saaa Wrote:

 Would you give them all then?  : )

Sure :) For graphics I'm using Ogre, which is a complete 3d engine, while SDL is a low-level graphics library. You can compile Ogre to use SDL as a backend, but by default it uses GLX on Linux and D3D on Windows, so I stick with the default. Sound is a similar story, SDL is a low-level library (afaik, I haven't used it that much), while Audiere is more high level. It's likely that I'll switch to OpenAL at some point, because it's more mature and supports hardware 3d audio, but it's not a high priority. And input: I originally used SDL for input back when Ogre used SDL as the default. But I had some problems with using SDL and GLX together, because SDL-input is integrated with the SDL video component. It worked, but not well. So instead I switched to OIS which is the "preferred" input method for Ogre apps. I really liked SDL better though. I heard some rumors that SDL might be decoupling the input component from the video in the future, and if that's true (and it works) I will switch back.
 First off, I do not know much about byte-code compilation but I'd reckon
 the optimizations in java are probably not something you can replicate on
 your own.

That's true. Promising Java-like performance is probably biting off a little too much, especially since Java has JIT. (There are open source JITs though, like LLVM.) On the other hand, there are languages like Lua which are considered fast enough for games, and which are completely open source as well (Lua uses the MIT-license.) What I'm trying to say is that I probably won't have to reinvent this wheel completely, only to modify an existing wheel to fit my type of carriage.
 When I think about it, its more important that the (user created) injected
 code
 (using the monster library) won't hurt the injected program, at least for 
 use in
 games. How do you cope with this? (ie. infinite loop)

That's a very good point, and it's one of the design goals of Monster. Scripts (even compiled scripts) should be safe to run no matter where they come from. That means not only handling infinite loops, infinite recursion, infinite memory allocations and other bugs, but also protecting against intentionally malicious code, ie. checking all input, protecting the file system, never exposing pointers, etc. Infinite loops and recursions are handled quite nicely already (currently it throws an exception, but later you may have the option of killing the offending object / function, etc.) Memory allocations will be handled in a similar fashion later. The rest is largely up to the library user - for example, if you expose a function that allows scripts to overwrite any file in the file system, then all the security measures in the world won't help you. I want to make tools and docs that makes this as easy as possible though. Nico
Jun 07 2008
parent reply "Saaa" <empty needmail.com> writes:
 Saaa Wrote:

 Would you give them all then?  : )

Sure :) For graphics I'm using Ogre, which is a complete 3d engine, while

 SDL is a low-level graphics library. You can compile Ogre to use SDL as
 a backend, but by default it uses GLX on Linux and D3D on Windows,
 so I stick with the default.

 Sound is a similar story, SDL is a low-level library (afaik, I haven't 
 used
 it that much), while Audiere is more high level. It's likely that I'll 
 switch
 to OpenAL at some point, because it's more mature and supports
 hardware 3d audio, but it's not a high priority.

 And input: I originally used SDL for input back when Ogre used SDL as
 the default. But I had some problems with using SDL and GLX together,
 because SDL-input is integrated with the SDL video component. It
 worked, but not well. So instead I switched to OIS which is the
 "preferred" input method for Ogre apps. I really liked SDL better
 though. I heard some rumors that SDL might be decoupling the input
 component from the video in the future, and if that's true (and it works)
 I will switch back.

 First off, I do not know much about byte-code compilation but I'd reckon
 the optimizations in java are probably not something you can replicate on
 your own.

That's true. Promising Java-like performance is probably biting off a little too much, especially since Java has JIT. (There are open source JITs though, like LLVM.) On the other hand, there are languages like Lua which are considered fast enough for games, and which are completely open source as well (Lua uses the MIT-license.) What I'm trying to say is that I probably won't have to reinvent this wheel completely, only to modify an existing wheel to fit my type of carriage.
 When I think about it, its more important that the (user created) 
 injected
 code
 (using the monster library) won't hurt the injected program, at least for
 use in
 games. How do you cope with this? (ie. infinite loop)

That's a very good point, and it's one of the design goals of Monster. Scripts (even compiled scripts) should be safe to run no matter where they come from. That means not only handling infinite loops, infinite recursion, infinite memory allocations and other bugs, but also protecting against intentionally malicious code, ie. checking all input, protecting the file system, never exposing pointers, etc. Infinite loops and recursions are handled quite nicely already (currently it

 throws an exception, but later you may have the option of killing the
 offending object / function, etc.) Memory allocations will be handled in a
 similar fashion later. The rest is largely up to the library user - for 
 example,
 if you expose a function that allows scripts to overwrite any file in the 
 file
 system, then all the security measures in the world won't help you. I want
 to make tools and docs that makes this as easy as possible though.

keyword is for, right :) I would like to be able to cap the size/cpu-time/memory usage.
Jun 07 2008
next sibling parent reply Extrawurst <spam extrawurst.org> writes:
Saaa schrieb:
 For graphics I'm using Ogre, which is a complete 3d engine, while
     


are no simple bindings to such an engine. as he exposed in his initial post you have to write a wrapper for such a thing so that you can expose these wrapper methods to D via extern(C). i am not sure how this has changed with D2.
 SDL is a low-level graphics library. You can compile Ogre to use SDL as
 a backend, but by default it uses GLX on Linux and D3D on Windows,
 so I stick with the default.
     


is responsible for window creation and D3D for graphic rendering. under SDL+OGL, SDL is the part creating window and some other stuff aswell. DirectX alone cannot exist, since it cannot create windows and those necessary things.
 I'm also interested in using OpenAL
   

SoundSystem build around OpenAL. For 3DSound it is not much more of work.
Jun 07 2008
parent reply "Saaa" <empty needmail.com> writes:
 I guess he means Ogre uses Win32-api+D3D under windows where Win32-api
 is responsible for window creation and D3D for graphic rendering. under 
 SDL+OGL, SDL is the part creating window and some other stuff aswell. 
 DirectX alone cannot exist, since it cannot create windows and those 
 necessary things.

Yeah, my bad, I thought some part of DX could create a window.
 I'm also interested in using OpenAL

SoundSystem build around OpenAL. For 3DSound it is not much more of work.

in comparison with using derelict and raw OpenAL?
Jun 07 2008
parent reply Extrawurst <spam extrawurst.org> writes:
Saaa schrieb:
 ArcLib looks interesting, what is the advantage of using it for sound only 
 in
 comparison with using derelict and raw OpenAL? 
   

You could of course also simply use the Sound package of ArcLib if it is sufficient for your needs.
Jun 07 2008
parent reply "Saaa" <empty needmail.com> writes:
 ArcLib looks interesting, what is the advantage of using it for sound 
 only in
 comparison with using derelict and raw OpenAL?

could of course also simply use the Sound package of ArcLib if it is sufficient for your needs.

I meant it more like: Does ArcLib offer stuff OpenAl doesn't?
Jun 08 2008
next sibling parent Extrawurst <spam extrawurst.org> writes:
Saaa schrieb:
 ArcLib looks interesting, what is the advantage of using it for sound 
 only in
 comparison with using derelict and raw OpenAL?
       

could of course also simply use the Sound package of ArcLib if it is sufficient for your needs.

I meant it more like: Does ArcLib offer stuff OpenAl doesn't?

offer anything more. Ogg-loading is an exception cause loading these is supported by ArcLib nativly. OpenAL nativly can only load from raw-bytes or wav-files.
Jun 08 2008
prev sibling parent reply Clay Smith <clayasaurus gmail.com> writes:
Saaa wrote:
 ArcLib looks interesting, what is the advantage of using it for sound 
 only in
 comparison with using derelict and raw OpenAL?

could of course also simply use the Sound package of ArcLib if it is sufficient for your needs.

I meant it more like: Does ArcLib offer stuff OpenAl doesn't?

ArcLib / Yage (where the base code of ArcLib openAL module came from) will allow you to get quickly set up using OpenAL. It's the difference between writing openal.sound.open(); SoundFX snd = new SoundFX("file.ogg / wav"); snd.play(); openal.sound.process(); versus writing 500 lines of code yourself to accomplish the same thing. You don't have to use ArcLib, just grab the zlib/png licensed openal module, modify it, and use it in your own project. ;)
Jun 09 2008
next sibling parent Lutger <lutger.blijdestin gmail.com> writes:
Clay Smith wrote:
...
 ArcLib / Yage (where the base code of ArcLib openAL module came from)
 will allow you to get quickly set up using OpenAL.

I agree, having to write 90% less code than OpenAl is something you can't do with OpenAl ;) Furthermore, you may get better integration with arclib if you use that and less chance of bugs.
Jun 10 2008
prev sibling parent "Saaa" <empty needmail.com> writes:
 ArcLib / Yage (where the base code of ArcLib openAL module came from)
 will allow you to get quickly set up using OpenAL. It's the difference 
 between writing

 openal.sound.open();

 SoundFX snd = new SoundFX("file.ogg / wav");
 snd.play();

 openal.sound.process();

 versus writing 500 lines of code yourself to accomplish the same thing.

 You don't have to use ArcLib, just grab the zlib/png licensed openal 
 module, modify it, and use it in your own project. ;)

Ok, I'll look into it : )
Jun 10 2008
prev sibling parent Nicolay Korslund <korslund gmail.com> writes:
Saaa Wrote:

 Did you just solve the halting problem  ; ) or is it timebased?

Since it's a VM, I can limit the number of instructions a function / object is allowed to execute per call. It's some ridiculously high number, but it typically stops an infinite loop in less than a second.
 I would like to be able to cap the size/cpu-time/memory usage.

I agree. This is something I will look into later. Nico
Jun 07 2008
prev sibling next sibling parent "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Nicolay Korslund" <korslund gmail.com> wrote in message 
news:g2aofv$2nl7$1 digitalmars.com...
 Hi guys.

 I'd like to announce a project I've been working on for some time - an 
 open
 source reimplementation of Morrowind. I've called it OpenMW, see
 http://openmw.snaptoad.com

 OpenMW is written almost completely in D, with some minor parts in C++
 to interface with libraries. I'm using Ogre for graphics, and there's 
 already
 a discussion going (with screenshots) at
 http://www.ogre3d.org/phpBB2/viewtopic.php?p=288886

 My hope is that this project will make D more visible in the game dev
 community, and prove that D has merit in that field.

 One specific thing is that it connects D code with Ogre and other C++
 libraries. Even though this is done in a way that is very specific to
 openmw, it show that it can be done without much problem. (In fact it
 isn't hard at all, the interface is not a complete low-level wrapper but
 a small set of "medium-level" functions using extern(C). It will be even
 easier with D2.0-style C++ linkage.)

 For more info and shots, check http://openmw.snaptoad.com . Note
 that it's still in a very early stage of development though.

Effing sweet!
Jun 06 2008
prev sibling next sibling parent reply Chris R. Miller <lordSaurontheGreat gmail.com> writes:
Nicolay Korslund Wrote:

 Hi guys.
 
 I'd like to announce a project I've been working on for some time - an open
 source reimplementation of Morrowind. I've called it OpenMW, see
 http://openmw.snaptoad.com
 
 OpenMW is written almost completely in D, with some minor parts in C++
 to interface with libraries. I'm using Ogre for graphics, and there's already
 a discussion going (with screenshots) at
 http://www.ogre3d.org/phpBB2/viewtopic.php?p=288886
 
 My hope is that this project will make D more visible in the game dev
 community, and prove that D has merit in that field.
 
 One specific thing is that it connects D code with Ogre and other C++
 libraries. Even though this is done in a way that is very specific to
 openmw, it show that it can be done without much problem. (In fact it
 isn't hard at all, the interface is not a complete low-level wrapper but
 a small set of "medium-level" functions using extern(C). It will be even
 easier with D2.0-style C++ linkage.)
 
 For more info and shots, check http://openmw.snaptoad.com . Note
 that it's still in a very early stage of development though.

Very very cool, though I do wonder about the legality of what you have just done. I seem to remember a large number of commercial EULAs being more or less Xerox copies of each other, and all of them very expressly disallow reverse engineering of any kind. I don't mean to be hostile, I'm just trying to know whether I can expect to see you in the news. I can see it now... "Electronic Entertainment Mega-corp Ubisoft sues Open-Source Programmer Nicolay Korslund for the reverse-engineering of their ancient and discontinued title "Elder Scrolls III: Morrowind," claiming that the act of writing a redundant game data file reader constitutes reverse engineering and is a violation of Intellectual Property. Representatives from Ubigroup were unable to comment at this time, though damages in the order of $500,000 were discussed if programmer Korslund refuses to completely destroy his work and renounce computer science for ever. Details at eleven." Perhaps not that dramatic, but you understand where I'm coming from. Then again, I live in America. Lawyers are a dime a dozen, so we get to put up with a lot of legal nonsense. I think it's made me more than a little paranoid.
Jun 06 2008
next sibling parent reply Nicolay Korslund <korslund gmail.com> writes:
Chris R. Miller Wrote:

 Very very cool, though I do wonder about the legality of what you have just
done.  I seem to remember a large number of commercial EULAs being more or less
Xerox copies of each other, and all of them very expressly disallow reverse
engineering of any kind.

Well, that's a very valid point. I'm not a lawyer so I can't definitely say anything about the legality of the software. Technically it is probably already in violation of the DMCA just by allowing access to the data without a copy protection scheme. That being said, openmw does not do anything that hasn't been done by other software for years already (I did very little of the reverse engineering myself), and there are similar projects (like ScummVM) that are going strong. Furthermore, I'm not based in the US. I honestly don't see how openmw could possibly contribute to piracy or hurt Bethesda Softworks in any way. In fact it's more likely to increase the shelf-life of one of their products, but that doesn't mean they will see it that way. But I have no reason to make enemies with anyone, and I will do what they ask me to, in the worst case take it down (but I hope it doesn't come to that.) One thing to be very careful about though is trademarks, since companies basically _have_ to protect them or else they risk losing them. That's why I've called it OpenMW, but I will even remove the MW if I have to. Nico
Jun 07 2008
parent reply Chris R. Miller <lordSaurontheGreat gmail.com> writes:
Nicolay Korslund Wrote:

 Chris R. Miller Wrote:
 
 Very very cool, though I do wonder about the legality of what you have just
done.  I seem to remember a large number of commercial EULAs being more or less
Xerox copies of each other, and all of them very expressly disallow reverse
engineering of any kind.

Well, that's a very valid point. I'm not a lawyer so I can't definitely say anything about the legality of the software. Technically it is probably already in violation of the DMCA just by allowing access to the data without a copy protection scheme. That being said, openmw does not do anything that hasn't been done by other software for years already (I did very little of the reverse engineering myself), and there are similar projects (like ScummVM) that are going strong. Furthermore, I'm not based in the US. I honestly don't see how openmw could possibly contribute to piracy or hurt Bethesda Softworks in any way. In fact it's more likely to increase the shelf-life of one of their products, but that doesn't mean they will see it that way. But I have no reason to make enemies with anyone, and I will do what they ask me to, in the worst case take it down (but I hope it doesn't come to that.) One thing to be very careful about though is trademarks, since companies basically _have_ to protect them or else they risk losing them. That's why I've called it OpenMW, but I will even remove the MW if I have to.

Cool. The presence of other engine replacements does make your work much less legally violate in my mind. I don't know if you already have some place to keep your work, but in the event that you'd like some place to stick your code you can use my site if you'd like (http://www.fsdev.net/). It has Subversion and some nice bug trackers and other gizmos that can be interesting to play with. Again, I don't know if you already have something, but if you want it you're welcome to register a project on my site.
Jun 09 2008
parent reply Nicolay Korslund <korslund gmail.com> writes:
 Chris R. Miller Wrote:

 I don't know if you already have some place to keep your work, but in the
event that you'd like some place to stick your code you can use my site if
you'd like (http://www.fsdev.net/).  It has Subversion and some nice bug
trackers and other gizmos that can be interesting to play with.  Again, I don't
know if you already have something, but if you want it you're welcome to
register a project on my site.
 

Thanks for the offer. But I've just recently signed up at sourceforge though, and I'm in the process of moving my files over there. Nico
Jun 14 2008
parent "Chris R. Miller" <lordSaurontheGreat gmail.com> writes:
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable

Nicolay Korslund wrote:
 Chris R. Miller Wrote:

 I don't know if you already have some place to keep your work, but in =


te if you'd like (http://www.fsdev.net/). It has Subversion and some nic= e bug trackers and other gizmos that can be interesting to play with. Ag= ain, I don't know if you already have something, but if you want it you'r= e welcome to register a project on my site.

Thanks for the offer. But I've just recently signed up at sourceforge t=

Excellent. <server_admin hat>They have better uptime than I do,=20 anyways.</server_admin hat>
Jun 14 2008
prev sibling parent reply "Koroskin Denis" <2korden gmail.com> writes:
On Sat, 07 Jun 2008 02:08:09 +0400, Chris R. Miller  
<lordSaurontheGreat gmail.com> wrote:

 Nicolay Korslund Wrote:

 Hi guys.

 I'd like to announce a project I've been working on for some time - an  
 open
 source reimplementation of Morrowind. I've called it OpenMW, see
 http://openmw.snaptoad.com

 OpenMW is written almost completely in D, with some minor parts in C++
 to interface with libraries. I'm using Ogre for graphics, and there's  
 already
 a discussion going (with screenshots) at
 http://www.ogre3d.org/phpBB2/viewtopic.php?p=288886

 My hope is that this project will make D more visible in the game dev
 community, and prove that D has merit in that field.

 One specific thing is that it connects D code with Ogre and other C++
 libraries. Even though this is done in a way that is very specific to
 openmw, it show that it can be done without much problem. (In fact it
 isn't hard at all, the interface is not a complete low-level wrapper but
 a small set of "medium-level" functions using extern(C). It will be even
 easier with D2.0-style C++ linkage.)

 For more info and shots, check http://openmw.snaptoad.com . Note
 that it's still in a very early stage of development though.

Very very cool, though I do wonder about the legality of what you have just done. I seem to remember a large number of commercial EULAs being more or less Xerox copies of each other, and all of them very expressly disallow reverse engineering of any kind. I don't mean to be hostile, I'm just trying to know whether I can expect to see you in the news. I can see it now... "Electronic Entertainment Mega-corp Ubisoft sues Open-Source Programmer Nicolay Korslund for the reverse-engineering of their ancient and discontinued title "Elder Scrolls III: Morrowind," claiming that the act of writing a redundant game data file reader constitutes reverse engineering and is a violation of Intellectual Property. Representatives from Ubigroup were unable to comment at this time, though damages in the order of $500,000 were discussed if programmer Korslund refuses to completely destroy his work and renounce computer science for ever. Details at eleven." Perhaps not that dramatic, but you understand where I'm coming from. Then again, I live in America. Lawyers are a dime a dozen, so we get to put up with a lot of legal nonsense. I think it's made me more than a little paranoid.

Well, it's not like that. There are tens of engine reimplementations out there[1]. Nothing is wrong if your game doesn't supply any copyrighted data, but those users who own the original game should be able to attach game data files to the engine and play game. Otherwise, it would be piracy. Complete remake is another story. I know of a few projects, that got Cease and Desist letter from copyright holders, and that forced the project to close. Good example is Chrono Trigger Resurrection[2], but AFAIK those guys were going to sell the game after release. Anyway, try to avoid usage of trademarked names (like "Morrowind" or "Elder Scroll"), in a title at the very least. ---- [1] Links: http://rewiki.regengedanken.de/wiki/Links_to_engine_reimplementations http://scummvm.org/compatibility.php http://en.wikipedia.org/wiki/Game_engine_recreations [2] http://www.opcoder.com/projects/chrono/
Jun 07 2008
parent Nicolay Korslund <korslund gmail.com> writes:
Thanks for the input. BTW I've added openmw to both the wiki lists you gave ;-)

Nico
Jun 07 2008
prev sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Nicolay Korslund wrote:
 Hi guys.
 
 I'd like to announce a project I've been working on for some time - an open
 source reimplementation of Morrowind. I've called it OpenMW, see
 http://openmw.snaptoad.com
 
 OpenMW is written almost completely in D, with some minor parts in C++
 to interface with libraries. I'm using Ogre for graphics, and there's already
 a discussion going (with screenshots) at
 http://www.ogre3d.org/phpBB2/viewtopic.php?p=288886
 
 My hope is that this project will make D more visible in the game dev
 community, and prove that D has merit in that field.
 
 One specific thing is that it connects D code with Ogre and other C++
 libraries. Even though this is done in a way that is very specific to
 openmw, it show that it can be done without much problem. (In fact it
 isn't hard at all, the interface is not a complete low-level wrapper but
 a small set of "medium-level" functions using extern(C). It will be even
 easier with D2.0-style C++ linkage.)
 
 For more info and shots, check http://openmw.snaptoad.com . Note
 that it's still in a very early stage of development though.
 
 Nico

Would you add an entry in http://www.prowiki.org/wiki4d/wiki.cgi?Games It seems quite worthy of mention :) -- Bruno Medeiros - Software Developer, MSc. in CS/E graduate http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Jun 13 2008
parent Nicolay Korslund <korslund gmail.com> writes:
Bruno Medeiros Wrote:
 Would you add an entry in http://www.prowiki.org/wiki4d/wiki.cgi?Games
 It seems quite worthy of mention :)

Good idea. I have added it now. Nico
Jun 14 2008