www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Crash handler with stack trace

reply Jeremie Pelletier <jeremiep gmail.com> writes:
I've been asked for my runtime crash handler with it's CodeView reader earlier
in digitalmars.D so here it is:

http://jump.fm/UVYHG

It includes the runtime handler, a PE reader, CodeView reader and crash report
window.

So far it supports Windows and some bits are already done for posix platforms.
It does a register dump, stack trace with resolved symbol names with their
declaration file and line, as well as a listing of all loaded modules.

Sorry for the lack of phobos/tango support, this is part of a runtime project
I'm building which should hopefully be released to dsource in a few months. I'm
releasing it now so if someone wants to implement a phobos or tango port
they're free to do it.
Sep 19 2009
next sibling parent reply Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
Jeremie Pelletier wrote:
 I've been asked for my runtime crash handler with it's CodeView reader earlier
in digitalmars.D so here it is:
 
 http://jump.fm/UVYHG
 
 It includes the runtime handler, a PE reader, CodeView reader and crash report
window.
 
 So far it supports Windows and some bits are already done for posix platforms.
It does a register dump, stack trace with resolved symbol names with their
declaration file and line, as well as a listing of all loaded modules.
 
 Sorry for the lack of phobos/tango support, this is part of a runtime project
I'm building which should hopefully be released to dsource in a few months. I'm
releasing it now so if someone wants to implement a phobos or tango port
they're free to do it.

Thanks a lot, Jeremie! The code looks really good. I think I'll be stealing the PE && CV parsing parts for Tango :) I'm curious about the runtime project of yours :o Can you shed a bit of the secret? How will it compare to druntime? What's the target audience? Does it have anything to do with 'meshes and textures'? ;) -- Tomasz Stachowiak http://h3.team0xf.com/ h3/h3r3tic on #D freenode
Sep 19 2009
next sibling parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Tom S Wrote:

 Jeremie Pelletier wrote:
 I've been asked for my runtime crash handler with it's CodeView reader earlier
in digitalmars.D so here it is:
 
 http://jump.fm/UVYHG
 
 It includes the runtime handler, a PE reader, CodeView reader and crash report
window.
 
 So far it supports Windows and some bits are already done for posix platforms.
It does a register dump, stack trace with resolved symbol names with their
declaration file and line, as well as a listing of all loaded modules.
 
 Sorry for the lack of phobos/tango support, this is part of a runtime project
I'm building which should hopefully be released to dsource in a few months. I'm
releasing it now so if someone wants to implement a phobos or tango port
they're free to do it.

Thanks a lot, Jeremie! The code looks really good. I think I'll be stealing the PE && CV parsing parts for Tango :)

I'm glad to hear that! Just keep my name in the credits and I'll be happy :)
 I'm curious about the runtime project of yours :o Can you shed a bit of 
 the secret? How will it compare to druntime? What's the target audience? 
 Does it have anything to do with 'meshes and textures'? ;)

Well it started a bit over two years ago, back then there was no druntime, I was digging into phobos to learn more about the language and decided it would be a fun and challenging task to write my own runtime. Then once I had it running stable I found out druntime had been created in the meantime! In any ways, I learned quite a bit of the language's internals and ABI while coding and debugging it; I began by removing all default paths and libraries from dmd (both in sc.ini and compiling with "--defaultlib= --debuglib=--") and made an empty object.d and built my way up until there was no more compiler crashes (dmd is quite whiny about not having its "intrinsic" symbols declared, mostly the typeinfo's) and all unresolved symbols were added. Since then the runtime was ported to D2 and kept up to date with every new release, I've been using it for these two years and it works great. I don't really know how it will compare to druntime, I'm not out there to compete with them really. I use DMD on both windows and posix platforms (Win7 and Ubuntu respectively) so its the only supported compiler right now, haven't tested 64-bit support either. It would be able to support libraries such as phobos and tango with a few changes to the places that call into the compiler runtime or memory manager. Right now the target audience is me. I haven't really considered who the target audience might be since I touch a bit of everything. I try and make it as generic and as modular as I can, so if someone doesn't need a package they can remove it from their build without getting tons of compile errors. I myself am focusing towards GUI application development with unified 2D/3D rendering and custom controls on top of that rendering as part of the runtime library, among many other things. So you could say it has to do with meshes and textures, although I haven't gotten to making interfaces for that part yet. The most I did on that level with my runtime was to write a reader for Blizzard's MPQ archives and extract model and texture data to render it in opengl, but apart from the MPQ reader it was all pretty ugly mixing win32 and gl code along with other stuff in the same routine to get the model on screen as fast as possible ;) I really am just beginning to notice people are interested in my project, I don't know when I'll put it on dsource and/or sourceforge. And given that D2 is still under heavy development it wouldn't be practical right now. I don't mind releasing bits and pieces of it but I would much rather see the first public release of it to be an usable one, although the compiler implementation part is pretty much stable at this point, if there's enough demand for it I might begin with that. Jeremie
Sep 19 2009
next sibling parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Vladimir Panteleev Wrote:

 On Sun, 20 Sep 2009 08:25:30 +0300, Jeremie Pelletier <jeremiep gmail.com>  
 wrote:
 
 I'm curious about the runtime project of yours :o Can you shed a bit of
 the secret? How will it compare to druntime? What's the target audience?
 Does it have anything to do with 'meshes and textures'? ;)

[snip]

This is cool stuff. I'd be very interested in finding out how you handled memory management / garbage collection. Or have you used the standard GC that comes with Phobos/Tango?

I forked the original D1 gc and rewrote it almost entirely. I made numerous speed improvements over it, such as greatly reducing the number of pool lookups, smarter alloc/realloc algorithms, removed the 'freebits' and 'scan' bitfields from pools and rewrote the garbage collector to only use the 'mark' one, I also merged the sweep and freelist rebuild phases together to save on the overhead of yet another loop over the entire working set, also added support for the gc to free memory pools if they're entirely free. I have it on pastebin if you want to take a look: http://pastebin.com/f7a3b4c4a Jeremie
Sep 20 2009
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Jeremie Pelletier:

I forked the original D1 gc and rewrote it almost entirely. I made numerous
speed improvements over it, such as greatly reducing the number of pool
lookups, smarter alloc/realloc algorithms, removed the 'freebits' and 'scan'
bitfields from pools and rewrote the garbage collector to only use the 'mark'
one, I also merged the sweep and freelist rebuild phases together to save on
the overhead of yet another loop over the entire working set, also added
support for the gc to free memory pools if they're entirely free.<

That looks like lot of work. Too bad we don't have a suite of GC-heavy benchmarks to test it. (Maybe Lucarella has collected something in the meantime). Bye, bearophile
Sep 20 2009
parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
bearophile wrote:
 Jeremie Pelletier:
 
 I forked the original D1 gc and rewrote it almost entirely. I made numerous
speed improvements over it, such as greatly reducing the number of pool
lookups, smarter alloc/realloc algorithms, removed the 'freebits' and 'scan'
bitfields from pools and rewrote the garbage collector to only use the 'mark'
one, I also merged the sweep and freelist rebuild phases together to save on
the overhead of yet another loop over the entire working set, also added
support for the gc to free memory pools if they're entirely free.<

That looks like lot of work. Too bad we don't have a suite of GC-heavy benchmarks to test it. (Maybe Lucarella has collected something in the meantime). Bye, bearophile

I couldn't agree more about the need for a GC benchmark suite, writing this module involved some of the hardest debugging work I ever did, even with printfs, memory dumps and windbg. I do believe it to be stable enough for general usage in its current state. Its been some time now since I last had any issues with it. If you come across such a benchmark suite, let me know, I would love to see how it performs against druntime's GC. Jeremie
Sep 20 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Jeremie Pelletier:

 I couldn't agree more about the need for a GC benchmark suite,

I am slowly creating one (a general one) that later may be used for LDC :-)
 If you come across such a benchmark suite, let me know, I would love to 
 see how it performs against druntime's GC.

Similar GC benchmark suites don't grow on trees. I don't think one exists yet for D (ask to Lucarella too). You can start testing your runtime against programs that you can find in the "binarytrees" and "gc" subdirectories here: http://www.fantascienza.net/leonardo/js/slow_d.zip And I'd like to know the results with both runtimes :-) Bye, bearophile
Sep 20 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Leandro Lucarella:

 I was unable
 to find any real programs (suitable for a GC benchmarks at least) for
 D that are properly maintained except Dil,

Have you tried Blaze? Bye, bearophile
Sep 22 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Leandro Lucarella:

 Blaze is an engine AFAIK, not a program. And an engine to build games
 AFAIK, and game aren't usually a good candidate for benchmarking since
 they can't be automated.

Blaze is a program that simulates 2D physics written in D. http://www.dsource.org/projects/blaze It's an engine, but it comes with an extensive demo that is and can be used as benchmark. So I think it can be used for your purposes (but I don't know how many memory allocations it performs). Bye, bearophile
Sep 23 2009
parent Jeremie Pelletier <jeremiep gmail.com> writes:
Denis Koroskin wrote:
 On Wed, 23 Sep 2009 20:03:47 +0400, Jarrett Billingsley 
 <jarrett.billingsley gmail.com> wrote:
 
 On Wed, Sep 23, 2009 at 11:58 AM, bearophile 
 <bearophileHUGS lycos.com> wrote:
 Leandro Lucarella:

 Blaze is an engine AFAIK, not a program. And an engine to build games
 AFAIK, and game aren't usually a good candidate for benchmarking since
 they can't be automated.

Blaze is a program that simulates 2D physics written in D. http://www.dsource.org/projects/blaze It's an engine, but it comes with an extensive demo that is and can be used as benchmark. So I think it can be used for your purposes (but I don't know how many memory allocations it performs).

It performs *way too fucking many*. ;) Which, I suppose, would make it a good GC benchmark.

A game engine that does *any* allocations per frame? My employee would kill me for writing something like that!

You have no idea how many professional game companies write sloppy code, they're not all over-optimizing code zealots like Carmack :) EA is notoriously good for that, but they do spend 3x as much on marketing as they do on development so go figure. Their games run way slower at medium settings than games by, say Blizzard, would at high settings.
Sep 23 2009
prev sibling parent reply Kagamin <spam here.lot> writes:
Jeremie Pelletier Wrote:

 I forked the original D1 gc and rewrote it almost entirely.

Did you try to address bug 2093?
Sep 25 2009
parent Jeremie Pelletier <jeremiep gmail.com> writes:
Kagamin wrote:
 Jeremie Pelletier Wrote:
 
 I forked the original D1 gc and rewrote it almost entirely.

Did you try to address bug 2093?

I just did a quick compile of the bug's test case and it worked correctly. I wasn't even aware of that bug to tell the truth.
Sep 25 2009
prev sibling parent Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
Jeremie Pelletier wrote:
 Tom S Wrote:
 
 Jeremie Pelletier wrote:
 I've been asked for my runtime crash handler with it's CodeView reader earlier
in digitalmars.D so here it is:

 http://jump.fm/UVYHG

 It includes the runtime handler, a PE reader, CodeView reader and crash report
window.

 So far it supports Windows and some bits are already done for posix platforms.
It does a register dump, stack trace with resolved symbol names with their
declaration file and line, as well as a listing of all loaded modules.

 Sorry for the lack of phobos/tango support, this is part of a runtime project
I'm building which should hopefully be released to dsource in a few months. I'm
releasing it now so if someone wants to implement a phobos or tango port
they're free to do it.

stealing the PE && CV parsing parts for Tango :)

I'm glad to hear that! Just keep my name in the credits and I'll be happy :)

Sure thing, I generally stick to the same philosophy :)
 I'm curious about the runtime project of yours :o Can you shed a bit of 
 the secret? How will it compare to druntime? What's the target audience? 
 Does it have anything to do with 'meshes and textures'? ;)

Well it started a bit over two years ago, back then there was no druntime, I was digging into phobos to learn more about the language and decided it would be a fun and challenging task to write my own runtime. Then once I had it running stable I found out druntime had been created in the meantime! In any ways, I learned quite a bit of the language's internals and ABI while coding and debugging it; I began by removing all default paths and libraries from dmd (both in sc.ini and compiling with "--defaultlib= --debuglib=--") and made an empty object.d and built my way up until there was no more compiler crashes (dmd is quite whiny about not having its "intrinsic" symbols declared, mostly the typeinfo's) and all unresolved symbols were added. Since then the runtime was ported to D2 and kept up to date with every new release, I've been using it for these two years and it works great. I don't really know how it will compare to druntime, I'm not out there to compete with them really. I use DMD on both windows and posix platforms (Win7 and Ubuntu respectively) so its the only supported compiler right now, haven't tested 64-bit support either. It would be able to support libraries such as phobos and tango with a few changes to the places that call into the compiler runtime or memory manager.

Thanks for the in-depth background story! Looks like you went a similar route that folks trying to do kernels in D choose. Perhaps if you had mentioned the development of this runtime earlier, druntime could be a joint project... But maybe it's not too late and druntime could still benefit from your expertise? There are scarcely few guys who understand the guts of writing D runtimes - I could probably count them all on the fingers of one palm ;) And I don't necessarily mean the concrete implementations but also the top-level structure.
 Right now the target audience is me. I haven't really considered who the
target audience might be since I touch a bit of everything. I try and make it
as generic and as modular as I can, so if someone doesn't need a package they
can remove it from their build without getting tons of compile errors. I myself
am focusing towards GUI application development with unified 2D/3D rendering
and custom controls on top of that rendering as part of the runtime library,
among many other things.

So it's floor wax *and* dessert topping? Sounds like you're set on doing _everything_ from scratch :o I don't think nowadays I'd be willing to live without Tango's streams...
 So you could say it has to do with meshes and textures, although I haven't
gotten to making interfaces for that part yet. The most I did on that level
with my runtime was to write a reader for Blizzard's MPQ archives and extract
model and texture data to render it in opengl, but apart from the MPQ reader it
was all pretty ugly mixing win32 and gl code along with other stuff in the same
routine to get the model on screen as fast as possible ;)

Haha, well that's a noble goal. I'll have to get myself some readers for third-party formats as well. In retrospect, writing a custom 3ds max exporter and supporting only it was a bad idea :P
 I really am just beginning to notice people are interested in my project, I
don't know when I'll put it on dsource and/or sourceforge. And given that D2 is
still under heavy development it wouldn't be practical right now. I don't mind
releasing bits and pieces of it but I would much rather see the first public
release of it to be an usable one, although the compiler implementation part is
pretty much stable at this point, if there's enough demand for it I might begin
with that.

I'm sure people would become interested way earlier, had you mentioned the project anywhere :P I'm still in D1 land, but there are folks trying to use D2 and craving for some code to look into and build upon. The more code we release to the general public, the bigger D feels like and the greater chances it has of attracting new users... at least that's what I like to believe :P -- Tomasz Stachowiak http://h3.team0xf.com/ h3/h3r3tic on #D freenode
Sep 20 2009
prev sibling parent reply "Vladimir Panteleev" <thecybershadow gmail.com> writes:
On Sun, 20 Sep 2009 08:25:30 +0300, Jeremie Pelletier <jeremiep gmail.com>  
wrote:

 I'm curious about the runtime project of yours :o Can you shed a bit of
 the secret? How will it compare to druntime? What's the target audience?
 Does it have anything to do with 'meshes and textures'? ;)

[snip]

This is cool stuff. I'd be very interested in finding out how you handled memory management / garbage collection. Or have you used the standard GC that comes with Phobos/Tango? -- Best regards, Vladimir mailto:thecybershadow gmail.com
Sep 20 2009
parent Leandro Lucarella <llucax gmail.com> writes:
Jarrett Billingsley, el 23 de septiembre a las 12:03 me escribiste:
 On Wed, Sep 23, 2009 at 11:58 AM, bearophile <bearophileHUGS lycos.com> wrote:
 Leandro Lucarella:

 Blaze is an engine AFAIK, not a program. And an engine to build games
 AFAIK, and game aren't usually a good candidate for benchmarking since
 they can't be automated.

Blaze is a program that simulates 2D physics written in D. http://www.dsource.org/projects/blaze It's an engine, but it comes with an extensive demo that is and can be used as benchmark. So I think it can be used for your purposes (but I don't know how many memory allocations it performs).

It performs *way too fucking many*. ;) Which, I suppose, would make it a good GC benchmark.

I'll take a look at the demos then, thanks for the tips. -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ ---------------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------------- Aprendan de la primavera, que antecede al verano, precede al inverno y no lo anda diciendo por ahí. -- Ricardo Vaporeso. Llanos de Luzuriaga, 1914.
Sep 23 2009
prev sibling next sibling parent Leandro Lucarella <llucax gmail.com> writes:
bearophile, el 20 de septiembre a las 22:31 me escribiste:
 Jeremie Pelletier:
 
 I couldn't agree more about the need for a GC benchmark suite,

I am slowly creating one (a general one) that later may be used for LDC :-)
 If you come across such a benchmark suite, let me know, I would love
 to see how it performs against druntime's GC.

Similar GC benchmark suites don't grow on trees. I don't think one exists yet for D (ask to Lucarella too).

Unfortunately I had not much success at making a benchmark suite. I think the main reason is the... "diversity" you find in D. You have D1/Phobos, D1/Tango and D2/Druntime. Doing a benchmark suite that works with all is very hard, you have to "port" the benchmark to 3 almost different languages. And any decent GC benchmark test *real* programs. I was unable to find any real programs (suitable for a GC benchmarks at least) for D that are properly maintained except Dil, which is the only program I plan to use in my benchmarks (I had a lot of problems lately with a bug in my Naive collector that I'm still trying to find). Picking Dil means to stay in the D1 Tango world, because I don't have the time to port Dil to D1/Phobos nor D2/Druntime, so there it goes the "cross-D-version" benchmark. I only have that, and a few micro-benchmarks I collected in the 3 or 4 years I've been lurking this newsgroup (there are very few though). I'll have to make at least a few more micro-benchmarks for multi-threaded applications, since my plan is to add concurrency to the GC, so I'll have to measure that scenario. You can find my benchmark sources in this git repo (is not very interesting for now, really): http://git.llucax.com.ar/w/software/dgc/dgcbench.git In my blog are a few (meaningless but nice) graphs for the Naive GC =) http://proj.llucax.com.ar/blog/dgc/blog I'll posting the results of the tests using the Tango basic collector soon. -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ ---------------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------------- El día que falten los niños, que sobren las mujeres y que se prenda fuego el último árbol, será el Apocalípsis. -- Ricardo Vaporeso. Camino Negro, 1916.
Sep 21 2009
prev sibling next sibling parent Leandro Lucarella <llucax gmail.com> writes:
bearophile, el 22 de septiembre a las 07:28 me escribiste:
 Leandro Lucarella:
 
 I was unable
 to find any real programs (suitable for a GC benchmarks at least) for
 D that are properly maintained except Dil,

Have you tried Blaze?

Blaze is an engine AFAIK, not a program. And an engine to build games AFAIK, and game aren't usually a good candidate for benchmarking since they can't be automated. -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ ---------------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------------- VECINOS RESCATARON A CABALLITO ATROPELLADO -- Crónica TV
Sep 23 2009
prev sibling next sibling parent Jarrett Billingsley <jarrett.billingsley gmail.com> writes:
On Wed, Sep 23, 2009 at 11:58 AM, bearophile <bearophileHUGS lycos.com> wrote:
 Leandro Lucarella:

 Blaze is an engine AFAIK, not a program. And an engine to build games
 AFAIK, and game aren't usually a good candidate for benchmarking since
 they can't be automated.

Blaze is a program that simulates 2D physics written in D. http://www.dsource.org/projects/blaze It's an engine, but it comes with an extensive demo that is and can be used as benchmark. So I think it can be used for your purposes (but I don't know how many memory allocations it performs).

It performs *way too fucking many*. ;) Which, I suppose, would make it a good GC benchmark.
Sep 23 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 23 Sep 2009 20:03:47 +0400, Jarrett Billingsley  
<jarrett.billingsley gmail.com> wrote:

 On Wed, Sep 23, 2009 at 11:58 AM, bearophile <bearophileHUGS lycos.com>  
 wrote:
 Leandro Lucarella:

 Blaze is an engine AFAIK, not a program. And an engine to build games
 AFAIK, and game aren't usually a good candidate for benchmarking since
 they can't be automated.

Blaze is a program that simulates 2D physics written in D. http://www.dsource.org/projects/blaze It's an engine, but it comes with an extensive demo that is and can be used as benchmark. So I think it can be used for your purposes (but I don't know how many memory allocations it performs).

It performs *way too fucking many*. ;) Which, I suppose, would make it a good GC benchmark.

A game engine that does *any* allocations per frame? My employee would kill me for writing something like that!
Sep 23 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 23 Sep 2009 20:03:47 +0400, Jarrett Billingsley  
<jarrett.billingsley gmail.com> wrote:

 On Wed, Sep 23, 2009 at 11:58 AM, bearophile <bearophileHUGS lycos.com>  
 wrote:
 Leandro Lucarella:

 Blaze is an engine AFAIK, not a program. And an engine to build games
 AFAIK, and game aren't usually a good candidate for benchmarking since
 they can't be automated.

Blaze is a program that simulates 2D physics written in D. http://www.dsource.org/projects/blaze It's an engine, but it comes with an extensive demo that is and can be used as benchmark. So I think it can be used for your purposes (but I don't know how many memory allocations it performs).

It performs *way too fucking many*. ;) Which, I suppose, would make it a good GC benchmark.

A game engine that does *any* allocations per frame? My employer would kill me for writing something like that!
Sep 23 2009
prev sibling parent reply grauzone <none example.net> writes:
Jeremie Pelletier wrote:
 I've been asked for my runtime crash handler with it's CodeView reader earlier
in digitalmars.D so here it is:
 
 http://jump.fm/UVYHG

Wanted to take a look at it again, but the site is defunct.
Sep 27 2009
parent Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
grauzone wrote:
 Jeremie Pelletier wrote:
 I've been asked for my runtime crash handler with it's CodeView reader 
 earlier in digitalmars.D so here it is:

 http://jump.fm/UVYHG

Wanted to take a look at it again, but the site is defunct.

http://www.mediafire.com/download.php?wy2m2hrgzdo -- Tomasz Stachowiak http://h3.team0xf.com/ h3/h3r3tic on #D freenode
Sep 27 2009