www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DMD 2.055 Crashing on Windows 7 x64

reply Mehrdad <wfunction hotmail.com> writes:
DMD 2.055 seems to be crashing on my computer (Windows 7 x64).

When I press Debug, I see it's doing so in CloseHandle, probably because 
I've turned on "Enable close exception" with Visual Studio's "Global 
Flags" tool. (It takes a reboot to take effect.)

So this means there's an incorrect handle being used somewhere... but 
interestingly, by the time it crashes, the output file is already 
produced. I don't have the time to make a debug build right now, but I 
thought I'd mention this so that it could be looked into at some point.
Sep 26 2011
next sibling parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 7:55 AM, Mehrdad wrote:
 DMD 2.055 seems to be crashing on my computer (Windows 7 x64).

 When I press Debug, I see it's doing so in CloseHandle, probably 
 because I've turned on "Enable close exception" with Visual Studio's 
 "Global Flags" tool. (It takes a reboot to take effect.)

 So this means there's an incorrect handle being used somewhere... but 
 interestingly, by the time it crashes, the output file is already 
 produced. I don't have the time to make a debug build right now, but I 
 thought I'd mention this so that it could be looked into at some point.

too. Is there a bug in snn.lib or something?
Sep 26 2011
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/26/2011 8:06 AM, Mehrdad wrote:
 Actually, I just found out that the program itself crashes the same way, too.

What program?
Sep 26 2011
parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 12:19 PM, Steven Schveighoffer wrote:
 On Mon, 26 Sep 2011 15:05:44 -0400, Walter Bright 
 <newshound2 digitalmars.com> wrote:

 On 9/26/2011 8:06 AM, Mehrdad wrote:
 Actually, I just found out that the program itself crashes the same 
 way, too.

What program?

I think he means both dmd and the program it generates crash in the same way. -Steve

It seems to be failing in the _close() method, but ONLY when I run it from my text editor, SciTE. It runs fine from the command prompt. I don't believe it's a problem with the text editor (since it's just a pipe, and since it works fine with other languages) but I'll keep looking. Does anyone happen to know if there's a stray _close() method in snn.lib that closes a handle that might be already closed?
Sep 26 2011
next sibling parent Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:
 AHA!

 Yes, there is a bug in snn.lib regarding pipes.  And I fixed it, 
 waiting for Walter to incorporate it :)  I needed it for the new 
 std.process.

 What it comes down to is, DMC's FILE * implementation does not expect 
 some of the quirks of pipe HANDLEs.  For instance, if you open a FILE 
 * around a pipe handle, it still tries to do a seek on that handle, 
 and crashes.  Also, when the write end of a pipe is closed, reading 
 from the read end results in EPIPE from ReadFile, but this is 
 translated to EBADF by the runtime.  Therefore, FILE * sets an error 
 instead of EOF.

 Is the email address you have for this message correct?  If so, I can 
 send you a new version of snn.lib to try linking your code against (if 
 you are willing to go through these steps), to see if it fixes your 
 problem.

 Using the command line dmd to build should be sufficient (I think).

 -Steve

Yup, feel free to send it to this one. It's not my main one but I do check it so I'll get it. :) Thanks!
Sep 26 2011
prev sibling parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:
 AHA!

 Yes, there is a bug in snn.lib regarding pipes.  And I fixed it, 
 waiting for Walter to incorporate it :)  I needed it for the new 
 std.process.

 What it comes down to is, DMC's FILE * implementation does not expect 
 some of the quirks of pipe HANDLEs.  For instance, if you open a FILE 
 * around a pipe handle, it still tries to do a seek on that handle, 
 and crashes.  Also, when the write end of a pipe is closed, reading 
 from the read end results in EPIPE from ReadFile, but this is 
 translated to EBADF by the runtime.  Therefore, FILE * sets an error 
 instead of EOF.

 Is the email address you have for this message correct?  If so, I can 
 send you a new version of snn.lib to try linking your code against (if 
 you are willing to go through these steps), to see if it fixes your 
 problem.

 Using the command line dmd to build should be sufficient (I think).

 -Steve

It seems like it still crashes from SciTE, though. :( When I try debugging, I see it's doing so inside _close() which is /immediately/ above kernel32.dll! BaseThreadInitThunk 12() -- in other words, it seems to be called directly from the entrypoint of the program, assuming it hasn't undergone optimizations. The error is: 0xC0000008: An invalid handle was specified. Is there any other place in which this can happen?
Sep 26 2011
next sibling parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 2:30 PM, Mehrdad wrote:
 On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:
 AHA!

 Yes, there is a bug in snn.lib regarding pipes.  And I fixed it, 
 waiting for Walter to incorporate it :)  I needed it for the new 
 std.process.

 What it comes down to is, DMC's FILE * implementation does not expect 
 some of the quirks of pipe HANDLEs.  For instance, if you open a FILE 
 * around a pipe handle, it still tries to do a seek on that handle, 
 and crashes.  Also, when the write end of a pipe is closed, reading 
 from the read end results in EPIPE from ReadFile, but this is 
 translated to EBADF by the runtime.  Therefore, FILE * sets an error 
 instead of EOF.

 Is the email address you have for this message correct?  If so, I can 
 send you a new version of snn.lib to try linking your code against 
 (if you are willing to go through these steps), to see if it fixes 
 your problem.

 Using the command line dmd to build should be sufficient (I think).

 -Steve

It seems like it still crashes from SciTE, though. :( When I try debugging, I see it's doing so inside _close() which is /immediately/ above kernel32.dll! BaseThreadInitThunk 12() -- in other words, it seems to be called directly from the entrypoint of the program, assuming it hasn't undergone optimizations. The error is: 0xC0000008: An invalid handle was specified. Is there any other place in which this can happen?

silly, but why not just use msvcrt.dll (if not the whole library, at least the I/O functions)? It's not like it introduces a new dependency (it's already in every version of Windows) and it's also proven to work. Not to mention that it reduces code size as well...
Sep 26 2011
next sibling parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 2:53 PM, Steven Schveighoffer wrote:
 I think the reason Walter always gives is that dmd would then have to 
 output COFF objects, whereas it currently outputs OMF.  OPTLINK also 
 does not accept COFF objects.

of msvcrt.lib, like we're doing for all the other Windows libraries? I'm confused as to how this is an issue.
 I think a lot of people would very much welcome this change, but it's 
 somewhat of an underwhelming change.  Instead of using DMC's runtime 
 for C functions, you are using VC++'s runtime.  But who cares?  I want 
 D's runtime to be better :)

Microsoft's C runtime that's packaged with Windows (which Microsoft ironically doesn't want you to use, but which compilers like GCC use anyway... and it turns out better than anything bundled with Visual Studio). You don't actually need Visual Studio to use it (in fact, it's not even included) -- the Windows Driver Kit has the bundled stuff required. In all honesty I don't see a single thing about it that I would call "better". Do you have any examples of things that snn.lib does 'better' than msvcrt.lib? The only incompatibility (which isn't "better" or "worse"... it's just an incompatibility) I can see is exception handling, but that's obviously an exception that can be left in snn.lib. Almost all the rest of the runtime, though, can be easily removed and redirected to msvcrt.dll... and I don't think there'd be any problems. (Are there?)
 I think the best path is weaning ourselves off of C dependencies.  
 Starting with FILE *, which has a lot of limitations.  It's part of 
 the reason I'm working on reworking std.stdio.
 For example, with my version of std.stdio, you would never have this 
 problem, because it deals with the HANDLE directly instead of through 
 DMC's file descriptor layer.

is all regarding what's /already/ taking place in snn.lib /before/ D runs, which is obviously unrelated to what's going on in std.stdio. I'm really curious to know what specific problems are keeping us from doing the switch (i.e. what snn.lib does better than msvcrt.dll). :)
Sep 26 2011
parent Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 3:38 PM, Steven Schveighoffer wrote:
 On Mon, 26 Sep 2011 18:10:05 -0400, Mehrdad <wfunction hotmail.com> 
 wrote:

 On 9/26/2011 2:53 PM, Steven Schveighoffer wrote:
 I think the reason Walter always gives is that dmd would then have 
 to output COFF objects, whereas it currently outputs OMF.  OPTLINK 
 also does not accept COFF objects.

version of msvcrt.lib, like we're doing for all the other Windows libraries? I'm confused as to how this is an issue.

I don't know, maybe there's another reason.
 I think a lot of people would very much welcome this change, but 
 it's somewhat of an underwhelming change.  Instead of using DMC's 
 runtime for C functions, you are using VC++'s runtime.  But who 
 cares?  I want D's runtime to be better :)

Microsoft's C runtime that's packaged with Windows (which Microsoft ironically doesn't want you to use, but which compilers like GCC use anyway... and it turns out better than anything bundled with Visual Studio). You don't actually need Visual Studio to use it (in fact, it's not even included) -- the Windows Driver Kit has the bundled stuff required.

It's the same thing. MSVCRT.dll is included with every windows. Just the version may be different from the runtime bundled with Visual Studio. That's why you can install the visual studio redistributable package. The lib file (which contains the linking information) is not included (with Windows). Neither is the statically linked version. But make no mistake, it's the Visual C++ runtime (MicroSoftVisualCRunTime).
 In all honesty I don't see a single thing about it that I would call 
 "better". Do you have any examples of things that snn.lib does 
 'better' than msvcrt.lib?

I think it may conform to the standard better. But I don't think any C runtime is better than what we could make in D :)
 The only incompatibility (which isn't "better" or "worse"... it's 
 just an incompatibility) I can see is exception handling, but that's 
 obviously an exception that can be left in snn.lib. Almost all the 
 rest of the runtime, though, can be easily removed and redirected to 
 msvcrt.dll... and I don't think there'd be any problems. (Are there?)

I don't know. For certain, there are some pieces of phobos written specifically knowing the internals of Digital Mars' runtime (like std.stdio).
 I think the best path is weaning ourselves off of C dependencies.  
 Starting with FILE *, which has a lot of limitations.  It's part of 
 the reason I'm working on reworking std.stdio.
 For example, with my version of std.stdio, you would never have this 
 problem, because it deals with the HANDLE directly instead of 
 through DMC's file descriptor layer.

This is all regarding what's /already/ taking place in snn.lib /before/ D runs, which is obviously unrelated to what's going on in std.stdio. I'm really curious to know what specific problems are keeping us from doing the switch (i.e. what snn.lib does better than msvcrt.dll). :)

I'm not really qualified to answer that. It seems like the right move to me, so there must be good reasons compiler-wise. -Steve

I hope this isn't too huge of a request (Walter?), but would it be possible to PLEASE release an object or library file which would be helpful for implementing the use of msvcrt.lib? Right now, most crucial thing I need is an object file which defines this symbol correctly (since it seems to be providing an anchor for the rest runtime and the GC): phobos.lib(memory) Error 42: Symbol Undefined __xi_a Having something that also implements the following would also be awesome, but I can't tell if it's as crucial as the above until I have the above and try it out first: Test.obj(Test) Error 42: Symbol Undefined __except_list phobos.lib(deh) Error 42: Symbol Undefined __tls_array phobos.lib(deh) Error 42: Symbol Undefined __tls_index phobos.lib(thread) Error 42: Symbol Undefined __tlsend phobos.lib(thread) Error 42: Symbol Undefined __tlsstart phobos.lib(memory) Error 42: Symbol Undefined __end If I could _only_ have these, then I could very well try to see if I can integrate the C runtime with msvcrt.dll, which I think many people would appreciate. I'm pretty sure that the definition of __xi_a itself must be incredibly small (maybe 2 lines of code total?) so I hope it's not too much trouble to release something about it... I have no idea how to define it myself. If a working definition was released then it'd be pretty awesome. Thank you!
Sep 26 2011
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, September 26, 2011 14:53 Steven Schveighoffer wrote:
 On Mon, 26 Sep 2011 17:35:20 -0400, Mehrdad <wfunction hotmail.com> wrote:
 On 9/26/2011 2:30 PM, Mehrdad wrote:
 On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:
 AHA!
 
 Yes, there is a bug in snn.lib regarding pipes. And I fixed it,
 waiting for Walter to incorporate it :) I needed it for the new
 std.process.
 
 What it comes down to is, DMC's FILE * implementation does not expect
 some of the quirks of pipe HANDLEs. For instance, if you open a FILE
 * around a pipe handle, it still tries to do a seek on that handle,
 and crashes. Also, when the write end of a pipe is closed, reading
 from the read end results in EPIPE from ReadFile, but this is
 translated to EBADF by the runtime. Therefore, FILE * sets an error
 instead of EOF.
 
 Is the email address you have for this message correct? If so, I can
 send you a new version of snn.lib to try linking your code against (if
 you are willing to go through these steps), to see if it fixes your
 problem.
 
 Using the command line dmd to build should be sufficient (I think).
 
 -Steve

Thanks for the email. It seems like it still crashes from SciTE, though. :( When I try debugging, I see it's doing so inside _close() which is /immediately/ above kernel32.dll! BaseThreadInitThunk 12() -- in other words, it seems to be called directly from the entrypoint of the program, assuming it hasn't undergone optimizations. The error is: 0xC0000008: An invalid handle was specified. Is there any other place in which this can happen?

Also, now that we're on the topic... this might or might not sound silly, but why not just use msvcrt.dll (if not the whole library, at least the I/O functions)? It's not like it introduces a new dependency (it's already in every version of Windows) and it's also proven to work. Not to mention that it reduces code size as well...

I think the reason Walter always gives is that dmd would then have to output COFF objects, whereas it currently outputs OMF. OPTLINK also does not accept COFF objects. I think a lot of people would very much welcome this change, but it's somewhat of an underwhelming change. Instead of using DMC's runtime for C functions, you are using VC++'s runtime. But who cares? I want D's runtime to be better :)

The gain in switching to COFF would be _huge_, because then D code would be properly interoperable with most of the C code on Windows - and in particular C code compiled with Microsoft's compiler. The only thing saving the current situation from being a complete disaster is that you can still use dlls. - Jonathan M Davis
Sep 26 2011
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 27.09.2011 2:33, Jonathan M Davis wrote:
 On Monday, September 26, 2011 14:53 Steven Schveighoffer wrote:
 On Mon, 26 Sep 2011 17:35:20 -0400, Mehrdad<wfunction hotmail.com>  wrote:
 On 9/26/2011 2:30 PM, Mehrdad wrote:
 On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:
 AHA!

 Yes, there is a bug in snn.lib regarding pipes. And I fixed it,
 waiting for Walter to incorporate it :) I needed it for the new
 std.process.

 What it comes down to is, DMC's FILE * implementation does not expect
 some of the quirks of pipe HANDLEs. For instance, if you open a FILE
 * around a pipe handle, it still tries to do a seek on that handle,
 and crashes. Also, when the write end of a pipe is closed, reading
 from the read end results in EPIPE from ReadFile, but this is
 translated to EBADF by the runtime. Therefore, FILE * sets an error
 instead of EOF.

 Is the email address you have for this message correct? If so, I can
 send you a new version of snn.lib to try linking your code against (if
 you are willing to go through these steps), to see if it fixes your
 problem.

 Using the command line dmd to build should be sufficient (I think).

 -Steve

Thanks for the email. It seems like it still crashes from SciTE, though. :( When I try debugging, I see it's doing so inside _close() which is /immediately/ above kernel32.dll! BaseThreadInitThunk 12() -- in other words, it seems to be called directly from the entrypoint of the program, assuming it hasn't undergone optimizations. The error is: 0xC0000008: An invalid handle was specified. Is there any other place in which this can happen?

Also, now that we're on the topic... this might or might not sound silly, but why not just use msvcrt.dll (if not the whole library, at least the I/O functions)? It's not like it introduces a new dependency (it's already in every version of Windows) and it's also proven to work. Not to mention that it reduces code size as well...

I think the reason Walter always gives is that dmd would then have to output COFF objects, whereas it currently outputs OMF. OPTLINK also does not accept COFF objects. I think a lot of people would very much welcome this change, but it's somewhat of an underwhelming change. Instead of using DMC's runtime for C functions, you are using VC++'s runtime. But who cares? I want D's runtime to be better :)

The gain in switching to COFF would be _huge_, because then D code would be properly interoperable with most of the C code on Windows - and in particular C code compiled with Microsoft's compiler. The only thing saving the current situation from being a complete disaster is that you can still use dlls.

It's not that easy. I mean to gain this level of compatibility we'd have to do both: switch to MS runtime and use COFF object format. Linking together two C runtimes is *ehm* tricky. But most importantly newer versions of VC++ use newer runtimes, I'm not sure how compatible they are, but judging by my previous experience - you still has to keep things separated like in DLLs and never mix allocation/FILE* stuff between them.
 - Jonathan M Davis

-- Dmitry Olshansky
Sep 26 2011
prev sibling parent Kagamin <spam here.lot> writes:
Mehrdad Wrote:

 Also, now that we're on the topic... this might or might not sound 
 silly, but why not just use msvcrt.dll (if not the whole library, at 
 least the I/O functions)? It's not like it introduces a new dependency 
 (it's already in every version of Windows) and it's also proven to work. 
 Not to mention that it reduces code size as well...

gdc already does it.
Sep 27 2011
prev sibling parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 2:36 PM, Steven Schveighoffer wrote:
 Hm... its hard for me to say.  Why would it be calling _close before 
 main() is called?  Very strange.  Is there any more stack information?

 Can you get it to crash by using a pipe from the command line?

 In other words:

 myprogram | more

 or echo hi | miprogram

 ?

 If this isn't killing the program in the same way, then it might be a 
 different issue than the one I fixed.

 -Steve

Nope, pipes/redirections from the command prompt are all fine -- it just crashes when run from SciTE (and also Notepad++, which I just verified). After somewhat painful debugging, it seems like the stack trace is a little off -- it actually happens when _exit() is called, which in turn calls _fcloseall() or something. Not sure what else I can do about it, though -- it's pretty clear the handle was destroyed sometime before, but I don't know when.
Sep 26 2011
next sibling parent Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 3:07 PM, Andrej Mitrovic wrote:
 SciTE (and probably N++ since it's based on scintilla) has a subsystem
 setting which can affect how it invokes a process. See the "Command
 subsystem" section here: http://www.scintilla.org/SciTEDoc.html

 Usually I run with subsystem 0.

(1) Scintilla is just the text editor. It doesn't have anything to do with the integrated command line. (2) I tried both 0 and 1, and neither made a difference. In fact, while using 1 DID pop up an extra command-line window, in either case the I/O took place inside SciTE itself. And in both cases, both DMD and my program crashed. You should be able to easily reproduce this if you set the "Enable close exception" and "Enable bad handles detection" Global Flags, using the Global Flags utility that comes with Debugging Tools for Windows. (Just make sure you reboot before you expect to see the change.)
Sep 26 2011
prev sibling next sibling parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 2:58 PM, Steven Schveighoffer wrote:
 This is likely a DMC issue, and is probably best reported as a DMC 
 bug.  I don't know how much more you want to pursue this, but the next 
 steps I'd recommend are:

 1. obtain the dmc compiler (it's free) if you haven't already.
 2. Compile an empty C program and run it using SciTE and Notepad++.  
 Verify the same error occurs
 3. report the failure using DMC's bugzilla.

 I'm guessing there's something in the way SciTE or Notepad++ sets up 
 the pipes before executing the process which causes the problem.  I 
 have a sinking feeling we'll see more of this issue when the new 
 std.process is released :(

 -Steve

used handle in snn.lib), because I don't see how it could affect my D program this way, if it was a DMC issue. And yeah, I just tested it with a simple hello world C example using DMC as well, and got the same error. Pretty darn sure now that the handle is getting closed twice or something...
Sep 26 2011
parent Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 3:37 PM, Steven Schveighoffer wrote:
 On Mon, 26 Sep 2011 18:17:49 -0400, Mehrdad <wfunction hotmail.com> 
 wrote:

 On 9/26/2011 2:58 PM, Steven Schveighoffer wrote:
 This is likely a DMC issue, and is probably best reported as a DMC 
 bug.  I don't know how much more you want to pursue this, but the 
 next steps I'd recommend are:

 1. obtain the dmc compiler (it's free) if you haven't already.
 2. Compile an empty C program and run it using SciTE and Notepad++.  
 Verify the same error occurs
 3. report the failure using DMC's bugzilla.

 I'm guessing there's something in the way SciTE or Notepad++ sets up 
 the pipes before executing the process which causes the problem.  I 
 have a sinking feeling we'll see more of this issue when the new 
 std.process is released :(

 -Steve

improperly used handle in snn.lib), because I don't see how it could affect my D program this way, if it was a DMC issue. And yeah, I just tested it with a simple hello world C example using DMC as well, and got the same error. Pretty darn sure now that the handle is getting closed twice or something...

My apologies, I meant DMC runtime issue (whose library is called snn.lib). At least with this test, we can rule out D being the culprit. However, we still have to deal with this problem... -Steve

DMC-related as well. :)
Sep 26 2011
prev sibling parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 2:58 PM, Steven Schveighoffer wrote:
 This is likely a DMC issue, and is probably best reported as a DMC 
 bug.  I don't know how much more you want to pursue this, but the next 
 steps I'd recommend are:

 1. obtain the dmc compiler (it's free) if you haven't already.
 2. Compile an empty C program and run it using SciTE and Notepad++.  
 Verify the same error occurs
 3. report the failure using DMC's bugzilla.

 I'm guessing there's something in the way SciTE or Notepad++ sets up 
 the pipes before executing the process which causes the problem.  I 
 have a sinking feeling we'll see more of this issue when the new 
 std.process is released :(

 -Steve

originally: A handle is being closed that isn't actually valid. Which handle? stderr. stderr is being closed when in fact it is the same handle as stdout. So when _fcloseallp() is called, it goes through the structures in __iob and closes each one in turn -- which causes a duplicate close operation on stderr, because it already gets closed when stdout gets closed. So it's an snn.lib issue. Is it easy to fix? (I hope it is...) Thanks!
Sep 26 2011
parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 3:41 PM, Steven Schveighoffer wrote:
 Probably -- let the OS close the handles instead of manually closing 
 them :)

 -Steve

internal buffering of the data, which would be lost if the stream isn't closed.
Sep 26 2011
parent Rainer Schuetze <r.sagitario gmx.de> writes:
I think, http://d.puremagic.com/issues/show_bug.cgi?id=1550 is very much 
related.

IIRC, the handle for stdin, stdout and stderr are retreived by 
GetStdHandle(), which should not be closed at all. Definitely should be 
fixed in snn.lib.


On 27.09.2011 01:15, Mehrdad wrote:
 On 9/26/2011 3:41 PM, Steven Schveighoffer wrote:
 Probably -- let the OS close the handles instead of manually closing
 them :)

 -Steve

internal buffering of the data, which would be lost if the stream isn't closed.

Sep 27 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 15:05:44 -0400, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 9/26/2011 8:06 AM, Mehrdad wrote:
 Actually, I just found out that the program itself crashes the same  
 way, too.

What program?

I think he means both dmd and the program it generates crash in the same way. -Steve
Sep 26 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 16:38:10 -0400, Mehrdad <wfunction hotmail.com> wrote:

 On 9/26/2011 12:19 PM, Steven Schveighoffer wrote:
 On Mon, 26 Sep 2011 15:05:44 -0400, Walter Bright  
 <newshound2 digitalmars.com> wrote:

 On 9/26/2011 8:06 AM, Mehrdad wrote:
 Actually, I just found out that the program itself crashes the same  
 way, too.

What program?

I think he means both dmd and the program it generates crash in the same way. -Steve

It seems to be failing in the _close() method, but ONLY when I run it from my text editor, SciTE. It runs fine from the command prompt. I don't believe it's a problem with the text editor (since it's just a pipe, and since it works fine with other languages) but I'll keep looking. Does anyone happen to know if there's a stray _close() method in snn.lib that closes a handle that might be already closed?

AHA! Yes, there is a bug in snn.lib regarding pipes. And I fixed it, waiting for Walter to incorporate it :) I needed it for the new std.process. What it comes down to is, DMC's FILE * implementation does not expect some of the quirks of pipe HANDLEs. For instance, if you open a FILE * around a pipe handle, it still tries to do a seek on that handle, and crashes. Also, when the write end of a pipe is closed, reading from the read end results in EPIPE from ReadFile, but this is translated to EBADF by the runtime. Therefore, FILE * sets an error instead of EOF. Is the email address you have for this message correct? If so, I can send you a new version of snn.lib to try linking your code against (if you are willing to go through these steps), to see if it fixes your problem. Using the command line dmd to build should be sufficient (I think). -Steve
Sep 26 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 17:30:42 -0400, Mehrdad <wfunction hotmail.com> wrote:

 On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:
 AHA!

 Yes, there is a bug in snn.lib regarding pipes.  And I fixed it,  
 waiting for Walter to incorporate it :)  I needed it for the new  
 std.process.

 What it comes down to is, DMC's FILE * implementation does not expect  
 some of the quirks of pipe HANDLEs.  For instance, if you open a FILE *  
 around a pipe handle, it still tries to do a seek on that handle, and  
 crashes.  Also, when the write end of a pipe is closed, reading from  
 the read end results in EPIPE from ReadFile, but this is translated to  
 EBADF by the runtime.  Therefore, FILE * sets an error instead of EOF.

 Is the email address you have for this message correct?  If so, I can  
 send you a new version of snn.lib to try linking your code against (if  
 you are willing to go through these steps), to see if it fixes your  
 problem.

 Using the command line dmd to build should be sufficient (I think).

 -Steve

It seems like it still crashes from SciTE, though. :( When I try debugging, I see it's doing so inside _close() which is /immediately/ above kernel32.dll! BaseThreadInitThunk 12() -- in other words, it seems to be called directly from the entrypoint of the program, assuming it hasn't undergone optimizations. The error is: 0xC0000008: An invalid handle was specified. Is there any other place in which this can happen?

Hm... its hard for me to say. Why would it be calling _close before main() is called? Very strange. Is there any more stack information? Can you get it to crash by using a pipe from the command line? In other words: myprogram | more or echo hi | miprogram ? If this isn't killing the program in the same way, then it might be a different issue than the one I fixed. -Steve
Sep 26 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 17:35:20 -0400, Mehrdad <wfunction hotmail.com> wrote:

 On 9/26/2011 2:30 PM, Mehrdad wrote:
 On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:
 AHA!

 Yes, there is a bug in snn.lib regarding pipes.  And I fixed it,  
 waiting for Walter to incorporate it :)  I needed it for the new  
 std.process.

 What it comes down to is, DMC's FILE * implementation does not expect  
 some of the quirks of pipe HANDLEs.  For instance, if you open a FILE  
 * around a pipe handle, it still tries to do a seek on that handle,  
 and crashes.  Also, when the write end of a pipe is closed, reading  
 from the read end results in EPIPE from ReadFile, but this is  
 translated to EBADF by the runtime.  Therefore, FILE * sets an error  
 instead of EOF.

 Is the email address you have for this message correct?  If so, I can  
 send you a new version of snn.lib to try linking your code against (if  
 you are willing to go through these steps), to see if it fixes your  
 problem.

 Using the command line dmd to build should be sufficient (I think).

 -Steve

It seems like it still crashes from SciTE, though. :( When I try debugging, I see it's doing so inside _close() which is /immediately/ above kernel32.dll! BaseThreadInitThunk 12() -- in other words, it seems to be called directly from the entrypoint of the program, assuming it hasn't undergone optimizations. The error is: 0xC0000008: An invalid handle was specified. Is there any other place in which this can happen?

silly, but why not just use msvcrt.dll (if not the whole library, at least the I/O functions)? It's not like it introduces a new dependency (it's already in every version of Windows) and it's also proven to work. Not to mention that it reduces code size as well...

I think the reason Walter always gives is that dmd would then have to output COFF objects, whereas it currently outputs OMF. OPTLINK also does not accept COFF objects. I think a lot of people would very much welcome this change, but it's somewhat of an underwhelming change. Instead of using DMC's runtime for C functions, you are using VC++'s runtime. But who cares? I want D's runtime to be better :) I think the best path is weaning ourselves off of C dependencies. Starting with FILE *, which has a lot of limitations. It's part of the reason I'm working on reworking std.stdio. For example, with my version of std.stdio, you would never have this problem, because it deals with the HANDLE directly instead of through DMC's file descriptor layer. -Steve
Sep 26 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 17:52:02 -0400, Mehrdad <wfunction hotmail.com> wrote:

 On 9/26/2011 2:36 PM, Steven Schveighoffer wrote:
 Hm... its hard for me to say.  Why would it be calling _close before  
 main() is called?  Very strange.  Is there any more stack information?

 Can you get it to crash by using a pipe from the command line?

 In other words:

 myprogram | more

 or echo hi | miprogram

 ?

 If this isn't killing the program in the same way, then it might be a  
 different issue than the one I fixed.

 -Steve

Nope, pipes/redirections from the command prompt are all fine -- it just crashes when run from SciTE (and also Notepad++, which I just verified). After somewhat painful debugging, it seems like the stack trace is a little off -- it actually happens when _exit() is called, which in turn calls _fcloseall() or something. Not sure what else I can do about it, though -- it's pretty clear the handle was destroyed sometime before, but I don't know when.

This is likely a DMC issue, and is probably best reported as a DMC bug. I don't know how much more you want to pursue this, but the next steps I'd recommend are: 1. obtain the dmc compiler (it's free) if you haven't already. 2. Compile an empty C program and run it using SciTE and Notepad++. Verify the same error occurs 3. report the failure using DMC's bugzilla. I'm guessing there's something in the way SciTE or Notepad++ sets up the pipes before executing the process which causes the problem. I have a sinking feeling we'll see more of this issue when the new std.process is released :( -Steve
Sep 26 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
SciTE (and probably N++ since it's based on scintilla) has a subsystem
setting which can affect how it invokes a process. See the "Command
subsystem" section here: http://www.scintilla.org/SciTEDoc.html

Usually I run with subsystem 0.
Sep 26 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 18:17:49 -0400, Mehrdad <wfunction hotmail.com> wrote:

 On 9/26/2011 2:58 PM, Steven Schveighoffer wrote:
 This is likely a DMC issue, and is probably best reported as a DMC  
 bug.  I don't know how much more you want to pursue this, but the next  
 steps I'd recommend are:

 1. obtain the dmc compiler (it's free) if you haven't already.
 2. Compile an empty C program and run it using SciTE and Notepad++.   
 Verify the same error occurs
 3. report the failure using DMC's bugzilla.

 I'm guessing there's something in the way SciTE or Notepad++ sets up  
 the pipes before executing the process which causes the problem.  I  
 have a sinking feeling we'll see more of this issue when the new  
 std.process is released :(

 -Steve

used handle in snn.lib), because I don't see how it could affect my D program this way, if it was a DMC issue. And yeah, I just tested it with a simple hello world C example using DMC as well, and got the same error. Pretty darn sure now that the handle is getting closed twice or something...

My apologies, I meant DMC runtime issue (whose library is called snn.lib). At least with this test, we can rule out D being the culprit. However, we still have to deal with this problem... -Steve
Sep 26 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 18:10:05 -0400, Mehrdad <wfunction hotmail.com> wrote:

 On 9/26/2011 2:53 PM, Steven Schveighoffer wrote:
 I think the reason Walter always gives is that dmd would then have to  
 output COFF objects, whereas it currently outputs OMF.  OPTLINK also  
 does not accept COFF objects.

of msvcrt.lib, like we're doing for all the other Windows libraries? I'm confused as to how this is an issue.

I don't know, maybe there's another reason.
 I think a lot of people would very much welcome this change, but it's  
 somewhat of an underwhelming change.  Instead of using DMC's runtime  
 for C functions, you are using VC++'s runtime.  But who cares?  I want  
 D's runtime to be better :)

Microsoft's C runtime that's packaged with Windows (which Microsoft ironically doesn't want you to use, but which compilers like GCC use anyway... and it turns out better than anything bundled with Visual Studio). You don't actually need Visual Studio to use it (in fact, it's not even included) -- the Windows Driver Kit has the bundled stuff required.

It's the same thing. MSVCRT.dll is included with every windows. Just the version may be different from the runtime bundled with Visual Studio. That's why you can install the visual studio redistributable package. The lib file (which contains the linking information) is not included (with Windows). Neither is the statically linked version. But make no mistake, it's the Visual C++ runtime (MicroSoftVisualCRunTime).
 In all honesty I don't see a single thing about it that I would call  
 "better". Do you have any examples of things that snn.lib does 'better'  
 than msvcrt.lib?

I think it may conform to the standard better. But I don't think any C runtime is better than what we could make in D :)
 The only incompatibility (which isn't "better" or "worse"... it's just  
 an incompatibility) I can see is exception handling, but that's  
 obviously an exception that can be left in snn.lib. Almost all the rest  
 of the runtime, though, can be easily removed and redirected to  
 msvcrt.dll... and I don't think there'd be any problems. (Are there?)

I don't know. For certain, there are some pieces of phobos written specifically knowing the internals of Digital Mars' runtime (like std.stdio).
 I think the best path is weaning ourselves off of C dependencies.   
 Starting with FILE *, which has a lot of limitations.  It's part of the  
 reason I'm working on reworking std.stdio.
 For example, with my version of std.stdio, you would never have this  
 problem, because it deals with the HANDLE directly instead of through  
 DMC's file descriptor layer.

is all regarding what's /already/ taking place in snn.lib /before/ D runs, which is obviously unrelated to what's going on in std.stdio. I'm really curious to know what specific problems are keeping us from doing the switch (i.e. what snn.lib does better than msvcrt.dll). :)

I'm not really qualified to answer that. It seems like the right move to me, so there must be good reasons compiler-wise. -Steve
Sep 26 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 18:36:16 -0400, Mehrdad <wfunction hotmail.com> wrote:

 On 9/26/2011 2:58 PM, Steven Schveighoffer wrote:
 This is likely a DMC issue, and is probably best reported as a DMC  
 bug.  I don't know how much more you want to pursue this, but the next  
 steps I'd recommend are:

 1. obtain the dmc compiler (it's free) if you haven't already.
 2. Compile an empty C program and run it using SciTE and Notepad++.   
 Verify the same error occurs
 3. report the failure using DMC's bugzilla.

 I'm guessing there's something in the way SciTE or Notepad++ sets up  
 the pipes before executing the process which causes the problem.  I  
 have a sinking feeling we'll see more of this issue when the new  
 std.process is released :(

 -Steve

originally: A handle is being closed that isn't actually valid. Which handle? stderr. stderr is being closed when in fact it is the same handle as stdout. So when _fcloseallp() is called, it goes through the structures in __iob and closes each one in turn -- which causes a duplicate close operation on stderr, because it already gets closed when stdout gets closed. So it's an snn.lib issue. Is it easy to fix? (I hope it is...)

Probably -- let the OS close the handles instead of manually closing them :) -Steve
Sep 26 2011
prev sibling next sibling parent "Regan Heath" <regan netmail.co.nz> writes:
On Mon, 26 Sep 2011 23:14:43 +0100, Mehrdad <wfunction hotmail.com> wrote:

 On 9/26/2011 3:07 PM, Andrej Mitrovic wrote:
 SciTE (and probably N++ since it's based on scintilla) has a subsystem
 setting which can affect how it invokes a process. See the "Command
 subsystem" section here: http://www.scintilla.org/SciTEDoc.html

 Usually I run with subsystem 0.

(1) Scintilla is just the text editor. It doesn't have anything to do with the integrated command line. (2) I tried both 0 and 1, and neither made a difference. In fact, while using 1 DID pop up an extra command-line window, in either case the I/O took place inside SciTE itself. And in both cases, both DMD and my program crashed. You should be able to easily reproduce this if you set the "Enable close exception" and "Enable bad handles detection" Global Flags, using the Global Flags utility that comes with Debugging Tools for Windows. (Just make sure you reboot before you expect to see the change.)

On newer versions of windows you can use Application Verifier.. provided you run the exe you want to verify in a debugger (WinDbg or MSVC). -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Sep 27 2011
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 19:15:08 -0400, Mehrdad <wfunction hotmail.com> wrote:

 On 9/26/2011 3:41 PM, Steven Schveighoffer wrote:
 Probably -- let the OS close the handles instead of manually closing  
 them :)

 -Steve

internal buffering of the data, which would be lost if the stream isn't closed.

That's what fflush is for. The handles should simply be closed by the OS. -Steve
Sep 27 2011
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, September 26, 2011 16:07 Dmitry Olshansky wrote:
 On 27.09.2011 2:33, Jonathan M Davis wrote:
 On Monday, September 26, 2011 14:53 Steven Schveighoffer wrote:
 On Mon, 26 Sep 2011 17:35:20 -0400, Mehrdad<wfunction hotmail.com> 



 On 9/26/2011 2:30 PM, Mehrdad wrote:
 On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:
 AHA!
 
 Yes, there is a bug in snn.lib regarding pipes. And I fixed it,
 waiting for Walter to incorporate it :) I needed it for the new
 std.process.
 
 What it comes down to is, DMC's FILE * implementation does not expect
 some of the quirks of pipe HANDLEs. For instance, if you open a FILE
 * around a pipe handle, it still tries to do a seek on that handle,
 and crashes. Also, when the write end of a pipe is closed, reading
 from the read end results in EPIPE from ReadFile, but this is
 translated to EBADF by the runtime. Therefore, FILE * sets an error
 instead of EOF.
 
 Is the email address you have for this message correct? If so, I can
 send you a new version of snn.lib to try linking your code against
 (if you are willing to go through these steps), to see if it fixes
 your problem.
 
 Using the command line dmd to build should be sufficient (I think).
 
 -Steve

Thanks for the email. It seems like it still crashes from SciTE, though. :( When I try debugging, I see it's doing so inside _close() which is /immediately/ above kernel32.dll! BaseThreadInitThunk 12() -- in other words, it seems to be called directly from the entrypoint of the program, assuming it hasn't undergone optimizations. The error is: 0xC0000008: An invalid handle was specified. Is there any other place in which this can happen?

Also, now that we're on the topic... this might or might not sound silly, but why not just use msvcrt.dll (if not the whole library, at least the I/O functions)? It's not like it introduces a new dependency (it's already in every version of Windows) and it's also proven to work. Not to mention that it reduces code size as well...

I think the reason Walter always gives is that dmd would then have to output COFF objects, whereas it currently outputs OMF. OPTLINK also does not accept COFF objects. I think a lot of people would very much welcome this change, but it's somewhat of an underwhelming change. Instead of using DMC's runtime for C functions, you are using VC++'s runtime. But who cares? I want D's runtime to be better :)

The gain in switching to COFF would be _huge_, because then D code would be properly interoperable with most of the C code on Windows - and in particular C code compiled with Microsoft's compiler. The only thing saving the current situation from being a complete disaster is that you can still use dlls.

It's not that easy. I mean to gain this level of compatibility we'd have to do both: switch to MS runtime and use COFF object format. Linking together two C runtimes is *ehm* tricky. But most importantly newer versions of VC++ use newer runtimes, I'm not sure how compatible they are, but judging by my previous experience - you still has to keep things separated like in DLLs and never mix allocation/FILE* stuff between them.

That may be, but the fact that I can't just link C code built with Visual Studio with D code is a major problem in enviroments where Visual Studio is what is used on Windows (such as where I work). In comparison, in Linux it works, because dmd actually uses the compiler which is the de facto standard on Linux - gcc. But since dmd uses optlink on Windows, you just don't have that level of compatibility. dmc and optlink may very well be solid tools, but as a D programmer trying to interoperate with C code, compatability matters a lot, and dmd on Windows just doesn't have it. - Jonathan M Davis
Sep 26 2011
prev sibling parent Don <nospam nospam.com> writes:
On 26.09.2011 16:55, Mehrdad wrote:
 DMD 2.055 seems to be crashing on my computer (Windows 7 x64).

 When I press Debug, I see it's doing so in CloseHandle, probably because
 I've turned on "Enable close exception" with Visual Studio's "Global
 Flags" tool. (It takes a reboot to take effect.)

 So this means there's an incorrect handle being used somewhere... but
 interestingly, by the time it crashes, the output file is already
 produced. I don't have the time to make a debug build right now, but I
 thought I'd mention this so that it could be looked into at some point.

That's interesting. I recently found another bug, which is also Windows7 specific. http://d.puremagic.com/issues/show_bug.cgi?id=6660 Although I assumed that it's stack corruption, it could also be the same handle issue.
Sep 26 2011