www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Array ops give sharing violation under Windows 7 64 bit?

reply Walter Bright <newshound2 digitalmars.com> writes:
I've been upgrading to a Windows 64 bit box. Running the D test suite, I ran 
into a very strange problem. Here's the program:

---------------------------
  extern(C) int printf(const char*, ...);

  int main()
  {
      byte[3] a;
      byte[3] b;
      byte[3] c;

      a[] = b[] + c[];

      printf("Success\n");
      return 0;
  }
---------------------------

I run it from a cc.bat file that has the contents:

--------------------------
  ..\dmd test
  test
  ..\dmd test
--------------------------

and the result is:
--------------------------
  C:\test>..\dmd test

  C:\test>test
  Success

  C:\test>..\dmd test
  GetLastError = 32 test.exe
  OPTLINK (R) for Win32  Release 8.00.12
  Copyright (C) Digital Mars 1989-2010  All rights reserved.
  http://www.digitalmars.com/ctg/optlink.html
  OPTLINK : Error 3: Cannot Create File test.exe
  --- errorlevel 1
---------------------------

Note the failure to write out test.exe. I instrumented Optlink to figure out 
why, and the CreateFile() returns error 32, which is "The process cannot access 
the file because it is being used by another process."
If you run the commands by typing them in (not via a .bat file) it works. If
the 
array
operations are removed, it works. It works on Windows XP.

I'm mystified. Does anyone have any ideas?
Mar 24 2012
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/24/2012 11:55 AM, Walter Bright wrote:
 I'm mystified. Does anyone have any ideas?

If I add a del test.exe to the cc.bat file: -------------------- ..\dmd test test del test.exe ..\dmd test ------------------- It now fails with GetLastError() of 5, meaning "Access is denied." The cc.bat file has to be executed repeatedly for this to happen. I wonder if it may have something to do with the operating system delayed writes?
Mar 24 2012
next sibling parent reply "Xinok" <xinok live.com> writes:
On Saturday, 24 March 2012 at 19:08:00 UTC, Walter Bright wrote:
 On 3/24/2012 11:55 AM, Walter Bright wrote:
 I'm mystified. Does anyone have any ideas?

If I add a del test.exe to the cc.bat file: -------------------- ..\dmd test test del test.exe ..\dmd test ------------------- It now fails with GetLastError() of 5, meaning "Access is denied." The cc.bat file has to be executed repeatedly for this to happen. I wonder if it may have something to do with the operating system delayed writes?

If you have an antivirus, try disabling it before compiling.
Mar 24 2012
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/24/2012 12:18 PM, Xinok wrote:
 If you have an antivirus, try disabling it before compiling.

I have a brand new vanilla install of Windows 7 home premium.
Mar 24 2012
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/25/2012 2:50 PM, Kagamin wrote:
 Microsoft has antivirus bundled with windows. Go to security center and see
 whether Windows Defender is working.

Well, I'll be hornswoggled. That did the trick! Defender has a feature where it will to real time scanning of new executables. Apparently, it was trying to scan the exe's built by the test suite while the test suite was trying to rewrite them. I just disabled my dev directory from Defender scanning it. Thanks!
Mar 25 2012
parent reply Don <nospam nospam.com> writes:
On 27.03.2012 00:42, Andrej Mitrovic wrote:
 On 3/26/12, Walter Bright<newshound2 digitalmars.com>  wrote:
 On 3/25/2012 2:50 PM, Kagamin wrote:
 Microsoft has antivirus bundled with windows. Go to security center and
 see
 whether Windows Defender is working.

Well, I'll be hornswoggled. That did the trick!

I really don't think that's the bottom issue. I've had defender off, and I can still reproduce the issue but it seems to happen in random phases. Several hundred runs it's ok, and then it's not ok.

Can't reproduce. I've run it more than a thousand times with no failures. Just looks like a bug in Windows Defender.
Mar 28 2012
parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/28/2012 12:10 AM, Don wrote:
 On 27.03.2012 00:42, Andrej Mitrovic wrote:
 On 3/26/12, Walter Bright<newshound2 digitalmars.com> wrote:
 On 3/25/2012 2:50 PM, Kagamin wrote:
 Microsoft has antivirus bundled with windows. Go to security center and
 see
 whether Windows Defender is working.

Well, I'll be hornswoggled. That did the trick!

I really don't think that's the bottom issue. I've had defender off, and I can still reproduce the issue but it seems to happen in random phases. Several hundred runs it's ok, and then it's not ok.

Can't reproduce. I've run it more than a thousand times with no failures. Just looks like a bug in Windows Defender.

Also, it only happened with .exe files, another arrow that points at Defender, because Defender only mucks with .exe's.
Mar 29 2012
prev sibling next sibling parent reply Brad Roberts <braddr puremagic.com> writes:
On 3/24/2012 12:07 PM, Walter Bright wrote:
 On 3/24/2012 11:55 AM, Walter Bright wrote:
 I'm mystified. Does anyone have any ideas?

If I add a del test.exe to the cc.bat file: -------------------- ..\dmd test test del test.exe ..\dmd test ------------------- It now fails with GetLastError() of 5, meaning "Access is denied." The cc.bat file has to be executed repeatedly for this to happen. I wonder if it may have something to do with the operating system delayed writes?

I saw this sort of behavior in the auto-tester (was win7 at the time, now it's running on a windows server 2008 box) during the various combinations of compilation options until I switched to adding a counter and a suffix to each compilation output. I don't recall there being a particular pattern to the failures (like being tied to array ops). Later, Brad
Mar 24 2012
parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/24/2012 12:19 PM, Brad Roberts wrote:
 I saw this sort of behavior in the auto-tester (was win7 at the time,

At least I'm not losing my mind :-)
 now it's running on a windows server 2008 box)
 during the various combinations of compilation options until I switched to
adding a counter and a suffix to each
 compilation output.  I don't recall there being a particular pattern to the
failures (like being tied to array ops).

More investigation: If I delete the array ops, but add the import: import core.cpuid; it fails. I'm also running a 6 core AMD FX 6100 CPU.
Mar 24 2012
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
On Saturday, 24 March 2012 at 19:22:03 UTC, Walter Bright wrote:
 On 3/24/2012 12:18 PM, Xinok wrote:
 If you have an antivirus, try disabling it before compiling.

I have a brand new vanilla install of Windows 7 home premium.

Microsoft has antivirus bundled with windows. Go to security center and see whether Windows Defender is working.
Mar 25 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 3/26/12, Walter Bright <newshound2 digitalmars.com> wrote:
 On 3/25/2012 2:50 PM, Kagamin wrote:
 Microsoft has antivirus bundled with windows. Go to security center and
 see
 whether Windows Defender is working.

Well, I'll be hornswoggled. That did the trick!

I really don't think that's the bottom issue. I've had defender off, and I can still reproduce the issue but it seems to happen in random phases. Several hundred runs it's ok, and then it's not ok.
Mar 26 2012
prev sibling parent "Michael" <pr m1xa.com> writes:
This behavior of linker caused by a protection suit with sandbox 
or similiar technology. Affected systems are win xp, win 7. With 
a standard antivirus is all ok.
Mar 31 2012
prev sibling next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Walter Bright" <newshound2 digitalmars.com> wrote in message 
news:jkl5ab$1ocp$1 digitalmars.com...
 I've been upgrading to a Windows 64 bit box. Running the D test suite, I 
 ran into a very strange problem. Here's the program:

 ---------------------------
  extern(C) int printf(const char*, ...);

  int main()
  {
      byte[3] a;
      byte[3] b;
      byte[3] c;

      a[] = b[] + c[];

      printf("Success\n");
      return 0;
  }
 ---------------------------

 I run it from a cc.bat file that has the contents:

 --------------------------
  ..\dmd test
  test
  ..\dmd test
 --------------------------

 and the result is:
 --------------------------
  C:\test>..\dmd test

  C:\test>test
  Success

  C:\test>..\dmd test
  GetLastError = 32 test.exe
  OPTLINK (R) for Win32  Release 8.00.12
  Copyright (C) Digital Mars 1989-2010  All rights reserved.
  http://www.digitalmars.com/ctg/optlink.html
  OPTLINK : Error 3: Cannot Create File test.exe
  --- errorlevel 1
 ---------------------------

 Note the failure to write out test.exe. I instrumented Optlink to figure 
 out why, and the CreateFile() returns error 32, which is "The process 
 cannot access the file because it is being used by another process."
 If you run the commands by typing them in (not via a .bat file) it works. 
 If the array
 operations are removed, it works. It works on Windows XP.

 I'm mystified. Does anyone have any ideas?

Windows is known to enjoy holding locks on files just for the fuck of it. See if the behavior changes if you add some delay just before the last line. Or, if you adjust it so that compiling test.d takes longer.
Mar 24 2012
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/24/2012 12:31 PM, Nick Sabalausky wrote:
 Windows is known to enjoy holding locks on files just for the fuck of it.
 See if the behavior changes if you add some delay just before the last line.
 Or, if you adjust it so that compiling test.d takes longer.

I tried those. No dice.
Mar 24 2012
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/24/2012 12:36 PM, Walter Bright wrote:
 On 3/24/2012 12:31 PM, Nick Sabalausky wrote:
 Windows is known to enjoy holding locks on files just for the fuck of it.
 See if the behavior changes if you add some delay just before the last line.
 Or, if you adjust it so that compiling test.d takes longer.

I tried those. No dice.

Even running a program in between that does a sleep for a full second does not fix it.
Mar 24 2012
parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/24/2012 12:48 PM, Walter Bright wrote:
 On 3/24/2012 12:36 PM, Walter Bright wrote:
 On 3/24/2012 12:31 PM, Nick Sabalausky wrote:
 Windows is known to enjoy holding locks on files just for the fuck of it.
 See if the behavior changes if you add some delay just before the last line.
 Or, if you adjust it so that compiling test.d takes longer.

I tried those. No dice.

Even running a program in between that does a sleep for a full second does not fix it.

Even replacing the sleep program with the bat command PAUSE doesn't fix it. :-(
Mar 24 2012
prev sibling next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Nick Sabalausky" <a a.a> wrote in message 
news:jkl7fd$1rqs$1 digitalmars.com...
 Windows is known to enjoy holding locks on files just for the fuck of it. 
 See if the behavior changes if you add some delay just before the last 
 line. Or, if you adjust it so that compiling test.d takes longer.

If it still fails with the delay, you can probably find out (for whatever help it might be) what process is holding the file open by adding a pause right after running "test". Then before "Press[ing] a key" as it suggests, run SysInternal's Process Explorer to see who has the file open. My guess would be some basic Windows system process, but who knows.
Mar 24 2012
next sibling parent "Nick Sabalausky" <a a.a> writes:
"Nick Sabalausky" <a a.a> wrote in message 
news:jkl816$1t08$1 digitalmars.com...
 "Nick Sabalausky" <a a.a> wrote in message 
 news:jkl7fd$1rqs$1 digitalmars.com...
 Windows is known to enjoy holding locks on files just for the fuck of it. 
 See if the behavior changes if you add some delay just before the last 
 line. Or, if you adjust it so that compiling test.d takes longer.

If it still fails with the delay, you can probably find out (for whatever help it might be) what process is holding the file open by adding a pause right after running "test". Then before "Press[ing] a key" as it suggests, run SysInternal's Process Explorer to see who has the file open. My guess would be some basic Windows system process, but who knows.

Acutally, my guess would be 'explorer.exe', that's the one that's really known for hogging files. Not sure know much knowing that would help though. But if it's *not* that, and is something unexpected, then maybe it would provide a clue.
Mar 24 2012
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/24/2012 12:40 PM, Nick Sabalausky wrote:
 If it still fails with the delay, you can probably find out (for whatever
 help it might be) what process is holding the file open by adding a pause
 right after running "test". Then before "Press[ing] a key" as it suggests,
 run SysInternal's Process Explorer to see who has the file open. My guess
 would be some basic Windows system process, but who knows.

It's looking more and more like I cannot migrate to Windows 7. First off, I cannot run any 16 bit code. Second, I cannot run the D test suite. Bleh. I'm glad I decided to do Win7 on a separate box rather than "upgrade" my XP box. What a disaster that would have been. (More info: it isn't core.cpuid, that just increases the likelihood of the failure. Also, looking at the process list when doing a PAUSE shows no new processes, but it still fails.)
Mar 24 2012
next sibling parent "Nick Sabalausky" <a a.a> writes:
"Walter Bright" <newshound2 digitalmars.com> wrote in message 
news:jklc1h$25l6$1 digitalmars.com...
 Also, looking at the process list when doing a PAUSE shows no new 
 processes, but it still fails.

No, no, first download this, it's like Task Manager on steriods: http://technet.microsoft.com/en-us/sysinternals/bb896653 You can use that to look up a file and see what process has it open. IIRC, you do "Find" -> "Find Handle or DLL...", type in the filename (in this case "test.exe"), hit the "Search" button, and if the instructions I'm giving you here are actually correct, then it should show you what process is holding "test.exe" open.
Mar 24 2012
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/24/2012 2:08 PM, Andrej Mitrovic wrote:
 But it's not Optlink's fault either. Unilink has the same issue:

I did initially suspect Optlink. It's good to know that that is not the problem.
Mar 24 2012
parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/24/2012 2:39 PM, Andrej Mitrovic wrote:
 On 3/24/12, Walter Bright<newshound2 digitalmars.com>  wrote:
 On 3/24/2012 2:08 PM, Andrej Mitrovic wrote:
 But it's not Optlink's fault either. Unilink has the same issue:

I did initially suspect Optlink. It's good to know that that is not the problem.

In the meantime if you're just building and testing execution of multiple .d files individually you can use a for loop and the -of switch. E.g.: setlocal EnableDelayedExpansion For %%a in (*.d) do dmd -of%%~na.exe %%a&& %%~na.exe

Yeah, looks like I'll have to add an iteration count to the exe name.
Mar 24 2012
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/24/2012 2:55 PM, Andrej Mitrovic wrote:
 On 3/24/12, Andrej Mitrovic<andrej.mitrovich gmail.com>  wrote:
 It could be a Win7 x64 bug. If you'll give me 20 minutes I'll get a
 hold of an x86 Win7 installation and see if I can reproduce the bug
 there.

I can't reproduce this on Win7 x86. I've tried numerous times but it never failed. It seems it's x64-related (or just a Win7 x64 bug).

Your's and Brad's idea of putting an iteration count on the executable works in my tests. It looks like the only way to workaround it for now.
Mar 24 2012
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/24/2012 2:55 PM, Andrej Mitrovic wrote:
 I can't reproduce this on Win7 x86. I've tried numerous times but it
 never failed. It seems it's x64-related (or just a Win7 x64 bug).

I submitted a bug report to Microsoft.
Mar 24 2012
parent reply "Mehrdad" <wfunction hotmail.com> writes:
Is it really a bug, or is it this feature (NTFS tunneling)?

http://blogs.msdn.com/b/oldnewthing/archive/2005/07/15/439261.aspx

"Walter Bright"  wrote in message news:jkmceb$r5f$1 digitalmars.com... 

On 3/24/2012 2:55 PM, Andrej Mitrovic wrote:
 I can't reproduce this on Win7 x86. I've tried numerous times but it
 never failed. It seems it's x64-related (or just a Win7 x64 bug).

I submitted a bug report to Microsoft.
Mar 29 2012
parent Walter Bright <newshound2 digitalmars.com> writes:
The names involved were short, so it should not have triggered that.

On 3/29/2012 1:01 AM, Mehrdad wrote:
 Is it really a bug, or is it this feature (NTFS tunneling)?

 http://blogs.msdn.com/b/oldnewthing/archive/2005/07/15/439261.aspx

 "Walter Bright" wrote in message news:jkmceb$r5f$1 digitalmars.com...
 On 3/24/2012 2:55 PM, Andrej Mitrovic wrote:
 I can't reproduce this on Win7 x86. I've tried numerous times but it
 never failed. It seems it's x64-related (or just a Win7 x64 bug).

I submitted a bug report to Microsoft.

Mar 29 2012
prev sibling next sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
On Sat, 24 Mar 2012 20:48:47 +0100, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 3/24/2012 12:36 PM, Walter Bright wrote:
 On 3/24/2012 12:31 PM, Nick Sabalausky wrote:
 Windows is known to enjoy holding locks on files just for the fuck of  
 it.
 See if the behavior changes if you add some delay just before the last  
 line.
 Or, if you adjust it so that compiling test.d takes longer.

I tried those. No dice.

Even running a program in between that does a sleep for a full second does not fix it.

You could try Process Explorer or http://blad.wordpress.com/2006/05/11/fuser-on-win32/ to find out who is locking the file.
Mar 24 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 3/24/12, Walter Bright <newshound2 digitalmars.com> wrote:
 Running the D test suite, I ran into a very strange problem.

I can reproduce the problem on Win7 x64. It doesn't seem to have anything to do with DMD. This is my cc.bat: dmd -c test.obj link test.obj test link test.obj But it's not Optlink's fault either. Unilink has the same issue: dmd -c test.d ulink test.obj test.exe ulink test.obj C:\dev>cc.bat C:\dev>dmd -c test.d C:\dev>ulink test.obj UniLink v1.07 [beta] (build 3.35) C:\dev>test.exe Success C:\dev>ulink test.obj UniLink v1.07 [beta] (build 3.35) Fatal: Unable to create file 'test.exe' (program still running?) It could be a Win7 x64 bug. If you'll give me 20 minutes I'll get a hold of an x86 Win7 installation and see if I can reproduce the bug there.
Mar 24 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 3/24/12, Walter Bright <newshound2 digitalmars.com> wrote:
 On 3/24/2012 2:08 PM, Andrej Mitrovic wrote:
 But it's not Optlink's fault either. Unilink has the same issue:

I did initially suspect Optlink. It's good to know that that is not the problem.

In the meantime if you're just building and testing execution of multiple .d files individually you can use a for loop and the -of switch. E.g.: setlocal EnableDelayedExpansion For %%a in (*.d) do dmd -of%%~na.exe %%a && %%~na.exe
Mar 24 2012
prev sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 3/24/12, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:
 It could be a Win7 x64 bug. If you'll give me 20 minutes I'll get a
 hold of an x86 Win7 installation and see if I can reproduce the bug
 there.

I can't reproduce this on Win7 x86. I've tried numerous times but it never failed. It seems it's x64-related (or just a Win7 x64 bug).
Mar 24 2012
prev sibling next sibling parent Don <nospam nospam.com> writes:
On 24.03.2012 19:55, Walter Bright wrote:
 I've been upgrading to a Windows 64 bit box. Running the D test suite, I
 ran into a very strange problem. Here's the program:

 ---------------------------
 extern(C) int printf(const char*, ...);

 int main()
 {
 byte[3] a;
 byte[3] b;
 byte[3] c;

 a[] = b[] + c[];

 printf("Success\n");
 return 0;
 }
 ---------------------------

 I run it from a cc.bat file that has the contents:

 --------------------------
 ..\dmd test
 test
 ..\dmd test
 --------------------------

 and the result is:
 --------------------------
 C:\test>..\dmd test

 C:\test>test
 Success

 C:\test>..\dmd test
 GetLastError = 32 test.exe
 OPTLINK (R) for Win32 Release 8.00.12
 Copyright (C) Digital Mars 1989-2010 All rights reserved.
 http://www.digitalmars.com/ctg/optlink.html
 OPTLINK : Error 3: Cannot Create File test.exe
 --- errorlevel 1
 ---------------------------

 Note the failure to write out test.exe. I instrumented Optlink to figure
 out why, and the CreateFile() returns error 32, which is "The process
 cannot access the file because it is being used by another process."
 If you run the commands by typing them in (not via a .bat file) it
 works. If the array
 operations are removed, it works. It works on Windows XP.

 I'm mystified. Does anyone have any ideas?

Bug 6660. I don't understand it, but the minimal test case is very simple, just a few asm instructions, no function calls. As I reported there, all that is required is a use of CPUID 2 or 4. Any use of CPUID 1 or 3 doesn't trigger the bug. It seems to me as though somehow, one processor is being kept alive after the process has ended. I wasn't sure if this was an OS issue or a processor issue. But if you're seeing this after upgrading an individual machine, it definitely has to be a Windows 7 bug. I'm relieved to hear that someone else can reproduce it.
Mar 25 2012
prev sibling next sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
Seems like you got your answer (anti-virus).

I've been struggling with this for creating a temp file, then
deleting. Disabling AV isn't a solution so I have it run delete
when the program exits :P

Also, I've been using system("pause") to do waiting ha, if the
user closes the console with the X, this command reads it as "any
key" and will execute the next command, which can complete before
the application receives TERM. So now all my system("pause")s
have a thread wait so nothing is run.

MS needs to get rid of the damn file locks!
Mar 26 2012
prev sibling parent dennis luehring <dl.soluz gmx.net> writes:
try

handle:
http://technet.microsoft.com/de-de/sysinternals/bb896655

or

process explorer:
http://technet.microsoft.com/de-de/sysinternals/bb896653

to find the blocking process

Am 24.03.2012 19:55, schrieb Walter Bright:
 Note the failure to write out test.exe. I instrumented Optlink to figure out
 why, and the CreateFile() returns error 32, which is "The process cannot access
 the file because it is being used by another process."
 If you run the commands by typing them in (not via a .bat file) it works. If
the
 array
 operations are removed, it works. It works on Windows XP.

 I'm mystified. Does anyone have any ideas?

Mar 26 2012