www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Disk write in a "for" loop with RwMutex never happens

reply Gavin Ray <ray.gavin97 gmail.com> writes:
I've put the code, stripped to a minimal example here:
- https://ldc.godbolt.org/z/fzsx3Tnnn

You can see that the single write + read version of the code 
works just fine:
```
pageData[0..4] = [1, 2, 3, 4]
readData[0..4] = [1, 2, 3, 4]
```

Where here, `pageData` is the data to be written to a file, and 
`readData` is the result of trying to read the freshly written 
file data.

But if the same code is placed inside of a `for` loop, suddenly 
no writes occur:
```d
pageData[0..4] = [0, 0, 0, 0]
readData[0..4] = [0, 0, 0, 0]

pageData[0..4] = [0, 0, 0, 1]
readData[0..4] = [0, 0, 0, 0]

pageData[0..4] = [0, 0, 0, 2]
readData[0..4] = [0, 0, 0, 0]

pageData[0..4] = [0, 0, 0, 3]
readData[0..4] = [0, 0, 0, 0]

pageData[0..4] = [0, 0, 0, 4]
readData[0..4] = [0, 0, 0, 0]

// ...
```

Does anyone know what is happening here? It's really puzzling.
Aug 28 2022
next sibling parent reply bauss <jacobbauss gmail.com> writes:
On Sunday, 28 August 2022 at 22:46:17 UTC, Gavin Ray wrote:

 I've put the code, stripped to a minimal example here:
 - https://ldc.godbolt.org/z/fzsx3Tnnn

 You can see that the single write + read version of the code 
 works just fine:
 ```
 pageData[0..4] = [1, 2, 3, 4]
 readData[0..4] = [1, 2, 3, 4]
 ```

 Where here, `pageData` is the data to be written to a file, and 
 `readData` is the result of trying to read the freshly written 
 file data.

 But if the same code is placed inside of a `for` loop, suddenly 
 no writes occur:
 ```d
 pageData[0..4] = [0, 0, 0, 0]
 readData[0..4] = [0, 0, 0, 0]

 pageData[0..4] = [0, 0, 0, 1]
 readData[0..4] = [0, 0, 0, 0]

 pageData[0..4] = [0, 0, 0, 2]
 readData[0..4] = [0, 0, 0, 0]

 pageData[0..4] = [0, 0, 0, 3]
 readData[0..4] = [0, 0, 0, 0]

 pageData[0..4] = [0, 0, 0, 4]
 readData[0..4] = [0, 0, 0, 0]

 // ...
 ```

 Does anyone know what is happening here? It's really puzzling.


You probably need to flush the output.
Aug 29 2022
parent Gavin Ray <ray.gavin97 gmail.com> writes:
On Monday, 29 August 2022 at 07:04:49 UTC, bauss wrote:

 Does anyone know what is happening here? It's really puzzling.


 You probably need to flush the output.


That's a good idea. I gave it a shot, and the following doesn't 
seem to change anything unfortunately:

```d
void writePage(PageId pageId, ubyte[PAGE_SIZE] pageData)
{
     synchronized (dbIOMutex.writer)
     {
         dbFile.seek(pageId * PAGE_SIZE);
         dbFile.rawWrite(pageData);
         dbFile.flush();
         dbFile.sync();
     }
}
```
Aug 29 2022
prev sibling next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
After a bunch of playing around I managed to determine that it is as 
simple as the mode.

exists(dbFileName) ? "r+" : "w+"



Will fix it.

Of course you shouldn't delete the file like that method is doing. It 
should probably reinitialize the FILE* descriptor.
Aug 29 2022
parent reply Gavin Ray <ray.gavin97 gmail.com> writes:
On Monday, 29 August 2022 at 15:52:31 UTC, rikki cattermole wrote:

 After a bunch of playing around I managed to determine that it 
 is as simple as the mode.

 exists(dbFileName) ? "r+" : "w+"



 Will fix it.

 Of course you shouldn't delete the file like that method is 
 doing. It should probably reinitialize the FILE* descriptor.


D'oh!

I didn't even think about the mode:

   > `a+ or ab+ or a+b`
   > "Append; open or create file for update, writing at 
end-of-file."

It must have been the "writing at end of file" bit?

Thanks Rikki. /embarrassed
Aug 29 2022
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 30/08/2022 8:16 AM, Gavin Ray wrote:

 It must have been the "writing at end of file" bit?


I don't know.

It read like it should work.

The offsets were correct, it just didn't work *shrug*.
Aug 29 2022
prev sibling parent reply ag0aep6g <anonymous example.com> writes:
On Sunday, 28 August 2022 at 22:46:17 UTC, Gavin Ray wrote:

 I've put the code, stripped to a minimal example here:
 - https://ldc.godbolt.org/z/fzsx3Tnnn


[...]

 But if the same code is placed inside of a `for` loop, suddenly 
 no writes occur:


[...]

 Does anyone know what is happening here? It's really puzzling.


Relevant pieces of the code:

```d
class DiskManager
{
     void writePage(PageId pageId, ubyte[PAGE_SIZE] pageData)
     {
         synchronized (dbIOMutex.writer)
         {
             dbFile.seek(pageId * PAGE_SIZE);
             dbFile.rawWrite(pageData);
         }
     }
}

void singleReadWrite()
{
     PageId pageId = 0;

     diskManager.writePage(pageId, pageData);
}

void multiReadWrite()
{
     PageId pageId = 0;

     foreach (i; 0 .. 10)
     {
         diskManager.writePage(pageId, pageData);
     }
}
```

You never change `pageId`. So as far as I can tell, you're always 
`seek`-ing to the same position, and you just overwrite the same 
piece of the file again and again.
Aug 29 2022
parent ag0aep6g <anonymous example.com> writes:
On Monday, 29 August 2022 at 16:21:53 UTC, ag0aep6g wrote:

 You never change `pageId`. So as far as I can tell, you're 
 always `seek`-ing to the same position, and you just overwrite 
 the same piece of the file again and again.


Whoops. I guess I missed the point of the question there.
Aug 29 2022