www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.mmfile

reply James Dunne <james.jdunne gmail.com> writes:
Has anyone used std.mmfile in their projects?  If so, how successful 
were you in using it?

I'm attempting to write a few test routines using it to create, read, 
write, and extend files in a paged manner (i.e. for a database).

Currently, there's no easy way to resize a MmFile without closing it and 
re-opening it with a bigger size parameter.  Is this a limitation of 
memory-mapped files in general?  On Win32 platform?  On Linux platform? 
  Or is it just a limitation of the implementation contained within 
std.mmfile?

Also, the MmFile should have an Open() and Close() method instead of 
relying only on constructors/destructors.

-- 
Regards,
James Dunne
Sep 14 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"James Dunne" <james.jdunne gmail.com> wrote in message 
news:dgaevi$2dr6$1 digitaldaemon.com...

 Has anyone used std.mmfile in their projects?  If so, how successful were 
 you in using it?

 I'm attempting to write a few test routines using it to create, read, 
 write, and extend files in a paged manner (i.e. for a database).

 Currently, there's no easy way to resize a MmFile without closing it and 
 re-opening it with a bigger size parameter.  Is this a limitation of 
 memory-mapped files in general?  On Win32 platform?  On Linux platform? Or 
 is it just a limitation of the implementation contained within std.mmfile?


The OS (Posix and Win32) forces mem-mapped file sizes to be fixed - so what 
you describe is all we can do.


 Also, the MmFile should have an Open() and Close() method instead of 
 relying only on constructors/destructors.


I thought about that, too, before sending my windowing changes to Walter. It 
is a toss-up to me having open/close in addition to delete. The benefits are 
more flexibility since with the delete you have to know about all the 
references to the mmfile - if anyone holds onto a deleted mmfile they'll get 
a seg-v when they try to use it. If we had open/close then people could 
check if the file is closed before trying to use it. The downside is that it 
adds (a small) bit of complexity. If you wrap the MmFile in a MmFileStream 
then you can close and safely test later that the stream is closed 
(MmFileStream deletes the underlying MmFile)


 -- 
 Regards,
 James Dunne
Sep 14 2005
parent reply James Dunne <james.jdunne gmail.com> writes:
Ben Hinkle wrote:

 "James Dunne" <james.jdunne gmail.com> wrote in message 
 news:dgaevi$2dr6$1 digitaldaemon.com...
 

Has anyone used std.mmfile in their projects?  If so, how successful were 
you in using it?

I'm attempting to write a few test routines using it to create, read, 
write, and extend files in a paged manner (i.e. for a database).

Currently, there's no easy way to resize a MmFile without closing it and 
re-opening it with a bigger size parameter.  Is this a limitation of 
memory-mapped files in general?  On Win32 platform?  On Linux platform? Or 
is it just a limitation of the implementation contained within std.mmfile?


 
 
 The OS (Posix and Win32) forces mem-mapped file sizes to be fixed - so what 
 you describe is all we can do.
 
 

Also, the MmFile should have an Open() and Close() method instead of 
relying only on constructors/destructors.


 
 
 I thought about that, too, before sending my windowing changes to Walter. It 
 is a toss-up to me having open/close in addition to delete. The benefits are 
 more flexibility since with the delete you have to know about all the 
 references to the mmfile - if anyone holds onto a deleted mmfile they'll get 
 a seg-v when they try to use it. If we had open/close then people could 
 check if the file is closed before trying to use it. The downside is that it 
 adds (a small) bit of complexity. If you wrap the MmFile in a MmFileStream 
 then you can close and safely test later that the stream is closed 
 (MmFileStream deletes the underlying MmFile)
 
 

-- 
Regards,
James Dunne


 
 
 
 


What windowing changes have you got in store?  Personally, I'd love it 
if I could just map a certain section of the file (not the entire file) 
and swap that in and out, while still being performant.  Is this 
possible with memory-mapped files?
Sep 15 2005
parent "Ben Hinkle" <ben.hinkle gmail.com> writes:
 What windowing changes have you got in store?  Personally, I'd love it if 
 I could just map a certain section of the file (not the entire file) and 
 swap that in and out, while still being performant.  Is this possible with 
 memory-mapped files?


They are in dmd.130 http://www.digitalmars.com/d/phobos/std_mmfile.html. The 
window size must be a multiple of the window size allowed by the OS - which 
is a page size on the OSes I checked (Linux/win32). The page size when I 
checked was 64K. Once you set the window size it automatically manages the 
mapping and unmapping multiples of the window size when you request an index 
i or a range i..j to ensure the data is mapped.
Sep 15 2005