www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.mmfile

reply James Dunne <james.jdunne gmail.com> writes:
Has anyone used std.mmfile in their projects?  If so, how successful 
were you in using it?

I'm attempting to write a few test routines using it to create, read, 
write, and extend files in a paged manner (i.e. for a database).

Currently, there's no easy way to resize a MmFile without closing it and 
re-opening it with a bigger size parameter.  Is this a limitation of 
memory-mapped files in general?  On Win32 platform?  On Linux platform? 
  Or is it just a limitation of the implementation contained within 
std.mmfile?

Also, the MmFile should have an Open() and Close() method instead of 
relying only on constructors/destructors.

-- 
Regards,
James Dunne
Sep 14 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"James Dunne" <james.jdunne gmail.com> wrote in message 
news:dgaevi$2dr6$1 digitaldaemon.com...
 Has anyone used std.mmfile in their projects?  If so, how successful were 
 you in using it?

 I'm attempting to write a few test routines using it to create, read, 
 write, and extend files in a paged manner (i.e. for a database).

 Currently, there's no easy way to resize a MmFile without closing it and 
 re-opening it with a bigger size parameter.  Is this a limitation of 
 memory-mapped files in general?  On Win32 platform?  On Linux platform? Or 
 is it just a limitation of the implementation contained within std.mmfile?

The OS (Posix and Win32) forces mem-mapped file sizes to be fixed - so what you describe is all we can do.
 Also, the MmFile should have an Open() and Close() method instead of 
 relying only on constructors/destructors.

I thought about that, too, before sending my windowing changes to Walter. It is a toss-up to me having open/close in addition to delete. The benefits are more flexibility since with the delete you have to know about all the references to the mmfile - if anyone holds onto a deleted mmfile they'll get a seg-v when they try to use it. If we had open/close then people could check if the file is closed before trying to use it. The downside is that it adds (a small) bit of complexity. If you wrap the MmFile in a MmFileStream then you can close and safely test later that the stream is closed (MmFileStream deletes the underlying MmFile)
 -- 
 Regards,
 James Dunne

Sep 14 2005
parent reply James Dunne <james.jdunne gmail.com> writes:
Ben Hinkle wrote:
 "James Dunne" <james.jdunne gmail.com> wrote in message 
 news:dgaevi$2dr6$1 digitaldaemon.com...
 
Has anyone used std.mmfile in their projects?  If so, how successful were 
you in using it?

I'm attempting to write a few test routines using it to create, read, 
write, and extend files in a paged manner (i.e. for a database).

Currently, there's no easy way to resize a MmFile without closing it and 
re-opening it with a bigger size parameter.  Is this a limitation of 
memory-mapped files in general?  On Win32 platform?  On Linux platform? Or 
is it just a limitation of the implementation contained within std.mmfile?

The OS (Posix and Win32) forces mem-mapped file sizes to be fixed - so what you describe is all we can do.
Also, the MmFile should have an Open() and Close() method instead of 
relying only on constructors/destructors.

I thought about that, too, before sending my windowing changes to Walter. It is a toss-up to me having open/close in addition to delete. The benefits are more flexibility since with the delete you have to know about all the references to the mmfile - if anyone holds onto a deleted mmfile they'll get a seg-v when they try to use it. If we had open/close then people could check if the file is closed before trying to use it. The downside is that it adds (a small) bit of complexity. If you wrap the MmFile in a MmFileStream then you can close and safely test later that the stream is closed (MmFileStream deletes the underlying MmFile)
-- 
Regards,
James Dunne


What windowing changes have you got in store? Personally, I'd love it if I could just map a certain section of the file (not the entire file) and swap that in and out, while still being performant. Is this possible with memory-mapped files?
Sep 15 2005
parent "Ben Hinkle" <ben.hinkle gmail.com> writes:
 What windowing changes have you got in store?  Personally, I'd love it if 
 I could just map a certain section of the file (not the entire file) and 
 swap that in and out, while still being performant.  Is this possible with 
 memory-mapped files?

They are in dmd.130 http://www.digitalmars.com/d/phobos/std_mmfile.html. The window size must be a multiple of the window size allowed by the OS - which is a page size on the OSes I checked (Linux/win32). The page size when I checked was 64K. Once you set the window size it automatically manages the mapping and unmapping multiples of the window size when you request an index i or a range i..j to ensure the data is mapped.
Sep 15 2005