www.digitalmars.com         C & C++   DMDScript  

D - VFS

reply BenjiSmith <BenjiSmith_member pathlink.com> writes:
I'm finding a need for a Virtual File System module. For those of you not
familiar with VFS, it gives you the ability to navigate through a series of
"directories" to locate and manipulate "files". The trick is that the VFS is not
technically implemented in a traditional file system. It's usually all globbed
together into a single file. A zip archive is a good example of a VFS, since it
can have a heirarchical directory structure with all sorts of files places into
any level of that structure.

Since zip archives don't meet my requirements, and since I don't know of any VFS
implementations that exist yet in D, I'm starting the design process for one of
my own.

So far, my VFS module will include the following features:

1) Serialization as a single binary file.
2) CreateDate & ModifyDate attributes for each dir/file.
3) URL navigation through the director structure.
4) A user database, complete with usernames and password-hashes for each user.
5) Per-file and per-directory read/write access settings.
6) Per-file compression settings (some files won't benefit from compression &
would be better off not compressed).
7) Per-file encryption settings (some files need stronger encryption than
others; some files don't need any encryption at all).
8) Separate parsing of the VFS directory and the data in the files
9) Pluggable interface for adding new compression/encryption algorithms.
10) XML export of directory tree (with all attribute information)

I will make my VFS code available (probably under LGPL or BSD license) when it's
done. Please let me know what other features you might be interested in seeing
in a VFS system. My primary interest in creating a VFS is so that I can package
up a bunch of resources (icons, scripts, settings, etc.) in a directory
structure that won't be accessible to users outside of my application. If you
have a different type of need for a VFS, it might necessetate different
features. Let me know what you're interested in.
Jul 02 2003
next sibling parent <sorry no.spam> writes:
You may save som time looking at SQLite. Your project sounds like much the
same thing except that this is an embeddable SQL database.
http://www.hwaci.com/sw/sqlite/

Roald

"BenjiSmith" <BenjiSmith_member pathlink.com> skrev i melding
news:bdvk0v$mi5$1 digitaldaemon.com...

 I'm finding a need for a Virtual File System module. For those of you not
 familiar with VFS, it gives you the ability to navigate through a series


of

 "directories" to locate and manipulate "files". The trick is that the VFS


is not

 technically implemented in a traditional file system. It's usually all


globbed

 together into a single file. A zip archive is a good example of a VFS,


since it

 can have a heirarchical directory structure with all sorts of files places


into

 any level of that structure.

 Since zip archives don't meet my requirements, and since I don't know of


any VFS

 implementations that exist yet in D, I'm starting the design process for


one of

 my own.

 So far, my VFS module will include the following features:

 1) Serialization as a single binary file.
 2) CreateDate & ModifyDate attributes for each dir/file.
 3) URL navigation through the director structure.
 4) A user database, complete with usernames and password-hashes for each


user.

 5) Per-file and per-directory read/write access settings.
 6) Per-file compression settings (some files won't benefit from


compression &

 would be better off not compressed).
 7) Per-file encryption settings (some files need stronger encryption than
 others; some files don't need any encryption at all).
 8) Separate parsing of the VFS directory and the data in the files
 9) Pluggable interface for adding new compression/encryption algorithms.
 10) XML export of directory tree (with all attribute information)

 I will make my VFS code available (probably under LGPL or BSD license)


when it's

 done. Please let me know what other features you might be interested in


seeing

 in a VFS system. My primary interest in creating a VFS is so that I can


package

 up a bunch of resources (icons, scripts, settings, etc.) in a directory
 structure that won't be accessible to users outside of my application. If


you

 have a different type of need for a VFS, it might necessetate different
 features. Let me know what you're interested in.
Jul 02 2003
prev sibling next sibling parent Mark Evans <Mark_member pathlink.com> writes:
http://www.datacompression.info/ArithmeticCoding.shtml
Jul 02 2003
prev sibling parent reply Ilya Minkov <midiclub 8ung.at> writes:
Oh, cool.

I believe i stumbled over compression algorithms which provide complete 
random read access to data. They are not particularly good in general, 
but perform well for text, in particular XML packfiles like OpenOffice 
ones would draw advantage of that, because such packfiles don't have to 
be uncompressed for use. Thus one thing to consider in the plug-in 
interface is to enable such plug-ins, which don't decompress all at 
once, but in the strategy they like.

Obviously, the datafiles in the archive cannot be changed, not really 
even if they are uncompressed. How about enabling writing to files, but 
storing "patches" in some other memory area, to be serialised later? The 
uses of this probably hard-to-implement feature are endless. This has to 
work transparently, with changes masking the actually unchanged contents 
of the file. While this could also be done within plugins with a usual 
file interface, i believe an interface addition is necessary which would 
allow the pakfile manager to somehow manage these "patches", i.e. save 
them somewhere, or ask compression plug-in to apply them.

This also may go along with extending the file interface in Phobos. It 
and the library have to complement each other in a harmonic matter.

-i.

BenjiSmith wrote:

 I'm finding a need for a Virtual File System module. For those of you not
 familiar with VFS, it gives you the ability to navigate through a series of
 "directories" to locate and manipulate "files". The trick is that the VFS is
not
 technically implemented in a traditional file system. It's usually all globbed
 together into a single file. A zip archive is a good example of a VFS, since it
 can have a heirarchical directory structure with all sorts of files places into
 any level of that structure.
 
 Since zip archives don't meet my requirements, and since I don't know of any
VFS
 implementations that exist yet in D, I'm starting the design process for one of
 my own.
 
 So far, my VFS module will include the following features:
 
 1) Serialization as a single binary file.
 2) CreateDate & ModifyDate attributes for each dir/file.
 3) URL navigation through the director structure.
 4) A user database, complete with usernames and password-hashes for each user.
 5) Per-file and per-directory read/write access settings.
 6) Per-file compression settings (some files won't benefit from compression &
 would be better off not compressed).
 7) Per-file encryption settings (some files need stronger encryption than
 others; some files don't need any encryption at all).
 8) Separate parsing of the VFS directory and the data in the files
 9) Pluggable interface for adding new compression/encryption algorithms.
 10) XML export of directory tree (with all attribute information)
 
 I will make my VFS code available (probably under LGPL or BSD license) when
it's
 done. Please let me know what other features you might be interested in seeing
 in a VFS system. My primary interest in creating a VFS is so that I can package
 up a bunch of resources (icons, scripts, settings, etc.) in a directory
 structure that won't be accessible to users outside of my application. If you
 have a different type of need for a VFS, it might necessetate different
 features. Let me know what you're interested in.
Jul 03 2003
parent BenjiSmith <BenjiSmith_member pathlink.com> writes:
In article <be1n7k$2tlm$1 digitaldaemon.com>, Ilya Minkov says...


This also may go along with extending the file interface in Phobos. It 
and the library have to complement each other in a harmonic matter.

-i.


What I'd really like to see in Phobos is a FileSystem interface (or a virtual
class) so that I can inherit from it when I create my VFS class. That way,
whenever someone's code needs to access a filesystem, it won't care whether it's
accessing a real filesystem or a virtual filesystem.

Now that would be cool.

--Benji
Jul 03 2003