www.digitalmars.com         C & C++   DMDScript  

c++.stlsoft - File as string

reply "christopher diggins" <cdiggins videotron.ca> writes:
In order to feed large files (e.g. the ones like Zz uses) to YARD 
efficiently, I am creating a class which allows files to be treated exactly 
like std::string. Am I reinventing the wheel here, and there is something 
else I should know about?

Btw, I got the Imperfect C++ book a few days ago, and I am having fun 
reading it. A very impressive work Matthew!

-- 
Christopher Diggins
Object Oriented Template Library (OOTL)
http://www.ootl.org 
Mar 12 2005
parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"christopher diggins" <cdiggins videotron.ca> wrote in message 
news:d0vvsl$q6l$1 digitaldaemon.com...
 In order to feed large files (e.g. the ones like Zz uses) to YARD 
 efficiently, I am creating a class which allows files to be treated 
 exactly like std::string. Am I reinventing the wheel here, and there 
 is something else I should know about?

Not really sure what you mean. Can you give an example of where you might use it? FYI: both UNIXSTL and WinSTL have the path class (basic_path<char-type> template), which does a great many path-y things. I copied the Boost idea of using / and /= for path concatenation, but I didn't look any deeper into their implementation. New to STLSoft 1.8.3 beta 2 (http://www.stlsoft.org/downloads.html) is the full and proper implememtation of the canonicalise() method, which'll remove and rationalise any embedded "." and ".." within the paths. FYI-2: The two path implementations are almost identical. I plan to refactor that somehow in the future, but haven't given it any detailed thought as yet.
 Btw, I got the Imperfect C++ book a few days ago, and I am having fun 
 reading it. A very impressive work Matthew!

Thanks!
Mar 12 2005
parent reply "christopher diggins" <cdiggins videotron.ca> writes:
"Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message 
news:d100ja$r0q$1 digitaldaemon.com...
 "christopher diggins" <cdiggins videotron.ca> wrote in message 
 news:d0vvsl$q6l$1 digitaldaemon.com...
 In order to feed large files (e.g. the ones like Zz uses) to YARD 
 efficiently, I am creating a class which allows files to be treated 
 exactly like std::string. Am I reinventing the wheel here, and there is 
 something else I should know about?

Not really sure what you mean. Can you give an example of where you might use it?

You would use it in generic code. You pass to your algorithm a template parameter which is either std::string or ootl::file_as_string, and you can use the same code-base for parsing files or for parsing strings. I want to have iterators, and the ability to construct sub-strings, and the other stuff would be fun as well. Think of it as a way, to have arbitrarily large strings stored in secondary storage.
 FYI: both UNIXSTL and WinSTL have the path class (basic_path<char-type> 
 template), which does a great many path-y things. I copied the Boost idea 
 of using / and /= for path concatenation, but I didn't look any deeper 
 into their implementation. New to STLSoft 1.8.3 beta 2 
 (http://www.stlsoft.org/downloads.html) is the full and proper 
 implememtation of the canonicalise() method, which'll remove and 
 rationalise any embedded "." and ".." within the paths.

 FYI-2: The two path implementations are almost identical. I plan to 
 refactor that somehow in the future, but haven't given it any detailed 
 thought as yet.

Thanks for bringing these to my attention. -- Christopher Diggins Object Oriented Template Library (OOTL) http://www.ootl.org
Mar 13 2005
next sibling parent "Matthew" <admin.hat stlsoft.dot.org> writes:
"christopher diggins" <cdiggins videotron.ca> wrote in message
news:d11kmc$29kc$1 digitaldaemon.com...
 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message
news:d100ja$r0q$1 digitaldaemon.com...
 "christopher diggins" <cdiggins videotron.ca> wrote in message
news:d0vvsl$q6l$1 digitaldaemon.com...
 In order to feed large files (e.g. the ones like Zz uses) to YARD efficiently,
I am creating a class which allows 
 files to be treated exactly like std::string. Am I reinventing the wheel here,
and there is something else I should 
 know about?

Not really sure what you mean. Can you give an example of where you might use it?

You would use it in generic code. You pass to your algorithm a template parameter which is either std::string or ootl::file_as_string, and you can use the same code-base for parsing files or for parsing strings. I want to have iterators, and the ability to construct sub-strings, and the other stuff would be fun as well. Think of it as a way, to have arbitrarily large strings stored in secondary storage.

You'd have to give me a small example to get what you mean.
Mar 13 2005
prev sibling parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"christopher diggins" <cdiggins videotron.ca> wrote in message
news:d11kmc$29kc$1 digitaldaemon.com...
 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message
 news:d100ja$r0q$1 digitaldaemon.com...
 "christopher diggins" <cdiggins videotron.ca> wrote in message
 news:d0vvsl$q6l$1 digitaldaemon.com...
 In order to feed large files (e.g. the ones like Zz uses) to YARD
 efficiently, I am creating a class which allows files to be treated
 exactly like std::string. Am I reinventing the wheel here, and there
 is something else I should know about?

Not really sure what you mean. Can you give an example of where you might use it?

You would use it in generic code. You pass to your algorithm a template parameter which is either std::string or ootl::file_as_string, and you can use the same code-base for parsing files or for parsing strings. I want to have iterators, and the ability to construct sub-strings, and the other stuff would be fun as well. Think of it as a way, to have arbitrarily large strings stored in secondary storage.

(One bike ride later and) I think I understand what you mean now. It presents a string interface, but is based on a file. Well, why not use (basic_)string_view coupled to a memory-mapped file? That should provide all that you need, I think. Of course, a little wrapper to that might not be a bad thing. I'd suggest you try it first, and let me know if that satisfies your needs. btw, I'm assuming that you have good grounds for not using istream_iterator??
Mar 13 2005
next sibling parent reply "christopher diggins" <cdiggins videotron.ca> writes:
"Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message 
news:d12d0b$31ca$1 digitaldaemon.com...
 "christopher diggins" <cdiggins videotron.ca> wrote in message
 news:d11kmc$29kc$1 digitaldaemon.com...
 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message
 news:d100ja$r0q$1 digitaldaemon.com...
 "christopher diggins" <cdiggins videotron.ca> wrote in message
 news:d0vvsl$q6l$1 digitaldaemon.com...
 In order to feed large files (e.g. the ones like Zz uses) to YARD
 efficiently, I am creating a class which allows files to be treated
 exactly like std::string. Am I reinventing the wheel here, and there
 is something else I should know about?

Not really sure what you mean. Can you give an example of where you might use it?

You would use it in generic code. You pass to your algorithm a template parameter which is either std::string or ootl::file_as_string, and you can use the same code-base for parsing files or for parsing strings. I want to have iterators, and the ability to construct sub-strings, and the other stuff would be fun as well. Think of it as a way, to have arbitrarily large strings stored in secondary storage.

(One bike ride later and) I think I understand what you mean now. It presents a string interface, but is based on a file.

Yes, well put.
 Well, why not use (basic_)string_view coupled to a memory-mapped file? 
 That should provide all that you need, I think.

I am not familiar with this technique, what is a string_view? I know the theory of memory-mapped files, but I haven't used them. What is a good api for them?
 Of course, a little wrapper to that might not be a bad thing. I'd suggest 
 you try it first, and let me know if that satisfies your needs.

Sure.
 btw, I'm assuming that you have good grounds for not using 
 istream_iterator??

It's not bi-directional. A recursive descent parser moves forwards and backwards a lot, and needs to be able to capture sub-strings from the input. Also didn't you and Zz say that istreams were too slow? -- Christopher Diggins Object Oriented Template Library (OOTL) http://www.ootl.org
Mar 14 2005
next sibling parent reply "christopher diggins" <cdiggins videotron.ca> writes:
Revisiting the problem of inputing to the YARD parser, I realize I simply 
need a forward iterator for files. The STL only provides input iterators for 
file streams which is insufficient. So I have to roll my own. The question 
is whether I would be better off using stdio.h routines for the 
implementation or should I instead using an ifstream based implementation. 
Any suggestions?

Thanks,

-- 
Christopher Diggins
Object Oriented Template Library (OOTL)
http://www.ootl.org 
Mar 14 2005
next sibling parent "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"christopher diggins" <cdiggins videotron.ca> wrote in message 
news:d150mj$2k8a$1 digitaldaemon.com...
 Revisiting the problem of inputing to the YARD parser, I realize I 
 simply need a forward iterator for files. The STL only provides input 
 iterators for file streams which is insufficient. So I have to roll my 
 own. The question is whether I would be better off using stdio.h 
 routines for the implementation or should I instead using an ifstream 
 based implementation. Any suggestions?

To get a forward iterator for a file, you'd have to dup file handles, which would suggest using stdio. To be honest, it'd be a lot more effort that using a string_view, which remains my recommendation. Also, in the general case, duping file handles to enumate Forward Iterator semantics is frought with danger, since the file might change.
Mar 14 2005
prev sibling parent "Zz" <Zz Zz.com> writes:
"christopher diggins" <cdiggins videotron.ca> wrote in message 
news:d150mj$2k8a$1 digitaldaemon.com...
 Revisiting the problem of inputing to the YARD parser, I realize I simply 
 need a forward iterator for files. The STL only provides input iterators 
 for file streams which is insufficient. So I have to roll my own. The 
 question is whether I would be better off using stdio.h routines for the 
 implementation or should I instead using an ifstream based implementation. 
 Any suggestions?

stdio.h, but on the other hand you are creating a library for C++ user and some people might like the idea of having something based on io-streams. Also quite a number new programmers I've meet don't even know what stdio.h is. Matthew would be very helpful here, you should look at the article he wrote for windows developer back in 2003 titled "C# Performance: Comparison with C, C++, D and Java Part 1 & 2 (It was very interesting). Zz
Mar 14 2005
prev sibling parent "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"christopher diggins" <cdiggins videotron.ca> wrote in message 
news:d14d8s$1vvb$1 digitaldaemon.com...
 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message 
 news:d12d0b$31ca$1 digitaldaemon.com...
 "christopher diggins" <cdiggins videotron.ca> wrote in message
 news:d11kmc$29kc$1 digitaldaemon.com...
 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message
 news:d100ja$r0q$1 digitaldaemon.com...
 "christopher diggins" <cdiggins videotron.ca> wrote in message
 news:d0vvsl$q6l$1 digitaldaemon.com...
 In order to feed large files (e.g. the ones like Zz uses) to YARD
 efficiently, I am creating a class which allows files to be 
 treated
 exactly like std::string. Am I reinventing the wheel here, and 
 there
 is something else I should know about?

Not really sure what you mean. Can you give an example of where you might use it?

You would use it in generic code. You pass to your algorithm a template parameter which is either std::string or ootl::file_as_string, and you can use the same code-base for parsing files or for parsing strings. I want to have iterators, and the ability to construct sub-strings, and the other stuff would be fun as well. Think of it as a way, to have arbitrarily large strings stored in secondary storage.

(One bike ride later and) I think I understand what you mean now. It presents a string interface, but is based on a file.

Yes, well put.
 Well, why not use (basic_)string_view coupled to a memory-mapped 
 file? That should provide all that you need, I think.

I am not familiar with this technique, what is a string_view?

(basic_)string_view is a (template) class that presents a String Model interface to the world, but on someone else's storage. It's appeared in STLSoft publicly as of 1.8.3 beta. It is essentially a length + char ptr. One thing I use it for is in combination with STLSoft's string_tokeniser, such that each split token does not have to have a separate block of memory for it. Naturally this makes it more efficient. The obvious downside is that if the underlying memory goes away, then accessing the string view is not a good idea. Also, calling c_str() is to be avoided if possible, since that will have to result in allocation in order to be able to append a nul character. But it sounds ideal for your purposes, because you'd have that memory there, and it'd be little effort to make sure you're not taking it away prematurely.
  I know the theory of memory-mapped files, but I haven't used them. 
 What is a good api for them?

This is on the STLSoft to-do, for both UNIXSTL and WinSTL. For the moment, assuming you're developing on Windows first, just use the long handed way, calling CreateFile(), CreateFileMapping() and MapViewOfFile().
 Of course, a little wrapper to that might not be a bad thing. I'd 
 suggest you try it first, and let me know if that satisfies your 
 needs.

Sure.
 btw, I'm assuming that you have good grounds for not using 
 istream_iterator??

It's not bi-directional.

Of course. Big Doh! to me. :-)
 A recursive descent parser moves forwards and backwards a lot, and 
 needs to be able to capture sub-strings from the input. Also didn't 
 you and Zz say that istreams were too slow?

Can be. :-)
Mar 14 2005
prev sibling parent reply "Zz" <Zz Zz.com> writes:
.. snip..
 (One bike ride later and) I think I understand what you mean now. It
 presents a string interface, but is based on a file.

 Well, why not use (basic_)string_view coupled to a memory-mapped file? 
 That should provide all that you need, I think.

If I recall correctly CreateFileMapping allows only a maximum of just over 2GB under windows.
 Of course, a little wrapper to that might not be a bad thing. I'd suggest 
 you try it first, and let me know if that satisfies your needs.

 btw, I'm assuming that you have good grounds for not using 
 istream_iterator??

I don't like io-streams but wouldn't istreambuf_iterator be better than istream_iterator in the case that io-streams would be used? Zz
Mar 14 2005
parent reply "christopher diggins" <cdiggins videotron.ca> writes:
"Zz" <Zz Zz.com> wrote in message news:d158n7$2sh5$1 digitaldaemon.com...
 I don't like io-streams but wouldn't istreambuf_iterator be better than 
 istream_iterator in the case that io-streams would be used?

Sorry, I forgot to reply to this, istreambuf_iterator is an input iterator, which means you can't use it in multi-pass algorithms (as needed by an R-D parser). I have implemented a forward iterator for files based on fopen, fseek and fread which is available at http://www.ootl.org/ootl/ootl_file_reader.hpp.htm . -- Christopher Diggins Object Oriented Template Library (OOTL) http://www.ootl.org
Mar 19 2005
parent reply "Zz" <Zz Zz.com> writes:
 Sorry, I forgot to reply to this,

 istreambuf_iterator is an input iterator, which means you can't use it in 
 multi-pass algorithms (as needed by an R-D parser). I have implemented a 
 forward iterator for files based on fopen, fseek and fread which is 
 available at http://www.ootl.org/ootl/ootl_file_reader.hpp.htm .

It looks good but... The only problem I personaly have with fseek and ftell is that you must represent the underlying file as a long and this will bring up issues with large files. fsetpos and fgetpos don't have the above limitation on size but you loose random file access. While parsing you may get the current position and allways jump back to the previous point by using fsetpos. Note: while SEEK_END is not really very portable for getting file sizes. Zz
Mar 21 2005
parent reply "christopher diggins" <cdiggins videotron.ca> writes:
"Zz" <Zz Zz.com> wrote in message news:d1lviu$2crf$1 digitaldaemon.com...
 Sorry, I forgot to reply to this,

 istreambuf_iterator is an input iterator, which means you can't use it in 
 multi-pass algorithms (as needed by an R-D parser). I have implemented a 
 forward iterator for files based on fopen, fseek and fread which is 
 available at http://www.ootl.org/ootl/ootl_file_reader.hpp.htm .

It looks good but... The only problem I personaly have with fseek and ftell is that you must represent the underlying file as a long and this will bring up issues with large files.

Ah yes. Are there any compilers which provide 64 bit longs yet? It seems that this is inevitable soon aren't they? I believe that with Visual C++ we could at least use the _fseek64 and _ftell64 for the time being. Do any other compilers provide similar 64 bit implementations of the C runtime libraries? What about ifstream.tellg and ifstream.seekg? How often are they implemented as 64 bits? Specifically what are your most common system compiler combinations?
 fsetpos and fgetpos don't have the above limitation on size but you loose 
 random file access. While parsing you may get the current position and 
 allways jump back to the previous point by using fsetpos.

Ah, I see. Perhaps I can make this work then using a combination of fsetpos and fseek.
 Note: while SEEK_END is not really very portable for getting file sizes.

Again, because of the long problem right? Hmmm..... I have some other tricks up my sleeve. Unfortunately the large file parsing problem will have to be put on hold temporarily, I have to finish up an article and make a release of one of my other libraries. Thanks, -D
Mar 21 2005
parent reply "Zz" <Zz Zz.com> writes:
.. snip..
 Ah yes. Are there any compilers which provide 64 bit longs yet? It seems 
 that this is inevitable soon aren't they? I believe that with Visual C++ 
 we could at least use the _fseek64 and _ftell64 for the time being. Do any 
 other compilers provide similar 64 bit implementations of the C runtime 
 libraries?

Metrowerks does not have it.
 What about ifstream.tellg and ifstream.seekg? How often are they 
 implemented as 64 bits? Specifically what are your most common system 
 compiler combinations?

I maily use Metrowerks, I'm just used to it and I think it has the best C/C++ debugger out there. I looked at Visual Studio 2005 Beta, and I liked how it understood STL containers. For final compilation Intel may be used.
 fsetpos and fgetpos don't have the above limitation on size but you loose 
 random file access. While parsing you may get the current position and 
 allways jump back to the previous point by using fsetpos.

Ah, I see. Perhaps I can make this work then using a combination of fsetpos and fseek.

The only time I really need to jump to a specific part of the file is when handling images, when doing regular text parsing I really don't jump to any part of the file since at the most it will be an LL(2) parser. Even with memory mapped files jumping to different portions can affect performance (thats why both windows and posix have special optimizations for sequential access).
 Note: while SEEK_END is not really very portable for getting file sizes.


This may come as a supprise but a C89 compiler is not required to support SEEK_END for files opened with "rb".
 Again, because of the long problem right?

 Hmmm..... I have some other tricks up my sleeve. Unfortunately the large 
 file parsing problem will have to be put on hold temporarily, I have to 
 finish up an article and make a release of one of my other libraries.

I totally understand (it's better to get thing rolling and then attacking the problem later on). Cheers, Zz
Mar 21 2005
parent reply "christopher diggins" <cdiggins videotron.ca> writes:
"Zz" <Zz Zz.com> wrote in message news:d1nkmd$10hv$1 digitaldaemon.com...
 Note: while SEEK_END is not really very portable for getting file sizes.


This may come as a supprise but a C89 compiler is not required to support SEEK_END for files opened with "rb".

Thanks for telling me about that. I did not know.
 Again, because of the long problem right?

 Hmmm..... I have some other tricks up my sleeve. Unfortunately the large 
 file parsing problem will have to be put on hold temporarily, I have to 
 finish up an article and make a release of one of my other libraries.

I totally understand (it's better to get thing rolling and then attacking the problem later on).

Speaking of which, have you had a chance to try the latest release? -- Christopher Diggins Object Oriented Template Library (OOTL) http://www.ootl.org
Mar 22 2005
parent reply "Zz" <Zz Zz.com> writes:
 Hmmm..... I have some other tricks up my sleeve. Unfortunately the large 
 file parsing problem will have to be put on hold temporarily, I have to 
 finish up an article and make a release of one of my other libraries.

I totally understand (it's better to get thing rolling and then attacking the problem later on).

Speaking of which, have you had a chance to try the latest release?

Zz
Mar 24 2005
parent reply "christopher diggins" <cdiggins videotron.ca> writes:
"Zz" <Zz Zz.com> wrote in message news:d1vgsm$sgo$1 digitaldaemon.com...
 Hmmm..... I have some other tricks up my sleeve. Unfortunately the 
 large file parsing problem will have to be put on hold temporarily, I 
 have to finish up an article and make a release of one of my other 
 libraries.

I totally understand (it's better to get thing rolling and then attacking the problem later on).

Speaking of which, have you had a chance to try the latest release?

Zz

Thanks. I had rewritten the large file reader using ifstream routines recently and learned the hard way some new things. At least for gcc 3.4 the following appears to be true: - ifstream.readsome always returns 0 - ifstream.seekg(0, ios::end) fails for large files - if you use ifstream.read at the end of the file, calls to ifstream.tellg() fails This really ties my hands as to what I can do. I can probably rewrite it again to make things work using FILE* routines, but I need to know whether fread(FILE, 0) will work dependably for large files. Can you give me any hints or advice? -- Christopher Diggins Object Oriented Template Library (OOTL) http://www.ootl.org
Mar 25 2005
parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
Chris

Have you tried using string_view + mmf yet?

If you're interested, I'd be happy to sketch out a skeleton example to show how
it'd be used.

Cheers

Matthew

"christopher diggins" <cdiggins videotron.ca> wrote in message
news:d217hl$2uk3$1 digitaldaemon.com...
 "Zz" <Zz Zz.com> wrote in message news:d1vgsm$sgo$1 digitaldaemon.com...
 Hmmm..... I have some other tricks up my sleeve. Unfortunately the large file
parsing problem will have to be put 
 on hold temporarily, I have to finish up an article and make a release of one
of my other libraries.

I totally understand (it's better to get thing rolling and then attacking the problem later on).

Speaking of which, have you had a chance to try the latest release?

Zz

Thanks. I had rewritten the large file reader using ifstream routines recently and learned the hard way some new things. At least for gcc 3.4 the following appears to be true: - ifstream.readsome always returns 0 - ifstream.seekg(0, ios::end) fails for large files - if you use ifstream.read at the end of the file, calls to ifstream.tellg() fails This really ties my hands as to what I can do. I can probably rewrite it again to make things work using FILE* routines, but I need to know whether fread(FILE, 0) will work dependably for large files. Can you give me any hints or advice? -- Christopher Diggins Object Oriented Template Library (OOTL) http://www.ootl.org

Mar 25 2005
parent reply "christopher diggins" <cdiggins videotron.ca> writes:
"Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message 
news:d2214j$rej$1 digitaldaemon.com...
 Chris

 Have you tried using string_view + mmf yet?

 If you're interested, I'd be happy to sketch out a skeleton example to 
 show how it'd be used.

Thanks, but I haven't tried yet because Zz mentioned there is a 2(or was it 4) GIG limit on MMF's in windows. Can you either confirm or affirm this? Do you know if fread() is reliable? -D
Mar 27 2005
next sibling parent reply "christopher diggins" <cdiggins videotron.ca> writes:
"christopher diggins" <cdiggins videotron.ca> wrote in message 
news:d26ker$2gma$1 digitaldaemon.com...
 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message 
 news:d2214j$rej$1 digitaldaemon.com...
 Chris

 Have you tried using string_view + mmf yet?

 If you're interested, I'd be happy to sketch out a skeleton example to 
 show how it'd be used.

Thanks, but I haven't tried yet because Zz mentioned there is a 2(or was it 4) GIG limit on MMF's in windows. Can you either confirm or affirm this?

^^^^^^ refute Sorry. Too early in the morning.
Mar 27 2005
parent "Zz" <Zz Zz.com> writes:
Here is the info from MATLAB on memory mapped files.

 Maximum Size of a Memory Map.   Due to limits set by the operating system, 
the maximum amount of data you can map with a single instance of a memory 
map is 2^31 - 1 (or 2 GB). If you need to map more than 2 GB, you can either 
create separate maps for different regions of the file, or you can move the 
2 GB window of one map to different locations in the file.
The 2 GB limit also applies to 64-bit platforms. However, because 64-bit 
platforms have a much larger address space, they can support having many 
more map instances in memory at any given time.

Aligned Access on Sol2 and HP-UX.   The Sol2 and HP-UX platforms only 
support aligned data access. This means that numeric values of type double 
that are to be read from a memory-mapped file must start at some multiple of 
8 bytes from the start of the file. (Note that this is from the start of the 
file, and not the start of the mapped region.) Furthermore, numeric values 
of type single and also 32-bit integers must start at multiples of 4 bytes, 
and 16-bit integers at 2-byte multiples.

If you attempt to map a file on Sol2 or HP-UX that does not take into 
account these alignment considerations, MATLAB generates an error.


"christopher diggins" <cdiggins videotron.ca> wrote in message 
news:d26ku8$2h3q$1 digitaldaemon.com...
 "christopher diggins" <cdiggins videotron.ca> wrote in message 
 news:d26ker$2gma$1 digitaldaemon.com...
 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message 
 news:d2214j$rej$1 digitaldaemon.com...
 Chris

 Have you tried using string_view + mmf yet?

 If you're interested, I'd be happy to sketch out a skeleton example to 
 show how it'd be used.

Thanks, but I haven't tried yet because Zz mentioned there is a 2(or was it 4) GIG limit on MMF's in windows. Can you either confirm or affirm this?

^^^^^^ refute Sorry. Too early in the morning.

Mar 27 2005
prev sibling parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"christopher diggins" <cdiggins videotron.ca> wrote in message
news:d26ker$2gma$1 digitaldaemon.com...
 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message
news:d2214j$rej$1 digitaldaemon.com...
 Chris

 Have you tried using string_view + mmf yet?

 If you're interested, I'd be happy to sketch out a skeleton example to show
how it'd be used.

Thanks, but I haven't tried yet because Zz mentioned there is a 2(or was it 4) GIG limit on MMF's in windows. Can you either confirm or affirm this?

I can't offhand, but it does sound familiar. I was more meaning that string_view + mmf was an option, rather than the complete solution.
 Do you know if fread() is reliable?

In what sense? Cheers Matthew
Mar 27 2005
parent reply "christopher diggins" <cdiggins videotron.ca> writes:
"Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message 
news:d27bcb$6sp$1 digitaldaemon.com...
 I can't offhand, but it does sound familiar.

 I was more meaning that string_view + mmf was an option, rather than the 
 complete solution.

Definitely I want to support "string_view + mmf" as an option. If you could post the skeleton example, I will write a yard example using it.
 Do you know if fread() is reliable?

In what sense?

Does it do what it is supposed to in most cases, on most platforms. For instance, Zz pointed out that fseekg(f, 0, SEEK_END) isn't specifically require to do what it is supposed to. I just learned the hard way that ifstream.seekg(0, ios::end) doesn't work on large files. So what I am asking is specifically, whether fread will do what is supposed to on most platforms, even at the end of extremely large files. Thanks a lot, Christopher Diggins http://www.ootl.org
Mar 28 2005
parent "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"christopher diggins" <cdiggins videotron.ca> wrote in message
news:d29b60$2a4h$1 digitaldaemon.com...
 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message
news:d27bcb$6sp$1 digitaldaemon.com...
 I can't offhand, but it does sound familiar.

 I was more meaning that string_view + mmf was an option, rather than the
complete solution.

Definitely I want to support "string_view + mmf" as an option. If you could post the skeleton example, I will write a yard example using it.

Done. Posted.
 Do you know if fread() is reliable?

In what sense?

Does it do what it is supposed to in most cases, on most platforms. For instance, Zz pointed out that fseekg(f, 0, SEEK_END) isn't specifically require to do what it is supposed to. I just learned the hard way that ifstream.seekg(0, ios::end) doesn't work on large files. So what I am asking is specifically, whether fread will do what is supposed to on most platforms, even at the end of extremely large files.

My assumption is that if off_t is 64-bit, then fseek() will work for 64-bit files. If not, then it won't. But that's all supposition. The only times I've worked with 64-bit-sized files is on Win32, whence I've used the Win32 API.
Mar 28 2005