www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Tempfiles in unittests

reply Magnus Lie Hetland <magnus hetland.org> writes:
I'm testing a class that writes various things to a file, and I'd like 
to test that behavior, and I'm wondering what the standard solution 
here is.

In Python, I'd just use a StringIO class, for example. It seems sort of 
impractical to make std.outbuffer to work as a File (even if the 
file-like parameter is a template parameter, the outbuffer methods 
don't work like the File methods, as far as I can see), so I've 
concluded that using std.stdio.tmpfile is the way to go.

However, when I'm done with the tempfile, I'd like to have its contents 
as a string. It seems that File has no method for reading the entire 
file contents into a string, so I'd have to read and concatenate lines 
or chunks or something? Now, if the File returned by tmpfile had a 
name, I could use std.file.readText(). But no such luck (as far as I 
can see, tmpfile().name is null).

So I'm just wondering -- what's "the way" to do this sort of testing? 
And if it is along the lines of what I'm doing, is there some 
function/method for reading the contents of a File as a string?

(I guess this goes for much file-related functionality; much of it 
seems to be very related to file-names and actual files, rather than 
"file-like" objects, i.e., those with the appropriate methods...)

-- 
Magnus Lie Hetland
http://hetland.org
Mar 09 2012
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Magnus Lie Hetland:

 It seems that File has no method for reading the entire file 
 contents into a string, so I'd have to read and concatenate 
 lines or chunks or something?

There are std.file.read() and std.file.readText(). But I agree that std.stdio.File should have a readAll() method :-) If you think this is a good idea, add it to Bugzilla. Bye, bearophile
Mar 09 2012
parent reply Magnus Lie Hetland <magnus hetland.org> writes:
On 2012-03-09 15:08:42 +0000, bearophile said:

 Magnus Lie Hetland:
 
 It seems that File has no method for reading the entire file contents 
 into a string, so I'd have to read and concatenate lines or chunks or 
 something?

There are std.file.read() and std.file.readText().

Yeah, I found those. I guess my post was a bit verbose, but the main point was that I have a File object, but no file name (because I'm using std.stdio.tmpfile() in a unit test), so std.file.read() and std.file.readText() are useless to me... :)
 But I agree that std.stdio.File should have a readAll() method
 :-) If you think this is a good idea, add it to Bugzilla.

Sure. For now, though: What's The Way[TM] to write unit tests for code that writes to files? Use tmpfile and read/concatenate the file contents myself? Something else that's more practical? (I would have thought this kind of test would be common?-) -- Magnus Lie Hetland http://hetland.org
Mar 09 2012
parent reply Magnus Lie Hetland <magnus hetland.org> writes:
On 2012-03-09 18:42:01 +0000, Jonathan M Davis said:

 File has a name property. You can do something like
 
 auto file = File.tmpfile();
 auto filename = file.name;
 auto str = readText(filename);

Yeah -- again, sorry my original post was so verbose, but part of the "problem spec" was that my tempfiles don't seem to have names (i.e., they are unnamed tempfiles, and file.name is null; platform-dependent, I think). Otherwise, this wouldn't have been a problem :) -- Magnus Lie Hetland http://hetland.org
Mar 10 2012
parent reply Magnus Lie Hetland <magnus hetland.org> writes:
On 2012-03-10 17:10:52 +0000, Jonathan M Davis said:

 Otherwise, this wouldn't have been a problem :)

Then you'll probably have to write a wrapper function which just uses byLine and join to grab the string line by line and then put it back together again.

Indeed. Not a problem. Just surprised me that there wasn't a standard way of writing file-related unit tests :) -- Magnus Lie Hetland http://hetland.org
Mar 11 2012
next sibling parent Ary Manzana <ary esperanto.org.ar> writes:
On 03/11/2012 06:14 PM, Jonathan M Davis wrote:
 On Sunday, March 11, 2012 13:33:34 Magnus Lie Hetland wrote:
 On 2012-03-10 17:10:52 +0000, Jonathan M Davis said:
 Otherwise, this wouldn't have been a problem :)

Then you'll probably have to write a wrapper function which just uses byLine and join to grab the string line by line and then put it back together again.

Just surprised me that there wasn't a standard way of writing file-related unit tests :)

Well, most of the file-related unit tests in Phobos don't care much about the contents of the file, and if they do, it's becaus they're testing one of the functions for reading one (such as byLine or read), in which case, they're not looking to read in the whole file at once. So, as far as Phobos' unit tests are concerned, there's not really any need for what you're trying to do. That doesn't mean that it shouldn't be supported, but it just doesn't appear to have ever come up. - Jonathan M Davis

What real world product did the designers of the D language do?
Mar 11 2012
prev sibling parent Magnus Lie Hetland <magnus hetland.org> writes:
On 2012-03-11 21:14:59 +0000, Jonathan M Davis said:

 So, as far as Phobos' unit tests are  concerned, there's not really any 
 need for what you're trying to do.

Sure. Just would have thought that maybe some unit-testing D user out there had written a function that wrote something to a file ;-) But no problem -- not many lines to add to my own code. -- Magnus Lie Hetland http://hetland.org
Mar 12 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 09, 2012 16:48:33 Magnus Lie Hetland wrote:
 On 2012-03-09 15:08:42 +0000, bearophile said:
 Magnus Lie Hetland:
 It seems that File has no method for reading the entire file contents
 into a string, so I'd have to read and concatenate lines or chunks or
 something?

There are std.file.read() and std.file.readText().

Yeah, I found those. I guess my post was a bit verbose, but the main point was that I have a File object, but no file name (because I'm using std.stdio.tmpfile() in a unit test), so std.file.read() and std.file.readText() are useless to me... :)

File has a name property. You can do something like auto file = File.tmpfile(); auto filename = file.name; auto str = readText(filename); I do grant you though that it's a bit odd to need to get the file name and operate on that rather than operating on the File object. It works though, and is a lot cleaner than reading in the file line-by-line, if you don't actually need to do that. - Jonathan M Davis
Mar 09 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Mar 09, 2012 at 01:42:01PM -0500, Jonathan M Davis wrote:
 On Friday, March 09, 2012 16:48:33 Magnus Lie Hetland wrote:
 On 2012-03-09 15:08:42 +0000, bearophile said:
 Magnus Lie Hetland:
 It seems that File has no method for reading the entire file
 contents into a string, so I'd have to read and concatenate lines
 or chunks or something?

There are std.file.read() and std.file.readText().

Yeah, I found those. I guess my post was a bit verbose, but the main point was that I have a File object, but no file name (because I'm using std.stdio.tmpfile() in a unit test), so std.file.read() and std.file.readText() are useless to me... :)

File has a name property. You can do something like auto file = File.tmpfile(); auto filename = file.name; auto str = readText(filename); I do grant you though that it's a bit odd to need to get the file name and operate on that rather than operating on the File object. It works though, and is a lot cleaner than reading in the file line-by-line, if you don't actually need to do that.

This seems like a strange limitation. Shouldn't it be possible to slurp the entire contents of an already-open file into a string? Perhaps a new method in File? Also, using file.name is prone to race conditions in certain situations. (Not that it'd be a problem usually, but just sayin'.) T -- Designer clothes: how to cover less by paying more.
Mar 09 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 09, 2012 11:34:07 H. S. Teoh wrote:
 On Fri, Mar 09, 2012 at 01:42:01PM -0500, Jonathan M Davis wrote:
 On Friday, March 09, 2012 16:48:33 Magnus Lie Hetland wrote:
 On 2012-03-09 15:08:42 +0000, bearophile said:
 Magnus Lie Hetland:
 It seems that File has no method for reading the entire file
 contents into a string, so I'd have to read and concatenate lines
 or chunks or something?

There are std.file.read() and std.file.readText().

Yeah, I found those. I guess my post was a bit verbose, but the main point was that I have a File object, but no file name (because I'm using std.stdio.tmpfile() in a unit test), so std.file.read() and std.file.readText() are useless to me... :)

File has a name property. You can do something like auto file = File.tmpfile(); auto filename = file.name; auto str = readText(filename); I do grant you though that it's a bit odd to need to get the file name and operate on that rather than operating on the File object. It works though, and is a lot cleaner than reading in the file line-by-line, if you don't actually need to do that.

[...] This seems like a strange limitation. Shouldn't it be possible to slurp the entire contents of an already-open file into a string? Perhaps a new method in File? Also, using file.name is prone to race conditions in certain situations. (Not that it'd be a problem usually, but just sayin'.)

Of course it's possible to read in the open file into a string. It's just that the only function in Phobos for doing that is in std.file and does not operate on a File. All of the File functions read in only a portion of the file. So, either a new function would have to be added to File, or you'd have to write one yourself (which wouldn't be all that hard using what's already there). - Jonathan M Davis
Mar 09 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, March 10, 2012 15:44:26 Magnus Lie Hetland wrote:
 On 2012-03-09 18:42:01 +0000, Jonathan M Davis said:
 File has a name property. You can do something like
 
 auto file = File.tmpfile();
 auto filename = file.name;
 auto str = readText(filename);

Yeah -- again, sorry my original post was so verbose, but part of the "problem spec" was that my tempfiles don't seem to have names (i.e., they are unnamed tempfiles, and file.name is null; platform-dependent, I think). Otherwise, this wouldn't have been a problem :)

Then you'll probably have to write a wrapper function which just uses byLine and join to grab the string line by line and then put it back together again. As annoying as it may be, it wouldn't be hard. - Jonathan M Davis
Mar 10 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, March 11, 2012 13:33:34 Magnus Lie Hetland wrote:
 On 2012-03-10 17:10:52 +0000, Jonathan M Davis said:
 Otherwise, this wouldn't have been a problem :)

Then you'll probably have to write a wrapper function which just uses byLine and join to grab the string line by line and then put it back together again.

Just surprised me that there wasn't a standard way of writing file-related unit tests :)

Well, most of the file-related unit tests in Phobos don't care much about the contents of the file, and if they do, it's becaus they're testing one of the functions for reading one (such as byLine or read), in which case, they're not looking to read in the whole file at once. So, as far as Phobos' unit tests are concerned, there's not really any need for what you're trying to do. That doesn't mean that it shouldn't be supported, but it just doesn't appear to have ever come up. - Jonathan M Davis
Mar 11 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Mar 11, 2012 at 02:14:59PM -0700, Jonathan M Davis wrote:
 On Sunday, March 11, 2012 13:33:34 Magnus Lie Hetland wrote:
 On 2012-03-10 17:10:52 +0000, Jonathan M Davis said:
 Otherwise, this wouldn't have been a problem :)

Then you'll probably have to write a wrapper function which just uses byLine and join to grab the string line by line and then put it back together again.

Just surprised me that there wasn't a standard way of writing file-related unit tests :)

Well, most of the file-related unit tests in Phobos don't care much about the contents of the file, and if they do, it's becaus they're testing one of the functions for reading one (such as byLine or read), in which case, they're not looking to read in the whole file at once. So, as far as Phobos' unit tests are concerned, there's not really any need for what you're trying to do. That doesn't mean that it shouldn't be supported, but it just doesn't appear to have ever come up.

I think he's talking about unit tests in general D code, not unittests in Phobos. T -- Heuristics are bug-ridden by definition. If they didn't have bugs, they'd be algorithms.
Mar 11 2012
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Mar 12, 2012 at 01:54:08PM +0100, Magnus Lie Hetland wrote:
 On 2012-03-11 21:14:59 +0000, Jonathan M Davis said:
 
So, as far as Phobos' unit tests are  concerned, there's not
really any need for what you're trying to do.

Sure. Just would have thought that maybe some unit-testing D user out there had written a function that wrote something to a file ;-)

I've written unittests that create files to test code with. However, I never thought far enough to make it something generic that can be used elsewhere. :-) So I use actual filenames (not tmpfiles) so that later unittests can reference actual filenames to load different datasets into the program for testing. Not the nicest solution, since it will overwrite files in the working directory, but presumably only developers will bother running unittests anyway so this shouldn't be a problem. T -- Truth, Sir, is a cow which will give [skeptics] no more milk, and so they are gone to milk the bull. -- Sam. Johnson
Mar 12 2012