www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 7972] New: std.file.read allocate a buffer the size of the file even when given a upper limit

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7972

           Summary: std.file.read allocate a buffer the size of the file
                    even when given a upper limit
           Product: D
           Version: future
          Platform: All
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Phobos
        AssignedTo: nobody puremagic.com
        ReportedBy: tbanelwebmin free.fr


--- Comment #0 from tbanelwebmin free.fr 2012-04-23 02:19:18 PDT ---
import std.file;
read ("/path/to/bigfile", 1024);

a core.exception.OutOfMemoryError is thrown.

--------

This is because a buffer the size of "/path/to/bigfile" is allocated,
instead of 1024

--------

Fix

Go to file std/file.d line 327
change maxInitialAlloc to minInitialAlloc

When done, the maxInitialAlloc is no longer usefull and may be removed.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 23 2012
parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7972



--- Comment #1 from tbanelwebmin free.fr 2013-01-11 13:50:05 PST ---
OutOfMemoryError is still there in version 2.061
calling:
   read("/path/to/bigfile", 1024);

The function
   void[] read(in char[] name, size_t upTo)
is supposed to return at most "upTo" bytes,
even for a very large file.

But internally, in the Posix version,
the allocated buffer is the size of the file
(line 222:
   immutable initialAlloc = ...
 )

The fix is to consider "upTo" when computing "initialAlloc":
  immutable initialAlloc = to!size_t(min(upTo,statbuf.st_size
      ? min(statbuf.st_size + 1, maxInitialAlloc)
      : minInitialAlloc));

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 11 2013