• d-bugmail puremagic.com (72/72) Oct 02 2022 https://issues.dlang.org/show_bug.cgi?id=23383

d-bugmail puremagic.com writes:

https://issues.dlang.org/show_bug.cgi?id=23383

          Issue ID: 23383
           Summary: csvReader throws ArrayIndexError when Contents has
                    more fields than header.length
           Product: D
           Version: D2
          Hardware: x86_64
               URL: https://run.dlang.io/?compiler=dmd&args=-unittest&sour
                    ce=void%20main()%0A%7B%0A%20%20%20%20import%20std.csv;
                    %0A%20%20%20%20import%20std.stdio:%20write,%20writeln,
                    %20writef,%20writefln;%0A%20%20%20%20import%20std.algo
                    rithm.comparison%20:%20equal;%0A%20%20%20%20string%20t
                    ext%20%3D%20%22a,b,c%5CnHello,65,true%5CnWorld,123,fal
                    se%22;%0A%20%20%20%20struct%20Layout%0A%20%20%20%20%7B
                    %0A%20%20%20%20%20%20%20%20int%20value;%0A%20%20%20%20
                    %20%20%20%20bool%20other;%0A%20%20%20%20%20%20%20%20st
                    ring%20name;%0A%20%20%20%20%09bool%20ok;%0A%20%20%20%2
                    0%7D%0A%20%20%20%20%0A%20%20%20%20auto%20records%20%3D
                    %20text.csvReader!Layout(%5B%22b%22,%22c%22,%22a%22%5D
                    );%0A%20%20%20%20assert(records.equal(%5B%0A%20%20%20%
                    20%20%20%20%20Layout(65,%20false,%20%22Hello%22),%0A%2
                    0%20%20%20%20%20%20%20Layout(123,%20false,%20%22World%
                    22)%0A%20%20%20%20%5D));%0A%20%20%20%20%0A%20%20%20%20
                    %0A%7D
                OS: Linux
            Status: NEW
          Severity: minor
          Priority: P1
         Component: phobos
          Assignee: nobody puremagic.com
          Reporter: rassoc posteo.de

User mw in the forums stumbled upon this issue: 
https://forum.dlang.org/thread/kownwaxqbcecwefntylj forum.dlang.org

The user's use case and verdict:

"We have a class/struct for a data record, some of its data fields need to be
saved/loaded from CSV files; while there are other helper fields which are
useful for various computation tasks (e.g. caching some intermediate
computation results), these fields do not need to be saved/loaded from the csv
files.
A CSV library should consider all the use cases, and allow users to ignore
certain fields."

Docs say:

"An optional header can be provided. The first record will be read in as the
header. If Contents is a struct then the header provided is expected to
correspond to the fields in the struct."

But this is never enforced and just fails with index error as seen in this
slightly modified docs example:

        ---
        import std.algorithm.comparison : equal;
        string text = "a,b,c\nHello,65,2.5\nWorld,123,7.5";
        struct Layout
        {
                int value;
                double other;
                string name;
                bool ok; // added, now 4 fields, but header.length == 3
        }

        auto records = text.csvReader!Layout(["b","c","a"]);
        assert(records.equal([
                Layout(65, 2.5, "Hello"),
                Layout(123, 7.5, "World")
        ]));
        ---

Error:
core.exception.ArrayIndexError /dlang/dmd/linux/bin64/../../src/phobos/std/csv.d(1209):
index [3] is out of bounds for array of length 3 

Where it fails:
https://github.com/dlang/phobos/blob/8e8aaae5080ccc2e0a2202cbe9778dca96496a95/std/csv.d#L1209

This needs, should the impl remain the same, a nicer assert error message in
the constructer enforcing that content's field count is not higher than
colHeaders.length.

--