www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Read csv data into a struct

reply tjb <broughtj gmail.com> writes:
Hello,

I am trying to read some data from a csv file into a struct.  I'm just learning
to use
D.  This little bit of code reads the data and prints to standard output just
fine:

import std.stdio : writeln;
import std.stream;

void main() {
  auto fin = new File("temp.csv");
  char[] line;

  int count;
  while(!fin.eof()) {
    line = fin.readLine();
    writeln(line);
  }

  auto dt1 = SysTime.from
}

But I don't know how to parse the input to a structure. Another complication is
that
some of my fields are dates and times (possibly with non-standard formats). Can
you
suggest an approach?

Here are the first few lines of output from the code above:

1/2/08,9:30:07,Q,70780,X,A,4.75,61,5,71,N,R,36.68,1,36.73,1
1/2/08,9:30:08,Q,73028,X,A,4.75,61,5,61,N,R,36.68,1,36.73,1
1/2/08,9:30:09,Q,79462,X,A,4.75,73,5,88,N,R,36.6,2,36.73,1
1/2/08,9:30:10,Q,86182,A,,4.75,11,5,11,N,R,36.61,2,36.71,1
1/2/08,9:30:10,Q,87054,X,A,4.75,73,5,76,N,R,36.61,2,36.71,1

Thanks!

TJB
Mar 02 2012
parent reply tjb <tyler.brough usu.edu> writes:
Woops.  I have a mistake in the code.  Should be:

import std.stdio : writeln;
import std.stream;

void main() {
  auto fin = new File("temp.csv");
  char[] line;

  int count;
  while(!fin.eof()) {
    line = fin.readLine();
    writeln(line);
  }
}

Thanks!

TJB
Mar 02 2012
parent reply "Yao Gomez" <yao.gomez gmail.com> writes:
On Friday, 2 March 2012 at 21:50:12 UTC, tjb wrote:

 Woops.  I have a mistake in the code.  Should be:

 import std.stdio : writeln;
 import std.stream;

 void main() {
   auto fin = new File("temp.csv");
   char[] line;

   int count;
   while(!fin.eof()) {
     line = fin.readLine();
     writeln(line);
   }
 }

 Thanks!

 TJB


Check this module: http://dlang.org/phobos/std_csv.html

It can parse the CVS file content into a custom class/struct.
Mar 02 2012
next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 03/02/2012 02:01 PM, Yao Gomez wrote:

 On Friday, 2 March 2012 at 21:50:12 UTC, tjb wrote:

 Woops. I have a mistake in the code. Should be:

 import std.stdio : writeln;
 import std.stream;

 void main() {
 auto fin = new File("temp.csv");
 char[] line;

 int count;
 while(!fin.eof()) {
 line = fin.readLine();
 writeln(line);
 }
 }

 Thanks!

 TJB


 Check this module: http://dlang.org/phobos/std_csv.html

 It can parse the CVS file content into a custom class/struct.


Thanks, that's very helpful.

1) I have modified the example from that module's documentation to take 
advantage of tuple expansions in foreach loops. Notice firstName, etc. 
instead of record[0], record[1], etc. (This is a relatively new feature.)

2) I've also used a struct instead of a tuple. (This is what TJB wants.)

3) And finally created an array of objects.

import std.csv;
import std.stdio;
import std.typecons;
import std.array;

struct Worker
{
     string firstName;
     string occupation;
     int amount;
}

void main()
{
     auto text = "Joe,Carpenter,300000\nFred,Blacksmith,400000\r\n";

     /* Take advantage of tuple expansion in foreach loops */
     foreach(firstName, occupation, amount;
             csvReader!(Tuple!(string,string,int))(text))
     {
         writefln("%s works as a %s and earns $%d per year",
                  firstName, occupation, amount);
     }

     /* Use a struct instead of a tuple */
     foreach (worker; csvReader!Worker(text)) {
         writeln("Worker in foreach: ", worker);
     }

     /* Make an array of Workers directly from the data */
     Worker[] workers = array(csvReader!Worker(text));
     writeln("Workers in array: ", workers);
}

Ali
Mar 02 2012
parent reply tjb <tyler.brough usu.edu> writes:
Ali,

Thanks.  That helps me see how to use a structure.  I just need to
figure out the date and time conversion.

Thanks!

TJB
Mar 02 2012
parent reply "Yao Gomez" <yao.gomez gmail.com> writes:
On Friday, 2 March 2012 at 22:56:49 UTC, tjb wrote:

 Ali,

 Thanks.  That helps me see how to use a structure.  I just need 
 to
 figure out the date and time conversion.

 Thanks!

 TJB


I don't remember who did it or where is located, but there's 
actually a project/code that can be used for convert from 
different kind of date and time string representations, to 
instances of Date and Time from the std.datetime library 
(parseDate or something like that).

Let me search for it. I even remember that there was a dicussion 
post about it.
Mar 02 2012
parent "Jesse Phillips" <jessekphillips+D gmail.com> writes:
On Saturday, 3 March 2012 at 00:03:02 UTC, Yao Gomez wrote:

 On Friday, 2 March 2012 at 22:56:49 UTC, tjb wrote:

 Ali,

 Thanks.  That helps me see how to use a structure.  I just 
 need to
 figure out the date and time conversion.

 Thanks!

 TJB


 I don't remember who did it or where is located, but there's 
 actually a project/code that can be used for convert from 
 different kind of date and time string representations, to 
 instances of Date and Time from the std.datetime library 
 (parseDate or something like that).

 Let me search for it. I even remember that there was a 
 dicussion post about it.


It is a conversion of the deprecated dateparse. There is going to 
be some new added to std but for now I guess this is all

https://gist.github.com/1283011
Mar 02 2012
prev sibling parent tjb <tyler.brough usu.edu> writes:
Yao,

Thanks.  That looks perfect.  I'll play with it until I figure it out.

Any suggestions for the date and time variables?

Thanks again!

TJB
Mar 02 2012