• bearophile (14/14) Nov 22 2010 I have just found some slides that show DataRush, a library that allows ...

bearophile <bearophileHUGS lycos.com> writes:

I have just found some slides that show DataRush, a library that allows to use
in Java another programming paradigm (dataflow programming):

http://qconsf.com/sf2010/file?path=/qcon-sanfran-2010/slides/JimFalgout_DataflowProgrammingAScalableDataCentricApproachToParallelism.pdf

The dataflow code for the Java DataRush library doesn't look too much bad, and
there is even a GUI to draw it:

ApplicationGraph app = GraphFactory.newApplicationGraph();
ReadDelimitedTextProperties rdprops = ...
RecordFlow leftFlow = app.add(new ReadDelimitedText("UnitPriceSorted.txt",
rdprops), "readLeft").getOutput();
RecordFlow rightFlow = app.add(new ReadDelimitedText("UnitSalesSorted.txt",
rdprops), "readRight").getOutput();
String[] keyNames = { "PRODUCT_ID", "CHANNEL_NAME" };
RecordFlow joinedFlow = app.add(new JoinSortedRows(leftFlow, rightFlow,
FULL_OUTER, keyNames)).getOutput();
app.add(new WriteDelimitedText(joinedFlow, “output.txt", WriteMode.OVERWRITE),
"write");
app.run();

A similar library that allows similar code is probably possible in D2 too.

Bye,
bearophile