www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Re: streaming redux

Andrei Alexandrescu Wrote:

 On 12/28/10 11:39 AM, Michel Fortin wrote:

 But what I'd like to see is something
 like this:

 interface Serializable(F) {
 void writeTo(F formatter);

Let me make sure I understand correctly. So when I define a class I commit to its possible representations? Doesn't seem good design to me. What if I later come with a new Formatter? I'd need to change my entire class hierarchy too.

What about defining this trait externally? It would limit the formatter to accessing public data members though.
 Any object can implement a serialization for a given formatter by
 implementing the interface above parametrized with the formatter type.

If only one formatter would be allowed that would be even worse. But you can allow several: class Widget : Serializable!Json, Serializable!Binary { ... } Sorry, I think this is poor design.

I think a distinction should be drawn between unstructured formats (compression, encryption) and structured formats (json, xml, csv). In the latter case, each piece of data written may need a label, there may be some context-specific separation between elements, etc. One obviously knows the desired serialization structure at design time, so the issue is how to achieve it within the streaming mechanism. I've encountered two cases: first, where I'm serializing a set of objects in code I own that has a direct relation to the serialized structure, and second, where I don't have such distinct chunks of serializable data in memory and I assemble the output from more granular data. Steve's formatter works very well for the first case, and this is by far the most common (I can't think of a single case where I've needed to format data objects that I can't alter, ie. from a third-party library). The latter is mostly an issue when a distinct serialized element is quite large and/or the app is generating the output somehow. I tend to work entirely with the last category of data and so don't expect any in-stream formatter to work for me, but the closer it can get the better :-) This could be tabular data where each row is quite large (a CSV stream needs some way to denote the end of a row for the newline, for example), input translated dynamically from another source and pumped to an output stream in some structured format like XML or JSON, etc. The problem with supporting this design with a stream formatter is that it often only works for output--unformatting the input typically requires a parser of some sort. It makes for one task a novice programmer can do (writing output), but the asymmetry is a bit weird from a stream design perspective.
 So basically, I'm not proposing you dump Formatter, just that you make
 it part of a reusable pattern for
 formatting/serializing/unformatting/unserializing things using other
 things that your Formatter interface.

I may be misunderstanding, but to me it seems that this design brings more problems than it solves.

It solves the (common) first problem above of reading/writing structured data formats where the data is available in-memory. That covers quite a lot.
Dec 29 2010