www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.compress

reply Walter Bright <newshound2 digitalmars.com> writes:
https://github.com/WalterBright/phobos/blob/std_compress/std/compress.d

I wrote this to add components to compress and expand ranges.

Highlights:

1. doesn't do any memory allocation
2. can handle arbitrarily large sets of data
3. it's lazy
4. takes an InputRange, and outputs an InputRange

Comments welcome.
Jun 03 2013
next sibling parent reply "Diggory" <diggsey googlemail.com> writes:
On Tuesday, 4 June 2013 at 03:44:05 UTC, Walter Bright wrote:
 https://github.com/WalterBright/phobos/blob/std_compress/std/compress.d

 I wrote this to add components to compress and expand ranges.

 Highlights:

 1. doesn't do any memory allocation
 2. can handle arbitrarily large sets of data
 3. it's lazy
 4. takes an InputRange, and outputs an InputRange

 Comments welcome.

Nice! What happens if R is not a ubyte range?
Jun 03 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/3/2013 10:41 PM, Diggory wrote:
 Nice! What happens if R is not a ubyte range?

It'll work with char and ubyte, too. Anything else you'll need to cast or use an adapter.
Jun 03 2013
prev sibling next sibling parent reply Timothee Cour <thelastmammoth gmail.com> writes:
--089e0141abac10303404de4e5ac2
Content-Type: text/plain; charset=ISO-8859-1

A)
there already is std.zlib; why not have:
std.compress.zlib: public import std.zlib
std.compress.lzw: put this new module there instead of in std.compress
std.compress.image.png
std.compress.image.jpg

B)
rename:
std.compress.lzwCompress => std.compress.lzw.compress
std.compress. lzwExpand => std.compress.lzw.uncompress

which is more consistent with compress/uncompress from std.zlib

C)
maybe add a link to
https://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Welch or other
source

D)
CircularBuffer belongs somewhere else; maybe std.range or std.container





On Mon, Jun 3, 2013 at 8:44 PM, Walter Bright <newshound2 digitalmars.com>wrote:

 https://github.com/**WalterBright/phobos/blob/std_**
 compress/std/compress.d<https://github.com/WalterBright/phobos/blob/std_compress/std/compress.d>

 I wrote this to add components to compress and expand ranges.

 Highlights:

 1. doesn't do any memory allocation
 2. can handle arbitrarily large sets of data
 3. it's lazy
 4. takes an InputRange, and outputs an InputRange

 Comments welcome.

--089e0141abac10303404de4e5ac2 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable A)<div>there already is std.zlib; why not have:</div><div><div>std.compress= .zlib: public import std.zlib</div></div><div><div>std.compress.lzw: put th= is new module there instead of in std.compress</div></div><div><div>std.com= press.image.png</div> </div><div><div><div>std.compress.image.jpg</div></div></div><div><br></div=
<div><div>B)</div><div>rename:</div><div>std.compress.lzwCompress =3D&gt; =

.compress.lzw.uncompress</div> <div><br></div><div>which is more consistent with compress/uncompress from = std.zlib</div></div><div><br></div><div>C)</div><div>maybe add a link to=A0= <a href=3D"https://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Welch">= https://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Welch</a> or other= source</div> <div><br></div><div>D)</div><div>CircularBuffer belongs somewhere else; may= be std.range or std.container</div><div><br></div><div><br></div><div><br><= /div><div><br></div><div><br></div><div><div class=3D"gmail_quote">On Mon, = Jun 3, 2013 at 8:44 PM, Walter Bright <span dir=3D"ltr">&lt;<a href=3D"mail= to:newshound2 digitalmars.com" target=3D"_blank">newshound2 digitalmars.com= </a>&gt;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><a href=3D"https://github.com/WalterBright/p= hobos/blob/std_compress/std/compress.d" target=3D"_blank">https://github.co= m/<u></u>WalterBright/phobos/blob/std_<u></u>compress/std/compress.d</a><br=

<br> I wrote this to add components to compress and expand ranges.<br> <br> Highlights:<br> <br> 1. doesn&#39;t do any memory allocation<br> 2. can handle arbitrarily large sets of data<br> 3. it&#39;s lazy<br> 4. takes an InputRange, and outputs an InputRange<br> <br> Comments welcome.<br> </blockquote></div><br></div> --089e0141abac10303404de4e5ac2--
Jun 03 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/3/2013 11:40 PM, Timothee Cour wrote:
 D)
 CircularBuffer belongs somewhere else; maybe std.range or std.container

I have mixed feelings about that. If you'll notice, std.compress doesn't have any imports! I wanted to make at least one module that doesn't pull in 100% of everything in Phobos (one of my pet peeves).
Jun 04 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 1:00 AM, Walter Bright wrote:
 On 6/3/2013 11:40 PM, Timothee Cour wrote:
 D)
 CircularBuffer belongs somewhere else; maybe std.range or std.container

I have mixed feelings about that. If you'll notice, std.compress doesn't have any imports! I wanted to make at least one module that doesn't pull in 100% of everything in Phobos (one of my pet peeves).

Note also I didn't document it, so it is private and can be moved.
Jun 04 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 1:15 AM, Peter Alexander wrote:
 On Tuesday, 4 June 2013 at 08:03:15 UTC, Walter Bright wrote:
 On 6/4/2013 1:00 AM, Walter Bright wrote:
 On 6/3/2013 11:40 PM, Timothee Cour wrote:
 D)
 CircularBuffer belongs somewhere else; maybe std.range or std.container

I have mixed feelings about that. If you'll notice, std.compress doesn't have any imports! I wanted to make at least one module that doesn't pull in 100% of everything in Phobos (one of my pet peeves).

Note also I didn't document it, so it is private and can be moved.

Then it should be private.

I agree.
 You should also mangle the name so that it doesn't
 pollute the unqualified symbol namespace (either that or fix visibility of
 private symbols).

If it proves useful, it will be moved into some more proper and public place. I think it's a bad idea to 'mangle' the name. First off, if it is private, it is not visible. And even being public, the anti-hijacking language features make it a non-problem. The whole point is to avoid the wretched C problems with a global name space, by not having a global name space.
Jun 04 2013
next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
04-Jun-2013 12:23, Walter Bright пишет:
 On 6/4/2013 1:15 AM, Peter Alexander wrote:

 You should also mangle the name so that it doesn't
 pollute the unqualified symbol namespace (either that or fix
 visibility of
 private symbols).

If it proves useful, it will be moved into some more proper and public place. I think it's a bad idea to 'mangle' the name. First off, if it is private, it is not visible. And even being public, the anti-hijacking language features make it a non-problem. The whole point is to avoid the wretched C problems with a global name space, by not having a global name space.

now is the time fix this bug? -- Dmitry Olshansky
Jun 04 2013
prev sibling parent Martin Nowak <code dawg.eu> writes:
On 06/04/2013 11:52 AM, Peter Alexander wrote:
 Well, the fix is currently in an unapproved DIP. I have no idea whether
 Walter intends to accept it or reject it. The discussion thread just
 seems to have died off.

 http://wiki.dlang.org/DIP22

I should really submit some ideas from my implementation to the DIP. https://github.com/D-Programming-Language/dmd/pull/739
Jun 04 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 1:08 AM, Jakob Ovrum wrote:
 On Tuesday, 4 June 2013 at 08:00:03 UTC, Walter Bright wrote:
 I have mixed feelings about that. If you'll notice, std.compress doesn't have
 any imports! I wanted to make at least one module that doesn't pull in 100% of
 everything in Phobos (one of my pet peeves).

I think this is a workaround, not a proper solution.

Yes, it is.
 It probably means Phobos' granularity is horribly wrong.

Yup. Phobos is hard to work on because of the complexity of everything importing and depending on everything else all in mutually referential cycles. I deliberately set out to create compress as a non-trivial module that did not do that. I hope that splitting things up into packages will improve things.
Jun 04 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 5:17 AM, Adam D. Ruppe wrote:
 I actually wish we could have multiple modules in a single file.

I don't see much point to that in modern file systems.
Jun 04 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 5:13 PM, Marco Leise wrote:
 Probably seek time if the files are scattered and not in cache.
 That's hardly a show stopper unless you have 17.156 files like
 the Java Runtime. But they 'solved' it by zipping them up.

Actually, I've often thought of making dmd able to read everything it needs out of a zip file.
Jun 04 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-06-05 02:58, Walter Bright wrote:

 Actually, I've often thought of making dmd able to read everything it
 needs out of a zip file.

I think it's better to have a proper package manager. -- /Jacob Carlborg
Jun 05 2013
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/4/13 4:00 AM, Walter Bright wrote:
 On 6/3/2013 11:40 PM, Timothee Cour wrote:
 D)
 CircularBuffer belongs somewhere else; maybe std.range or std.container

I have mixed feelings about that. If you'll notice, std.compress doesn't have any imports! I wanted to make at least one module that doesn't pull in 100% of everything in Phobos (one of my pet peeves).

The downside of that is reinventing everything. I haven't looked at the code yet, but std.range has http://dlang.org/phobos/std_range.html#cycle which implements a circular buffer. Andrei
Jun 04 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 5:26 AM, Andrei Alexandrescu wrote:
 The downside of that is reinventing everything. I haven't looked at the code
 yet, but std.range has http://dlang.org/phobos/std_range.html#cycle which
 implements a circular buffer.

cycle only reads from a circular buffer. CircularBuffer can be filled as well as emptied at the same time (it has a put() method).
Jun 04 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-06-04 08:40, Timothee Cour wrote:
 A)
 there already is std.zlib; why not have:
 std.compress.zlib: public import std.zlib
 std.compress.lzw: put this new module there instead of in std.compress
 std.compress.image.png
 std.compress.image.jpg

 B)
 rename:
 std.compress.lzwCompress => std.compress.lzw.compress
 std.compress. lzwExpand => std.compress.lzw.uncompress

 which is more consistent with compress/uncompress from std.zlib

 C)
 maybe add a link to
 https://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Welch or other
 source

 D)
 CircularBuffer belongs somewhere else; maybe std.range or std.container

I agree with all of these. Perhaps it should be put in the review queue as well. -- /Jacob Carlborg
Jun 04 2013
prev sibling next sibling parent "Dicebot" <m.strashun gmail.com> writes:
On Tuesday, 4 June 2013 at 08:00:03 UTC, Walter Bright wrote:
 On 6/3/2013 11:40 PM, Timothee Cour wrote:
 D)
 CircularBuffer belongs somewhere else; maybe std.range or 
 std.container

I have mixed feelings about that. If you'll notice, std.compress doesn't have any imports! I wanted to make at least one module that doesn't pull in 100% of everything in Phobos (one of my pet peeves).

If that is an issue, it is an issue in DMD, not in module. Modules are supposed to use each other extensively, that is the very reason to have them!
Jun 04 2013
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Tuesday, 4 June 2013 at 08:00:03 UTC, Walter Bright wrote:
 I have mixed feelings about that. If you'll notice, 
 std.compress doesn't have any imports! I wanted to make at 
 least one module that doesn't pull in 100% of everything in 
 Phobos (one of my pet peeves).

I think this is a workaround, not a proper solution. It probably means Phobos' granularity is horribly wrong.
Jun 04 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Tuesday, 4 June 2013 at 08:03:15 UTC, Walter Bright wrote:
 On 6/4/2013 1:00 AM, Walter Bright wrote:
 On 6/3/2013 11:40 PM, Timothee Cour wrote:
 D)
 CircularBuffer belongs somewhere else; maybe std.range or 
 std.container

I have mixed feelings about that. If you'll notice, std.compress doesn't have any imports! I wanted to make at least one module that doesn't pull in 100% of everything in Phobos (one of my pet peeves).

Note also I didn't document it, so it is private and can be moved.

Then it should be private. You should also mangle the name so that it doesn't pollute the unqualified symbol namespace (either that or fix visibility of private symbols).
Jun 04 2013
prev sibling next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
--001a11c209d2b1a72604de4fe5e3
Content-Type: text/plain; charset=ISO-8859-1

There's no point in having modules reinvent the wheel everytime. CircularBuffer
is clearly usable in other contexts.

Reusing such code makes sure bug fixes and efficiency gains are done once
and for all and work across the board.

 I wanted to make at least one module that doesn't pull in 100% of


That seems like a very artificial exercise leading to unnecessary contorsions. On Tue, Jun 4, 2013 at 1:00 AM, Walter Bright <newshound2 digitalmars.com>wrote:
 On 6/3/2013 11:40 PM, Timothee Cour wrote:

 D)
 CircularBuffer belongs somewhere else; maybe std.range or std.container

I have mixed feelings about that. If you'll notice, std.compress doesn't have any imports! I wanted to make at least one module that doesn't pull in 100% of everything in Phobos (one of my pet peeves).

--001a11c209d2b1a72604de4fe5e3 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable There&#39;s no point in having modules reinvent the wheel everytime.=A0<spa= n style=3D"font-size:13px;color:rgb(80,0,80);font-family:arial,sans-serif;b= ackground-color:rgb(255,255,255)">CircularBuffer is clearly usable in other= contexts.=A0</span><div> <font color=3D"#500050" face=3D"arial, sans-serif"><br></font></div><div><f= ont color=3D"#500050" face=3D"arial, sans-serif">Reusing such code makes su= re bug fixes and efficiency gains are done once and for all and work across= the board.<br> </font><div><font color=3D"#500050" face=3D"arial, sans-serif"><br></font><= /div><div>&gt;&gt; I wanted to make at least one module that doesn&#39;t pu= ll in 100% of everything in Phobos</div><div><br></div><div>That seems like= a very artificial exercise leading to unnecessary contorsions.<br> <br><div class=3D"gmail_quote">On Tue, Jun 4, 2013 at 1:00 AM, Walter Brigh= t <span dir=3D"ltr">&lt;<a href=3D"mailto:newshound2 digitalmars.com" targe= t=3D"_blank">newshound2 digitalmars.com</a>&gt;</span> wrote:<br><blockquot= e class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc sol= id;padding-left:1ex"> <div class=3D"im">On 6/3/2013 11:40 PM, Timothee Cour wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> D)<br> CircularBuffer belongs somewhere else; maybe std.range or std.container<br> </blockquote> <br></div> I have mixed feelings about that. If you&#39;ll notice, std.compress doesn&= #39;t have any imports! I wanted to make at least one module that doesn&#39= ;t pull in 100% of everything in Phobos (one of my pet peeves).<br> </blockquote></div><br></div></div> --001a11c209d2b1a72604de4fe5e3--
Jun 04 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Tuesday, 4 June 2013 at 08:23:52 UTC, Walter Bright wrote:
 You should also mangle the name so that it doesn't
 pollute the unqualified symbol namespace (either that or fix 
 visibility of
 private symbols).

If it proves useful, it will be moved into some more proper and public place. I think it's a bad idea to 'mangle' the name. First off, if it is private, it is not visible. And even being public, the anti-hijacking language features make it a non-problem. The whole point is to avoid the wretched C problems with a global name space, by not having a global name space.

import std.compress; import mylib.circularbuffer; CircularBuffer!(ubyte[1024]) buf; ERROR: conflicting names, even though std.compress.CircularBuffer is private! I have to fully qualify CircularBuffer, or use alias to get around the problem. D may not have a global namespace, but it does have unqualified name lookup, and private symbols still pollute that pseudo-namespace.
Jun 04 2013
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/3/2013 8:44 PM, Walter Bright wrote:
 https://github.com/WalterBright/phobos/blob/std_compress/std/compress.d

 I wrote this to add components to compress and expand ranges.

 Highlights:

 1. doesn't do any memory allocation
 2. can handle arbitrarily large sets of data
 3. it's lazy
 4. takes an InputRange, and outputs an InputRange

 Comments welcome.

BTW, I also wrote this because it is a tricky component to write. There is not a 1:1 correspondence between input and output - the relationship is not predictable. Worse, there are "look backs" on input and "back patches" on output. Hence, sliding buffers have to be used on both input and output. I like to think of it as an example of how to do such. It took me a bit of time to figure out a way to do it that wasn't too numbingly complex.
Jun 04 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Tuesday, 4 June 2013 at 08:33:29 UTC, Peter Alexander wrote:
 On Tuesday, 4 June 2013 at 08:23:52 UTC, Walter Bright wrote:
 You should also mangle the name so that it doesn't
 pollute the unqualified symbol namespace (either that or fix 
 visibility of
 private symbols).

If it proves useful, it will be moved into some more proper and public place. I think it's a bad idea to 'mangle' the name. First off, if it is private, it is not visible. And even being public, the anti-hijacking language features make it a non-problem. The whole point is to avoid the wretched C problems with a global name space, by not having a global name space.

import std.compress; import mylib.circularbuffer; CircularBuffer!(ubyte[1024]) buf; ERROR: conflicting names, even though std.compress.CircularBuffer is private! I have to fully qualify CircularBuffer, or use alias to get around the problem.

Is this according to the specs though, or a bug? It was my understanding that another module's private symbols should not even be "seen" ?
Jun 04 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Tuesday, 4 June 2013 at 09:11:49 UTC, monarch_dodra wrote:
 On Tuesday, 4 June 2013 at 08:33:29 UTC, Peter Alexander wrote:
 On Tuesday, 4 June 2013 at 08:23:52 UTC, Walter Bright wrote:
 You should also mangle the name so that it doesn't
 pollute the unqualified symbol namespace (either that or fix 
 visibility of
 private symbols).

If it proves useful, it will be moved into some more proper and public place. I think it's a bad idea to 'mangle' the name. First off, if it is private, it is not visible. And even being public, the anti-hijacking language features make it a non-problem. The whole point is to avoid the wretched C problems with a global name space, by not having a global name space.

import std.compress; import mylib.circularbuffer; CircularBuffer!(ubyte[1024]) buf; ERROR: conflicting names, even though std.compress.CircularBuffer is private! I have to fully qualify CircularBuffer, or use alias to get around the problem.

Is this according to the specs though, or a bug? It was my understanding that another module's private symbols should not even be "seen" ?

Well, the fix is currently in an unapproved DIP. I have no idea whether Walter intends to accept it or reject it. The discussion thread just seems to have died off. http://wiki.dlang.org/DIP22
Jun 04 2013
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Timothee Cour:

 D)
 CircularBuffer belongs somewhere else; maybe std.range or 
 std.container

If you are interested in adding a CircularBuffer to Phobos, then I'd like both that fixed sized one and a growing one like this: http://rosettacode.org/wiki/Queue/Usage#Faster_Version Bye, bearophile
Jun 04 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Tuesday, 4 June 2013 at 11:18:45 UTC, bearophile wrote:
 If you are interested in adding a CircularBuffer to Phobos, 
 then I'd like both that fixed sized one and a growing one like 
 this:
 http://rosettacode.org/wiki/Queue/Usage#Faster_Version

Nitpick; head = (head + 1) & ((cast(size_t)1 << power2) - 1); can be head = (head + 1) & (A.length - 1); No? power2 seems superfluous. Also, left/right shifts by variable amount are very slow on some processors Anyway, we'll really need allocators before we can add more allocating containers. Andrei? :-)
Jun 04 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
I actually wish we could have multiple modules in a single file. 
Correct me if I'm wrong, but if imported something and only used 
one type there, the linker should strip out the others, right?

But this doesn't happen because ModuleInfo references all kinds 
of things, and moduleinfo is referenced for constructors and 
such. This is useful and removing it is probably a bad idea.

Breaking up into packages is one idea but you can't always do it. 
What if you're doing some big string mixins? A single file is 
also a little easier to distribute.

But mixins is the case that is hard to work around since they by 
definition go into one file. If we could do something like 
mixin("module foo.mixin"~name~" { code }"); you could work around 
it.

Then you could isolate sections of generated code in their own 
logical modules, letting the linker kill those sections if they 
aren't actually used.
Jun 04 2013
prev sibling next sibling parent reply David <d dav1d.de> writes:
Am 04.06.2013 05:44, schrieb Walter Bright:
 https://github.com/WalterBright/phobos/blob/std_compress/std/compress.d
 
 I wrote this to add components to compress and expand ranges.
 
 Highlights:
 
 1. doesn't do any memory allocation
 2. can handle arbitrarily large sets of data
 3. it's lazy
 4. takes an InputRange, and outputs an InputRange
 
 Comments welcome.

Why do we need that? I would much rather have a deflate which doesn't depend on a C zlib (a proper std.zlib written in 100% D) and followed by a less buggy, less pita, less limited std.zip (btw. I think I fxed one of the bugs a while ago but it is still open and listed as bug on dlang.org). I personally never used lzw compression and from what I know it is only used in GIF and TIFF (I might be wrong here), in comparison to deflate which is used in a varity of formats. So making std.compress only contain a rarely used compression algorithm feels wrong, having in it std.compress.* ok.
Jun 04 2013
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, June 04, 2013 14:48:34 David wrote:
 (btw. I think I fxed one
 of the bugs a while ago but it is still open and listed as bug on
 dlang.org).

If you're sure that it's fixed, then close it. - Jonathan M Davis
Jun 04 2013
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Peter Alexander:

 Nitpick;

 head = (head + 1) & ((cast(size_t)1 << power2) - 1);

 can be

 head = (head + 1) & (A.length - 1);

 No? power2 seems superfluous.

I see. Thank you. I will improve it later. Bye, bearophile
Jun 04 2013
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-04 05:44, Walter Bright wrote:
 https://github.com/WalterBright/phobos/blob/std_compress/std/compress.d

 I wrote this to add components to compress and expand ranges.

 Highlights:

 1. doesn't do any memory allocation
 2. can handle arbitrarily large sets of data
 3. it's lazy
 4. takes an InputRange, and outputs an InputRange

I'm wondering if (un)compress can take the compressing algorithm as a template parameter. Does that make sense? Something like: auto result = data.compress!(LZW); Then we could pass different compressing algorithms to the compress function. -- /Jacob Carlborg
Jun 04 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 6:34 AM, Jacob Carlborg wrote:
 I'm wondering if (un)compress can take the compressing algorithm as a template
 parameter. Does that make sense?

 Something like:

 auto result = data.compress!(LZW);

 Then we could pass different compressing algorithms to the compress function.

I don't see the point. Furthermore, it requires that the compress template know about all the compression algorithms available, which limits future expansion.
Jun 04 2013
next sibling parent reply David <d dav1d.de> writes:
Am 04.06.2013 18:09, schrieb Walter Bright:
 On 6/4/2013 6:34 AM, Jacob Carlborg wrote:
 I'm wondering if (un)compress can take the compressing algorithm as a
 template
 parameter. Does that make sense?

 Something like:

 auto result = data.compress!(LZW);

 Then we could pass different compressing algorithms to the compress
 function.

I don't see the point. Furthermore, it requires that the compress template know about all the compression algorithms available, which limits future expansion.

No the compression type only has to provide a certain api.
Jun 04 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 9:34 AM, David wrote:
 Am 04.06.2013 18:09, schrieb Walter Bright:
 On 6/4/2013 6:34 AM, Jacob Carlborg wrote:
 I'm wondering if (un)compress can take the compressing algorithm as a
 template
 parameter. Does that make sense?

 Something like:

 auto result = data.compress!(LZW);

 Then we could pass different compressing algorithms to the compress
 function.

I don't see the point. Furthermore, it requires that the compress template know about all the compression algorithms available, which limits future expansion.

No the compression type only has to provide a certain api.

Again, I'm not seeing the added value with this.
Jun 04 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 9:33 AM, John Colvin wrote:
 On Tuesday, 4 June 2013 at 16:09:09 UTC, Walter Bright wrote:
 On 6/4/2013 6:34 AM, Jacob Carlborg wrote:
 I'm wondering if (un)compress can take the compressing algorithm as a template
 parameter. Does that make sense?

 Something like:

 auto result = data.compress!(LZW);

 Then we could pass different compressing algorithms to the compress function.

I don't see the point. Furthermore, it requires that the compress template know about all the compression algorithms available, which limits future expansion.

Not necessarily. If the compression algorithms were free functions in the module you could just be passing an alias to one, which compress would then call. (which would also allow people to specify their own algorithms)

What value does a function which just passes an alias to another one add?
Jun 04 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 11:04 AM, John Colvin wrote:
 On Tuesday, 4 June 2013 at 17:50:47 UTC, Walter Bright wrote:
 What value does a function which just passes an alias to another one add?

A unified interface called "compress" that takes a compression function as an alias (with e.g. lzwCompress as a default) seems like a nicer way of working, seeing as people don't necessarily care/know about which algorithm they're using, they just want to compress something a bit. Also, it would be cool if a range could remember which algorithm it was compressed with (as it's type? I.e. LzwRange), so a generic function "expand" could call the appropriate ***Expand

What is the improvement of typing: compress(lzw) over: lzwCompress() ?
Jun 04 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 11:43 AM, Timothee Cour wrote:
 writing generic code.
 same reason as why we prefer:
 auto y=to!double(x) over auto y=to_double(x);

The situations aren't comparable. The to!double case is parameterizing with a type, the compress one is not. Secondly, compress(lzw) does ABSOLUTELY NOTHING but turn around and call lzw. It adds nothing.
Jun 04 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 11:55 AM, Jonathan M Davis wrote:
 Well, I'd expect it to be compress!lzw(), but in any case, what it buys you is
 that you can pass the algorithm around without caring what it is so that while
 code higher up on the stack may have to know that it's lzw, code deeper down
 doesn't have to care what type of algorithm it's using. Now, whether that
 flexibility is all that useful in this particular case, I don't know, but it
 _does_ help with generic code. It's like how a lot of std.algorithm takes its
 predicate as an alias.

There is zero utility in this: auto compress(alias dg) { return dg(); } Not even for generic code.
Jun 04 2013
next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
05-Jun-2013 00:30, Byron Heads пишет:
 On Tue, 04 Jun 2013 13:15:07 -0700, Walter Bright wrote:

 On 6/4/2013 11:55 AM, Jonathan M Davis wrote:
 Well, I'd expect it to be compress!lzw(), but in any case, what it buys
 you is that you can pass the algorithm around without caring what it is
 so that while code higher up on the stack may have to know that it's
 lzw, code deeper down doesn't have to care what type of algorithm it's
 using. Now, whether that flexibility is all that useful in this
 particular case, I don't know, but it _does_ help with generic code.
 It's like how a lot of std.algorithm takes its predicate as an alias.

There is zero utility in this: auto compress(alias dg) { return dg(); } Not even for generic code.

but a compress interface would be nice: interface Compress { ubyte[] compress(ubyte[]); ubyte[] uncompress(ubyte[]); } that way you can use any compress algorithm bool send(Compress)(Socket sock);

It's a range already thus composable. Ranged I/O though is something to come some time in near future (Steve?) -- Dmitry Olshansky
Jun 04 2013
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 1:30 PM, Byron Heads wrote:
 but a compress interface would be nice:

 interface Compress
 {
      ubyte[] compress(ubyte[]);
      ubyte[] uncompress(ubyte[]);
 }

 that way you can use any compress algorithm
 bool send(Compress)(Socket sock);

That isn't how ranges work. Ranges already define an input and an output interface. We don't need to invent another scheme.
Jun 04 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 12:41 PM, Peter Alexander wrote:
 I think this is over-engineering. It's unlikely that an application will need
to
 support multiple compression algorithms in the same piece of code, and even if
 it did, it would be trivial to implement this on top of the simple interface
 that Walter is using.

Yup. My experience with abstractions that have no use cases is all the wrong things get abstracted. And by my experience, I include every one I've seen other people write as well as my own. My favorite is windows.h. It was originally written for 16 bit Windows, and had all kinds of abstractions to make it portable for a future 32 bit Windows. Unfortunately, apparently nobody working on windows.h had any experience with 32 bit code, and the abstractions turned out to be all wrong.
Jun 04 2013
parent Paulo Pinto <pjmlp progtools.org> writes:
Am 04.06.2013 22:20, schrieb Walter Bright:
 On 6/4/2013 12:41 PM, Peter Alexander wrote:
 I think this is over-engineering. It's unlikely that an application
 will need to
 support multiple compression algorithms in the same piece of code, and
 even if
 it did, it would be trivial to implement this on top of the simple
 interface
 that Walter is using.

Yup. My experience with abstractions that have no use cases is all the wrong things get abstracted. And by my experience, I include every one I've seen other people write as well as my own. My favorite is windows.h. It was originally written for 16 bit Windows, and had all kinds of abstractions to make it portable for a future 32 bit Windows. Unfortunately, apparently nobody working on windows.h had any experience with 32 bit code, and the abstractions turned out to be all wrong.

Yep, it brings back some memories.
Jun 05 2013
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/4/13 2:46 PM, Walter Bright wrote:
 On 6/4/2013 11:43 AM, Timothee Cour wrote:
 writing generic code.
 same reason as why we prefer:
 auto y=to!double(x) over auto y=to_double(x);

The situations aren't comparable. The to!double case is parameterizing with a type, the compress one is not. Secondly, compress(lzw) does ABSOLUTELY NOTHING but turn around and call lzw. It adds nothing.

Not absolutely nothing. Almost nothing. The distinction is important. Andrei
Jun 04 2013
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/5/13 12:44 AM, Max Samukha wrote:
 On Tuesday, 4 June 2013 at 18:46:49 UTC, Walter Bright wrote:
 On 6/4/2013 11:43 AM, Timothee Cour wrote:
 writing generic code.
 same reason as why we prefer:
 auto y=to!double(x) over auto y=to_double(x);

The situations aren't comparable. The to!double case is parameterizing with a type, the compress one is not. Secondly, compress(lzw) does ABSOLUTELY NOTHING but turn around and call lzw. It adds nothing.

That "absolutely" based on limited personal experience is the biggest D's problem.

It's a point, but "biggest" is also kind of too much and based on limited personal experience :o). Andrei
Jun 04 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 9:44 PM, Max Samukha wrote:
 On Tuesday, 4 June 2013 at 18:46:49 UTC, Walter Bright wrote:
 On 6/4/2013 11:43 AM, Timothee Cour wrote:
 writing generic code.
 same reason as why we prefer:
 auto y=to!double(x) over auto y=to_double(x);

The situations aren't comparable. The to!double case is parameterizing with a type, the compress one is not. Secondly, compress(lzw) does ABSOLUTELY NOTHING but turn around and call lzw. It adds nothing.

That "absolutely" based on limited personal experience is the biggest D's problem.

I've seen an awful lot of abstractions over the years that provided zero value. You need to provide a compelling use case to justify another layer of complexity. "generic code" is not a compelling use case. It's already generic. Note how these components are to be used: src.lzwCompress.copy(dst); Your proposal is: src.compress(lzw).copy(dst); I.e. zero value, as so far all compress() does is call lzw(). The whole point of range-based pipeline programming is you can just plug in different components. There is no demonstrated use case for adding another layer. I am actually wrong in saying it has zero value. It has negative value :-)
Jun 04 2013
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, June 04, 2013 23:55:05 Timothee Cour wrote:
 What I suggested in my original post didn't involve any
 indirection/abstraction; simply a renaming to be consistent with existing
 zlib (see my points A+B in my 1st post on this thread):
 
 std.compress.zlib.compress
 std.compress.zlib.uncompress
 std.compress.lzw.compress
 std.compress.lzw.uncompress
 
 same reason we have: std.file.write, std.stdio.write, etc, and not
 std.fileWrite, std.stdioWrite.

So, you want to create whole modules for each compression algorithm? That seems like overkill to me. What Walter currently has isn't even 1000 lines long (and that's including the CircularBuffer helper struct). Splitting it up like that seems like over-modularation to me. - Jonathan M Daivs
Jun 04 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/4/2013 11:59 PM, Jonathan M Davis wrote:
 So, you want to create whole modules for each compression algorithm?

Yes.
 That seems like overkill to me. What Walter currently has isn't even 1000 lines
 long (and that's including the CircularBuffer helper struct). Splitting it up
 like that seems like over-modularation to me.

When two modules have nothing to do with each other, they should be in separate modules.
Jun 05 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/5/2013 12:29 AM, Jonathan M Davis wrote:
 On Wednesday, June 05, 2013 00:14:59 Walter Bright wrote:
 When two modules have nothing to do with each other, they should be in
 separate modules.

Except that they're all compression algorithms, so they _are_ related.

No, they are not related. They don't share code, and it is unlikely more than one would be called in any particular use case. Remember, module contents have private access to other parts of the module. This violates encapsulation when the parts are unrelated.
 Having
 modules that are only a few hundred lines long is very counterproductive IMHO.

Why? On the other hand, when you are trying to understand a module, having thousands of lines of things that have no connection to each other makes it difficult. It also makes debugging them harder than necessary.
 It's highly annoying how Java insists on splitting everything up into different
 files. You end up with a lot of small files to wade through.

Wade through for what? If you're having a problem with the lzw compressor, why would you find it more productive to wade through the huffman compressor to get to it?
 Fortunately, D
 doesn't force that, and I don't think that we should go that route by choice.
 There's no more reason to split all of these up then there is to put each
 algorithm in std.algorithm in its own module. And yes, I know that you like
 that idea, but it seems ridiculous to me to try and have only one or two
 functions per module. We don't want them to be huge, but having them be very
 small is just as harmful IMHO.

You need a better case as to why it is harmful. I've spent many miserable hours trying to find a bug in a phobos module that is a zillion lines of code, trying to strip out what is not necessary to repro the problem. I don't see what problem kitchen sink modules solve - my experience is that smaller, better contained abstractions are more productive than kitchen sinks.
Jun 05 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-05 08:59, Jonathan M Davis wrote:

 So, you want to create whole modules for each compression algorithm? That
 seems like overkill to me. What Walter currently has isn't even 1000 lines
 long (and that's including the CircularBuffer helper struct). Splitting it up
 like that seems like over-modularation to me.

The current modules in Phobos already contains too much. We shouldn't make the same mistake again. -- /Jacob Carlborg
Jun 05 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/5/2013 12:38 AM, Jonathan M Davis wrote:
 Maybe some do, but many don't, and 1000 lines is _far_ from too much. If we
 start making modules that small, we're going to end up with tons of them to
 wade through to find anything.

1. It isn't any harder to find things in multiple files than in one file. 2. If there's a ton in one file, you have to wade through the ton to find what you're looking for. Your argument has merit if you are using a floppy disk drive for storage, as floppies are agonizingly slow to read files off of. But that problem disappeared 30 years ago. (At an SD conference back in the 80's, I was on a compiler panel with the compiler guys from Microsoft, Borland, etc. We were each asked how our respective compilers worked on floppy systems. The guys would say "well, you set it up this way, configure it that way, juggle what goes on which floppy, and you can do it!" I was the third of five guys, and my response was: "We charge $200 extra for the floppy disk development system, and ship you a hard disk with it." That was the end of that discussion, and I never heard that question again.
Jun 05 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-06-05 16:31, John Colvin wrote:

 Agreed.

 To be honest, it's a trivial matter easily solved by a variety of tools,
 but I'm often just lazy and end up reading code with gedit or similar.

Gedit has a file tree sidebar, at least as a plugin. -- /Jacob Carlborg
Jun 09 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-06-05 09:38, Jonathan M Davis wrote:

 Maybe some do, but many don't, and 1000 lines is _far_ from too much. If we
 start making modules that small, we're going to end up with tons of them to
 wade through to find anything.

I completely agree with Walter and he mad my point a lot better than I could. -- /Jacob Carlborg
Jun 05 2013
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Wednesday, 5 June 2013 at 07:00:14 UTC, Jonathan M Davis wrote:
 So, you want to create whole modules for each compression 
 algorithm? That
 seems like overkill to me. What Walter currently has isn't even 
 1000 lines
 long (and that's including the CircularBuffer helper struct). 
 Splitting it up
 like that seems like over-modularation to me.

Modules are the unit of encapsulation in D (private), so they should always be as small as possible. As Andrei would say: Destroyed? David
Jun 05 2013
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/5/13 2:55 AM, Timothee Cour wrote:
 What I suggested in my original post didn't involve any
 indirection/abstraction; simply a renaming to be consistent with
 existing zlib (see my points A+B in my 1st post on this thread):

 std.compress.zlib.compress
 std.compress.zlib.uncompress
 std.compress.lzw.compress
 std.compress.lzw.uncompress

I think that's nice. Andrei
Jun 05 2013
next sibling parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> wrote in message 
news:koncgm$9f5$1 digitalmars.com...
 On 6/5/13 2:55 AM, Timothee Cour wrote:
 What I suggested in my original post didn't involve any
 indirection/abstraction; simply a renaming to be consistent with
 existing zlib (see my points A+B in my 1st post on this thread):

 std.compress.zlib.compress
 std.compress.zlib.uncompress
 std.compress.lzw.compress
 std.compress.lzw.uncompress

I think that's nice. Andrei

This has the problem that you now can't import more than one compression module and still use ufcs. The annoying one I keep hitting in phobos is std.file.write vs std.stdio.write. For range-based APIs it is a huge pita to have to switch away from ufcs. I think xyzCompress is still pretty sweet, consistent, and completely fixes the problem. It has the added benefit that you can tell which compression algorithm is being used without having to know what is imported. I would not have a problem with each module providing both 'compress' and 'xyzCompress', but that is against phobos policy.
Jun 09 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 09, 2013 17:12:16 Daniel Murphy wrote:
 "Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> wrote in message
 news:koncgm$9f5$1 digitalmars.com...
 
 On 6/5/13 2:55 AM, Timothee Cour wrote:
 What I suggested in my original post didn't involve any
 indirection/abstraction; simply a renaming to be consistent with
 existing zlib (see my points A+B in my 1st post on this thread):
 
 std.compress.zlib.compress
 std.compress.zlib.uncompress
 std.compress.lzw.compress
 std.compress.lzw.uncompress

I think that's nice. Andrei

This has the problem that you now can't import more than one compression module and still use ufcs. The annoying one I keep hitting in phobos is std.file.write vs std.stdio.write. For range-based APIs it is a huge pita to have to switch away from ufcs. I think xyzCompress is still pretty sweet, consistent, and completely fixes the problem. It has the added benefit that you can tell which compression algorithm is being used without having to know what is imported.

That can be fixed by using a local alias, but it's true that it's an extra annoyance. - Jonathan M Davis
Jun 09 2013
prev sibling parent reply Timothee Cour <thelastmammoth gmail.com> writes:
--089e0118289ef9201b04dec296b8
Content-Type: text/plain; charset=ISO-8859-1

On Sun, Jun 9, 2013 at 12:53 AM, Jonathan M Davis <jmdavisProg gmx.com>wrote:

 On Sunday, June 09, 2013 17:12:16 Daniel Murphy wrote:
 "Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> wrote in message
 news:koncgm$9f5$1 digitalmars.com...

 On 6/5/13 2:55 AM, Timothee Cour wrote:
 What I suggested in my original post didn't involve any
 indirection/abstraction; simply a renaming to be consistent with
 existing zlib (see my points A+B in my 1st post on this thread):

 std.compress.zlib.compress
 std.compress.zlib.uncompress
 std.compress.lzw.compress
 std.compress.lzw.uncompress

I think that's nice. Andrei

This has the problem that you now can't import more than one compression module and still use ufcs. The annoying one I keep hitting in phobos is std.file.write vs std.stdio.write. For range-based APIs it is a huge

 to have to switch away from ufcs.  I think xyzCompress is still pretty
 sweet, consistent, and completely fixes the problem.  It has the added
 benefit that you can tell which compression algorithm is being used

 having to know what is imported.

That can be fixed by using a local alias, but it's true that it's an extra annoyance. - Jonathan M Davis

which is why I have suggested supporting UFCS with fully qualified function names: auto a="".(std.path.join)("\n"); myfile.(std.file.write)(text); text.(std.stdio.write); see post: support UFCS with fully qualified function names (was in "digitalmars.D.learn") http://forum.dlang.org/post/mailman.1453.1369099708.4724.digitalmars-d puremagic.com it also helps searchability: if one uses local aliases such as import std.stdio:write2=write, naive searching via grep 'write(' will miss such cases. The increase in complexity is minimal, and the feature makes sense with the rest of the language. --089e0118289ef9201b04dec296b8 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <div>On Sun, Jun 9, 2013 at 12:53 AM, Jonathan M Davis <span dir=3D"ltr">&l= t;<a href=3D"mailto:jmdavisProg gmx.com" target=3D"_blank">jmdavisProg gmx.= com</a>&gt;</span> wrote:</div><div class=3D"gmail_quote"><blockquote class= =3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd= ing-left:1ex"> <div class=3D"im">On Sunday, June 09, 2013 17:12:16 Daniel Murphy wrote:<br=

il erdani.org">SeeWebsiteForEmail erdani.org</a>&gt; wrote in message<br> &gt; news:koncgm$9f5$1 digitalmars.com...<br> &gt;<br> &gt; &gt; On 6/5/13 2:55 AM, Timothee Cour wrote:<br> &gt; &gt;&gt; What I suggested in my original post didn&#39;t involve any<b= r> &gt; &gt;&gt; indirection/abstraction; simply a renaming to be consistent w= ith<br> &gt; &gt;&gt; existing zlib (see my points A+B in my 1st post on this threa= d):<br> &gt; &gt;&gt;<br> &gt; &gt;&gt; std.compress.zlib.compress<br> &gt; &gt;&gt; std.compress.zlib.uncompress<br> &gt; &gt;&gt; std.compress.lzw.compress<br> &gt; &gt;&gt; std.compress.lzw.uncompress<br> &gt; &gt;<br> &gt; &gt; I think that&#39;s nice.<br> &gt; &gt;<br> &gt; &gt; Andrei<br> &gt;<br> &gt; This has the problem that you now can&#39;t import more than one compr= ession<br> &gt; module and still use ufcs. =A0The annoying one I keep hitting in phobo= s is<br> &gt; std.file.write vs std.stdio.write. =A0For range-based APIs it is a hug= e pita<br> &gt; to have to switch away from ufcs. =A0I think xyzCompress is still pret= ty<br> &gt; sweet, consistent, and completely fixes the problem. =A0It has the add= ed<br> &gt; benefit that you can tell which compression algorithm is being used wi= thout<br> &gt; having to know what is imported.<br> <br> </div>That can be fixed by using a local alias, but it&#39;s true that it&#= 39;s an extra<br> annoyance.<br> <span class=3D"HOEnZb"><font color=3D"#888888"><br> - Jonathan M Davis<br> </font></span></blockquote></div><br><div><div>which is why I have suggeste= d supporting UFCS with fully qualified function names:</div><div><br></div>= <div>auto a=3D&quot;&quot;.(std.path.join)(&quot;\n&quot;);</div><div><div> myfile.(std.file.write)(text);</div></div><div><div>text.(std.stdio.write);= </div></div><div><br></div><div>see post: support UFCS with fully qualified= function names (was in &quot;digitalmars.D.learn&quot;)</div><div><a href= =3D"http://forum.dlang.org/post/mailman.1453.1369099708.4724.digitalmars-d = puremagic.com">http://forum.dlang.org/post/mailman.1453.1369099708.4724.dig= italmars-d puremagic.com</a></div> <br></div><div>it also helps searchability: if one uses local aliases such = as import std.stdio:write2=3Dwrite, naive searching via grep &#39;write(&#3= 9; will miss such cases. The increase in complexity is minimal, and the fea= ture makes sense with the rest of the language.</div> --089e0118289ef9201b04dec296b8--
Jun 09 2013
parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Timothee Cour" <thelastmammoth gmail.com> wrote in message 
news:mailman.999.1370827257.13711.digitalmars-d puremagic.com...
 which is why I have suggested supporting UFCS with fully qualified 
 function
 names:

 auto a="".(std.path.join)("\n");
 myfile.(std.file.write)(text);
 text.(std.stdio.write);

 see post: support UFCS with fully qualified function names (was in
 "digitalmars.D.learn")
 http://forum.dlang.org/post/mailman.1453.1369099708.4724.digitalmars-d puremagic.com

I'm not a huge fan of this syntax. If we were adding syntax, I would prefer a new operator with lower precedence than '.' eg auto a = "" -> std.path.join("\n"); But I'm not sure the problem is big enough to warrant new syntax.
 it also helps searchability: if one uses local aliases such as import
 std.stdio:write2=write, naive searching via grep 'write(' will miss such
 cases. The increase in complexity is minimal, and the feature makes sense
 with the rest of the language.

I agree, renamed imports make code harder to understand, and harder to refactor. In this case we can prevent problem simply by not giving functions generic names like 'compress'. Ideally you should be able to import the entire standard library with no name conflicts.
Jun 09 2013
parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jakob Ovrum" <jakobovrum gmail.com> wrote in message 
news:fjmuuahorgbwkcvygnqq forum.dlang.org...
 On Tuesday, 11 June 2013 at 13:13:56 UTC, Daniel Murphy wrote:
 Also, compress is a ridiculously general name for a function.

We have module-level functions called "copy" (multiple), "read", "write", "map", etc. already, and it's not a bad thing!

It is.
 It's OK because the full name is not "compress", but 
 "std.compression.lz77.compress". This way, how specific the code wants to 
 be depends on the user and the particular use-case, instead of 
 one-size-fits-all alternatives like "lz77Compress". There's no redundancy 
 in the name yet we still have the option to be pin-point specific (e.g. 
 static import), and yes, we still get to use UFCS!

There is a reason we don't call every function in phobos 'process' and let the module name tell us what is actually does - when you see the name in your source code, it is easy to recognize what is being done.
 To eliminate the UFCS problem - which doesn't happen very often (how often 
 do you want to use two different compression algorithms in the same 
 unit?), we can (must?) use renamed symbols when importing.

My workplace has a fire extinguisher, but this doesn't mean lighting fires is a good idea. I know we have the tools to disambiguate, but they come at a syntax and/or clarity cost. Why create a problem when we don't have to?
 Since any example using multiple "compress" functions would be contrived, 
 I'll use an existing conflict - the case of "copy".

Eg. Code which implements http compression with support for multiple algorithms. tl;dr We have great tools to disambiguate when we have to. Let's not have to.
Jun 11 2013
parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jakob Ovrum" <jakobovrum gmail.com> wrote in message 
news:sdgqfozqnysbnumynkvp forum.dlang.org...
 On Tuesday, 11 June 2013 at 22:34:55 UTC, Daniel Murphy wrote:
 There is a reason we don't call every function in phobos 'process' and 
 let
 the module name tell us what is actually does - when you see the name in
 your source code, it is easy to recognize what is being done.

"copy", "write" and "compress" are perfectly recognizable names.

Ok, how exactly is the data compressed in the following snippet? No scrolling up to the top of the module to see what's imported! newdata = data.compress();
 tl;dr We have great tools to disambiguate when we have to.  Let's not 
 have
 to.

The way I see it, you're asking that all code should pay for the benefit of a minority of cases. I'd choose the inverse.

This is not a function that will be used every few lines. Making the name a little longer for an increase in clarity is usually seen as a good idea.
Jun 13 2013
parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"David Nadlinger" <code klickverbot.at> wrote in message 
news:ahqzxzjbhmfiacwjgfkj forum.dlang.org...
 On Thursday, 13 June 2013 at 11:36:16 UTC, Daniel Murphy wrote:
 Ok, how exactly is the data compressed in the following snippet?  No
 scrolling up to the top of the module to see what's imported!

I don't need to scroll to the top of the module, just a few lines up because I'm using function-local imports anyway. :P If you want extra verbosity (which can be good *sometimes*), just write "import lz77 = std.compression.lz77" and you are good to go.

I don't think 4 characters is a high price to pay for the added clarity. Then there is no ambiguity, no need to rename imports, no problems using ufcs. Every time I see lz77Compress in anybody's code I know exactly what it does! I understand the motivation for shortening function names that will be used frequently... but this is not in that category.
Jun 13 2013
next sibling parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"David Nadlinger" <code klickverbot.at> wrote in message 
news:gzniyhyeuhjturqffgan forum.dlang.org...
 On Thursday, 13 June 2013 at 23:45:12 UTC, Daniel Murphy wrote:
 I don't think 4 characters is a high price to pay for the added clarity.
 Then there is no ambiguity, no need to rename imports, no problems using
 ufcs.  Every time I see lz77Compress in anybody's code I know exactly 
 what
 it does!

import std.compression : lz77Compress = lz78Compress; ;)

:(
Jun 13 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-06-14 01:53, David Nadlinger wrote:

 import std.compression : lz77Compress = lz78Compress;

 ;)

If you do that you only have yourself to blame. What if someone uses monkey patching and replaces all your functions at runtime. -- /Jacob Carlborg
Jun 14 2013
prev sibling next sibling parent "SomeDude" <lovelydear mailmetrash.com> writes:
On Wednesday, 5 June 2013 at 07:00:14 UTC, Jonathan M Davis wrote:
 So, you want to create whole modules for each compression 
 algorithm? That
 seems like overkill to me. What Walter currently has isn't even 
 1000 lines
 long (and that's including the CircularBuffer helper struct). 
 Splitting it up
 like that seems like over-modularation to me.

 - Jonathan M Daivs

Well, as the author of a 15,000 lines datetime module, I think your opinion is a little biased. *I* think 1,000 lines is a perfect size for a module.
Jun 05 2013
prev sibling next sibling parent "Xiaoxi" <xiaoxi 163.com> writes:
On Wednesday, 5 June 2013 at 19:01:28 UTC, SomeDude wrote:
 On Wednesday, 5 June 2013 at 07:00:14 UTC, Jonathan M Davis 
 wrote:
 So, you want to create whole modules for each compression 
 algorithm? That
 seems like overkill to me. What Walter currently has isn't 
 even 1000 lines
 long (and that's including the CircularBuffer helper struct). 
 Splitting it up
 like that seems like over-modularation to me.

 - Jonathan M Daivs

Well, as the author of a 15,000 lines datetime module, I think your opinion is a little biased. *I* think 1,000 lines is a perfect size for a module.

are cross module / file, inling working on all d compilers? if not, bigger modules are better.
Jun 06 2013
prev sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 6 June 2013 at 13:34:42 UTC, Xiaoxi wrote:
 are cross module / file, inling working on all d compilers? if 
 not, bigger modules are better.

This is not at all relevant if either a) the functions in question are templates, as it is the case here or b) the functions in a bigger module don't call each other anyway, such as in many kitchen-sink modules that just group vaguely related functionality together. David
Jun 06 2013
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/4/13 2:43 PM, Timothee Cour wrote:
     What is the improvement of typing:

         compress(lzw)

     over:

         lzwCompress()

     ?


 writing generic code.
 same reason as why we prefer:
 auto y=to!double(x) over auto y=to_double(x);

I think the application here is a bit more tenuous. It's natural to think of a type-parameterized algorithm that needs to!T. But it's more of a long shot to think of an algorithm statically parameterized on the compression method. That could definitely intervene, but it's not likely to be frequent; and if it's not, a mixin can always take care of it. Andrei
Jun 04 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 4 June 2013 at 16:09:09 UTC, Walter Bright wrote:
 On 6/4/2013 6:34 AM, Jacob Carlborg wrote:
 I'm wondering if (un)compress can take the compressing 
 algorithm as a template
 parameter. Does that make sense?

 Something like:

 auto result = data.compress!(LZW);

 Then we could pass different compressing algorithms to the 
 compress function.

I don't see the point. Furthermore, it requires that the compress template know about all the compression algorithms available, which limits future expansion.

Not necessarily. If the compression algorithms were free functions in the module you could just be passing an alias to one, which compress would then call. (which would also allow people to specify their own algorithms)
Jun 04 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 4 June 2013 at 17:50:47 UTC, Walter Bright wrote:
 On 6/4/2013 9:33 AM, John Colvin wrote:
 On Tuesday, 4 June 2013 at 16:09:09 UTC, Walter Bright wrote:
 On 6/4/2013 6:34 AM, Jacob Carlborg wrote:
 I'm wondering if (un)compress can take the compressing 
 algorithm as a template
 parameter. Does that make sense?

 Something like:

 auto result = data.compress!(LZW);

 Then we could pass different compressing algorithms to the 
 compress function.

I don't see the point. Furthermore, it requires that the compress template know about all the compression algorithms available, which limits future expansion.

Not necessarily. If the compression algorithms were free functions in the module you could just be passing an alias to one, which compress would then call. (which would also allow people to specify their own algorithms)

What value does a function which just passes an alias to another one add?

A unified interface called "compress" that takes a compression function as an alias (with e.g. lzwCompress as a default) seems like a nicer way of working, seeing as people don't necessarily care/know about which algorithm they're using, they just want to compress something a bit. Also, it would be cool if a range could remember which algorithm it was compressed with (as it's type? I.e. LzwRange), so a generic function "expand" could call the appropriate ***Expand
Jun 04 2013
prev sibling next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
--089e015370f4d8640804de5874d6
Content-Type: text/plain; charset=ISO-8859-1

On Tue, Jun 4, 2013 at 11:37 AM, Walter Bright
<newshound2 digitalmars.com>wrote:

 On 6/4/2013 11:04 AM, John Colvin wrote:

 On Tuesday, 4 June 2013 at 17:50:47 UTC, Walter Bright wrote:

 What value does a function which just passes an alias to another one add?

A unified interface called "compress" that takes a compression function as an alias (with e.g. lzwCompress as a default) seems like a nicer way of working, seeing as people don't necessarily care/know about which algorithm they're using, they just want to compress something a bit. Also, it would be cool if a range could remember which algorithm it was compressed with (as it's type? I.e. LzwRange), so a generic function "expand" could call the appropriate ***Expand

What is the improvement of typing: compress(lzw) over: lzwCompress() ?

writing generic code. same reason as why we prefer: auto y=to!double(x) over auto y=to_double(x); --089e015370f4d8640804de5874d6 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Tue, Jun 4, 2013 at 11:37 AM, Walter Bright <span dir=3D"ltr">&lt;<a hre= f=3D"mailto:newshound2 digitalmars.com" target=3D"_blank">newshound2 digita= lmars.com</a>&gt;</span> wrote:<br><div class=3D"gmail_quote"><blockquote c= lass=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;= padding-left:1ex"> <div class=3D"im">On 6/4/2013 11:04 AM, John Colvin wrote:<br> </div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"><div class=3D"im"> On Tuesday, 4 June 2013 at 17:50:47 UTC, Walter Bright wrote:<br> </div><div class=3D"im"><blockquote class=3D"gmail_quote" style=3D"margin:0= 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> What value does a function which just passes an alias to another one add?<b= r> </blockquote> <br> A unified interface called &quot;compress&quot; that takes a compression fu= nction as an<br> alias (with e.g. lzwCompress as a default) seems like a nicer way of workin= g,<br> seeing as people don&#39;t necessarily care/know about which algorithm they= &#39;re<br> using, they just want to compress something a bit.<br> <br> Also, it would be cool if a range could remember which algorithm it was<br> compressed with (as it&#39;s type? I.e. LzwRange), so a generic function &q= uot;expand&quot;<br> could call the appropriate ***Expand<br> </div></blockquote> <br> What is the improvement of typing:<br> <br> =A0 =A0compress(lzw)<br> <br> over:<br> <br> =A0 =A0lzwCompress()<br> <br> ?<br> </blockquote></div><br><div>writing generic code.</div><div>same reason as = why we prefer:</div><div>auto y=3Dto!double(x) over auto y=3Dto_double(x);<= /div><div><br></div> --089e015370f4d8640804de5874d6--
Jun 04 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 4 June 2013 at 18:46:49 UTC, Walter Bright wrote:
 On 6/4/2013 11:43 AM, Timothee Cour wrote:
 writing generic code.
 same reason as why we prefer:
 auto y=to!double(x) over auto y=to_double(x);

The situations aren't comparable. The to!double case is parameterizing with a type, the compress one is not. Secondly, compress(lzw) does ABSOLUTELY NOTHING but turn around and call lzw. It adds nothing.

Currently. However, compress could become more feature-rich in the future. Perhaps there's some scope for automatic algorithm/parameter selection based on the type and length(if available) of what gets passed.
Jun 04 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Tuesday, 4 June 2013 at 19:00:35 UTC, John Colvin wrote:
 On Tuesday, 4 June 2013 at 18:46:49 UTC, Walter Bright wrote:
 On 6/4/2013 11:43 AM, Timothee Cour wrote:
 writing generic code.
 same reason as why we prefer:
 auto y=to!double(x) over auto y=to_double(x);

The situations aren't comparable. The to!double case is parameterizing with a type, the compress one is not. Secondly, compress(lzw) does ABSOLUTELY NOTHING but turn around and call lzw. It adds nothing.

Currently. However, compress could become more feature-rich in the future. Perhaps there's some scope for automatic algorithm/parameter selection based on the type and length(if available) of what gets passed.

I think this is over-engineering. It's unlikely that an application will need to support multiple compression algorithms in the same piece of code, and even if it did, it would be trivial to implement this on top of the simple interface that Walter is using.
Jun 04 2013
prev sibling next sibling parent "Max Samukha" <maxsamukha gmail.com> writes:
On Tuesday, 4 June 2013 at 18:46:49 UTC, Walter Bright wrote:
 On 6/4/2013 11:43 AM, Timothee Cour wrote:
 writing generic code.
 same reason as why we prefer:
 auto y=to!double(x) over auto y=to_double(x);

The situations aren't comparable. The to!double case is parameterizing with a type, the compress one is not. Secondly, compress(lzw) does ABSOLUTELY NOTHING but turn around and call lzw. It adds nothing.

That "absolutely" based on limited personal experience is the biggest D's problem.
Jun 04 2013
prev sibling next sibling parent "Max Samukha" <maxsamukha gmail.com> writes:
On Wednesday, 5 June 2013 at 04:54:46 UTC, Andrei Alexandrescu 
wrote:
 On 6/5/13 12:44 AM, Max Samukha wrote:
 On Tuesday, 4 June 2013 at 18:46:49 UTC, Walter Bright wrote:
 On 6/4/2013 11:43 AM, Timothee Cour wrote:
 writing generic code.
 same reason as why we prefer:
 auto y=to!double(x) over auto y=to_double(x);

The situations aren't comparable. The to!double case is parameterizing with a type, the compress one is not. Secondly, compress(lzw) does ABSOLUTELY NOTHING but turn around and call lzw. It adds nothing.

That "absolutely" based on limited personal experience is the biggest D's problem.

It's a point, but "biggest" is also kind of too much and based on limited personal experience :o). Andrei

Yeah, I noticed that.
Jun 04 2013
prev sibling next sibling parent "Max Samukha" <maxsamukha gmail.com> writes:
On Wednesday, 5 June 2013 at 06:18:54 UTC, Walter Bright wrote:
 On 6/4/2013 9:44 PM, Max Samukha wrote:
 On Tuesday, 4 June 2013 at 18:46:49 UTC, Walter Bright wrote:
 On 6/4/2013 11:43 AM, Timothee Cour wrote:
 writing generic code.
 same reason as why we prefer:
 auto y=to!double(x) over auto y=to_double(x);

The situations aren't comparable. The to!double case is parameterizing with a type, the compress one is not. Secondly, compress(lzw) does ABSOLUTELY NOTHING but turn around and call lzw. It adds nothing.

That "absolutely" based on limited personal experience is the biggest D's problem.

I've seen an awful lot of abstractions over the years that provided zero value.

I understand. But I've also seen a lot of abstractions over the years that seemed useless initially but were discovered to be extremely useful later (Bayes theorem is an example - it took 300 years to find a concrete use for it). So "a compelling use case" is not a sufficient criterion for evaluating usefulness of abstractions.
 You need to provide a compelling use case to justify another 
 layer of complexity. "generic code" is not a compelling use 
 case. It's already generic.

 Note how these components are to be used:

     src.lzwCompress.copy(dst);

 Your proposal is:

     src.compress(lzw).copy(dst);

 I.e. zero value, as so far all compress() does is call lzw().

That's not my proposal. Honestly I didn't even take a close look at it. I just felt like it was time to attack you - there is an explicit permission for casual trolling you gave.
 The whole point of range-based pipeline programming is you can 
 just plug in different components. There is no demonstrated use 
 case for adding another layer.

Ok.
 I am actually wrong in saying it has zero value. It has 
 negative value :-)

In this particular case, maybe.
Jun 04 2013
prev sibling next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
--001a11c2abf879630804de62ad27
Content-Type: text/plain; charset=ISO-8859-1

What I suggested in my original post didn't involve any
indirection/abstraction; simply a renaming to be consistent with existing
zlib (see my points A+B in my 1st post on this thread):

std.compress.zlib.compress
std.compress.zlib.uncompress
std.compress.lzw.compress
std.compress.lzw.uncompress

same reason we have: std.file.write, std.stdio.write, etc, and not
std.fileWrite, std.stdioWrite.

On Tue, Jun 4, 2013 at 11:18 PM, Walter Bright
<newshound2 digitalmars.com>wrote:

 On 6/4/2013 9:44 PM, Max Samukha wrote:

 On Tuesday, 4 June 2013 at 18:46:49 UTC, Walter Bright wrote:

 On 6/4/2013 11:43 AM, Timothee Cour wrote:

 writing generic code.
 same reason as why we prefer:
 auto y=to!double(x) over auto y=to_double(x);

The situations aren't comparable. The to!double case is parameterizing with a type, the compress one is not. Secondly, compress(lzw) does ABSOLUTELY NOTHING but turn around and call lzw. It adds nothing.

That "absolutely" based on limited personal experience is the biggest D's problem.

I've seen an awful lot of abstractions over the years that provided zero value. You need to provide a compelling use case to justify another layer of complexity. "generic code" is not a compelling use case. It's already generic. Note how these components are to be used: src.lzwCompress.copy(dst); Your proposal is: src.compress(lzw).copy(dst); I.e. zero value, as so far all compress() does is call lzw(). The whole point of range-based pipeline programming is you can just plug in different components. There is no demonstrated use case for adding another layer. I am actually wrong in saying it has zero value. It has negative value :-)

--001a11c2abf879630804de62ad27 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable What I suggested in my original post didn&#39;t involve any indirection/abs= traction; simply a renaming to be consistent with existing zlib (see my poi= nts A+B in my 1st post on this thread):<div><div style=3D"font-size:13px;co= lor:rgb(34,34,34);font-family:arial,sans-serif;background-color:rgb(255,255= ,255)"> <br></div><div style=3D"font-size:13px;color:rgb(34,34,34);font-family:aria= l,sans-serif;background-color:rgb(255,255,255)">std.compress.zlib.compress<= /div><div style=3D"font-size:13px;color:rgb(34,34,34);font-family:arial,san= s-serif;background-color:rgb(255,255,255)"> std.compress.zlib.uncompress</div><div style=3D"font-size:13px;color:rgb(34= ,34,34);font-family:arial,sans-serif;background-color:rgb(255,255,255)">std= .compress.lzw.compress</div><div style=3D"font-size:13px;color:rgb(34,34,34= );font-family:arial,sans-serif;background-color:rgb(255,255,255)"> std.compress.lzw.uncompress</div><div style=3D"font-size:13px;color:rgb(34,= 34,34);font-family:arial,sans-serif;background-color:rgb(255,255,255)"><br>= </div><div style=3D"font-size:13px;color:rgb(34,34,34);font-family:arial,sa= ns-serif;background-color:rgb(255,255,255)"> same reason we have: std.file.write, std.stdio.write, etc, and not std.file= Write, std.stdioWrite.</div><div style=3D"font-size:13px;color:rgb(34,34,34= );font-family:arial,sans-serif;background-color:rgb(255,255,255)"><br></div=

span dir=3D"ltr">&lt;<a href=3D"mailto:newshound2 digitalmars.com" target= =3D"_blank">newshound2 digitalmars.com</a>&gt;</span> wrote:<br><blockquote= class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc soli= d;padding-left:1ex"> <div class=3D"HOEnZb"><div class=3D"h5">On 6/4/2013 9:44 PM, Max Samukha wr= ote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On Tuesday, 4 June 2013 at 18:46:49 UTC, Walter Bright wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On 6/4/2013 11:43 AM, Timothee Cour wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> writing generic code.<br> same reason as why we prefer:<br> auto y=3Dto!double(x) over auto y=3Dto_double(x);<br> </blockquote> <br> The situations aren&#39;t comparable. The to!double case is parameterizing = with a<br> type, the compress one is not. Secondly, compress(lzw) does ABSOLUTELY NOTH= ING<br> but turn around and call lzw. It adds nothing.<br> </blockquote> <br> That &quot;absolutely&quot; based on limited personal experience is the big= gest D&#39;s problem.<br> </blockquote> <br></div></div> I&#39;ve seen an awful lot of abstractions over the years that provided zer= o value.<br> <br> You need to provide a compelling use case to justify another layer of compl= exity. &quot;generic code&quot; is not a compelling use case. It&#39;s alre= ady generic.<br> <br> Note how these components are to be used:<br> <br> =A0 =A0 src.lzwCompress.copy(dst);<br> <br> Your proposal is:<br> <br> =A0 =A0 src.compress(lzw).copy(dst);<br> <br> I.e. zero value, as so far all compress() does is call lzw().<br> <br> The whole point of range-based pipeline programming is you can just plug in= different components. There is no demonstrated use case for adding another= layer.<br> <br> I am actually wrong in saying it has zero value. It has negative value :-)<= br> </blockquote></div><br></div> --001a11c2abf879630804de62ad27--
Jun 04 2013
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Wednesday, 5 June 2013 at 12:55:50 UTC, Andrei Alexandrescu
wrote:
 On 6/5/13 2:55 AM, Timothee Cour wrote:
 What I suggested in my original post didn't involve any
 indirection/abstraction; simply a renaming to be consistent 
 with
 existing zlib (see my points A+B in my 1st post on this 
 thread):

 std.compress.zlib.compress
 std.compress.zlib.uncompress
 std.compress.lzw.compress
 std.compress.lzw.uncompress

I think that's nice.

+1. D has many powerful features for handling module namespacing (e.g. "import lzw = std.compress.lzw"), let's enable people to make use of them. David
Jun 05 2013
prev sibling parent "Zach the Mystic" <reachzach gggggmail.com> writes:
On Wednesday, 5 June 2013 at 04:54:46 UTC, Andrei Alexandrescu 
wrote:
 That "absolutely" based on limited personal experience is the 
 biggest
 D's problem.

It's a point, but "biggest" is also kind of too much and based on limited personal experience :o). Andrei

Hey, if you ever need someone who can reliably answer with limited personal experience, I'm available. :-)
Jun 05 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, June 04, 2013 01:23:52 Walter Bright wrote:
 I think it's a bad idea to 'mangle' the name. First off, if it is private,
 it is not visible. And even being public, the anti-hijacking language
 features make it a non-problem. The whole point is to avoid the wretched C
 problems with a global name space, by not having a global name space.

Not visible? When was that fixed? Last time I checked, access level had zero effect on visibility, just your ability to actually call it. Access level is taken into account after overload resolution. So, if there's another, public symbol with the same name which would be as good a match as this one aside from access level, then you're going to get a compilation error - which is exactly why most of us argue that inaccessible symbols should not be visible. But that requires a language change (which should definitely happen IMHO, but AFAIK, it still hasn't). - Jonathan M Davis
Jun 04 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, June 04, 2013 11:46:48 Walter Bright wrote:
 On 6/4/2013 11:43 AM, Timothee Cour wrote:
 writing generic code.
 same reason as why we prefer:
 auto y=to!double(x) over auto y=to_double(x);

The situations aren't comparable. The to!double case is parameterizing with a type, the compress one is not. Secondly, compress(lzw) does ABSOLUTELY NOTHING but turn around and call lzw. It adds nothing.

Well, I'd expect it to be compress!lzw(), but in any case, what it buys you is that you can pass the algorithm around without caring what it is so that while code higher up on the stack may have to know that it's lzw, code deeper down doesn't have to care what type of algorithm it's using. Now, whether that flexibility is all that useful in this particular case, I don't know, but it _does_ help with generic code. It's like how a lot of std.algorithm takes its predicate as an alias. - Jonathan M Davis
Jun 04 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, June 04, 2013 13:15:07 Walter Bright wrote:
 On 6/4/2013 11:55 AM, Jonathan M Davis wrote:
 Well, I'd expect it to be compress!lzw(), but in any case, what it buys
 you is that you can pass the algorithm around without caring what it is
 so that while code higher up on the stack may have to know that it's lzw,
 code deeper down doesn't have to care what type of algorithm it's using.
 Now, whether that flexibility is all that useful in this particular case,
 I don't know, but it _does_ help with generic code. It's like how a lot
 of std.algorithm takes its predicate as an alias.

There is zero utility in this: auto compress(alias dg) { return dg(); } Not even for generic code.

If that's all it's doing, then no, it wouldn't be useful to pass it as an argument. I was just pointing out that there are plenty of cases where passing functions to generic algorithms is an improvement. I haven't looked at what you've done yet, so I can't really comment on the details of this particular case. - Jonathan M Davis
Jun 04 2013
prev sibling next sibling parent Byron Heads <byron.heads gmail.com> writes:
On Tue, 04 Jun 2013 13:15:07 -0700, Walter Bright wrote:

 On 6/4/2013 11:55 AM, Jonathan M Davis wrote:
 Well, I'd expect it to be compress!lzw(), but in any case, what it buys
 you is that you can pass the algorithm around without caring what it is
 so that while code higher up on the stack may have to know that it's
 lzw, code deeper down doesn't have to care what type of algorithm it's
 using. Now, whether that flexibility is all that useful in this
 particular case, I don't know, but it _does_ help with generic code.
 It's like how a lot of std.algorithm takes its predicate as an alias.

There is zero utility in this: auto compress(alias dg) { return dg(); } Not even for generic code.

but a compress interface would be nice: interface Compress { ubyte[] compress(ubyte[]); ubyte[] uncompress(ubyte[]); } that way you can use any compress algorithm bool send(Compress)(Socket sock);
Jun 04 2013
prev sibling next sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Tue, 04 Jun 2013 09:06:22 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 On 6/4/2013 5:17 AM, Adam D. Ruppe wrote:
 I actually wish we could have multiple modules in a single file.

I don't see much point to that in modern file systems.

Probably seek time if the files are scattered and not in cache. That's hardly a show stopper unless you have 17.156 files like the Java Runtime. But they 'solved' it by zipping them up. -- Marco
Jun 04 2013
prev sibling next sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Mon, 03 Jun 2013 20:44:04 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 Comments welcome.

LZW is a nice and fast general purpose algorithm and I welcome its addition to Phobos to build file format readers from it (MS-DOS compress, GIF, TIFF) or even just to compress data on the fly in RAM. Most people seem to have moved on to zlib though for pretty much anything else. Actually I just happened to attempt something similar. Influenced by your talk about modularity and bioinfornatic's micro benchmarking with reading FASTA files I try to wrap up the concepts of bit streams and algorithms processing them. But some of my design goals are different: a) Not-Invented-Here must take precedence. :D b) There is no other measure than bytes/second. c) Every algorithm must run in its own thread for maximal parallelism. (like Unix process piping) So it is not about parallel algorithms, but building processing pipelines that work like Unix where only circular buffers need to be shared from one algorithm to the next. Am Mon, 3 Jun 2013 23:40:06 -0700 schrieb Timothee Cour <thelastmammoth gmail.com>:
 A)
 there already is std.zlib; why not have:
 std.compress.zlib: public import std.zlib
 std.compress.lzw: put this new module there instead of in std.compress
 std.compress.image.png
 std.compress.image.jpg

Yes and no. Compression algorithms should be in std.compress and share the same API, but image file formats in std.image.* or std.fileformat.*. You don't look into std.compress when you want to open *.bmps and *.jpgs. Am Tue, 04 Jun 2013 01:00:03 -0700 schrieb Walter Bright <newshound2 digitalmars.com>:
 On 6/3/2013 11:40 PM, Timothee Cour wrote:
 D)
 CircularBuffer belongs somewhere else; maybe std.range or std.container

I have mixed feelings about that. If you'll notice, std.compress doesn't have any imports! I wanted to make at least one module that doesn't pull in 100% of everything in Phobos (one of my pet peeves).

I have nothing to add to the discussion on THAT matter, but a compromise should be found between few massive imports (D) and hundreds of tiny imports (Java). :) -- Marco
Jun 04 2013
prev sibling next sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Tue, 04 Jun 2013 17:58:01 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 On 6/4/2013 5:13 PM, Marco Leise wrote:
 Probably seek time if the files are scattered and not in cache.
 That's hardly a show stopper unless you have 17.156 files like
 the Java Runtime. But they 'solved' it by zipping them up.

Actually, I've often thought of making dmd able to read everything it needs out of a zip file.

That would have been difficult for editors and IDEs that can look up file names from include paths only when they are not zipped up. It is good the way it is. -- Marco
Jun 04 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, June 05, 2013 00:14:59 Walter Bright wrote:
 When two modules have nothing to do with each other, they should be in
 separate modules.

Except that they're all compression algorithms, so they _are_ related. Having modules that are only a few hundred lines long is very counterproductive IMHO. It's highly annoying how Java insists on splitting everything up into different files. You end up with a lot of small files to wade through. Fortunately, D doesn't force that, and I don't think that we should go that route by choice. There's no more reason to split all of these up then there is to put each algorithm in std.algorithm in its own module. And yes, I know that you like that idea, but it seems ridiculous to me to try and have only one or two functions per module. We don't want them to be huge, but having them be very small is just as harmful IMHO. - Jonathan M Davis
Jun 05 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, June 05, 2013 09:31:01 Jacob Carlborg wrote:
 On 2013-06-05 08:59, Jonathan M Davis wrote:
 So, you want to create whole modules for each compression algorithm? That
 seems like overkill to me. What Walter currently has isn't even 1000 lines
 long (and that's including the CircularBuffer helper struct). Splitting it
 up like that seems like over-modularation to me.

The current modules in Phobos already contains too much. We shouldn't make the same mistake again.

Maybe some do, but many don't, and 1000 lines is _far_ from too much. If we start making modules that small, we're going to end up with tons of them to wade through to find anything. - Jonathan M Davis
Jun 05 2013
prev sibling next sibling parent "eles" <eles eles.com> writes:
On Wednesday, 5 June 2013 at 00:58:02 UTC, Walter Bright wrote:
 On 6/4/2013 5:13 PM, Marco Leise wrote:
 Actually, I've often thought of making dmd able to read 
 everything it needs out of a zip file.

I support that. It would make distributing source code cleaner. Most of the time you don't need to look at the code, just compile it, while still knowing that you have it available in an archive if you need it. Maybe that kind of support could improve the distribution of closed-source libraries, too: the generated .di files and the binaries could be packaged together in a zip file. More, the zip file could be really easy tested for self-containment. It happens sometime that a folder of code compiles, then when you package the whole thing and ship it, you discover that you forget to package inside some kind of file/dependency, and the customers are complaining about it. With a zip file, you just do a compile-check on the final package and, if ok, then it is ready for shipment. Btw, I cannot not resist, just adding here my favorite quote in software development: "It compiles. Let's ship it!" :)
Jun 05 2013
prev sibling next sibling parent "eles" <eles eles.com> writes:
On Wednesday, 5 June 2013 at 02:23:54 UTC, Marco Leise wrote:
 Am Tue, 04 Jun 2013 17:58:01 -0700
 schrieb Walter Bright <newshound2 digitalmars.com>:
 That would have been difficult for editors and IDEs that can
 look up file names from include paths only when they are not
 zipped up. It is good the way it is.

True, but Java also had the same issue with its .jar files and the editors adapted.
Jun 05 2013
prev sibling next sibling parent "Regan Heath" <regan netmail.co.nz> writes:
On Wed, 05 Jun 2013 11:17:06 +0100, Jacob Carlborg <doob me.com> wrote:

 On 2013-06-05 02:58, Walter Bright wrote:

 Actually, I've often thought of making dmd able to read everything it
 needs out of a zip file.

I think it's better to have a proper package manager.

I think it's better to have both :) R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Jun 05 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 5 June 2013 at 08:11:14 UTC, Walter Bright wrote:
 On 6/5/2013 12:38 AM, Jonathan M Davis wrote:
 Maybe some do, but many don't, and 1000 lines is _far_ from 
 too much. If we
 start making modules that small, we're going to end up with 
 tons of them to
 wade through to find anything.

1. It isn't any harder to find things in multiple files than in one file.

Although I think you're right about having smaller modules, I generally find it easier to browse through a larger file than many smaller files. Multiple files is ok if you know what you're looking for (grep) but when you're just trying to scan across a system to get a feel for how it's working, juggling many files is a real pita.
Jun 05 2013
prev sibling next sibling parent "Diggory" <diggsey googlemail.com> writes:
On Wednesday, 5 June 2013 at 11:30:10 UTC, John Colvin wrote:
 On Wednesday, 5 June 2013 at 08:11:14 UTC, Walter Bright wrote:
 On 6/5/2013 12:38 AM, Jonathan M Davis wrote:
 Maybe some do, but many don't, and 1000 lines is _far_ from 
 too much. If we
 start making modules that small, we're going to end up with 
 tons of them to
 wade through to find anything.

1. It isn't any harder to find things in multiple files than in one file.

Although I think you're right about having smaller modules, I generally find it easier to browse through a larger file than many smaller files. Multiple files is ok if you know what you're looking for (grep) but when you're just trying to scan across a system to get a feel for how it's working, juggling many files is a real pita.

Surely you would know which compression algorithm you wanted to change? If it's a general renaming or something not specific to a particular use then a file search is necessary anyway.
Jun 05 2013
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Wednesday, 5 June 2013 at 07:39:12 UTC, Jonathan M Davis wrote:
 Maybe some do, but many don't, and 1000 lines is _far_ from too 
 much. If we
 start making modules that small, we're going to end up with 
 tons of them to
 wade through to find anything.

 - Jonathan M Davis

We have a standard library in disagreement with the language's encapsulation mechanics. The module/package system in D is almost ignored in Phobos (and that's probably why the package system still has all these little things needing ironing out). It seems to owe influence to typical C and C++ library structure, which is simply suboptimal in D's module system. Third-party libraries tend to do a much better job at this. For example, Tango goes all out and embraces the package and module system, and the result is an extremely organized tree of modules with appropriate granularity. Code isn't hard to find because everything isn't just dumped into (bloated) blobs in a flat structure like in Phobos; it's organized into a tree. It seems like a no-brainer with the D language, and Phobos is the only D library I know that doesn't embrace this style of organization. The result is awful coupling throughout; with Phobos, we can't even write Hello World without pulling in half of the standard library. It's not just about the actual dependencies a module has, but the perceived dependencies; important from a readability perspective. I know a lot of D programmers embrace selective imports when working with Phobos, because just seeing a plain import statement such as "import std.datetime;" tells you very little about what the importing module actually does, and it's harder to figure out exactly where unqualified symbols come from when reading the module's code. I think the programmer should have a choice of convenience versus readability/fine dependency management when importing. The current module system does a decent job at enabling this already, and it's bound to get better with improvements like DIP37. Scripts and certain application code may want to prioritize productivity over finely managed dependencies, while library code - especially the *standard* library! - should definitely aim for lean coupling that makes sense. To that end, I think a lot of improvements can be made without breaking user code, but I'd be very much willing to see all kinds of breakage if it means we can get rid of the present standard library of substandard quality. The language may have been declared stable, but Phobos is in no laudable state.
Jun 05 2013
prev sibling next sibling parent reply "Tiago Martinez" <tiago.martinez gmail.com> writes:
On Tuesday, 4 June 2013 at 03:44:05 UTC, Walter Bright wrote:
 https://github.com/WalterBright/phobos/blob/std_compress/std/compress.d

 I wrote this to add components to compress and expand ranges.

 Highlights:

 1. doesn't do any memory allocation
 2. can handle arbitrarily large sets of data
 3. it's lazy
 4. takes an InputRange, and outputs an InputRange

 Comments welcome.

I may have misunderstood something, but the code does not implement LZW (a variant of LZ78), but a variant of LZ77 (i.e. deflate/ZIP). See https://en.wikipedia.org/wiki/LZ77_and_LZ78
Jun 05 2013
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
05-Jun-2013 16:16, Tiago Martinez пишет:
 On Tuesday, 4 June 2013 at 03:44:05 UTC, Walter Bright wrote:
 https://github.com/WalterBright/phobos/blob/std_compress/std/compress.d

 I wrote this to add components to compress and expand ranges.

 Highlights:

 1. doesn't do any memory allocation
 2. can handle arbitrarily large sets of data
 3. it's lazy
 4. takes an InputRange, and outputs an InputRange

 Comments welcome.

I may have misunderstood something, but the code does not implement LZW (a variant of LZ78), but a variant of LZ77 (i.e. deflate/ZIP).

+1 I thought to chime in with this too, keywords are: sliding window ===> LZ77 dictionary ===> LZW
 See https://en.wikipedia.org/wiki/LZ77_and_LZ78

-- Dmitry Olshansky
Jun 05 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/5/2013 10:46 AM, Dmitry Olshansky wrote:
 05-Jun-2013 16:16, Tiago Martinez пишет:
 I may have misunderstood something, but the code does not
 implement LZW (a variant of LZ78), but a variant of LZ77 (i.e.
 deflate/ZIP).

+1 I thought to chime in with this too, keywords are: sliding window ===> LZ77 dictionary ===> LZW
 See https://en.wikipedia.org/wiki/LZ77_and_LZ78


Thanks, you're both right.
Jun 05 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 5 June 2013 at 11:57:19 UTC, Diggory wrote:
 On Wednesday, 5 June 2013 at 11:30:10 UTC, John Colvin wrote:
 On Wednesday, 5 June 2013 at 08:11:14 UTC, Walter Bright wrote:
 On 6/5/2013 12:38 AM, Jonathan M Davis wrote:
 Maybe some do, but many don't, and 1000 lines is _far_ from 
 too much. If we
 start making modules that small, we're going to end up with 
 tons of them to
 wade through to find anything.

1. It isn't any harder to find things in multiple files than in one file.

Although I think you're right about having smaller modules, I generally find it easier to browse through a larger file than many smaller files. Multiple files is ok if you know what you're looking for (grep) but when you're just trying to scan across a system to get a feel for how it's working, juggling many files is a real pita.

Surely you would know which compression algorithm you wanted to change? If it's a general renaming or something not specific to a particular use then a file search is necessary anyway.

I eas speaking more generally, about phobos as a whole.
Jun 05 2013
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Wednesday, 5 June 2013 at 11:30:10 UTC, John Colvin wrote:
 Although I think you're right about having smaller modules, I 
 generally find it easier to browse through a larger file than 
 many smaller files.

 Multiple files is ok if you know what you're looking for (grep) 
 but when you're just trying to scan across a system to get a 
 feel for how it's working, juggling many files is a real pita.

Use an editor with a file tree sidebar? Quite on the contrary, I find many files to be much preferable, because you automatically have "bookmarks" in the source to come back to, and having the functionality already grouped in manageable logical units saves you from inferring that structure again, as it is the case when scrolling through a huge file. On a lighter note, if it's really a problem for you that module files are too small, what about just concatenating all the files in a given directory using a little shell magic? ;) David
Jun 05 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 5 June 2013 at 14:17:43 UTC, David Nadlinger wrote:
 On Wednesday, 5 June 2013 at 11:30:10 UTC, John Colvin wrote:
 Although I think you're right about having smaller modules, I 
 generally find it easier to browse through a larger file than 
 many smaller files.

 Multiple files is ok if you know what you're looking for 
 (grep) but when you're just trying to scan across a system to 
 get a feel for how it's working, juggling many files is a real 
 pita.

Use an editor with a file tree sidebar? Quite on the contrary, I find many files to be much preferable, because you automatically have "bookmarks" in the source to come back to, and having the functionality already grouped in manageable logical units saves you from inferring that structure again, as it is the case when scrolling through a huge file. On a lighter note, if it's really a problem for you that module files are too small, what about just concatenating all the files in a given directory using a little shell magic? ;) David

Agreed. To be honest, it's a trivial matter easily solved by a variety of tools, but I'm often just lazy and end up reading code with gedit or similar.
Jun 05 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, June 05, 2013 14:02:37 Jakob Ovrum wrote:
 We have a standard library in disagreement with the language's
 encapsulation mechanics. The module/package system in D is almost
 ignored in Phobos (and that's probably why the package system
 still has all these little things needing ironing out). It seems
 to owe influence to typical C and C++ library structure, which is
 simply suboptimal in D's module system.

I honestly don't see how Phobos is in disagreement with the module system. No, it doesn't use hierarchy as much as it should, and there are a few modules that are overly large (like std.algorithm or std.datetime), but for the most part, I don't see any problem with its level of encapsulation. It's mainly just its organization which could have been better. My primary objection here is that it seems ridiculous to me create lots of tiny modules. I hate how Java does that sort of thing, but there you're _forced_ to in many cases, whereas we have the opportunity to actually group things together in a single module where appropriate. And having whole modules with only one or two functions is way too small IMHO, and that seems to be what we're proposing here. - Jonathan M Davis
Jun 05 2013
prev sibling next sibling parent "Diggory" <diggsey googlemail.com> writes:
On Wednesday, 5 June 2013 at 17:21:01 UTC, Jonathan M Davis wrote:
 On Wednesday, June 05, 2013 14:02:37 Jakob Ovrum wrote:
 We have a standard library in disagreement with the language's
 encapsulation mechanics. The module/package system in D is 
 almost
 ignored in Phobos (and that's probably why the package system
 still has all these little things needing ironing out). It 
 seems
 to owe influence to typical C and C++ library structure, which 
 is
 simply suboptimal in D's module system.

I honestly don't see how Phobos is in disagreement with the module system. No, it doesn't use hierarchy as much as it should, and there are a few modules that are overly large (like std.algorithm or std.datetime), but for the most part, I don't see any problem with its level of encapsulation. It's mainly just its organization which could have been better. My primary objection here is that it seems ridiculous to me create lots of tiny modules. I hate how Java does that sort of thing, but there you're _forced_ to in many cases, whereas we have the opportunity to actually group things together in a single module where appropriate. And having whole modules with only one or two functions is way too small IMHO, and that seems to be what we're proposing here. - Jonathan M Davis

I agree with one or two functions it's far too small, but I'm in favour of having only one or two top-level classes/structs per module (there will be exceptional cases but in general) For examples: std.regex - I think it would be better if each implementation had its own module, plus a separate module for the parts common to all of them. Importing std.regex would publicly import the lot using the new package system. std.range - module for tests, ie. isXXX and hasXXX, module for algorithms ie. retro, take, etc., module for class wrappers std.datetime - split each class/struct into own module, systime alone is ~8000 lines
Jun 05 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jun 05, 2013 at 01:20:48PM -0400, Jonathan M Davis wrote:
 On Wednesday, June 05, 2013 14:02:37 Jakob Ovrum wrote:
 We have a standard library in disagreement with the language's
 encapsulation mechanics. The module/package system in D is almost
 ignored in Phobos (and that's probably why the package system
 still has all these little things needing ironing out). It seems
 to owe influence to typical C and C++ library structure, which is
 simply suboptimal in D's module system.

I honestly don't see how Phobos is in disagreement with the module system. No, it doesn't use hierarchy as much as it should, and there are a few modules that are overly large (like std.algorithm or std.datetime), but for the most part, I don't see any problem with its level of encapsulation. It's mainly just its organization which could have been better. My primary objection here is that it seems ridiculous to me create lots of tiny modules. I hate how Java does that sort of thing, but there you're _forced_ to in many cases, whereas we have the opportunity to actually group things together in a single module where appropriate. And having whole modules with only one or two functions is way too small IMHO, and that seems to be what we're proposing here.

As Andrei pointed out, I think we need to look at this not from a size perspective (number of lines, number of functions, etc.), but from an API perspective: do these functions/structs belong together, or are they only marginally related? More precisely, if some user code uses function X, is that code equally likely to also use Y? Are there common use cases in which only Y is used, not X? If the use of function X almost always implies the use of function Y (and vice versa), then they belong in the same module. Otherwise, I'd say they are candidates for splitting up. If function X uses function Z, and function Y also uses function Z, but the use of X does not necessarily imply the use of Y (and vice versa), then I'd argue that X, Y, and Z should be in separate modules to maximize reuse and reduce the amount of code you have to pull in (you shouldn't be forced to pull in Z just because you use X which calls Y, which Z happens to also call). This may be a bit heavy-handed for user code, but for Phobos, the standard library, I think the bar should be set higher. After all, one of the stated goals of Phobos is that you shouldn't need to pull in a whole ton of code just because you call a single function. Right now I think we're a bit short of that goal. T -- All men are mortal. Socrates is mortal. Therefore all men are Socrates.
Jun 05 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Wednesday, 5 June 2013 at 18:21:04 UTC, H. S. Teoh wrote:
 On Wed, Jun 05, 2013 at 01:20:48PM -0400, Jonathan M Davis 
 wrote:
 On Wednesday, June 05, 2013 14:02:37 Jakob Ovrum wrote:
 We have a standard library in disagreement with the 
 language's
 encapsulation mechanics. The module/package system in D is 
 almost
 ignored in Phobos (and that's probably why the package system
 still has all these little things needing ironing out). It 
 seems
 to owe influence to typical C and C++ library structure, 
 which is
 simply suboptimal in D's module system.

I honestly don't see how Phobos is in disagreement with the module system. No, it doesn't use hierarchy as much as it should, and there are a few modules that are overly large (like std.algorithm or std.datetime), but for the most part, I don't see any problem with its level of encapsulation. It's mainly just its organization which could have been better. My primary objection here is that it seems ridiculous to me create lots of tiny modules. I hate how Java does that sort of thing, but there you're _forced_ to in many cases, whereas we have the opportunity to actually group things together in a single module where appropriate. And having whole modules with only one or two functions is way too small IMHO, and that seems to be what we're proposing here.

As Andrei pointed out, I think we need to look at this not from a size perspective (number of lines, number of functions, etc.), but from an API perspective: do these functions/structs belong together, or are they only marginally related? More precisely, if some user code uses function X, is that code equally likely to also use Y? Are there common use cases in which only Y is used, not X? If the use of function X almost always implies the use of function Y (and vice versa), then they belong in the same module. Otherwise, I'd say they are candidates for splitting up. If function X uses function Z, and function Y also uses function Z, but the use of X does not necessarily imply the use of Y (and vice versa), then I'd argue that X, Y, and Z should be in separate modules to maximize reuse and reduce the amount of code you have to pull in (you shouldn't be forced to pull in Z just because you use X which calls Y, which Z happens to also call). This may be a bit heavy-handed for user code, but for Phobos, the standard library, I think the bar should be set higher. After all, one of the stated goals of Phobos is that you shouldn't need to pull in a whole ton of code just because you call a single function. Right now I think we're a bit short of that goal.

Massive +1 Modules are for grouping functions/types that are commonly used together or have interdependencies, not for grouping things that are in a similar category (although these things can be related). I don't care if levenshteinDistance is a "classic algorithm", I don't want to have to compile it every time I want to take the minimum of two numbers. Barely anyone is ever going to use it, so it should be off in a module on its own. There's absolutely nothing wrong with having lots of small modules provided that you don't end up importing the same sets of modules over and over. There are numerous advantages: 1. Makes it easier to manage dependencies. 1a. reduces compile times. 1b. reduces binary size. 1c. benefits incremental and distributed/parallel compilation. 2. Makes version control easier as more files means merge conflicts are less likely. 3. Makes it easier to navigate files. The only downside is that you may occasionally have to import more modules.
Jun 06 2013
prev sibling next sibling parent "SomeDude" <lovelydear mailmetrash.com> writes:
On Thursday, 6 June 2013 at 14:26:51 UTC, Peter Alexander wrote:
 Modules are for grouping functions/types that are commonly used 
 together or have interdependencies, not for grouping things 
 that are in a similar category (although these things can be 
 related).

 I don't care if levenshteinDistance is a "classic algorithm", I 
 don't want to have to compile it every time I want to take the 
 minimum of two numbers. Barely anyone is ever going to use it, 
 so it should be off in a module on its own.

 There's absolutely nothing wrong with having lots of small 
 modules provided that you don't end up importing the same sets 
 of modules over and over. There are numerous advantages:

 1. Makes it easier to manage dependencies.
 1a. reduces compile times.
 1b. reduces binary size.
 1c. benefits incremental and distributed/parallel compilation.
 2. Makes version control easier as more files means merge 
 conflicts are less likely.
 3. Makes it easier to navigate files.

 The only downside is that you may occasionally have to import 
 more modules.

Wise words !
Jun 06 2013
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 10, 2013 11:44:56 Daniel Murphy wrote:
 In this case we can prevent problem simply by not giving functions generic
 names like 'compress'.  Ideally you should be able to import the entire
 standard library with no name conflicts.

We've actually made the opposite choice when discussing this in the past. We've specifically gone for making functions which do the same thing in different modules having the same name (e.g. std.ascii and std.uni), which makes swapping one for the other easy and avoids having to come up with distinct names, though it does obviously create more naming conflicts when you try and mix and match such modules. I'd also point out that it's been argued that it's a failure of the module system if we're specifically trying to avoid having different modules have functions with the same name. It's the module system's job to differentiate such functions, and specifically avoiding naming stuff the same to avoid naming conflicts means that you're pretty much ignoring the module system. So, the general approach has been to name functions differently when they do different things and name them the same when they do the same thing and then let the module system take care of differentiating between the two when you need to. - Jonathan M Davis
Jun 09 2013
parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1001.1370829569.13711.digitalmars-d puremagic.com...
 On Monday, June 10, 2013 11:44:56 Daniel Murphy wrote:
 In this case we can prevent problem simply by not giving functions 
 generic
 names like 'compress'.  Ideally you should be able to import the entire
 standard library with no name conflicts.

We've actually made the opposite choice when discussing this in the past. We've specifically gone for making functions which do the same thing in different modules having the same name (e.g. std.ascii and std.uni), which makes swapping one for the other easy and avoids having to come up with distinct names, though it does obviously create more naming conflicts when you try and mix and match such modules. I'd also point out that it's been argued that it's a failure of the module system if we're specifically trying to avoid having different modules have functions with the same name. It's the module system's job to differentiate such functions, and specifically avoiding naming stuff the same to avoid naming conflicts means that you're pretty much ignoring the module system. So, the general approach has been to name functions differently when they do different things and name them the same when they do the same thing and then let the module system take care of differentiating between the two when you need to. - Jonathan M Davis

The difference here is these are range functions and you lose ufcs. It doesn't make much difference unless you are trying to chain them. Ranges, and call chaining of range-based functions using ufcs, are among the most attractive features of phobos. Let's define a new general approach, and keep them conflict-free when possible. Also, compress is a ridiculously general name for a function.
Jun 11 2013
prev sibling next sibling parent reply Timothee Cour <thelastmammoth gmail.com> writes:
--001a11c309a0fa199e04dec3396a
Content-Type: text/plain; charset=ISO-8859-1

ok I found what I think is the best solution to this problem :-)
see:
http://forum.dlang.org/post/mailman.1002.1370829729.13711.digitalmars-d-learn puremagic.com



On Sun, Jun 9, 2013 at 6:59 PM, Jonathan M Davis <jmdavisProg gmx.com>wrote:

 On Monday, June 10, 2013 11:44:56 Daniel Murphy wrote:
 In this case we can prevent problem simply by not giving functions

 names like 'compress'.  Ideally you should be able to import the entire
 standard library with no name conflicts.

We've actually made the opposite choice when discussing this in the past. We've specifically gone for making functions which do the same thing in different modules having the same name (e.g. std.ascii and std.uni), which makes swapping one for the other easy and avoids having to come up with distinct names, though it does obviously create more naming conflicts when you try and mix and match such modules. I'd also point out that it's been argued that it's a failure of the module system if we're specifically trying to avoid having different modules have functions with the same name. It's the module system's job to differentiate such functions, and specifically avoiding naming stuff the same to avoid naming conflicts means that you're pretty much ignoring the module system. So, the general approach has been to name functions differently when they do different things and name them the same when they do the same thing and then let the module system take care of differentiating between the two when you need to. - Jonathan M Davis

--001a11c309a0fa199e04dec3396a Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable ok I found what I think is the best solution to this problem=A0:-)<div>see:= <br><div><a href=3D"http://forum.dlang.org/post/mailman.1002.1370829729.137= 11.digitalmars-d-learn puremagic.com">http://forum.dlang.org/post/mailman.1= 002.1370829729.13711.digitalmars-d-learn puremagic.com</a><br> <div><br></div><div><br><br><div class=3D"gmail_quote">On Sun, Jun 9, 2013 = at 6:59 PM, Jonathan M Davis <span dir=3D"ltr">&lt;<a href=3D"mailto:jmdavi= sProg gmx.com" target=3D"_blank">jmdavisProg gmx.com</a>&gt;</span> wrote:<= br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left= :1px #ccc solid;padding-left:1ex"> <div class=3D"im">On Monday, June 10, 2013 11:44:56 Daniel Murphy wrote:<br=

eric<br> &gt; names like &#39;compress&#39;. =A0Ideally you should be able to import= the entire<br> &gt; standard library with no name conflicts.<br> <br> </div>We&#39;ve actually made the opposite choice when discussing this in t= he past.<br> We&#39;ve specifically gone for making functions which do the same thing in= <br> different modules having the same name (e.g. std.ascii and std.uni), which<= br> makes swapping one for the other easy and avoids having to come up with<br> distinct names, though it does obviously create more naming conflicts when = you<br> try and mix and match such modules. I&#39;d also point out that it&#39;s be= en argued<br> that it&#39;s a failure of the module system if we&#39;re specifically tryi= ng to avoid<br> having different modules have functions with the same name. It&#39;s the mo= dule<br> system&#39;s job to differentiate such functions, and specifically avoiding= naming<br> stuff the same to avoid naming conflicts means that you&#39;re pretty much = ignoring<br> the module system.<br> <br> So, the general approach has been to name functions differently when they d= o<br> different things and name them the same when they do the same thing and the= n<br> let the module system take care of differentiating between the two when you= <br> need to.<br> <span class=3D"HOEnZb"><font color=3D"#888888"><br> - Jonathan M Davis<br> </font></span></blockquote></div><br></div></div></div> --001a11c309a0fa199e04dec3396a--
Jun 09 2013
parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Timothee Cour" <thelastmammoth gmail.com> wrote in message 
news:mailman.1003.1370829991.13711.digitalmars-d puremagic.com...
 ok I found what I think is the best solution to this problem :-)
 see:
 http://forum.dlang.org/post/mailman.1002.1370829729.13711.digitalmars-d-learn puremagic.com

That's pretty awesome, but still much much much uglier than not having to disambiguate in the first place.
Jun 11 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 10 June 2013 at 01:59:29 UTC, Jonathan M Davis wrote:
 On Monday, June 10, 2013 11:44:56 Daniel Murphy wrote:
 In this case we can prevent problem simply by not giving 
 functions generic
 names like 'compress'.  Ideally you should be able to import 
 the entire
 standard library with no name conflicts.

We've actually made the opposite choice when discussing this in the past. We've specifically gone for making functions which do the same thing in different modules having the same name (e.g. std.ascii and std.uni), which makes swapping one for the other easy and avoids having to come up with distinct names, though it does obviously create more naming conflicts when you try and mix and match such modules. I'd also point out that it's been argued that it's a failure of the module system if we're specifically trying to avoid having different modules have functions with the same name. It's the module system's job to differentiate such functions, and specifically avoiding naming stuff the same to avoid naming conflicts means that you're pretty much ignoring the module system. So, the general approach has been to name functions differently when they do different things and name them the same when they do the same thing and then let the module system take care of differentiating between the two when you need to. - Jonathan M Davis

You are wise and speak the truth :P
Jun 09 2013
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Tuesday, 11 June 2013 at 13:13:56 UTC, Daniel Murphy wrote:
 Also, compress is a ridiculously general name for a function.

We have module-level functions called "copy" (multiple), "read", "write", "map", etc. already, and it's not a bad thing! It's OK because the full name is not "compress", but "std.compression.lz77.compress". This way, how specific the code wants to be depends on the user and the particular use-case, instead of one-size-fits-all alternatives like "lz77Compress". There's no redundancy in the name yet we still have the option to be pin-point specific (e.g. static import), and yes, we still get to use UFCS! To eliminate the UFCS problem - which doesn't happen very often (how often do you want to use two different compression algorithms in the same unit?), we can (must?) use renamed symbols when importing. Since any example using multiple "compress" functions would be contrived, I'll use an existing conflict - the case of "copy". The following program backs up the specified files and writes a nicely formatted message to stdout (OK, so a tiny bit contrived): ---- void main(string[] args) { import std.algorithm : chain, copy, joiner; import std.array : empty; import std.file : fileCopy = copy; // `fileCopy` is std.file.copy import std.stdio : stdout; auto fileNames = args[1 .. $]; foreach(fileName; fileNames) fileName.fileCopy(fileName ~ ".bak"); if(!fileNames.empty) "Backed up the following files: " .chain(fileNames.joiner(", ")) .copy(stdout.lockingTextWriter()); } ---- By eliminating redundancies from symbol names, we empower the user, and the module system offers all the tools necessary to solve conflicts in a variety of ways.
Jun 11 2013
prev sibling next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
--089e0118289e312b3d04dee54641
Content-Type: text/plain; charset=ISO-8859-1

On Tue, Jun 11, 2013 at 11:22 AM, Jakob Ovrum <jakobovrum gmail.com> wrote:

 On Tuesday, 11 June 2013 at 13:13:56 UTC, Daniel Murphy wrote:

 Also, compress is a ridiculously general name for a function.

We have module-level functions called "copy" (multiple), "read", "write", "map", etc. already, and it's not a bad thing! It's OK because the full name is not "compress", but "std.compression.lz77. **compress". This way, how specific the code wants to be depends on the user and the particular use-case, instead of one-size-fits-all alternatives like "lz77Compress". There's no redundancy in the name yet we still have the option to be pin-point specific (e.g. static import), and yes, we still get to use UFCS! To eliminate the UFCS problem - which doesn't happen very often (how often do you want to use two different compression algorithms in the same unit?), we can (must?) use renamed symbols when importing.

I have found a better way to do that: see http://forum.dlang.org/post/mailman.1002.1370829729.13711.digitalmars-d-learn puremagic.com subject: 'best way to handle UFCS with ambiguous names: using std.typetuple.Alias!' syntax: 'arg1.Alias!(std.file.write).arg2'* see related discussion for reasoning. I'd like to push this as standard way to deal with ambiguities.
 Since any example using multiple "compress" functions would be contrived,
 I'll use an existing conflict - the case of "copy".

 The following program backs up the specified files and writes a nicely
 formatted message to stdout (OK, so a tiny bit contrived):
 ----
 void main(string[] args)
 {
         import std.algorithm : chain, copy, joiner;
         import std.array : empty;
         import std.file : fileCopy = copy; // `fileCopy` is std.file.copy
         import std.stdio : stdout;

         auto fileNames = args[1 .. $];

         foreach(fileName; fileNames)
                 fileName.fileCopy(fileName ~ ".bak");

         if(!fileNames.empty)
                 "Backed up the following files: "
                         .chain(fileNames.joiner(", "))
                         .copy(stdout.**lockingTextWriter());
 }
 ----

 By eliminating redundancies from symbol names, we empower the user, and
 the module system offers all the tools necessary to solve conflicts in a
 variety of ways.

--089e0118289e312b3d04dee54641 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Tue, Jun 11, 2013 at 11:22 AM, Jakob Ovrum <span dir=3D"ltr">&lt;<a href= =3D"mailto:jakobovrum gmail.com" target=3D"_blank">jakobovrum gmail.com</a>= &gt;</span> wrote:<br><div class=3D"gmail_quote"><blockquote class=3D"gmail= _quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:= 1ex"> <div class=3D"im">On Tuesday, 11 June 2013 at 13:13:56 UTC, Daniel Murphy w= rote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Also, compress is a ridiculously general name for a function.<br> </blockquote> <br></div> We have module-level functions called &quot;copy&quot; (multiple), &quot;re= ad&quot;, &quot;write&quot;, &quot;map&quot;, etc. already, and it&#39;s no= t a bad thing!<br> <br> It&#39;s OK because the full name is not &quot;compress&quot;, but &quot;st= d.compression.lz77.<u></u>compress&quot;. This way, how specific the code w= ants to be depends on the user and the particular use-case, instead of one-= size-fits-all alternatives like &quot;lz77Compress&quot;. There&#39;s no re= dundancy in the name yet we still have the option to be pin-point specific = (e.g. static import), and yes, we still get to use UFCS!<br> <br> To eliminate the UFCS problem - which doesn&#39;t happen very often (how of= ten do you want to use two different compression algorithms in the same uni= t?), we can (must?) use renamed symbols when importing.<br></blockquote> <div><br></div><div>I have found a better way to do that: see <a href=3D"ht= tp://forum.dlang.org/post/mailman.1002.1370829729.13711.digitalmars-d-learn= puremagic.com">http://forum.dlang.org/post/mailman.1002.1370829729.13711.d= igitalmars-d-learn puremagic.com</a></div> <div>subject: &#39;best way to handle UFCS with ambiguous names: using std.= typetuple.Alias!&#39;</div><div>syntax: &#39;arg1.Alias!(std.file.write).ar= g2&#39;*=A0</div><div>see related discussion for reasoning. I&#39;d like to= push this as standard way to deal with ambiguities.</div> <div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;= border-left:1px #ccc solid;padding-left:1ex"> <br> Since any example using multiple &quot;compress&quot; functions would be co= ntrived, I&#39;ll use an existing conflict - the case of &quot;copy&quot;.<= br> <br> The following program backs up the specified files and writes a nicely form= atted message to stdout (OK, so a tiny bit contrived):<br> ----<br> void main(string[] args)<br> {<br> =A0 =A0 =A0 =A0 import std.algorithm : chain, copy, joiner;<br> =A0 =A0 =A0 =A0 import std.array : empty;<br> =A0 =A0 =A0 =A0 import std.file : fileCopy =3D copy; // `fileCopy` is std.f= ile.copy<br> =A0 =A0 =A0 =A0 import std.stdio : stdout;<br> <br> =A0 =A0 =A0 =A0 auto fileNames =3D args[1 .. $];<br> <br> =A0 =A0 =A0 =A0 foreach(fileName; fileNames)<br> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 fileName.fileCopy(fileName ~ &quot;.bak&quo= t;);<br> <br> =A0 =A0 =A0 =A0 if(!fileNames.empty)<br> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 &quot;Backed up the following files: &quot;= <br> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 .chain(fileNames.joiner(&qu= ot;, &quot;))<br> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 .copy(stdout.<u></u>locking= TextWriter());<br> }<br> ----<br> <br> By eliminating redundancies from symbol names, we empower the user, and the= module system offers all the tools necessary to solve conflicts in a varie= ty of ways.<br> </blockquote></div><br><div><br></div><div><br></div> --089e0118289e312b3d04dee54641--
Jun 11 2013
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Tuesday, 11 June 2013 at 18:43:45 UTC, Timothee Cour wrote:
 I have found a better way to do that: see
 http://forum.dlang.org/post/mailman.1002.1370829729.13711.digitalmars-d-learn puremagic.com
 subject: 'best way to handle UFCS with ambiguous names: using
 std.typetuple.Alias!'
 syntax: 'arg1.Alias!(std.file.write).arg2'*
 see related discussion for reasoning. I'd like to push this as 
 standard way
 to deal with ambiguities.

It's clearly an option, but I think it's too syntactically heavy, causing more harm than good (the idea of UFCS is, of course, readability!). Since these conflicting symbols are in the minority for the vast majority of code units, I think renamed symbols are much, much better.
Jun 11 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, June 11, 2013 20:50:17 Jakob Ovrum wrote:
 On Tuesday, 11 June 2013 at 18:43:45 UTC, Timothee Cour wrote:
 I have found a better way to do that: see
 http://forum.dlang.org/post/mailman.1002.1370829729.13711.digitalmars-d-le
 arn puremagic.com subject: 'best way to handle UFCS with ambiguous names:
 using
 std.typetuple.Alias!'
 syntax: 'arg1.Alias!(std.file.write).arg2'*
 see related discussion for reasoning. I'd like to push this as
 standard way
 to deal with ambiguities.

It's clearly an option, but I think it's too syntactically heavy, causing more harm than good (the idea of UFCS is, of course, readability!). Since these conflicting symbols are in the minority for the vast majority of code units, I think renamed symbols are much, much better.

Agreed. - Jonathan M Davis
Jun 11 2013
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Tuesday, 11 June 2013 at 22:34:55 UTC, Daniel Murphy wrote:
 There is a reason we don't call every function in phobos 
 'process' and let
 the module name tell us what is actually does - when you see 
 the name in
 your source code, it is easy to recognize what is being done.

"copy", "write" and "compress" are perfectly recognizable names.
 tl;dr We have great tools to disambiguate when we have to.  
 Let's not have
 to.

The way I see it, you're asking that all code should pay for the benefit of a minority of cases. I'd choose the inverse.
Jun 12 2013
prev sibling next sibling parent "Michal Minich" <michal.minich gmail.com> writes:
On Thursday, 13 June 2013 at 11:36:16 UTC, Daniel Murphy wrote:

 Ok, how exactly is the data compressed in the following 
 snippet?  No
 scrolling up to the top of the module to see what's imported!

 newdata = data.compress();

You can have that argument for any single overload and virtual call. At least you know it statically; with virtual you don't know until runtime... In many languages you would have interface ICompressor { Stream compress (Stream s) }...
Jun 13 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Thursday, 13 June 2013 at 11:36:16 UTC, Daniel Murphy wrote:
 Ok, how exactly is the data compressed in the following 
 snippet?  No
 scrolling up to the top of the module to see what's imported!

 newdata = data.compress();

If it's not obvious from the context, just be explicit. newdata = std.compression.lz77.compress(data); Don't force verbosity on everyone just in case someone wants it.
Jun 13 2013
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 6/13/13, Peter Alexander <peter.alexander.au gmail.com> wrote:
 If it's not obvious from the context, just be explicit.

 newdata = std.compression.lz77.compress(data);

 Don't force verbosity on everyone just in case someone wants it.

What happens when we get std.compression.lz78 and you end up accidentally calling compress on with lz77 and expand with lz78? Pseudocoding: module deserialize; import std.compression.lz77; auto readfile(string filename) { return readFile(filename).expand; } module serialize; import std.compression.lz78; // oops! void writeFile(T)(T[] data, string filename) { writeFile(filename, data.compress); } Imports are incredibly easy to screw up. But if we used types instead of global modules then we could not only make our calling code clearer (and less buggy), but it would also allow us to use package imports so we can use any compression algorithm: module deserialize; import std.compression; // package import, e.g. imports lz77, lz78, etc modules auto readfile(string filename) { return filename.readFile.lz77.expand; } module serialize; import std.compression; // package import void writeFile(T)(T[] data, string filename) { data.lz77.compress.writeFile(filename); } "lz77" would be an auto function which takes the buffer and returns a Lz77 struct that has expand/compress methods.
Jun 13 2013
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 6/13/13, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:
 But if we used types instead of global modules

*global functions*
Jun 13 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Thursday, 13 June 2013 at 13:15:03 UTC, Andrej Mitrovic wrote:
 What happens when we get std.compression.lz78 and you end up
 accidentally calling compress on with lz77 and expand with lz78?

The exact same typo could happen with your structs. You haven't solved anything:
 module serialize;
 import std.compression;  // package import
 void writeFile(T)(T[] data, string filename) {
 data.lz77.compress.writeFile(filename); }

void writeFile(T)(T[] data, string filename) { data.lz78.compress.writeFile(filename); } oops!
Jun 13 2013
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 13 June 2013 at 13:15:03 UTC, Andrej Mitrovic wrote:
 What happens when we get std.compression.lz78 and you end up
 accidentally calling compress on with lz77 and expand with lz78?
 […]
 Imports are incredibly easy to screw up.

I think this argument is invalid: A typo in an import statement is just as likely as in a function call. David
Jun 13 2013
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 13 June 2013 at 11:36:16 UTC, Daniel Murphy wrote:
 Ok, how exactly is the data compressed in the following 
 snippet?  No
 scrolling up to the top of the module to see what's imported!

I don't need to scroll to the top of the module, just a few lines up because I'm using function-local imports anyway. :P If you want extra verbosity (which can be good *sometimes*), just write "import lz77 = std.compression.lz77" and you are good to go. David
Jun 13 2013
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 13 June 2013 at 23:45:12 UTC, Daniel Murphy wrote:
 I don't think 4 characters is a high price to pay for the added 
 clarity.
 Then there is no ambiguity, no need to rename imports, no 
 problems using
 ufcs.  Every time I see lz77Compress in anybody's code I know 
 exactly what
 it does!

import std.compression : lz77Compress = lz78Compress; ;)
Jun 13 2013
prev sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Thursday, 13 June 2013 at 23:45:12 UTC, Daniel Murphy wrote:
 I don't think 4 characters is a high price to pay for the added 
 clarity.
 Then there is no ambiguity, no need to rename imports, no 
 problems using
 ufcs.  Every time I see lz77Compress in anybody's code I know 
 exactly what
 it does!

I recommend you just use local imports if it bother you that much, then it's obvious: import std.compression.lz77; auto newdata = compress(data); Really, it should be obvious from the context which compression algorithm you are using.
 I understand the motivation for shortening function names that 
 will be used
 frequently... but this is not in that category.

This is not the motivation. The problem with lz77compress is that it is redundant: std.compression.lz77.lz77compress It's bad style to repeat the module name in module identifiers. It completely defeats the purpose of using modules as namespaces. If all the compression algorithms were inside std.compression instead of having their own modules then yes, lz77compress would be a fantastic name, but they're not, so it's not.
Jun 14 2013