www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - enforce()?

reply bearophile <bearophileHUGS lycos.com> writes:
I have counted about 200 usages of std.contracts.enforce() inside Phobos. Can
you tell me what's the purpose of enforce() in a language that has built-in
Contract Programming?

And what are the purposes of std.contracts.AssumeSorted()? Is it useful for
something?

Bye,
bearophile
(I know this is not the digitalmars.D.learn newsgroup).
Jun 15 2010
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
bearophile wrote:
 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?

You need to read TDPL for that :o).
 And what are the purposes of std.contracts.AssumeSorted()? Is it
 useful for something?

AssumeSorted was an experiment. I think it has drawbacks that I don't know how to address, so I'll retire it. Andrei
Jun 15 2010
next sibling parent reply Bernard Helyer <b.helyer gmail.com> writes:
You need to read TDPL for that :o).

Please don't start replying to queries in this fashion. Not everyone has the wherewithal to get a copy of a book such as TDPL. Especially seeing as you're the author, this kind of reply just looks like whoring for the book. I'm not saying that's what it is, just what it can look like. I've got TDPL on the way from Amazon, by the way. I just don't want to see this reply, and wanted to express my distaste.
Jun 16 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bernard Helyer wrote:
 You need to read TDPL for that :o).

Please don't start replying to queries in this fashion. Not everyone has the wherewithal to get a copy of a book such as TDPL. Especially seeing as you're the author, this kind of reply just looks like whoring for the book. I'm not saying that's what it is, just what it can look like. I've got TDPL on the way from Amazon, by the way. I just don't want to see this reply, and wanted to express my distaste.

All right, all right. Basically there's a marked difference between contract checking (which verifies the architectural integrity of a program) and error handling (which deals with errors that occur in correct programs). Contracts help with the former, enforce helps with the latter. The differences are marked enough that TDPL dedicates a separate chapter to each. Andrei
Jun 16 2010
parent Walter Bright <newshound1 digitalmars.com> writes:
Andrei Alexandrescu wrote:
 Basically there's a marked difference between contract checking (which 
 verifies the architectural integrity of a program) and error handling 
 (which deals with errors that occur in correct programs). Contracts help 
 with the former, enforce helps with the latter.
 
 The differences are marked enough that TDPL dedicates a separate chapter 
 to each.

Yes, I agree it is extremely important to separate the concepts of contract checking from error handling.
Jun 16 2010
prev sibling next sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Wed, 16 Jun 2010 00:18:03 -0700, Andrei Alexandrescu wrote:

 Bernard Helyer wrote:
 You need to read TDPL for that :o).

Please don't start replying to queries in this fashion. Not everyone has the wherewithal to get a copy of a book such as TDPL. Especially seeing as you're the author, this kind of reply just looks like whoring for the book. I'm not saying that's what it is, just what it can look like. I've got TDPL on the way from Amazon, by the way. I just don't want to see this reply, and wanted to express my distaste.

All right, all right. Basically there's a marked difference between contract checking (which verifies the architectural integrity of a program) and error handling (which deals with errors that occur in correct programs). Contracts help with the former, enforce helps with the latter.

I think any confusion regarding this may stem from the fact that enforce () resides in std.contracts. Personally, I think it's worth moving it to object.d, but maybe it's too late for that? Anyway, I love enforce() -- it's become my standard error handling tool. -Lars
Jun 16 2010
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 15 Jun 2010 22:23:15 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 bearophile wrote:
 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?

You need to read TDPL for that :o).
 And what are the purposes of std.contracts.AssumeSorted()? Is it
 useful for something?

AssumeSorted was an experiment. I think it has drawbacks that I don't know how to address, so I'll retire it.

Hm... what are the drawbacks (besides it not being enforced)? I thought it was a good solution. -Steve
Jun 16 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 06/16/2010 05:47 AM, Steven Schveighoffer wrote:
 On Tue, 15 Jun 2010 22:23:15 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 bearophile wrote:
 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?

You need to read TDPL for that :o).
 And what are the purposes of std.contracts.AssumeSorted()? Is it
 useful for something?

AssumeSorted was an experiment. I think it has drawbacks that I don't know how to address, so I'll retire it.

Hm... what are the drawbacks (besides it not being enforced)? I thought it was a good solution.

Sorry I took so long (over one month!) to reply to this. I've delayed the reply to the point when it could be integrated within the upcoming thread about improving std.algorithm.find. The problem with AssumeSorted is that generally predicates in D (and also in most other languages) are not easy to compare. Let's first recall AssumSorted's definition. It's just a wrapper: /** Passes the type system the information that $(D range) is already sorted by predicate $(D pred). No checking is performed; debug builds may insert checks randomly. To insert a check, see $(XREF algorithm, isSorted). */ struct AssumeSorted(Range, alias pred = "a < b") { /// Alias for $(D Range). alias Range AssumeSorted; /// The passed-in range. Range assumeSorted; /// The sorting predicate. alias pred assumeSortedBy; } /// Ditto AssumeSorted!(Range, pred) assumeSorted(alias pred = "a < b", Range) (Range r) { AssumeSorted!(Range, pred) result; result.assumeSorted = r; return result; } The recommended way to use the facility is: int[] a = [ -1, 0, 1, 2, 3, 4, 5 ]; assert(find(assumeSorted(a), 3) == [ 3, 4, 5 ]); find() uses simple means to detect that its first argument has type AssumeSorted and takes advantage of that when searching (specifically by doing binary search). So far so good. The problem ensues when we want to make sure that the sorting predicate is in sync with the search predicate. For example, if the search predicate is "==" then it's okay to use "<" or ">" as a sorting predicate. But searching for "a.zip == b.zip" in a range sorted by "a.name < b.name" is not okay. If predicates were all expressed as strings, probably some string manipulation could be done to see whether they are compatible. But as things stand, assertSorted has quite limited power. Andrei
Jul 17 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/19/2010 06:36 AM, Steven Schveighoffer wrote:
 Just thinking out loud here, couldn't you use the predicate already in
 AssumeSorted? I mean, if you're going to pass AssumeSorted into find,
 you don't want to also specify the predicate as then the range just
 becomes a standard range.

 There must be some kind of way to use template constraints to kill the
 predicate arg to find when the range is an AssumeSorted struct. If not,
 there should be.

That's a good idea. The find predicate that could be derived from AssumeSorted's predicate pred would be !pred(a, b) && !pred(b, a). Thanks, Steve. Andrei
Jul 19 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/19/2010 09:27 AM, Steven Schveighoffer wrote:
 On Mon, 19 Jul 2010 09:36:54 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 On 07/19/2010 06:36 AM, Steven Schveighoffer wrote:
 Just thinking out loud here, couldn't you use the predicate already in
 AssumeSorted? I mean, if you're going to pass AssumeSorted into find,
 you don't want to also specify the predicate as then the range just
 becomes a standard range.

 There must be some kind of way to use template constraints to kill the
 predicate arg to find when the range is an AssumeSorted struct. If not,
 there should be.

That's a good idea. The find predicate that could be derived from AssumeSorted's predicate pred would be !pred(a, b) && !pred(b, a). Thanks, Steve.

You're welcome :) BTW, you don't need the combo predicate until the very end. Basically, you do a binary search for the first element where pred(a, E) is false (where E is the target), and then see if pred(E, a) is also false on that element (to test for equality).

You mean like this? :o) http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/algorithm.d?rev=1279#L4703 Andrei
Jul 19 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/19/2010 10:24 AM, Steven Schveighoffer wrote:
 On Mon, 19 Jul 2010 11:10:03 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 On 07/19/2010 09:27 AM, Steven Schveighoffer wrote:
 On Mon, 19 Jul 2010 09:36:54 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 On 07/19/2010 06:36 AM, Steven Schveighoffer wrote:
 Just thinking out loud here, couldn't you use the predicate already in
 AssumeSorted? I mean, if you're going to pass AssumeSorted into find,
 you don't want to also specify the predicate as then the range just
 becomes a standard range.

 There must be some kind of way to use template constraints to kill the
 predicate arg to find when the range is an AssumeSorted struct. If
 not,
 there should be.

That's a good idea. The find predicate that could be derived from AssumeSorted's predicate pred would be !pred(a, b) && !pred(b, a). Thanks, Steve.

You're welcome :) BTW, you don't need the combo predicate until the very end. Basically, you do a binary search for the first element where pred(a, E) is false (where E is the target), and then see if pred(E, a) is also false on that element (to test for equality).

You mean like this? :o) http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/algorithm.d?rev=1279#L4703

Yep. I realized after I wrote this that you probably were already doing it :)

Yah, it's quite the STL classic. STL commonly defines implicitly equivalence in terms of !less(a, b) && !less(b, a) but uses only one of the two comparisons until the last leg, when it's testing the opposite way.
 Interestingly, I found that when doing the redblacktree that I tried to
 do some optimization on the lookup of an element. Basically, while I'm
 going down the tree looking for a single element, I'm using the binary
 predicate to move left or right. However, if I move left (i.e. it's not
 less), then that could be the element I'm looking for! So I try the
 opposite of the predicate to see if I should return.

Indeed that's 100% what STL's lower_bound and rb_tree.find do. By the way, I'm still eagerly waiting for your red-black tree implementation. I think it would be pure awesomeness if you massaged the red/black bit inside one of the pointers. I figured out a way of doing that without throwing off the garbage collector: union { unsigned byte * _gcHelper; size_t _bits; } bool setRed() { _bits |= 1; } bool setBlack() { _bits &= ~(cast(size_t) 1); } bool isRed() { return _bits & 1; } RBTreeNode * left() { return cast(RBTreeNode *) cast(size_t) (_bits & ~(cast(size_t) 1)); } The idea is to leave _gcHelper in there as a valid pointer to either a RBTreeNode or a pointer to one byte inside the RBTreeNode. That way the GC is never confused - it will keep the node. I think putting that one bit inside the pointer has important consequences. I also suggest you read up on "left-leaning red-black trees" for a recent alternative approach that simplifies the code a fair amount. Andrei
Jul 19 2010
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/19/2010 12:23 PM, Steven Schveighoffer wrote:
 On Mon, 19 Jul 2010 12:21:36 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 By the way, I'm still eagerly waiting for your red-black tree
 implementation.

Sorry for the delay, I've been very busy at work, and I wanted to slip in a couple druntime fixes for array appending. All that is left really is the unit testing, and making the docs more phobos-ish.
 I think it would be pure awesomeness if you massaged the red/black bit
 inside one of the pointers. I figured out a way of doing that without
 throwing off the garbage collector:

Yes, that works (BTW, you don't need the union, I hate unions :), just substitute _bits for _left everywhere, I think it would even work with a moving GC).

Walter told me that union is instrumental to keeping the compiler in the know about such shenanigans. What does your idea look like? You mean keeping a possibly misaligned RBTreeNode pointer and manipulating that? I think that's a bit worse than unions because it transforms a sure thing into a maybe works thing.
 But I don't know how important it is to save that extra 4 bytes/node. A
 redblack node already has 3 pointers in it, the flag puts it to 16 bytes
 instead of overhead instead of 12. It certainly can be an implementation
 choice.

 I can look at left-leaning trees (I've had it on my todo list for
 dcollections too).

Sounds great. If the payload is one word, on a 32-bit system we'd have 20 bytes per node. I seem to recall the current GC can allocate 16 bytes and then 32 bytes and then 48 bytes, so with the embedded bit we're looking at halving the total allocated size. Not too shoddy! Then the relative overhead of that extra word is not felt up until a payload of 20 bytes, at which point again it jumps to 33%. I wonder what things look like (alignment, granularity) for the 64-bit implementation. Andrei
Jul 19 2010
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/19/2010 01:50 PM, Steven Schveighoffer wrote:
 On Mon, 19 Jul 2010 13:47:38 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 On 07/19/2010 12:23 PM, Steven Schveighoffer wrote:
 On Mon, 19 Jul 2010 12:21:36 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 By the way, I'm still eagerly waiting for your red-black tree
 implementation.

Sorry for the delay, I've been very busy at work, and I wanted to slip in a couple druntime fixes for array appending. All that is left really is the unit testing, and making the docs more phobos-ish.
 I think it would be pure awesomeness if you massaged the red/black bit
 inside one of the pointers. I figured out a way of doing that without
 throwing off the garbage collector:

Yes, that works (BTW, you don't need the union, I hate unions :), just substitute _bits for _left everywhere, I think it would even work with a moving GC).

Walter told me that union is instrumental to keeping the compiler in the know about such shenanigans. What does your idea look like? You mean keeping a possibly misaligned RBTreeNode pointer and manipulating that? I think that's a bit worse than unions because it transforms a sure thing into a maybe works thing.

I don't pretend to know what ominous problems Walter knows about regarding the compiler's view, but here is what I'm thinking: If a pointer points to the beginning of a node, and a node has at least one pointer in it (which it must, since it's a tree), then pointing one byte into the node means you're still pointing at the same block, making sure the GC doesn't collect. Really, the generated code will be exactly the same as your solution, but it's less of a misuse of the type system IMO (believe it or not). I'd rather use casts when you are trying to use something that's typed as one thing as something else. When using unions, I usually expect only one member of the union to be valid at any one time. And wouldn't a union be more egregious with the upcoming mostly-precise scanner?

I don't think so (applied to all of the above) for reasons of various degree of obviousness, but perhaps it's not worth expanding on such a minor issue. Andrei
Jul 19 2010
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:

 Not quite :)  There is one byte for padding because (insert gasp-inspiring  
 music accent) all struct heap allocations are allocated through newArray  
 with a size of 1.  I discovered this when working on the array append  
 patch.

How much more hidden shit like this do I have to see? I have filed a bug report: http://d.puremagic.com/issues/show_bug.cgi?id=4487 Maybe Walter has to fix this before porting dmd to 64 bits.
 Most of this is mitigated if you have a custom allocator that allocates an  
 array of nodes at once

The GC is supposed to not suck that much. If I want to do all manually and use custom allocators then maybe it's better if I start to switc to C language. Thank you Steven.
Jul 19 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:
 What's so horrible about it?

Using about 210% of the RAM I was planning to use. And not even saying it in a small print somewhere in the docs.
 It's a corner case.

4-word structs are quite common. It's not a common corner.
 All I'm saying is the pad byte itself isn't a huge issue, even if it  
 didn't exist, there would always be inefficient allocations.

That is too much inefficient.
 If you want to tune your app to be the most  
 efficient at memory allocation, you need to study the allocator and learn  
 its tricks and nuances.

The allocator can have some overhead, but this is offensive.
  I think you have some misguided notion that C's  
 allocator is perfect, and there's no hidden cost to it.  That's not the  
 case.

I have written plenty code in C and its cost is not that high. I have seen you have changed my bug report, the first I have signed as 'major' into an 'enhancement'. And you have said:
 DMD functions exactly as designed.

Then I can answer it's a *design* bug. I feel offended.
Jul 19 2010
parent bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:
 That is the cost of allocation schemes that use fixed size memory blocks.   
 Especially when they grow in powers of 2.  Tune your app for it, and you  
 won't have this problem.

I did know about the power of 2 allocations for small memory blocks, and I know it's useful to reduce memory fragmentation. So I have tuned my code for that, that's why I have several structs 16 bytes long, but now I have to target 15 bytes, that is not a power of 2 :o)
 A design bug *is* an enhancement :)

I see, I didn't know this. Sorry for losing my temper Steven... Bye, bearophile
Jul 19 2010
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/19/2010 03:09 PM, Steven Schveighoffer wrote:
 Take the
 redblack tree node for instance. Get rid of the pad byte, and it's tuned
 for word-size payload. But go over that, and you're back to pretty much
 wasting 50% of your memory.

I think this characterization is a bit inaccurate because it suggests that there are gains only for one-word payloads. I think the truth is that there are gain for several payloads. Their relative value decreases with the size of the payload. Long story short - the less slack (byte of overhead, bool for red/black information), the better (in various quanta and of various values). Andrei
Jul 19 2010
parent bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:
 Isn't everyone always  
 saying around here how cheap memory is these days? ;)

RAM is cheap, but the CPU doesn't used RAM, it mostly uses L1 cache (and a bit L2/L3 caches too), and they cost a lot :-) The more space your data structure uses, the less you can fit in the cache. Today cache effects are important for the code performance. This is a nice example, shows how reducing the size of the data structure and changing its arrangement (the original was a normal tree, transversed for each pixel) can improve the code performance by something like one order of magnitude for ray-tracing: http://www.cs.utah.edu/~bes/papers/fastRT/paper-node12.html Bye, bearophile
Jul 19 2010
prev sibling parent BLS <windevguy hotmail.de> writes:
On 19/07/2010 18:21, Andrei Alexandrescu wrote:
 I also suggest you read up on "left-leaning red-black trees" for a
 recent alternative approach that simplifies the code a fair amount.

A few month ago (12-15) I have also made that suggestion to Steve. Meanwhile I am not that sure that LL RB Trees do offer significant complexity reduction... . R. Sedgewick's original implementation in Java is not bullet proofed. Don't get me wrong LL RBTree have a certain appeal but read your self. --In case that you don't want to use this link : http://t-t-travails.blogspot.com/2008/04/left-leaning-red-black-trees-are-hard.html --Here a quote < Last Monday, I started implementing left-leaning red-black trees, expecting to spend perhaps 15 hours on the project. I'm here more than 60 hours of work later to tell you that left-leaning red-black trees are hard to implement, and contrary to Sedgewick's claims, their implementation appears to require approximately the same amount of code and complexity as standard red-black trees.

alternative data-structure. Besidet QT folks are using the skiplist algo. for their MAP implementation. Andrei, hope you have noticed that Steve's dcollections allow the replacement of the underlaying data-structute. ;) So IMHO let's spend some time in implementing the skiplist data-structure. Finally > I would like to see std.datastructures. for core tree,list,graphs etc structures.. A+. Bjoern
Jul 19 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sat, 17 Jul 2010 16:25:16 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 06/16/2010 05:47 AM, Steven Schveighoffer wrote:
 On Tue, 15 Jun 2010 22:23:15 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 bearophile wrote:
 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?

You need to read TDPL for that :o).
 And what are the purposes of std.contracts.AssumeSorted()? Is it
 useful for something?

AssumeSorted was an experiment. I think it has drawbacks that I don't know how to address, so I'll retire it.

Hm... what are the drawbacks (besides it not being enforced)? I thought it was a good solution.

Sorry I took so long (over one month!) to reply to this. I've delayed the reply to the point when it could be integrated within the upcoming thread about improving std.algorithm.find. The problem with AssumeSorted is that generally predicates in D (and also in most other languages) are not easy to compare. Let's first recall AssumSorted's definition. It's just a wrapper: /** Passes the type system the information that $(D range) is already sorted by predicate $(D pred). No checking is performed; debug builds may insert checks randomly. To insert a check, see $(XREF algorithm, isSorted). */ struct AssumeSorted(Range, alias pred = "a < b") { /// Alias for $(D Range). alias Range AssumeSorted; /// The passed-in range. Range assumeSorted; /// The sorting predicate. alias pred assumeSortedBy; } /// Ditto AssumeSorted!(Range, pred) assumeSorted(alias pred = "a < b", Range) (Range r) { AssumeSorted!(Range, pred) result; result.assumeSorted = r; return result; } The recommended way to use the facility is: int[] a = [ -1, 0, 1, 2, 3, 4, 5 ]; assert(find(assumeSorted(a), 3) == [ 3, 4, 5 ]); find() uses simple means to detect that its first argument has type AssumeSorted and takes advantage of that when searching (specifically by doing binary search). So far so good. The problem ensues when we want to make sure that the sorting predicate is in sync with the search predicate. For example, if the search predicate is "==" then it's okay to use "<" or ">" as a sorting predicate. But searching for "a.zip == b.zip" in a range sorted by "a.name < b.name" is not okay. If predicates were all expressed as strings, probably some string manipulation could be done to see whether they are compatible. But as things stand, assertSorted has quite limited power.

Just thinking out loud here, couldn't you use the predicate already in AssumeSorted? I mean, if you're going to pass AssumeSorted into find, you don't want to also specify the predicate as then the range just becomes a standard range. There must be some kind of way to use template constraints to kill the predicate arg to find when the range is an AssumeSorted struct. If not, there should be. -Steve
Jul 19 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 09:36:54 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 07/19/2010 06:36 AM, Steven Schveighoffer wrote:
 Just thinking out loud here, couldn't you use the predicate already in
 AssumeSorted? I mean, if you're going to pass AssumeSorted into find,
 you don't want to also specify the predicate as then the range just
 becomes a standard range.

 There must be some kind of way to use template constraints to kill the
 predicate arg to find when the range is an AssumeSorted struct. If not,
 there should be.

That's a good idea. The find predicate that could be derived from AssumeSorted's predicate pred would be !pred(a, b) && !pred(b, a). Thanks, Steve.

You're welcome :) BTW, you don't need the combo predicate until the very end. Basically, you do a binary search for the first element where pred(a, E) is false (where E is the target), and then see if pred(E, a) is also false on that element (to test for equality). -Steve
Jul 19 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 11:10:03 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 07/19/2010 09:27 AM, Steven Schveighoffer wrote:
 On Mon, 19 Jul 2010 09:36:54 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 On 07/19/2010 06:36 AM, Steven Schveighoffer wrote:
 Just thinking out loud here, couldn't you use the predicate already in
 AssumeSorted? I mean, if you're going to pass AssumeSorted into find,
 you don't want to also specify the predicate as then the range just
 becomes a standard range.

 There must be some kind of way to use template constraints to kill the
 predicate arg to find when the range is an AssumeSorted struct. If  
 not,
 there should be.

That's a good idea. The find predicate that could be derived from AssumeSorted's predicate pred would be !pred(a, b) && !pred(b, a). Thanks, Steve.

You're welcome :) BTW, you don't need the combo predicate until the very end. Basically, you do a binary search for the first element where pred(a, E) is false (where E is the target), and then see if pred(E, a) is also false on that element (to test for equality).

You mean like this? :o) http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/algorithm.d?rev=1279#L4703

Yep. I realized after I wrote this that you probably were already doing it :) Interestingly, I found that when doing the redblacktree that I tried to do some optimization on the lookup of an element. Basically, while I'm going down the tree looking for a single element, I'm using the binary predicate to move left or right. However, if I move left (i.e. it's not less), then that could be the element I'm looking for! So I try the opposite of the predicate to see if I should return. But when I allow multiple identical elements (i.e. multiset), I want to find the *first* instance of the element, the code is much simpler. If I move to the left child, I store that as the "Best result so far". Then at the end, I simply run the opposite predicate once on the aforementioned best result. The benefit of running the opposite predicate sooner is that if the element is higher in the tree, I'll return quicker, but I think it ends up being a wash. I'll probably change it to be the same as the multi style tree. It all comes from the original code which used an int return for the comparison, making it just as simple to detect equality as it is to detect less-than. Maybe I'm the first one to make that mistake :) -Steve
Jul 19 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 12:21:36 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:
 By the way, I'm still eagerly waiting for your red-black tree  
 implementation.

Sorry for the delay, I've been very busy at work, and I wanted to slip in a couple druntime fixes for array appending. All that is left really is the unit testing, and making the docs more phobos-ish.
 I think it would be pure awesomeness if you massaged the red/black bit  
 inside one of the pointers. I figured out a way of doing that without  
 throwing off the garbage collector:

Yes, that works (BTW, you don't need the union, I hate unions :), just substitute _bits for _left everywhere, I think it would even work with a moving GC). But I don't know how important it is to save that extra 4 bytes/node. A redblack node already has 3 pointers in it, the flag puts it to 16 bytes instead of overhead instead of 12. It certainly can be an implementation choice. I can look at left-leaning trees (I've had it on my todo list for dcollections too). -Steve
Jul 19 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 13:47:38 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 07/19/2010 12:23 PM, Steven Schveighoffer wrote:
 On Mon, 19 Jul 2010 12:21:36 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 By the way, I'm still eagerly waiting for your red-black tree
 implementation.

Sorry for the delay, I've been very busy at work, and I wanted to slip in a couple druntime fixes for array appending. All that is left really is the unit testing, and making the docs more phobos-ish.
 I think it would be pure awesomeness if you massaged the red/black bit
 inside one of the pointers. I figured out a way of doing that without
 throwing off the garbage collector:

Yes, that works (BTW, you don't need the union, I hate unions :), just substitute _bits for _left everywhere, I think it would even work with a moving GC).

Walter told me that union is instrumental to keeping the compiler in the know about such shenanigans. What does your idea look like? You mean keeping a possibly misaligned RBTreeNode pointer and manipulating that? I think that's a bit worse than unions because it transforms a sure thing into a maybe works thing.

I don't pretend to know what ominous problems Walter knows about regarding the compiler's view, but here is what I'm thinking: If a pointer points to the beginning of a node, and a node has at least one pointer in it (which it must, since it's a tree), then pointing one byte into the node means you're still pointing at the same block, making sure the GC doesn't collect. Really, the generated code will be exactly the same as your solution, but it's less of a misuse of the type system IMO (believe it or not). I'd rather use casts when you are trying to use something that's typed as one thing as something else. When using unions, I usually expect only one member of the union to be valid at any one time. And wouldn't a union be more egregious with the upcoming mostly-precise scanner?
 But I don't know how important it is to save that extra 4 bytes/node. A
 redblack node already has 3 pointers in it, the flag puts it to 16 bytes
 instead of overhead instead of 12. It certainly can be an implementation
 choice.

 I can look at left-leaning trees (I've had it on my todo list for
 dcollections too).

Sounds great. If the payload is one word, on a 32-bit system we'd have 20 bytes per node. I seem to recall the current GC can allocate 16 bytes and then 32 bytes and then 48 bytes, so with the embedded bit we're looking at halving the total allocated size. Not too shoddy!

Not quite :) There is one byte for padding because (insert gasp-inspiring music accent) all struct heap allocations are allocated through newArray with a size of 1. I discovered this when working on the array append patch. So even a 16-byte struct requires a 32-byte block.
 Then the relative overhead of that extra word is not felt up until a  
 payload of 20 bytes, at which point again it jumps to 33%.

Most of this is mitigated if you have a custom allocator that allocates an array of nodes at once (what I do in dcollections). As a simple implementation, you could allocate enough nodes to be under a certain threshold of wasted space.
 I wonder what things look like (alignment, granularity) for the 64-bit  
 implementation.

They must be 8-byte aligned, and have 3 8-byte pointers, so that means at least 24 bytes. If you store an int, then it will still fit in the 32-byte block. I don't know what's planned as the minimum size for 64-bit GC. -Steve
Jul 19 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 15:53:46 -0400, bearophile <bearophileHUGS lycos.com>  
wrote:

 Steven Schveighoffer:

 Not quite :)  There is one byte for padding because (insert  
 gasp-inspiring
 music accent) all struct heap allocations are allocated through newArray
 with a size of 1.  I discovered this when working on the array append
 patch.

How much more hidden shit like this do I have to see? I have filed a bug report: http://d.puremagic.com/issues/show_bug.cgi?id=4487 Maybe Walter has to fix this before porting dmd to 64 bits.
 Most of this is mitigated if you have a custom allocator that allocates  
 an
 array of nodes at once

The GC is supposed to not suck that much. If I want to do all manually and use custom allocators then maybe it's better if I start to switc to C language. Thank you Steven.

What's so horrible about it? It's a corner case. If you were allocating a 20-byte struct, you are wasting 12 bytes per value anyways. Or what about a 36-byte struct? All I'm saying is the pad byte itself isn't a huge issue, even if it didn't exist, there would always be inefficient allocations. Take the redblack tree node for instance. Get rid of the pad byte, and it's tuned for word-size payload. But go over that, and you're back to pretty much wasting 50% of your memory. If you want to tune your app to be the most efficient at memory allocation, you need to study the allocator and learn its tricks and nuances. I think you have some misguided notion that C's allocator is perfect, and there's no hidden cost to it. That's not the case. That being said, it would be good if we could get rid of the 1-byte pad for single struct allocations on the heap. As a bonus, the code will be more efficient because it doesn't have to deal with the "array" notion. -Steve
Jul 19 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 16:24:04 -0400, bearophile <bearophileHUGS lycos.com>  
wrote:

 Steven Schveighoffer:
 What's so horrible about it?

Using about 210% of the RAM I was planning to use. And not even saying it in a small print somewhere in the docs.
 It's a corner case.

4-word structs are quite common. It's not a common corner.

You mean it *is* a common corner? I agree, and I think the bug report is warranted, we should get it fixed. But again, you have specifically designed your input to thwart the system. Those examples will always be possible to construct, unless you have a perfect memory allocator, which probably will be so slow that it's unusable.
 All I'm saying is the pad byte itself isn't a huge issue, even if it
 didn't exist, there would always be inefficient allocations.

That is too much inefficient.

That is the cost of allocation schemes that use fixed size memory blocks. Especially when they grow in powers of 2. Tune your app for it, and you won't have this problem.
 If you want to tune your app to be the most
 efficient at memory allocation, you need to study the allocator and  
 learn
 its tricks and nuances.

The allocator can have some overhead, but this is offensive.

I guess my answer is, tune your app to the allocator. If you allocate a lot of little items, you will go a long way to allocate by allocating an array of them instead and use a free list.
  I think you have some misguided notion that C's
 allocator is perfect, and there's no hidden cost to it.  That's not the
 case.

I have written plenty code in C and its cost is not that high.

So C does not use pools of fixed-size memory? All its blocks it hands out are exactly the size you ask for? Hm... let me test... Yep, C does the same thing. I wrote a small program to allocate 1,000,000 blocks of a command-line-supplied size. Up to 12 bytes per block allocates 17MiB, even if the blocks I request are 1 byte. For a single pointer, that's 300% overhead. How horrid, let's lambaste the C allocator developers. WHERE'S THE FINE PRINT!???? 16 bytes per block allocates 24.6MiB. Wait, shouldn't it be 17, surely that holds 16MiB of data? What's going on here? How can anyone be asked to deal with this shit? Oh wait, I didn't *tune my app for the allocator*. Above that, the C allocator does a good job of minimizing overhead, but that's for a plastic example of simply calling malloc 1M times in a row. And C has less to worry about than D when it comes to memory allocation and is far more experienced at it. But it's not perfect in all situations. It's much easier to tune your app to the allocator than tune the allocator to your app.
 I have seen you have changed my bug report, the first I have signed as  
 'major' into an 'enhancement'. And you have said:
 DMD functions exactly as designed.

Then I can answer it's a *design* bug. I feel offended.

A design bug *is* an enhancement :) -Steve
Jul 19 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 16:41:50 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 07/19/2010 03:09 PM, Steven Schveighoffer wrote:
 Take the
 redblack tree node for instance. Get rid of the pad byte, and it's tuned
 for word-size payload. But go over that, and you're back to pretty much
 wasting 50% of your memory.

I think this characterization is a bit inaccurate because it suggests that there are gains only for one-word payloads. I think the truth is that there are gain for several payloads. Their relative value decreases with the size of the payload. Long story short - the less slack (byte of overhead, bool for red/black information), the better (in various quanta and of various values).

There is a cost though... which was my point. Isn't everyone always saying around here how cheap memory is these days? ;) -Steve
Jul 19 2010
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 17:01:34 -0400, bearophile <bearophileHUGS lycos.com>  
wrote:

 Steven Schveighoffer:
 That is the cost of allocation schemes that use fixed size memory  
 blocks.
 Especially when they grow in powers of 2.  Tune your app for it, and you
 won't have this problem.

I did know about the power of 2 allocations for small memory blocks, and I know it's useful to reduce memory fragmentation. So I have tuned my code for that, that's why I have several structs 16 bytes long, but now I have to target 15 bytes, that is not a power of 2 :o)

Hm... unfortunately, I think you will end up in the same boat. Because any struct of size 15 is aligned to be on a 16-byte boundary. From my memory, I don't think the array allocation code takes into account if the final element coincides with the pad byte, but I may be wrong. Make sure to test this theory before going through and trying to trim bytes off all your structs. I think if you use 12-byte structs, it will fit fine, but then of course, you are wasting 25% memory :) If you can deal with some manual memory management, you may want to pre-allocate a large array of the structs and then use a free list to "allocate" and "deallocate" them. This should pack them in as tightly as possible with almost no overhead. Of course, if you depend on the GC to free your elements, then it might be more of a burden to change all your code to contain manual memory management. -Steve
Jul 19 2010
prev sibling next sibling parent reply Lutger <lutger.blijdestijn gmail.com> writes:
bearophile wrote:

 I have counted about 200 usages of std.contracts.enforce() inside Phobos.
 Can you tell me what's the purpose of enforce() in a language that has
 built-in Contract Programming?

I'd think of it this way: enforce() is part of defensive programming, and contracts are related to software testing.
 And what are the purposes of std.contracts.AssumeSorted()? Is it useful
 for something?
 
 Bye,
 bearophile
 (I know this is not the digitalmars.D.learn newsgroup).

Jun 15 2010
next sibling parent reply Jonathan M Davis <jmdavisProg gmail.com> writes:
Lutger wrote:

 bearophile wrote:
 
 I have counted about 200 usages of std.contracts.enforce() inside Phobos.
 Can you tell me what's the purpose of enforce() in a language that has
 built-in Contract Programming?

I'd think of it this way: enforce() is part of defensive programming, and contracts are related to software testing.

That's probably a pretty good way of putting it. It's essentially the difference between when you use assertions and when you use exceptions. Assertions assert that something is _always_ true and that if it isn't, the program is wrong, while exceptions are for exceptional circumstances (as opposed to _never_) and indicate an error of some kind which is likely outside the control of the program - such as something happening with the file system, user input, or the amount of available memory. enforce() appears to effectively be the exception equivalent to assert(). You use it when you want an exception thrown rather than when you want to kill your program due to an error. Unfortunately, the difference between when assertions should be used and when exceptions should be used is one that is just subtle enough that it often trips people up, even though in theory it should be fairly straightforward. - Jonathan M Davis
Jun 16 2010
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
Jonathan M Davis wrote:

 Assertions assert that something is _always_ true and that if it 

 program is wrong, while exceptions are for exceptional circumstances

Makes sense.
 You use [enforce] when you want an exception thrown rather than when 

 kill your program due to an error.

To further confuse the issue, assert throws too: import std.stdio; import std.algorithm; void main() { try { assert(false); } catch (Throwable) { writeln("an assertion failed"); } } The difference is just the exception that is thrown. Throwable seems to be most general. From what I've read so far, I take enforce as a replacement to what it exactly is: if (condition) { throw /* ... */; } Since I never use assert for that purpose, I take enforce as a shortcut for the above. Ali
Jun 16 2010
parent Jonathan M Davis <jmdavisProg gmail.com> writes:
Ali Çehreli wrote:

 Jonathan M Davis wrote:
 
  > Assertions assert that something is _always_ true and that if it
 isn't, the
  > program is wrong, while exceptions are for exceptional circumstances
 
 Makes sense.
 
  > You use [enforce] when you want an exception thrown rather than when
 you want to
  > kill your program due to an error.
 
 To further confuse the issue, assert throws too:
 
 import std.stdio;
 import std.algorithm;
 
 void main()
 {
      try {
          assert(false);
      } catch (Throwable) {
          writeln("an assertion failed");
      }
 }
 
 The difference is just the exception that is thrown. Throwable seems to
 be most general.
 
  From what I've read so far, I take enforce as a replacement to what it
 exactly is:
 
 if (condition) {
      throw /* ... */;
 }
 
 Since I never use assert for that purpose, I take enforce as a shortcut
 for the above.
 
 Ali

Well, in a sense, the fact that assertions throw is an implementation detail since that's not the case in all languages. The concepts of assertions and exceptions are distinctly different. However, while assertions do throw in D, they throw AssertErrors which are Errors and not exceptions, albeit both are Throwable. So, they're still different. You _can_ catch Errors, but you probably shouldn't. I believe that they're intended for pretty much unrecoverable errors. The fact that they're thrown likely makes it easier to exit the program semi-gracefully - or at least makes it easier for the generated program to properly indicate an error rather than simply dying - but they're still distinctly separate from exceptions and shouldn't generally be caught. I suppose that it's kind of like the difference between checked and unchecked exceptions in Java. You generally only worry about the checked ones. You are right though in that the fact that Errors are Throwable does muddle things somewhat. - Jonathan M Davis
Jun 16 2010
prev sibling parent "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Ali =C3=87ehreli <acehreli yahoo.com> wrote:

 To further confuse the issue, assert throws too:

 import std.stdio;
 import std.algorithm;

 void main()
 {
      try {
          assert(false);
      } catch (Throwable) {
          writeln("an assertion failed");
      }
 }

 The difference is just the exception that is thrown. Throwable seems t=

 be most general.

Seeing as how Error is supposed to be unrecoverable, and Exception might= be recoverable, and both inherit from Throwable, one should only very ve= ry rarely catch Exception, and by extension, Throwable. One might in fact argue that Error and Exception should have no common ancestor but Object= .
  From what I've read so far, I take enforce as a replacement to what i=

 exactly is:

 if (condition) {
      throw /* ... */;
 }

That is indeed basically what it is. -- = Simen
Jun 16 2010
prev sibling next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
bearophile wrote:
 I have counted about 200 usages of std.contracts.enforce() inside Phobos. Can
you tell me what's the purpose of enforce() in a language that has built-in
Contract Programming?

I can see two benefits: 1) enforce throws object.Exception, which is "the root of the exception hierarchy"; hence enforce errors can be caught with the same general catch(Exception) clause [*]. On the other hand, assert throws a type that is out of the Exception hierarchy: core.exception.AssertError 2) As a bonus, the word 'enforce' fits the purpose better than 'assert' 3) (the other 2 :p) The format of the message of the uncaught exceptions is a little better (e.g. no sign before the file name) Ali * Note: Actually, Throwable is at the top of the exception hierarchy, but I've heard before that the top exception class should be taken to be Exception; perhaps for user applications?
Jun 16 2010
next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Ali Çehreli wrote:
 bearophile wrote:
 I have counted about 200 usages of std.contracts.enforce() inside 
 Phobos. Can you tell me what's the purpose of enforce() in a language 
 that has built-in Contract Programming?

I can see two benefits:

The difference is not based on those 3 points, but on what Andrei wrote here. Contracts and error checking are completely distinct activities and should not be conflated.
Jun 16 2010
next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 06/16/2010 04:15 PM, Walter Bright wrote:
 Ali Çehreli wrote:
 bearophile wrote:
 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?

I can see two benefits:

The difference is not based on those 3 points, but on what Andrei wrote here. Contracts and error checking are completely distinct activities and should not be conflated.

Could you please explain them? There are many people here that don't understand the difference between these two concepts (including me). So maybe we are too dumb, maybe those concepts are not generally known or maybe the explanation is not very well clear in the documentation.
Jun 16 2010
next sibling parent reply Alex Makhotin <alex bitprox.com> writes:
Steven Schveighoffer wrote:
  
 Think of enforce as "throw if"
 

So why not concatenating the two and rename it to exactly 'throwif'? Self descriptive is better than cryptic 'enforce'. -- Alex Makhotin, the founder of BITPROX, http://bitprox.com
Jun 16 2010
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Alex Makhotin wrote:
 Steven Schveighoffer wrote:
  
 Think of enforce as "throw if"

So why not concatenating the two and rename it to exactly 'throwif'? Self descriptive is better than cryptic 'enforce'.

Well throwif describes mechanism and enforce describes intent. After all assert is not abortif :o). Andrei
Jun 16 2010
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 16/06/2010 15:45, Andrei Alexandrescu wrote:
 Well throwif describes mechanism and enforce describes intent. After all
 assert is not abortif :o).

 Andrei

Indeed, especially given that other code program may use throwing as mechanism, but with a different intent. And you would not be able to distinguish it if you called it throwif. -- Bruno Medeiros - Software Engineer
Jun 17 2010
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Leandro Lucarella wrote:
 Steven Schveighoffer, el 16 de junio a las 06:55 me escribiste:
 On Wed, 16 Jun 2010 05:28:46 -0400, Ary Borenszweig
 <ary esperanto.org.ar> wrote:

 On 06/16/2010 04:15 PM, Walter Bright wrote:
 Ali Çehreli wrote:
 bearophile wrote:
 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?


here. Contracts and error checking are completely distinct activities and should not be conflated.

don't understand the difference between these two concepts (including me). So maybe we are too dumb, maybe those concepts are not generally known or maybe the explanation is not very well clear in the documentation.

expectation to an exception in a single expression. For example, take some system call that returns -1 on error, you could do this: if(result < 0) throw new Exception("oops!"); or you could do this: enforce(result >= 0, "oops!"); Think of enforce as "throw if"

So maybe throw_if() would be a better name =) Anyway, I think enforce() is poisson,

Indeed it is a bit fishy :o).
 because it make the programmer to
 not think about errors at all, just add and enforce() and there you go.
 But when you need to be fault tolerant, is very important to know what's
 the nature of the error, but thanks to enforce(), almost every error is
 a plain Exception, no hierarchy, no extra info, all you can do to get
 a little more info about what happened is to parse the exception string,
 and that's not really an option.

I think there is no real need for exception hierarchies. I occasionally dream of eliminating all of the useless exceptions defined left and right in Phobos.
 And in fact, I think there's an errnoEnforce which throws a standard
 exception with the string error from the system.

That's the only useful case of enforce, because it includes the *important* information (the actual errno). There is also enforceEx!(), to use a custom exception, which practically nobody uses (I counted only 4 uses in phobos).

I'd be hard pressed to find good examples of exception hierarchy use. Everybody talks about them but I've seen none. The fact that the coder doesn't need to think hard to use enforce() effectively is a plus, not a minus. An overdesigned enforce that adds extra burden to its user would have been a mistake. Andrei
Jun 16 2010
next sibling parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 Everybody talks about them but I've seen none.
 The fact that the coder doesn't need to think hard to use enforce()
 effectively is a plus, not a minus. An overdesigned enforce that adds
 extra burden to its user would have been a mistake.
 Andrei

IMHO the presence of a simple method of handling errors, even if it's far from perfect, is a good thing. If you have to think about a whole exception hierarchy every time you hit a possible error condition in your code, you tend to put this tedious task off until forever, leading to programs that fail for unknown reasons because some error condition was never reported. Well-designed exception hierarchies are nice, but forcing their use all the time would be making the perfect the enemy of the good. Furthermore, I love enforce() because sometimes I want just some subset of assertions checked in release mode, usually whichever ones can be checked at negligible performance cost. I tend to use it a lot as an assert-even-in-release-mode function.
Jun 16 2010
prev sibling next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-06-16 10:53:12 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:

 Leandro Lucarella wrote:
 Steven Schveighoffer, el 16 de junio a las 06:55 me escribiste:
 On Wed, 16 Jun 2010 05:28:46 -0400, Ary Borenszweig
 <ary esperanto.org.ar> wrote:
 
 On 06/16/2010 04:15 PM, Walter Bright wrote:
 Ali ehreli wrote:
 bearophile wrote:
 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?


here. Contracts and error checking are completely distinct activities and should not be conflated.

don't understand the difference between these two concepts (including me). So maybe we are too dumb, maybe those concepts are not generally known or maybe the explanation is not very well clear in the documentation.

expectation to an exception in a single expression. For example, take some system call that returns -1 on error, you could do this: if(result < 0) throw new Exception("oops!"); or you could do this: enforce(result >= 0, "oops!"); Think of enforce as "throw if"

So maybe throw_if() would be a better name =) Anyway, I think enforce() is poisson,

Indeed it is a bit fishy :o).
 because it make the programmer to
 not think about errors at all, just add and enforce() and there you go.
 But when you need to be fault tolerant, is very important to know what's
 the nature of the error, but thanks to enforce(), almost every error is
 a plain Exception, no hierarchy, no extra info, all you can do to get
 a little more info about what happened is to parse the exception string,
 and that's not really an option.

I think there is no real need for exception hierarchies. I occasionally dream of eliminating all of the useless exceptions defined left and right in Phobos.

The need is not really for a hierarchy. The hierarchy serves the need, which is to: 1. Be able to programatically check the kind of the error and so your program can act appropriately. 2. Propagate additional information related to the error and the context it occured. Displaying a proper error message to a user and offering relevant recovery choices often need both. Sometime, a program won't ask the user and attempt something by itself as a recovery. In both cases, you need to know the kind of error, and may need context information.
 And in fact, I think there's an errnoEnforce which throws a standard
 exception with the string error from the system.

That's the only useful case of enforce, because it includes the *important* information (the actual errno). There is also enforceEx!(), to use a custom exception, which practically nobody uses (I counted only 4 uses in phobos).

I'd be hard pressed to find good examples of exception hierarchy use. Everybody talks about them but I've seen none.

The need is not really for a hierarchy. The hierarchy serves the need, which is to: 1. Programatically check the kind of the error and so your program can act appropriately. 2. Propagate additional information related to the error and the context in which it occurred. Displaying a proper error message to a user and offering relevant recovery choices often need both. Sometime, a program won't ask the user and attempt something by itself as a recovery. In both cases, you need to know the kind of error, and may need context information. That said, hierarchies are often abused, and aren't universally useful. But exceptions should provide the above information in a way or another when useful. Think about a GUI program, if an exception is thrown somewhere during a complex operation (say, reading a lot of files), I could catch it as some level, create a wrapper exception with the context (file=hello.d error=access denied) and rethrow it to unwind until reatching the GUI error handler. Or the file function could throw a useful exception from the start. In either cases, the code in charge of that operation can display a message such as "Creating the archive failed. File 'hello.d' could not be read because you do not have read permissions to it." with options "Retry as Administrator", "Exclude 'hello.d'" or "Cancel". Knowing programatically what has gone wrong is important in many cases.
 The fact that the coder doesn't need to think hard to use enforce() 
 effectively is a plus, not a minus. An overdesigned enforce that adds 
 extra burden to its user would have been a mistake.

That's indeed true. Throwing an Exception with no info is still better than not throwing at all, and creating useful exceptions isn't always easy, nor economically rewarding. What's important is to make it easy to improve the thrown exception when it becomes relevant. For instance // first version: throws standard exception enforce(1 == 1, "access denied to " ~ filename); // refined version: throws custom exception enforce(1 == 1, new FileException("access denied to " ~ filename, accessDeniedError, filename)); -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Jun 16 2010
prev sibling parent reply Jonathan M Davis <jmdavisProg gmail.com> writes:
Andrei Alexandrescu wrote:

 Anyway, I think enforce() is poisson,

Indeed it is a bit fishy :o).

LOL. My thoughts exactly.
 
 because it make the programmer to
 not think about errors at all, just add and enforce() and there you go.
 But when you need to be fault tolerant, is very important to know what's
 the nature of the error, but thanks to enforce(), almost every error is
 a plain Exception, no hierarchy, no extra info, all you can do to get
 a little more info about what happened is to parse the exception string,
 and that's not really an option.

I think there is no real need for exception hierarchies. I occasionally dream of eliminating all of the useless exceptions defined left and right in Phobos.
 And in fact, I think there's an errnoEnforce which throws a standard
 exception with the string error from the system.

That's the only useful case of enforce, because it includes the *important* information (the actual errno). There is also enforceEx!(), to use a custom exception, which practically nobody uses (I counted only 4 uses in phobos).

I'd be hard pressed to find good examples of exception hierarchy use. Everybody talks about them but I've seen none. The fact that the coder doesn't need to think hard to use enforce() effectively is a plus, not a minus. An overdesigned enforce that adds extra burden to its user would have been a mistake. Andrei

I think that exception hierarchies can be quite useful, but in most cases, I haven't seen projects bother with them. I do think that certain types of exceptions can be useful as separate types as long as they inherit from the base exception type and you therefore don't _have_ to worry about the hierarchy. A good example of a useful exception type IMO is Java's IOException. It makes good sense to handle them in a specific way separate from general exceptions. You can frequently recover just fine from them, and it allows you to handle I/O-related exceptions gracefully while other exceptions might be considered fatal. However, those other exceptions - especially those which are from more or less unrecoverable errors NullPointerExceptions or OutOfMemoryExceptions - don't necessarily gain much from an exception hierarchy. So, I think that it really depends on the exception. I do think, however, that there are certain types of exceptions which can benefit from having their own type because it allows you to handle them in a specific manner separate from general and/or unrecoverable exceptions. - Jonathan M Davis
Jun 16 2010
parent Jason Spencer <spencer8 sbcglobal.net> writes:
I think about it roughly this way (in reverse priority):

Contracts/assertions concern problems in the program(ming) domain.

Exceptions concern problems in the system domain.

Problems in the actual problem domain should be modeled in the design
and have their own abstractions.

These interact a little bit, so I have an excuse to bend my rules
whenever I want :)  For instance, if the system is part of your
problem domain (e.g. embedded code), then exceptions are probably not
the right approach.  That's why I indicate a false idea of priority.

Jason
Jun 16 2010
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Ary Borenszweig wrote:
 On 06/16/2010 04:15 PM, Walter Bright wrote:
 The difference is not based on those 3 points, but on what Andrei wrote
 here. Contracts and error checking are completely distinct activities
 and should not be conflated.

Could you please explain them? There are many people here that don't understand the difference between these two concepts (including me). So maybe we are too dumb, maybe those concepts are not generally known or maybe the explanation is not very well clear in the documentation.

It has nothing to do with being dumb, as it is not obvious. Contracts are for verifying that your program is in a state that it is designed to be in. A contract failure is defined as a program bug. Errors, on the other hand, are things that can go wrong at run time, like your disk is full when trying to write a file. These are NOT program bugs. Another way to look at it is your program should continue to operate correctly if all the contracts are removed. This is not true of removing all error checking and handling. Furthermore, errors are something a program can recover from and continue operating. Contract failures are ALWAYS fatal. A common newbie (and some expert) misconception is that contract failures can or even must be recovered. This comes from a misunderstanding of the basic principles of engineering a safe and reliable system.
Jun 16 2010
next sibling parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Walter Bright wrote:

 Ary Borenszweig wrote:
 On 06/16/2010 04:15 PM, Walter Bright wrote:
 The difference is not based on those 3 points, but on what Andrei wrote
 here. Contracts and error checking are completely distinct activities
 and should not be conflated.

Could you please explain them? There are many people here that don't understand the difference between these two concepts (including me). So maybe we are too dumb, maybe those concepts are not generally known or maybe the explanation is not very well clear in the documentation.

It has nothing to do with being dumb, as it is not obvious. Contracts are for verifying that your program is in a state that it is designed to be in. A contract failure is defined as a program bug. Errors, on the other hand, are things that can go wrong at run time, like your disk is full when trying to write a file. These are NOT program bugs. Another way to look at it is your program should continue to operate correctly if all the contracts are removed. This is not true of removing all error checking and handling. Furthermore, errors are something a program can recover from and continue operating. Contract failures are ALWAYS fatal. A common newbie (and some expert) misconception is that contract failures can or even must be recovered. This comes from a misunderstanding of the basic principles of engineering a safe and reliable system.

I am not so sure about this last point, usually you want to fail but perhaps not always. This is about what to do after detection of a program bug vs how to handle an exceptional condition. Suppose for example (actually this is from real life) there is an important operation which, as a service, also sends an e-mail notification as part of that operation. It is very bad if the operation fails, but a failed notification is not that bad. What to do in case of a bug with the e-mail notification? 1. crash (gracefully), do not complete the operation. 2. log the error for the devs to look into (or crash) *after* the operation is complete, let the operation go through without the e-mail notification. Option 1 is annoying and prevents people from getting work done due to a 'minor' bug. Option 2 however probably results in this bug either not getting noticed quite early enough or ignored in the face of other issues that always seems to have higher priority. Choosing for option 2 can also lead to bugs being swallowed silently or mistaken for exceptional conditions, which is more dangerous. I don't mean to split hairs, I bet a lot of software has these kind of cases.
Jun 16 2010
next sibling parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Simen kjaeraas wrote:

 Lutger <lutger.blijdestijn gmail.com> wrote:
 
 Suppose for example (actually this is from real life) there is an
 important
 operation which, as a service, also sends an e-mail notification as part
 of that
 operation. It is very bad if the operation fails, but a failed
 notification is
 not that bad. What to do in case of a bug with the e-mail notification?

 1. crash (gracefully), do not complete the operation.
 2. log the error for the devs to look into (or crash) *after* the
 operation is
 complete, let the operation go through without the e-mail notification.

 Option 1 is annoying and prevents people from getting work done due to a
 'minor'
 bug. Option 2 however probably results in this bug either not getting
 noticed
 quite early enough or ignored in the face of other issues that always
 seems to
 have higher priority. Choosing for option 2 can also lead to bugs being
 swallowed silently or mistaken for exceptional conditions, which is more
 dangerous.

 I don't mean to split hairs, I bet a lot of software has these kind of
 cases.

How did you end up with an email system that is so horribly broken that it spits Errors instead of Exceptions when things are not quite the way it wants them to be?

Not Errors, it is not in D and does not distinguish between Errors and Exceptions. It was an example, a (design) question. It's very simple: sendEmail() // possibly die here because something relatively unimportant thing is buggy vs: try { sendEmail() } catch(BadShitThatCanHappen) { RecoverFromBadShitThatCanHappen() // ok, this is good and according to spec } catch(Exception ex) { logError() // now crash? assume we know this must be programmer's fault }
 If it cannot send the email, it may throw an Exception. If you try and
 pass it a handwritten letter, it should throw an Error.
 
 Basically, throwing an Exception means 'Your attention please, reactor 5
 has a cooling problem you might want to look at', whereas an Error means
 'Explosion imminent, get the fuck off outta here!'.
 

No, an Error means the program has a bug. Programs have thousands of bugs, this is not related to how critical it is. An Exception can be way more important to fix than a bug. WebServerDownException for example, is often not a bug in the code that drives websites, but for sure I will contact the sysadmin before even thinking of going back to work. The question is how to proceed after the fact.
Jun 16 2010
parent Walter Bright <newshound1 digitalmars.com> writes:
Simen kjaeraas wrote:
 That said, a bug in a rarely-used function may indeed be significantly
 less important than getting a server back online. However, I would still
 say an error indicates something is fundamentally wrong.

The contract failing means you do not know what went wrong. That means there's no way the program can determine if it is recoverable or not. For all you know, malware may have infected your process and continuing to execute may send your credit card database to a thief.
Jun 17 2010
prev sibling next sibling parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Simen kjaeraas wrote:

...
 If it cannot send the email, it may throw an Exception. If you try and
 pass it a handwritten letter, it should throw an Error.
 

This is the question: should I segfault on a handwritten letter even if it is not such an important letter and could just go on operating?
Jun 16 2010
parent =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Simen kjaeraas wrote:
 Lutger <lutger.blijdestijn gmail.com> wrote:
=20
 Simen kjaeraas wrote:

 ...
 If it cannot send the email, it may throw an Exception. If you try an=



 pass it a handwritten letter, it should throw an Error.

This is the question: should I segfault on a handwritten letter even if it is not such an important letter and could just go on operating?

Yes. If someone is passing your email system a handwritten letter, something is so wrong, the program should balk and exit. It's not just a small mixup, it's an indication something is completely wrong. =20

should signal the mistake and recover. External input must *always* be checked and wrong inputs must be recovered from gracefully. However, if you take (and check) the user input, then put it in a queue, then take things from the queue for processing, and you get a handwritten letter out of the queue, *then* it is an error and cannot be recovered from (because this should have been checked for before putting the letter in the queue and if this is messed up, you don't know what else may be messed up nor how bad the situation is). Which is what you say after:
 As Walter put it, an Error, be it an AssertError or otherwise, means
 your program has ventured into uncharted territory, and behavior from
 this point on is undefined. "Permissible undefined behavior ranges
 from ignoring the situation completely with unpredictable results, to
 having demons fly out of your nose."[1]
=20

Jerome --=20 mailto:jeberger free.fr http://jeberger.free.fr Jabber: jeberger jabber.fr
Jun 17 2010
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Lutger wrote:
 Walter Bright wrote:
 Furthermore, errors are something a program can recover from and continue
 operating. Contract failures are ALWAYS fatal. A common newbie (and some
 expert) misconception is that contract failures can or even must be recovered.
 This comes from a misunderstanding of the basic principles of engineering a
 safe and reliable system.

I am not so sure about this last point, usually you want to fail but perhaps not always. This is about what to do after detection of a program bug vs how to handle an exceptional condition.

First you need to decide if it is a program bug or not. If it is not a program bug, it shouldn't be done with contracts. If it is a program bug, then the only proper thing to do is exit the program. The program cannot decide if it is a minor bug or not, nor can it decide if it is recoverable. It is, by definition, in an unknown state, and continuing to execute may cause anything to happen. (For example, malware may have installed itself and that may get executed.) If you need notifications that the program failed, a separate monitor program should be used. This is how people who design safe systems do it. People who believe that programs can "recover" from bugs design systems that fail, sometimes with terrible consequences. My articles on the topic: http://www.drdobbs.com/blog/archives/2009/10/safe_systems_fr.html http://www.drdobbs.com/blog/archives/2009/11/designing_safe.html
Jun 16 2010
next sibling parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Walter Bright wrote:

 Lutger wrote:
 Walter Bright wrote:
 Furthermore, errors are something a program can recover from and continue
 operating. Contract failures are ALWAYS fatal. A common newbie (and some
 expert) misconception is that contract failures can or even must be
 recovered. This comes from a misunderstanding of the basic principles of
 engineering a safe and reliable system.

I am not so sure about this last point, usually you want to fail but perhaps not always. This is about what to do after detection of a program bug vs how to handle an exceptional condition.

First you need to decide if it is a program bug or not. If it is not a program bug, it shouldn't be done with contracts. If it is a program bug, then the only proper thing to do is exit the program. The program cannot decide if it is a minor bug or not, nor can it decide if it is recoverable. It is, by definition, in an unknown state, and continuing to execute may cause anything to happen. (For example, malware may have installed itself and that may get executed.)

I didn't really get this point from your articles on the subject, but that does clarify it for me. The assumption one makes when recovering is that it is indeed possible and safe. Even if it may be likely, it is never reliable to count on it.
Jun 16 2010
parent Walter Bright <newshound1 digitalmars.com> writes:
Lutger wrote:
 The assumption one makes when recovering is that it is indeed 
 possible and safe. Even if it may be likely, it is never reliable to count on 
 it.

Exactly.
Jun 16 2010
prev sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 17/06/2010 00:27, Walter Bright wrote:
 Lutger wrote:
 Walter Bright wrote:
 Furthermore, errors are something a program can recover from and
 continue
 operating. Contract failures are ALWAYS fatal. A common newbie (and some
 expert) misconception is that contract failures can or even must be
 recovered.
 This comes from a misunderstanding of the basic principles of
 engineering a
 safe and reliable system.

I am not so sure about this last point, usually you want to fail but perhaps not always. This is about what to do after detection of a program bug vs how to handle an exceptional condition.

First you need to decide if it is a program bug or not. If it is not a program bug, it shouldn't be done with contracts.

I would go further and state that anything outside the direct control of a process (such as network state, disk state, OS state, other processes behavior, user interaction, etc.) should be modeled as an error and not a contract violation. Such externals errors may be a "bug" in the system as whole, but they are not a bug in the particular process, and thus should not be modeled as a contract violation. In other words, _contract violations should always be situations that you can prevent by changing the code of the underlying process_. You can't do that for network errors, disk state, etc.. But you can do that for stuff like ensuring a variable is never null, an object in your program is in some particular state at a particular point in execution, etc. -- Bruno Medeiros - Software Engineer
Jun 17 2010
next sibling parent Walter Bright <newshound1 digitalmars.com> writes:
Bruno Medeiros wrote:
 I would go further and state that anything outside the direct control of 
 a process (such as network state, disk state, OS state, other processes 
 behavior, user interaction, etc.) should be modeled as an error and not 
 a contract violation.
 Such externals errors may be a "bug" in the system as whole, but they 
 are not a bug in the particular process, and thus should not be modeled 
 as a contract violation.
 In other words, _contract violations should always be situations that 
 you can prevent by changing the code of the underlying process_. You 
 can't do that for network errors, disk state, etc.. But you can do that 
 for stuff like ensuring a variable is never null, an object in your 
 program is in some particular state at a particular point in execution, 
 etc.

That's a reasonable way of looking at it.
Jun 17 2010
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
Bruno Medeiros <brunodomedeiros+spam com.gmail> wrote:
 On 17/06/2010 00:27, Walter Bright wrote:
 Lutger wrote:
 Walter Bright wrote:
 Furthermore, errors are something a program can recover from and
 continue
 operating. Contract failures are ALWAYS fatal. A common newbie (and
 some
 expert) misconception is that contract failures can or even must be
 recovered.
 This comes from a misunderstanding of the basic principles of
 engineering a
 safe and reliable system.

I am not so sure about this last point, usually you want to fail but perhaps not always. This is about what to do after detection of a program bug vs how to handle an exceptional condition.

First you need to decide if it is a program bug or not. If it is not a program bug, it shouldn't be done with contracts.

I would go further and state that anything outside the direct control of a process (such as network state, disk state, OS state, other processes behavior, user interaction, etc.) should be modeled as an error and not a contract violation. Such externals errors may be a "bug" in the system as whole, but they are not a bug in the particular process, and thus should not be modeled as a contract violation. In other words, _contract violations should always be situations that you can prevent by changing the code of the underlying process_. You can't do that for network errors, disk state, etc.. But you can do that for stuff like ensuring a variable is never null, an object in your program is in some particular state at a particular point in execution, etc.

Right. I'd say contracts are to catch logic errors.
Jun 18 2010
prev sibling parent Ary Borenszweig <ary esperanto.org.ar> writes:
On 06/16/2010 11:44 PM, Walter Bright wrote:
 Ary Borenszweig wrote:
 On 06/16/2010 04:15 PM, Walter Bright wrote:
 The difference is not based on those 3 points, but on what Andrei wrote
 here. Contracts and error checking are completely distinct activities
 and should not be conflated.

Could you please explain them? There are many people here that don't understand the difference between these two concepts (including me). So maybe we are too dumb, maybe those concepts are not generally known or maybe the explanation is not very well clear in the documentation.

It has nothing to do with being dumb, as it is not obvious. Contracts are for verifying that your program is in a state that it is designed to be in. A contract failure is defined as a program bug. Errors, on the other hand, are things that can go wrong at run time, like your disk is full when trying to write a file. These are NOT program bugs. Another way to look at it is your program should continue to operate correctly if all the contracts are removed. This is not true of removing all error checking and handling. Furthermore, errors are something a program can recover from and continue operating. Contract failures are ALWAYS fatal. A common newbie (and some expert) misconception is that contract failures can or even must be recovered. This comes from a misunderstanding of the basic principles of engineering a safe and reliable system.

Ah, ok, now I understand. Thanks.
Jun 16 2010
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-06-16 05:15:24 -0400, Walter Bright <newshound1 digitalmars.com> said:

 The difference is not based on those 3 points, but on what Andrei wrote 
 here. Contracts and error checking are completely distinct activities 
 and should not be conflated.

True. Yet, enforce is inside std.contracts. If that isn't conflating the two concepts I wonder what it is. :-) -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Jun 16 2010
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Michel Fortin wrote:
 On 2010-06-16 05:15:24 -0400, Walter Bright <newshound1 digitalmars.com> 
 said:
 
 The difference is not based on those 3 points, but on what Andrei 
 wrote here. Contracts and error checking are completely distinct 
 activities and should not be conflated.

True. Yet, enforce is inside std.contracts. If that isn't conflating the two concepts I wonder what it is. :-)

You're right! I think Lars' suggestion is sensible - we should move enforce to object. Better yet we should find a better name for std.contracts. Ideas? Andrei
Jun 16 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 06/17/2010 04:10 AM, Lars T. Kyllingstad wrote:
 On Wed, 16 Jun 2010 07:31:39 -0700, Andrei Alexandrescu wrote:

 Michel Fortin wrote:
 On 2010-06-16 05:15:24 -0400, Walter Bright
 <newshound1 digitalmars.com>  said:

 The difference is not based on those 3 points, but on what Andrei
 wrote here. Contracts and error checking are completely distinct
 activities and should not be conflated.

True. Yet, enforce is inside std.contracts. If that isn't conflating the two concepts I wonder what it is. :-)

You're right! I think Lars' suggestion is sensible - we should move enforce to object. Better yet we should find a better name for std.contracts. Ideas? Andrei

A few suggestions (even though I still think it belongs in object.d), in no particular order: std.enforce std.assumptions std.constraints std.checks std.tests std.error std.errcheck -Lars

We haven't reached consensus on where to put enforce() and friends. Any other ideas? Of the above, I like std.checks. Better yet, how about defining std.exception that includes a host of exception-related functionality (such as defining exceptions that retain file and line, perhaps stack traces etc.)? Andrei
Jun 27 2010
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Lars T. Kyllingstad wrote:
 On Sun, 27 Jun 2010 18:09:02 -0500, Andrei Alexandrescu wrote:
 
 On 06/17/2010 04:10 AM, Lars T. Kyllingstad wrote:
 On Wed, 16 Jun 2010 07:31:39 -0700, Andrei Alexandrescu wrote:

 Michel Fortin wrote:
 On 2010-06-16 05:15:24 -0400, Walter Bright
 <newshound1 digitalmars.com>  said:

 The difference is not based on those 3 points, but on what Andrei
 wrote here. Contracts and error checking are completely distinct
 activities and should not be conflated.

Yet, enforce is inside std.contracts. If that isn't conflating the two concepts I wonder what it is. :-)

enforce to object. Better yet we should find a better name for std.contracts. Ideas? Andrei

A few suggestions (even though I still think it belongs in object.d), in no particular order: std.enforce std.assumptions std.constraints std.checks std.tests std.error std.errcheck -Lars

other ideas? Of the above, I like std.checks. Better yet, how about defining std.exception that includes a host of exception-related functionality (such as defining exceptions that retain file and line, perhaps stack traces etc.)?

TDPL mentions several times that enforce() is in std.contracts. Doesn't that preclude moving it or renaming the module?

I plan to move it to std.exception in a backward-compatible way (have std.conv consist of only one import, then deprecate it). Andrei
Jun 28 2010
prev sibling parent torhu <no spam.invalid> writes:
On 28.06.2010 01:09, Andrei Alexandrescu wrote:
[...]
 We haven't reached consensus on where to put enforce() and friends. Any
 other ideas? Of the above, I like std.checks.

 Better yet, how about defining std.exception that includes a host of
 exception-related functionality (such as defining exceptions that retain
 file and line, perhaps stack traces etc.)?

How will std.exception relate to core.exception? Seems to me having two module with that similiar names could easily be confusing.
Jul 05 2010
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Michel Fortin wrote:
 On 2010-06-16 05:15:24 -0400, Walter Bright <newshound1 digitalmars.com> 
 said:
 
 The difference is not based on those 3 points, but on what Andrei 
 wrote here. Contracts and error checking are completely distinct 
 activities and should not be conflated.

True. Yet, enforce is inside std.contracts. If that isn't conflating the two concepts I wonder what it is. :-)

I agree completely. enforce must move.
Jun 16 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Walter Bright wrote:
 Michel Fortin wrote:
 On 2010-06-16 05:15:24 -0400, Walter Bright 
 <newshound1 digitalmars.com> said:

 The difference is not based on those 3 points, but on what Andrei 
 wrote here. Contracts and error checking are completely distinct 
 activities and should not be conflated.

True. Yet, enforce is inside std.contracts. If that isn't conflating the two concepts I wonder what it is. :-)

I agree completely. enforce must move.

Where to? Andrei
Jun 16 2010
next sibling parent reply Jonathan M Davis <jmdavisProg gmail.com> writes:
Andrei Alexandrescu wrote:

 Walter Bright wrote:
 Michel Fortin wrote:
 On 2010-06-16 05:15:24 -0400, Walter Bright
 <newshound1 digitalmars.com> said:

 The difference is not based on those 3 points, but on what Andrei
 wrote here. Contracts and error checking are completely distinct
 activities and should not be conflated.

True. Yet, enforce is inside std.contracts. If that isn't conflating the two concepts I wonder what it is. :-)

I agree completely. enforce must move.

Where to? Andrei

I would point out that pretty much nothing in std.contracts actually relates to contracts. Rather, it relates to error handling. So, it would probably be a good idea to simply rename the module - perhaps to std.error. - Jonathan M Davis
Jun 16 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-06-16 14:10:17 -0400, Jonathan M Davis <jmdavisProg gmail.com> said:

 I would point out that pretty much nothing in std.contracts actually relates
 to contracts. Rather, it relates to error handling. So, it would probably be
 a good idea to simply rename the module - perhaps to std.error.

I concur: the module is misnamed. The only things not related to error handling are assumeUnique and assumeSorted, and I fail to see the link with design by contract for either one. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Jun 16 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-06-16 14:44:29 -0400, Michel Fortin <michel.fortin michelf.com> said:

 On 2010-06-16 14:10:17 -0400, Jonathan M Davis <jmdavisProg gmail.com> said:
 
 I would point out that pretty much nothing in std.contracts actually relates
 to contracts. Rather, it relates to error handling. So, it would probably be
 a good idea to simply rename the module - perhaps to std.error.

I concur: the module is misnamed. The only things not related to error handling are assumeUnique and assumeSorted, and I fail to see the link with design by contract for either one.

Oh, forgot about "pointsTo" too. What's the link with contracts, or error handling? -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Jun 16 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Michel Fortin wrote:
 On 2010-06-16 14:44:29 -0400, Michel Fortin <michel.fortin michelf.com> 
 said:
 
 On 2010-06-16 14:10:17 -0400, Jonathan M Davis <jmdavisProg gmail.com> 
 said:

 I would point out that pretty much nothing in std.contracts actually 
 relates
 to contracts. Rather, it relates to error handling. So, it would 
 probably be
 a good idea to simply rename the module - perhaps to std.error.

I concur: the module is misnamed. The only things not related to error handling are assumeUnique and assumeSorted, and I fail to see the link with design by contract for either one.

Oh, forgot about "pointsTo" too. What's the link with contracts, or error handling?

Certain functions (notably swap) must make sure that there's no mutual aliasing between two objects. Andrei
Jun 16 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-06-16 14:59:45 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:

 Michel Fortin wrote:
 On 2010-06-16 14:44:29 -0400, Michel Fortin <michel.fortin michelf.com> said:
 
 On 2010-06-16 14:10:17 -0400, Jonathan M Davis <jmdavisProg gmail.com> said:
 
 I would point out that pretty much nothing in std.contracts actually relates
 to contracts. Rather, it relates to error handling. So, it would probably be
 a good idea to simply rename the module - perhaps to std.error.

I concur: the module is misnamed. The only things not related to error handling are assumeUnique and assumeSorted, and I fail to see the link with design by contract for either one.

Oh, forgot about "pointsTo" too. What's the link with contracts, or error handling?

Certain functions (notably swap) must make sure that there's no mutual aliasing between two objects.

Ok, so you're using "pointsTo" to check this in a contract? But isn't that just a utility function which can be used for contracts as much as for everything else? Does it really belong in std.contracts because at some place you use it in a contract? I don't think so. But that's something for you to decide. And unfortunately I'm not sure where you put it. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Jun 16 2010
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-06-16 20:45:47 -0400, Michel Fortin <michel.fortin michelf.com> said:

 Ok, so you're using "pointsTo" to check this in a contract? But isn't 
 that just a utility function which can be used for contracts as much as 
 for everything else? Does it really belong in std.contracts because at 
 some place you use it in a contract? I don't think so. But that's 
 something for you to decide. And unfortunately I'm not sure where you 
 put it.

Should have concluded by: "I'm not sure where you *should* put it either." -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Jun 16 2010
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Andrei Alexandrescu wrote:
 Walter Bright wrote:
 Michel Fortin wrote:
 On 2010-06-16 05:15:24 -0400, Walter Bright 
 <newshound1 digitalmars.com> said:

 The difference is not based on those 3 points, but on what Andrei 
 wrote here. Contracts and error checking are completely distinct 
 activities and should not be conflated.

True. Yet, enforce is inside std.contracts. If that isn't conflating the two concepts I wonder what it is. :-)

I agree completely. enforce must move.

Where to?

Dunno.
Jun 16 2010
parent reply Don <nospam nospam.com> writes:
Walter Bright wrote:
 Andrei Alexandrescu wrote:
 Walter Bright wrote:
 Michel Fortin wrote:
 On 2010-06-16 05:15:24 -0400, Walter Bright 
 <newshound1 digitalmars.com> said:

 The difference is not based on those 3 points, but on what Andrei 
 wrote here. Contracts and error checking are completely distinct 
 activities and should not be conflated.

True. Yet, enforce is inside std.contracts. If that isn't conflating the two concepts I wonder what it is. :-)

I agree completely. enforce must move.

Where to?

Dunno.

import std.dunno; Works for me.
Jun 16 2010
next sibling parent Walter Bright <newshound1 digitalmars.com> writes:
Don wrote:
 import std.dunno;
 Works for me.

cut & print.
Jun 16 2010
prev sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
Don wrote:

 import std.dunno;
 Works for me.

Or std.poisson... :p Ali
Jun 16 2010
parent biozic <dransic free.fr> writes:
Le 16/06/10 22:36, Ali Çehreli a écrit :
 Don wrote:

 import std.dunno;
 Works for me.

Or std.poisson... :p

Better name it std.fishy, because std.poisson could be mistaken for a statistical distribution module!
Jun 16 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 16 Jun 2010 05:28:46 -0400, Ary Borenszweig <ary esperanto.org.ar>  
wrote:

 On 06/16/2010 04:15 PM, Walter Bright wrote:
 Ali Çehreli wrote:
 bearophile wrote:
 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?

I can see two benefits:

The difference is not based on those 3 points, but on what Andrei wrote here. Contracts and error checking are completely distinct activities and should not be conflated.

Could you please explain them? There are many people here that don't understand the difference between these two concepts (including me). So maybe we are too dumb, maybe those concepts are not generally known or maybe the explanation is not very well clear in the documentation.

I think of enforce as a convenient way translating an error in an expectation to an exception in a single expression. For example, take some system call that returns -1 on error, you could do this: if(result < 0) throw new Exception("oops!"); or you could do this: enforce(result >= 0, "oops!"); Think of enforce as "throw if" And in fact, I think there's an errnoEnforce which throws a standard exception with the string error from the system. I'd say the difference between enforce and assert is exactly what Andrei said -- enforce is meant to catch errors that can occur during normal operation. Assert is meant to catch errors that are not expected during normal operation. Assert's more like a sanity check. Also, assert is turned off in release mode, enforce is left on. -Steve
Jun 16 2010
prev sibling next sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Wed, 16 Jun 2010 06:55:21 -0400, Steven Schveighoffer wrote:

 On Wed, 16 Jun 2010 05:28:46 -0400, Ary Borenszweig
 <ary esperanto.org.ar> wrote:
 
 On 06/16/2010 04:15 PM, Walter Bright wrote:
 Ali Çehreli wrote:
 bearophile wrote:
 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a
 language that has built-in Contract Programming?

I can see two benefits:

The difference is not based on those 3 points, but on what Andrei wrote here. Contracts and error checking are completely distinct activities and should not be conflated.

Could you please explain them? There are many people here that don't understand the difference between these two concepts (including me). So maybe we are too dumb, maybe those concepts are not generally known or maybe the explanation is not very well clear in the documentation.

I think of enforce as a convenient way translating an error in an expectation to an exception in a single expression. For example, take some system call that returns -1 on error, you could do this: if(result < 0) throw new Exception("oops!"); or you could do this: enforce(result >= 0, "oops!"); Think of enforce as "throw if"

It also adds a file and a line number to the error message, so the problem is easier to track down. Very handy. :)
 And in fact, I think there's an errnoEnforce which throws a standard
 exception with the string error from the system.

That's right, and there's even an enforceEx() which lets you specify which exception type to throw: http://digitalmars.com/d/2.0/phobos/std_contracts.html#enforceEx -Lars
Jun 16 2010
prev sibling next sibling parent "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Lutger <lutger.blijdestijn gmail.com> wrote:

 Suppose for example (actually this is from real life) there is an  
 important
 operation which, as a service, also sends an e-mail notification as part  
 of that
 operation. It is very bad if the operation fails, but a failed  
 notification is
 not that bad. What to do in case of a bug with the e-mail notification?

 1. crash (gracefully), do not complete the operation.
 2. log the error for the devs to look into (or crash) *after* the  
 operation is
 complete, let the operation go through without the e-mail notification.

 Option 1 is annoying and prevents people from getting work done due to a  
 'minor'
 bug. Option 2 however probably results in this bug either not getting  
 noticed
 quite early enough or ignored in the face of other issues that always  
 seems to
 have higher priority. Choosing for option 2 can also lead to bugs being
 swallowed silently or mistaken for exceptional conditions, which is more
 dangerous.

 I don't mean to split hairs, I bet a lot of software has these kind of  
 cases.

How did you end up with an email system that is so horribly broken that it spits Errors instead of Exceptions when things are not quite the way it wants them to be? If it cannot send the email, it may throw an Exception. If you try and pass it a handwritten letter, it should throw an Error. Basically, throwing an Exception means 'Your attention please, reactor 5 has a cooling problem you might want to look at', whereas an Error means 'Explosion imminent, get the fuck off outta here!'. -- Simen
Jun 16 2010
prev sibling next sibling parent "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Lutger <lutger.blijdestijn gmail.com> wrote:

 Simen kjaeraas wrote:

 ...
 If it cannot send the email, it may throw an Exception. If you try and
 pass it a handwritten letter, it should throw an Error.

This is the question: should I segfault on a handwritten letter even if it is not such an important letter and could just go on operating?

Yes. If someone is passing your email system a handwritten letter, something is so wrong, the program should balk and exit. It's not just a small mixup, it's an indication something is completely wrong. As Walter put it, an Error, be it an AssertError or otherwise, means your program has ventured into uncharted territory, and behavior from this point on is undefined. "Permissible undefined behavior ranges from ignoring the situation completely with unpredictable results, to having demons fly out of your nose."[1] -- Simen [1] http://groups.google.com/group/comp.std.c/msg/dfe1ef367547684b?pli=1
Jun 17 2010
prev sibling next sibling parent "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Lutger <lutger.blijdestijn gmail.com> wrote:

 How did you end up with an email system that is so horribly broken th=


 it spits Errors instead of Exceptions when things are not quite the w=


 it wants them to be?

Not Errors, it is not in D and does not distinguish between Errors and=

 Exceptions. It was an example, a (design) question. It's very simple:

Ah. Then no, handling the exception is perfectly acceptable, and probabl= y the right thing to do.
 Basically, throwing an Exception means 'Your attention please, reacto=


 has a cooling problem you might want to look at', whereas an Error me=


 'Explosion imminent, get the fuck off outta here!'.

No, an Error means the program has a bug. Programs have thousands of =

 bugs, this
 is not related to how critical it is. An Exception can be way more  =

 important to
 fix than a bug. WebServerDownException for example, is often not a bug=

 in the
 code that drives websites, but for sure I will contact the sysadmin  =

 before even
 thinking of going back to work. The question is how to proceed after t=

 fact.

Yes and no. Throwing an error puts the program into an undefined state, and everything may happen. Because everything happening at once would strain space-time and cause Bad Things=E2=84=A2 to happen, we would like= to limit the time in which this does occur. Hence, we bail out. That said, a bug in a rarely-used function may indeed be significantly less important than getting a server back online. However, I would still= say an error indicates something is fundamentally wrong. -- = Simen
Jun 17 2010
prev sibling next sibling parent reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Wed, 16 Jun 2010 07:31:39 -0700, Andrei Alexandrescu wrote:

 Michel Fortin wrote:
 On 2010-06-16 05:15:24 -0400, Walter Bright
 <newshound1 digitalmars.com> said:
 
 The difference is not based on those 3 points, but on what Andrei
 wrote here. Contracts and error checking are completely distinct
 activities and should not be conflated.

True. Yet, enforce is inside std.contracts. If that isn't conflating the two concepts I wonder what it is. :-)

You're right! I think Lars' suggestion is sensible - we should move enforce to object. Better yet we should find a better name for std.contracts. Ideas? Andrei

A few suggestions (even though I still think it belongs in object.d), in no particular order: std.enforce std.assumptions std.constraints std.checks std.tests std.error std.errcheck -Lars
Jun 17 2010
parent Jonathan M Davis <jmdavisprog gmail.com> writes:
On Sunday 27 June 2010 16:09:02 Andrei Alexandrescu wrote:
 
 We haven't reached consensus on where to put enforce() and friends. Any
 other ideas? Of the above, I like std.checks.
 
 Better yet, how about defining std.exception that includes a host of
 exception-related functionality (such as defining exceptions that retain
 file and line, perhaps stack traces etc.)?
 
 
 Andrei

std.exception sounds like a good plan. I'm not overly fond of any of the other names, and I'm not sure that I care much one way or the other if we pick one, but std.exception with a bunch of exception-related stuff sounds particularly useful and could help standardize some of the way exceptions are used in D code. - Jonathan M Davis
Jun 27 2010
prev sibling next sibling parent "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:

 Better yet, how about defining std.exception that includes a host of  
 exception-related functionality (such as defining exceptions that retain  
 file and line, perhaps stack traces etc.)?

Sounds good. -- Simen
Jun 27 2010
prev sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Sun, 27 Jun 2010 18:09:02 -0500, Andrei Alexandrescu wrote:

 On 06/17/2010 04:10 AM, Lars T. Kyllingstad wrote:
 On Wed, 16 Jun 2010 07:31:39 -0700, Andrei Alexandrescu wrote:

 Michel Fortin wrote:
 On 2010-06-16 05:15:24 -0400, Walter Bright
 <newshound1 digitalmars.com>  said:

 The difference is not based on those 3 points, but on what Andrei
 wrote here. Contracts and error checking are completely distinct
 activities and should not be conflated.

True. Yet, enforce is inside std.contracts. If that isn't conflating the two concepts I wonder what it is. :-)

You're right! I think Lars' suggestion is sensible - we should move enforce to object. Better yet we should find a better name for std.contracts. Ideas? Andrei

A few suggestions (even though I still think it belongs in object.d), in no particular order: std.enforce std.assumptions std.constraints std.checks std.tests std.error std.errcheck -Lars

We haven't reached consensus on where to put enforce() and friends. Any other ideas? Of the above, I like std.checks. Better yet, how about defining std.exception that includes a host of exception-related functionality (such as defining exceptions that retain file and line, perhaps stack traces etc.)?

TDPL mentions several times that enforce() is in std.contracts. Doesn't that preclude moving it or renaming the module? -Lars
Jun 28 2010
prev sibling next sibling parent Leandro Lucarella <luca llucax.com.ar> writes:
Steven Schveighoffer, el 16 de junio a las 06:55 me escribiste:
 On Wed, 16 Jun 2010 05:28:46 -0400, Ary Borenszweig
 <ary esperanto.org.ar> wrote:
 
On 06/16/2010 04:15 PM, Walter Bright wrote:
Ali Çehreli wrote:
bearophile wrote:
I have counted about 200 usages of std.contracts.enforce() inside
Phobos. Can you tell me what's the purpose of enforce() in a language
that has built-in Contract Programming?

I can see two benefits:

The difference is not based on those 3 points, but on what Andrei wrote here. Contracts and error checking are completely distinct activities and should not be conflated.

Could you please explain them? There are many people here that don't understand the difference between these two concepts (including me). So maybe we are too dumb, maybe those concepts are not generally known or maybe the explanation is not very well clear in the documentation.

I think of enforce as a convenient way translating an error in an expectation to an exception in a single expression. For example, take some system call that returns -1 on error, you could do this: if(result < 0) throw new Exception("oops!"); or you could do this: enforce(result >= 0, "oops!"); Think of enforce as "throw if"

So maybe throw_if() would be a better name =) Anyway, I think enforce() is poisson, because it make the programmer to not think about errors at all, just add and enforce() and there you go. But when you need to be fault tolerant, is very important to know what's the nature of the error, but thanks to enforce(), almost every error is a plain Exception, no hierarchy, no extra info, all you can do to get a little more info about what happened is to parse the exception string, and that's not really an option.
 And in fact, I think there's an errnoEnforce which throws a standard
 exception with the string error from the system.

That's the only useful case of enforce, because it includes the *important* information (the actual errno). There is also enforceEx!(), to use a custom exception, which practically nobody uses (I counted only 4 uses in phobos). -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- Y será el día en que la electricidad deje de ser rayo y sea depilador femenino. -- Ricardo Vaporeso
Jun 16 2010
prev sibling parent Leandro Lucarella <luca llucax.com.ar> writes:
Andrei Alexandrescu, el 16 de junio a las 07:53 me escribiste:
because it make the programmer to
not think about errors at all, just add and enforce() and there you go.
But when you need to be fault tolerant, is very important to know what's
the nature of the error, but thanks to enforce(), almost every error is
a plain Exception, no hierarchy, no extra info, all you can do to get
a little more info about what happened is to parse the exception string,
and that's not really an option.

I think there is no real need for exception hierarchies. I occasionally dream of eliminating all of the useless exceptions defined left and right in Phobos.

Exception hierarchy is only one way to discriminate error types. Extra info, is another (like an error code). I agree that a *large* exception hierarchy hurts more than it helps.
And in fact, I think there's an errnoEnforce which throws a standard
exception with the string error from the system.

That's the only useful case of enforce, because it includes the *important* information (the actual errno). There is also enforceEx!(), to use a custom exception, which practically nobody uses (I counted only 4 uses in phobos).

I'd be hard pressed to find good examples of exception hierarchy use. Everybody talks about them but I've seen none.

I think Python has a good one. I find myself discriminating between ValueError, IndexError, KeyError, OSError and IOError all the time.
 The fact that the coder doesn't need to think hard to use enforce()
 effectively is a plus, not a minus. An overdesigned enforce that
 adds extra burden to its user would have been a mistake.

That is, if you don't care on handling errors and let the program crash with a backtrace, or add a big try {} catch (Exception) in the main. If that's not the case, it only produce a false feeling that D (standard library) is good handling errors when it's not, it's just a binary "there is an error" - "there is no errors". -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- The average person laughs 13 times a day
Jun 16 2010