www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.experimental.checkedint is ready for comments!

reply Robert burner Schadek <rburners gmail.com> writes:
As with many other languages (C, C++, Java, etc.), D's built-in 
integer data types are quite difficult to use correctly.

It is tempting to think of int, for example, as if it were an 
actual mathematical integer. Doing so, however leads to buggy 
code due to unintuitive behaviour like:

* Wrapped overflow
* Reinterpretation of signed values as unsigned in mixed 
expressions
* Floating Point Exceptions which aren't Exceptions and have 
nothing to do with
   floating point
* Formally "undefined behaviour" with some inputs for various 
operations

This checkedint module provides alternative operations and types 
(SafeInt,SmartInt) that protect the user from most difficulties 
of this sort, while maintaining good performance (provided that 
inlining and optimizations are enabled).

== SmartInt ==
SmartInt smartOp strive to actually give the mathematically 
correct answer whenever possible, rather than just signaling an 
error.

== SafeInt ==
SafeInt safeOp strive to match the behaviour of the basic 
integral types exactly, $(B except) that where the behaviour of 
the basic type is wrong, or very unintuitive, an error is 
signaled instead.


The main downsides to using checkedint are:

Some added friction when interfacing to non-checkedint-aware code.
Slower compilation and larger binaries.


PR: https://github.com/dlang/phobos/pull/4407
DUB: http://code.dlang.org/packages/checkedint

I will do the review management.
Jun 07 2016
next sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Tuesday, 7 June 2016 at 08:50:07 UTC, Robert burner Schadek 
wrote:

 I will do the review management.


Thanks.
Jun 07 2016
prev sibling next sibling parent Jack Stouffer <jack jackstouffer.com> writes:
On Tuesday, 7 June 2016 at 08:50:07 UTC, Robert burner Schadek 
wrote:

 ...


I left my commentary in the PR. Overall it looks pretty good 
design wise, and I would totally vote for it's inclusion in 
Phobos.
Jun 09 2016
prev sibling next sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
In two weeks I will talk to tsbockmann how much time he needs to 
work in all comments.
After he is done I will start the formal review phase.

p.s.
 everybody please take an interest. This module can give D 
another strategic advantage over our competition.
Jun 14 2016
parent reply Andrea Fontana <nospam example.com> writes:
On Tuesday, 14 June 2016 at 10:20:58 UTC, Robert burner Schadek 
wrote:

 In two weeks I will talk to tsbockmann how much time he needs 
 to work in all comments.
 After he is done I will start the formal review phase.

 p.s.
  everybody please take an interest. This module can give D 
 another strategic advantage over our competition.


Any documentation/example?
Jun 14 2016
parent reply Andrea Fontana <nospam example.com> writes:
On Tuesday, 14 June 2016 at 10:33:34 UTC, Andrea Fontana wrote:

 On Tuesday, 14 June 2016 at 10:20:58 UTC, Robert burner Schadek 
 wrote:

 In two weeks I will talk to tsbockmann how much time he needs 
 to work in all comments.
 After he is done I will start the formal review phase.

 p.s.
  everybody please take an interest. This module can give D 
 another strategic advantage over our competition.


 Any documentation/example?


Whoops I see it's in PR. You should link here too :)
Jun 14 2016
parent tsbockman <thomas.bockman gmail.com> writes:
On Tuesday, 14 June 2016 at 10:34:49 UTC, Andrea Fontana wrote:

 On Tuesday, 14 June 2016 at 10:33:34 UTC, Andrea Fontana wrote:

 Any documentation/example?


 Whoops I see it's in PR. You should link here too :)


I can't, because the link is not stable. The docs are rebuilt and 
given a new URL every time I update the PR.
Jun 14 2016
prev sibling next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 7 June 2016 at 08:50:07 UTC, Robert burner Schadek 
wrote:

 PR: https://github.com/dlang/phobos/pull/4407


I also have

https://github.com/nordlow/phobos-next/blob/master/src/bound.d
providing Ada-style range type at
https://github.com/nordlow/phobos-next/blob/master/src/bound.d#L286

and

https://github.com/nordlow/phobos-next/blob/master/src/modulo.d
providing Ada-style module type at
https://github.com/nordlow/phobos-next/blob/master/src/modulo.d#L46

I'm currently using `Mod` in an useful way here
https://github.com/nordlow/phobos-next/blob/master/src/typecons_ex.d#L227
unittested here
https://github.com/nordlow/phobos-next/blob/master/src/typecons_ex.d#L286

to automatically get bounded integers when searching for elements 
in static arrays. In turn, I'm using this in my radix tree 
implementation here

https://github.com/nordlow/phobos-next/blob/master/src/trie.d#L922

to provide more type-safe indexing of statically sized arryas.

I know, it's big, but it *is* motivated when accessing static 
arrays in specialized containers like radix trees.

Have you thought about extending checkedint to something similar 
to bounded integer wrapper type like my `bound.d`?

Not that my `Bound`-type probably should be used inplace of 
`Mod`, but I haven't had time to do the switch yet.
Jun 14 2016
next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 14 June 2016 at 10:39:01 UTC, Nordlöw wrote:

 providing Ada-style module type at


Should be:
providing Ada-style modulo type at
Jun 14 2016
prev sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Tuesday, 14 June 2016 at 10:39:01 UTC, Nordlöw wrote:

 Have you thought about extending checkedint to something 
 similar to bounded integer wrapper type like my `bound.d`?


I spent some time studying the possibility of a `BoundInt` type. 
Some conclusions I reached:

    1) Designing and implementing `BoundInt` to my standards for 
quality and
       performance would be a large project of similar magnitude 
to what has
       already been done on `checkedint`, which took me about a 
year. (And I
       was building on the earlier work of  burner and others.)

    2) `BoundInt` is not a replacement for `SafeInt` or 
`SmartInt`, although
       they would likely share some parts of the implementation.

    3) Adding `BoundInt` later should not require any breaking 
changes to the
       public API of `checkedint`.

    4) Pervasive, natural use of `BoundInt` in large systems (like 
Phobos)
       may cause *awful* template bloat issues, depending on the 
design used.
       (Improvements to the compiler front-end could mitigate this 
issue, in
       the long run.)

Given the above, I believe we should move forward with 
`checkedint` as-is. Someone can add a `BoundInt` type to it 
later, if there is demand.
Jun 14 2016
parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 14 June 2016 at 13:58:29 UTC, tsbockman wrote:

 Given the above, I believe we should move forward with 
 `checkedint` as-is. Someone can add a `BoundInt` type to it 
 later, if there is demand.


Ok. Thanks.
Jun 14 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Thanks for doing this!

On 6/7/2016 1:50 AM, Robert burner Schadek wrote:

 == SmartInt ==
 SmartInt smartOp strive to actually give the mathematically correct answer
 whenever possible, rather than just signaling an error.

 == SafeInt ==
 SafeInt safeOp strive to match the behaviour of the basic integral types
 exactly, $(B except) that where the behaviour of the basic type is wrong, or
 very unintuitive, an error is signaled instead.



What the differences between these types are is unclear, for example, the
former 
says "correct" and the latter says "not wrong". Also, "very unintuitive" has a 
pretty slippery meaning.
Jun 14 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/14/2016 4:25 PM, Walter Bright wrote:

 Thanks for doing this!

 On 6/7/2016 1:50 AM, Robert burner Schadek wrote:

 == SmartInt ==
 SmartInt smartOp strive to actually give the mathematically correct answer
 whenever possible, rather than just signaling an error.

 == SafeInt ==
 SafeInt safeOp strive to match the behaviour of the basic integral types
 exactly, $(B except) that where the behaviour of the basic type is wrong, or
 very unintuitive, an error is signaled instead.



 What the differences between these types are is unclear, for example, the
former
 says "correct" and the latter says "not wrong". Also, "very unintuitive" has a
 pretty slippery meaning.


A better explanation:

http://dtest.thecybershadow.net/artifact/website-7cc6e938154f90aa49fa6451a85b13e35ab2de99-bc50ab8ca1b076e361ddc71d706010b7/web/phobos-prerelease/std_experimental_checkedint.html
Jun 14 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Overall, I find the documentation to be unusually good. Nice work!

================================================

.noex

   The .noex member is oddly named, being a negative and no idea what 'ex' 
means. It sets a sticky flag on error, so perhaps .sticky?

---
N is a basic integral type, it needs a better name than 'N'. How about 
'BaseType' or even 'BaseIntegralType'? A complete list of what types are 
acceptable for 'N' would be desirous, too.

---
'bscal' is a complete mystery without reading the documentation. Perhaps
'value' 
instead?

---
enum IntFlagPolicy policy;

   Should contain a link to the complete explanation.

---
Should have a "References:" section with link to core.checkedint

---
-O (DMD) should be a link to the -O flag instructions 
http://dlang.org/dmd-windows.html#switch-O

---
'--inline' is not a DMD switch

---
Remove all use of 'you' and 'your' from the documentation. For example:

"If you really need more speed, try switching to DebugInt for the hottest code 
in your program (like inner loops) before giving up on checkedint entirely."

becomes:

"For more speed, switch to DebugInt for the hottest code in the program (like 
inner loops)."

(The internal link to DebugInt is wrong.)

And:

"This way, you can use SafeInt!N to debug your integer logic in testing, but 
switch to basic N in release mode for maximum speed and the smallest binaries."

becomes:

"This way, use SafeInt!N to debug integer logic in testing, but switch to basic 
N in release mode for maximum speed and the smallest binaries."

Reference material should never contain "you" or "your".

---
"debuggin" => "debugging"

---
There's an overall lack of use of internal links to navigate within the page.

---
Functions are tagged  safe. Are there any unsafe functions?
Jun 14 2016
next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 00:16:12 UTC, Walter Bright wrote:

 Overall, I find the documentation to be unusually good. Nice 
 work!


Thanks. :-D


 ================================================

 .noex

   The .noex member is oddly named, being a negative and no idea 
 what 'ex' means. It sets a sticky flag on error, so perhaps 
 .sticky?


Originally I wanted to have the policies just be `throws` and 
`nothrow` - but of course `nothrow` is a keyword, so I chose 
`noex` (short for "no exceptions") instead. I agree it looks kind 
of odd though, especially since I later added the `asserts` 
policy.

I'll consider renaming `noex`, or at least add an explanation of 
the mnemonic to the docs.


 ---
 N is a basic integral type, it needs a better name than 'N'. 
 How about 'BaseType' or even 'BaseIntegralType'? A complete 
 list of what types are acceptable for 'N' would be desirous, 
 too.


`N` is not a public symbol, and it's used all over the place. Do 
I really need to give it some giant multi-word name?

Besides which, Phobos uses single-letter names for template 
parameter type names all over the place.


 ---
 'bscal' is a complete mystery without reading the 
 documentation. Perhaps 'value' instead?


`value` is a very popular name for function parameters and local 
variables. `bscal` needs a more obscure name to prevent the free 
function version (a shim to facilitate uniform treatment of basic 
integral types and checkedint types in generic code) from 
shadowing or being shadowed by local symbols.

(I originally called it `value`, but changed it once I realized 
how annoying it is not to be able to use that name for anything 
else in generic code that uses `checkedint`.)

Also, I prefer a more unfriendly name for `bscal` to remind 
people that accessing it entails giving up all of the safety 
benefits of `checkedint`. The fact that it's grep-able, too, 
could be helpful to linters and the like.


 ---
 enum IntFlagPolicy policy;

   Should contain a link to the complete explanation.

 ---
 Should have a "References:" section with link to core.checkedint

 ---
 -O (DMD) should be a link to the -O flag instructions 
 http://dlang.org/dmd-windows.html#switch-O

 ---
 '--inline' is not a DMD switch

 ---
 Remove all use of 'you' and 'your' from the documentation. For 
 example:

 "If you really need more speed, try switching to DebugInt for 
 the hottest code in your program (like inner loops) before 
 giving up on checkedint entirely."

 becomes:

 "For more speed, switch to DebugInt for the hottest code in the 
 program (like inner loops)."

 (The internal link to DebugInt is wrong.)

 And:

 "This way, you can use SafeInt!N to debug your integer logic in 
 testing, but switch to basic N in release mode for maximum 
 speed and the smallest binaries."

 becomes:

 "This way, use SafeInt!N to debug integer logic in testing, but 
 switch to basic N in release mode for maximum speed and the 
 smallest binaries."

 Reference material should never contain "you" or "your".

 ---
 "debuggin" => "debugging"

 ---
 There's an overall lack of use of internal links to navigate 
 within the page.


I will work on the docs some more.


 ---
 Functions are tagged  safe. Are there any unsafe functions?


Sort of. There are a small number of template functions in 
`checkedint` that call some outside function whose  safety is 
unspecified, requiring the  safe-ty of the `checkedint` function 
to be inferred.

I believe the complete list currently is:

* SmartInt.toString(sink, fmt)
* SafeInt.toString(sink, fmt)
* checkedint.to()
* IntFlag.toString(sink, fmt)
* IntFlags.toString(sink, fmt)

(It would be great if we could fill in the gaps in D's attribute 
system - it's kind of ridiculous that for every attribute, there 
is at least one possible mode that cannot be indicated explicitly 
in any way.)
Jun 14 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/14/2016 8:15 PM, tsbockman wrote:

 N is a basic integral type, it needs a better name than 'N'. How about
 'BaseType' or even 'BaseIntegralType'? A complete list of what types are
 acceptable for 'N' would be desirous, too.


 `N` is not a public symbol, and it's used all over the place.


It appears all over the public documentation.


 Do I really need to give it some giant multi-word name?


Something better than 'N'.


 Besides which, Phobos uses single-letter names for template parameter type
names
 all over the place.


Generally 'T' is used for 'any type'. 'N' has no corresponding significance.


 ---
 'bscal' is a complete mystery without reading the documentation. Perhaps
 'value' instead?


 `value` is a very popular name for function parameters and local variables.
 `bscal` needs a more obscure name to prevent the free function version (a shim
 to facilitate uniform treatment of basic integral types and checkedint types in
 generic code) from shadowing or being shadowed by local symbols.

 (I originally called it `value`, but changed it once I realized how annoying it
 is not to be able to use that name for anything else in generic code that uses
 `checkedint`.)

 Also, I prefer a more unfriendly name for `bscal` to remind people that
 accessing it entails giving up all of the safety benefits of `checkedint`. The
 fact that it's grep-able, too, could be helpful to linters and the like.


Greppability is inddeed a plus, but 'bscal' is needlessly uninformative. Note 
that publicly facing names should not be so.



 I will work on the docs some more.


Thank you.



 * SmartInt.toString(sink, fmt)
 * SafeInt.toString(sink, fmt)
 * checkedint.to()
 * IntFlag.toString(sink, fmt)
 * IntFlags.toString(sink, fmt)


I see no love for output ranges :-(
Jun 14 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 03:42:52 UTC, Walter Bright wrote:

 On 6/14/2016 8:15 PM, tsbockman wrote:

 Do I really need to give it some giant multi-word name?


 Something better than 'N'.


`Int`? `Base`?

Whatever it is needs to be short; `BaseIntegralType` is *way* too 
long for this and would make many of the signatures painfully 
verbose.


 Besides which, Phobos uses single-letter names for template 
 parameter type names
 all over the place.


 Generally 'T' is used for 'any type'. 'N' has no corresponding 
 significance.


I have seen all of `S`, `F`, `G`, `X`, `R`, `C`, `A`, and `B` 
used as template parameter names in Phobos. Often there is no 
particular significance to the letter chosen, but the purpose of 
the parameter is obvious from the context, anyway.

Using short template parameter names helps keep D's 
already-very-long signatures from growing even longer than they 
already are.


 Greppability is inddeed a plus, but 'bscal' is needlessly 
 uninformative. Note that publicly facing names should not be so.


`basic`? `base`?

Again, this needs to be short for readability and usability. 
Normally it's not needed at all, but when it is needed (like in 
the implementation of `SmartInt` and `SafeInt`), it tends to be 
needed *a lot*.

The obvious choices - `value`, `val`, or `raw` - are out because 
of the shadowing problem.


 I will work on the docs some more.


 Thank you.



 * SmartInt.toString(sink, fmt)
 * SafeInt.toString(sink, fmt)
 * checkedint.to()
 * IntFlag.toString(sink, fmt)
 * IntFlags.toString(sink, fmt)


 I see no love for output ranges :-(


I will add support. Someone should update this wiki page with 
whatever the current best practice is:
     http://wiki.dlang.org/Defining_custom_print_format_specifiers
Jun 14 2016
next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 04:48:02 UTC, tsbockman wrote:

 On Wednesday, 15 June 2016 at 03:42:52 UTC, Walter Bright wrote:

 * SmartInt.toString(sink, fmt)
 * SafeInt.toString(sink, fmt)
 * checkedint.to()
 * IntFlag.toString(sink, fmt)
 * IntFlags.toString(sink, fmt)


 I see no love for output ranges :-(


 I will add support.


Done. (It turns out that they were actually already supported, 
but I updated the docs to make this clearer.)
Jun 14 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/14/2016 11:17 PM, tsbockman wrote:

 Done.


Pretty dazz!



 (It turns out that they were actually already supported, but I updated the
 docs to make this clearer.)


Ain't it cool when that happens?
Jun 15 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/14/2016 9:48 PM, tsbockman wrote:

 On Wednesday, 15 June 2016 at 03:42:52 UTC, Walter Bright wrote:

 On 6/14/2016 8:15 PM, tsbockman wrote:

 Do I really need to give it some giant multi-word name?


 Something better than 'N'.


 `Int`? `Base`?


'Integer' would work fine.



 Whatever it is needs to be short; `BaseIntegralType` is *way* too long for this
 and would make many of the signatures painfully verbose.


Since they aren't recursive, that shouldn't be a problem. Besides, the
signature 
will be the actual type name, not the alias for it.



 Besides which, Phobos uses single-letter names for template parameter type
names
 all over the place.


 Generally 'T' is used for 'any type'. 'N' has no corresponding significance.


 I have seen all of `S`, `F`, `G`, `X`, `R`, `C`, `A`, and `B` used as template
 parameter names in Phobos. Often there is no particular significance to the
 letter chosen, but the purpose of the parameter is obvious from the context,
 anyway.


1. Each one should be evaluated on its own merits.
2. Style is not something cast in stone, we try to constant evolve better ways.
3. Bad practice in one case is not a rationale to use bad practice elsewhere :-)
4. You mentioned greppability - 'N' is as ungreppable as it gets!



 Using short template parameter names helps keep D's already-very-long
signatures
 from growing even longer than they already are.


The parameter names don't appear in the signature.



 Greppability is inddeed a plus, but 'bscal' is needlessly uninformative. Note
 that publicly facing names should not be so.


 `basic`? `base`?


'integer'? (to go with 'Integer' for the type)



 Again, this needs to be short for readability and usability. Normally it's not
 needed at all, but when it is needed (like in the implementation of `SmartInt`
 and `SafeInt`), it tends to be needed *a lot*.


Internal to the implementation does not require it to be short.



 I see no love for output ranges :-(


 I will add support.


Thank you.
Jun 15 2016
parent tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 07:16:18 UTC, Walter Bright wrote:

 On 6/14/2016 9:48 PM, tsbockman wrote:

 `Int`? `Base`?


 'Integer' would work fine.


`BaseInt`?

`SmartInt!Integer` looks weird to me, because of the repetition. 
Also, if we're going to use a long name like that I think it 
should hint at the fact that the true base type of a checked 
integer type is always a basic type, not a custom type like 
`SmartInt` or `BigInt`.


 Besides, the signature will be the actual type name, not the 
 alias for it.


I'm not sure what you're getting at here?

I'm concerned about how stuff like this appears in the docs 
(which is the only place where the user will ever see this name). 
Obviously the docs must use the alias, since they are for the 
generic type, rather than a specific instantiation.


 I have seen all of `S`, `F`, `G`, `X`, `R`, `C`, `A`, and `B` 
 used as template
 parameter names in Phobos. Often there is no particular 
 significance to the
 letter chosen, but the purpose of the parameter is obvious 
 from the context,
 anyway.


 1. Each one should be evaluated on its own merits.
 2. Style is not something cast in stone, we try to constant 
 evolve better ways.


Obviously I don't think your way is better.  :-)


 3. Bad practice in one case is not a rationale to use bad 
 practice elsewhere :-)


But when a practice works out fine in one case, that *is* 
evidence that it is likely to work out in others.


 4. You mentioned greppability - 'N' is as ungreppable as it 
 gets!


Which is irrelevant, because `N`/`BaseIntegralType`/whatever will 
never appear in the user's code (but `bscal` will).


 Using short template parameter names helps keep D's 
 already-very-long signatures
 from growing even longer than they already are.


 The parameter names don't appear in the signature.


They appear in the docs. I'm talking about the docs, because 
that's the only place anyone outside the Phobos dev team will 
ever see the `N`/`Integer`/whatever name.


 Again, this needs to be short for readability and usability. 
 Normally it's not
 needed at all, but when it is needed (like in the 
 implementation of `SmartInt`
 and `SafeInt`), it tends to be needed *a lot*.


 Internal to the implementation does not require it to be short.


Keeping names that don't need to be long short(ish) makes it a 
lot easier for me to read code.

Also, in practice people usually choose convenience over safety. 
If writing an algorithm using `checkedint` instead of the basic 
integer types makes the algorithm twice as long, the vast 
majority of people simply won't use `checkedint`.

For me, there is no point to pursuing inclusion of `checkedint` 
in Phobos if doing so requires making the API so verbose that no 
one will want to use it. (Which is not to say that `.bscal` => 
`.integer` is such a problem.)
Jun 15 2016
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2016-06-15 05:15, tsbockman wrote:


 Originally I wanted to have the policies just be `throws` and `nothrow`
 - but of course `nothrow` is a keyword, so I chose `noex` (short for "no
 exceptions") instead. I agree it looks kind of odd though, especially
 since I later added the `asserts` policy.


nothrow is actually two words, "no" and "throw". Therefore the symbol 
should be noThrow. Solving both the conflict with the keyword and using 
correct grammar :)

-- 
/Jacob Carlborg
Jun 14 2016
prev sibling next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 00:16:12 UTC, Walter Bright wrote:

 A complete list of what types are acceptable for 'N' would be 
 desirous, too.


This is specified fully in the template constraints:
     if (isIntegral!N && isUnqual!N)
     if ((isIntegral!N && !isUnqual!N) || isCheckedint!N)

The second overload simply forwards to the first, after applying 
BasicScalar!(Unqual!N).

However, I recall that new users tend to get confused/intimidated 
by template constraints, so I suppose I should add a prose 
explanation to the docs, as well.
Jun 14 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/14/2016 8:23 PM, tsbockman wrote:

 On Wednesday, 15 June 2016 at 00:16:12 UTC, Walter Bright wrote:

 A complete list of what types are acceptable for 'N' would be desirous, too.


 This is specified fully in the template constraints:
     if (isIntegral!N && isUnqual!N)
     if ((isIntegral!N && !isUnqual!N) || isCheckedint!N)

 The second overload simply forwards to the first, after applying
 BasicScalar!(Unqual!N).


Why would a checkedint be a base type for a checkedint?



 However, I recall that new users tend to get confused/intimidated by template
 constraints, so I suppose I should add a prose explanation to the docs, as
well.


The fact that anyone has to write !isUnqual, i.e. a double negative, is crummy 
design. But that's not your fault :-)
Jun 14 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 03:45:39 UTC, Walter Bright wrote:

 On 6/14/2016 8:23 PM, tsbockman wrote:

 This is specified fully in the template constraints:
     if (isIntegral!N && isUnqual!N)
     if ((isIntegral!N && !isUnqual!N) || isCheckedint!N)

 The second overload simply forwards to the first, after 
 applying
 BasicScalar!(Unqual!N).


 Why would a checkedint be a base type for a checkedint?


Generic code (contrived, oversimplified example):

     import checkedint.throws, checkedint.traits;

     SmartInt!(typeof(A.init + B.init)) add(A, B)(const A a, const 
B b)
         if (isIntegral!A && isIntegral!B)
     {
         SmartInt!A ma = a;
         SmartInt!B mb = b;

         return ma + mb;
     }

Of course I could force the user to write `BasicScalar` 
everywhere - but why? The intent is just as clear this way, and 
it's less verbose.
Jun 14 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/14/2016 9:57 PM, tsbockman wrote:

 The intent is just as clear this way, and it's less verbose.


Ok. I'd just change the constraint to:

     if (isIntegral!N || isCheckedint!N)

You can do the qualification machinations using a static if inside the template.
Jun 15 2016
parent tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 07:08:22 UTC, Walter Bright wrote:

 On 6/14/2016 9:57 PM, tsbockman wrote:

 The intent is just as clear this way, and it's less verbose.


 Ok. I'd just change the constraint to:

     if (isIntegral!N || isCheckedint!N)

 You can do the qualification machinations using a static if 
 inside the template.


This was a deliberate design decision, which I do not think 
should be changed (although I will if you insist):

     * With your signature, the user may wrongly expect SmartInt to
       support qualified `N`. My scheme encodes the fact that only 
unqualified
       types are *really* supported in the public signatures.

       (I should add an explanatory note about this in the docs, 
too.)

     * Your scheme will make uglier and more verbose error 
messages and fully
       qualified type signatures, as `SmartInt` will become 
`SmartInt.SmartInt`
       or `SmartIntImpl`. This is confusing for users who are not 
familiar with
       the details of D's template system, and still annoying for 
the rest of us.

     * Your scheme will also complicate the docs, as it will 
replace the top-level
       entry for `struct SmartInt` with a nested
       `template SmartInt` => `struct SmartInt` entry.
Jun 15 2016
prev sibling next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 00:16:12 UTC, Walter Bright wrote:

 Remove all use of 'you' and 'your' from the documentation.


Done.


 "debuggin" => "debugging"


Done.
Jun 14 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/14/2016 11:16 PM, tsbockman wrote:

 On Wednesday, 15 June 2016 at 00:16:12 UTC, Walter Bright wrote:

 Remove all use of 'you' and 'your' from the documentation.


 Done.


I hope you like the results, and are not doing it just because I asked.
Jun 15 2016
parent tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 07:17:56 UTC, Walter Bright wrote:

 On 6/14/2016 11:16 PM, tsbockman wrote:

 On Wednesday, 15 June 2016 at 00:16:12 UTC, Walter Bright 
 wrote:

 Remove all use of 'you' and 'your' from the documentation.


 Done.


 I hope you like the results, and are not doing it just because 
 I asked.


Definitely the latter.  :-)

I have heard such rules before, and they seem silly to me, 
personally. But, it's not a big deal either way, so I changed it.
Jun 15 2016
prev sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 00:16:12 UTC, Walter Bright wrote:

 -O (DMD) should be a link to the -O flag instructions 
 http://dlang.org/dmd-windows.html#switch-O


Done.


 '--inline' is not a DMD switch


Fixed and linked, like -O.
Jun 14 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/14/2016 11:32 PM, tsbockman wrote:

 On Wednesday, 15 June 2016 at 00:16:12 UTC, Walter Bright wrote:

 -O (DMD) should be a link to the -O flag instructions
 http://dlang.org/dmd-windows.html#switch-O


 Done.


 '--inline' is not a DMD switch


 Fixed and linked, like -O.


Tip 'o the hat.
Jun 15 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
 PR: https://github.com/dlang/phobos/pull/4407
 DUB: http://code.dlang.org/packages/checkedint


Thanks for this work. Documentation can be seen here:

http://dtest.thecybershadow.net/artifact/website-7cc6e938154f90aa49fa6451a85b13e35ab2de99-1cfc1d833df5be7c570307da6f7bf5eb/web/phobos-prerelease/std_experimental_checkedint.html

I think there are a few considerable issues with the proposal, but also 
that all are fixable. Here are a few notes:

* The opening documentation states this proposal is concerned with 
adding checking capabilities to integral types. It is a full-blown 
package with 6 modules totaling 4690 lines as wc counts. (For 
comparison: std.experimental.allocator, which offers many facilities and 
implements a number of difficult structures and algorithms, has 11831 
lines.) That's a high budget and a high imposition on users for adding 
checks to integral types. Such extensive style of coding goes against 
the D style we want to promote. Also it is worrisome that one type 
wasn't enough (in spite of extensive parameterization with policies) and 
two are needed, with subtle differences between them that need to be 
summarized in a table. The budget I'd establish for this is one 
parameterized type in one module of manageable size. Several 
parameterized types would be okay if they characterize distinct 
abstractions and use the same backend. Anything else is an indication of 
a design run awry.

* std.experimental.checkedint says "Normally this module should not be 
imported directly." -> this doesn't bode well. I suspect that's also the 
case for std.experimental.checkedint.flags. Modules should be import units.

* Naming: "Safe" has a different meaning throughout D and Phobos. Using 
it here is liable to confuse.

* Naming: The name of a type ("Int") in a name is often a red flag, and 
indeed here it's redundant. One should be able to say Smart!int, not 
stutter SmartInt!int (or Checked!int or whatever). Also, this suggests 
that other types should be considered, how about Smart!bool and 
Smart!double?

* One of the first things I looked for was establishing bounds for 
numbers, like Smart!(int, 0, 100) for percentage. For all its might, 
this package does not offer this basic facility, and from what I can 
tell does not allow users to enforce it via policies.

* Naming: I have no idea what "N bscal;" means.

* Documentation suddenly mentions isFixedPoint without having discussed 
it. Does this package do something about fixed-point numbers?

* Not sure about the value of wrapping bsf, bsr, ilogb and probably 
others. Unlike overflow etc,. calling them with the appropriate values 
is trivial on the user side. A completeness argument could be made but 
all of these wrappers are weight of little value.

* Typos: "equiavlents"

* Documentation uses "you" in several places, colloquialism that should 
be avoided.

* The idea of a loose federation of smart operators that sanitize the 
result of usual arithmetic operators is valuable. The idea of using 
different imports for the same name depending on design is clever, but 
can be simplified. Define functions such as enforcedOp, assertedOp (you 
don't need "unary" and "binary", they can be overloads), and then:

import std.checkedint : smartOp = enforcedOp;
...
auto x = smartOp!"+"(a, b);

* Not sure why divPow2 is needed, why not some enforcedOp!"<<" etc. Same 
about pow, why not enforcedOp!"^^"?

* Getting to the design: the root of the problem is a byzantine design 
that is closed to extension. The current definition goes:

struct SafeInt(N, IntFlagPolicy _policy, Flag!"bitOps" bitOps = Yes.bitOps);

Looking at the IntFlagPolicy, it offers three canned behavior: throws, 
asserts, and noex. Users cannot customize behavior and there is no 
information passed into the policy (e.g. the operands in case of 
overflow, or the numerator in case of division by zero). Passing the 
appropriate information would make it possible to implement various 
policies, e.g. the policy cannot say "I'm actually okay with this" or 
"here I'll implement a saturation policy".

A better design would use the policy as a last-resort hook, allowing it 
to substitute its own answer for the result if it can't be computed 
safely. For example, consider addition:

     bool overflow;
     auto result = myAdd(payload, rhs.payload, overflow);
     if (!overflow) return result;
     return onOverflow(payload, rhs.payload);

The onOverflow function is a policy, which allows user code to exactly 
define what should happen (including all current policies). The same 
goes about all other hooks - multiplication, division, power etc. Each 
should be configurable.

* I see little value in free functions such as e.g. abs() because they 
are trivial one-liners. I understand the need for completeness, but it 
seems a good aspiration for consistency is being marred by a bunch of 
code pulp that really does nothing interesting. Probably not worth it.

===

This suggests a much simpler design that uses DbI to choose behaviors in 
a flexible manner. The definition would go:

struct Checked(T, Hook, T min = T.min, T max = T.max) if (...)
{
     // state {
     static if (stateSize!Hook > 0) private Hook hook;
     else private alias hook = Hook;
     private T payload;
     // }
     ...
}

The struct Hook may have no member at all, in which case Checked!T will 
be as close to plain T as possible. Things become interesting when Hook 
defines optional methods that can be detected by introspection, such as 
onOverflow, onDiv0, onOutOfRangeCmp, etc. - essentially all hooks that 
may be ever needed. Whenever an operation cannot complete with the right 
behavior, the appropriate hook is statically introspected; if 
nonexistent, fine, return with the builtin behavior. If the hook does 
exist, pass responsibility to the hook along with enough information to 
actually let the hook supplant the computation.

The hook may have state (e.g. hysteresis, NaN flag, error state, etc) so 
that's why it may be embedded within Checked.

The only remaining matter is to implement a few preexisting policies 
(Hook implementations) to implement typical choices (such as the ones 
present today), and the core algorithms for doing bounded operations. 
The most interesting algorithms are for computing the bounds of 
operations such as |, &, and ^. The D compiler needs those as well, and 
currently implements them incorrectly. I have these in my mind and I can 
help with those.


Thanks,

Andrei
Jun 15 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/15/2016 9:40 AM, Andrei Alexandrescu wrote:

 Looking at the IntFlagPolicy, it offers three canned behavior: throws, asserts,
 and noex. Users cannot customize behavior and there is no information passed
 into the policy (e.g. the operands in case of overflow, or the numerator in
case
 of division by zero). Passing the appropriate information would make it
possible
 to implement various policies, e.g. the policy cannot say "I'm actually okay
 with this" or "here I'll implement a saturation policy".


Just a note to add to these excellent comments. One style of integer arithmetic 
is "saturation" arithmetic, whereby if it overflows, instead of an error it
gets 
"stuck" at T.max. Same for underflow.

   https://en.wikipedia.org/wiki/Saturation_arithmetic

A good test for a policy API design is if saturation behavior could be done
with 
a user-defined policy, instead of it being a pre-defined one.
Jun 15 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 06/15/2016 02:26 PM, Walter Bright wrote:

 On 6/15/2016 9:40 AM, Andrei Alexandrescu wrote:

 Looking at the IntFlagPolicy, it offers three canned behavior: throws,
 asserts,
 and noex. Users cannot customize behavior and there is no information
 passed
 into the policy (e.g. the operands in case of overflow, or the
 numerator in case
 of division by zero). Passing the appropriate information would make
 it possible
 to implement various policies, e.g. the policy cannot say "I'm
 actually okay
 with this" or "here I'll implement a saturation policy".


 Just a note to add to these excellent comments. One style of integer
 arithmetic is "saturation" arithmetic, whereby if it overflows, instead
 of an error it gets "stuck" at T.max. Same for underflow.

    https://en.wikipedia.org/wiki/Saturation_arithmetic

 A good test for a policy API design is if saturation behavior could be
 done with a user-defined policy, instead of it being a pre-defined one.


Also, the proposed design mentions "hysteresis" and allows Hook to 
introduce state. -- Andrei
Jun 15 2016
prev sibling next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 16:40:19 UTC, Andrei Alexandrescu 
wrote:

 Thanks for this work.
 [...]
 I think there are a few considerable issues with the proposal, 
 but also that all are fixable.


Your message was very long, so for the moment I'm going to filter 
it down to just the high-level design criticism. (The rest is 
unimportant unless/until we reach consensus on the design, 
anyway.)


 * The opening documentation states this proposal is concerned 
 with adding checking capabilities to integral types. It is a 
 full-blown package with 6 modules totaling 4690 lines as wc 
 counts. (For comparison: std.experimental.allocator, which 
 offers many facilities and implements a number of difficult 
 structures and algorithms, has 11831 lines.) That's a high 
 budget and a high imposition on users for adding checks to 
 integral types. Such extensive style of coding goes against the 
 D style we want to promote.
 [...]
 The budget I'd  establish for this is one parameterized type in
 one module of manageable size. Several parameterized types would
 be okay if they characterize distinct abstractions and use the 
 same backend. Anything else is an indication of a design run 
 awry.


This can be summarized as, "It's too big and complicated."

`checkedint` as it stands is, I believe, fairly close to the 
smallest implementation possible in D today, within the 
constraints of the features demanded by the community in past 
discussions, coupled with my high-level design goals.

If you want something shorter or simpler, you will have to cut 
features or compromise the design in other ways. (Or improve the 
D language to facilitate a more elegant design.)

Some features and design goals that combine to motivate my design:

    1) Checked types must signal an error (somehow) whenever their 
behaviour
       deviates from that of an ideal mathematical integer.

    2) It should be possible to recover from errors - using 
`assert(false)` or
       a deliberate divide-by-zero to crash the program is bad 
design unless
       the condition that triggers it is never supposed to happen, 
ever.

    3) Performance (with respect to both speed and memory use) 
should be as
       close as possible to that of the built-in machine integer 
types.

    4) The API should minimize verbosity and ceremony, because 
otherwise hardly
       anyone will use it - people generally prefer convenience 
over safety.

    5) Writing generic code that works correctly with both checked 
and unchecked
       types must be easy.

    6) The API must make safe usage easy, and (accidental) unsafe 
usage hard,
       because people generally don't pay much attention to the 
docs (even if
       they're good). A false sense of security is worse than none 
at all.

    7) The API must be usable in `nothrow  nogc` code.

    8) The number of distinct template instantiations generated in 
natural use
       must be finite, to prevent excessive combinatorial 
explosion when
       checked types are used in public APIs. (Templates that are 
just aliases,
       and small functions that always inline don't count against 
this.)


 Also it is worrisome that one type wasn't enough (in spite of 
 extensive
 parameterization with policies) and two are needed, with subtle 
 differences
 between them that need to be summarized in a table.


The reason for the `SmartInt` versus `SafeInt` split is that with 
D's current semantics
(4) and (5) conflict.

`SmartInt` prioritizes (4); `SafeInt` and `DebugInt` prioritize 
(5).


 Getting to the design: the root of the problem is a byzantine 
 design that is closed to extension.


The design was closed deliberately because of (8). Template bloat 
is a major concern, even with the current finite design.

I want `checkedint` to be usable in public APIs, and that 
requires some standardization of error handling and base types to 
be enforced upon the users. Otherwise, everyone will choose 
something different and all template instantiations involving 
integer types will become practically single-use.


 Looking at the IntFlagPolicy, it offers three canned behavior: 
 throws, asserts, and noex.


The choice of policies is motivated by the natural 
incompatibility of (2), (4), (6), and (7). I built in enough 
variety to allow people to choose their own priorities among 
those goals, and no more because of (8).


 * One of the first things I looked for was establishing bounds 
 for numbers, like Smart!(int, 0, 100) for percentage. For all 
 its might, this package does not offer this basic facility, and 
 from what I can tell does not allow users to enforce it via 
 policies.


Here you are suggesting adding even more complexity to a design 
that you have already de-facto rejected as overly complex. As 
discussed earlier in this very thread, I studied adding support 
for arbitrary bounds and decided not to pursue that right now 
because implementing it in a performant way would greatly 
increase the complexity of `checkedint`, and make the template 
bloat problem much worse.


 Also, this suggests that other types should be considered, how 
 about Smart!bool and Smart!double?


Neither `bool` nor `double` has the kind of severe-but-fixable 
safety and correctness issues that the machine integer types do, 
which motivates the `checkedint` design. No doubt someone can 
make up some sort of meaning to attach to those symbols, but it 
will most likely have nothing to do with `SmartInt`.


 * Not sure why divPow2 is needed, why not some enforcedOp!"<<"


Because (although similar) a bit shift is actually semantically 
different than dividing or multiplying by a power of two. The bit 
shift implies different rules for rounding and overflow.

I realized in testing that even for `SmartInt`, the bit shift 
semantics are still useful sometimes, and decided that it was 
better not to confuse the two. A `smartOp` version of the bit 
shifts is necessary because the built-in bit shifts have some 
undefined behaviour that needs to be fixed.


 etc. Same about pow, why not enforcedOp!"^^"?


`pow()` exists as a free function to satisfy (5), and because the 
`^^` and `^^=` operators both have language-level bugs that 
currently make their use incompatible with (6).


 I see little value in free functions such as e.g. abs() because 
 they are trivial one-liners. I understand the need for 
 completeness, but it seems a good aspiration for consistency is 
 being marred by a bunch of code pulp that really does nothing 
 interesting. Probably not worth it.


`checkedint`-aware versions of functions like `abs()` are 
necessary to satisfy (4) and (6) together.


 The hook may have state (e.g. hysteresis, NaN flag, error 
 state, etc) so that's why it may be embedded within Checked.


The early iterations of `checkedint` worked this way (although I 
had no plans to support user-defined hooks). I implemented and 
debugged it, and thought it was about ready to submit many months 
ago.

Then I actually tried *using* it, and hated it. The problem with 
using a NaN state, is that in nothrow code you have to manually 
check it before *every* call to an external 
(non-`checkedint`-aware) function, or you may accidentally lose 
it.

As a result, safe NaN-based APIs violate (4), while concise APIs 
violate (6). The IEEE-inspired sticky flags feature is my 
solution to this problem, and it is far more pleasant to work 
with in practice - as well as faster and more memory efficient.

Meanwhile, in code that allows exceptions, there is no reason to 
pay the speed and memory penalty of carting around the NaN state 
to check later - just throw on the spot whenever something goes 
wrong.
Jun 15 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 18:34:15 UTC, tsbockman wrote:

 On Wednesday, 15 June 2016 at 16:40:19 UTC, Andrei Alexandrescu 
 wrote:

 Getting to the design: the root of the problem is a byzantine 
 design that is closed to extension.


 The design was closed deliberately because of (8). Template 
 bloat is a major concern, even with the current finite design.

 I want `checkedint` to be usable in public APIs, and that 
 requires some standardization of error handling and base types 
 to be enforced upon the users. Otherwise, everyone will choose 
 something different and all template instantiations involving 
 integer types will become practically single-use.


 Looking at the IntFlagPolicy, it offers three canned behavior: 
 throws, asserts, and noex.


 The choice of policies is motivated by the natural 
 incompatibility of (2), (4), (6), and (7). I built in enough 
 variety to allow people to choose their own priorities among 
 those goals, and no more because of (8).


Re-reading what I wrote there, I makes it sound like the closed 
design was motivated solely by template bloat concerns. But, 
that's not true either.

Standardizing the error handling methods is also important for 
other interoperability-related reasons:

     * If every third-party library designs a different error 
handling method,
       people writing applications that depend on many libraries 
will have to
       study the `checkedint` error signaling of each one, and 
make sure that
       their code which interacts with each library knows how to 
detect and
       respond to its particular signaling mechanism.

       My design for `checkedint` reduces the number of 
`checkedint`-specific
       signaling methods that need to be studied down to one (the 
sticky flags
       policy). The other two policies are just using D's standard 
error-
       handling facilities, and don't require any user 
intervention anyway,
       unless you want to catch the exception for some reason.

       Even the sticky flags policy is only of concern for 
`nothrow  nogc` APIs,
       which I suspect in practice will mostly mean game libraries.

     * The relative simplicity of my policy set has allowed me to 
arrange them
       into a strict linear hierarchy, so that the results of 
mixing policies
       is predictable and safe.
Jun 15 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 06/15/2016 07:13 PM, tsbockman wrote:

 Standardizing the error handling methods is also important for other
 interoperability-related reasons:

      * If every third-party library designs a different error handling
 method,
        people writing applications that depend on many libraries will
 have to
        study the `checkedint` error signaling of each one, and make sure
 that
        their code which interacts with each library knows how to detect and
        respond to its particular signaling mechanism.

        My design for `checkedint` reduces the number of
 `checkedint`-specific
        signaling methods that need to be studied down to one (the sticky
 flags
        policy). The other two policies are just using D's standard error-
        handling facilities, and don't require any user intervention anyway,
        unless you want to catch the exception for some reason.

        Even the sticky flags policy is only of concern for `nothrow
  nogc` APIs,
        which I suspect in practice will mostly mean game libraries.

      * The relative simplicity of my policy set has allowed me to
 arrange them
        into a strict linear hierarchy, so that the results of mixing
 policies
        is predictable and safe.


I don't agree with this. On the face of it, three built-in policies 
solve the matter of interoperability: if you want interop, use one of 
those. Done. But interop is not a reason to preclude any flexibility. -- 
Andrei
Jun 15 2016
parent tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 16 June 2016 at 00:06:13 UTC, Andrei Alexandrescu 
wrote:

 On 06/15/2016 07:13 PM, tsbockman wrote:

 Standardizing the error handling methods is also important for 
 other interoperability-related reasons:
 [...]


 I don't agree with this. On the face of it, three built-in 
 policies solve the matter of interoperability: if you want 
 interop, use one of those. Done. But interop is not a reason to 
 preclude any flexibility. -- Andrei


Well, I'm satisfied with my design in this respect, and 
completely lack motivation, time, emotional energy, etc. to 
rewrite `checkedint` again from the ground up to properly account 
for arbitrary pluggable error handling policies.

So, the community will have to decide how important this is, and 
find someone else to take over the project if you all demand a 
rewrite.
Jun 15 2016
prev sibling next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 16:40:19 UTC, Andrei Alexandrescu 
wrote:

 I think there are a few considerable issues with the proposal, 
 but also that all are fixable.


I already sent a much longer message detailing some of the 
reasons why I believe my design is sensible. But, before we 
continue this discussion much further, I need to stop and make 
one thing clear:

What you are proposing is *not* "fixing" my design - it is 
basically scrapping it and replacing it with a ground-up rewrite, 
with perhaps some bits and pieces and general inspiration taken 
from my work.

I'm OK with the community rejecting my design if that's what 
people really want to do, but I will not be implementing your 
proposal myself. I will simply leave `checkedint` on DUB, and 
move on with my life.

So, I'm putting out a general call now for the community to 
download the DUB package, try actually *using* it for something, 
and speak up as to whether the design seems good, or not.

If the decision is made to accept the high-level design, then we 
can go back to bikeshedding about names, fixing typos, 
tweaking/trimming the API, etc.
Jun 15 2016
next sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 18:50:32 UTC, tsbockman wrote:

 What you are proposing is *not* "fixing" my design - it is 
 basically scrapping it and replacing it with a ground-up 
 rewrite, with perhaps some bits and pieces and general 
 inspiration taken from my work.
 [...]
 If the decision is made to accept the high-level design, then 
 we can go back to bikeshedding about names, fixing typos, 
 tweaking/trimming the API, etc.


I should mention that one possible "trimming" would be to remove 
either `SmartInt`/`smartOp` or `SafeInt`/`safeOp`. I don't *want* 
to do this, obviously, but I wouldn't categorize it as "scrapping 
the design", either.
Jun 15 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 06/15/2016 02:50 PM, tsbockman wrote:

 On Wednesday, 15 June 2016 at 16:40:19 UTC, Andrei Alexandrescu wrote:

 I think there are a few considerable issues with the proposal, but
 also that all are fixable.


 I already sent a much longer message detailing some of the reasons why I
 believe my design is sensible. But, before we continue this discussion
 much further, I need to stop and make one thing clear:

 What you are proposing is *not* "fixing" my design - it is basically
 scrapping it and replacing it with a ground-up rewrite, with perhaps
 some bits and pieces and general inspiration taken from my work.


I'd say that's a fair assessment on the face of it. But the pieces are 
there and the grit and expertise to assemble them are already there. The 
real question is what's the "right" design. Ideally we should optimize 
for that.


 I'm OK with the community rejecting my design if that's what people
 really want to do, but I will not be implementing your proposal myself.
 I will simply leave `checkedint` on DUB, and move on with my life.


That is sensible, and a course of action I thought would transpire. But 
I suggest you to reconsider. Your PR is a solid implementation of the 
wrong idea. Before anyone gets offended, let me add that I think there's 
a lot of good in that. For this kind of work (not algorithm-intensive) 
ideas are cheap; the solid implementation matters a lot more than the 
idea - the same expertise can be put to use on another idea.

I tried to offer a careful review by a competent peer, with an eye for 
taking an okay design toward a great design. The kind of review I wish 
to get now as much as at any point of my career. They are difficult to 
receive but often have a lot of potential in them.


 So, I'm putting out a general call now for the community to download the
 DUB package, try actually *using* it for something, and speak up as to
 whether the design seems good, or not.

 If the decision is made to accept the high-level design, then we can go
 back to bikeshedding about names, fixing typos, tweaking/trimming the
 API, etc.


I don't think checkedint in a form close to its current one is for 
Phobos. I would argue it's not for D. Starting with the problem 
statement "we want to check integral operations for overflow and other 
surprising behaviors" and ending with 4.5 KLOC defining a bunch of names 
in 6 modules makes it highly suspicious there is an overgrown underbelly 
somewhere. The language features are not orchestrated properly. Even if 
it were the case that there's no smaller design that conforms with the 
requirements, that means requirements have a problem.


Andrei
Jun 15 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 16 June 2016 at 00:31:38 UTC, Andrei Alexandrescu 
wrote:

 On 06/15/2016 02:50 PM, tsbockman wrote:

 What you are proposing is *not* "fixing" my design - it is 
 basically
 scrapping it and replacing it with a ground-up rewrite, with 
 perhaps
 some bits and pieces and general inspiration taken from my 
 work.


 I'd say that's a fair assessment on the face of it. But the 
 pieces are there and the grit and expertise to assemble them 
 are already there. The real question is what's the "right" 
 design. Ideally we should optimize for that.


I set out to solve a specific problem with `checkedint`, and I 
believe my design succeeds in the areas that it is intended to, 
to the extent that is reasonably possible with D today.

In order to convince me to rewrite this, you would need to 
convince me that *my* goals can be met significantly better by a 
different library design. So far, I see no evidence of this.

You are of course free to set different goals than I have, but 
from my perspective this is *major* scope creep, and not what I 
volunteered for.


 I'm OK with the community rejecting my design if that's what 
 people
 really want to do, but I will not be implementing your 
 proposal myself.
 I will simply leave `checkedint` on DUB, and move on with my 
 life.


 That is sensible, and a course of action I thought would 
 transpire. But I suggest you to reconsider. Your PR is a solid 
 implementation of the wrong idea. Before anyone gets offended, 
 let me add that I think there's a lot of good in that.


There is only one thing about all this that really offends me:

`checkedint` (and  burner's `SafeInt` before it) have been under 
development in the open for over a year now. There have been 
several discussions in the forums, with feedback being actively 
solicited. Significant design changes were made to address 
various people's needs. `SafeInt` was an open pull request for 
many months with 100+ comments accumulating in that time.

Why didn't you make your design requirements known at any earlier 
point in this process? If you are ultimate gate keeper for Phobos 
(as you seem to be), you ought to make your requirements known 
*before* the implementation is finished.

Even so, I'm not particularly upset. I've learned a great deal 
through this project, and `checkedint` will still be available 
for anyone who wants to use it through DUB.

The code is Boost Licensed, so anyone who wants to is free to 
fork it and try to fulfill your vision.


 For this kind of work (not algorithm-intensive) ideas are 
 cheap; the solid implementation matters a lot more than the 
 idea - the same expertise can be put to use on another idea.

 I tried to offer a careful review by a competent peer, with an 
 eye for taking an okay design toward a great design. The kind 
 of review I wish to get now as much as at any point of my 
 career. They are difficult to receive but often have a lot of 
 potential in them.


I don't mind a scathing review, so long as it has been properly 
researched and thought through. I do object to being criticized 
for not including features that no one asked me for until now, 
and which have little to do with the original purpose of the 
project, though.


 So, I'm putting out a general call now for the community to 
 download the
 DUB package, try actually *using* it for something, and speak 
 up as to
 whether the design seems good, or not.

 If the decision is made to accept the high-level design, then 
 we can go
 back to bikeshedding about names, fixing typos, 
 tweaking/trimming the
 API, etc.


 I don't think checkedint in a form close to its current one is 
 for Phobos. I would argue it's not for D. Starting with the 
 problem statement "we want to check integral operations for 
 overflow and other surprising behaviors" and ending with 4.5 
 KLOC defining a bunch of names in 6 modules makes it highly 
 suspicious there is an overgrown underbelly somewhere. The 
 language features are not orchestrated properly. Even if it 
 were the case that there's no smaller design that conforms with 
 the requirements, that means requirements have a problem.


 Andrei


I believe a fundamental part of the problem, is that something 
like `SmartInt` should be built in to the language as the default 
integer type, not tacked on as a library. (Stuff like bound types 
would still be implemented in the library.)

However, my understanding is that this approach was reject by 
Walter Bright about two years ago (and obviously actually making 
it default would involve unacceptable breakage at this point). 
So, I tried to work with what we have.
Jun 15 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/15/16 9:34 PM, tsbockman wrote:

 `checkedint` (and  burner's `SafeInt` before it) have been under
 development in the open for over a year now. There have been several
 discussions in the forums, with feedback being actively solicited.
 Significant design changes were made to address various people's needs.
 `SafeInt` was an open pull request for many months with 100+ comments
 accumulating in that time.

 Why didn't you make your design requirements known at any earlier point
 in this process? If you are ultimate gate keeper for Phobos (as you seem
 to be), you ought to make your requirements known *before* the
 implementation is finished.


Apologies about that. I've done a bit of spelunking to see what 
happened. Indeed the first reference to SafeInt is on a forum post on 
6/7/2015, followed immediately by 
https://github.com/dlang/phobos/pull/3389 which entailed a long discussion.

You first posted about checkedint here on 6/30/2015, in a large thread.

At that time, I had the std.allocator review going on (started on 
6/11/2015), a newborn baby, and a move across the continent to worry 
about (which happened at the end of June). It is entirely possible I 
just missed that discussion, or more likely saw it and had no meaningful 
input at the time. There has been a gap in forum posts with "checkedint" 
in the title between 7/3/2015 and 6/7/2016, so it's not like there was a 
continuing presence I was working hard to ignore. I honestly think 
there's nothing to be offended over.

This underlies a larger issue. There must be a protocol that guarantees 
a proposal is brought to consideration to the D leadership. Dicebot is 
leading such an initiative (which can be seen as a revamping of DIPs) 
and we hope to get it finalized soon.


Andrei
Jun 15 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 16 June 2016 at 02:53:38 UTC, Andrei Alexandrescu 
wrote:

 On 6/15/16 9:34 PM, tsbockman wrote:

 Why didn't you make your design requirements known at any 
 earlier point
 in this process? If you are ultimate gate keeper for Phobos 
 (as you seem
 to be), you ought to make your requirements known *before* the
 implementation is finished.


 Apologies about that. I've done a bit of spelunking to see what 
 happened. Indeed the first reference to SafeInt is on a forum 
 post on 6/7/2015, followed immediately by 
 https://github.com/dlang/phobos/pull/3389 which entailed a long 
 discussion.

 You first posted about checkedint here on 6/30/2015, in a large 
 thread.

 At that time, I had the std.allocator review going on (started 
 on 6/11/2015), a newborn baby, and a move across the continent 
 to worry about (which happened at the end of June). It is 
 entirely possible I just missed that discussion, or more likely 
 saw it and had no meaningful input at the time. There has been 
 a gap in forum posts with "checkedint" in the title between 
 7/3/2015 and 6/7/2016,


Numerous other mentions were made of this project in various 
contexts on the forums, in GitHub pull requests, and on the bug 
tracker - including discussions in which you participated. 'posts 
with "checkedint" in the title' is too narrow of a search filter.


 so it's not like there was a continuing presence I was working 
 hard to ignore. I honestly think there's nothing to be offended 
 over.


Malicious intent is not required to make the act offensive; 
you're still jumping into a project a year in the making and 
demanding that I choose between investing an additional six 
months (wild guess) of my time working on things I don't care 
about (at best), or canceling the project (which has otherwise 
received generally positive feedback so far).

I am not too upset mostly because I had a variety of reasons for 
pursuing this, not all of which depend on getting it into Phobos.


 This underlies a larger issue. There must be a protocol that 
 guarantees a proposal is brought to consideration to the D 
 leadership. Dicebot is leading such an initiative (which can be 
 seen as a revamping of DIPs) and we hope to get it finalized 
 soon.


 Andrei


That is part of the problem, but this is also a fine example of a 
broader pattern that I have noticed in D's review process:

Pull requests are routinely reviewed in an upside-down fashion:

1) Formatting
2) Typos
3) Names
4) Tests (and names again)
6) Docs (and names)
8) Design (and more about names)
9) Does this even belong in Phobos?

I don't think people are doing it on purpose - it's just easier 
to start with the trivial nit-picks, because you don't need a 
deep understanding of the code and the problem domain (or 
decision-making authority) to complain about a missing ' ' or 
something.

But, that doesn't change the fact that the process still feels 
almost perfectly designed to waste contributors' time.

Unless the PR is a complete mess, (9) and (8) should be debated 
*first*, before worrying about any of the other stuff. Why waste 
people's time fixing trivialities, if it's all going to just be 
deleted or rewritten anyway?
Jun 15 2016
next sibling parent reply lobo <swamplobo gmail.com> writes:
On Thursday, 16 June 2016 at 03:56:02 UTC, tsbockman wrote:

 On Thursday, 16 June 2016 at 02:53:38 UTC, Andrei Alexandrescu 
 wrote:

 [...]


 Numerous other mentions were made of this project in various 
 contexts on the forums, in GitHub pull requests, and on the bug 
 tracker - including discussions in which you participated. 
 'posts with "checkedint" in the title' is too narrow of a 
 search filter.


 [...]


 Malicious intent is not required to make the act offensive; 
 you're still jumping into a project a year in the making and 
 demanding that I choose between investing an additional six 
 months (wild guess) of my time working on things I don't care 
 about (at best), or canceling the project (which has otherwise 
 received generally positive feedback so far).

 I am not too upset mostly because I had a variety of reasons 
 for pursuing this, not all of which depend on getting it into 
 Phobos.


 [...]


 That is part of the problem, but this is also a fine example of 
 a broader pattern that I have noticed in D's review process:

 Pull requests are routinely reviewed in an upside-down fashion:

 1) Formatting
 2) Typos
 3) Names
 4) Tests (and names again)
 6) Docs (and names)
 8) Design (and more about names)
 9) Does this even belong in Phobos?

 I don't think people are doing it on purpose - it's just easier 
 to start with the trivial nit-picks, because you don't need a 
 deep understanding of the code and the problem domain (or 
 decision-making authority) to complain about a missing ' ' or 
 something.

 But, that doesn't change the fact that the process still feels 
 almost perfectly designed to waste contributors' time.

 Unless the PR is a complete mess, (9) and (8) should be debated 
 *first*, before worrying about any of the other stuff. Why 
 waste people's time fixing trivialities, if it's all going to 
 just be deleted or rewritten anyway?




Has this work/design been submitted as a DIP? I cannot find it.

I thought all Phobos additions of any magnitude were required to 
pass the DIP submission first in order to avoid this sort of 
situation. If there is a DIP that was accepted then to have it 
knocked back now would suck.

Without a DIP you have to expect the design could be turned down 
by any core developer when they first get the opportunity to 
review it, no matter how long after the work was initiated.

bye,
lobo
Jun 15 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/16/16 2:55 AM, lobo wrote:

 Without a DIP you have to expect the design could be turned down by any
 core developer when they first get the opportunity to review it, no
 matter how long after the work was initiated.


We're still working on making DIPs a process that requires leadership to 
intervene, but I'd say this is already true now.

Per the title of this thread: "std.experimental.checkedint is ready for 
comments!". What kind of comments were expected?


Andrei
Jun 16 2016
prev sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 16 June 2016 at 06:55:00 UTC, lobo wrote:

 Has this work/design been submitted as a DIP? I cannot find it.

 I thought all Phobos additions of any magnitude were required 
 to pass the DIP submission first in order to avoid this sort of 
 situation. If there is a DIP that was accepted then to have it 
 knocked back now would suck.

 Without a DIP you have to expect the design could be turned 
 down by any core developer when they first get the opportunity 
 to review it, no matter how long after the work was initiated.

 bye,
 lobo


My observation has been that the DIP process is not followed, in 
practice. The DIP for this project (which I did not start; I 
merely continued  burner's work) is actually the same one as that 
of the recently accepted `ndslice`: https://wiki.dlang.org/DIP80
Jun 16 2016
prev sibling next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Thursday, 16 June 2016 at 03:56:02 UTC, tsbockman wrote:

 That is part of the problem, but this is also a fine example of 
 a broader pattern that I have noticed in D's review process:

 Pull requests are routinely reviewed in an upside-down fashion:

 1) Formatting
 2) Typos
 3) Names
 4) Tests (and names again)
 6) Docs (and names)
 8) Design (and more about names)
 9) Does this even belong in Phobos?

 I don't think people are doing it on purpose - it's just easier 
 to start with the trivial nit-picks, because you don't need a 
 deep understanding of the code and the problem domain (or 
 decision-making authority) to complain about a missing ' ' or 
 something.

 But, that doesn't change the fact that the process still feels 
 almost perfectly designed to waste contributors' time.

 Unless the PR is a complete mess, (9) and (8) should be debated 
 *first*, before worrying about any of the other stuff. Why 
 waste people's time fixing trivialities, if it's all going to 
 just be deleted or rewritten anyway?


I think anything sufficiently large is likely to be reviewed in 
that order. In a lot of cases all the work for 1 - 8 is 
progressively done while working out 9. Should people not mention 
the smaller mistakes / disagreements they find along the way 
until they've reached the end and can provide a final judgement?
Jun 16 2016
parent tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 16 June 2016 at 07:02:21 UTC, John Colvin wrote:

 I think anything sufficiently large is likely to be reviewed in 
 that order. In a lot of cases all the work for 1 - 8 is 
 progressively done while working out 9. Should people not 
 mention the smaller mistakes / disagreements they find along 
 the way until they've reached the end and can provide a final 
 judgement?


I think that consideration should be given to splitting reviews 
into two phases by policy:

     1. The big picture: refining the overall design, and debating
        whether the change is worthwhile or not. This ends when the
        change has been formally approved by someone who has the
        authority to do so.

     2. Completing and polishing the implementation, until it is
        actually ready to merge.

Distinguish clearly between these phases, and make it clear to 
submitters that they are not required or expected to fix/finish 
all the little stuff until (1) is over, since there's a good 
chance it will all be irrelevant, anyway.

Obviously there will be some fuzziness as to whether an issue 
belongs in (1) or (2), but there's lots and lots of stuff that 
clearly falls into one or the other.

One of the things that such a policy would accomplish is to 
highlight the essential (but often ignored) question, "Who 
actually has authority to approve this?"
Jun 16 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/15/16 11:56 PM, tsbockman wrote:

 Numerous other mentions were made of this project in various contexts on
 the forums, in GitHub pull requests, and on the bug tracker - including
 discussions in which you participated. 'posts with "checkedint" in the
 title' is too narrow of a search filter.


I am sure there were, which was especially visible to you because you 
were following the project. Some examples would be helpful so I can 
learn from them.


 so it's not like there was a continuing presence I was working hard to
 ignore. I honestly think there's nothing to be offended over.


 Malicious intent is not required to make the act offensive; you're still
 jumping into a project a year in the making and demanding that I choose
 between investing an additional six months (wild guess) of my time
 working on things I don't care about (at best), or canceling the project
 (which has otherwise received generally positive feedback so far).


Agreed malice is not required. But I'm still having trouble seeing the 
offense. Annoyance at a negative review, sure, we're all human. But 
taking offense? The closest anything came to "demanding" anything has been:


 This suggests a much simpler design [...]
 But I suggest you to reconsider.


How could I have phrased my review and follow-up in ways that are not 
offensive? Should I have just accepted the proposal on grounds that a 
lot of work has been put into it and the deadline has passed for 
influencing it? (Non-rhetorical questions.)


 Pull requests are routinely reviewed in an upside-down fashion:

 1) Formatting
 2) Typos
 3) Names
 4) Tests (and names again)
 6) Docs (and names)
 8) Design (and more about names)
 9) Does this even belong in Phobos?

 I don't think people are doing it on purpose - it's just easier to start
 with the trivial nit-picks, because you don't need a deep understanding
 of the code and the problem domain (or decision-making authority) to
 complain about a missing ' ' or something.


I can see how that could be happening. Often (and in this case) there 
are different folks touching on the different points.


Andrei
Jun 16 2016
parent tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 16 June 2016 at 12:25:39 UTC, Andrei Alexandrescu 
wrote:

 Pull requests are routinely reviewed in an upside-down fashion:

 1) Formatting
 2) Typos
 3) Names
 4) Tests (and names again)
 6) Docs (and names)
 8) Design (and more about names)
 9) Does this even belong in Phobos?

 I don't think people are doing it on purpose - it's just 
 easier to start
 with the trivial nit-picks, because you don't need a deep 
 understanding
 of the code and the problem domain (or decision-making 
 authority) to
 complain about a missing ' ' or something.


 I can see how that could be happening. Often (and in this case) 
 there are different folks touching on the different points.


Yes, it's mostly a process issue, not an individual one. See my 
earlier reply to John Colvin, for a practical suggestion as to 
how to improve with this.
Jun 16 2016
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/15/16 11:56 PM, tsbockman wrote:

 On Thursday, 16 June 2016 at 02:53:38 UTC, Andrei Alexandrescu wrote:

 On 6/15/16 9:34 PM, tsbockman wrote:

 Why didn't you make your design requirements known at any earlier point
 in this process? If you are ultimate gate keeper for Phobos (as you seem
 to be), you ought to make your requirements known *before* the
 implementation is finished.


 Apologies about that. I've done a bit of spelunking to see what
 happened. Indeed the first reference to SafeInt is on a forum post on
 6/7/2015, followed immediately by
 https://github.com/dlang/phobos/pull/3389 which entailed a long
 discussion.

 You first posted about checkedint here on 6/30/2015, in a large thread.

 At that time, I had the std.allocator review going on (started on
 6/11/2015), a newborn baby, and a move across the continent to worry
 about (which happened at the end of June). It is entirely possible I
 just missed that discussion, or more likely saw it and had no
 meaningful input at the time. There has been a gap in forum posts with
 "checkedint" in the title between 7/3/2015 and 6/7/2016,


 Numerous other mentions were made of this project in various contexts on
 the forums, in GitHub pull requests, and on the bug tracker - including
 discussions in which you participated. 'posts with "checkedint" in the
 title' is too narrow of a search filter.


 so it's not like there was a continuing presence I was working hard to
 ignore. I honestly think there's nothing to be offended over.


 Malicious intent is not required to make the act offensive; you're still
 jumping into a project a year in the making and demanding that I choose
 between investing an additional six months (wild guess) of my time
 working on things I don't care about (at best), or canceling the project
 (which has otherwise received generally positive feedback so far).

 I am not too upset mostly because I had a variety of reasons for
 pursuing this, not all of which depend on getting it into Phobos.


 This underlies a larger issue. There must be a protocol that
 guarantees a proposal is brought to consideration to the D leadership.
 Dicebot is leading such an initiative (which can be seen as a
 revamping of DIPs) and we hope to get it finalized soon.


 Andrei


 That is part of the problem, but this is also a fine example of a
 broader pattern that I have noticed in D's review process:

 Pull requests are routinely reviewed in an upside-down fashion:

 1) Formatting
 2) Typos
 3) Names
 4) Tests (and names again)
 6) Docs (and names)
 8) Design (and more about names)
 9) Does this even belong in Phobos?


This is a good point. I personally have avoided doing anything in 1-8 
unless I first agree it's a change I think we should accept. Which then 
results in 0) nobody is looking at this!???

I will occasionally chime in when there is a particular pain point with 
design philosophy, but the problem is, this gives the impression that I 
am willing to accept the change after X nitpicks are done. I try not to 
do this unless there is another reviewer that seems motivated to do a 
full review who has already posted, and they haven't covered that piece.

On the other hand, peoples minds can change.

e.g:

"The early iterations of `checkedint` worked this way (although I had no 
plans to support user-defined hooks). I implemented and debugged it, and 
thought it was about ready to submit many months ago.

Then I actually tried *using* it, and hated it."

And there is a further problem -- Walter and Andrei are gatekeepers, but 
are stretched incredibly thin. So a reviewer that is keen on a certain 
addition may put in a lot of time on steps 1-8 assuming that step 9 is a 
given, and when W/A come around to looking at it, they say no.

How to fix this? I don't know. The only recommendation I have is to do 2 
things:

1. Get pre-approval if you have your heart set on Phobos inclusion. Make 
SURE you understand the expectations from Andrei and Walter.
2. Be willing to put it on code.dlang.org instead, and Phobos as a 
secondary goal (your approach).

I had a similar experience with dcollections, though I never really 
intended on it being in Phobos, it was meant for Tango. But that didn't 
work out either, as the gatekeeper there was working on his own version 
of containers.


 Unless the PR is a complete mess, (9) and (8) should be debated *first*,
 before worrying about any of the other stuff. Why waste people's time
 fixing trivialities, if it's all going to just be deleted or rewritten
 anyway?


I understand your frustration. All I can say is, open source 
contributors have to have a thicker skin (and I'm not saying you don't). 
We are all human and have our faults, and any team in any context can 
have miscommunication, or misunderstandings. I can assure you it's not 
the plan to waste people's time or to cause frustration. Even with a 
mitigating plan in place, these can happen. Have a plan that you can 
control, with a viable path if the things you can't don't go your way.

-Steve
Jun 16 2016
next sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 16 June 2016 at 13:56:26 UTC, Steven Schveighoffer 
wrote:

 And there is a further problem -- Walter and Andrei are 
 gatekeepers, but are stretched incredibly thin.


Having had some time to think about all it now, I believe this 
was my actual problem.

When I started working on this, I was new to the D project and, 
based on following the GitHub discussions and observing the 
voting and review process for `std.logger` and `std.ndslice`, I 
believed that the acceptance process for Phobos was somewhat 
democratic: the more people who approved of a design, the more 
likely it would be accepted (with, of course, substantially more 
weight placed on the opinions of the leadership, who at the time 
I identified more or less as, "those with merge rights").

Having seen the outcome of the recent auto-decoding discussion, 
and the way that this review effectively ended abruptly the 
moment that Andrei Alexandrescu decided that he didn't like my 
design for `checkedint` - despite a number of other people 
earlier speaking positively of it - I now perceive that the 
democratic aspects of the Phobos process are essentially advisory 
in nature, with little-to-no real authority attached to them.

I do not intend this as a criticism, as I am not a stickler for 
democracy. I do think this should be communicated clearly to 
future new contributors, though, to avoid confusion. (Keep in 
mind that I joined the project before the "all new Phobos symbols 
must be approved by Andrei" rule was announced.)

Anyway, I think I see things from Andrei's perspective now, and 
would like to apologize for taking offense, now that I understand 
his role better.

I will, however, be ending my participation in the D development 
process, as I am not personally interested in working in an 
environment where everything I do needs pre-approval from someone 
with his opinions and leadership style (this is not specifically 
about `checkedint`) - especially since he seems too busy to 
effectively manage more contributors using the current system, 
anyway.

I will continue to be available for follow-up on the work I have 
already done, and will try to finish at least some of my current 
pending pull requests. I wish D well, and may still post on the 
forums once in a while.
Jun 16 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/16/2016 6:56 AM, Steven Schveighoffer wrote:

 I understand your frustration. All I can say is, open source contributors have
 to have a thicker skin (and I'm not saying you don't). We are all human and
have
 our faults, and any team in any context can have miscommunication, or
 misunderstandings. I can assure you it's not the plan to waste people's time or
 to cause frustration. Even with a mitigating plan in place, these can happen.
 Have a plan that you can control, with a viable path if the things you can't
 don't go your way.


+1

Also, if it isn't approved for Phobos, that doesn't necessarily mean the work
is 
wasted. Dub is full of add on libraries, add it there. Approval from others is 
not necessary.

My own Sargon is for things I wrote that were not approved for Phobos for one 
reason or another:

https://www.digitalmars.com/sargon/halffloat.html

Work is wasted only when the author gives up on it.
Jun 16 2016
parent Jack Stouffer <jack jackstouffer.com> writes:
On Thursday, 16 June 2016 at 21:52:16 UTC, Walter Bright wrote:

 https://www.digitalmars.com/sargon/halffloat.html


Your HTTPS cert seems to be broken
Jun 16 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/15/2016 8:56 PM, tsbockman wrote:

 Pull requests are routinely reviewed in an upside-down fashion:

 1) Formatting
 2) Typos
 3) Names
 4) Tests (and names again)
 6) Docs (and names)
 8) Design (and more about names)
 9) Does this even belong in Phobos?


It's a consequence of starting to read a document cold, i.e. the first thing 
noticed is the formatting, then the typos, then the names, etc.

A deep understanding of the design comes much later, only after having 
thoroughly read it and hit all the speedbumps of typos, etc.
Jun 16 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 16 June 2016 at 21:02:32 UTC, Walter Bright wrote:

 On 6/15/2016 8:56 PM, tsbockman wrote:

 Pull requests are routinely reviewed in an upside-down fashion:

 1) Formatting
 2) Typos
 3) Names
 4) Tests (and names again)
 6) Docs (and names)
 8) Design (and more about names)
 9) Does this even belong in Phobos?


 It's a consequence of starting to read a document cold, i.e. 
 the first thing noticed is the formatting, then the typos, then 
 the names, etc.

 A deep understanding of the design comes much later, only after 
 having thoroughly read it and hit all the speedbumps of typos, 
 etc.


Yes I understand - it's a very natural thing to do (and I have 
done it myself).

I think my suggestion fundamentally boils down to, "Try to 
communicate to submitters early on who needs to approve their 
change, and that they may delay addressing all the lesser issues 
until after their basic proposal has been (tentatively) approved."

My problem with `checkedint` was that I formed a wrong idea of 
whose approval I needed (interested Phobos devs collectively, 
versus Andrei specifically).
Jun 16 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/16/2016 2:11 PM, tsbockman wrote:

 My problem with `checkedint` was that I formed a wrong idea of whose approval I
 needed (interested Phobos devs collectively, versus Andrei specifically).


Andrei is in charge of the library and has my full support.

We've talked many times about raising the bar much higher on Phobos. It's a 
tough competitive environment for programming languages out there, and we
should 
all expect the review process going forward for Phobos to be brutal.

On the other hand, getting an endorsement from Andrei for a design is something 
anyone should be proud of, and I suspect will be worth the blood, sweat and
tears.

I know my own code has gotten a lot better from Andrei's professional criticism.
Jun 16 2016
next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 16 June 2016 at 22:30:48 UTC, Walter Bright wrote:

 Andrei is in charge of the library and has my full support.

 We've talked many times about raising the bar much higher on 
 Phobos. It's a tough competitive environment for programming 
 languages out there, and we should all expect the review 
 process going forward for Phobos to be brutal.

 On the other hand, getting an endorsement from Andrei for a 
 design is something anyone should be proud of, and I suspect 
 will be worth the blood, sweat and tears.

 I know my own code has gotten a lot better from Andrei's 
 professional criticism.


There are many ways to go about "raising the bar"; making Andrei 
the sole arbiter of a design's quality will not necessarily 
accomplish this, but will make the need for constant 
communication with him a major bottleneck for development, 
regardless. It also risks making him into a single point of 
failure.

I do not think this approach will scale in the long run, no 
matter who is given that role. Of course, I would be happy to be 
proven wrong.
Jun 16 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Thursday, 16 June 2016 at 23:27:06 UTC, tsbockman wrote:

 I do not think this approach will scale in the long run, no 
 matter who is given that role. Of course, I would be happy to 
 be proven wrong.


I think maybe the C++ approach is better. If you want something 
into C++ std your best bet is to try to get it into Boost first.
Jun 16 2016
parent reply deadalnix <deadalnix gmail.com> writes:
On Friday, 17 June 2016 at 06:54:20 UTC, Ola Fosheim Grøstad 
wrote:

 On Thursday, 16 June 2016 at 23:27:06 UTC, tsbockman wrote:

 I do not think this approach will scale in the long run, no 
 matter who is given that role. Of course, I would be happy to 
 be proven wrong.


 I think maybe the C++ approach is better. If you want something 
 into C++ std your best bet is to try to get it into Boost first.


std.experimental should probably be our boost.
Jun 17 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Friday, 17 June 2016 at 07:16:20 UTC, deadalnix wrote:

 On Friday, 17 June 2016 at 06:54:20 UTC, Ola Fosheim Grøstad 
 wrote:

 On Thursday, 16 June 2016 at 23:27:06 UTC, tsbockman wrote:

 I do not think this approach will scale in the long run, no 
 matter who is given that role. Of course, I would be happy to 
 be proven wrong.


 I think maybe the C++ approach is better. If you want 
 something into C++ std your best bet is to try to get it into 
 Boost first.


 std.experimental should probably be our boost.


It could be, but "experimental" sends signals of not being 
production ready. If it isn't ready for production it also won't 
be tested to the same extent.

Anyway, having something like Boost with a different custodian 
than Phobos, would allow people to vote with their feet. If a 
library is popular it should be a candidate.

C++ has "experimental" for unstable APIs:
http://en.cppreference.com/w/cpp/experimental/gcd
http://en.cppreference.com/w/cpp/experimental/to_array
Jun 17 2016
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-06-17 00:30, Walter Bright wrote:


 Andrei is in charge of the library and has my full support.


We have to make it very clear, somewhere, exactly what requires Andrei's 
approval and what doesn't. No, I cannot do that because I don't know the 
requirements.

-- 
/Jacob Carlborg
Jun 17 2016
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/17/2016 1:58 AM, Jacob Carlborg wrote:

 On 2016-06-17 00:30, Walter Bright wrote:


 Andrei is in charge of the library and has my full support.


 We have to make it very clear, somewhere, exactly what requires Andrei's
 approval and what doesn't. No, I cannot do that because I don't know the
 requirements.


Certainly, adding new modules would.
Jun 17 2016
prev sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Friday, 17 June 2016 at 08:58:51 UTC, Jacob Carlborg wrote:

 On 2016-06-17 00:30, Walter Bright wrote:


 Andrei is in charge of the library and has my full support.


 We have to make it very clear, somewhere, exactly what requires 
 Andrei's approval and what doesn't. No, I cannot do that 
 because I don't know the requirements.


What caught me off guard here isn't that Andrei had to approve it 
in some sense - it's that the judgment of everyone else who 
approved my design seemingly became irrelevant the moment that 
Andrei decided he could do better.

This implies that the only safe way to go about design a new 
module, is to collaborate closely with Andrei from the start.
Jun 17 2016
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Friday, 17 June 2016 at 14:52:27 UTC, tsbockman wrote:

 What caught me off guard here isn't that Andrei had to approve 
 it in some sense - it's that the judgment of everyone else who 
 approved my design seemingly became irrelevant the moment that 
 Andrei decided he could do better.


I hope you don't get too discouraged. Even if not in phobos, the 
project is still worth having.
Jun 17 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/17/2016 7:52 AM, tsbockman wrote:

 What caught me off guard here isn't that Andrei had to approve it in some sense
 - it's that the judgment of everyone else who approved my design seemingly
 became irrelevant the moment that Andrei decided he could do better.


History is full of examples of individuals who decide they can do better, 
against all conventional wisdom, and are eventually proven correct. Andrei has 
an enviable track record in the industry of having disruptive ideas that later 
became mainstream. He's also very well paid as a consultant to give advice to 
the programming teams of companies. His comments are worthy of serious 
consideration.



 This implies that the only safe way to go about design a new module, is to
 collaborate closely with Andrei from the start.


The goal is to have only the best in Phobos, even if the process of getting 
there is inefficient, frustrating, and slow. D has moved beyond the point where 
we can accept anything less.

A beneficial, leveraging side effect is improving the skills of the people
doing 
the work by coaching, reviewing, and critiquing the work. Andrei's review of 
SafeInt has elements that apply to just about everything in Phobos - things we 
all need to get better at.

I'd like to see his review here morphed into a best practices checklist others 
can use when reviewing Phobos code.
Jun 17 2016
next sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
What you said is true, but IMO you're missing the point.

IMO the current D Process is just backward. Having people working 
on stuff openly, with intend for phobos inclusion, for a lot of 
months and that putting it up for review clearly does not work.
And if you and/or Andrei at that point say "Na". That is just 
frustrating, no matter how good the reasons are.

So I propose that you and Andrei create a list of high level 
goals for the compiler as well as for phobos/druntime. Nothing 
abstract, specific stuff properly also design specifics. And 
attach explanations why those things should be done.

For example:

Stuff that needs to be in phobos:
* sync/async Database abstraction that is nogc
* Value Range Integer type
* ....

I think you get the idea about the list.
Than you write in big letters.
"To everybody that wants to contribute a modules to phobos look 
at the above list and pick you module. All other module ideas may 
be rejected for whatever reason by Andrei. If you have an idea 
for a module not on the list get the design approved by Andrei 
before you start coding to avoid frustration later."

I know, this is an open source project and you can't tell people 
what to do because you don't pay us any money. But you could tell 
us what you think is important to make D number one. At least 
then people know what the lay of the land is and don't have to 
guess. And by saying "yes" or "no" at the end of the development 
of new stuff, you're actually doing this already. Only at the 
wrong point in time, IMO.


Additionally, IMO at some point you two have to delegate/trust 
other people/lieutenants. You two simply don't have the time to 
do all the necessary reviewing to make the D Process scale. And 
you two make mistakes as well. core.checkedint and etc.c.odbc is 
just deadbeef. (Please nobody start a discussion about that here. 
Even if those two module are a success, last time I checked 
Walter and Andrei are human so they make mistakes. And Andrei 
actually said so at DConf as well.)


So an action list from my POV.
* Andrei, Walter create a specific task list and designs
* make this list obvious to find. github README (See comment 
above) !!!!
* disable the auto quote feature of the forum. This feature is 
just poisonous for the discussion culture of this forum. 
Everybody is just trying to disprove the other without actually 
trying to bring the discussion/D forward.
* something like https://github.com/facebook/mention-bot would be 
nice. It will properly also help finding lieutenants.

rant off

p.s. Warning: Parent post is not quoted. Ignore Explain Fit it 
for me
I f****** hate it.
Jun 17 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/17/2016 3:17 PM, Robert burner Schadek wrote:

 * Andrei, Walter create a specific task list and designs


I do all the time. The net result - nothing happens.

People do what they want to do when it's volunteer work. Presenting people with 
a list of things to work on does not result in work getting done on it.



 you two have to delegate/trust other people/lieutenants


We've done a lot of that. For example, Andrei is in charge of the library. 
Several people are empowered to pull stuff. Brad is in charge of the autotester 
and bugzilla. Martin handles releases. Mike is in charge of the blog. Dicebot 
recently took charge of DIPs. Etc. etc.

A common theme there, however, is they are self-selected. Assigning people to
do 
things has never worked.

Realistically, however, the review of SafeInt was delegated. tsbockman received 
a lot of feedback, advice, and ideas from the community. If Andrei's comments 
are all wrong-headed, that's one thing. But if they are on target, then we as a 
community need to get better when we advise and comment on proposals. Including
me.



 rant off


So please excuse my counter-rant :-)
Jun 17 2016
parent reply Robert burner Schadek <rburners gmail.com> writes:
Ok, where is this list? And where is list for phobos?

Sure, some people are a allowed to pull stuff into phobos. But 
only stuff that does not introduce new names and Andrei does not 
veto.

Of course you can not appoint people, but you should have an idea 
who you think is sharing your overall vision of D. And then you 
can ask them. Delegating work is nice, but I would like you to 
delegate power.
Jun 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/18/2016 1:52 AM, Robert burner Schadek wrote:

 Ok, where is this list? And where is list for phobos?


I've posted them from time to time. Most recently I put a "call to action" in 
the thread "size_t vs uintptr_t". Many people have come up to me and asked for
a 
list of ideas, I've given them a list, and they did something else, every time.

Off the top of my head, work items for phobos I've repeatedly posted in various 
forms:

1. eliminate all gratuitous use of gc
2. review all Phobos modules for compatibility with ranges - std.zip, for 
example, was done before ranges and does not work with them
3. use -cov to improve code coverage of Phobos modules
4. make sure every function in Phobos has an example
5. make sure every function in Phobos has Params: and Returns: sections
http://www.digitalmars.com/d/archives/digitalmars/D/Phobos_Documentation_-_call_to_action_258777.html
6. replace std.xml with something we can be proud of
7. compile a list of popular modules in other languages and see what we should
have
8. create a greenthreads module that works like Goroutines
9. create a module that enables code to be run on GPUs
10. now that D can interface to C++ templates and exceptions, create the 
interface code to the C++ STL
11. review all of Phobos for  safe compatibility
12. remove dependency on autodecode from Phobos
13. there was recently a thread on Phobos desperately needing a BigDecimal 
module, someone helpfully posted links to a couple of free C versions, I 
suggested that someone build a D wrapper for one or convert one to D. Sadly, 
that was the end of the discussion :-(
14. I regularly ask people to write articles about D, and the situation is 
slowly getting better, in that more and more good ones are getting written

I've learned to not ask for a gui library :-)

There is no shortage of action items. Just hang out in the n.g. for a bit.



 Of course you can not appoint people, but you should have an idea who you think
 is sharing your overall vision of D. And then you can ask them. Delegating work
 is nice, but I would like you to delegate power.


I do and I do. Andrei, for example, is in charge of Phobos. Martin does the 
releases. Etc. I do not remotely make all the decisions. I do my best not to 
undermine or micromanage others who do make decisions, trying to only step in 
when I felt something has really gone wrong.

-----------------------

What do you want to work on?
Jun 18 2016
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-06-18 12:25, Walter Bright wrote:


 Off the top of my head, work items for phobos I've repeatedly posted in
 various forms:

 1. eliminate all gratuitous use of gc
 2. review all Phobos modules for compatibility with ranges - std.zip,
 for example, was done before ranges and does not work with them
 3. use -cov to improve code coverage of Phobos modules
 4. make sure every function in Phobos has an example
 5. make sure every function in Phobos has Params: and Returns: sections
 http://www.digitalmars.com/d/archives/digitalmars/D/Phobos_Documentation_-_call_to_action_258777.html

 6. replace std.xml with something we can be proud of
 7. compile a list of popular modules in other languages and see what we
 should have
 8. create a greenthreads module that works like Goroutines
 9. create a module that enables code to be run on GPUs
 10. now that D can interface to C++ templates and exceptions, create the
 interface code to the C++ STL
 11. review all of Phobos for  safe compatibility
 12. remove dependency on autodecode from Phobos
 13. there was recently a thread on Phobos desperately needing a
 BigDecimal module, someone helpfully posted links to a couple of free C
 versions, I suggested that someone build a D wrapper for one or convert
 one to D. Sadly, that was the end of the discussion :-(
 14. I regularly ask people to write articles about D, and the situation
 is slowly getting better, in that more and more good ones are getting
 written

 I've learned to not ask for a gui library :-)

 There is no shortage of action items. Just hang out in the n.g. for a bit.


Now you're doing exactly as you're telling us not to do. Writing an 
important post deep inside a thread, with a completely different 
subject, that nobody will see. Put it on the front page of dlang.org or 
somewhere visible.

-- 
/Jacob Carlborg
Jun 18 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/18/2016 4:21 AM, Jacob Carlborg wrote:

 Now you're doing exactly as you're telling us not to do. Writing an important
 post deep inside a thread, with a completely different subject, that nobody
will
 see. Put it on the front page of dlang.org or somewhere visible.


Good point.
Jun 18 2016
prev sibling next sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
I created this: https://wiki.dlang.org/Walter_Andrei_Action_List
And PR's to dmd, druntime, and phobos' README.md that points to 
it.
If you could keep that list up to date, would be really hopeful 
IMO.
https://github.com/dlang/druntime/pull/1597
https://github.com/dlang/dmd/pull/5874
https://github.com/dlang/phobos/pull/4435


 What do you want to work on?


* I'm working on mentoring the GSoC xml replacement 
(https://github.com/lodo1995/experimental.xml) it is looking 
really good btw.
* I have https://github.com/dlang/phobos/pull/2995 in the 
pipeline which I think will help us to spot performance 
regeression in phobos and in any other D code for that matter
* I'm also working on getting that PR into unit-threaded as I 
fear the PR will sit in the queue for some time to come.
* recently I worked on std.parallelism and tried to get all tests 
run by default. https://github.com/dlang/phobos/pull/4399 No fun! 
std.parallelism or at least way the author indented its use, as 
shown in the unittests, is just broken.
* Then I'm working on my DConf project.
* And I'm reviewing phobos PR's
Jun 18 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/18/2016 6:46 AM, Robert burner Schadek wrote:

 I created this: https://wiki.dlang.org/Walter_Andrei_Action_List
 And PR's to dmd, druntime, and phobos' README.md that points to it.
 If you could keep that list up to date, would be really hopeful IMO.


Having it in the readme is a good idea.



 What do you want to work on?


 * I'm working on mentoring the GSoC xml replacement
 (https://github.com/lodo1995/experimental.xml) it is looking really good btw.
 * I have https://github.com/dlang/phobos/pull/2995 in the pipeline which I
think
 will help us to spot performance regeression in phobos and in any other D code
 for that matter
 * I'm also working on getting that PR into unit-threaded as I fear the PR will
 sit in the queue for some time to come.
 * recently I worked on std.parallelism and tried to get all tests run by
 default. https://github.com/dlang/phobos/pull/4399 No fun! std.parallelism or
at
 least way the author indented its use, as shown in the unittests, is just
broken.
 * Then I'm working on my DConf project.
 * And I'm reviewing phobos PR's


That is a most excellent and valuable set of projects you're working on!
Jun 18 2016
prev sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 18 June 2016 at 10:25:43 UTC, Walter Bright wrote:

 [...]
 I've given them a list, and they did something else, every time.

 Off the top of my head, work items for phobos I've repeatedly 
 posted in various forms:
 [...]


People actually have been working on a lot of the stuff in your 
list.

The sweeping Phobos-wide tasks like cutting back on GC use, 
making more stuff ` safe`, improving the docs, etc. is getting 
worked on all the time. No one is going to go and convert the 
entirety of Phobos in one fell swoop, because Phobos is huge and 
there is no individual who has the domain knowledge required to 
work on every part of it.

Nevertheless, there has been steady progress on those tasks 
because whenever people work on some part of Phobos (for whatever 
reason), they tend to also do some general cleanup while they're 
at it. (This is hindered by the insistence upon splitting work 
into many small pull requests, though.)

As for the big module ideas - people have discussed work already 
in progress toward a better `std.xml` and a GPU compute library 
both here on the forums, and (in the case of the latter) in a 
talk at the most recent DConf.

There are specific reasons, aired in the discussions, as to why 
people may not be interested in working on BigDecimal (perceived 
as being of importance mainly to the finance industry, who is 
unlikely to contribute anything back in return) or removing auto 
decode (what's the point, when Andrei is emphatic that the 
feature itself cannot ever be deprecated?). Even so, those 
discussions were very recent; it is quite possible that someone 
around here intends to work on them (or already is) and just 
hasn't got far enough to want feedback yet.
Jun 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/18/2016 8:38 AM, tsbockman wrote:

 There are specific reasons, aired in the discussions, as to why people may not
 be interested in working on BigDecimal (perceived as being of importance mainly
 to the finance industry, who is unlikely to contribute anything back in return)
 or removing auto decode (what's the point, when Andrei is emphatic that the
 feature itself cannot ever be deprecated?). Even so, those discussions were
very
 recent; it is quite possible that someone around here intends to work on them
 (or already is) and just hasn't got far enough to want feedback yet.


Actually, Andrei has agreed with the issues autodecoding has, he's just 
(legitimately) wary of the extent of disruption it would cause by removing it. 
He is also in favor of fixing Phobos so it is not dependent on it.

(Note that I have fixed quite a few bits of Phobos in this manner already, see 
std.string and std.path)
Jun 18 2016
parent tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 18 June 2016 at 19:49:07 UTC, Walter Bright wrote:

 On 6/18/2016 8:38 AM, tsbockman wrote:

 removing auto decode (what's the point, when Andrei is
 emphatic that the feature itself cannot ever be deprecated?).
 Even so, those discussions were very recent; it is quite
 possible that someone around here intends to work on them
 (or already is) and just hasn't got far enough to want 
 feedback yet.


 Actually, Andrei has agreed with the issues autodecoding has, 
 he's just (legitimately) wary of the extent of disruption it 
 would cause by removing it. He is also in favor of fixing 
 Phobos so it is not dependent on it.


Unless you're trying to say that Andrei has now agreed to 
deprecate autodecoding, you're not actually disagreeing with what 
I wrote.

The most enticing reason to work on removing Phobos' dependency 
on autodecoding, is to pave the way for deprecating the feature 
itself. The motivation for the Phobos changes is much reduced if 
the deprecation itself is not planned.

Anyway, I'm not trying to restart the autodecoding debate here 
(nor to discourage anyone who wants to from working on this), so 
I suggest you start a new thread if you have something to 
announce about this issue that wasn't already said at the 
conclusion of the last debate.
Jun 18 2016
prev sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Friday, 17 June 2016 at 20:49:54 UTC, Walter Bright wrote:

 On 6/17/2016 7:52 AM, tsbockman wrote:

 What caught me off guard here isn't that Andrei had to approve 
 it in some sense
 - it's that the judgment of everyone else who approved my 
 design seemingly
 became irrelevant the moment that Andrei decided he could do 
 better.


 History is full of examples of individuals who decide they can 
 do better, against all conventional wisdom, and are eventually 
 proven correct.


That has nothing to do with my point: "better" is subjective, and 
I thought the judgment and goals of others would be given more 
weight.


 Andrei has an enviable track record in the industry of having 
 disruptive ideas that later became mainstream. He's also very 
 well paid as a consultant to give advice to the programming 
 teams of companies. His comments are worthy of serious 
 consideration.


I have a finite amount of time, energy, and motivation to work on 
this.

I might have considered his proposed expansion of the scope of 
the project if it had come earlier, when I was actively seeking 
input from people about what features were important to them. 
That connection did not happen, though.

Regardless, it is not clear to me that his requirements are 
actually compatible with mine; I don't really care how wonderful 
his ideas are if he's trying to solve a (subtly) different 
problem than I am. As far as I can tell, Andrei is perfectly 
willing to sacrifice my requirements to make room in his 
complexity "budget" for his own.

This is, of course, his prerogative as lead designer for Phobos, 
but it also kills any interest I have in working on the project 
for free.
Jun 17 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/17/2016 4:38 PM, tsbockman wrote:

 This is, of course, his prerogative as lead designer for Phobos, but it also
 kills any interest I have in working on the project for free.



You know, I do sympathize, even if I don't agree. One big motivation for D in 
the first place was the C++ community not being interested in my contributions. 
Ironically, they're adopting them one by one anyway :-)

If you're not interested in continuing with this, I suggest packaging it up and 
making it available via Dub. Sort of let the marketplace decide.
Jun 17 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 18 June 2016 at 01:28:38 UTC, Walter Bright wrote:

 On 6/17/2016 4:38 PM, tsbockman wrote:

 This is, of course, his prerogative as lead designer for 
 Phobos, but it also
 kills any interest I have in working on the project for free.



 You know, I do sympathize, even if I don't agree. One big 
 motivation for D in the first place was the C++ community not 
 being interested in my contributions. Ironically, they're 
 adopting them one by one anyway :-)


Indeed, D has many excellent ideas. I hope that some day you'll 
consider allowing a breaking "D3" though, as the D language 
design also has a great deal of technical debt accumulated from 
the blitz of feature additions in D2.

If that debt can be payed off, I don't think that any of the 
currently popular languages could compete with it.


 If you're not interested in continuing with this, I suggest 
 packaging it up and making it available via Dub. Sort of let 
 the marketplace decide.


It's been on DUB since before this review started, and I plan to 
support it (including incorporating some of the feedback from 
this thread).
Jun 17 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/17/2016 7:20 PM, tsbockman wrote:

 It's been on DUB since before this review started, and I plan to support it
 (including incorporating some of the feedback from this thread).


Thanks, that's a good plan.
Jun 17 2016
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 15.06.2016 18:40, Andrei Alexandrescu wrote:

 The only remaining matter is to implement a few preexisting policies
 (Hook implementations) to implement typical choices (such as the ones
 present today), and the core algorithms for doing bounded operations.
 The most interesting algorithms are for computing the bounds of
 operations such as |, &, and ^. The D compiler needs those as well, and
 currently implements them incorrectly. I have these in my mind and I can
 help with those.


https://github.com/tgehr/d-compiler/blob/master/vrange.d#L338
Jun 15 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 06/15/2016 05:32 PM, Timon Gehr wrote:

 On 15.06.2016 18:40, Andrei Alexandrescu wrote:

 The only remaining matter is to implement a few preexisting policies
 (Hook implementations) to implement typical choices (such as the ones
 present today), and the core algorithms for doing bounded operations.
 The most interesting algorithms are for computing the bounds of
 operations such as |, &, and ^. The D compiler needs those as well, and
 currently implements them incorrectly. I have these in my mind and I can
 help with those.


 https://github.com/tgehr/d-compiler/blob/master/vrange.d#L338


Nice! On first glance (number of steps :o)) these looks like what I had 
in mind. Any chance you could make PRs for integrating these into the 
compiler? Thanks! -- Andrei
Jun 15 2016
prev sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 15 June 2016 at 16:40:19 UTC, Andrei Alexandrescu 
wrote:

 One of the first things I looked for was establishing bounds 
 for numbers, like Smart!(int, 0, 100) for percentage. For all 
 its might, this package does not offer this basic facility, and 
 from what I can tell does not allow users to enforce it via 
 policies.


Another angle on why `checkedint` does not currently include a 
bounded type: although some of the internals might be similar, 
conceptually a `BoundInt` type is actually the *opposite* of the 
ideal which `SmartInt` and `SafeInt` strive toward.

The problem which this package was written to solve, is that when 
programming people frequently want to use "integers" as a data 
type. A real "integer" is, by definition, UNbounded, with an 
infinite range of values available. This makes designing 
algorithms simpler, but of course leads to problems when a value 
strays outside the narrow range in which the behaviour of machine 
integers correctly emulates that of mathematical integers.

The whole point of `checkedint` is to be able to write algorithms 
based on the simplifying assumption that your variables behave 
like real, UNbounded mathematical integers, secure in the 
knowledge that an error message will be generated if that 
assumption is invalidated.

Intentionally limiting values to a specific range is a different 
problem, and never the one that I was trying to solve - although 
I agree that the `checkedint` package would be a good namespace 
in which to park a `BoundInt` type, if someone wants to submit 
one.
Jun 15 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 06/15/2016 07:34 PM, tsbockman wrote:

 The whole point of `checkedint` is to be able to write algorithms based
 on the simplifying assumption that your variables behave like real,
 UNbounded mathematical integers, secure in the knowledge that an error
 message will be generated if that assumption is invalidated.


Fair point, though one could make the same claim for integers within 
other limits than the machine's. -- Andrei
Jun 15 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 16 June 2016 at 00:03:26 UTC, Andrei Alexandrescu 
wrote:

 On 06/15/2016 07:34 PM, tsbockman wrote:

 The whole point of `checkedint` is to be able to write 
 algorithms based
 on the simplifying assumption that your variables behave like 
 real,
 UNbounded mathematical integers, secure in the knowledge that 
 an error
 message will be generated if that assumption is invalidated.


 Fair point, though one could make the same claim for integers 
 within other limits than the machine's. -- Andrei


Sure. I'm just trying to explain my design goals and philosophy - 
not to argue that no other approach could be valid.

But, I would appreciate it if comments distinguished between:

1) My goals and philosophy differ from yours, versus
2) My implementation is bad.

I think a lot of the high-level criticism you brought is really 
rooted in (1), even though you present it as (2).
Jun 15 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 06/15/2016 08:07 PM, tsbockman wrote:

 On Thursday, 16 June 2016 at 00:03:26 UTC, Andrei Alexandrescu wrote:

 On 06/15/2016 07:34 PM, tsbockman wrote:

 The whole point of `checkedint` is to be able to write algorithms based
 on the simplifying assumption that your variables behave like real,
 UNbounded mathematical integers, secure in the knowledge that an error
 message will be generated if that assumption is invalidated.


 Fair point, though one could make the same claim for integers within
 other limits than the machine's. -- Andrei


 Sure. I'm just trying to explain my design goals and philosophy - not to
 argue that no other approach could be valid.

 But, I would appreciate it if comments distinguished between:

 1) My goals and philosophy differ from yours, versus
 2) My implementation is bad.

 I think a lot of the high-level criticism you brought is really rooted
 in (1), even though you present it as (2).


My perception is your implementation is professional. -- Andrei
Jun 15 2016
parent tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 16 June 2016 at 00:33:38 UTC, Andrei Alexandrescu 
wrote:

 On 06/15/2016 08:07 PM, tsbockman wrote:

 But, I would appreciate it if comments distinguished between:

 1) My goals and philosophy differ from yours, versus
 2) My implementation is bad.

 I think a lot of the high-level criticism you brought is 
 really rooted
 in (1), even though you present it as (2).


 My perception is your implementation is professional. -- Andrei


Thank you.
Jun 15 2016