www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - size_t index=-1;

reply Laeeth Isharc <laeethnospam nospam.laeeth.com> writes:
should it be a compiler warning to assign a negative literal to 
an unsigned without a cast ?
Mar 16 2016
next sibling parent Mathias Lang <pro.mathias.lang gmail.com> writes:
On Wednesday, 16 March 2016 at 18:40:56 UTC, Laeeth Isharc wrote:
 should it be a compiler warning to assign a negative literal to 
 an unsigned without a cast ?
yes it should. https://issues.dlang.org/show_bug.cgi?id=3468
Mar 16 2016
prev sibling next sibling parent Jonathan M Davis via Digitalmars-d-learn writes:
On Wednesday, March 16, 2016 18:40:56 Laeeth Isharc via Digitalmars-d-learn 
wrote:
 should it be a compiler warning to assign a negative literal to
 an unsigned without a cast ?
Maybe? It's a common enough thing to do that I'm willing to bet that Walter would object, but what you're really looking to do in most cases like that is to get something like uint.max, in which case it's better to just use the built-in constant. But I doubt that assigning negative literals to unsigned variables causes much in the way of bugs. The bigger problem is comparing signed and unsigned types, and a number of folks have argued that that should be a warning or error. - Jonathan M Davis
Mar 16 2016
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 3/16/16 2:40 PM, Laeeth Isharc wrote:
 should it be a compiler warning to assign a negative literal to an
 unsigned without a cast ?
Why? They implicitly convert. int x = -1; uint y = x; I don't see a difference between this and your code. And we can't change this behavior of the second line, too much arguably valid code would break. -Steve
Mar 16 2016
parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
On Wednesday, 16 March 2016 at 20:11:41 UTC, Steven Schveighoffer 
wrote:
 On 3/16/16 2:40 PM, Laeeth Isharc wrote:
 should it be a compiler warning to assign a negative literal 
 to an
 unsigned without a cast ?
Why? They implicitly convert. int x = -1; uint y = x; I don't see a difference between this and your code. And we can't change this behavior of the second line, too much arguably valid code would break. -Steve
We can change it, and we should. But it should be deprecated properly, and we should put in place enough candy to make it viable (See http://forum.dlang.org/post/vbeohujwdsoqfgwqgasa forum.dlang.org ).
Mar 16 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 3/16/16 4:55 PM, Mathias Lang wrote:
 On Wednesday, 16 March 2016 at 20:11:41 UTC, Steven Schveighoffer wrote:
 On 3/16/16 2:40 PM, Laeeth Isharc wrote:
 should it be a compiler warning to assign a negative literal to an
 unsigned without a cast ?
Why? They implicitly convert. int x = -1; uint y = x; I don't see a difference between this and your code. And we can't change this behavior of the second line, too much arguably valid code would break.
We can change it, and we should. But it should be deprecated properly, and we should put in place enough candy to make it viable (See http://forum.dlang.org/post/vbeohujwdsoqfgwqgasa forum.dlang.org ).
No, please don't. Assigning a signed value to an unsigned (and vice versa) is very useful, and there is no good reason to break this. -Steve
Mar 16 2016
next sibling parent reply Anonymouse <asdf asdf.com> writes:
On Wednesday, 16 March 2016 at 21:49:05 UTC, Steven Schveighoffer 
wrote:
 No, please don't. Assigning a signed value to an unsigned (and 
 vice versa) is very useful, and there is no good reason to 
 break this.

 -Steve
I agree, but implicitly allowing for comparisons between the two allows for easy and *silent* mistakes. Also for comparisons of less-than-zero, for those functions we have that return -1 on failure. import std.string : indexOf; size_t pos = "banana".indexOf("c"); if (pos > 0) { // oops } The above is naturally a programmer error but it's not something that will net you an immediate crash. It will just silently not behave as you meant, and you'll find yourself with a lot of fake bananas.
Mar 16 2016
parent Johan Engelen <j j.nl> writes:
On Wednesday, 16 March 2016 at 22:07:39 UTC, Anonymouse wrote:
 
 size_t pos = "banana".indexOf("c");
 if (pos > 0) {
Although I also think it makes sense to warn (in specific cases) about mixed-sign comparisons, the example you give here does nothing that we can warn about. It is a comparison of an unsigned "pos" with a literal that is unsigned too. ("0" literal must be considered signed and unsigned without any warnings)
Mar 16 2016
prev sibling parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
On Wednesday, 16 March 2016 at 21:49:05 UTC, Steven Schveighoffer 
wrote:
 No, please don't. Assigning a signed value to an unsigned (and 
 vice versa) is very useful, and there is no good reason to 
 break this.

 -Steve
I'm not talking about removing it completely. The implicit conversion should only happen when it's safe: ``` int s; if (s >= 0) // VRP saves the day { uint u = s; } ``` ``` uint u; if (u > short.max) throw new Exception("Argument out of range"); // Or `assert` short s = u; ```
Mar 16 2016
next sibling parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Wednesday, March 16, 2016 22:37:40 Mathias Lang via Digitalmars-d-learn 
wrote:
 On Wednesday, 16 March 2016 at 21:49:05 UTC, Steven Schveighoffer

 wrote:
 No, please don't. Assigning a signed value to an unsigned (and
 vice versa) is very useful, and there is no good reason to
 break this.

 -Steve
I'm not talking about removing it completely. The implicit conversion should only happen when it's safe: ``` int s; if (s >= 0) // VRP saves the day { uint u = s; } ``` ``` uint u; if (u > short.max) throw new Exception("Argument out of range"); // Or `assert` short s = u; ```
Now, you're talking about comparing signed and unsigned values, which is a completely different ballgame. Just assigning one to the other really isn't a problem, and sometimes you _want_ the wraparound. If you assume that it's always the case that assigning a negative value to an unsigned type is something that programmers don't want to do, then you haven't programmed in C enough. And while it could still be done by requiring casts, consider that every time you do a cast, you're telling the compiler to just shut up and do what you want, which makes it easy to hide stuff that you don't want hidden - especially when code changes later. D purposefully allows converting between signed and unsigned types of the same or greater size. And based on what he's said on related topics in the past, there's pretty much no way that you're going to convince Walter that it's a bad idea. And I really don't see a problem with the current behavior as far as assignment goes. It's comparisons which are potentially problematic, and that's where you'd have some chance of getting a warning or error added to the compiler. If you want to actually have the values check to ensure that a negative value isn't assigned to an unsigned integer, then use std.conv.to to do conversions or wrap your integers in types that have more restrictive rules. IIRC, at least one person around here has done that already so that they can catch integer overflow - which is basically what you're complaining about here. - Jonathan M Davis
Mar 16 2016
next sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 17 March 2016 at 01:57:16 UTC, Jonathan M Davis 
wrote:
 Just assigning one to the other really isn't a problem, and 
 sometimes you _want_ the wraparound. If you assume that it's 
 always the case that assigning a negative value to an unsigned 
 type is something that programmers don't want to do, then you 
 haven't programmed in C enough.
Greater than 90% of the time, even in low level code, an assignment, comparison, or any other operation that mixes signed and unsigned types is being done directly (without bounds checking) only for speed, laziness, or ignorance - not because 2's complement mapping of negative to positive values is actually desired. Forcing deliberate invocations of 2's complement mapping between signed and unsigned types to be explicitly marked is a good thing, seeing as the intended semantics are fundamentally different. I interpret any resistance to this idea, simply as evidence that we haven't yet made it sufficiently easy/pretty to be explicit. Any idea that it's actually *desirable* for code to be ambiguous in this way is just silly.
Mar 16 2016
prev sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 17 March 2016 at 01:57:16 UTC, Jonathan M Davis 
wrote:
 or wrap your integers in types that have more restrictive 
 rules. IIRC, at least one person around here has done that 
 already so that they can catch integer overflow - which is 
 basically what you're complaining about here.
That's me (building on Robert Schadek's work): https://code.dlang.org/packages/checkedint Although I should point out that my `SmartInt` actually has *less* restrictive rules than the built-in types - all possible combinations of size and signedness are both allowed and safe for all operations, without any explicit casts. A lot of what `SmartInt` does depends on (minimal) extra runtime logic, which imposes a ~30% performance penalty (when integer math is actually the bottleneck) with good compiler optimizations (GDC or LDC). But, a lot of it could also be done at no runtime cost, by leveraging VRP. C's integer math rules are really pretty bad, even when taking performance into account. Something as simple as by default promoting to a signed, rather than unsigned, type would prevent many bugs in practice, at zero cost (except that it would be a breaking change). There is also `SafeInt` with "more restrictive rules", if it is for some reason necessary to work inside the limitations of the built-in basic integer types.
Mar 16 2016
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 3/16/16 6:37 PM, Mathias Lang wrote:
 On Wednesday, 16 March 2016 at 21:49:05 UTC, Steven Schveighoffer wrote:
 No, please don't. Assigning a signed value to an unsigned (and vice
 versa) is very useful, and there is no good reason to break this.
I'm not talking about removing it completely. The implicit conversion should only happen when it's safe: ``` int s; if (s >= 0) // VRP saves the day { uint u = s; } ``` ``` uint u; if (u > short.max) throw new Exception("Argument out of range"); // Or `assert` short s = u; ```
Converting unsigned to signed or vice versa (of the same size type) is safe. No information is lost. It's the comparison between the two which confuses the heck out of people. I think we can solve 80% of the problems by just fixing that. And the bug report says it's preapproved from Walter and Andrei. VRP on steroids would be nice, but I don't think it's as trivial to solve. -Steve
Mar 17 2016
next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 17 March 2016 at 17:09:46 UTC, Steven Schveighoffer 
wrote:
 Converting unsigned to signed or vice versa (of the same size 
 type) is safe. No information is lost.
Saying that "no information is lost" in such a case, is like saying that if I encrypt my hard drive and then throw away the password, "no information is lost". Technically this is true: the bit count is the same as it was before. In practice, though, the knowledge of how information is encoded is essential to actually using it. In the same way, using `cast(ulong)` to pass `-1L` to a function that expects a `ulong` results in a de-facto loss of information, because that `-1L` can no longer distinguished from `ulong.max`, despite the fundamental semantic difference between the two.
 VRP on steroids would be nice, but I don't think it's as 
 trivial to solve.
D's current VRP is actually surprisingly anemic: it doesn't even understand integer comparisons, or the range restrictions implied by the predicate when a certain branch of an `if` statement is taken. Lionello Lunesu made a PR a while back that adds these two features, and it makes the compiler feel a lot smarter. (The PR was not accepted at the time, but I have since revived it: https://github.com/D-Programming-Language/dmd/pull/5229)
Mar 17 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWY=?= writes:
On Thursday, 17 March 2016 at 22:46:01 UTC, tsbockman wrote:
 In the same way, using `cast(ulong)` to pass `-1L` to a 
 function that expects a `ulong` results in a de-facto loss of 
 information, because that `-1L` can no longer distinguished 
 from `ulong.max`, despite the fundamental semantic difference 
 between the two.
Only providing modular arithmetics is a significant language design flaw, but as long as all integers are defined to be modular then there is no fundamental semantic difference either. Of course, comparisons beyond equality doesn't work for modular arithmetics either, irrespective of sign... You basically have to decide whether you want a line or a circle; Walter chose the circle for integers and the line for floating point. The circle is usually the wrong model, but that does not change the language definition...
Mar 17 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Friday, 18 March 2016 at 05:20:35 UTC, Ola Fosheim Grøstaf 
wrote:
 Only providing modular arithmetics is a significant language 
 design flaw, but as long as all integers are defined to be 
 modular then there is no fundamental semantic difference either.
`ulong.max` and `-1L` are fundamentally different semantically, even with two's complement modular arithmetic. Just because a few operations (addition and subtraction, mainly) can use a common implementation for both, does not change that. Division, for example, cannot be done correctly without knowing whether the inputs are signed or not.
Mar 18 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Friday, 18 March 2016 at 23:35:42 UTC, tsbockman wrote:
 `ulong.max` and `-1L` are fundamentally different semantically, 
 even with two's complement modular arithmetic.
Different types implies different semantics, but not the literals in isolation. Under modular arithmetics for an ubyte the literals -128 and 128 both refer to 128. This follows from -128 == 0 - (128). Unfortunately in D, the actual arithmetics is not done modulo 2^8, but modulo 2^32. So, what we should object to is modular arithmetics over integers as defined in D.
 Just because a few operations (addition and subtraction, 
 mainly) can use a common implementation for both, does not 
 change that. Division, for example, cannot be done correctly 
 without knowing whether the inputs are signed or not.
Yes, both multiplication and division change with the type, but you usually don't want signed values in modular arithmetics? The major flaw is in how D defines arithmetics for integers.
Mar 19 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 19 March 2016 at 08:49:29 UTC, Ola Fosheim Grøstad 
wrote:
 On Friday, 18 March 2016 at 23:35:42 UTC, tsbockman wrote:
 `ulong.max` and `-1L` are fundamentally different 
 semantically, even with two's complement modular arithmetic.
Different types implies different semantics, but not the literals in isolation.
Both of the literals I used in my example explicitly indicate the type, not just the value.
Mar 19 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Saturday, 19 March 2016 at 09:35:00 UTC, tsbockman wrote:
 Both of the literals I used in my example explicitly indicate 
 the type, not just the value.
Yes, but few people specify unsigned literals and relies on them being implicitly cast to unsigned. You don't want to type 0UL and 1UL all the time. This is a another thing that Go does better, numeric literals ought to not be bound to a concrete type. So while I agree with you that the integer situation is messy, changing it to something better requires many changes. Which I am all for.
Mar 19 2016
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 3/17/16 6:46 PM, tsbockman wrote:
 On Thursday, 17 March 2016 at 17:09:46 UTC, Steven Schveighoffer wrote:
 Converting unsigned to signed or vice versa (of the same size type) is
 safe. No information is lost.
Saying that "no information is lost" in such a case, is like saying that if I encrypt my hard drive and then throw away the password, "no information is lost". Technically this is true: the bit count is the same as it was before.
It's hard to throw away the "key" of 2's complement math.
 In practice, though, the knowledge of how information is encoded is
 essential to actually using it.
In practice, a variable that is unsigned or signed is expected to behave like it is declared. I don't think anyone expects differently. When I see: size_t x = -1; I expect x to behave like an unsigned size_t that represents -1. There is no ambiguity here. Where it gets confusing is if you didn't mean to type size_t. But the compiler can't know that. When you start doing comparisons, then ambiguity creeps in. The behavior is well defined, but not very intuitive. You can get into trouble even without mixing signed/unsigned types. For example: for(size_t i = 0; i < a.length - 1; ++i) This is going to crash when a.length == 0. Better to do this: for(size_t i = 0; i + 1 < a.length; ++i) unsigned math can be difficult, there is no doubt. But we can't just disable it, or disable unsigned conversions.
 In the same way, using `cast(ulong)` to pass `-1L` to a function that
 expects a `ulong` results in a de-facto loss of information, because
 that `-1L` can no longer distinguished from `ulong.max`, despite the
 fundamental semantic difference between the two.
Any time you cast a type, the original type information is lost. But in this case, no bits are lost. In this case, the function is declaring "I don't care what your original type was, I want to use ulong". If it desires to know the original type, it should use a template parameter instead. Note, I have made these mistakes myself, and I understand what you are asking for and why you are asking for it. But these are bugs. The user is telling the compiler to do one thing, and expecting it to do something else. It's not difficult to fix, and in fact, many lines of code are written specifically to take advantage of these rules. This is why we cannot remove them. The benefit is not worth the cost.
 VRP on steroids would be nice, but I don't think it's as trivial to
 solve.
D's current VRP is actually surprisingly anemic: it doesn't even understand integer comparisons, or the range restrictions implied by the predicate when a certain branch of an `if` statement is taken. Lionello Lunesu made a PR a while back that adds these two features, and it makes the compiler feel a lot smarter. (The PR was not accepted at the time, but I have since revived it: https://github.com/D-Programming-Language/dmd/pull/5229)
I'm not compiler-savvy enough to have an opinion on the PR, but I think more sophisticated VRP would be good. -Steve
Mar 18 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Friday, 18 March 2016 at 14:51:34 UTC, Steven Schveighoffer 
wrote:
 Note, I have made these mistakes myself, and I understand what 
 you are asking for and why you are asking for it. But these are 
 bugs. The user is telling the compiler to do one thing, and 
 expecting it to do something else. It's not difficult to fix, 
 and in fact, many lines of code are written specifically to 
 take advantage of these rules. This is why we cannot remove 
 them. The benefit is not worth the cost.
Actually, I think I confused things for you by mentioning to `cast(ulong)`. I'm not asking for a Java-style "no unsigned" system (I hate that; it's one of my biggest annoyances with Java). Rather, I'm picking on *implicit* conversions between signed and unsigned. I'm basically saying, "because information is lost when casting between signed and unsigned, all such casts should be explicit". This could make code rather verbose - except that from my experiments, with decent VRP the compiler can actually be surprisingly smart about warning only in those cases where implicit casting is really a bad idea.
Mar 18 2016
next sibling parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Friday, March 18, 2016 23:48:32 tsbockman via Digitalmars-d-learn wrote:
 I'm basically saying, "because information is lost when casting
 between signed and unsigned, all such casts should be explicit".
See. Here's the fundamental disagreement. _No_ information is lost when converting between signed and unsigned integers. e.g. int i = -1; uint ui = i; int j = i; assert(j == -1); But even if you convinced us, you'd have to convince Walter. And based on previously discussions on this subject, I think that you have an _extremely_ low chance of that. He doesn't even think that there's a problem that void foo(bool bar) {} void foo(long bar) {} foo(1); resulted in call to the bool overload was a problem when pretty much everyone else did. The only thing that I'm aware of that Walter has thought _might_ be something that we should change is allowing the comparison between signed and unsigned integers, and if you read what he says in the bug report for it, he clearly doesn't think it's a big problem: https://issues.dlang.org/show_bug.cgi?id=259 And that's something that clearly causes bugs in way that converting between signed and unsigned integers does not. You're fighting for a lost cause on this one. - Jonathan M Davis
Mar 18 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 19 March 2016 at 04:17:42 UTC, Jonathan M Davis 
wrote:
 The only thing that I'm aware of that Walter has thought 
 _might_ be something that we should change is allowing the 
 comparison between signed and unsigned integers, and if you 
 read what he says in the bug report for it, he clearly doesn't 
 think it's a big problem:

 https://issues.dlang.org/show_bug.cgi?id=259

 And that's something that clearly causes bugs in way that 
 converting between signed and unsigned integers does not. 
 You're fighting for a lost cause on this one.

 - Jonathan M Davis
You do realize that, technically, there are no comparisons between basic signed and unsigned integers in D? The reason that *attempting* such a comparison produces such weird results, is because the signed value is being implicitly cast to an unsigned type. The thing you say *is* a problem, is directly caused by the thing that you say is *not* a problem.
Mar 19 2016
parent reply Basile B. <b2.temp gmx.com> writes:
On Saturday, 19 March 2016 at 09:33:25 UTC, tsbockman wrote:
 [...] The reason that *attempting* such a comparison produces 
 such weird results, is because the signed value is being 
 implicitly cast to an unsigned type.
Yes and that's the opposite that should happend: when signed and unsigned are mixed in a comparison, the unsigned value should be implictly cast to a wider signed value. And then it works! - https://issues.dlang.org/show_bug.cgi?id=15805 - https://github.com/BBasile/iz/blob/v0.5.8/import/iz/sugar.d#L1017
Mar 19 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Saturday, 19 March 2016 at 10:01:41 UTC, Basile B. wrote:
 On Saturday, 19 March 2016 at 09:33:25 UTC, tsbockman wrote:
 [...] The reason that *attempting* such a comparison produces 
 such weird results, is because the signed value is being 
 implicitly cast to an unsigned type.
Yes and that's the opposite that should happend: when signed and unsigned are mixed in a comparison, the unsigned value should be implictly cast to a wider signed value. And then it works! - https://issues.dlang.org/show_bug.cgi?id=15805 - https://github.com/BBasile/iz/blob/v0.5.8/import/iz/sugar.d#L1017
I have no problem with C++ compilers complaining about signed/unsigned comparisons. It sometimes means you should reconsider the comparison, so it leads to better code. The better solution is to add 7, 15, 31 and 63 bit unsigned integer types that safely converts to signed (this is what Ada does) and remove implicit conversion for unsigned 8,16,32, and 64 bit integers.
Mar 19 2016
parent Basile B. <b2.temp gmx.com> writes:
On Saturday, 19 March 2016 at 10:24:41 UTC, Ola Fosheim Grøstad 
wrote:
 On Saturday, 19 March 2016 at 10:01:41 UTC, Basile B. wrote:
 On Saturday, 19 March 2016 at 09:33:25 UTC, tsbockman wrote:
 [...] The reason that *attempting* such a comparison produces 
 such weird results, is because the signed value is being 
 implicitly cast to an unsigned type.
Yes and that's the opposite that should happend: when signed and unsigned are mixed in a comparison, the unsigned value should be implictly cast to a wider signed value. And then it works! - https://issues.dlang.org/show_bug.cgi?id=15805 - https://github.com/BBasile/iz/blob/v0.5.8/import/iz/sugar.d#L1017
I have no problem with C++ compilers complaining about signed/unsigned comparisons. It sometimes means you should reconsider the comparison, so it leads to better code. The better solution is to add 7, 15, 31 and 63 bit unsigned integer types that safely converts to signed (this is what Ada does)
FPC (Object Pascal) too, but that not a surpise since it's in the same family
 and remove implicit conversion for unsigned 8,16,32, and 64 bit 
 integers.
Yes that's almost that but in D the only solution I see is like in my template: widening. When widening is not possible (mainly on X86_64) then warning. The problem is that cent and ucent are not implemented, otherwise it would always work even on 64 bit OS. I'd like to propose a PR for this (not for cent/ucent but for the widening) but it looks a bit overcomplicated for a first contrib in the compiler...
Mar 19 2016
prev sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 19 March 2016 at 10:01:41 UTC, Basile B. wrote:
 Yes and that's the opposite that should happend: when signed 
 and unsigned are mixed in a comparison, the unsigned value 
 should be implictly cast to a wider signed value. And then it 
 works!
That would be reasonable. Whether it's actually faster than just inserting an extra check for `signed_value < 0` in mixed comparisons is likely platform dependent, though. Honestly though - even just changing the rules to implicitly convert both operands to a signed type of the same size, instead of an unsigned type of the same size, would be a huge improvement. Small negative values are way more common than huge (greater than signed_type.max) positive ones in almost all code. (This change will never happen, of course, as it would be far too subtle of a breaking change for existing code.) Regardless, the first step is to implement the pre-approved solution to DMD 259: deprecate the current busted behavior.
Mar 19 2016
prev sibling next sibling parent Jonathan M Davis via Digitalmars-d-learn writes:
On Friday, March 18, 2016 21:17:42 Jonathan M Davis via Digitalmars-d-learn 
wrote:
 On Friday, March 18, 2016 23:48:32 tsbockman via Digitalmars-d-learn wrote:
 I'm basically saying, "because information is lost when casting
 between signed and unsigned, all such casts should be explicit".
See. Here's the fundamental disagreement. _No_ information is lost when converting between signed and unsigned integers. e.g. int i = -1; uint ui = i; int j = i; assert(j == -1); But even if you convinced us, you'd have to convince Walter. And based on previously discussions on this subject, I think that you have an _extremely_ low chance of that. He doesn't even think that there's a problem that void foo(bool bar) {} void foo(long bar) {} foo(1); resulted in call to the bool overload was a problem when pretty much everyone else did. The only thing that I'm aware of that Walter has thought _might_ be something that we should change is allowing the comparison between signed and unsigned integers, and if you read what he says in the bug report for it, he clearly doesn't think it's a big problem: https://issues.dlang.org/show_bug.cgi?id=259 And that's something that clearly causes bugs in way that converting between signed and unsigned integers does not. You're fighting for a lost cause on this one.
And I really should have proofread this message before sending it... :( Hopefully, you get what I meant though. - Jonathan M Davis
Mar 19 2016
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 3/18/16 7:48 PM, tsbockman wrote:
 On Friday, 18 March 2016 at 14:51:34 UTC, Steven Schveighoffer wrote:
 Note, I have made these mistakes myself, and I understand what you are
 asking for and why you are asking for it. But these are bugs. The user
 is telling the compiler to do one thing, and expecting it to do
 something else. It's not difficult to fix, and in fact, many lines of
 code are written specifically to take advantage of these rules. This
 is why we cannot remove them. The benefit is not worth the cost.
Actually, I think I confused things for you by mentioning to `cast(ulong)`. I'm not asking for a Java-style "no unsigned" system (I hate that; it's one of my biggest annoyances with Java). Rather, I'm picking on *implicit* conversions between signed and unsigned.
No, I understood you meant implicit casting.
 I'm basically saying, "because information is lost when casting between
 signed and unsigned, all such casts should be explicit". This could make
 code rather verbose - except that from my experiments, with decent VRP
 the compiler can actually be surprisingly smart about warning only in
 those cases where implicit casting is really a bad idea.
Your definition of when "implicit casting is really a bad idea" is almost certainly going to include cases where it really isn't a bad idea. The compiler isn't all-knowing, and there will always be cases where the user knows best (and did the conversion intentionally). An obvious one is: void foo(ubyte[] x) { int len = x.length; } (let's assume 32-bit CPU) I'm assuming the compiler would complain about this, since technically, len could be negative! Disallowing such code or requiring a cast is probably too much. -Steve
Mar 21 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Monday, 21 March 2016 at 17:38:35 UTC, Steven Schveighoffer 
wrote:
 Your definition of when "implicit casting is really a bad idea" 
 is almost certainly going to include cases where it really 
 isn't a bad idea.
This logic can be applied to pretty much any warning condition or safety/correctness-related compiler feature; if it were followed consistently the compiler would just always trust the programmer, like an ancient C or C++ compiler with warnings turned off.
 The compiler isn't all-knowing, and there will always be cases 
 where the user knows best (and did the conversion 
 intentionally).
That's what explicit casts are for.
 An obvious one is:

 void foo(ubyte[] x)
 {
   int len = x.length;
 }

 (let's assume 32-bit CPU) I'm assuming the compiler would 
 complain about this, since technically, len could be negative! 
 Disallowing such code or requiring a cast is probably too much.
But that *is* a bad idea - there have been real-world bugs caused by doing stuff like that without checking. With respect to your specific example: 1) The memory limit on a true 32-bit system is 4GiB, not 2GiB. Even with an OS that reserves some of the address space, as much as 3GiB or 3.5GiB may be exposed to a user-space process in practice. 2) Many 32-bit CPUs have Physical Address Extension, which allows them to support way more than 4GiB. Even a non-PAE-aware process will probably still be offered at least 3GiB on such a system. 3) Just because your program is 32-bit, does *not* mean that it will only ever run on 32-bit CPUs. On a 64-bit system, a single 32-bit process could easily have access to ~3GiB of memory. 4) Even on an embedded system (which D doesn't really support right now, anyway) with a true, 2GiB memory limit, you still have the problem that such highly platform-dependent code is difficult to find and update when the time comes to port the code to more powerful hardware. These kinds of things are why D has fixed-size integer types: to encourage writing portable code, without too many invisible assumptions about the precise details of the execution environment.
Mar 21 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 3/21/16 4:27 PM, tsbockman wrote:
 On Monday, 21 March 2016 at 17:38:35 UTC, Steven Schveighoffer wrote:
 Your definition of when "implicit casting is really a bad idea" is
 almost certainly going to include cases where it really isn't a bad idea.
This logic can be applied to pretty much any warning condition or safety/correctness-related compiler feature; if it were followed consistently the compiler would just always trust the programmer, like an ancient C or C++ compiler with warnings turned off.
Right, if we were starting over, I'd say let's make sure you can't make these kinds of mistakes. We are not starting over though, and existing code will have intentional uses of the existing behavior that are NOT bugs. Even that may have been rejected by Walter since a goal is making C code easy to port. Note that we already have experience with such a thing: if(arr). Fixing is easy, just put if(arr.ptr). It was rejected because major users of this "feature" did not see any useful improvements -- all their usages were sound.
 The compiler isn't all-knowing, and there will always be cases where
 the user knows best (and did the conversion intentionally).
That's what explicit casts are for.
Existing code doesn't need to cast. People are lazy. I only would insert a cast if I needed to. Most valid code just works fine without casts, so you are going to flag lots of valid code as a nuisance.
 An obvious one is:

 void foo(ubyte[] x)
 {
   int len = x.length;
 }

 (let's assume 32-bit CPU) I'm assuming the compiler would complain
 about this, since technically, len could be negative! Disallowing such
 code or requiring a cast is probably too much.
But that *is* a bad idea - there have been real-world bugs caused by doing stuff like that without checking.
It depends on the situation. foo may know that x is going to be short enough to fit in an int. The question becomes, if 99% of cases the user knows that he was converting to a signed value intentionally, and in the remaining 1% of cases, 99% of those were harmless "errors", then this is going to be just a nuisance update, and it will fail to be accepted.
 With respect to your specific example:

 1) The memory limit on a true 32-bit system is 4GiB, not 2GiB. Even with
 an OS that reserves some of the address space, as much as 3GiB or 3.5GiB
 may be exposed to a user-space process in practice.
Then make it long len = x.length on a 64-bit system. Only reason I said assume it's 32-bit, is because on 64-bit CPU, using int is already an error. The architecture wasn't important for the example. -Steve
Mar 21 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Monday, 21 March 2016 at 22:29:46 UTC, Steven Schveighoffer 
wrote:
 It depends on the situation. foo may know that x is going to be 
 short enough to fit in an int.

 The question becomes, if 99% of cases the user knows that he 
 was converting to a signed value intentionally, and in the 
 remaining 1% of cases, 99% of those were harmless "errors", 
 then this is going to be just a nuisance update, and it will 
 fail to be accepted.
My experimentation strongly suggests that your "99.99% false positive" figure is way, *way* off. This stuff is both: 1) Harder for people to get right than you think (you can't develop good intuition about the extent of the problem, unless you spend some time thoroughly auditing existing code bases specifically looking for this kind of problem), and also 2) Easier for the compiler to figure out than you think - I was really surprised at how short the list of problems flagged by the compiler was, when I tested Lionello Lunesu's work on the current D codebase. The false positive rate would certainly be *much* lower than your outlandish 10,000 : 1 estimate, given a good compiler implementation.
 With respect to your specific example:

 1) The memory limit on a true 32-bit system is 4GiB, not 2GiB. 
 Even with
 an OS that reserves some of the address space, as much as 3GiB 
 or 3.5GiB
 may be exposed to a user-space process in practice.
Then make it long len = x.length on a 64-bit system. Only reason I said assume it's 32-bit, is because on 64-bit CPU, using int is already an error. The architecture wasn't important for the example.
Huh? The point of mine which you quoted applies specifically to 32-bit systems. 32-bit array lengths can be greater than `int.max`. Did you mean to reply to point #3, instead?
Mar 21 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 3/21/16 7:43 PM, tsbockman wrote:
 On Monday, 21 March 2016 at 22:29:46 UTC, Steven Schveighoffer wrote:
 It depends on the situation. foo may know that x is going to be short
 enough to fit in an int.

 The question becomes, if 99% of cases the user knows that he was
 converting to a signed value intentionally, and in the remaining 1% of
 cases, 99% of those were harmless "errors", then this is going to be
 just a nuisance update, and it will fail to be accepted.
My experimentation strongly suggests that your "99.99% false positive" figure is way, *way* off. This stuff is both:
Maybe, what would be a threshold that people would find acceptable?
 1) Harder for people to get right than you think (you can't develop good
 intuition about the extent of the problem, unless you spend some time
 thoroughly auditing existing code bases specifically looking for this
 kind of problem), and also
It matters not to the person who is very aware of the issue and doesn't write buggy code. His code "breaks" too. I would estimate that *most* uses of if(arr) in the wild were/are incorrect. However, in one particular user's code *0* were incorrect, even though he used it extensively. This kind of problem is what lead to the change being reverted. I suspect this change would be far more likely to create more headaches than help.
 2) Easier for the compiler to figure out than you think - I was really
 surprised at how short the list of problems flagged by the compiler was,
 when I tested Lionello Lunesu's work on the current D codebase.
This is highly subjective to whose code you use it on.
 The false positive rate would certainly be *much* lower than your
 outlandish 10,000 : 1 estimate, given a good compiler implementation.
I wouldn't say it's outlandish given my understanding of the problem. The question is, does the pain justify the update? I haven't run it against my code or any code really, but I can see how someone is very good at making correct uses of the implicit conversion.
 With respect to your specific example:

 1) The memory limit on a true 32-bit system is 4GiB, not 2GiB. Even with
 an OS that reserves some of the address space, as much as 3GiB or 3.5GiB
 may be exposed to a user-space process in practice.
Then make it long len = x.length on a 64-bit system. Only reason I said assume it's 32-bit, is because on 64-bit CPU, using int is already an error. The architecture wasn't important for the example.
Huh? The point of mine which you quoted applies specifically to 32-bit systems. 32-bit array lengths can be greater than `int.max`. Did you mean to reply to point #3, instead?
You seem to spend a lot of time focusing on 32-bit architecture, which was not my point at all. My point is that most arrays and uses are short enough to be handled with a signed value as the length. If this is a generic library function, sure, we should handle all possibilities. This doesn't mean someone's command line utility processing strings from the argument list should have to worry about that (as an example). Breaking perfectly good code is something we should strive against. -Steve
Mar 21 2016
parent tsbockman <thomas.bockman gmail.com> writes:
On Tuesday, 22 March 2016 at 00:18:54 UTC, Steven Schveighoffer 
wrote:
 On 3/21/16 7:43 PM, tsbockman wrote:
 The false positive rate would certainly be *much* lower than 
 your
 outlandish 10,000 : 1 estimate, given a good compiler 
 implementation.
I wouldn't say it's outlandish given my understanding of the problem. The question is, does the pain justify the update? I haven't run it against my code or any code really, but I can see how someone is very good at making correct uses of the implicit conversion.
Well that's the real problem here then, isn't it? I wouldn't want this stuff "fixed" either, if I thought false positives would outnumber useful warnings by 10,000 : 1. However, I already *know* that's not the case, from my own tests. But at this point I'm obviously not going to convince you, except by compiling some concrete statistics on what got flagged in some real code bases. And this I plan to do (in some form or other), once `checkedint` and/or the fix for DMD issue 259 are really ready. People can make an informed decision about the trade-offs then.
Mar 21 2016
prev sibling parent Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Thursday, 17 March 2016 at 17:09:46 UTC, Steven Schveighoffer 
wrote:
 On 3/16/16 6:37 PM, Mathias Lang wrote:
 On Wednesday, 16 March 2016 at 21:49:05 UTC, Steven 
 Schveighoffer wrote:
 No, please don't. Assigning a signed value to an unsigned 
 (and vice
 versa) is very useful, and there is no good reason to break 
 this.
I'm not talking about removing it completely. The implicit conversion should only happen when it's safe: ``` int s; if (s >= 0) // VRP saves the day { uint u = s; } ``` ``` uint u; if (u > short.max) throw new Exception("Argument out of range"); // Or `assert` short s = u; ```
Converting unsigned to signed or vice versa (of the same size type) is safe. No information is lost.
Strictly speaking yes, but typically, an `int` isn't used as a bit-pattern but as an integer (it's in the name). Such behaviour is very undesirable for integers.
 It's the comparison between the two which confuses the heck out 
 of people. I think we can solve 80% of the problems by just 
 fixing that.
That's probably true, anyway.
Mar 18 2016