www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Struct should be invalid after move

reply Sebastiaan Koppe <mail skoppe.eu> writes:
I have a non-copyable struct and I really want a compiler error 
whenever I access it after it has been moved.

---
struct Handle {
     ...
      disable this(this);
     ...
}

void main() {
     import std.algorithm : move;
     auto handle = getOne();
     auto second = handle.move;  /// line 14
     auto third = handle.move;    ///  <- compiler error, variable 
handle is invalid after line 14
}
---

I believe this would prevent some nasty bugs when dealing with 
these structs.

What do you think?
Nov 27 2018
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Tuesday, 27 November 2018 at 08:00:22 UTC, Sebastiaan Koppe 
wrote:
 I have a non-copyable struct and I really want a compiler error 
 whenever I access it after it has been moved.

 ---
 struct Handle {
     ...
      disable this(this);
     ...
 }

 void main() {
     import std.algorithm : move;
     auto handle = getOne();
     auto second = handle.move;  /// line 14
     auto third = handle.move;    ///  <- compiler error, 
 variable handle is invalid after line 14
 }
 ---

 I believe this would prevent some nasty bugs when dealing with 
 these structs.

 What do you think?
we need an ` invalidate`s attribute for that, would apply to pointers passed to free etc. Probably would need a DIP though. I like it.
Nov 27 2018
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Tuesday, 27 November 2018 at 08:37:32 UTC, Nicholas Wilson 
wrote:
 On Tuesday, 27 November 2018 at 08:00:22 UTC, Sebastiaan Koppe 
 wrote:
 I have a non-copyable struct and I really want a compiler 
 error whenever I access it after it has been moved.

 ---
 struct Handle {
     ...
      disable this(this);
     ...
 }

 void main() {
     import std.algorithm : move;
     auto handle = getOne();
     auto second = handle.move;  /// line 14
     auto third = handle.move;    ///  <- compiler error, 
 variable handle is invalid after line 14
 }
 ---

 I believe this would prevent some nasty bugs when dealing with 
 these structs.

 What do you think?
we need an ` invalidate`s attribute for that, would apply to pointers passed to free etc. Probably would need a DIP though. I like it.
Yeah, that could be an awesome addition. However, it's not as simple, because this should be legal too: ``` auto handle = getOne(); auto second = handle.move; // `handle` becomes invalid // ... handle = getOne(); // `handle` is valid again ```
Nov 27 2018
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Tue, Nov 27, 2018 at 1:08 AM Stanislav Blinov via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Tuesday, 27 November 2018 at 08:37:32 UTC, Nicholas Wilson
 wrote:
 On Tuesday, 27 November 2018 at 08:00:22 UTC, Sebastiaan Koppe
 wrote:
 I have a non-copyable struct and I really want a compiler
 error whenever I access it after it has been moved.

 ---
 struct Handle {
     ...
      disable this(this);
     ...
 }

 void main() {
     import std.algorithm : move;
     auto handle = getOne();
     auto second = handle.move;  /// line 14
     auto third = handle.move;    ///  <- compiler error,
 variable handle is invalid after line 14
 }
 ---

 I believe this would prevent some nasty bugs when dealing with
 these structs.

 What do you think?
we need an ` invalidate`s attribute for that, would apply to pointers passed to free etc. Probably would need a DIP though. I like it.
Yeah, that could be an awesome addition. However, it's not as simple, because this should be legal too: ``` auto handle = getOne(); auto second = handle.move; // `handle` becomes invalid // ... handle = getOne(); // `handle` is valid again ```
The language goes to great effort to return everything to it's `init` state after being moved or destroyed. The whole point of that is to cover the cases you are concerned about here. If it was invalid to access a thing after it's moved (or after destruction), then there's no reason for any of the work that resets thing to their `init` to happen... that's quite a different set of language semantics.
Nov 27 2018
parent reply kinke <noone nowhere.com> writes:
On Tuesday, 27 November 2018 at 19:51:12 UTC, Manu wrote:
 The language goes to great effort to return everything to it's 
 `init`
 state after being moved or destroyed. The whole point of that 
 is to
 cover the cases you are concerned about here.
 If it was invalid to access a thing after it's moved (or after
 destruction), then there's no reason for any of the work that 
 resets
 thing to their `init` to happen... that's quite a different set 
 of
 language semantics.
Yeah, I also totally fail to see a reason why a moved-from instance should be treated any different than a default-allocated one. Ownership taken away, reset to init, memory obviously still claimed and accessible as long as in scope.
Nov 27 2018
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 27 November 2018 at 22:49:44 UTC, kinke wrote:
 Yeah, I also totally fail to see a reason why a moved-from 
 instance should be treated any different than a 
 default-allocated one.
You are right, it shouldn't. A Handle.init or one after a move are both invalid. Invalid from the standpoint that a Handle should always refer to some underlying resource. And when it provably doesn't, I want to compiler to help me.
Nov 27 2018
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, November 27, 2018 12:51:12 PM MST Manu via Digitalmars-d wrote:
 The language goes to great effort to return everything to it's `init`
 state after being moved or destroyed. The whole point of that is to
 cover the cases you are concerned about here.
 If it was invalid to access a thing after it's moved (or after
 destruction), then there's no reason for any of the work that resets
 thing to their `init` to happen... that's quite a different set of
 language semantics.
I'd forgotten that move reset the variable to its init value. But given that it does, that pretty much does eliminate this entire problem. I can understand it if someone would still consider it a bug in their code to reuse the variable given that it then doesn't hold the value that it did before, but it's perfectly safe to access it at that point, and memory safety issues would have been the biggest reason to be concerned about touching the variable again after it's been moved. As such, having the compiler flag a variable being used after a move would be pretty much equivalent to having it flag it if you used a variable that was simply default-initialized rather than having been given an explicit value. - Jonathan M Davis
Nov 27 2018
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Wednesday, 28 November 2018 at 00:28:35 UTC, Jonathan M Davis 
wrote:
 As such, having the compiler flag a variable being used after a 
 move would be pretty much equivalent to having it flag it if 
 you used a variable that was simply default-initialized rather 
 than having been given an explicit value.
Yep. Exactly. I would love to opt-in such a feature via e.g.: struct Handle { ... disable Handle.init; ... }
Nov 27 2018
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, November 28, 2018 12:48:54 AM MST Sebastiaan Koppe via 
Digitalmars-d wrote:
 On Wednesday, 28 November 2018 at 00:28:35 UTC, Jonathan M Davis

 wrote:
 As such, having the compiler flag a variable being used after a
 move would be pretty much equivalent to having it flag it if
 you used a variable that was simply default-initialized rather
 than having been given an explicit value.
Yep. Exactly. I would love to opt-in such a feature via e.g.: struct Handle { ... disable Handle.init; ... }
If you have disable this(); in your struct, then that makes it illegal to default-initialize that type. However, it still has an init value (since _all_ types in D have init values), and so something like move would still use the init value, and any type introspection using init would continue to work. It's just the default initialization which is then illegal, forcing you to explicitly initialize all variables of that type. That being said, I wouldn't advise disabling default initialization unless you really need to, because a number of things rely on it (e.g. I'm not sure if you can stick a type which can't be default-initialized into an array, because certain array operations depend on default initialization). So, while it's occasionally useful, it's one of those things that tends to cause problems when you do it. - Jonathan M Davis
Nov 28 2018
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, November 27, 2018 1:00:22 AM MST Sebastiaan Koppe via 
Digitalmars-d wrote:
 I have a non-copyable struct and I really want a compiler error
 whenever I access it after it has been moved.

 ---
 struct Handle {
      ...
       disable this(this);
      ...
 }

 void main() {
      import std.algorithm : move;
      auto handle = getOne();
      auto second = handle.move;  /// line 14
      auto third = handle.move;    ///  <- compiler error, variable
 handle is invalid after line 14
 }
 ---

 I believe this would prevent some nasty bugs when dealing with
 these structs.

 What do you think?
Well, the DIP to add opPostMove has been approved, and if opPostMove is then disabled, then that should disable moving entirely (making it a compile error for it to happen at all, not to access the struct after it's been moved), which is probably a better approach. However, that will require that the DIP actually be implemented, and AFAIK, there's no ETA on that. - Jonathan M Davis
Nov 27 2018
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 27 November 2018 at 09:39:13 UTC, Jonathan M Davis 
wrote:
 Well, the DIP to add opPostMove has been approved, and if 
 opPostMove is then  disabled, then that should disable moving 
 entirely (making it a compile error for it to happen at all, 
 not to access the struct after it's been moved), which is 
 probably a better approach. However, that will require that the 
 DIP actually be implemented, and AFAIK, there's no ETA on that.

 - Jonathan M Davis
But I actually want to move it. Just so that the variable moved from is considered invalid afterwards.
Nov 27 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, November 27, 2018 3:32:40 AM MST Sebastiaan Koppe via 
Digitalmars-d wrote:
 On Tuesday, 27 November 2018 at 09:39:13 UTC, Jonathan M Davis

 wrote:
 Well, the DIP to add opPostMove has been approved, and if
 opPostMove is then  disabled, then that should disable moving
 entirely (making it a compile error for it to happen at all,
 not to access the struct after it's been moved), which is
 probably a better approach. However, that will require that the
 DIP actually be implemented, and AFAIK, there's no ETA on that.

 - Jonathan M Davis
But I actually want to move it. Just so that the variable moved from is considered invalid afterwards.
Well, if the compiler were doing the move, then there would be no move if there were any possibility of the variable being used after it was moved. So, basically, you're looking at the case where an system function that the compiler almost ceratinly does not understand is doing any moves has moved an object, and you want the compiler to then complain if the variable is used after the move. That would require some way for the compiler to be told that a move has occurred, which is likely possible, but I don't know how it could be reasonably done - especially since the move could be happening inside a function that the compiler has no visibility into. We'd probably have to add yet another attribute to indicate that a function is doing a move, and I think that most everyone would consider that to be a terrible idea. The other issue that comes to mind is that there are cases where you actually _do_ want to use a variable after a move has occurred - usually because you want to move something else into that variable (e.g. with swap). So, even if the compiler could perfectly detect when a move had occurred, it would then have to somehow know when using the variable again was then okay and when it wasn't. And I'm not sure that that's really solvable. If it already could detect all moves, then it could know whether another move was being attempted and consider that case safe, but I doubt that it could be made to understand when using opAssign would be safe. In many cases, it would not be, but for some types it would be (in particular when opAssign doesn't care about the previous value). So, it seems like trying to detect what you're trying to detect could easily require flagging cases which are actually valid as being illegal. All in all, at best, this just sounds like it would be adding a lot of extra complexity for detecting problems with an operation that is system by its very nature and which most programs aren't going to do. If you can come up with a simple way for the kind of problem that you want detected to be detected while not having it complain about valid code, then maybe it could be implemented, but thinking about it, I'm pretty sure that a new function attribute would be required, and I don't see that flying at this point. As it is, most folks think that D has too many attributes. So, any new ones have to really be worth it, whereas this just seems like it would be dealing with an uncommon edge case. It would certainly be valuable in that context, but I seriously question that it would be worth the extra language complexity. Also, remember that Walter is likely to shoot down most solutions (to pretty much any problem) that require code flow analysis, and this seems like the kind of problem that would require it (at least if it isn't going to have false positives or miss cases which it should flag). But if you want to solve this problem, then it looks to me like the first thing that you need to solve is how the compiler knows when a move has even occurred. And simply flagging common functions that do moves isn't going to be enough to catch all cases - especially when all it takes to hide the fact that a move occurred is to wrap the move call in another function that the compiler doesn't have visibility into thanks to separate compilation. - Jonathan M Davis
Nov 27 2018
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Tuesday, 27 November 2018 at 12:14:17 UTC, Jonathan M Davis 
wrote:

 Well, if the compiler were doing the move...
[wall snipped]
 And simply flagging common functions that do moves isn't going 
 to be enough to catch all cases - especially when all it takes 
 to hide the fact that a move occurred is to wrap the move call 
 in another function that the compiler doesn't have visibility 
 into thanks to separate compilation.
"Easy" enough. The move and emplace families should be compiler intrinsics, not library functions.
Nov 27 2018
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, November 27, 2018 5:37:29 AM MST Stanislav Blinov via 
Digitalmars-d wrote:
 On Tuesday, 27 November 2018 at 12:14:17 UTC, Jonathan M Davis

 wrote:
 Well, if the compiler were doing the move...
[wall snipped]
 And simply flagging common functions that do moves isn't going
 to be enough to catch all cases - especially when all it takes
 to hide the fact that a move occurred is to wrap the move call
 in another function that the compiler doesn't have visibility
 into thanks to separate compilation.
"Easy" enough. The move and emplace families should be compiler intrinsics, not library functions.
Well, they're not currently. And even if they were, that doesn't solve the problem of detecting when functions are called that do moves that the compiler does not have the source code for due to separate compilation or because the move is actually being done in C code that gets called from D code. Without that, at best, you're catching the really simple cases, and even that likely requires code flow analysis. It also has the problem that it gives the impression that the compiler catches a certain class of problem for you in general when in fact it only catches it in a few cases. It's basically getting into the same territory as detecting when a pointer or reference is never initialized with anything other null and making that an error - something that Walter has already refused to do, because only the most basic cases would be caught (especially if you want to avoid false positives), and he doesn't want to add features that require code flow analysis. Honestly, this seems like the sort of thing that's better suited to a linter that detects some cases where it thinks you might be accessing a variable in an invalid manner after it's been moved (allowing you to examine the code to see whether it's right or not), whereas the compiler has to get it right 100% of the time - especially if it's an error. - Jonathan M Davis
Nov 27 2018
prev sibling next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 27 November 2018 at 08:00:22 UTC, Sebastiaan Koppe 
wrote:
 I have a non-copyable struct and I really want a compiler error 
 whenever I access it after it has been moved.

 ---
 struct Handle {
     ...
      disable this(this);
     ...
 }

 void main() {
     import std.algorithm : move;
     auto handle = getOne();
     auto second = handle.move;  /// line 14
     auto third = handle.move;    ///  <- compiler error, 
 variable handle is invalid after line 14
 }
 ---

 I believe this would prevent some nasty bugs when dealing with 
 these structs.

 What do you think?
void foo(int a) { import std.algorithm : move; auto handle = getOne(); if (a > 0) auto second = handle.move; auto third = handle.move; // compile error? } Not a trivial problem in the general case.
Nov 27 2018
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 27 November 2018 at 09:50:25 UTC, John Colvin wrote:
 void foo(int a)
 {
     import std.algorithm : move;
     auto handle = getOne();
     if (a > 0)
         auto second = handle.move;
     auto third = handle.move; // compile error?
 }

 Not a trivial problem in the general case.
Not a trivial problem indeed. But I really believe we need these things (or similar) for safety's sake. One other issue I can see is if the handle is passed by ref into opaque function. At this point we can't reason about whether the struct is still valid after the call to bar: --- void bar(ref Handle); void main() { auto handle = getOne(); bar(handle); auto second = handle.move(); // compile error? don't know } --- Possible solution is to always disallow move after passing as ref. And maybe move could be disallowed on a scope ref param, then we could safely pass Handle to functions that take by ref while provably keeping the caller's Handle valid.
Nov 27 2018
parent reply Alex <sascha.orlov gmail.com> writes:
On Tuesday, 27 November 2018 at 10:53:21 UTC, Sebastiaan Koppe 
wrote:
 Not a trivial problem indeed. But I really believe we need 
 these things (or similar) for safety's sake.

 One other issue I can see is if the handle is passed by ref 
 into opaque function. At this point we can't reason about 
 whether the struct is still valid after the call to bar:

 ---
 void bar(ref Handle);

 void main() {
   auto handle = getOne();
   bar(handle);
   auto second = handle.move(); // compile error? don't know
 }
 ---

 Possible solution is to always disallow move after passing as 
 ref.

 And maybe move could be disallowed on a scope ref param, then 
 we could safely pass Handle to functions that take by ref while 
 provably keeping the caller's Handle valid.
There exist auto val = Handle.init; 1. How do you treat this? 2. Why do you don't want to treat the handle after movement the same way?
Nov 27 2018
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 27 November 2018 at 10:59:03 UTC, Alex wrote:
 There exist

 auto val = Handle.init;

 1. How do you treat this?
I have no idea. Logically I would say that - in my case - the val is invalid.
 2. Why do you don't want to treat the handle after movement the 
 same way?
Because the handle refers to an underlying resource, and any access to that resource through that handle is invalid after a move. Much like one doesn't want to call .release() twice on an unique(T) wrapper type. Sure, I could put in a runtime check, but then I get runtime errors and I rather have compile time errors.
Nov 27 2018
next sibling parent reply Alex <sascha.orlov gmail.com> writes:
On Tuesday, 27 November 2018 at 12:03:20 UTC, Sebastiaan Koppe 
wrote:
 On Tuesday, 27 November 2018 at 10:59:03 UTC, Alex wrote:
 There exist

 auto val = Handle.init;

 1. How do you treat this?
I have no idea. Logically I would say that - in my case - the val is invalid.
This is a source of bugs. The T.init value is the one and only value, which exists, independently from the definition (of Handle). Therefore, your own functions on Handle have to consider this case and have to behave properly. I.e. define some behavior. I mean, what would happen, if you take an existent function and pass Handle.init to it? Would it assert? Would it yield 42? Some reproducible behavior is defined well enough.
 2. Why do you don't want to treat the handle after movement 
 the same way?
Because the handle refers to an underlying resource, and any access to that resource through that handle is invalid after a move. Much like one doesn't want to call .release() twice on an unique(T) wrapper type. Sure, I could put in a runtime check, but then I get runtime errors and I rather have compile time errors.
This is not an answer to my question. The question was: why you would like to have a different behavior w.r.t. some other behavior, which exists, independent of your actions.
Nov 27 2018
next sibling parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Tuesday, 27 November 2018 at 12:42:43 UTC, Alex wrote:
 On Tuesday, 27 November 2018 at 12:03:20 UTC, Sebastiaan Koppe 
 wrote:
 auto val = Handle.init;

 1. How do you treat this?
I have no idea. Logically I would say that - in my case - the val is invalid.
This is a source of bugs. The T.init value is the one and only value, which exists, independently from the definition (of Handle). Therefore, your own functions on Handle have to consider this case and have to behave properly. I.e. define some behavior.
No. The only thing that must be valid to do with a value in .init state is destruct it. Standard library move et al. assume as much. In fact, what they do is exactly that - wipe out the moved from value, emplacing the initializer there. What also *may* be valid is reinitialization/assignment.
 Sure, I could put in a runtime check, but then I get runtime 
 errors and I rather have compile time errors.
This is not an answer to my question. The question was: why you would like to have a different behavior w.r.t. some other behavior, which exists, independent of your actions.
Because .init does not generally exert any useful behavior.
Nov 27 2018
parent reply Alex <sascha.orlov gmail.com> writes:
On Tuesday, 27 November 2018 at 12:51:58 UTC, Stanislav Blinov 
wrote:
 On Tuesday, 27 November 2018 at 12:42:43 UTC, Alex wrote:
 On Tuesday, 27 November 2018 at 12:03:20 UTC, Sebastiaan Koppe 
 wrote:
 auto val = Handle.init;

 1. How do you treat this?
I have no idea. Logically I would say that - in my case - the val is invalid.
This is a source of bugs. The T.init value is the one and only value, which exists, independently from the definition (of Handle). Therefore, your own functions on Handle have to consider this case and have to behave properly. I.e. define some behavior.
No. The only thing that must be valid to do with a value in .init state is destruct it.
I have something completely different in mind... std.traits is full of examples, where T.init is used beyond of destruction. And the behavior of a function for a .init value is also defined at all times. Independently from how the function is written. It can be easily tested by plugging the .init value as the input parameter.
 Standard library move et al. assume as much. In fact, what they 
 do is exactly that - wipe out the moved from value, emplacing 
 the initializer there. What also *may* be valid is 
 reinitialization/assignment.
Sure. And I don't question this. I just would like to know, how the custom functions of OP handle the case of .init. I mean... If they don't, then they assume the value is not in the .init state and perform some actions with it, which can lead to some assert. Which would be perfectly ok.
 Sure, I could put in a runtime check, but then I get runtime 
 errors and I rather have compile time errors.
This is not an answer to my question. The question was: why you would like to have a different behavior w.r.t. some other behavior, which exists, independent of your actions.
Because .init does not generally exert any useful behavior.
Hmm... sure. But some behavior exists. And my question is: why (not even how) the behavior should differ between the .init case and the case after a move.
Nov 27 2018
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Tuesday, 27 November 2018 at 13:17:24 UTC, Alex wrote:

 I have something completely different in mind...
 std.traits is full of examples, where T.init is used beyond of 
 destruction.
 And the behavior of a function for a .init value is also 
 defined at all times. Independently from how the function is 
 written. It can be easily tested by plugging the .init value as 
 the input parameter.
For compile-time checks, sure, as it is a convenient way to get at the underlying type. But that's all that is.
 Standard library move et al. assume as much. In fact, what 
 they do is exactly that - wipe out the moved from value, 
 emplacing the initializer there. What also *may* be valid is 
 reinitialization/assignment.
Sure. And I don't question this. I just would like to know, how the custom functions of OP handle the case of .init. I mean... If they don't, then they assume the value is not in the .init state and perform some actions with it, which can lead to some assert. Which would be perfectly ok.
Right now we have to resort to runtime checks. Which is hilarious considering we *can* disable default construction, thereby asserting that .init is indeed an invalid state.
 This is not an answer to my question. The question was: why 
 you would like to have a different behavior w.r.t. some other 
 behavior, which exists, independent of your actions.
Because .init does not generally exert any useful behavior.
Hmm... sure. But some behavior exists. And my question is: why (not even how) the behavior should differ between the .init case and the case after a move.
What Sebastiaan proposes is for there to be *no* behavior at all, as using an invalid value would become a compile-time error. Which would mean one wouldn't need extraneous run-time checks.
Nov 27 2018
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 27 November 2018 at 14:37:40 UTC, Stanislav Blinov 
wrote:
 What Sebastiaan proposes is for there to be *no* behavior at 
 all, as using an invalid value would become a compile-time 
 error. Which would mean one wouldn't need extraneous run-time 
 checks.
Yes. The only valid thing to do would be to move an existing Handle into it (explicitly or as a result of a function).
Nov 27 2018
prev sibling parent reply Alex <sascha.orlov gmail.com> writes:
On Tuesday, 27 November 2018 at 14:37:40 UTC, Stanislav Blinov 
wrote:
 On Tuesday, 27 November 2018 at 13:17:24 UTC, Alex wrote:

 For compile-time checks, sure, as it is a convenient way to get 
 at the underlying type. But that's all that is.

 Right now we have to resort to runtime checks. Which is 
 hilarious considering we *can* disable default construction, 
 thereby asserting that .init is indeed an invalid state.
Ok... so. Without default construction, int i; and int i = void; become the same. But you won't be able to prevent it from being an int, right? How would you detect an invalid state in this case?
 What Sebastiaan proposes is for there to be *no* behavior at 
 all, as using an invalid value would become a compile-time 
 error. Which would mean one wouldn't need extraneous run-time 
 checks.
I assume, that detecting invalid states becomes harder as types become simpler. And, it depends strongly on the application, whether a value is considered invalid... Doesn't Typedef already allow this functionality?
Nov 27 2018
parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Tuesday, 27 November 2018 at 15:14:13 UTC, Alex wrote:
 On Tuesday, 27 November 2018 at 14:37:40 UTC, Stanislav Blinov 
 wrote:
 On Tuesday, 27 November 2018 at 13:17:24 UTC, Alex wrote:

 For compile-time checks, sure, as it is a convenient way to 
 get at the underlying type. But that's all that is.

 Right now we have to resort to runtime checks. Which is 
 hilarious considering we *can* disable default construction, 
 thereby asserting that .init is indeed an invalid state.
Ok... so. Without default construction, int i; and int i = void; become the same. But you won't be able to prevent it from being an int, right? How would you detect an invalid state in this case?
You wouldn't. int doesn't have an invalid state. That's not what's in question here. This is: ``` struct S { private X* ptr; disable this(); disable this(this); this(X* p) { ptr = p; } ref get() return safe { return *p; } alias get this; } ``` ...or pointers in general.
 What Sebastiaan proposes is for there to be *no* behavior at 
 all, as using an invalid value would become a compile-time 
 error. Which would mean one wouldn't need extraneous run-time 
 checks.
I assume, that detecting invalid states becomes harder as types become simpler.
There's no difference. If the compiler would know what an invalid state is and what operations result in it, it doesn't matter what the type is.
 And, it depends strongly on the application, whether a value is 
 considered invalid... Doesn't Typedef already allow this 
 functionality?
No, it does not. We are talking about complie-time checking, not runtime checking.
Nov 27 2018
prev sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 27 November 2018 at 12:42:43 UTC, Alex wrote:
 On Tuesday, 27 November 2018 at 12:03:20 UTC, Sebastiaan Koppe
 I have no idea. Logically I would say that - in my case - the 
 val is invalid.
This is a source of bugs. The T.init value is the one and only value, which exists, independently from the definition (of Handle). Therefore, your own functions on Handle have to consider this case and have to behave properly. I.e. define some behavior. I mean, what would happen, if you take an existent function and pass Handle.init to it? Would it assert? Would it yield 42? Some reproducible behavior is defined well enough.
You are right, Handle.init is in the same state as it were after a move, so you would expect similar behaviour.
Nov 27 2018
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 27 November 2018 at 12:03:20 UTC, Sebastiaan Koppe 
wrote:
 On Tuesday, 27 November 2018 at 10:59:03 UTC, Alex wrote:
 There exist

 auto val = Handle.init;

 1. How do you treat this?
I have no idea. Logically I would say that - in my case - the val is invalid.
 2. Why do you don't want to treat the handle after movement 
 the same way?
Because the handle refers to an underlying resource, and any access to that resource through that handle is invalid after a move. Much like one doesn't want to call .release() twice on an unique(T) wrapper type. Sure, I could put in a runtime check, but then I get runtime errors and I rather have compile time errors.
So ... what you want is this: "Compiler, ensure that there is no possible execution-flow path in which this variable is ever set to 0." We can actually attempt to prove that, using SMT or other logic solvers, though it will most likely say that there is set up inputs and flow-decisions which will result in the variable set to 0, and it can even give you an example. You can then take that example and remove the invalid case, for the input provided. This is a massive amount of work to get right.
Nov 28 2018
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Wednesday, 28 November 2018 at 09:11:48 UTC, Stefan Koch wrote:
 So ... what you want is this:

 "Compiler, ensure that there is no possible execution-flow path 
 in which this variable is ever set to 0."
Almost. I want to prevent usage *after* it is set to 0. Similar to how the compiler already recognises this error: --- void main() { int* p = null; (*p) = 5; // <- Error: null dereference in function _Dmain } --- Which more and more languages already provide. Swift, Kotlin etc.
Nov 28 2018
parent John Colvin <john.loughran.colvin gmail.com> writes:
On Wednesday, 28 November 2018 at 09:30:21 UTC, Sebastiaan Koppe 
wrote:
 On Wednesday, 28 November 2018 at 09:11:48 UTC, Stefan Koch 
 wrote:
 So ... what you want is this:

 "Compiler, ensure that there is no possible execution-flow 
 path in which this variable is ever set to 0."
Almost. I want to prevent usage *after* it is set to 0. Similar to how the compiler already recognises this error: --- void main() { int* p = null; (*p) = 5; // <- Error: null dereference in function _Dmain } --- Which more and more languages already provide. Swift, Kotlin etc.
Sure, catching some simple cases is definitely doable, if someone has the compiler chops (or is prepared to learn) and some time to dedicate to it. I suspect if it was a small code change and had a zero false-positive rate (I.e. only caught things that *definitely* were wrong) then it would be accepted. Large code change -> uncertain. Any false positives that forbid valid programs -> unlikely to be accepted. That's just my feeling though, who knows....
Nov 28 2018
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/27/18 3:00 AM, Sebastiaan Koppe wrote:
 I have a non-copyable struct and I really want a compiler error whenever 
 I access it after it has been moved.
 
 ---
 struct Handle {
      ...
       disable this(this);
      ...
 }
 
 void main() {
      import std.algorithm : move;
      auto handle = getOne();
      auto second = handle.move;  /// line 14
      auto third = handle.move;    ///  <- compiler error, variable 
 handle is invalid after line 14
 }
 ---
 
 I believe this would prevent some nasty bugs when dealing with these 
 structs.
 
 What do you think?
You can do a runtime error (with opPostMove included), but I think a compiler error would require full flow analysis, and would have to default to allowing it, as otherwise you will get false errors that are more annoying than the original problem. -Steve
Nov 27 2018
parent reply burjui <bytefu gmail.com> writes:
On Tuesday, 27 November 2018 at 15:27:19 UTC, Steven 
Schveighoffer wrote:
 You can do a runtime error (with opPostMove included), but I 
 think a compiler error would require full flow analysis, and 
 would have to default to allowing it, as otherwise you will get 
 false errors that are more annoying than the original problem.
A good modern compiler of a statically compiled language has to have flow analysis enabled unconditionally. I don't see any reason not to enable it in D frontend, except Walter's stubbornness. The current situation with D doesn't even make sense: with optimizations enabled, the compiler can reject code that compiles just fine without them. I find it sad and ridiculous at the same time. This happens exactly because dataflow analysis is only enabled when optimizations are enabled: void main() { int* x; *x = 0; } This compiles just fine with `dmd x.d`, but emits an error with `dmd -O x.d`: Error: null dereference in function _Dmain
Nov 27 2018
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/27/18 11:49 AM, burjui wrote:
 On Tuesday, 27 November 2018 at 15:27:19 UTC, Steven Schveighoffer wrote:
 You can do a runtime error (with opPostMove included), but I think a 
 compiler error would require full flow analysis, and would have to 
 default to allowing it, as otherwise you will get false errors that 
 are more annoying than the original problem.
A good modern compiler of a statically compiled language has to have flow analysis enabled unconditionally.
Does g++ count as a modern compiler? It doesn't do flow analysis on the code you have below.
 I 
 find it sad and ridiculous at the same time. This happens exactly 
 because dataflow analysis is only enabled when optimizations are enabled:
 
 void main()
 {
      int* x;
      *x = 0;
 }
 
 This compiles just fine with `dmd x.d`, but emits an error with `dmd -O 
 x.d`:
 Error: null dereference in function _Dmain
Note that complete flow analysis is equivalent to the halting problem. So really, you can never get it perfect, and therefore, there are always going to be some things you can't catch. So it's impossible to fulfill that promise in any compiler. In any case, I can't imagine how to do it without some handshaking between std.algorithm.move and the compiler, which is likely to be met with resistance. -Steve
Nov 27 2018
next sibling parent Paul Backus <snarwin gmail.com> writes:
On Tuesday, 27 November 2018 at 19:28:13 UTC, Steven 
Schveighoffer wrote:
 void main()
 {
      int* x;
      *x = 0;
 }
 
 This compiles just fine with `dmd x.d`, but emits an error 
 with `dmd -O x.d`:
 Error: null dereference in function _Dmain
Note that complete flow analysis is equivalent to the halting problem. So really, you can never get it perfect, and therefore, there are always going to be some things you can't catch. So it's impossible to fulfill that promise in any compiler.
It's possible if you're willing to make the rules strict enough that some otherwise-valid programs get rejected--that's what Rust does, after all. But for D, that isn't really an option.
Nov 27 2018
prev sibling parent reply burjui <bytefu gmail.com> writes:
On Tuesday, 27 November 2018 at 19:28:13 UTC, Steven 
Schveighoffer wrote:
 Note that complete flow analysis is equivalent to the halting 
 problem. So really, you can never get it perfect, and 
 therefore, there are always going to be some things you can't 
 catch. So it's impossible to fulfill that promise in any 
 compiler.
That's a straw man. I was not talking about complete flow analysis and getting it perfect. Some form of flow analysis is already build into DMD, so I suggest to: 1. At least enable it unconditionally 2. Improve it to account for more complex scenarios than simple null dereferencing Anyway, the point I am trying to make here is that it's much better to make one compiler smarter, than to rely on many users' not-even-close-to-perfect discipline, and I think C++ proves it. Software development these days is too complex to comprehend without the help of tools, which must become smarter to really have an impact and stay in use.
Nov 30 2018
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/30/18 10:01 AM, burjui wrote:
 On Tuesday, 27 November 2018 at 19:28:13 UTC, Steven Schveighoffer wrote:
 Note that complete flow analysis is equivalent to the halting problem. 
 So really, you can never get it perfect, and therefore, there are 
 always going to be some things you can't catch. So it's impossible to 
 fulfill that promise in any compiler.
That's a straw man. I was not talking about complete flow analysis and getting it perfect. Some form of flow analysis is already build into DMD, so I suggest to: 1. At least enable it unconditionally
I'm not a compiler writer, and don't know the first thing about the requirements, but I would hazard a guess that the reason this is the case is if optimization isn't enabled, some critical pieces needed to do flow analysis aren't present. I don't think it's orthogonal.
 2. Improve it to account for more complex scenarios than simple null 
 dereferencing
I think any improvements to the flow analysis would be welcome! The more the compiler can flag as an obvious error, the better.
 Anyway, the point I am trying to make here is that it's much better to 
 make one compiler smarter, than to rely on many users' 
 not-even-close-to-perfect discipline, and I think C++ proves it. 
 Software development these days is too complex to comprehend without the 
 help of tools, which must become smarter to really have an impact and 
 stay in use.
Best effort to detect problems is generally what we have with D. My point was simply that any time you solve "how come it doesn't detect this", someone will highlight another problem that is harder or impossible to solve. -Steve
Nov 30 2018
prev sibling parent reply sanjayss <dummy dummy.dummy> writes:
On Tuesday, 27 November 2018 at 08:00:22 UTC, Sebastiaan Koppe 
wrote:
 I have a non-copyable struct and I really want a compiler error 
 whenever I access it after it has been moved.

 ---
 struct Handle {
     ...
      disable this(this);
     ...
 }

 void main() {
     import std.algorithm : move;
     auto handle = getOne();
     auto second = handle.move;  /// line 14
     auto third = handle.move;    ///  <- compiler error, 
 variable handle is invalid after line 14
 }
 ---

 I believe this would prevent some nasty bugs when dealing with 
 these structs.

 What do you think?
I have always wanted a feature in C that would let me explicitly tell the compiler that a variable is no longer in scope (some sort of unset of a variable). This would be useful to do defensive programming against use-after-free of pointers to allocated memory and such. Though it would be nice to have the compiler auto-detect these kinds of things, maybe having the programmer explicitly request this might be way easier to implement.
Nov 28 2018
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Wednesday, 28 November 2018 at 09:17:39 UTC, sanjayss wrote:
 I have always wanted a feature in C that would let me 
 explicitly tell the compiler that a variable is no longer in 
 scope (some sort of unset of a variable). This would be useful 
 to do defensive programming against use-after-free of pointers 
 to allocated memory and such.

 Though it would be nice to have the compiler auto-detect these 
 kinds of things, maybe having the programmer explicitly request 
 this might be way easier to implement.
Yeah. It would be awesome if the programmer could 'taint' a variable as invalid. Even if that doesn't propagate in all cases.
Nov 28 2018
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 28 November 2018 at 09:17:39 UTC, sanjayss wrote:
 On Tuesday, 27 November 2018 at 08:00:22 UTC, Sebastiaan Koppe 
 wrote:
 [...]
I have always wanted a feature in C that would let me explicitly tell the compiler that a variable is no longer in scope (some sort of unset of a variable). This would be useful to do defensive programming against use-after-free of pointers to allocated memory and such.
{ void* ptr = malloc(5); } // ptr no longer in scope
Nov 28 2018
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Nov 28, 2018 at 04:49:02PM +0000, Atila Neves via Digitalmars-d wrote:
 On Wednesday, 28 November 2018 at 09:17:39 UTC, sanjayss wrote:
 On Tuesday, 27 November 2018 at 08:00:22 UTC, Sebastiaan Koppe wrote:
 [...]
I have always wanted a feature in C that would let me explicitly tell the compiler that a variable is no longer in scope (some sort of unset of a variable). This would be useful to do defensive programming against use-after-free of pointers to allocated memory and such.
{ void* ptr = malloc(5); } // ptr no longer in scope
Yeah, this seems to be a not-very-well-known aspect of syntax common across C, C++, Java, and D: ReturnType myFunc(Args args) { int var1; someCode(); { int var2; someOtherCode(); } // var2 no longer in scope, but var1 still is. { // Can reuse identifier 'var2' without conflict float var2; yetMoreCode(); } etcetera(); } It's a very useful construct in ensuring that temporaries don't last longer than they ought to. (Syntactically anyway... the compiler may or may not actually translate that directly in the executable. But the point is to avoid programmer slip-ups.) T -- In theory, there is no difference between theory and practice.
Nov 28 2018
parent sanjayss <dummy dummy.dummy> writes:
On Wednesday, 28 November 2018 at 16:58:32 UTC, H. S. Teoh wrote:
 Yeah, this seems to be a not-very-well-known aspect of syntax 
 common across C, C++, Java, and D:

 	ReturnType myFunc(Args args) {
 		int var1;
 		someCode();
 		{
 			int var2;
 			someOtherCode();
 		}
 		// var2 no longer in scope, but var1 still is.
 		{
 			// Can reuse identifier 'var2' without conflict
 			float var2;
 			yetMoreCode();
 		}
 		etcetera();
 	}

 It's a very useful construct in ensuring that temporaries don't 
 last longer than they ought to. (Syntactically anyway... the 
 compiler may or may not actually translate that directly in the 
 executable. But the point is to avoid programmer slip-ups.)


 T
Yes, you can get what I asked for using blocks, but there are use cases where you can't always use blocks effectively. Also there is the aesthetic aspect -- all the curly braces and indentation makes code annoying to read.
Nov 28 2018
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/28/18 11:49 AM, Atila Neves wrote:
 On Wednesday, 28 November 2018 at 09:17:39 UTC, sanjayss wrote:
 On Tuesday, 27 November 2018 at 08:00:22 UTC, Sebastiaan Koppe wrote:
 [...]
I have always wanted a feature in C that would let me explicitly tell the compiler that a variable is no longer in scope (some sort of unset of a variable). This would be useful to do defensive programming against use-after-free of pointers to allocated memory and such.
{     void* ptr = malloc(5); } // ptr no longer in scope
I think what the request is that you have variables with overlapping scope. For example: void *ptr1 = malloc(5); void *ptr2 = malloc(5); ... free(ptr1); // end ptr1 scope ... free(ptr2); // end ptr2 scope Which isn't possible by adding a nested scope. But in any case, this doesn't fix all the problems anyway, you could have another alias to the same data, free that alias, and then ptr1 is still "valid". -Steve
Nov 28 2018
parent reply Neia Neutuladh <neia ikeran.org> writes:
On Wed, 28 Nov 2018 12:47:07 -0500, Steven Schveighoffer wrote:
 But in any case, this doesn't fix all the problems anyway, you could
 have another alias to the same data, free that alias, and then ptr1 is
 still "valid".
Move constructors and init'ing out the moved variable address that, no?
Nov 28 2018
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/28/18 1:59 PM, Neia Neutuladh wrote:
 On Wed, 28 Nov 2018 12:47:07 -0500, Steven Schveighoffer wrote:
 But in any case, this doesn't fix all the problems anyway, you could
 have another alias to the same data, free that alias, and then ptr1 is
 still "valid".
Move constructors and init'ing out the moved variable address that, no?
Not sure what you mean. What I'm talking about is this: int *ptr1 = (int *)malloc(sizeof(int)); int *ptr2 = ptr1; free(ptr2); *ptr1 = 5; If we have a hypothetical C compiler that prevents use of ptr2 after free, it probably doesn't know about the link to ptr1, to make that also unusable. Indeed the runtime solution is much more possible. -Steve
Nov 28 2018
next sibling parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Wednesday, 28 November 2018 at 19:11:14 UTC, Steven 
Schveighoffer wrote:

 int *ptr1 = (int *)malloc(sizeof(int));
 int *ptr2 = ptr1;

 free(ptr2);

 *ptr1 = 5;

 If we have a hypothetical C compiler that prevents use of ptr2 
 after free, it probably doesn't know about the link to ptr1, to 
 make that also unusable.

 Indeed the runtime solution is much more possible.
For that (or rather, to prevent that) we have non-copyable types. Like the one in the OP.
Nov 28 2018
prev sibling parent Neia Neutuladh <neia ikeran.org> writes:
On Wed, 28 Nov 2018 14:11:14 -0500, Steven Schveighoffer wrote:
 Not sure what you mean. What I'm talking about is this:
 
 int *ptr1 = (int *)malloc(sizeof(int));
 int *ptr2 = ptr1;
 
 free(ptr2);
 
 *ptr1 = 5;
Yes, and the compiler can't detect this very often (at least not without an ownership / borrowing system like Rust or Vala). Instead, you can eliminate aliasing, introduce reference counting or a garbage collector to avoid manually freeing, or use indirection to detect when the value is freed. There are a bunch of solutions that work some of the time. The end result is a bit of code that doesn't fit with any of it and needs special caution.
Nov 28 2018