www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - An Issue I Wish To Raise Awareness On

reply Jack Stouffer <jack jackstouffer.com> writes:
TL;DR: Issue 17658 [1] makes using shared very 
annoying/practically impossible.

Over the weekend, I attempted to fix Issue 15768 [2], which is an 
underlying problem in the std.stdio.File struct that stops it 
from closing properly when shared across threads. This is a big 
problem for users because stdin/out/err are all __gshared File, 
and are therefore unsafe. This makes for a really annoying 
situation where writeln("a") is  safe but stderr.writeln("a") 
isn't.

The obvious solution is to make stdin/out/err shared(File) and 
modify File to have shared overloads which either lock or use 
atomic ops safely. When I tried to write said functions, I ran 
into compilation issues that I couldn't diagnose until I ran 
across this thread by Atila [3]. The problem is, unlike a 
constructor, destructors and post-blits can't be overloaded with 
shared variants. Consider:

```
struct A
{
     this(string a) {}
     this(string a) shared {}

     ~this() {}
     ~this() shared {}

     this(this) {}
     this(this) shared {}
}

void main()
{
     shared f = A("");
}
```

Error: destructor f152.A.~this conflicts with destructor 
f152.A.~this at /d422/f152.d(6)
Error: function f152.A.__postblit conflicts with function 
f152.A.__postblit at /d422/f152.d(9)

This is further compounded with this

```
struct A
{
	~this() {}
}

void main()
{
	auto a = A();
	shared b = A();
}
```

Error: non-shared method f585.A.~this is not callable using a 
shared object

The only way to work around this is to create a new type that is 
defined as shared struct and copy over all of the code from the 
original type. This really hobbles shared in any real world 
context.

I ask that someone who knows the DMD code base could please take 
a look at this and see if this is something that can be fixed 
easily and without breakage.

[1] https://issues.dlang.org/show_bug.cgi?id=17658
[2] https://issues.dlang.org/show_bug.cgi?id=15768
[3] 
https://forum.dlang.org/post/sqazguejrcdtjimtjxtz forum.dlang.org
Jul 17
parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 17 July 2017 at 14:26:19 UTC, Jack Stouffer wrote:
 TL;DR: Issue 17658 [1] makes using shared very 
 annoying/practically impossible.

 [...]
I fixed this already, should be in the next release. Atila
Jul 17
parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Monday, 17 July 2017 at 17:41:58 UTC, Atila Neves wrote:
 On Monday, 17 July 2017 at 14:26:19 UTC, Jack Stouffer wrote:
 TL;DR: Issue 17658 [1] makes using shared very 
 annoying/practically impossible.

 [...]
I fixed this already, should be in the next release. Atila
Are you sure? Because DMD nightly still errors: https://run.dlang.io?compiler=dmd-nightly&source=struct%20A%0A%7B%0A%20%20%20%20this(string%20a)%20%7B%7D%0A%20%20%20%20this(string%20a)%20shared%20%7B%7D%0A%0A%20%20%20%20~this()%20%7B%7D%0A%20%20%20%20~this()%20shared%20%7B%7D%0A%0A%20%20%20%20this(this)%20%7B%7D%0A%20%20%20%20this(this)%20shared%20%7B%7D%0A%7D%0A%0Avoid%20main()%0A%7B%0A%20%20%20%20shared%20f%20%3D%20A(%22%22)%3B%0A%7D (maybe we should remove the ban on URL shorteners for our own sites)
Jul 17
next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Monday, 17 July 2017 at 19:30:53 UTC, Jack Stouffer wrote:
 On Monday, 17 July 2017 at 17:41:58 UTC, Atila Neves wrote:
 On Monday, 17 July 2017 at 14:26:19 UTC, Jack Stouffer wrote:
 TL;DR: Issue 17658 [1] makes using shared very 
 annoying/practically impossible.

 [...]
I fixed this already, should be in the next release. Atila
Are you sure? Because DMD nightly still errors: https://run.dlang.io?compiler=dmd-nightly&source=struct%20A%0A%7B%0A%20%20%20%20this(string%20a)%20%7B%7D%0A%20%20%20%20this(string%20a)%20shared%20%7B%7D%0A%0A%20%20%20%20~this()%20%7B%7D%0A%20%20%20%20~this()%20shared%20%7B%7D%0A%0A%20%20%20%20this(this)%20%7B%7D%0A%20%20%20%20this(this)%20shared%20%7B%7D%0A%7D%0A%0Avoid%20main()%0A%7B%0A%20%20%20%20shared%20f%20%3D%20A(%22%22)%3B%0A%7D (maybe we should remove the ban on URL shorteners for our own sites)
I think Atila was talking about this one: struct A { ~this() {} } void main() { auto a = A(); shared b = A(); } https://is.gd/kOYlWY
Jul 18
next sibling parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 18 July 2017 at 11:47:37 UTC, Petar Kirov 
[ZombineDev] wrote:
 On Monday, 17 July 2017 at 19:30:53 UTC, Jack Stouffer wrote:
 On Monday, 17 July 2017 at 17:41:58 UTC, Atila Neves wrote:
 On Monday, 17 July 2017 at 14:26:19 UTC, Jack Stouffer wrote:
 TL;DR: Issue 17658 [1] makes using shared very 
 annoying/practically impossible.

 [...]
I fixed this already, should be in the next release. Atila
Are you sure? Because DMD nightly still errors: https://run.dlang.io?compiler=dmd-nightly&source=struct%20A%0A%7B%0A%20%20%20%20this(string%20a)%20%7B%7D%0A%20%20%20%20this(string%20a)%20shared%20%7B%7D%0A%0A%20%20%20%20~this()%20%7B%7D%0A%20%20%20%20~this()%20shared%20%7B%7D%0A%0A%20%20%20%20this(this)%20%7B%7D%0A%20%20%20%20this(this)%20shared%20%7B%7D%0A%7D%0A%0Avoid%20main()%0A%7B%0A%20%20%20%20shared%20f%20%3D%20A(%22%22)%3B%0A%7D (maybe we should remove the ban on URL shorteners for our own sites)
I think Atila was talking about this one: struct A { ~this() {} } void main() { auto a = A(); shared b = A(); } https://is.gd/kOYlWY
https://github.com/dlang/dmd/pull/6752
Jul 18
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
On Tuesday, 18 July 2017 at 11:47:37 UTC, Petar Kirov 
[ZombineDev] wrote:
 I think Atila was talking about this one:
 struct A
 {
 	~this() {}
 }

 void main()
 {
 	auto a = A();
 	shared b = A();
 }
This is strange. There's nothing that suggests that struct A and its destructor is thread-safe, yet compiler assumes it is.
Jul 18
parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 18 July 2017 at 15:03:07 UTC, Kagamin wrote:
 On Tuesday, 18 July 2017 at 11:47:37 UTC, Petar Kirov 
 [ZombineDev] wrote:
 I think Atila was talking about this one:
 struct A
 {
 	~this() {}
 }

 void main()
 {
 	auto a = A();
 	shared b = A();
 }
This is strange. There's nothing that suggests that struct A and its destructor is thread-safe, yet compiler assumes it is.
Except for a programmer explicitly and manually calling the destructor (in which case, don't), the destructor is only ever called by one thread. Atila
Jul 18
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tuesday, July 18, 2017 18:06:56 Atila Neves via Digitalmars-d wrote:
 On Tuesday, 18 July 2017 at 15:03:07 UTC, Kagamin wrote:
 On Tuesday, 18 July 2017 at 11:47:37 UTC, Petar Kirov

 [ZombineDev] wrote:
 I think Atila was talking about this one:
 struct A
 {

    ~this() {}

 }

 void main()
 {

    auto a = A();
    shared b = A();

 }
This is strange. There's nothing that suggests that struct A and its destructor is thread-safe, yet compiler assumes it is.
Except for a programmer explicitly and manually calling the destructor (in which case, don't), the destructor is only ever called by one thread.
It could still be a problem if the struct has a member variable that is a reference type, because then something else could refer to that object, and if it's shared, then you would need to protect it, and the operations that shared prevents should still be prevented. For full-on value types, it should be a non-issue though. - Jonathan M Davis
Jul 18
next sibling parent reply Kagamin <spam here.lot> writes:
On Tuesday, 18 July 2017 at 19:24:18 UTC, Jonathan M Davis wrote:
 For full-on value types, it should be a non-issue though.
Not quite. Value types include resource identifiers, which may have threading requirements, e.g. GUI widget handles and OpenGL handles, assuming they are thread-safe and making them implicitly shared would be incorrect.
Jul 19
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Wed, 19 Jul 2017 08:50:11 +0000
schrieb Kagamin <spam here.lot>:

 On Tuesday, 18 July 2017 at 19:24:18 UTC, Jonathan M Davis wrote:
 For full-on value types, it should be a non-issue though.  
Not quite. Value types include resource identifiers, which may have threading requirements, e.g. GUI widget handles and OpenGL handles, assuming they are thread-safe and making them implicitly shared would be incorrect.
That's exactly what I was opposing in the other post. These handles are opaque and never change their value. Within the Dlang language barrier they can be immutable and as such, implicitly shared. Your thinking is less technical, trying to find a best fit between type system and foreign API, so that only handles with a thread-safe API may become `shared`. I like the idea, but it is impractical. It sometimes depends on whether a library was compiled with multi-threading support or not and a value type can be copied from and to shared anyways, rendering the safety argument void: int x; shared int y = x; int z = y; -- Marco
Jul 19
parent reply Kagamin <spam here.lot> writes:
On Wednesday, 19 July 2017 at 12:56:38 UTC, Marco Leise wrote:
 That's exactly what I was opposing in the other post. These
 handles are opaque and never change their value. Within the
 Dlang language barrier they can be immutable and as such,
 implicitly shared.
Given transitivity of immutability the handle should have the same immutability as the resource it represents.
 It sometimes depends on whether a library was
 compiled with multi-threading support or not
Then you can communicate multithreading support with type system.
 and a value type can be copied from and to shared anyways, 
 rendering the safety
 argument void:

 	int x;
 	shared int y = x;
 	int z = y;
If it's overlooked, it doesn't mean D can't have proper sharing.
Jul 20
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Thu, 20 Jul 2017 08:56:57 +0000
schrieb Kagamin <spam here.lot>:

 On Wednesday, 19 July 2017 at 12:56:38 UTC, Marco Leise wrote:
 That's exactly what I was opposing in the other post. These
 handles are opaque and never change their value. Within the
 Dlang language barrier they can be immutable and as such,
 implicitly shared.  
Given transitivity of immutability the handle should have the same immutability as the resource it represents.
I understand that you apply D keywords to C types in a best fit fashion, to get some errors from the compiler when you use them in the wrong context. It should work alright in some APIs (maybe you can show me an example). But since C does not have these qualifiers, one function may be used for shared and unshared resources or the library may even be compiled with or without thread-safety enabled and you have to query that at *runtime*, where you have no help from the type-system. So I believe, relying on the pattern will be frustrating at times as not general enough. What I seek to achieve by slapping immutable on things like file descriptors and opaque types is the use as hash table keys. As the hashed part of hash table keys must not change, this approach enables us to use these types as keys and statically verify immutability, too. Because opaque structs and integer handles are used in C APIs to *hide* the implementation details, the compiler is deliberately left blind. That FILE* could as well be an integer as far as transitivity of `immutable` goes. Nothing the compiler *can see of it* will ever change. And that's all the type system will ever care about really! Even if part of that hidden structure is actually returned mutably by some function, it is just an implementation detail. Whether you slap `immutable` on anything there is mostly cosmetic. Now what does that mean for type checks in practice? 1) Looking at POSIX' C `fgetc()` function, the stream is a plain FILE*: int fgetc (FILE *stream). Since I/O is thread-safe[1], it should ideally be `shared` the way you put it. And the way I look at it, it should be `immutable` on *our* side of the language barrier. (I.e. Dlang wont be able to change its contents or see the contents change.) 2) You can look up items by file descriptors or FILE* in hash tables implementations with immutable keys. So we can take this away: * Making a struct opaque, is implicitly making it immutable, because it's contents cannot be modified nor read directly - the compiler cannot reason about its contents. * Now we also have a layman's head-const, making resource pointers usable as immutable hash table keys. As you can see my thinking revolves around the idea that hash table keys must be immutable and that stems from the idea that once hashed and sorted into a table, the hashed data must not change. There are other approaches and druntime's AAs simply allow mutable keys: void main() { struct S { int* i; } int a = 1, b = 2; uint[S] aa; aa[S(&a)] = 42; foreach(ref s_key; aa.byKey()) // Allows "innocent" changes to the keys s_key.i = &b; foreach(ref s_key; aa.byKey()) // AA doesn't find the key it just returned! Range violation. uint u = aa[s_key]; } -- Marco
Jul 25
next sibling parent Kagamin <spam here.lot> writes:
On Tuesday, 25 July 2017 at 19:22:11 UTC, Marco Leise wrote:
 I understand that you apply D keywords to C types in a
 best fit fashion, to get some errors from the compiler when
 you use them in the wrong context. It should work alright in
 some APIs (maybe you can show me an example).
I currently only use value types, and they unfortunately implicitly convert to all qualifiers.
 But since C does not have these qualifiers, one function may be 
 used for shared and unshared resources or the library may even 
 be compiled with or without thread-safety enabled and you have 
 to query that at *runtime*, where you have no help from the 
 type-system. So I believe, relying on the pattern will be 
 frustrating at times as not general enough.
This means you use custom multithreading approach to work with the resources, shared qualifier would reflect it.
 What I seek to achieve by slapping immutable on things like 
 file descriptors and opaque types is the use as hash table keys.
Reference types could benefit from this too, I use it regularly in C#, so it's not exclusive to value types. You just fight hash table design here. If this use case is useful, you shouldn't fight with the library.
 As the hashed part of hash table keys must not change, this 
 approach enables us to use these types as keys and statically 
 verify immutability, too.
 Because opaque structs and integer handles are used in C APIs 
 to *hide* the implementation details, the compiler is 
 deliberately left blind.
Blind compiler can't verify anything: it's blind. And ftell and lseek clearly indicate that file stream is not immutable.
    Since I/O is thread-safe[1], it should ideally be `shared`
    the way you put it.
Indeed, but only C11 specifies this.
Jul 28
prev sibling parent Kagamin <spam here.lot> writes:
On Tuesday, 25 July 2017 at 19:22:11 UTC, Marco Leise wrote:
    Since I/O is thread-safe[1], it should ideally be `shared`
    the way you put it.
Also it sounds like a bad decision. void f() { printf("hello "); printf("world\n"); } If you have shared stdout and this function runs in 2 threads, you will get a character soup --- hello hello world world --- If you have a separate line buffered stdout for each thread, you will get whole lines in the output --- hello world hello world --- And incur no locking.
Jul 29
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 18 July 2017 at 19:24:18 UTC, Jonathan M Davis wrote:
 On Tuesday, July 18, 2017 18:06:56 Atila Neves via 
 Digitalmars-d wrote:
 On Tuesday, 18 July 2017 at 15:03:07 UTC, Kagamin wrote:
 On Tuesday, 18 July 2017 at 11:47:37 UTC, Petar Kirov

 [ZombineDev] wrote:
 I think Atila was talking about this one:
 struct A
 {

    ~this() {}

 }

 void main()
 {

    auto a = A();
    shared b = A();

 }
This is strange. There's nothing that suggests that struct A and its destructor is thread-safe, yet compiler assumes it is.
Except for a programmer explicitly and manually calling the destructor (in which case, don't), the destructor is only ever called by one thread.
It could still be a problem if the struct has a member variable that is a reference type, because then something else could refer to that object, and if it's shared, then you would need to protect it, and the operations that shared prevents should still be prevented. For full-on value types, it should be a non-issue though. - Jonathan M Davis
Mmm, I guess so. As Marco pointed out, it's a similar problem with immutable (because the compiler casts it away before calling the destructor). Although I dare say that anybody writing code that depends on such locking and destruction when shared is unlikely to get it right in the first place. Atila
Jul 19
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wednesday, July 19, 2017 2:29:04 PM MDT Atila Neves via Digitalmars-d 
wrote:
 On Tuesday, 18 July 2017 at 19:24:18 UTC, Jonathan M Davis wrote:
 On Tuesday, July 18, 2017 18:06:56 Atila Neves via
 Except for a programmer explicitly and manually calling the
 destructor (in which case, don't), the destructor is only ever
 called by one thread.
It could still be a problem if the struct has a member variable that is a reference type, because then something else could refer to that object, and if it's shared, then you would need to protect it, and the operations that shared prevents should still be prevented. For full-on value types, it should be a non-issue though. - Jonathan M Davis
Mmm, I guess so. As Marco pointed out, it's a similar problem with immutable (because the compiler casts it away before calling the destructor). Although I dare say that anybody writing code that depends on such locking and destruction when shared is unlikely to get it right in the first place.
Well, consider that something like a reference counted type would have to get this right. Now, a reference counted type that worled with shared would need to be written that way - simply slapping shared on such a smart pointer type isn't going to work - but it would then be really bad for the destructor to be treated as thread-local. It really looks to me like having a thread-local dstructor for shared data is just begging for problems. It may work in the simple cases, but it'll fall apart in the more complicated ones, and I don't see how that's acceptable. Only really basic types are going to work as shared without being specifically designed for it, so I'm inclined to think that it would be far better for the language to be changed so that it supports having both a shared and non-shared destructor rather than having shared objects work with non-shared destructors. - Jonathan M Davis
Jul 19
parent reply Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 19 July 2017 at 20:23:18 UTC, Jonathan M Davis 
wrote:
 On Wednesday, July 19, 2017 2:29:04 PM MDT Atila Neves via 
 Digitalmars-d wrote:
 On Tuesday, 18 July 2017 at 19:24:18 UTC, Jonathan M Davis 
 wrote:
 On Tuesday, July 18, 2017 18:06:56 Atila Neves via
 Except for a programmer explicitly and manually calling the 
 destructor (in which case, don't), the destructor is only 
 ever called by one thread.
It could still be a problem if the struct has a member variable that is a reference type, because then something else could refer to that object, and if it's shared, then you would need to protect it, and the operations that shared prevents should still be prevented. For full-on value types, it should be a non-issue though. - Jonathan M Davis
Mmm, I guess so. As Marco pointed out, it's a similar problem with immutable (because the compiler casts it away before calling the destructor). Although I dare say that anybody writing code that depends on such locking and destruction when shared is unlikely to get it right in the first place.
Well, consider that something like a reference counted type would have to get this right. Now, a reference counted type that worled with shared would need to be written that way - simply slapping shared on such a smart pointer type isn't going to work - but it would then be really bad for the destructor to be treated as thread-local.
Not necessarily - the reference counted smart pointer doesn't have to be `shared` itself to have a `shared` payload. I'm not even entirely sure what the advantage of it being `shared` would be, or even what that would really mean. The way `automem.RefCounted` works right now is by doing atomic reference count manipulations by using reflection to know if the payload is `shared`.
 It really looks to me like having a thread-local dstructor for 
 shared data is just begging for problems. It may work in the 
 simple cases, but it'll fall apart in the more complicated 
 ones, and I don't see how that's acceptable.
You've definitely made me wonder about complicated cases, but I'd argue that they'd be rare. Destructors are (bar manually calling them) run in one thread. I'm having trouble imagining a situation where two threads have references to a `shared` object/value that is going to be destroyed deterministically.
 Only really basic types are going to work as shared without 
 being specifically designed for it, so I'm inclined to think 
 that it would be far better for the language to be changed so 
 that it supports having both a shared and non-shared destructor 
 rather than having shared objects work with non-shared 
 destructors.
Perhaps. Atila
Jul 19
next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Wednesday, 19 July 2017 at 20:59:03 UTC, Atila Neves wrote:
 On Wednesday, 19 July 2017 at 20:23:18 UTC, Jonathan M Davis 
 wrote:
 On Wednesday, July 19, 2017 2:29:04 PM MDT Atila Neves via 
 Digitalmars-d wrote:
 On Tuesday, 18 July 2017 at 19:24:18 UTC, Jonathan M Davis 
 wrote:
 On Tuesday, July 18, 2017 18:06:56 Atila Neves via
 Except for a programmer explicitly and manually calling 
 the destructor (in which case, don't), the destructor is 
 only ever called by one thread.
It could still be a problem if the struct has a member variable that is a reference type, because then something else could refer to that object, and if it's shared, then you would need to protect it, and the operations that shared prevents should still be prevented. For full-on value types, it should be a non-issue though. - Jonathan M Davis
Mmm, I guess so. As Marco pointed out, it's a similar problem with immutable (because the compiler casts it away before calling the destructor). Although I dare say that anybody writing code that depends on such locking and destruction when shared is unlikely to get it right in the first place.
Well, consider that something like a reference counted type would have to get this right. Now, a reference counted type that worled with shared would need to be written that way - simply slapping shared on such a smart pointer type isn't going to work - but it would then be really bad for the destructor to be treated as thread-local.
Not necessarily - the reference counted smart pointer doesn't have to be `shared` itself to have a `shared` payload.
Agreed. I'm exploring doing the same for std.stdio.File.
 I'm not even entirely sure what the advantage of it being 
 `shared` would be, or even what that would really mean.
There are plenty of cases where you need the smart pointer to be shared itself - e.g. member composition and global variables. See also: [0] Note that this doesn't play well with regular [1] value types becuase e.g. you don't have control over the synthesized bit-blit for this(this) and so you can't assume that structs with a single pointer member are updated atomically, even if would write the opAssign that way. In C++17 atomic_shared_ptr has it's copy-constructor and assign operator deleted. You can only do atomic<T> like ops with it and derive a plain shared_ptr<T> from it, kind-of like core.atomic's HeadUnshared(T). [0]: https://www.justsoftwaresolutions.co.uk/threading/why-do-we-need-atomic_shared_ptr.html [1]: http://stepanovpapers.com/DeSt98.pdf (definition of regular types).
Jul 19
parent reply Kagamin <spam here.lot> writes:
On Wednesday, 19 July 2017 at 21:50:32 UTC, Petar Kirov 
[ZombineDev] wrote:
 Note that this doesn't play well with regular [1] value types 
 becuase e.g. you don't have control over the synthesized 
 bit-blit for this(this) and so you can't assume that structs 
 with a single pointer member are updated atomically, even if 
 would write the opAssign that way. In C++17 atomic_shared_ptr 
 has it's copy-constructor and assign operator deleted. You can 
 only do atomic<T> like ops with it and derive a plain 
 shared_ptr<T> from it, kind-of like core.atomic's 
 HeadUnshared(T).
Huh? Why opAssign can't just do what atomic<T> does?
Jul 20
next sibling parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 20 July 2017 at 10:16:21 UTC, Kagamin wrote:
 On Wednesday, 19 July 2017 at 21:50:32 UTC, Petar Kirov 
 [ZombineDev] wrote:
 Note that this doesn't play well with regular [1] value types 
 becuase e.g. you don't have control over the synthesized 
 bit-blit for this(this) and so you can't assume that structs 
 with a single pointer member are updated atomically, even if 
 would write the opAssign that way. In C++17 atomic_shared_ptr 
 has it's copy-constructor and assign operator deleted. You can 
 only do atomic<T> like ops with it and derive a plain 
 shared_ptr<T> from it, kind-of like core.atomic's 
 HeadUnshared(T).
Huh? Why opAssign can't just do what atomic<T> does?
opAssign is fine, the problem is with the this(this).
Jul 20
prev sibling parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 20 July 2017 at 10:16:21 UTC, Kagamin wrote:
 On Wednesday, 19 July 2017 at 21:50:32 UTC, Petar Kirov 
 [ZombineDev] wrote:
 Note that this doesn't play well with regular [1] value types 
 becuase e.g. you don't have control over the synthesized 
 bit-blit for this(this) and so you can't assume that structs 
 with a single pointer member are updated atomically, even if 
 would write the opAssign that way. In C++17 atomic_shared_ptr 
 has it's copy-constructor and assign operator deleted. You can 
 only do atomic<T> like ops with it and derive a plain 
 shared_ptr<T> from it, kind-of like core.atomic's 
 HeadUnshared(T).
Huh? Why opAssign can't just do what atomic<T> does?
Also note that atomic<T> doesn't have neither copy constructor nor assignment operator: http://en.cppreference.com/w/cpp/atomic/atomic/atomic
 atomic( const atomic& ) = delete;
http://en.cppreference.com/w/cpp/atomic/atomic/operator%3D
 atomic& operator=( const atomic& ) = delete;
 atomic& operator=( const atomic& ) volatile = delete;
Jul 20
prev sibling next sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wednesday, July 19, 2017 8:59:03 PM MDT Atila Neves via Digitalmars-d 
wrote:
 On Wednesday, 19 July 2017 at 20:23:18 UTC, Jonathan M Davis

 wrote:
 On Wednesday, July 19, 2017 2:29:04 PM MDT Atila Neves via

 Digitalmars-d wrote:
 On Tuesday, 18 July 2017 at 19:24:18 UTC, Jonathan M Davis

 wrote:
 On Tuesday, July 18, 2017 18:06:56 Atila Neves via

 Except for a programmer explicitly and manually calling the
 destructor (in which case, don't), the destructor is only
 ever called by one thread.
It could still be a problem if the struct has a member variable that is a reference type, because then something else could refer to that object, and if it's shared, then you would need to protect it, and the operations that shared prevents should still be prevented. For full-on value types, it should be a non-issue though. - Jonathan M Davis
Mmm, I guess so. As Marco pointed out, it's a similar problem with immutable (because the compiler casts it away before calling the destructor). Although I dare say that anybody writing code that depends on such locking and destruction when shared is unlikely to get it right in the first place.
Well, consider that something like a reference counted type would have to get this right. Now, a reference counted type that worled with shared would need to be written that way - simply slapping shared on such a smart pointer type isn't going to work - but it would then be really bad for the destructor to be treated as thread-local.
Not necessarily - the reference counted smart pointer doesn't have to be `shared` itself to have a `shared` payload. I'm not even entirely sure what the advantage of it being `shared` would be, or even what that would really mean. The way `automem.RefCounted` works right now is by doing atomic reference count manipulations by using reflection to know if the payload is `shared`.
Okay, but my point is that it's perfectly legal to have a shared object that refers to other shared objects that it does not own, and casting away shared in the destructor means that the compiler can no longer enforce shared like it's supposed to.
 It really looks to me like having a thread-local dstructor for
 shared data is just begging for problems. It may work in the
 simple cases, but it'll fall apart in the more complicated
 ones, and I don't see how that's acceptable.
You've definitely made me wonder about complicated cases, but I'd argue that they'd be rare. Destructors are (bar manually calling them) run in one thread. I'm having trouble imagining a situation where two threads have references to a `shared` object/value that is going to be destroyed deterministically.
The issue isn't the object being destroyed. It's what it refers to via its member variables. For instance, what if an object were to remove itself from a shared list when it's destroyed (e.g. because it's an observer in the observer pattern). The object has a reference to the list, but it doesn't own it. So, the fact that the destructor is only run in a single thread doesn't help any. You could have ten of these objects all being destroyed and accessing the same list at the same time from different threads, and if the destructor treats the objects as thread-local - and thus treats all of the member variables as thread-local, then you won't get the compiler errors that you're supposed to get when you do something non-atomic with shared. Sure, the list could be designed in such a way that it protects itself against threading issues, but it's perfectly legal to have it be shared and require that anyone using it lock a mutex, and destructors need to take that into account just like any other function would. Basically, what these changes have done is act like all destructors are part of a synchronized class (as described in TDPL) where shared is stripped off of the member variables - except that _all_ of shared has been stripped off instead of just the outer layer, but these aren't synchronized classes, and they don't provide any guarantees about references not escaping or locking occurring, and not even synchronized classes can safely cast away shared except for the outer layer. As such, from what I can tell, these changes are very broken. Yes, we need a solution which allows us to deal with shared destructors properly, but casting away shared without any guarantees beyond the fact that no other threads are calling any member functions on that object at that time simply does not guarantee that shared objects have been properly protected. To do that, you'd essentially need synchronized classes, and even they can only strip off the outer layer of shared, not the whole thing. Yes, destructors have fewer problems with shared than most functions, but there's nothing about then which makes them immune to issues with shared. - Jonathan M Davis
Jul 19
parent reply Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
On Wednesday, 19 July 2017 at 22:35:43 UTC, Jonathan M Davis 
wrote:
 The issue isn't the object being destroyed. It's what it refers 
 to via its member variables. For instance, what if an object 
 were to remove itself from a shared list when it's destroyed 
 (e.g. because it's an observer in the observer pattern). The 
 object has a reference to the list, but it doesn't own it.
So, even a thread-local object that has references to a shared list has to handle those as shared, even in its non-shared destructor. I can't follow your argument.
Jul 20
next sibling parent reply Kagamin <spam here.lot> writes:
On Thursday, 20 July 2017 at 07:40:35 UTC, Dominikus Dittes 
Scherkl wrote:
 On Wednesday, 19 July 2017 at 22:35:43 UTC, Jonathan M Davis 
 wrote:
 The issue isn't the object being destroyed. It's what it 
 refers to via its member variables. For instance, what if an 
 object were to remove itself from a shared list when it's 
 destroyed (e.g. because it's an observer in the observer 
 pattern). The object has a reference to the list, but it 
 doesn't own it.
So, even a thread-local object that has references to a shared list has to handle those as shared, even in its non-shared destructor. I can't follow your argument.
Thread local object can't be contained in a shared list, the list is referred as unqualified, and thread local object will be contained in a thread local list, and shared object will be contained in a shared list because of transitivity of the shared qualifier.
Jul 20
parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 20 July 2017 at 10:19:30 UTC, Kagamin wrote:
 On Thursday, 20 July 2017 at 07:40:35 UTC, Dominikus Dittes 
 Scherkl wrote:
 On Wednesday, 19 July 2017 at 22:35:43 UTC, Jonathan M Davis 
 wrote:
 The issue isn't the object being destroyed. It's what it 
 refers to via its member variables. For instance, what if an 
 object were to remove itself from a shared list when it's 
 destroyed (e.g. because it's an observer in the observer 
 pattern). The object has a reference to the list, but it 
 doesn't own it.
So, even a thread-local object that has references to a shared list has to handle those as shared, even in its non-shared destructor. I can't follow your argument.
Thread local object can't be contained in a shared list, the list is referred as unqualified, and thread local object will be contained in a thread local list, and shared object will be contained in a shared list because of transitivity of the shared qualifier.
It's the other way around: ThreadLocal tl; struct ThreadLocal { shared(ListNode*)* listHead; shared(ListNode)* listNode; ~this() { listHead.removeNodeFromList(listNode); } }
Jul 20
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thursday, July 20, 2017 07:40:35 Dominikus Dittes Scherkl via 
Digitalmars-d wrote:
 On Wednesday, 19 July 2017 at 22:35:43 UTC, Jonathan M Davis

 wrote:
 The issue isn't the object being destroyed. It's what it refers
 to via its member variables. For instance, what if an object
 were to remove itself from a shared list when it's destroyed
 (e.g. because it's an observer in the observer pattern). The
 object has a reference to the list, but it doesn't own it.
So, even a thread-local object that has references to a shared list has to handle those as shared, even in its non-shared destructor. I can't follow your argument.
You can't just strip off shared. To do so defeats the purpose of shared. If you have something like struct S { shared List<Foo> _list; ~this() { ... } } then inside of the destructor, _list is not treated as shared, meaning that none of the compiler protections for shared are in place, no locking has occurred, and the compiler is free to make optimizations based on the wrong assumption that all of the member variables are thread-local. If nothing else has access to that list, then it'll work, but if anything else does - and if it's a reference type, that's perfectly possible - then you have a threading problem, because shared has been violated. Except in cases where the member variables are all value types and thus no other references to them should exist when the destructor is called, stripping away shared from them means that the compiler can no longer properly enforce shared, and it's going to make the wrong assumptions about whether the data can be treated as thread-local or not. If we go with the assumption that nothing has pointers to the member variables (since doing so would be system, and they're only valid so long as the struct isn't moved anyway), you can probably strip off the outer layer of shared safely in the destructor, but if you're dealing with a reference type, anything it points to needs to still be treated as shared. - Jonathan M Davis
Jul 20
parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 20 July 2017 at 21:20:46 UTC, Jonathan M Davis wrote:
 On Thursday, July 20, 2017 07:40:35 Dominikus Dittes Scherkl 
 via Digitalmars-d wrote:
 On Wednesday, 19 July 2017 at 22:35:43 UTC, Jonathan M Davis

 wrote:
 The issue isn't the object being destroyed. It's what it 
 refers to via its member variables. For instance, what if an 
 object were to remove itself from a shared list when it's 
 destroyed (e.g. because it's an observer in the observer 
 pattern). The object has a reference to the list, but it 
 doesn't own it.
So, even a thread-local object that has references to a shared list has to handle those as shared, even in its non-shared destructor. I can't follow your argument.
You can't just strip off shared. To do so defeats the purpose of shared. If you have something like struct S { shared List<Foo> _list; ~this() { ... } }
This is fine. What dmd does now is strip shared off of the `this` pointer, not the member variables. There's only a problem if the sharedness of the member variable(s) depends on sharedness of the enclosing object.
 then inside of the destructor, _list is not treated as shared,
It is. Atila
Jul 21
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Friday, July 21, 2017 08:37:51 Atila Neves via Digitalmars-d wrote:
 On Thursday, 20 July 2017 at 21:20:46 UTC, Jonathan M Davis wrote:
 On Thursday, July 20, 2017 07:40:35 Dominikus Dittes Scherkl

 via Digitalmars-d wrote:
 On Wednesday, 19 July 2017 at 22:35:43 UTC, Jonathan M Davis

 wrote:
 The issue isn't the object being destroyed. It's what it
 refers to via its member variables. For instance, what if an
 object were to remove itself from a shared list when it's
 destroyed (e.g. because it's an observer in the observer
 pattern). The object has a reference to the list, but it
 doesn't own it.
So, even a thread-local object that has references to a shared list has to handle those as shared, even in its non-shared destructor. I can't follow your argument.
You can't just strip off shared. To do so defeats the purpose of shared. If you have something like struct S { shared List<Foo> _list; ~this() { ... } }
Wow. I've been doing too much C++ lately apparently, since I used <>. :|
 This is fine. What dmd does now is strip shared off of the `this`
 pointer, not the member variables. There's only a problem if the
 sharedness of the member variable(s) depends on sharedness of the
 enclosing object.
What happens with something like struct S { Foo* _foo; ~this() {...} } shared S s; Inside the destructor, is what _foo points to still treated as shared: shared(Foo)*? i.e. is the outer layer of shared the only layer being made thread-local - like what would supposedly happen with synchronized classes? If so, then that largely solves the problem. The only issue is pointers to the member variables, which would not be safe but would still technically be possible, in which case something outside the struct could still reference the member variables from another thread. But given that that's going to blow up in your face soon thereafter anyway, since the object is being destroyed (and thus you screwed up making sure that your system code was safe), that's probably fine. However, if _foo is treated as Foo* instead of shared(Foo)* in the destructor, then there definitely is a problem. The fact that when the member variable is explicitly shared, it continues to be treated as shared definitely reduces the problem, but it doesn't fully close the hole. The parts of the member variables not directly in the object still need to be treated as shared, because they aren't necessarily owned or controlled by the object and could legally and safely be manipulated from other threads even while the destructor is running. So, are they still treated as shared, or are they treated as fully thread-local? I would have thought that they'd still be treated as thread-local given that the shared part is then only known to the variable that was marked as shared and not the destructor itself, since it's the same destructor for thread-local and shared objects. And if the destructor treats the member variables as completely thread-local (rather than just the outer layer as thread local) even when the object itself was shared, then I don't think that this is a viable solution. It would either need to be made illegal to make an object shared if it has indirections and a destructor, or it needs to have a shared destructor. - Jonathan M Davis
Jul 21
parent reply Atila Neves <atila.neves gmail.com> writes:
On Friday, 21 July 2017 at 22:02:41 UTC, Jonathan M Davis wrote:
 On Friday, July 21, 2017 08:37:51 Atila Neves via Digitalmars-d 
 wrote:
 On Thursday, 20 July 2017 at 21:20:46 UTC, Jonathan M Davis 
 wrote:
 On Thursday, July 20, 2017 07:40:35 Dominikus Dittes Scherkl

 via Digitalmars-d wrote:
 On Wednesday, 19 July 2017 at 22:35:43 UTC, Jonathan M Davis

 wrote:
 The issue isn't the object being destroyed. It's what it 
 refers to via its member variables. For instance, what if 
 an object were to remove itself from a shared list when 
 it's destroyed (e.g. because it's an observer in the 
 observer pattern). The object has a reference to the 
 list, but it doesn't own it.
So, even a thread-local object that has references to a shared list has to handle those as shared, even in its non-shared destructor. I can't follow your argument.
You can't just strip off shared. To do so defeats the purpose of shared. If you have something like struct S { shared List<Foo> _list; ~this() { ... } }
Wow. I've been doing too much C++ lately apparently, since I used <>. :|
I noticed, but I wasn't going to say anything ;)
 This is fine. What dmd does now is strip shared off of the 
 `this` pointer, not the member variables. There's only a 
 problem if the sharedness of the member variable(s) depends on 
 sharedness of the enclosing object.
What happens with something like struct S { Foo* _foo; ~this() {...} } shared S s; Inside the destructor, is what _foo points to still treated as shared: shared(Foo)*?
No. This is what I meant by the sharedness depening on the enclosing object. However, there's a workaround: struct Foo { } struct S { Foo* _foo; bool _isShared; this(this T, U)(U foo) if(is(T == shared) && is(U == shared(Foo)*) || !is(T == shared) && is(U == Foo*)) { static if(is(T == shared)) _isShared = true; _foo = foo; } ~this() { import std.stdio: writeln; _isShared ? writeln("shared dtor") : writeln("non-shared dtor"); } } void main() { auto f = Foo(); auto sf = shared Foo(); auto s = S(&f); auto ss = shared S(&sf); } It's annoying to use that bool up memory-wise, but I assume it's not a big deal for most applications. In any case, that example wouldn't have worked anyway before my change to dmd - even creating the S struct would've been a compiler error. Atila
Jul 24
next sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Monday, July 24, 2017 2:30:01 PM MDT Atila Neves via Digitalmars-d wrote:
 This is fine. What dmd does now is strip shared off of the
 `this` pointer, not the member variables. There's only a
 problem if the sharedness of the member variable(s) depends on
 sharedness of the enclosing object.
What happens with something like struct S { Foo* _foo; ~this() {...} } shared S s; Inside the destructor, is what _foo points to still treated as shared: shared(Foo)*?
No. This is what I meant by the sharedness depening on the enclosing object. However, there's a workaround: struct Foo { } struct S { Foo* _foo; bool _isShared; this(this T, U)(U foo) if(is(T == shared) && is(U == shared(Foo)*) || !is(T == shared) && is(U == Foo*)) { static if(is(T == shared)) _isShared = true; _foo = foo; } ~this() { import std.stdio: writeln; _isShared ? writeln("shared dtor") : writeln("non-shared dtor"); } } void main() { auto f = Foo(); auto sf = shared Foo(); auto s = S(&f); auto ss = shared S(&sf); } It's annoying to use that bool up memory-wise, but I assume it's not a big deal for most applications. In any case, that example wouldn't have worked anyway before my change to dmd - even creating the S struct would've been a compiler error.
The problem with this is that this means that shared is not being properly enforced by the compiler. Your workaround is a way for the programmer to figure out if the object is shared and do something differently based on that, but for the compiler to do what it's supposed to be doing with shared (e.g. prevent non-atomic operations), any indirections in the member variables must continue to be typed as shared inside the destructor, and that's clearly not happening right now, which is a serious problem IMHO. The situation may be better thanks to your changes in that some stuff is now possible that should be possible and was not before, but it's not completely sound as far as the type system goes, and we really should be fixing it so that shared is properly enforced rather than just blindly stripped off. - Jonathan M Davis
Jul 24
parent Atila Neves <atila.neves gmail.com> writes:
On Monday, 24 July 2017 at 20:30:25 UTC, Jonathan M Davis wrote:
 On Monday, July 24, 2017 2:30:01 PM MDT Atila Neves via 
 Digitalmars-d wrote:
 This is fine. What dmd does now is strip shared off of the 
 `this` pointer, not the member variables. There's only a 
 problem if the sharedness of the member variable(s) depends 
 on sharedness of the enclosing object.
What happens with something like struct S { Foo* _foo; ~this() {...} } shared S s; Inside the destructor, is what _foo points to still treated as shared: shared(Foo)*?
No. This is what I meant by the sharedness depening on the enclosing object. However, there's a workaround: struct Foo { } struct S { Foo* _foo; bool _isShared; this(this T, U)(U foo) if(is(T == shared) && is(U == shared(Foo)*) || !is(T == shared) && is(U == Foo*)) { static if(is(T == shared)) _isShared = true; _foo = foo; } ~this() { import std.stdio: writeln; _isShared ? writeln("shared dtor") : writeln("non-shared dtor"); } } void main() { auto f = Foo(); auto sf = shared Foo(); auto s = S(&f); auto ss = shared S(&sf); } It's annoying to use that bool up memory-wise, but I assume it's not a big deal for most applications. In any case, that example wouldn't have worked anyway before my change to dmd - even creating the S struct would've been a compiler error.
The problem with this is that this means that shared is not being properly enforced by the compiler. Your workaround is a way for the programmer to figure out if the object is shared and do something differently based on that, but for the compiler to do what it's supposed to be doing with shared (e.g. prevent non-atomic operations), any indirections in the member variables must continue to be typed as shared inside the destructor, and that's clearly not happening right now, which is a serious problem IMHO. The situation may be better thanks to your changes in that some stuff is now possible that should be possible and was not before, but it's not completely sound as far as the type system goes, and we really should be fixing it so that shared is properly enforced rather than just blindly stripped off. - Jonathan M Davis
I agree that this could be a problem, and that the proper solution is probably to allow the user to define more than one destructor. The problem isn't just with shared - immutable is similar, since you'd be able to invoke undefined behaviour from the destructor since immutable would be cast away and the compiler wouldn't even warn you. And that was already the behaviour in dmd. I think the situation isn't ideal but better than before. I also think that while the problem exists, I don't think it'll be common. This would only affect structs that can be shared _and_ non-shared (or immutable). This will require a DIP, methinks. Atila Atila
Jul 25
prev sibling parent Kagamin <spam here.lot> writes:
On Monday, 24 July 2017 at 14:30:01 UTC, Atila Neves wrote:
 struct Foo { }


 struct S {

     Foo* _foo;
     bool _isShared;

     this(this T, U)(U foo) if(is(T == shared) && is(U == 
 shared(Foo)*) || !is(T == shared) && is(U == Foo*)) {
         static if(is(T == shared)) _isShared = true;
         _foo = foo;
     }

     ~this() {
         import std.stdio: writeln;
         _isShared ? writeln("shared dtor") : 
 writeln("non-shared dtor");
     }
 }

 void main() {
     auto f = Foo();
     auto sf = shared Foo();
     auto s = S(&f);
     auto ss = shared S(&sf);
 }
Exactly this. You must design struct to support shared type, in which case it's better and more straightforward to just write shared destructor rather than work it around. Same for immutable.
Jul 25
prev sibling parent reply Kagamin <spam here.lot> writes:
On Wednesday, 19 July 2017 at 20:59:03 UTC, Atila Neves wrote:
 Not necessarily - the reference counted smart pointer doesn't 
 have to be `shared` itself to have a `shared` payload.
Yes, but it can be done either way. It's actually what Jack is trying to do: make stdout shared and reference counted: https://issues.dlang.org/show_bug.cgi?id=15768#c7
 I'm not even entirely sure what the advantage of it being 
 `shared` would be, or even what that would really mean.
It will be thread safe and its lifetime will be automatically managed.
 You've definitely made me wonder about complicated cases, but 
 I'd argue that they'd be rare. Destructors are (bar manually 
 calling them) run in one thread. I'm having trouble imagining a 
 situation where two threads have references to a `shared` 
 object/value that is going to be destroyed deterministically.
A mutex, a file, a socket, any shareable resource. Though I agree that reference counting of shared resources should be optimized by thread local counters.
Jul 20
parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 20 July 2017 at 10:15:26 UTC, Kagamin wrote:
 On Wednesday, 19 July 2017 at 20:59:03 UTC, Atila Neves wrote:
 Not necessarily - the reference counted smart pointer doesn't 
 have to be `shared` itself to have a `shared` payload.
Yes, but it can be done either way. It's actually what Jack is trying to do: make stdout shared and reference counted: https://issues.dlang.org/show_bug.cgi?id=15768#c7
 I'm not even entirely sure what the advantage of it being 
 `shared` would be, or even what that would really mean.
It will be thread safe and its lifetime will be automatically managed.
 You've definitely made me wonder about complicated cases, but 
 I'd argue that they'd be rare. Destructors are (bar manually 
 calling them) run in one thread. I'm having trouble imagining 
 a situation where two threads have references to a `shared` 
 object/value that is going to be destroyed deterministically.
A mutex, a file, a socket, any shareable resource. Though I agree that reference counting of shared resources should be optimized by thread local counters.
Mutexes and sockets are classes, so not destroyed deterministically. Anything that is `shared` is likely to be a reference (pointer, class...), or a global. Either way the compiler-generated destructor call isn't going to exist, which means it's probably ok to cast away shared when the compiler inserts the automatic call to a destructor at the end of scope. Atila
Jul 21
parent reply Kagamin <spam here.lot> writes:
On Friday, 21 July 2017 at 08:51:27 UTC, Atila Neves wrote:
 Mutexes and sockets are classes, so not destroyed 
 deterministically.
They should, like any unmanaged resources, e.g. by wrapping in a smart pointer. Imagine 10000 lingering tcp connections accumulated over time due to poor timing of destruction, it becomes a stress test for the server.
 Anything that is `shared` is likely to be a reference (pointer, 
 class...), or a global. Either way the compiler-generated 
 destructor call isn't going to exist, which means it's probably 
 ok to cast away shared when the compiler inserts the automatic 
 call to a destructor at the end of scope.
These are contradictory. Does the automatic destructor call exist or not?
Jul 21
next sibling parent Kagamin <spam here.lot> writes:
Hmm, if proper implementation of a shared smart pointer is 
impossible, it probably means that such smart pointer should be 
typed unshared when passed around. But then it doesn't make sense 
to call unshared destructor on shared smart pointer anyway, 
because it's not designed to be typed shared. In this case 
absence of shared destructor will indicate that the object 
doesn't support being shared and the compiler should reject the 
code.
Jul 21
prev sibling next sibling parent Moritz Maxeiner <moritz ucworks.org> writes:
On Friday, 21 July 2017 at 11:57:11 UTC, Kagamin wrote:
 On Friday, 21 July 2017 at 08:51:27 UTC, Atila Neves wrote:
 Mutexes and sockets are classes, so not destroyed 
 deterministically.
They should, like any unmanaged resources
I tend to agree, although
 e.g. by wrapping in a smart pointer.
objects that manage their own lifetime limit the design space unnecessarily. It's better to use normal structs to wrap resources (RAII) and then build whatever object lifetime management scheme one wants (including reference counting) on top of that.
Jul 21
prev sibling parent Atila Neves <atila.neves gmail.com> writes:
On Friday, 21 July 2017 at 11:57:11 UTC, Kagamin wrote:
 On Friday, 21 July 2017 at 08:51:27 UTC, Atila Neves wrote:
 Mutexes and sockets are classes, so not destroyed 
 deterministically.
They should, like any unmanaged resources, e.g. by wrapping in a smart pointer. Imagine 10000 lingering tcp connections accumulated over time due to poor timing of destruction, it becomes a stress test for the server.
 Anything that is `shared` is likely to be a reference 
 (pointer, class...), or a global. Either way the 
 compiler-generated destructor call isn't going to exist, which 
 means it's probably ok to cast away shared when the compiler 
 inserts the automatic call to a destructor at the end of scope.
These are contradictory. Does the automatic destructor call exist or not?
What I'm trying to say is that `shared` values will usually be references or globals and therefore there won't be a compiler-generated call to the destructor at the end of scope. When the compiler _does_ generate a call to the destructor, the value is unlikely to be shared. Since then I've thought that sending a value to another thread does indeed create a shared value with defined scope. Or, for that matter, calling any function that takes shared values (but those are rare, so it'll usually be `send`). I think I've not done a good job of explaining the destructor fix: what it changes is the code the compiler writes for you when variables go out of scope, i.e. the automatic destructor call casts away shared. It already did the same thing for immutable, otherwise you wouldn't be able to put immutable values on the stack. Atila
Jul 21
prev sibling next sibling parent Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 18 July 2017 at 11:47:37 UTC, Petar Kirov 
[ZombineDev] wrote:
 On Monday, 17 July 2017 at 19:30:53 UTC, Jack Stouffer wrote:
 [...]
I think Atila was talking about this one: struct A { ~this() {} } void main() { auto a = A(); shared b = A(); } https://is.gd/kOYlWY
That's what I meant. I was on a train and only skimmed through. Atila
Jul 18
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 18 July 2017 at 11:47:37 UTC, Petar Kirov 
[ZombineDev] wrote:
 I think Atila was talking about this one:
 struct A
 {
 	~this() {}
 }

 void main()
 {
 	auto a = A();
 	shared b = A();
 }
Shouldn't it be : struct A { ~this() shared {} } void main() { auto a = A(); shared b = A(); } ? Because handling theard-local data as shared is safe as far as I remember, but not the other way round. And if you want a destructor which works with both immutable and normal, shouldn't it be a const destructor?
Jul 19
parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Wednesday, 19 July 2017 at 14:56:58 UTC, Dukc wrote:
 Shouldn't it be :
 struct A
 {
     ~this() shared {}
 }

 void main()
 {
     auto a = A();
     shared b = A();
 }
 ?

 Because handling theard-local data as shared is safe as far as 
 I remember, but not the other way round.
Non-shared structs/classes can't call shared methods. Unless you're saying that the above should work even though it currently doesn't. Even then, I don't know about that. If your type is complex enough to need a shared dtor then the dtor probably needs to do some locking or extra checks. You don't want to impose that cost on a struct instance which isn't shared.
Jul 19
parent Dukc <ajieskola gmail.com> writes:
On Wednesday, 19 July 2017 at 15:38:08 UTC, Jack Stouffer wrote:
 Unless you're saying that the above should work even though it 
 currently doesn't. Even then, I don't know about that. If your 
 type is complex enough to need a shared dtor then the dtor 
 probably needs to do some locking or extra checks. You don't 
 want to impose that cost on a struct instance which isn't 
 shared.
Yes, just what I meant. And perfectly explained why we need something better.
Jul 19
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 17 July 2017 at 19:30:53 UTC, Jack Stouffer wrote:
 On Monday, 17 July 2017 at 17:41:58 UTC, Atila Neves wrote:
 On Monday, 17 July 2017 at 14:26:19 UTC, Jack Stouffer wrote:
 TL;DR: Issue 17658 [1] makes using shared very 
 annoying/practically impossible.

 [...]
I fixed this already, should be in the next release. Atila
Are you sure? Because DMD nightly still errors: https://run.dlang.io?compiler=dmd-nightly&source=struct%20A%0A%7B%0A%20%20%20%20this(string%20a)%20%7B%7D%0A%20%20%20%20this(string%20a)%20shared%20%7B%7D%0A%0A%20%20%20%20~this()%20%7B%7D%0A%20%20%20%20~this()%20shared%20%7B%7D%0A%0A%20%20%20%20this(this)%20%7B%7D%0A%20%20%20%20this(this)%20shared%20%7B%7D%0A%7D%0A%0Avoid%20main()%0A%7B%0A%20%20%20%20shared%20f%20%3D%20A(%22%22)%3B%0A%7D (maybe we should remove the ban on URL shorteners for our own sites)
Now I've read your post properly: there is only one destructor. With the fix I mentioned, just don't defined the `shared` version, there's no need to. Postblit is still a problem, however. Atila
Jul 18
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Tue, 18 Jul 2017 18:10:58 +0000
schrieb Atila Neves <atila.neves gmail.com>:
 Now I've read your post properly: there is only one destructor. 
 With the fix I mentioned, just don't defined the `shared` 
 version, there's no need to. Postblit is still a problem, however.
 
 Atila
The issue is wider than just `shared` by the way: https://issues.dlang.org/show_bug.cgi?id=13628 Some may jump to say that an immutable struct can't be destructed, but my perspective here is that immutable only applies to what the compiler can introspect. A file descriptor or an opaque struct pointer from a C API are just flat values and escape the compiler. They can be stored in an immutable struct and still need `close()` called on them. Layman's head-const :p -- Marco
Jul 18
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 17 July 2017 at 19:30:53 UTC, Jack Stouffer wrote:
 On Monday, 17 July 2017 at 17:41:58 UTC, Atila Neves wrote:
 On Monday, 17 July 2017 at 14:26:19 UTC, Jack Stouffer wrote:
 TL;DR: Issue 17658 [1] makes using shared very 
 annoying/practically impossible.

 [...]
I fixed this already, should be in the next release. Atila
Are you sure? Because DMD nightly still errors: https://run.dlang.io?compiler=dmd-nightly&source=struct%20A%0A%7B%0A%20%20%20%20this(string%20a)%20%7B%7D%0A%20%20%20%20this(string%20a)%20shared%20%7B%7D%0A%0A%20%20%20%20~this()%20%7B%7D%0A%20%20%20%20~this()%20shared%20%7B%7D%0A%0A%20%20%20%20this(this)%20%7B%7D%0A%20%20%20%20this(this)%20shared%20%7B%7D%0A%7D%0A%0Avoid%20main()%0A%7B%0A%20%20%20%20shared%20f%20%3D%20A(%22%22)%3B%0A%7D (maybe we should remove the ban on URL shorteners for our own sites)
This works fine in dmd 2.075: struct A { this(string a) {} this(string a) shared {} ~this() {} this(this T)(this) {} // you can reflect to find out if shared } void main() { auto nonShared = A(""); auto shared_ = shared A(""); auto nonSharedCopy = nonShared; auto sharedCopy = shared_; } Atila
Jul 21
parent reply Arek <arychlinski gmail.com> writes:
On Friday, 21 July 2017 at 08:53:44 UTC, Atila Neves wrote:

 This works fine in dmd 2.075:

 struct A
 {
     this(string a) {}
     this(string a) shared {}

     ~this() {}

     this(this T)(this) {} // you can reflect to find out if 
 shared
 }

 void main()
 {
     auto nonShared = A("");
     auto shared_ = shared A("");
     auto nonSharedCopy = nonShared;
     auto sharedCopy = shared_;
 }


 Atila
This look interesting: this(this T)(this) {} What is it? Postblit? It compiles but doesn't work for me. this(this T)(this) {writeln("postblit"); } // doesn't print anything Arek
Aug 12
parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 12 August 2017 at 19:34:35 UTC, Arek wrote:
 On Friday, 21 July 2017 at 08:53:44 UTC, Atila Neves wrote:

 This works fine in dmd 2.075:

 struct A
 {
     this(string a) {}
     this(string a) shared {}

     ~this() {}

     this(this T)(this) {} // you can reflect to find out if 
 shared
 }

 void main()
 {
     auto nonShared = A("");
     auto shared_ = shared A("");
     auto nonSharedCopy = nonShared;
     auto sharedCopy = shared_;
 }


 Atila
This look interesting: this(this T)(this) {} What is it? Postblit? It compiles but doesn't work for me. this(this T)(this) {writeln("postblit"); } // doesn't print anything Arek
It's a template postblit constructor - it'd get instantiated differently depending on the type of the implicit `this` parameter and would be able to fix things up taking into account whether or not `this` was shared (or immutable). Atila
Aug 14
next sibling parent reply Arek <arychlinski gmail.com> writes:
On Monday, 14 August 2017 at 16:49:22 UTC, Atila Neves wrote:
 On Saturday, 12 August 2017 at 19:34:35 UTC, Arek wrote:
 On Friday, 21 July 2017 at 08:53:44 UTC, Atila Neves wrote:
... /cut/...
 It's a template postblit constructor - it'd get instantiated 
 differently depending on the type of the implicit `this` 
 parameter and would be able to fix things up taking into 
 account whether or not `this` was shared (or immutable).

 Atila
I've tested this code on dmd 2.075.0 and it doesn't behave like postblit. It's not executed in these statements: auto nonSharedCopy = nonShared; auto sharedCopy = shared_; To get it executed, you have to change the statements into auto nonSharedCopy = A(nonShared); auto sharedCopy = A(shared_); this(this T)(this) is compiled into 0000000000000000 <shared(ref shared(b.A) function(immutable(char)[])) b.A.__ctor>: 0: 55 push %rbp 1: 48 8b ec mov %rsp,%rbp 4: 48 89 f8 mov %rdi,%rax 7: 5d pop %rbp 8: c3 retq 9: 00 00 add %al,(%rax) ... what is exacly the same as the constructor: 0000000000000000 <ref b.A b.A.__ctor(immutable(char)[])>: 0: 55 push %rbp 1: 48 8b ec mov %rsp,%rbp 4: 48 89 f8 mov %rdi,%rax 7: 5d pop %rbp 8: c3 retq 9: 00 00 add %al,(%rax) ... The real postblit looks like this: Disassembly of section .text.void b.A.__postblit(): 0000000000000000 <void b.A.__postblit()>: 0: 55 push %rbp 1: 48 8b ec mov %rsp,%rbp 4: 48 83 ec 10 sub $0x10,%rsp 8: 48 89 7d f8 mov %rdi,-0x8(%rbp) c: b8 73 00 00 00 mov $0x73,%eax 11: b9 0a 00 00 00 mov $0xa,%ecx 16: 99 cltd 17: f7 f9 idiv %ecx 19: 48 83 7d f8 00 cmpq $0x0,-0x8(%rbp) 1e: 75 2e jne 4e <void b.A.__postblit()+0x4e> 20: 49 89 c8 mov %rcx,%r8 23: 48 8d 0d 00 00 00 00 lea 0x0(%rip),%rcx # 2a <void b.A.__postblit()+0x2a> 2a: b8 03 00 00 00 mov $0x3,%eax 2f: 48 89 c2 mov %rax,%rdx 32: 48 89 55 f0 mov %rdx,-0x10(%rbp) 36: 48 8d 15 00 00 00 00 lea 0x0(%rip),%rdx # 3d <void b.A.__postblit()+0x3d> 3d: bf 09 00 00 00 mov $0x9,%edi 42: 48 89 d6 mov %rdx,%rsi 45: 48 8b 55 f0 mov -0x10(%rbp),%rdx 49: e8 00 00 00 00 callq 4e <void b.A.__postblit()+0x4e> 4e: c9 leaveq 4f: c3 retq So, in my opinion, your example compiles (but doesn't work) because it doesn't have the user defined postblit. It's a pity, because it's looked promising. Arek
Aug 14
parent Atila Neves <atila.neves gmail.com> writes:
On Monday, 14 August 2017 at 18:59:17 UTC, Arek wrote:
 On Monday, 14 August 2017 at 16:49:22 UTC, Atila Neves wrote:
 [...]
... /cut/...
 [...]
I've tested this code on dmd 2.075.0 and it doesn't behave like postblit. [...]
I'd have to double check, but this seems like a bug to me. Atila
Aug 15
prev sibling parent Arek <arychlinski gmail.com> writes:
On Monday, 14 August 2017 at 16:49:22 UTC, Atila Neves wrote:
 On Saturday, 12 August 2017 at 19:34:35 UTC, Arek wrote:
 On Friday, 21 July 2017 at 08:53:44 UTC, Atila Neves wrote:
 It's a template postblit constructor - it'd get instantiated 
 differently depending on the type of the implicit `this` 
 parameter and would be able to fix things up taking into 
 account whether or not `this` was shared (or immutable).

 Atila
Sorry - my mistake. This fancy template compiles into Disassembly of section .text.ref b.A b.A.__ctor!(b.A).__ctor(b.A): 0000000000000000 <ref b.A b.A.__ctor!(b.A).__ctor(b.A)>: 0: 55 push %rbp 1: 48 8b ec mov %rsp,%rbp 4: 48 83 ec 30 sub $0x30,%rsp 8: 48 89 7d f8 mov %rdi,-0x8(%rbp) c: 48 8d 0d 00 00 00 00 lea 0x0(%rip),%rcx # 13 <ref b.A b.A.__ctor!(b.A).__ctor(b.A)+0x13> 13: b8 0a 00 00 00 mov $0xa,%eax ... cut... But it's still not the postblit and doesn't works like postblit. Arek
Aug 14