www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - pass-by-ref semantics for structs (was Deque impl.)

reply d coder <dlang.coder gmail.com> writes:
On Thu, Jan 31, 2013 at 5:30 PM, monarch_dodra <monarchdodra gmail.com> wrote:
 The pull is kind of stuck in limbo, specifically because of the problems
 associated with implementing reference semantics with structs :/

Thanks for the enlightening email. I am of the considered view that reference semantics with structs in D is tough (if not impossible) with default constructor and postblit constructor (used when passing objects). This is because you can not initialize any object (wrapped in the struct) in the default constructor and if you are passing the struct as a function parameter, it is not possible to initialize these internal objects (before passing them) in the postblit constructor. I faced this in some of my code. Do not know if you are facing the same scenario in the DList implementation. Are there any possible solutions in the pipe? Regards - Puneet
Jan 31 2013
next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 31 January 2013 at 12:28:00 UTC, d coder wrote:
 On Thu, Jan 31, 2013 at 5:30 PM, monarch_dodra 
 <monarchdodra gmail.com> wrote:
 The pull is kind of stuck in limbo, specifically because of 
 the problems
 associated with implementing reference semantics with structs 
 :/

Thanks for the enlightening email. I am of the considered view that reference semantics with structs in D is tough (if not impossible) with default constructor and postblit constructor (used when passing objects). This is because you can not initialize any object (wrapped in the struct) in the default constructor and if you are passing the struct as a function parameter, it is not possible to initialize these internal objects (before passing them) in the postblit constructor. I faced this in some of my code. Do not know if you are facing the same scenario in the DList implementation. Are there any possible solutions in the pipe? Regards - Puneet

This keeps coming up, so I'll wrap up how it goes: Conclusions: * The language will not change. * Use "static S opCall()" to emulate no-arg constructor The conversation: - "But opCall is not good enough: It's not a constructor" - "But it's all you'll get" If you want some history, specifically, you need to look for "no-arg constructor".
Jan 31 2013
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 07:27:08 -0500, d coder <dlang.coder gmail.com> wrote:

 On Thu, Jan 31, 2013 at 5:30 PM, monarch_dodra <monarchdodra gmail.com>  
 wrote:
 The pull is kind of stuck in limbo, specifically because of the problems
 associated with implementing reference semantics with structs :/

Thanks for the enlightening email. I am of the considered view that reference semantics with structs in D is tough (if not impossible) with default constructor and postblit constructor (used when passing objects). This is because you can not initialize any object (wrapped in the struct) in the default constructor and if you are passing the struct as a function parameter, it is not possible to initialize these internal objects (before passing them) in the postblit constructor. I faced this in some of my code. Do not know if you are facing the same scenario in the DList implementation.

It's pretty simple. Containers are reference types, and it makes most sense to pass them by reference. In order to be sane, containers must be implemented as classes. Otherwise, you have behavior like this: void foo(int[int] aa, int x, int y) { aa[x] = y; } void main() { int[int] aa; foo(aa, 1, 2); aa[3] = 4; foo(aa, 5, 6); assert(!(1 in aa)); assert(aa[3] == 4); assert(aa[5] == 6); } In other words, the only way to turn a default-instantiated struct into a valid struct is to use it once. Not the behavior you want. Classes don't have this problem, because you must instantiate them to use them. -Steve
Jan 31 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/13 10:03 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 07:27:08 -0500, d coder <dlang.coder gmail.com> wrote:

 On Thu, Jan 31, 2013 at 5:30 PM, monarch_dodra
 <monarchdodra gmail.com> wrote:
 The pull is kind of stuck in limbo, specifically because of the problems
 associated with implementing reference semantics with structs :/

Thanks for the enlightening email. I am of the considered view that reference semantics with structs in D is tough (if not impossible) with default constructor and postblit constructor (used when passing objects). This is because you can not initialize any object (wrapped in the struct) in the default constructor and if you are passing the struct as a function parameter, it is not possible to initialize these internal objects (before passing them) in the postblit constructor. I faced this in some of my code. Do not know if you are facing the same scenario in the DList implementation.

It's pretty simple. Containers are reference types, and it makes most sense to pass them by reference. In order to be sane, containers must be implemented as classes. Otherwise, you have behavior like this: void foo(int[int] aa, int x, int y) { aa[x] = y; } void main() { int[int] aa; foo(aa, 1, 2); aa[3] = 4; foo(aa, 5, 6); assert(!(1 in aa)); assert(aa[3] == 4); assert(aa[5] == 6); } In other words, the only way to turn a default-instantiated struct into a valid struct is to use it once. Not the behavior you want. Classes don't have this problem, because you must instantiate them to use them.

As far as I can tell classes have the same problem. Andrei
Jan 31 2013
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/13 10:18 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu
 As far as I can tell classes have the same problem.

Nope. void foo(someclass aa, int x, int y) { aa[x] = y; } void main() { someclass aa; foo(aa, 1, 2); // segfault ... }

We could easily arrange things to segfault just the same with a struct-based implementation. Andrei
Jan 31 2013
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/13 10:29 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:21:04 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu
 As far as I can tell classes have the same problem.

Nope. void foo(someclass aa, int x, int y) { aa[x] = y; } void main() { someclass aa; foo(aa, 1, 2); // segfault ... }

We could easily arrange things to segfault just the same with a struct-based implementation.

So you want to make a struct that acts just like a class? I'm not seeing the point.

It has a destructor. Andrei
Jan 31 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/13 11:00 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:40:15 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 It has a destructor.

One which is not called if allocated on the heap.

That is correct. My point was that with structs we get to implement full-fledged reference counting for containers. Reference counting makes a lot of sense for containers. They inherently own their internals, don't have cycles, and are massive enough to make pass by value effectively an anti-pattern (as it is for C++). At the same time memory reclamation is of high interest. Reference counting is really the sweet spot for containers.
 It's possible to make a class reference that is destroyed when going out
 of scope. Then you have the option, heap destroyed or stack destroyed.

 Not possible with structs (at least for now).

I don't understand this. Is it about the deprecated meaning of scope? Andrei
Jan 31 2013
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 31.01.2013 17:17, Andrei Alexandrescu wrote:
 On 1/31/13 11:00 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:40:15 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 It has a destructor.

One which is not called if allocated on the heap.

That is correct. My point was that with structs we get to implement full-fledged reference counting for containers. Reference counting makes a lot of sense for containers. They inherently own their internals, don't have cycles, and are massive enough to make pass by value effectively an anti-pattern (as it is for C++). At the same time memory reclamation is of high interest. Reference counting is really the sweet spot for containers.

I can understand interest in early and deterministic reclamation of memory, but I have some issues with reference counting. Maybe you can shed some light on these: - how do you reference count immutable containers? You'll have to cast the payload to mutable and assume it is not in read-only memory. - how do you do reference counting when accessing shared containers in multiple threads? Consider clearing the last reference to a shared reference counted object while another thread reads this reference. With usual atomic operations on the reference count only: 1. the reading thread reads the payload pointer 2. the writing thread reads the payload pointer, decrements the reference count and frees the array 3. the reading thread increments the reference count, but accesses the deallocated array. With atomic operations, I can only imagine fixing this with CAS2 operations reading/changing both pointer and reference count at the same time, but even then, it is not obvious. Anyway, this operation is not supported by the currently popular processors. In contrast, GC managed memory is memory- safe in multi-threading environments by design.
Jan 31 2013
next sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 31.01.2013 21:48, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 15:32:23 -0500, Rainer Schuetze <r.sagitario gmx.de>
 wrote:

 - how do you reference count immutable containers? You'll have to cast
 the payload to mutable and assume it is not in read-only memory.

The reference count part is not immutable. e.g.: struct refcountstruct { int count; immutable(containerimpl) impl; }

What about immutable(refcountstruct)? Maybe only implicitely as field of an immutable class.
 - how do you do reference counting when accessing shared containers in
 multiple threads? Consider clearing the last reference to a shared
 reference counted object while another thread reads this reference.
 With usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses
 the deallocated array.

I would be concerned if a thread can get ahold of a reference counted pointer without having the counter incremented already on its behalf.

You have to read the pointer to increment the counter to get hold of the reference.
Jan 31 2013
next sibling parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 31.01.2013 21:55, Rainer Schuetze wrote:
 On 31.01.2013 21:48, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 15:32:23 -0500, Rainer Schuetze <r.sagitario gmx.de>
 wrote:

 - how do you reference count immutable containers? You'll have to cast
 the payload to mutable and assume it is not in read-only memory.

The reference count part is not immutable. e.g.: struct refcountstruct { int count; immutable(containerimpl) impl; }

What about immutable(refcountstruct)? Maybe only implicitely as field of an immutable class.

That would probably be better immutable(refcountstruct*) as field of the immutable container struct.
Jan 31 2013
prev sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 31.01.2013 22:37, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 15:55:11 -0500, Rainer Schuetze <r.sagitario gmx.de>
 wrote:
 - how do you do reference counting when accessing shared containers in
 multiple threads? Consider clearing the last reference to a shared
 reference counted object while another thread reads this reference.
 With usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses
 the deallocated array.

I would be concerned if a thread can get ahold of a reference counted pointer without having the counter incremented already on its behalf.

You have to read the pointer to increment the counter to get hold of the reference.

Right, but where do you get the original pointer from? Wouldn't that reference have incremented the count?

Any reference that is accessible from multiple threads, e.g. a global shared variable. shared(Container) allObjects; Without any thread having a reference to it, the payload's reference count is 1. The reading thread above increments the counter when making a local copy, but the writing thread decrements the counter when assigning Container.init. If both run concurrently, with the sequence 1 to 3 above, trouble is inevitable.
Jan 31 2013
next sibling parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 31.01.2013 23:37, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 17:31:38 -0500, Rainer Schuetze <r.sagitario gmx.de>
 wrote:

 Any reference that is accessible from multiple threads, e.g. a global
 shared variable.

 shared(Container) allObjects;

 Without any thread having a reference to it, the payload's reference
 count is 1. The reading thread above increments the counter when
 making a local copy, but the writing thread decrements the counter
 when assigning Container.init. If both run concurrently, with the
 sequence 1 to 3 above, trouble is inevitable.

Right, but isn't that an issue with having ANY shared variable that isn't protected by a lock?

True, as soon as you start mutating the variable. In case of simple reference/pointer without reference count, the change is atomic though (depending on the CPU or the implementation of "shared") and the reader thread just gets the old or the new pointer, but never an invalid pointer. My main point actually was that a lot of people seem to think that reference counting with atomic incrementing/decrementing the counter (like it is used in COM) is thread safe in general. I think they are wrong.
 I was thinking you had passed the pointer in via a message or something.

I agree, message passing should not have this issue. It needs that single reference being accessed from different threads.
Jan 31 2013
prev sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 01.02.2013 00:14, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 17:37:39 -0500, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:

 On Thu, 31 Jan 2013 17:31:38 -0500, Rainer Schuetze
 <r.sagitario gmx.de> wrote:

 Any reference that is accessible from multiple threads, e.g. a global
 shared variable.

 shared(Container) allObjects;

 Without any thread having a reference to it, the payload's reference
 count is 1. The reading thread above increments the counter when
 making a local copy, but the writing thread decrements the counter
 when assigning Container.init. If both run concurrently, with the
 sequence 1 to 3 above, trouble is inevitable.

Right, but isn't that an issue with having ANY shared variable that isn't protected by a lock? I was thinking you had passed the pointer in via a message or something.

BTW, couldn't we solve this with a function that increments the retain count, and then returns a pointer to the object? Such a method would have to be atomic on shared instances.

The problem is to make it atomic without expensive locks. Lacking the CAS2 operation (that does a CAS on two arbitrary memory locations simultaneously), my first thought was that it is not possible. Considering it again, I guess it can be done by not using atomic increment/decrement, but splitting up the operations. The main idea is to no longer touch the data after the reference count has become 0 once: // CAS returns *var before operation, only modifies if *var == expected size_t CAS(size_t* var, size_t expected, size_t newval); struct PayLoad(T) { T* array; size_t size; size_t refCount; PayLoad!T* ref() { size_t cnt = refCount; while(cnt) { int old = CAS(&refCount, cnt, cnt+1); if(old == cnt) return this; cnt = refCount; } // in the rare case the payload has become invalid, // return a new empty object to avoid having to // check for null (could also be null if the container // always checks its payload pointer) return new PayLoad!T; } void deref() { size_t cnt, old; do { assert(refCount); cnt = refCount; old = CAS(&refCount, cnt, cnt-1); } while(old != cnt); if(refCount == 0) free(array); } } struct Container(T) { PayLoad!T* payload; this() // I know this doesn't work, but would be convenient { payload = new Payload!T; } this(this) { payload = payload.ref(); } ~this() { payload.deref(); } void opAssign(typeof(this) rhs) { if(rhs.payload !is payload) { payload.deref(); payload = rhs.payload.ref(); } } } PayLoad!T will still have to be garbage collected, though. BTW: Maybe I am misunderstanding it, but according to http://dlang.org/operatoroverloading.html overloading opAssign on the same type as in "void opAssign(typeof(this) rhs)" is not allowed. Is this correct? You find it in phobos as well. Another BTW: the links to anchors on the same page are broken as they don't include the html file.
Feb 01 2013
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 01.02.2013 21:02, Steven Schveighoffer wrote:
 On Fri, 01 Feb 2013 14:03:51 -0500, Rainer Schuetze <r.sagitario gmx.de>
 wrote:
 The problem is to make it atomic without expensive locks. Lacking the
 CAS2 operation (that does a CAS on two arbitrary memory locations
 simultaneously), my first thought was that it is not possible.

I don't think expensive locks are an issue here. Having an expensive lock to copy a pointer from a shared location into a thread-local location is worth having an expensive lock, and may even be necessary.

It depends on the application. At work we put a lot of effort into transferring data lock-free from/to a low latency thread processing audio. No blocking is allowed in this thread. (You cannot guarantee this with D at the moment, but IMO it only takes a few tweaks to the runtime.)
 Once you have the thread-local copy of the reference, incrementing and
 decrementing the reference count can be done via CAS.

Yes. My proposed code is only slightly more complex than atomic increments/decrements, but should also work for shared(Container).
Feb 01 2013
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
01-Feb-2013 00:32, Rainer Schuetze пишет:
 On 31.01.2013 17:17, Andrei Alexandrescu wrote:
 On 1/31/13 11:00 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:40:15 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 It has a destructor.

One which is not called if allocated on the heap.

That is correct. My point was that with structs we get to implement full-fledged reference counting for containers. Reference counting makes a lot of sense for containers. They inherently own their internals, don't have cycles, and are massive enough to make pass by value effectively an anti-pattern (as it is for C++). At the same time memory reclamation is of high interest. Reference counting is really the sweet spot for containers.

I can understand interest in early and deterministic reclamation of memory, but I have some issues with reference counting. Maybe you can shed some light on these: - how do you reference count immutable containers? You'll have to cast the payload to mutable and assume it is not in read-only memory.

Containers of immutables can have ref-count easily. And I'd say fully immutable containers are rare case and can rely on GC.
 - how do you do reference counting when accessing shared containers in
 multiple threads?

Atomic Inc/Dec.
 Consider clearing the last reference to a shared
 reference counted object while another thread reads this reference. With
 usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses
 the deallocated array.

Can't happen as reading thread got to have a reference and so does the writing thread, then refcount >= 2.
 With atomic operations, I can only imagine fixing this with CAS2
 operations reading/changing both pointer and reference count at the same
 time, but even then, it is not obvious. Anyway, this operation is not
 supported by the currently popular processors.

 In contrast, GC managed memory is memory- safe in multi-threading
 environments by design.

-- Dmitry Olshansky
Feb 01 2013
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 01.02.2013 09:06, Dmitry Olshansky wrote:
 01-Feb-2013 00:32, Rainer Schuetze пишет:
 - how do you reference count immutable containers? You'll have to cast
 the payload to mutable and assume it is not in read-only memory.

Containers of immutables can have ref-count easily. And I'd say fully immutable containers are rare case and can rely on GC.

Do you want different implementations of the containers depending on the mutable/immutable/shared modifier? Would that be possible?
 - how do you do reference counting when accessing shared containers in
 multiple threads?

Atomic Inc/Dec.

Not good enough as shown in the discussion with Steven.
 Consider clearing the last reference to a shared
 reference counted object while another thread reads this reference. With
 usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses
 the deallocated array.

Can't happen as reading thread got to have a reference and so does the writing thread, then refcount >= 2.

Can happen with a single shared reference accessed from two threads.
Feb 01 2013
next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
01-Feb-2013 23:10, Rainer Schuetze пишет:
 On 01.02.2013 09:06, Dmitry Olshansky wrote:
 01-Feb-2013 00:32, Rainer Schuetze пишет:
 - how do you reference count immutable containers? You'll have to cast
 the payload to mutable and assume it is not in read-only memory.

Containers of immutables can have ref-count easily. And I'd say fully immutable containers are rare case and can rely on GC.

Do you want different implementations of the containers depending on the mutable/immutable/shared modifier? Would that be possible?

In fact I do ;) It's possible base on element type not the whole container itself.
 - how do you do reference counting when accessing shared containers in
 multiple threads?

Atomic Inc/Dec.

Not good enough as shown in the discussion with Steven.
 Consider clearing the last reference to a shared
 reference counted object while another thread reads this reference. With
 usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses
 the deallocated array.

Can't happen as reading thread got to have a reference and so does the writing thread, then refcount >= 2.

Can happen with a single shared reference accessed from two threads.

You mean shared reference to shared ref-counter object? Awful and BTW impossible in D as you'd have to cast your way into it :) -- Dmitry Olshansky
Feb 01 2013
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 01.02.2013 20:36, Dmitry Olshansky wrote:
 01-Feb-2013 23:10, Rainer Schuetze пишет:
 Consider clearing the last reference to a shared
 reference counted object while another thread reads this reference.
 With
 usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses
 the deallocated array.

Can't happen as reading thread got to have a reference and so does the writing thread, then refcount >= 2.

Can happen with a single shared reference accessed from two threads.

You mean shared reference to shared ref-counter object? Awful and BTW impossible in D as you'd have to cast your way into it :)

Maybe I'm coding too much C++, but isn't this what the "shared" modifier is all about?
Feb 01 2013
prev sibling parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 01.02.2013 20:18, monarch_dodra wrote:
 On Friday, 1 February 2013 at 19:10:26 UTC, Rainer Schuetze wrote:
 On 01.02.2013 09:06, Dmitry Olshansky wrote:
 01-Feb-2013 00:32, Rainer Schuetze пишет:
 - how do you reference count immutable containers? You'll have to cast
 the payload to mutable and assume it is not in read-only memory.

Containers of immutables can have ref-count easily. And I'd say fully immutable containers are rare case and can rely on GC.

Do you want different implementations of the containers depending on the mutable/immutable/shared modifier? Would that be possible?

You can only do that on the qualifier of the *parameters*, not the container itself.

Ok, but that won't help changing reference count semantics, because the type of the elements are irrelevant here. It's about the container itself.
 This is not possible because an S "is a" const(S). If the implementation
 of an S was different from a const(S), then you'd violate that. At best,
 a const(S) is an S with restricted possibilities, bot not *different*
 possibilities.

I guess you would have to convert it to a different type, but it can be an efficient operation transferring the payload unmodified. Also, when converting to shared, you might even be able to verify that there are no other references left (that must be non-sharing).
Feb 01 2013
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
31-Jan-2013 19:21, Andrei Alexandrescu пишет:
 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu
 As far as I can tell classes have the same problem.

Nope. void foo(someclass aa, int x, int y) { aa[x] = y; } void main() { someclass aa; foo(aa, 1, 2); // segfault ... }

We could easily arrange things to segfault just the same with a struct-based implementation.

Structs are quite borked in this regard e.g. without extra efforts the following: somclass aa = someclass(); foor(aa, 1, 2); // segfault, surprize someclass() is someclass.init The current workaround I find the most sensible is: - disable this(); - make all constructors private - define opCall and forward it to private constructors. 0-arg versions have to pass dummy and/or default values to get struct constructed - automate this boilerplate until something in language is fixed? :) The advantage is that the following is illegal: someclass aa; and the following works as expected: auto aa = someclass(); //create empty container damn it! -- Dmitry Olshansky
Jan 31 2013
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
31-Jan-2013 23:32, Maxim Fomin пишет:
 On Thursday, 31 January 2013 at 19:17:56 UTC, Dmitry Olshansky wrote:
 31-Jan-2013 19:21, Andrei Alexandrescu пишет:
 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu
 As far as I can tell classes have the same problem.

Nope. void foo(someclass aa, int x, int y) { aa[x] = y; } void main() { someclass aa; foo(aa, 1, 2); // segfault ... }

We could easily arrange things to segfault just the same with a struct-based implementation.

Structs are quite borked in this regard e.g. without extra efforts the following: somclass aa = someclass(); foor(aa, 1, 2); // segfault, surprize someclass() is someclass.init The current workaround I find the most sensible is: - disable this(); - make all constructors private - define opCall and forward it to private constructors. 0-arg versions have to pass dummy and/or default values to get struct constructed - automate this boilerplate until something in language is fixed? :) The advantage is that the following is illegal: someclass aa; and the following works as expected: auto aa = someclass(); //create empty container damn it!

If desired default struct value is constant for all instances of that struct, there is another workaround by changing init values: ------------------- import std.stdio; class A { int x = 5; } struct S { int x = 5; A a; } void main() { S s; // defaults to preallocated a, a is not null assert(S.x is 100 && s.a !is null && s.a.x is 100); }

This is fine except that it doesn't solve the problem of: Container a = Container(); //should be empty container, not illegal null-container Basically dsiable this() is optional, but I like to help user avoid bugs.
 import runtime;

 mixin(declareExternInitZPointer!S);

 static this()
 {
      A a = new A;
      a.x = 100;
      S s =  S(100, a);
      rtSetDefaultHeapInitializer(s);
      rtSetDefaultStackInitializer(s, mixin(passExternInitZPointer!S));
 }
 ----------------------

 Where runtime module is http://dpaste.dzfl.pl/40b59a5d

 This does not fully replace default ctor, since all instances of S are
 affected.

-- Dmitry Olshansky
Jan 31 2013
next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
31-Jan-2013 23:54, Robert burner Schadek пишет:
 On 01/31/2013 08:34 PM, Dmitry Olshansky wrote:
 31-Jan-2013 23:32, Maxim Fomin пишет:
 On Thursday, 31 January 2013 at 19:17:56 UTC, Dmitry Olshansky wrote:
 31-Jan-2013 19:21, Andrei Alexandrescu пишет:
 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu
 As far as I can tell classes have the same problem.

Nope. void foo(someclass aa, int x, int y) { aa[x] = y; } void main() { someclass aa; foo(aa, 1, 2); // segfault ... }

We could easily arrange things to segfault just the same with a struct-based implementation.

Structs are quite borked in this regard e.g. without extra efforts the following: somclass aa = someclass(); foor(aa, 1, 2); // segfault, surprize someclass() is someclass.init The current workaround I find the most sensible is: - disable this(); - make all constructors private - define opCall and forward it to private constructors. 0-arg versions have to pass dummy and/or default values to get struct constructed - automate this boilerplate until something in language is fixed? :) The advantage is that the following is illegal: someclass aa; and the following works as expected: auto aa = someclass(); //create empty container damn it!

If desired default struct value is constant for all instances of that struct, there is another workaround by changing init values: ------------------- import std.stdio; class A { int x = 5; } struct S { int x = 5; A a; } void main() { S s; // defaults to preallocated a, a is not null assert(S.x is 100 && s.a !is null && s.a.x is 100); }

This is fine except that it doesn't solve the problem of: Container a = Container(); //should be empty container, not illegal null-container

operation.

Lazy evaluation, great. Except that is a useless opcode per every operation with container + keep in in mind the problem: void func(cont a) { //if a was empty and we go with lazy-initialization // then the outside cont is not updated a.insert(...); } void main() { ... cont x; ... //this could be non-trivial code that //may or may not touch x in all path func(x); ... } -- Dmitry Olshansky
Jan 31 2013
prev sibling next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
31-Jan-2013 23:51, Maxim Fomin пишет:
 On Thursday, 31 January 2013 at 19:34:29 UTC, Dmitry Olshansky wrote:
 This is fine except that it doesn't solve the problem of:
 Container a = Container(); //should be empty container, not illegal
 null-container

 Basically  dsiable this() is optional, but I like to help user avoid
 bugs.

You have static opCall for such things. Since when this was a problem?

Yes, that's why I use it still... it can't initialize a const struct *and* requires a T.init or T a = void; or forward to this(int dummy=42) container. -- Dmitry Olshansky
Jan 31 2013
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/13 2:54 PM, Robert burner Schadek wrote:
[snip]

Please avoid overquoting - this is the worst there is: one line inserted 
without whitespace in the middle of a long quote.

Thanks,

Andrei
Jan 31 2013
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/13 10:27 AM, monarch_dodra wrote:
 On Thursday, 31 January 2013 at 15:12:52 UTC, Andrei Alexandrescu wrote:
 As far as I can tell classes have the same problem.


 Andrei

Regarding classes, would there be any chance of being to create a class instance that isn't a child of "object"?

Using classes entails buying into an entire object model with its own pluses and minuses. Adding classes that don't inherit Object would wreck havoc all over the place.
 I'm not entirely sure what the implications are, but it seems that is
 the major "no-sell" argument against using classes.

 I know *I* would be using them a whole lot more if this was the case.

What would be the problem? Andrei
Jan 31 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 1/31/13 10:03 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 07:27:08 -0500, d coder <dlang.coder gmail.com>  
 wrote:

 On Thu, Jan 31, 2013 at 5:30 PM, monarch_dodra
 <monarchdodra gmail.com> wrote:
 The pull is kind of stuck in limbo, specifically because of the  
 problems
 associated with implementing reference semantics with structs :/

Thanks for the enlightening email. I am of the considered view that reference semantics with structs in D is tough (if not impossible) with default constructor and postblit constructor (used when passing objects). This is because you can not initialize any object (wrapped in the struct) in the default constructor and if you are passing the struct as a function parameter, it is not possible to initialize these internal objects (before passing them) in the postblit constructor. I faced this in some of my code. Do not know if you are facing the same scenario in the DList implementation.

It's pretty simple. Containers are reference types, and it makes most sense to pass them by reference. In order to be sane, containers must be implemented as classes. Otherwise, you have behavior like this: void foo(int[int] aa, int x, int y) { aa[x] = y; } void main() { int[int] aa; foo(aa, 1, 2); aa[3] = 4; foo(aa, 5, 6); assert(!(1 in aa)); assert(aa[3] == 4); assert(aa[5] == 6); } In other words, the only way to turn a default-instantiated struct into a valid struct is to use it once. Not the behavior you want. Classes don't have this problem, because you must instantiate them to use them.

As far as I can tell classes have the same problem.

Nope. void foo(someclass aa, int x, int y) { aa[x] = y; } void main() { someclass aa; foo(aa, 1, 2); // segfault ... } -Steve
Jan 31 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 31 January 2013 at 15:12:52 UTC, Andrei Alexandrescu 
wrote:
 As far as I can tell classes have the same problem.


 Andrei

Regarding classes, would there be any chance of being to create a class instance that isn't a child of "object"? I'm not entirely sure what the implications are, but it seems that is the major "no-sell" argument against using classes. I know *I* would be using them a whole lot more if this was the case.
Jan 31 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 31 January 2013 at 15:21:02 UTC, Andrei Alexandrescu 
wrote:
 We could easily arrange things to segfault just the same with a 
 struct-based implementation.

 Andrei

Not if you have a "lazy initialization scheme", which is the current scheme we are using... ...and will probably continue to use, since we have no standard way of forcing initialization :(
Jan 31 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 10:21:04 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu
 As far as I can tell classes have the same problem.

Nope. void foo(someclass aa, int x, int y) { aa[x] = y; } void main() { someclass aa; foo(aa, 1, 2); // segfault ... }

We could easily arrange things to segfault just the same with a struct-based implementation.

So you want to make a struct that acts just like a class? I'm not seeing the point. -Steve
Jan 31 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 10:40:15 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 1/31/13 10:29 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:21:04 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu
 As far as I can tell classes have the same problem.

Nope. void foo(someclass aa, int x, int y) { aa[x] = y; } void main() { someclass aa; foo(aa, 1, 2); // segfault ... }

We could easily arrange things to segfault just the same with a struct-based implementation.

So you want to make a struct that acts just like a class? I'm not seeing the point.

It has a destructor.

One which is not called if allocated on the heap. It's possible to make a class reference that is destroyed when going out of scope. Then you have the option, heap destroyed or stack destroyed. Not possible with structs (at least for now). -Steve
Jan 31 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, January 31, 2013 10:39:46 Andrei Alexandrescu wrote:
 On 1/31/13 10:27 AM, monarch_dodra wrote:
 On Thursday, 31 January 2013 at 15:12:52 UTC, Andrei Alexandrescu wrote:
 As far as I can tell classes have the same problem.
 
 
 Andrei

Regarding classes, would there be any chance of being to create a class instance that isn't a child of "object"?

Using classes entails buying into an entire object model with its own pluses and minuses. Adding classes that don't inherit Object would wreck havoc all over the place.

And what would be the gain, anyway? I don't understand what the downside is to having a single base object besides the issues with toString, toHash, opEquals, and opCmp. And we decided to remove all of them from Object (though no actual progress has been made beyond the decision), and with those gone, Object will have next to nothing on it anyway. But not having Object be the base of all classes would definitely cause quite a few problems. - Jonathan M Davis
Jan 31 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 31 January 2013 at 16:17:09 UTC, Jonathan M Davis 
wrote:
 On Thursday, January 31, 2013 10:39:46 Andrei Alexandrescu 
 wrote:
 On 1/31/13 10:27 AM, monarch_dodra wrote:
 On Thursday, 31 January 2013 at 15:12:52 UTC, Andrei 
 Alexandrescu wrote:
 As far as I can tell classes have the same problem.
 
 
 Andrei

Regarding classes, would there be any chance of being to create a class instance that isn't a child of "object"?

Using classes entails buying into an entire object model with its own pluses and minuses. Adding classes that don't inherit Object would wreck havoc all over the place.

And what would be the gain, anyway? I don't understand what the downside is to having a single base object besides the issues with toString, toHash, opEquals, and opCmp. And we decided to remove all of them from Object (though no actual progress has been made beyond the decision), and with those gone, Object will have next to nothing on it anyway. But not having Object be the base of all classes would definitely cause quite a few problems. - Jonathan M Davis

I guess it still has the vtable and monitor. In any case, the question was mostly asked to "test the waters" in regards to "class usage best practice", and to probe the consequences of such a change. Please disregard the question in further discussion.
Jan 31 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 11:17:14 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 1/31/13 11:00 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:40:15 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 It has a destructor.

One which is not called if allocated on the heap.

That is correct. My point was that with structs we get to implement full-fledged reference counting for containers.

But that won't work if the struct is on the heap. You will leak the memory. Not only that, but if the container uses the GC to allocate nodes, calling the destructor during the GC cycle would be a bad thing.
 Reference counting makes a lot of sense for containers. They inherently  
 own their internals, don't have cycles, and are massive enough to make  
 pass by value effectively an anti-pattern (as it is for C++). At the  
 same time memory reclamation is of high interest. Reference counting is  
 really the sweet spot for containers.

Reference counting makes a lot of sense for any reference type. Including classes. I think your point is that we can do reference counting with structs, and we can't with classes. I think: a) you can do reference counting with both (at the moment, this requires a wrapper struct for the class) b) structs that depend on destruction to free their memory do not work with the current runtime implementation.
 It's possible to make a class reference that is destroyed when going out
 of scope. Then you have the option, heap destroyed or stack destroyed.

 Not possible with structs (at least for now).

I don't understand this. Is it about the deprecated meaning of scope?

It means, classes have a mechanism for destruction via the heap (GC) or via the stack (scope, or whatever currently replaces scope (Scoped? I can't remember) ). Structs by default are easily destroyed from the stack, but lack the fundamental cogs to get GC-based destruction. In other words, you can't put a reference-counted struct on the heap, it won't work correctly (the destructor won't get called). This is not to say this can't be fixed, but if we are talking about current implementation limitations, classes are superior in how they can be used. -Steve
Jan 31 2013
prev sibling next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Thursday, 31 January 2013 at 18:59:20 UTC, Steven 
Schveighoffer wrote:
 Structs by default are easily destroyed from the stack, but 
 lack the fundamental cogs to get GC-based destruction.

 -Steve

Perhaps _d_newitemT() and buddies can check whether it creates structs, store a collection of pointers to structs it allocates and at the end of program some druntime routine loops through out of the collection to run dtors?
Jan 31 2013
prev sibling next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Thursday, 31 January 2013 at 19:17:56 UTC, Dmitry Olshansky 
wrote:
 31-Jan-2013 19:21, Andrei Alexandrescu пишет:
 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu
 As far as I can tell classes have the same problem.

Nope. void foo(someclass aa, int x, int y) { aa[x] = y; } void main() { someclass aa; foo(aa, 1, 2); // segfault ... }

We could easily arrange things to segfault just the same with a struct-based implementation.

Structs are quite borked in this regard e.g. without extra efforts the following: somclass aa = someclass(); foor(aa, 1, 2); // segfault, surprize someclass() is someclass.init The current workaround I find the most sensible is: - disable this(); - make all constructors private - define opCall and forward it to private constructors. 0-arg versions have to pass dummy and/or default values to get struct constructed - automate this boilerplate until something in language is fixed? :) The advantage is that the following is illegal: someclass aa; and the following works as expected: auto aa = someclass(); //create empty container damn it!

If desired default struct value is constant for all instances of that struct, there is another workaround by changing init values: ------------------- import std.stdio; class A { int x = 5; } struct S { int x = 5; A a; } void main() { S s; // defaults to preallocated a, a is not null assert(S.x is 100 && s.a !is null && s.a.x is 100); } import runtime; mixin(declareExternInitZPointer!S); static this() { A a = new A; a.x = 100; S s = S(100, a); rtSetDefaultHeapInitializer(s); rtSetDefaultStackInitializer(s, mixin(passExternInitZPointer!S)); } ---------------------- Where runtime module is http://dpaste.dzfl.pl/40b59a5d This does not fully replace default ctor, since all instances of S are affected.
Jan 31 2013
prev sibling next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Thursday, 31 January 2013 at 19:34:29 UTC, Dmitry Olshansky 
wrote:
 This is fine except that it doesn't solve the problem of:
 Container a = Container(); //should be empty container, not 
 illegal null-container

 Basically  dsiable this() is optional, but I like to help user 
 avoid bugs.

You have static opCall for such things. Since when this was a problem?
Jan 31 2013
prev sibling next sibling parent Robert burner Schadek <realburner gmx.de> writes:
On 01/31/2013 08:34 PM, Dmitry Olshansky wrote:
 31-Jan-2013 23:32, Maxim Fomin пишет:
 On Thursday, 31 January 2013 at 19:17:56 UTC, Dmitry Olshansky wrote:
 31-Jan-2013 19:21, Andrei Alexandrescu пишет:
 On 1/31/13 10:18 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:12:53 -0500, Andrei Alexandrescu
 As far as I can tell classes have the same problem.

Nope. void foo(someclass aa, int x, int y) { aa[x] = y; } void main() { someclass aa; foo(aa, 1, 2); // segfault ... }

We could easily arrange things to segfault just the same with a struct-based implementation.

Structs are quite borked in this regard e.g. without extra efforts the following: somclass aa = someclass(); foor(aa, 1, 2); // segfault, surprize someclass() is someclass.init The current workaround I find the most sensible is: - disable this(); - make all constructors private - define opCall and forward it to private constructors. 0-arg versions have to pass dummy and/or default values to get struct constructed - automate this boilerplate until something in language is fixed? :) The advantage is that the following is illegal: someclass aa; and the following works as expected: auto aa = someclass(); //create empty container damn it!

If desired default struct value is constant for all instances of that struct, there is another workaround by changing init values: ------------------- import std.stdio; class A { int x = 5; } struct S { int x = 5; A a; } void main() { S s; // defaults to preallocated a, a is not null assert(S.x is 100 && s.a !is null && s.a.x is 100); }

This is fine except that it doesn't solve the problem of: Container a = Container(); //should be empty container, not illegal null-container

operation.
 Basically  dsiable this() is optional, but I like to help user avoid 
 bugs.

 import runtime;

 mixin(declareExternInitZPointer!S);

 static this()
 {
      A a = new A;
      a.x = 100;
      S s =  S(100, a);
      rtSetDefaultHeapInitializer(s);
      rtSetDefaultStackInitializer(s, mixin(passExternInitZPointer!S));
 }
 ----------------------

 Where runtime module is http://dpaste.dzfl.pl/40b59a5d

 This does not fully replace default ctor, since all instances of S are
 affected.


Jan 31 2013
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Thursday, 31 January 2013 at 19:55:01 UTC, Robert burner 
Schadek wrote:
 Just use a payload struct and check whether that is null before 
 every operation.

The following problem is inherent to this approach: --- struct Array(T) { static struct Payload { T* data; size_t length; } Payload* p; // ... } void fill(T)(Array!T a) { a.length = 3; a[0] = 1; a[1] = 2; a[2] = 3; } void client() { Array!int a; fill(a); // Oops, a is still empty (with null payload). } --- Such containers are "almost, but not quite" reference types; reason enough to disable this() in order to improve the user experience, in my opinion. David
Jan 31 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 14:22:24 -0500, Maxim Fomin <maxim maxim-fomin.ru>  
wrote:

 On Thursday, 31 January 2013 at 18:59:20 UTC, Steven Schveighoffer wrote:
 Structs by default are easily destroyed from the stack, but lack the  
 fundamental cogs to get GC-based destruction.

 -Steve

Perhaps _d_newitemT() and buddies can check whether it creates structs, store a collection of pointers to structs it allocates and at the end of program some druntime routine loops through out of the collection to run dtors?

What is needed is a precise GC, where the struct typeinfo and destructor are stored along with the block. However, we ALSO need two functions, a destructor (Called from the stack deterministically) and a finalizer (called from the GC). Like I said, this isn't an unsolvable problem, but neither is making classes reference counted. -Steve
Jan 31 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 15:32:23 -0500, Rainer Schuetze <r.sagitario gmx.de>  
wrote:

 On 31.01.2013 17:17, Andrei Alexandrescu wrote:
 On 1/31/13 11:00 AM, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 10:40:15 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 It has a destructor.

One which is not called if allocated on the heap.

That is correct. My point was that with structs we get to implement full-fledged reference counting for containers. Reference counting makes a lot of sense for containers. They inherently own their internals, don't have cycles, and are massive enough to make pass by value effectively an anti-pattern (as it is for C++). At the same time memory reclamation is of high interest. Reference counting is really the sweet spot for containers.

I can understand interest in early and deterministic reclamation of memory, but I have some issues with reference counting. Maybe you can shed some light on these: - how do you reference count immutable containers? You'll have to cast the payload to mutable and assume it is not in read-only memory.

The reference count part is not immutable. e.g.: struct refcountstruct { int count; immutable(containerimpl) impl; }
 - how do you do reference counting when accessing shared containers in  
 multiple threads? Consider clearing the last reference to a shared  
 reference counted object while another thread reads this reference. With  
 usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the  
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses  
 the deallocated array.

I would be concerned if a thread can get ahold of a reference counted pointer without having the counter incremented already on its behalf. -Steve
Jan 31 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 15:55:11 -0500, Rainer Schuetze <r.sagitario gmx.de>  
wrote:

 On 31.01.2013 21:48, Steven Schveighoffer wrote:
 On Thu, 31 Jan 2013 15:32:23 -0500, Rainer Schuetze <r.sagitario gmx.de>
 wrote:

 - how do you reference count immutable containers? You'll have to cast
 the payload to mutable and assume it is not in read-only memory.

The reference count part is not immutable. e.g.: struct refcountstruct { int count; immutable(containerimpl) impl; }

What about immutable(refcountstruct)? Maybe only implicitely as field of an immutable class.

You wouldn't be able to do that. The reference count needs to be tail-immutable, meaning the reference count is not immutable, but the data it protects is. Anticipating your answer, no, there isn't a good way to do this at the moment, it's something D definitely lacks.
 - how do you do reference counting when accessing shared containers in
 multiple threads? Consider clearing the last reference to a shared
 reference counted object while another thread reads this reference.
 With usual atomic operations on the reference count only:

    1. the reading thread reads the payload pointer
    2. the writing thread reads the payload pointer, decrements the
 reference count and frees the array
    3. the reading thread increments the reference count, but accesses
 the deallocated array.

I would be concerned if a thread can get ahold of a reference counted pointer without having the counter incremented already on its behalf.

You have to read the pointer to increment the counter to get hold of the reference.

Right, but where do you get the original pointer from? Wouldn't that reference have incremented the count? -Steve
Jan 31 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 17:31:38 -0500, Rainer Schuetze <r.sagitario gmx.de>  
wrote:

 Any reference that is accessible from multiple threads, e.g. a global  
 shared variable.

 shared(Container) allObjects;

 Without any thread having a reference to it, the payload's reference  
 count is 1. The reading thread above increments the counter when making  
 a local copy, but the writing thread decrements the counter when  
 assigning Container.init. If both run concurrently, with the sequence 1  
 to 3 above, trouble is inevitable.

Right, but isn't that an issue with having ANY shared variable that isn't protected by a lock? I was thinking you had passed the pointer in via a message or something. -Steve
Jan 31 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 31 Jan 2013 17:37:39 -0500, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 On Thu, 31 Jan 2013 17:31:38 -0500, Rainer Schuetze <r.sagitario gmx.de>  
 wrote:

 Any reference that is accessible from multiple threads, e.g. a global  
 shared variable.

 shared(Container) allObjects;

 Without any thread having a reference to it, the payload's reference  
 count is 1. The reading thread above increments the counter when making  
 a local copy, but the writing thread decrements the counter when  
 assigning Container.init. If both run concurrently, with the sequence 1  
 to 3 above, trouble is inevitable.

Right, but isn't that an issue with having ANY shared variable that isn't protected by a lock? I was thinking you had passed the pointer in via a message or something.

BTW, couldn't we solve this with a function that increments the retain count, and then returns a pointer to the object? Such a method would have to be atomic on shared instances. -Steve
Jan 31 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
Whilst we're talking about containers and refcounted, I've been 
encountering on and off a very strange bug with Array and 
RefCounted.

It is creating stack overflows, and I'm *certain* is creating 
problems when trying to nest connectors. I also think it is the 
"tip" of a more serious bug.

Could anybody with the required skills try to investigate this 
one?

http://d.puremagic.com/issues/show_bug.cgi?id=9438
Feb 01 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 1 February 2013 at 19:10:26 UTC, Rainer Schuetze wrote:
 On 01.02.2013 09:06, Dmitry Olshansky wrote:
 01-Feb-2013 00:32, Rainer Schuetze пишет:
 - how do you reference count immutable containers? You'll 
 have to cast
 the payload to mutable and assume it is not in read-only 
 memory.

Containers of immutables can have ref-count easily. And I'd say fully immutable containers are rare case and can rely on GC.

Do you want different implementations of the containers depending on the mutable/immutable/shared modifier? Would that be possible?

You can only do that on the qualifier of the *parameters*, not the container itself. This is not possible because an S "is a" const(S). If the implementation of an S was different from a const(S), then you'd violate that. At best, a const(S) is an S with restricted possibilities, bot not *different* possibilities.
Feb 01 2013
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 01 Feb 2013 14:03:51 -0500, Rainer Schuetze <r.sagitario gmx.de>  
wrote:
 The problem is to make it atomic without expensive locks. Lacking the  
 CAS2 operation (that does a CAS on two arbitrary memory locations  
 simultaneously), my first thought was that it is not possible.

I don't think expensive locks are an issue here. Having an expensive lock to copy a pointer from a shared location into a thread-local location is worth having an expensive lock, and may even be necessary. Once you have the thread-local copy of the reference, incrementing and decrementing the reference count can be done via CAS. -Steve
Feb 01 2013