digitalmars.D - A Refcounted Array Type
- Walter Bright (85/85) Feb 23 2015 This is pretty straightforward. More could be done:
- Andrei Alexandrescu (11/12) Feb 23 2015 [snip]
- Jonathan M Davis via Digitalmars-d (4/10) Feb 23 2015 And delete is supposed to have been deprecated ages ago, but yeah, it
- Walter Bright (3/5) Feb 23 2015 Managing memory always is going to be unsafe. The idea is to encapsulate...
- Jonathan M Davis via Digitalmars-d (12/17) Feb 24 2015 That's fine. The issue is that delete is considered @safe by the compile...
- Walter Bright (3/4) Feb 24 2015 I thought more people would be interested in how to do a memory safe ref...
- matovitch (3/7) Feb 24 2015 This is really interesting ! Thought as a beginner, I am
- Namespace (9/18) Feb 24 2015 That's why:
- matovitch (2/22) Feb 24 2015 Hmm, I don't see why that's why... :(
- Namespace (4/28) Feb 24 2015 It's more effective if you figure it out on your own.
- "Ulrich =?UTF-8?B?S8O8dHRsZXIi?= <kuettler gmail.com> (2/13) Feb 24 2015 The counter is shared amount all "copies" of the array.
- matovitch (3/17) Feb 24 2015 Ah yes, what an idiot ! Thanks !
- Jonathan M Davis via Digitalmars-d (15/19) Feb 25 2015 Oh, I think that that there's definitely something there and that we sho...
- bearophile (12/13) Feb 23 2015 When you go past bounds of a built-in array you get an error
- Walter Bright (3/11) Feb 23 2015 This is off-topic. The point of the array class as presented is to show ...
- Max Klyga (5/108) Feb 23 2015 I thought that delete is deprecated, yet here Walter himself promotes a
- Walter Bright (5/9) Feb 23 2015 That isn't the point. I could have well have used malloc/free, but I wan...
- weaselcat (3/4) Feb 23 2015 exactly what is this doing? I don't see this explained in DIP25
- Walter Bright (3/6) Feb 23 2015 The 'this' is passed by reference. So the 'return' is really 'return ref...
- weaselcat (2/11) Feb 23 2015 Oh, I see. That makes a lot more sense.
- Andrei Alexandrescu (3/10) Feb 24 2015 We should amend DIP25 to explain that "this" is handled like a
- deadalnix (5/19) Feb 24 2015 Note that I'm not surprised by that behavior (this is indeed ref
- Chris (8/93) Feb 24 2015 Thanks for the example.
- ponce (4/89) Feb 24 2015 Does DIP25 means we can turn resources into RC types and be done
- Andrei Alexandrescu (2/5) Feb 24 2015 That's what we're aiming for. -- Andrei
- Michel Fortin (14/17) Feb 24 2015 Careful!
- Steven Schveighoffer (7/20) Feb 24 2015 Actually, RCArray can never be allocated on GC, or you may corrupt
- Walter Bright (6/10) Feb 24 2015 No, RCArray is not intended for shared access between threads.
- deadalnix (3/11) Feb 24 2015 Maybe we want to fix the GC, exceptions and delegates or disable
- Walter Bright (3/5) Feb 24 2015 Andrei already filed a bug report on the GC issue. If there aren't bug r...
- deadalnix (4/10) Feb 24 2015 Well I filled a bug on the delegate one ~2 years ago, and the 2
- Walter Bright (4/13) Feb 24 2015 Without examples and bug reports, I don't really know what you're talkin...
- Steven Schveighoffer (15/24) Feb 26 2015 No, the GC may have already deallocated the memory count points at, and
- Andrei Alexandrescu (4/6) Feb 26 2015 Yah, that will solve the nonatomic reference counting. What do you think...
- Steven Schveighoffer (6/11) Feb 26 2015 I saw that, it sounds reasonable. I have to mull over what it means.
- "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= (6/10) Feb 26 2015 Ugh, so you still have to identify objects before releasing the
- deadalnix (14/22) Feb 26 2015 class BazingaException : Exception {
- Andrei Alexandrescu (2/23) Feb 26 2015 Could you please walk me through what the matter is here. Thanks. -- And...
- Michel Fortin (24/54) Feb 26 2015 The exception is thrown by t.join() in another thread, after the
- deadalnix (7/9) Feb 26 2015 1/ The originating thread is likely to be dead by the time we
- "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> (4/6) Feb 26 2015 This probably implies forcibly destroying objects whose creating
- Steven Schveighoffer (6/11) Feb 26 2015 I don't think so, those objects can just be destroyed by the
- "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> (4/18) Feb 26 2015 That's true. However, what if the destructors access global
- Steven Schveighoffer (6/22) Feb 26 2015 Hm... I don't know. Is it "disallowed?" I don't think so (a simple test
- ketmar (3/9) Feb 26 2015 but what if i'm doing memory managing myself and i *know* that it's ok t...
- Steven Schveighoffer (11/20) Feb 26 2015 Accessing thread-local data in the GC-run destructor means you are
- ketmar (10/36) Feb 26 2015 let's assume that i have a program that has only one thread. i know it=2...
- Steven Schveighoffer (8/13) Feb 26 2015 In that case, you shouldn't be subject to any kind of race conditions.
- ketmar (2/5) Feb 26 2015 and a warning, please! a warning that i can silence, of course.=
- deadalnix (5/8) Feb 27 2015 His case is not @safe, so we can ignore that problem. However,
- ketmar (13/20) Feb 27 2015 the compiler tends to disagree:
- weaselcat (5/28) Feb 27 2015 Slighty off-topic,
- "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> (2/37) Feb 28 2015 Yeah, we should have distinct finalizers (with or without DIP74).
- Steven Schveighoffer (9/29) Mar 02 2015 I think in this case, the compiler can probably statically reject the
- ketmar (4/38) Mar 02 2015 it can't, 'cause it doesn't know how the given object is going to be=20
- Steven Schveighoffer (6/12) Mar 03 2015 Classes are expected to be GC allocated. But the point is moot, because
- ketmar (8/10) Mar 03 2015 O_O
- Kagamin (2/15) Mar 03 2015 And what's the problem?
- Steven Schveighoffer (14/35) Mar 03 2015 This:
- Kagamin (3/4) Mar 03 2015 You mean assert breaks something?
- Steven Schveighoffer (7/10) Mar 03 2015 OK, if you want to be pedantic:
- Kagamin (4/8) Mar 03 2015 You mean, the class destructor accesses a GC-allocated memory?
- Steven Schveighoffer (3/11) Mar 04 2015 Wow, you really aren't getting it. I give up, it's tiring.
- Kagamin (3/4) Mar 04 2015 Tiring? Did it take that much effort to not say what you mean?
- Steven Schveighoffer (6/9) Mar 04 2015 No tiring like it is to have to explain obvious details to a child.
- Kagamin (12/14) Mar 05 2015 Safety system is only partially proactive in a sense it doesn't
- ketmar (2/24) Mar 03 2015 the answer is in your quotation.=
- "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> (5/33) Feb 27 2015 By "disallowed", I indeed mean "banned". Unfortunately, something
- Steven Schveighoffer (6/11) Feb 26 2015 However, this has to be done carefully. For instance, you have to run
- Andrei Alexandrescu (5/18) Feb 24 2015 Good point. This is already a danger in existing code, e.g. File uses
- deadalnix (19/58) Feb 24 2015 This may not be @safe depending on the type of E. Also, this do
- Walter Bright (10/17) Feb 24 2015 Steven pointed out the correct fix for that.
- "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> (4/8) Feb 25 2015 Just try to do the same with opSlice() and you will see what he
- deadalnix (26/65) Feb 24 2015 This may not be @safe depending on the type of E. Also, this do
- monarch_dodra (7/11) Mar 05 2015 What is the point of keeping start/end? Aren't those baked into
- "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> (5/18) Mar 05 2015 `GC.free()` needs a pointer to the start of the allocated block;
- monarch_dodra (3/23) Mar 05 2015 Still, you shouldn't need "end", and bounds checking would "just
- Andrei Alexandrescu (2/4) Mar 05 2015 Correct. -- Andrei
This is pretty straightforward. More could be done: 1. small array optimization 2. support for ranges as constructor args 3. present a range interface 4. support for malloc/free instead of GC 5. bounds checking 6. the array[] and the count could be allocated together 7. array[] could be just a pointer but the basic idea is there, I didn't want to hide it behind all the other flesh a professional type would have. Note the return in opIndex(). This is DIP25 at work! Compile: dmd rcarray -unittest -main -dip25 =========================================== struct RCArray(E) { this(E[] a) { array = a.dup; start = 0; end = a.length; count = new int; *count = 1; } ~this() { if (count && --*count == 0) delete array; } this(this) { if (count) ++*count; } size_t length() { return end - start; } ref E opIndex(size_t i) return // here's the magic { return array[start + i]; } RCArray opSlice(size_t lwr, size_t upr) { RCArray result = this; result.start = start + lwr; result.end = start + upr; return result; } private: E[] array; size_t start, end; int* count; } unittest { static int[3] s = [7, 6, 4]; auto r = RCArray!int(s); assert(r.length == 3); assert(r[0] == 7); assert(r[1] == 6); assert(r[2] == 4); assert(*r.count == 1); { auto r2 = r; assert(r2[0] == 7); assert(r2[1] == 6); assert(r2[2] == 4); assert(*r.count == 2); r[1] = 3; assert(r2[0] == 7); assert(r2[1] == 3); assert(r2[2] == 4); } assert(*r.count == 1); auto r3 = r[1 .. 3]; r[2] = 9; assert(r3[0] == 3); assert(r3[1] == 9); /+ ref int test(ref RCArray!int rr) { return rr[1]; // this gives error } +/ }
Feb 23 2015
On 2/23/15 2:15 PM, Walter Bright wrote:This is pretty straightforward.[snip] The code builds if you slap a " safe:" at the top. There is one bug in the compiler: "delete" must not be allowed in safe code. The destructor must be trusted. Understanding that this code (sans delete) is safe (and the contribution of DIP25 to that) is of paramount importance. Making it possible to define safe structs that are still able to return reference to their internals is crucial. It paves the way for truly safe reference counted classes. Andrei
Feb 23 2015
On Monday, February 23, 2015 15:28:21 Andrei Alexandrescu via Digitalmars-d wrote:On 2/23/15 2:15 PM, Walter Bright wrote:And delete is supposed to have been deprecated ages ago, but yeah, it _definitely_ shouldn't be considered safe. - Jonathan M DavisThis is pretty straightforward.[snip] The code builds if you slap a " safe:" at the top. There is one bug in the compiler: "delete" must not be allowed in safe code. The destructor must be trusted.
Feb 23 2015
On 2/23/2015 9:59 PM, Jonathan M Davis via Digitalmars-d wrote:And delete is supposed to have been deprecated ages ago, but yeah, it _definitely_ shouldn't be considered safe.Managing memory always is going to be unsafe. The idea is to encapsulate that, which RCArray shows how to do.
Feb 23 2015
On Monday, February 23, 2015 23:02:03 Walter Bright via Digitalmars-d wrote:On 2/23/2015 9:59 PM, Jonathan M Davis via Digitalmars-d wrote:That's fine. The issue is that delete is considered safe by the compiler, not with the idea of RCArray. This code shouldn't compile: void main() safe { auto i = new int(5); delete i; } and it does. Of course, delete is supposed to have been deprecated, but no one has gotten around to doing it. So, maybe that's why no one made it so that it was treated correctly with regards to safe. - Jonathan M DavisAnd delete is supposed to have been deprecated ages ago, but yeah, it _definitely_ shouldn't be considered safe.Managing memory always is going to be unsafe. The idea is to encapsulate that, which RCArray shows how to do.
Feb 24 2015
On 2/24/2015 1:32 AM, Jonathan M Davis via Digitalmars-d wrote:The issue is that delete is considered safe by the compiler,I thought more people would be interested in how to do a memory safe reference counted container.
Feb 24 2015
On Tuesday, 24 February 2015 at 09:44:13 UTC, Walter Bright wrote:On 2/24/2015 1:32 AM, Jonathan M Davis via Digitalmars-d wrote:This is really interesting ! Thought as a beginner, I am wondering why the counter need to be a pointer ?The issue is that delete is considered safe by the compiler,I thought more people would be interested in how to do a memory safe reference counted container.
Feb 24 2015
On Tuesday, 24 February 2015 at 10:08:23 UTC, matovitch wrote:On Tuesday, 24 February 2015 at 09:44:13 UTC, Walter Bright wrote:That's why: ---- this(this) { if (count) ++*count; } ----On 2/24/2015 1:32 AM, Jonathan M Davis via Digitalmars-d wrote:This is really interesting ! Thought as a beginner, I am wondering why the counter need to be a pointer ?The issue is that delete is considered safe by the compiler,I thought more people would be interested in how to do a memory safe reference counted container.
Feb 24 2015
On Tuesday, 24 February 2015 at 10:11:02 UTC, Namespace wrote:On Tuesday, 24 February 2015 at 10:08:23 UTC, matovitch wrote:Hmm, I don't see why that's why... :(On Tuesday, 24 February 2015 at 09:44:13 UTC, Walter Bright wrote:That's why: ---- this(this) { if (count) ++*count; } ----On 2/24/2015 1:32 AM, Jonathan M Davis via Digitalmars-d wrote:This is really interesting ! Thought as a beginner, I am wondering why the counter need to be a pointer ?The issue is that delete is considered safe by the compiler,I thought more people would be interested in how to do a memory safe reference counted container.
Feb 24 2015
On Tuesday, 24 February 2015 at 10:13:36 UTC, matovitch wrote:On Tuesday, 24 February 2015 at 10:11:02 UTC, Namespace wrote:It's more effective if you figure it out on your own. But maybe this is a bit more clear? https://github.com/Dgame/m3/blob/master/SmartPointer.d#L106On Tuesday, 24 February 2015 at 10:08:23 UTC, matovitch wrote:Hmm, I don't see why that's why... :(On Tuesday, 24 February 2015 at 09:44:13 UTC, Walter Bright wrote:That's why: ---- this(this) { if (count) ++*count; } ----On 2/24/2015 1:32 AM, Jonathan M Davis via Digitalmars-d wrote:This is really interesting ! Thought as a beginner, I am wondering why the counter need to be a pointer ?The issue is that delete is considered safe by the compiler,I thought more people would be interested in how to do a memory safe reference counted container.
Feb 24 2015
On Tuesday, 24 February 2015 at 10:13:36 UTC, matovitch wrote:On Tuesday, 24 February 2015 at 10:11:02 UTC, Namespace wrote:The counter is shared amount all "copies" of the array.On Tuesday, 24 February 2015 at 10:08:23 UTC, matovitch wrote: That's why: ---- this(this) { if (count) ++*count; } ----Hmm, I don't see why that's why... :(
Feb 24 2015
On Tuesday, 24 February 2015 at 10:16:00 UTC, Ulrich Küttler wrote:On Tuesday, 24 February 2015 at 10:13:36 UTC, matovitch wrote:Ah yes, what an idiot ! Thanks !On Tuesday, 24 February 2015 at 10:11:02 UTC, Namespace wrote:The counter is shared amount all "copies" of the array.On Tuesday, 24 February 2015 at 10:08:23 UTC, matovitch wrote: That's why: ---- this(this) { if (count) ++*count; } ----Hmm, I don't see why that's why... :(
Feb 24 2015
On Tuesday, February 24, 2015 01:44:07 Walter Bright via Digitalmars-d wrote:On 2/24/2015 1:32 AM, Jonathan M Davis via Digitalmars-d wrote:Oh, I think that that there's definitely something there and that we should be interested. I don't think that either Andrei or I were trying to take away from that. It's just that code showed a problem in that delete was considered safe when it should be system, so Andrei pointed that out, and then I agreed with him and pointed out that delete was really supposed to have been deprecated by now anyway. But the concept is perfectly valid, and delete can be replaced with destroy and core.memory.GC.free for the same effect without actually needing delete, so it'll still work even if delete is finally deprecated - though honestly, I would think that if you're going to be doing manual memory management, it would just be better to use malloc and free and avoid the GC altogether (though if the element type contains any references or pointers, you probably need to tell the GC about the memory so that it can scan it). - Jonathan M DavisThe issue is that delete is considered safe by the compiler,I thought more people would be interested in how to do a memory safe reference counted container.
Feb 25 2015
Walter Bright:5. bounds checkingWhen you go past bounds of a built-in array you get an error located in the user code, while if you put a pre-condition in your Array struct to detect the same errors, you get a run-time error message located in that pre-condition instead. I'd like some way to solve this small problem of giving more correctly located error messages. In Contract programming lingo it's a problem of "blame management" (I'd also like a way to detect some compile-time out-of-bound errors for user-defined collections, but you said this is not worth the effort). Bye, bearophile
Feb 23 2015
On 2/23/2015 4:13 PM, bearophile wrote:This is off-topic. The point of the array class as presented is to show how a memory safe reference counted container can be built.5. bounds checkingWhen you go past bounds of a built-in array you get an error located in the user code, while if you put a pre-condition in your Array struct to detect the same errors, you get a run-time error message located in that pre-condition instead. I'd like some way to solve this small problem of giving more correctly located error messages. In Contract programming lingo it's a problem of "blame management" (I'd also like a way to detect some compile-time out-of-bound errors for user-defined collections, but you said this is not worth the effort).
Feb 23 2015
I thought that delete is deprecated, yet here Walter himself promotes a solution that uses it. Can we *PLEASE* finally deprecate things that are supposed to be deprecated and remove things that are supposed to be removed? On 2015-02-23 22:15:46 +0000, Walter Bright said:This is pretty straightforward. More could be done: 1. small array optimization 2. support for ranges as constructor args 3. present a range interface 4. support for malloc/free instead of GC 5. bounds checking 6. the array[] and the count could be allocated together 7. array[] could be just a pointer but the basic idea is there, I didn't want to hide it behind all the other flesh a professional type would have. Note the return in opIndex(). This is DIP25 at work! Compile: dmd rcarray -unittest -main -dip25 =========================================== struct RCArray(E) { this(E[] a) { array = a.dup; start = 0; end = a.length; count = new int; *count = 1; } ~this() { if (count && --*count == 0) delete array; } this(this) { if (count) ++*count; } size_t length() { return end - start; } ref E opIndex(size_t i) return // here's the magic { return array[start + i]; } RCArray opSlice(size_t lwr, size_t upr) { RCArray result = this; result.start = start + lwr; result.end = start + upr; return result; } private: E[] array; size_t start, end; int* count; } unittest { static int[3] s = [7, 6, 4]; auto r = RCArray!int(s); assert(r.length == 3); assert(r[0] == 7); assert(r[1] == 6); assert(r[2] == 4); assert(*r.count == 1); { auto r2 = r; assert(r2[0] == 7); assert(r2[1] == 6); assert(r2[2] == 4); assert(*r.count == 2); r[1] = 3; assert(r2[0] == 7); assert(r2[1] == 3); assert(r2[2] == 4); } assert(*r.count == 1); auto r3 = r[1 .. 3]; r[2] = 9; assert(r3[0] == 3); assert(r3[1] == 9); /+ ref int test(ref RCArray!int rr) { return rr[1]; // this gives error } +/ }
Feb 23 2015
On 2/23/2015 5:01 PM, Max Klyga wrote:I thought that delete is deprecated, yet here Walter himself promotes a solution that uses it. Can we *PLEASE* finally deprecate things that are supposed to be deprecated and remove things that are supposed to be removed?That isn't the point. I could have well have used malloc/free, but I wanted the code example to be about how to make a memory safe ref counted container, not about the details of memory allocation. It demonstrates how 'return ref' is to be used.
Feb 23 2015
On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:ref E opIndex(size_t i) return // here's the magicexactly what is this doing? I don't see this explained in DIP25 at all.
Feb 23 2015
On 2/23/2015 5:41 PM, weaselcat wrote:On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:The 'this' is passed by reference. So the 'return' is really 'return ref this', the rest is explained by DIP25.ref E opIndex(size_t i) return // here's the magicexactly what is this doing? I don't see this explained in DIP25 at all.
Feb 23 2015
On Tuesday, 24 February 2015 at 02:06:07 UTC, Walter Bright wrote:On 2/23/2015 5:41 PM, weaselcat wrote:Oh, I see. That makes a lot more sense.On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:The 'this' is passed by reference. So the 'return' is really 'return ref this', the rest is explained by DIP25.ref E opIndex(size_t i) return // here's the magicexactly what is this doing? I don't see this explained in DIP25 at all.
Feb 23 2015
On 2/23/15 6:05 PM, Walter Bright wrote:On 2/23/2015 5:41 PM, weaselcat wrote:We should amend DIP25 to explain that "this" is handled like a parameter. -- AndreiOn Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:The 'this' is passed by reference. So the 'return' is really 'return ref this', the rest is explained by DIP25.ref E opIndex(size_t i) return // here's the magicexactly what is this doing? I don't see this explained in DIP25 at all.
Feb 24 2015
On Tuesday, 24 February 2015 at 15:51:42 UTC, Andrei Alexandrescu wrote:On 2/23/15 6:05 PM, Walter Bright wrote:Note that I'm not surprised by that behavior (this is indeed ref for struct). But this is not ref for classes, which will defeat DIP25.On 2/23/2015 5:41 PM, weaselcat wrote:We should amend DIP25 to explain that "this" is handled like a parameter. -- AndreiOn Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:The 'this' is passed by reference. So the 'return' is really 'return ref this', the rest is explained by DIP25.ref E opIndex(size_t i) return // here's the magicexactly what is this doing? I don't see this explained in DIP25 at all.
Feb 24 2015
On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:This is pretty straightforward. More could be done: 1. small array optimization 2. support for ranges as constructor args 3. present a range interface 4. support for malloc/free instead of GC 5. bounds checking 6. the array[] and the count could be allocated together 7. array[] could be just a pointer but the basic idea is there, I didn't want to hide it behind all the other flesh a professional type would have. Note the return in opIndex(). This is DIP25 at work! Compile: dmd rcarray -unittest -main -dip25 =========================================== struct RCArray(E) { this(E[] a) { array = a.dup; start = 0; end = a.length; count = new int; *count = 1; } ~this() { if (count && --*count == 0) delete array; } this(this) { if (count) ++*count; } size_t length() { return end - start; } ref E opIndex(size_t i) return // here's the magic { return array[start + i]; } RCArray opSlice(size_t lwr, size_t upr) { RCArray result = this; result.start = start + lwr; result.end = start + upr; return result; } private: E[] array; size_t start, end; int* count; } unittest { static int[3] s = [7, 6, 4]; auto r = RCArray!int(s); assert(r.length == 3); assert(r[0] == 7); assert(r[1] == 6); assert(r[2] == 4); assert(*r.count == 1); { auto r2 = r; assert(r2[0] == 7); assert(r2[1] == 6); assert(r2[2] == 4); assert(*r.count == 2); r[1] = 3; assert(r2[0] == 7); assert(r2[1] == 3); assert(r2[2] == 4); } assert(*r.count == 1); auto r3 = r[1 .. 3]; r[2] = 9; assert(r3[0] == 3); assert(r3[1] == 9); /+ ref int test(ref RCArray!int rr) { return rr[1]; // this gives error } +/ }Thanks for the example. I think we need more practical stuff like this compiled somewhere on the homepage, especially when a new feature is introduced, to answer questions like: What is it? When should I use it? And how do I implement it? Saves a lot of time and the frustration of finding out later that there _is_ such a feature, only I never heard of it until now.
Feb 24 2015
Does DIP25 means we can turn resources into RC types and be done with non-deterministic destructor calls by the GC, without using RefCounted or Unique? On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:This is pretty straightforward. More could be done: 1. small array optimization 2. support for ranges as constructor args 3. present a range interface 4. support for malloc/free instead of GC 5. bounds checking 6. the array[] and the count could be allocated together 7. array[] could be just a pointer but the basic idea is there, I didn't want to hide it behind all the other flesh a professional type would have. Note the return in opIndex(). This is DIP25 at work! Compile: dmd rcarray -unittest -main -dip25 =========================================== struct RCArray(E) { this(E[] a) { array = a.dup; start = 0; end = a.length; count = new int; *count = 1; } ~this() { if (count && --*count == 0) delete array; } this(this) { if (count) ++*count; } size_t length() { return end - start; } ref E opIndex(size_t i) return // here's the magic { return array[start + i]; } RCArray opSlice(size_t lwr, size_t upr) { RCArray result = this; result.start = start + lwr; result.end = start + upr; return result; } private: E[] array; size_t start, end; int* count; } unittest { static int[3] s = [7, 6, 4]; auto r = RCArray!int(s); assert(r.length == 3); assert(r[0] == 7); assert(r[1] == 6); assert(r[2] == 4); assert(*r.count == 1); { auto r2 = r; assert(r2[0] == 7); assert(r2[1] == 6); assert(r2[2] == 4); assert(*r.count == 2); r[1] = 3; assert(r2[0] == 7); assert(r2[1] == 3); assert(r2[2] == 4); } assert(*r.count == 1); auto r3 = r[1 .. 3]; r[2] = 9; assert(r3[0] == 3); assert(r3[1] == 9); /+ ref int test(ref RCArray!int rr) { return rr[1]; // this gives error } +/ }
Feb 24 2015
On 2/24/15 3:14 AM, ponce wrote:Does DIP25 means we can turn resources into RC types and be done with non-deterministic destructor calls by the GC, without using RefCounted or Unique?That's what we're aiming for. -- Andrei
Feb 24 2015
On 2015-02-23 22:15:46 +0000, Walter Bright said:int* count; [...] if (count && --*count == 0) [...]Careful! This isn't memory safe and you have to thank the GC for it. If you ever use RCArray as a member variable in a class, the RCArray destructor is going to be called from a random thread when the class destructor is run. If some thread has a stack reference to the array you have a race. You have to use an atomic counter unless you can prove the RCArray struct will never be put in a GC-managed context. It is rather sad that the language has no way to enforce such a restriction, and also that safe cannot detect that this is a problem here. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Feb 24 2015
On 2/24/15 8:17 AM, Michel Fortin wrote:On 2015-02-23 22:15:46 +0000, Walter Bright said:Actually, RCArray can never be allocated on GC, or you may corrupt memory. count may be non-null, and point at invalid memory when the dtor is called. Only safe way to do this is to C malloc/free the count. And yes, at that point, you need atomics. -Steveint* count; [...] if (count && --*count == 0) [...]Careful! This isn't memory safe and you have to thank the GC for it. If you ever use RCArray as a member variable in a class, the RCArray destructor is going to be called from a random thread when the class destructor is run. If some thread has a stack reference to the array you have a race. You have to use an atomic counter unless you can prove the RCArray struct will never be put in a GC-managed context. It is rather sad that the language has no way to enforce such a restriction, and also that safe cannot detect that this is a problem here.
Feb 24 2015
On 2/24/2015 6:56 AM, Steven Schveighoffer wrote:Actually, RCArray can never be allocated on GC, or you may corrupt memory. count may be non-null, and point at invalid memory when the dtor is called.Just set count to null after the delete.Only safe way to do this is to C malloc/free the count. And yes, at that point, you need atomics.No, RCArray is not intended for shared access between threads. Shared containers and local containers are different enough that they merit being different types with different implementations altogether. Trying to just slap 'shared' on a container isn't going to work.
Feb 24 2015
On Tuesday, 24 February 2015 at 20:35:04 UTC, Walter Bright wrote:Maybe we want to fix the GC, exceptions and delegates or disable them in safe code because they all cause implicit sharing.Only safe way to do this is to C malloc/free the count. And yes, at that point, you need atomics.No, RCArray is not intended for shared access between threads. Shared containers and local containers are different enough that they merit being different types with different implementations altogether. Trying to just slap 'shared' on a container isn't going to work.
Feb 24 2015
On 2/24/2015 2:34 PM, deadalnix wrote:Maybe we want to fix the GC, exceptions and delegates or disable them in safe code because they all cause implicit sharing.Andrei already filed a bug report on the GC issue. If there aren't bug reports on the other two, file them.
Feb 24 2015
On Wednesday, 25 February 2015 at 00:13:03 UTC, Walter Bright wrote:On 2/24/2015 2:34 PM, deadalnix wrote:Well I filled a bug on the delegate one ~2 years ago, and the 2 others are features, not bugs, as far as I know.Maybe we want to fix the GC, exceptions and delegates or disable them in safe code because they all cause implicit sharing.Andrei already filed a bug report on the GC issue. If there aren't bug reports on the other two, file them.
Feb 24 2015
On 2/24/2015 4:32 PM, deadalnix wrote:On Wednesday, 25 February 2015 at 00:13:03 UTC, Walter Bright wrote:Without examples and bug reports, I don't really know what you're talking about. At least with a bug report, you have something easy to link to instead of trying to explain it over and over again.On 2/24/2015 2:34 PM, deadalnix wrote:Well I filled a bug on the delegate one ~2 years ago, and the 2 others are features, not bugs, as far as I know.Maybe we want to fix the GC, exceptions and delegates or disable them in safe code because they all cause implicit sharing.Andrei already filed a bug report on the GC issue. If there aren't bug reports on the other two, file them.
Feb 24 2015
On 2/24/15 3:34 PM, Walter Bright wrote:On 2/24/2015 6:56 AM, Steven Schveighoffer wrote:No, the GC may have already deallocated the memory count points at, and count is still non-null. This is why you can never refer to GC-allocated memory in dtors, if they can be called from the GC.Actually, RCArray can never be allocated on GC, or you may corrupt memory. count may be non-null, and point at invalid memory when the dtor is called.Just set count to null after the delete.GC is what shares the access. For instance, if you have an RCArray in one object that is being collected, it will attempt to decrement count. However, that doesn't mean that another RCArray reference in the originating thread won't also be trying to adjust count. In reality, it's only shared between the GC-collection running thread and the originating thread. I was not talking at all about making RCArray shared-usable. As talked about before, running dtors in the originating thread can solve this problem. -SteveOnly safe way to do this is to C malloc/free the count. And yes, at that point, you need atomics.No, RCArray is not intended for shared access between threads.
Feb 26 2015
On 2/26/15 8:51 AM, Steven Schveighoffer wrote:As talked about before, running dtors in the originating thread can solve this problem.Yah, that will solve the nonatomic reference counting. What do you think about http://forum.dlang.org/thread/mcllre$1abs$1 digitalmars.com? Andrei
Feb 26 2015
On 2/26/15 11:57 AM, Andrei Alexandrescu wrote:On 2/26/15 8:51 AM, Steven Schveighoffer wrote:I saw that, it sounds reasonable. I have to mull over what it means. I think possibly a better solution is to have a "finalize" function, similar to how tango does it, so the dtor is only called from destroy/delete, and the finalize method is called from the GC. -SteveAs talked about before, running dtors in the originating thread can solve this problem.Yah, that will solve the nonatomic reference counting. What do you think about http://forum.dlang.org/thread/mcllre$1abs$1 digitalmars.com?
Feb 26 2015
On Thursday, 26 February 2015 at 18:04:11 UTC, Steven Schveighoffer wrote:I think possibly a better solution is to have a "finalize" function, similar to how tango does it, so the dtor is only called from destroy/delete, and the finalize method is called from the GC.Ugh, so you still have to identify objects before releasing the memory? Put all objects that need cleanup on a separate GC heap then, to reduce impact.
Feb 26 2015
On Thursday, 26 February 2015 at 16:57:51 UTC, Andrei Alexandrescu wrote:On 2/26/15 8:51 AM, Steven Schveighoffer wrote:class BazingaException : Exception { RefCount!Stuff reallyImportantStuff; // ... } void main() { auto t = new Thread({ RefCount!Stuff s = ...; throw new BazingaException(s); }); t.start(); t.join(); }As talked about before, running dtors in the originating thread can solve this problem.Yah, that will solve the nonatomic reference counting. What do you think about http://forum.dlang.org/thread/mcllre$1abs$1 digitalmars.com? Andrei
Feb 26 2015
On 2/26/15 12:54 PM, deadalnix wrote:On Thursday, 26 February 2015 at 16:57:51 UTC, Andrei Alexandrescu wrote:Could you please walk me through what the matter is here. Thanks. -- AndreiOn 2/26/15 8:51 AM, Steven Schveighoffer wrote:class BazingaException : Exception { RefCount!Stuff reallyImportantStuff; // ... } void main() { auto t = new Thread({ RefCount!Stuff s = ...; throw new BazingaException(s); }); t.start(); t.join(); }As talked about before, running dtors in the originating thread can solve this problem.Yah, that will solve the nonatomic reference counting. What do you think about http://forum.dlang.org/thread/mcllre$1abs$1 digitalmars.com? Andrei
Feb 26 2015
On 2015-02-26 21:07:26 +0000, Andrei Alexandrescu said:On 2/26/15 12:54 PM, deadalnix wrote:The exception is thrown by t.join() in another thread, after the originating thread died. Thus, obviously, it cannot be destructed in the originating thread as stated above. But everyone already know that. But the example doesn't make one problem quite clear: Unless the GC destroys the exception in the thread join() was called, there can be a race. That's because join() moves the exception to another thread, and the thread that now owns the exception could make copies of that reallyImportantStuff and access the counter beyond the exception's lifetime. So it turns out that the GC heap needs call the exception's destructor in the thread calling join() to avoid races. Additionally, if the old thread has leftover objects still not yet collected, they'll need to be destroyed in the thread calling join() too. Otherwise you might get races when the exception is destroyed. So you could solve all that by changing of ownership for things originating from the worker thread to the thread that is calling join(). Or if no one calls join(), then you can destroy objects originating from the dead thread in any thread, as long as they are all destroyed *in the same thread* (because objects originating from the same thread might all points to the same thread-local reference counts). -- Michel Fortin michel.fortin michelf.com http://michelf.com/On Thursday, 26 February 2015 at 16:57:51 UTC, Andrei Alexandrescu wrote:Could you please walk me through what the matter is here. Thanks. -- AndreiOn 2/26/15 8:51 AM, Steven Schveighoffer wrote:class BazingaException : Exception { RefCount!Stuff reallyImportantStuff; // ... } void main() { auto t = new Thread({ RefCount!Stuff s = ...; throw new BazingaException(s); }); t.start(); t.join(); }As talked about before, running dtors in the originating thread can solve this problem.Yah, that will solve the nonatomic reference counting. What do you think about http://forum.dlang.org/thread/mcllre$1abs$1 digitalmars.com? Andrei
Feb 26 2015
On Thursday, 26 February 2015 at 21:07:26 UTC, Andrei Alexandrescu wrote:Could you please walk me through what the matter is here. Thanks. -- Andrei1/ The originating thread is likely to be dead by the time we collect. 2/ The RefCounted Stuff crossed thread boundaries so even if we kept the thread alive somehow to be able to destruct as mentioned, this is still not thread safe.
Feb 26 2015
On Thursday, 26 February 2015 at 16:51:30 UTC, Steven Schveighoffer wrote:As talked about before, running dtors in the originating thread can solve this problem.This probably implies forcibly destroying objects whose creating thread terminated.
Feb 26 2015
On 2/26/15 12:56 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" wrote:On Thursday, 26 February 2015 at 16:51:30 UTC, Steven Schveighoffer wrote:I don't think so, those objects can just be destroyed by the GC-collection running thread. If the thread is no longer present, there can't be a race. -SteveAs talked about before, running dtors in the originating thread can solve this problem.This probably implies forcibly destroying objects whose creating thread terminated.
Feb 26 2015
On Thursday, 26 February 2015 at 18:08:28 UTC, Steven Schveighoffer wrote:On 2/26/15 12:56 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" wrote:That's true. However, what if the destructors access global (thread-local) variables? Is that already disallowed?On Thursday, 26 February 2015 at 16:51:30 UTC, Steven Schveighoffer wrote:I don't think so, those objects can just be destroyed by the GC-collection running thread. If the thread is no longer present, there can't be a race.As talked about before, running dtors in the originating thread can solve this problem.This probably implies forcibly destroying objects whose creating thread terminated.
Feb 26 2015
On 2/26/15 4:40 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" wrote:On Thursday, 26 February 2015 at 18:08:28 UTC, Steven Schveighoffer wrote:Hm... I don't know. Is it "disallowed?" I don't think so (a simple test would suffice), but if any code exists today that does it, it's very very wrong :) I would suspect that such code should be banned. -SteveOn 2/26/15 12:56 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" wrote:That's true. However, what if the destructors access global (thread-local) variables? Is that already disallowed?On Thursday, 26 February 2015 at 16:51:30 UTC, Steven Schveighoffer wrote:I don't think so, those objects can just be destroyed by the GC-collection running thread. If the thread is no longer present, there can't be a race.As talked about before, running dtors in the originating thread can solve this problem.This probably implies forcibly destroying objects whose creating thread terminated.
Feb 26 2015
On Thu, 26 Feb 2015 18:04:28 -0500, Steven Schveighoffer wrote:but what if i'm doing memory managing myself and i *know* that it's ok to=20 use thread-locals in my destructor?=That's true. However, what if the destructors access global (thread-local) variables? Is that already disallowed?=20 Hm... I don't know. Is it "disallowed?" I don't think so (a simple test would suffice), but if any code exists today that does it, it's very very wrong :) I would suspect that such code should be banned.
Feb 26 2015
On 2/26/15 7:34 PM, ketmar wrote:On Thu, 26 Feb 2015 18:04:28 -0500, Steven Schveighoffer wrote:Accessing thread-local data in the GC-run destructor means you are accessing in the GC collecting thread. In the case where we make dtors run in the originating thread, it is OK. But if the thread is gone, then any running of that dtor may incorrectly use the TL data in that thread, which may not be correct. In any case, accessing thread local data today is most certainly wrong, since there isn't even a likelihood the same thread will destroy the data. I can't see a situation where this would be what you wanted. Accessing global data should be fine, but thread-local data is not. -Stevebut what if i'm doing memory managing myself and i *know* that it's ok to use thread-locals in my destructor?That's true. However, what if the destructors access global (thread-local) variables? Is that already disallowed?Hm... I don't know. Is it "disallowed?" I don't think so (a simple test would suffice), but if any code exists today that does it, it's very very wrong :) I would suspect that such code should be banned.
Feb 26 2015
On Thu, 26 Feb 2015 21:56:30 -0500, Steven Schveighoffer wrote:On 2/26/15 7:34 PM, ketmar wrote:let's assume that i have a program that has only one thread. i know it=20 for sure. and i don't like prepending ugly `__gshared` to my declarations. sure, it all depends of GC code (it can be multithreaded itself, or it=20 can deliberately run finalisers in another thread), but why i should not=20 be allowed to do what i want? i can write that GC myself, and i *know*=20 that there will be no other threads. this is the same as allocating in finaliser: as finalisers aren't made=20 implicitly nogc, i can see no sense in restricting 'em in any other way,=20 as the same arguments applies in both cases.=On Thu, 26 Feb 2015 18:04:28 -0500, Steven Schveighoffer wrote:Accessing thread-local data in the GC-run destructor means you are accessing in the GC collecting thread. In the case where we make dtors run in the originating thread, it is OK. But if the thread is gone, then any running of that dtor may incorrectly use the TL data in that thread, which may not be correct. =20 In any case, accessing thread local data today is most certainly wrong, since there isn't even a likelihood the same thread will destroy the data. =20 I can't see a situation where this would be what you wanted. Accessing global data should be fine, but thread-local data is not.but what if i'm doing memory managing myself and i *know* that it's ok to use thread-locals in my destructor?That's true. However, what if the destructors access global (thread-local) variables? Is that already disallowed?Hm... I don't know. Is it "disallowed?" I don't think so (a simple test would suffice), but if any code exists today that does it, it's very very wrong :) I would suspect that such code should be banned.
Feb 26 2015
On 2/26/15 10:56 PM, ketmar wrote:On Thu, 26 Feb 2015 21:56:30 -0500, Steven Schveighoffer wrote:In that case, you shouldn't be subject to any kind of race conditions. But we can't make the library/language based on this assumption. Your case is the exceptional case. I think it's going to be difficult to impossible to prevent people from using TLS in destructors anyway. It can probably be a guideline more than a compiler error. -SteveI can't see a situation where this would be what you wanted. Accessing global data should be fine, but thread-local data is not.let's assume that i have a program that has only one thread. i know it for sure. and i don't like prepending ugly `__gshared` to my declarations.
Feb 26 2015
On Thu, 26 Feb 2015 23:13:01 -0500, Steven Schveighoffer wrote:I think it's going to be difficult to impossible to prevent people from using TLS in destructors anyway. It can probably be a guideline more than a compiler error.and a warning, please! a warning that i can silence, of course.=
Feb 26 2015
On Friday, 27 February 2015 at 04:13:03 UTC, Steven Schveighoffer wrote:In that case, you shouldn't be subject to any kind of race conditions. But we can't make the library/language based on this assumption. Your case is the exceptional case.His case is not safe, so we can ignore that problem. However, this do not change the fact that the type qualifier do not provide the guarantee they are supposed to.
Feb 27 2015
On Fri, 27 Feb 2015 20:51:54 +0000, deadalnix wrote:On Friday, 27 February 2015 at 04:13:03 UTC, Steven Schveighoffer wrote:the compiler tends to disagree: =3D=3D=3D test.d =3D=3D=3D int myglobal; class A { ~this () safe { if (myglobal =3D=3D 42) assert(0); } } void main () { auto a =3D new A; } =3D=3D=3D=3D=3D=3D dmd -w -c -o- test.d wow! no warnings, no errors!=In that case, you shouldn't be subject to any kind of race conditions. But we can't make the library/language based on this assumption. Your case is the exceptional case.His case is not safe, so we can ignore that problem.
Feb 27 2015
On Saturday, 28 February 2015 at 04:18:38 UTC, ketmar wrote:On Fri, 27 Feb 2015 20:51:54 +0000, deadalnix wrote:Slighty off-topic, If DIP74 is approved it would be nice if destructors were redesigned(and possibly dropped altogether for non-deterministic lifetimes.)On Friday, 27 February 2015 at 04:13:03 UTC, Steven Schveighoffer wrote:the compiler tends to disagree: === test.d === int myglobal; class A { ~this () safe { if (myglobal == 42) assert(0); } } void main () { auto a = new A; } ====== dmd -w -c -o- test.d wow! no warnings, no errors!In that case, you shouldn't be subject to any kind of race conditions. But we can't make the library/language based on this assumption. Your case is the exceptional case.His case is not safe, so we can ignore that problem.
Feb 27 2015
On Saturday, 28 February 2015 at 04:27:36 UTC, weaselcat wrote:On Saturday, 28 February 2015 at 04:18:38 UTC, ketmar wrote:Yeah, we should have distinct finalizers (with or without DIP74).On Fri, 27 Feb 2015 20:51:54 +0000, deadalnix wrote:Slighty off-topic, If DIP74 is approved it would be nice if destructors were redesigned(and possibly dropped altogether for non-deterministic lifetimes.)On Friday, 27 February 2015 at 04:13:03 UTC, Steven Schveighoffer wrote:the compiler tends to disagree: === test.d === int myglobal; class A { ~this () safe { if (myglobal == 42) assert(0); } } void main () { auto a = new A; } ====== dmd -w -c -o- test.d wow! no warnings, no errors!In that case, you shouldn't be subject to any kind of race conditions. But we can't make the library/language based on this assumption. Your case is the exceptional case.His case is not safe, so we can ignore that problem.
Feb 28 2015
On 2/27/15 11:18 PM, ketmar wrote:On Fri, 27 Feb 2015 20:51:54 +0000, deadalnix wrote:I think in this case, the compiler can probably statically reject the code. But there are definitely ways around that, so I don't think full compliance is possible. I also think it's wrong-headed to have the compiler reject such code based on a GC deficiency. The only solution is to have GC-allocated data be destroyed in the same thread it's allocated in. This brings the multi-threaded program into the same realm as yours -- looks like one thread for unshared data :) -SteveOn Friday, 27 February 2015 at 04:13:03 UTC, Steven Schveighoffer wrote:the compiler tends to disagree: === test.d === int myglobal; class A { ~this () safe { if (myglobal == 42) assert(0); } } void main () { auto a = new A; } ====== dmd -w -c -o- test.d wow! no warnings, no errors!In that case, you shouldn't be subject to any kind of race conditions. But we can't make the library/language based on this assumption. Your case is the exceptional case.His case is not safe, so we can ignore that problem.
Mar 02 2015
On Mon, 02 Mar 2015 15:43:42 -0500, Steven Schveighoffer wrote:On 2/27/15 11:18 PM, ketmar wrote:it can't, 'cause it doesn't know how the given object is going to be=20 allocated. i may completely skip GC and use my own allocator, which has=20 no troubles with threads, for example.=On Fri, 27 Feb 2015 20:51:54 +0000, deadalnix wrote:I think in this case, the compiler can probably statically reject the code.On Friday, 27 February 2015 at 04:13:03 UTC, Steven Schveighoffer wrote:the compiler tends to disagree: =3D=3D=3D test.d =3D=3D=3D int myglobal; class A { ~this () safe { if (myglobal =3D=3D 42) assert(0); } } void main () { auto a =3D new A; } =3D=3D=3D=3D=3D=3D dmd -w -c -o- test.d wow! no warnings, no errors!In that case, you shouldn't be subject to any kind of race conditions. But we can't make the library/language based on this assumption. Your case is the exceptional case.His case is not safe, so we can ignore that problem.
Mar 02 2015
On 3/2/15 10:38 PM, ketmar wrote:On Mon, 02 Mar 2015 15:43:42 -0500, Steven Schveighoffer wrote:Classes are expected to be GC allocated. But the point is moot, because statically rejecting this does not guarantee abuse will not happen. And I don't think this restriction should be enshrined in the language anyway -- it's quite possible to write a GC that behaves correctly here. -SteveI think in this case, the compiler can probably statically reject the code.it can't, 'cause it doesn't know how the given object is going to be allocated. i may completely skip GC and use my own allocator, which has no troubles with threads, for example.
Mar 03 2015
On Tue, 03 Mar 2015 08:53:03 -0500, Steven Schveighoffer wrote:Classes are expected to be GC allocated.O_O classes are expected to be heap-allocated, methinks, and heap is not=20 necessary garbage collected.it's quite possible to write a GC that behaves correctly here.it's *almost* possible, i'd say. or, in another words, it's not possible=20 without turning threads to garbage-collected resource (at least it's not=20 possible without havy and dirty hackery and/or support from compiler=20 itself).=
Mar 03 2015
On Saturday, 28 February 2015 at 04:18:38 UTC, ketmar wrote:And what's the problem?His case is not safe, so we can ignore that problem.the compiler tends to disagree: === test.d === int myglobal; class A { ~this () safe { if (myglobal == 42) assert(0); } } void main () { auto a = new A; } ====== dmd -w -c -o- test.d wow! no warnings, no errors!
Mar 03 2015
On 3/3/15 9:22 AM, Kagamin wrote:On Saturday, 28 February 2015 at 04:18:38 UTC, ketmar wrote:This: void main() { myglobal = 42; spawnThread((){myglobal = 0; auto a = new A; a = null; while(1){}}); // trigger assert in A.dtor. int[] mem; while(1) { mem.length += 10000; } } -SteveAnd what's the problem?His case is not safe, so we can ignore that problem.the compiler tends to disagree: === test.d === int myglobal; class A { ~this () safe { if (myglobal == 42) assert(0); } } void main () { auto a = new A; } ====== dmd -w -c -o- test.d wow! no warnings, no errors!
Mar 03 2015
On Tuesday, 3 March 2015 at 14:42:40 UTC, Steven Schveighoffer wrote:// trigger assert in A.dtor.You mean assert breaks something?
Mar 03 2015
On 3/3/15 9:57 AM, Kagamin wrote:On Tuesday, 3 March 2015 at 14:42:40 UTC, Steven Schveighoffer wrote:OK, if you want to be pedantic: int *myglobal; repeat same code, but access what myglobal points to instead of setting to 42. Memory corruption is what safe code is trying to prevent. This would lead to memory corruption. -Steve// trigger assert in A.dtor.You mean assert breaks something?
Mar 03 2015
On Tuesday, 3 March 2015 at 18:38:11 UTC, Steven Schveighoffer wrote:int *myglobal; repeat same code, but access what myglobal points to instead of setting to 42. Memory corruption is what safe code is trying to prevent. This would lead to memory corruption.You mean, the class destructor accesses a GC-allocated memory? That's indeed unsafe, but unrelated to globals.
Mar 03 2015
On 3/4/15 2:26 AM, Kagamin wrote:On Tuesday, 3 March 2015 at 18:38:11 UTC, Steven Schveighoffer wrote:Wow, you really aren't getting it. I give up, it's tiring. -Steveint *myglobal; repeat same code, but access what myglobal points to instead of setting to 42. Memory corruption is what safe code is trying to prevent. This would lead to memory corruption.You mean, the class destructor accesses a GC-allocated memory? That's indeed unsafe, but unrelated to globals.
Mar 04 2015
On Wednesday, 4 March 2015 at 15:39:43 UTC, Steven Schveighoffer wrote:Wow, you really aren't getting it. I give up, it's tiring.Tiring? Did it take that much effort to not say what you mean?
Mar 04 2015
On 3/4/15 11:02 AM, Kagamin wrote:On Wednesday, 4 March 2015 at 15:39:43 UTC, Steven Schveighoffer wrote:No tiring like it is to have to explain obvious details to a child. In any case, either you are being intentionally obtuse or really ignorant. Either way, I don't have any desire to continue with this conversation. -SteveWow, you really aren't getting it. I give up, it's tiring.Tiring? Did it take that much effort to not say what you mean?
Mar 04 2015
On Wednesday, 4 March 2015 at 17:12:38 UTC, Steven Schveighoffer wrote:No tiring like it is to have to explain obvious details to a child.Safety system is only partially proactive in a sense it doesn't cover all possible scenarios, of one finds a bug, he provides an example, illustrating the bug (did int myglobal do it? if it was a child thing, why it was got wrong?). And scenarios tend to be very non-obvious, especially if one speaks about yet unseen bug. How an unknown bug can be obvious? And you believe, that expressing yourself in Nostradamus style greatly helps make things obvious? What things in safety system are on a child level at all? You should have a longer way to go down to a child argument.
Mar 05 2015
On Tue, 03 Mar 2015 14:22:15 +0000, Kagamin wrote:On Saturday, 28 February 2015 at 04:18:38 UTC, ketmar wrote:the answer is in your quotation.==20 And what's the problem?His case is not safe, so we can ignore that problem.the compiler tends to disagree: =3D=3D=3D test.d =3D=3D=3D int myglobal; class A { ~this () safe { if (myglobal =3D=3D 42) assert(0); } } void main () { auto a =3D new A; } =3D=3D=3D=3D=3D=3D dmd -w -c -o- test.d wow! no warnings, no errors!
Mar 03 2015
On Thursday, 26 February 2015 at 23:04:28 UTC, Steven Schveighoffer wrote:On 2/26/15 4:40 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" wrote:By "disallowed", I indeed mean "banned". Unfortunately, something that's just banned will sooner or later be used by someone, unless it's also statically prevented by the compiler.On Thursday, 26 February 2015 at 18:08:28 UTC, Steven Schveighoffer wrote:Hm... I don't know. Is it "disallowed?" I don't think so (a simple test would suffice), but if any code exists today that does it, it's very very wrong :) I would suspect that such code should be banned.On 2/26/15 12:56 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" wrote:That's true. However, what if the destructors access global (thread-local) variables? Is that already disallowed?On Thursday, 26 February 2015 at 16:51:30 UTC, Steven Schveighoffer wrote:I don't think so, those objects can just be destroyed by the GC-collection running thread. If the thread is no longer present, there can't be a race.As talked about before, running dtors in the originating thread can solve this problem.This probably implies forcibly destroying objects whose creating thread terminated.
Feb 27 2015
On 2/26/15 12:56 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" wrote:On Thursday, 26 February 2015 at 16:51:30 UTC, Steven Schveighoffer wrote:However, this has to be done carefully. For instance, you have to run dtors on next allocation, or when thread terminates, because if you do it inside stop-the-world signal handler, you still potentially have a race. -SteveAs talked about before, running dtors in the originating thread can solve this problem.This probably implies forcibly destroying objects whose creating thread terminated.
Feb 26 2015
On 2/24/15 5:17 AM, Michel Fortin wrote:On 2015-02-23 22:15:46 +0000, Walter Bright said:Good point. This is already a danger in existing code, e.g. File uses reference counting. I just submitted https://issues.dlang.org/show_bug.cgi?id=14221. Andreiint* count; [...] if (count && --*count == 0) [...]Careful! This isn't memory safe and you have to thank the GC for it. If you ever use RCArray as a member variable in a class, the RCArray destructor is going to be called from a random thread when the class destructor is run. If some thread has a stack reference to the array you have a race. You have to use an atomic counter unless you can prove the RCArray struct will never be put in a GC-managed context. It is rather sad that the language has no way to enforce such a restriction, and also that safe cannot detect that this is a problem here.
Feb 24 2015
On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:struct RCArray(E) { this(E[] a) { array = a.dup; start = 0; end = a.length; count = new int; *count = 1; }This may not be safe depending on the type of E. Also, this do not really solve the garbage problem~this() { if (count && --*count == 0) delete array; } this(this) { if (count) ++*count; }Here, we are going to run a bunch of null check because we can't enforce proper construction of struct. This is not per se a limitation of this piece of code, simply a long standing issue that should up. Also, there are no way to ensure that the array is going to be bound to one thread, even without shared (exception, delegate, destructor, pure return). You need at least atomic increment and/or decrement. But worse, as this can be assigned to static variables, you need to lock the whole damn struct unless you can update it atomically. The struct is 320 bits large and it is not updatable atomically in any arch I know of. You need a mutex.size_t length() { return end - start; } ref E opIndex(size_t i) return // here's the magic { return array[start + i]; }I have to admit, that is pretty cool. But it is only gonna work with value types.RCArray opSlice(size_t lwr, size_t upr) { RCArray result = this; result.start = start + lwr; result.end = start + upr; return result; }You need bound check: end <= start <= array.length .private: E[] array; size_t start, end; int* count; }
Feb 24 2015
On 2/24/2015 2:27 PM, deadalnix wrote:This may not be safe depending on the type of E.The code is a template, and so relies on inference.Also, this do not really solve the garbage problemSteven pointed out the correct fix for that.Also, there are no way to ensure that the array is going to be bound to one thread, even without shared (exception, delegate, destructor, pure return). You need at least atomic increment and/or decrement.No. The correct solution is to fix any cases of implicit sharing. If they don't have bugzilla issues, file them.I have to admit, that is pretty cool. But it is only gonna work with value types.I don't see that limitation at all.You need bound check: end <= start <= array.lengthSee the antecedent: "More could be done: [...] 5. bounds checking"
Feb 24 2015
On Wednesday, 25 February 2015 at 00:20:25 UTC, Walter Bright wrote:On 2/24/2015 2:27 PM, deadalnix wrote:Just try to do the same with opSlice() and you will see what he means.I have to admit, that is pretty cool. But it is only gonna work with value types.I don't see that limitation at all.
Feb 25 2015
On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:struct RCArray(E) { this(E[] a) { array = a.dup; start = 0; end = a.length; count = new int; *count = 1; }This may not be safe depending on the type of E. Also, this do not really solve the garbage problem as you need a GCed slice to build the RCed slice. That mean that without a nicer way to construct this, this is not solving any problem. Also, I guess that even if you use the GC in there, you want to make sure that the function is nogc .~this() { if (count && --*count == 0) delete array; } this(this) { if (count) ++*count; }Here, we are going to run a bunch of null check because we can't enforce proper construction of struct. This is not per se a limitation of this piece of code, simply a long standing issue that should up. Also, there are no way to ensure that the array is going to be bound to one thread, even without shared (exception, delegate, destructor, pure return). You need at least atomic increment and/or decrement. But worse, as this can be assigned to static variables, you need to lock the whole damn struct unless you can update it atomically. The struct is 320 bits large and it is not updatable atomically in any arch I know of. You need a mutex.size_t length() { return end - start; } ref E opIndex(size_t i) return // here's the magic { return array[start + i]; }I have to admit, that is pretty cool. But it is only gonna work with value types.RCArray opSlice(size_t lwr, size_t upr) { RCArray result = this; result.start = start + lwr; result.end = start + upr; return result; }You need bound check: end <= start <= array.length .private: E[] array; size_t start, end; int* count; }Generally, this need support for other array operations, is not usable in nogc, has threading issues, but this is a cool thing you can do with "ref return".
Feb 24 2015
On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:private: E[] array; size_t start, end; int* count;What is the point of keeping start/end? Aren't those baked into the array slice? Not storing start end means not having to do index arithmetic (minor), reducing struct size (always nice). But more importantly, it allows implicit (and conditional) bounds checking (awesome), which actually runs regardless anyways. Or did I miss something?
Mar 05 2015
On Thursday, 5 March 2015 at 15:20:47 UTC, monarch_dodra wrote:On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:`GC.free()` needs a pointer to the start of the allocated block; it will not release memory if it gets an interior pointer. But as far as I can see, one pointer that stores the original address should be enough.private: E[] array; size_t start, end; int* count;What is the point of keeping start/end? Aren't those baked into the array slice? Not storing start end means not having to do index arithmetic (minor), reducing struct size (always nice). But more importantly, it allows implicit (and conditional) bounds checking (awesome), which actually runs regardless anyways. Or did I miss something?
Mar 05 2015
On Thursday, 5 March 2015 at 16:19:09 UTC, Marc Schütz wrote:On Thursday, 5 March 2015 at 15:20:47 UTC, monarch_dodra wrote:Still, you shouldn't need "end", and bounds checking would "just work".On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:`GC.free()` needs a pointer to the start of the allocated block; it will not release memory if it gets an interior pointer. But as far as I can see, one pointer that stores the original address should be enough.private: E[] array; size_t start, end; int* count;What is the point of keeping start/end? Aren't those baked into the array slice? Not storing start end means not having to do index arithmetic (minor), reducing struct size (always nice). But more importantly, it allows implicit (and conditional) bounds checking (awesome), which actually runs regardless anyways. Or did I miss something?
Mar 05 2015
On 3/5/15 8:50 AM, monarch_dodra wrote:Still, you shouldn't need "end", and bounds checking would "just work".Correct. -- Andrei
Mar 05 2015