www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Are D classes proper reference types?

reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
D classes are distinct from structs because they are intended to 
be bound to a reference (pointer) and not addressed as a value 
(inline/copying).

But for efficiency reasons, scoped classes can stack-allocate, 
but my understanding is that the compiler can still allocate it 
on the GC heap? So it is still a reference type, I guess?

But how about "emplace", does "emplace" imply the the compiler 
cannot put it on the GC heap? Yes, I am aware that this is 
library construct, but still relevant.

The reason I am asking this is because implementing reference 
counting in a clean fashion requires classes to be proper 
reference types, so that means disabling scoped classes and 
emplaced classes, and only allow it where a reference/pointer to 
a class never exceed a reference count of 1.

The core of my question is this: is it at all reasonable for 
"user code" to assume that a class is not allocated on the heap? 
Is it reasonable to write code that assumes that it is allocated 
"inline" as with emplace? Because if it is, then then the 
advantages of having classes as reference types and making them 
different from structs are lost.
Jun 23 2021
parent reply kinke <noone nowhere.com> writes:
On Thursday, 24 June 2021 at 06:50:44 UTC, Ola Fosheim Grøstad 
wrote:

 [...]


Yes, class *refs* are always pointers. *scope* classes are 
deprecated (I don't think I've ever seen one); with `scope c = 
new Object`, you can have the compiler allocate a class 
*instance* on the stack for you, but `c` is still a *ref*. 
`emplace` doesn't allocate, you have to pass the memory 
explicitly. A class *instance* can also live in the static data 
segment (`static immutable myStaticObject = new Object;`); 
`extern(C++)` class instances can also live on the C++ heap/stack 
etc. etc.
Jun 24 2021
next sibling parent Adam D Ruppe <destructionator gmail.com> writes:
On Thursday, 24 June 2021 at 07:28:56 UTC, kinke wrote:

 *scope* classes are deprecated (I don't think I've ever seen 
 one);


I used it for my database thing where it is supposed to be 
destroyed reliably but also uses runtime polymorphism.

I now suggest people just stick `scope(exit) .destroy(obj);` any 
time you use it.
Jun 24 2021
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 24 June 2021 at 07:28:56 UTC, kinke wrote:

 On Thursday, 24 June 2021 at 06:50:44 UTC, Ola Fosheim Grøstad 
 wrote:

 [...]


 (I don't think I've ever seen one); with `scope c = new 
 Object`, you can have the compiler allocate a class *instance* 
 on the stack for you, but `c` is still a *ref*.


The dmd frontend uses them all the time to avoid allocation 
overhead for Visitors.
Jun 24 2021
parent kinke <noone nowhere.com> writes:
On Thursday, 24 June 2021 at 12:31:08 UTC, Stefan Koch wrote:

 On Thursday, 24 June 2021 at 07:28:56 UTC, kinke wrote:

 On Thursday, 24 June 2021 at 06:50:44 UTC, Ola Fosheim Grøstad 
 wrote:

 [...]


 (I don't think I've ever seen one); with `scope c = new 
 Object`, you can have the compiler allocate a class *instance* 
 on the stack for you, but `c` is still a *ref*.


 The dmd frontend uses them all the time to avoid allocation 
 overhead for Visitors.


I was talking about not having seen a `scope class C { ... }`, 
*not* the `scope` storage class as in the example.
Jun 24 2021
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 24 June 2021 at 07:28:56 UTC, kinke wrote:

 Yes, class *refs* are always pointers. *scope* classes are 
 deprecated (I don't think I've ever seen one); with `scope c = 
 new Object`, you can have the compiler allocate a class 
 *instance* on the stack for you, but `c` is still a *ref*.


But the user code cannot depend on it being stack allocated? So I 
could replace the Object reference with a reference counting 
pointer and put the counter at a negative offset?


 `emplace` doesn't allocate, you have to pass the memory 
 explicitly.


This is more of a problem. I was thinking about arrays that 
provide an emplace method, then one could replace emplace with 
heap allocation. I guess it isn't really possible to make 
`emplace` with custom memory work gracefully with reference 
counting with ref count at negative offset.


 A class *instance* can also live in the static data segment 
 (`static immutable myStaticObject = new Object;`);


But it isn't required to? It certainly wouldn't work with 
reference counting if it is stored in read only memory...


 `extern(C++)` class instances can also live on the C++ 
 heap/stack etc. etc.


Yes, that cannot be avoided.
Jun 24 2021
parent reply kinke <noone nowhere.com> writes:
On Friday, 25 June 2021 at 06:09:17 UTC, Ola Fosheim Grøstad 
wrote:

 On Thursday, 24 June 2021 at 07:28:56 UTC, kinke wrote:

 Yes, class *refs* are always pointers. *scope* classes are 
 deprecated (I don't think I've ever seen one); with `scope c = 
 new Object`, you can have the compiler allocate a class 
 *instance* on the stack for you, but `c` is still a *ref*.


 But the user code cannot depend on it being stack allocated? So 
 I could replace the Object reference with a reference counting 
 pointer and put the counter at a negative offset?


Well AFAIK it's mandated by the language, so an RC scheme 
replacing such allocations by heap ones seems like a definite 
step backwards - it's a useful pattern, and as Stefan pointed 
out, definitely in use. You could still stack-allocate but 
accommodate for the counter prefix in the compiler.


 `emplace` doesn't allocate, you have to pass the memory 
 explicitly.


 This is more of a problem. I was thinking about arrays that 
 provide an emplace method, then one could replace emplace with 
 heap allocation. I guess it isn't really possible to make 
 `emplace` with custom memory work gracefully with reference 
 counting with ref count at negative offset.


It's certainly possible as it's a library thing; some existing 
code may assume the returned reference to point to the beginning 
of the passed memory though (where there'd be your counter). What 
you'd definitely need to adjust is `__traits(classInstanceSize)`, 
accomodating for the extra counter prefix.
There's very likely existing code out there which doesn't use 
druntime's emplace[Initializer], but does it manually.


 A class *instance* can also live in the static data segment 
 (`static immutable myStaticObject = new Object;`);


 But it isn't required to? It certainly wouldn't work with 
 reference counting if it is stored in read only memory...


Not required to AFAIK, but if it's not statically allocated, 
you'd need to allocate it at runtime via some module or CRT ctor. 
It's probably easier to have the compiler put it into static but 
writable memory, so that you can mess with the counter.

---

All in all, I think a more interesting/feasible approach would be 
abusing the monitor field of extern(D) classes for the reference 
counter. It's the 2nd field (of pointer size) of each class 
instance, directly after the vptr (pointer to vtable). I think 
all monitor access goes through a druntime call, so you could 
hook into there, disallowing any regular monitor access, and put 
this (AFAIK, seldomly used) monitor field to some good use.
Jun 25 2021
next sibling parent reply kinke <noone nowhere.com> writes:
Wrt. manual non-heap allocations (stack/data segment/emplace 
etc.), you could e.g. reserve the most significant bit of the 
counter to denote such instances and prevent them from being 
free'd (and possibly finalization/destruction too; this would 
need some more thought I suppose).
Jun 25 2021
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 25 June 2021 at 07:17:20 UTC, kinke wrote:

 Wrt. manual non-heap allocations (stack/data segment/emplace 
 etc.), you could e.g. reserve the most significant bit of the 
 counter to denote such instances and prevent them from being 
 free'd (and possibly finalization/destruction too; this would 
 need some more thought I suppose).


Destruction is a bit tricky. If people rely on the destructor to 
run when the function returns then that cannot be moved to a 
reference counter. For instance if they have implemented some 
kind of locking mechanism or transaction mechanism with classes…

The most tricky one is emplace though as you have no way of 
releasing the memory without an extra function pointer.

Regarding using high bits in the counter; What you would want is 
to have a cheap increment/decrement and instead take the hit when 
the object is released. So you might actually instead want to 
keep track of the allcation-status in the lower 3 bits and 
instead do ±8, but I am not sure how that affects different CPUs. 
The basic idea, would be to make it so you don't trigger 
destruction on 0, but when the result is/becomes negative.
Jun 25 2021
prev sibling parent reply IGotD- <nise nise.com> writes:
On Friday, 25 June 2021 at 07:17:20 UTC, kinke wrote:

 Wrt. manual non-heap allocations (stack/data segment/emplace 
 etc.), you could e.g. reserve the most significant bit of the 
 counter to denote such instances and prevent them from being 
 free'd (and possibly finalization/destruction too; this would 
 need some more thought I suppose).


You cannot use the most significant bit as it will not work with 
some 32-bit systems. Linux with a 3G kernel position for example. 
Better to use the least significant bit as all allocated memory 
is guaranteed to be aligned. Regardless this requires compiler 
support for masking off this bit.

Now where going into halfway fat pointer support. Then we can 
just use fat pointers instead and have full freedom.
Jun 25 2021
next sibling parent IGotD- <nise nise.com> writes:
On Friday, 25 June 2021 at 17:37:13 UTC, IGotD- wrote:

 You cannot use the most significant bit as it will not work 
 with some 32-bit systems. Linux with a 3G kernel position for 
 example. Better to use the least significant bit as all 
 allocated memory is guaranteed to be aligned. Regardless this 
 requires compiler support for masking off this bit.

 Now where going into halfway fat pointer support. Then we can 
 just use fat pointers instead and have full freedom.


One thing I have found out over the years that if you want to 
have full versatility, have a pointer to your free function in 
your fat pointer. By having this you have generic method to free 
your object when it goes out of scope. You have the ability to 
use custom allocators and even change allocators on the fly. If 
you for some reason don't want to free your object automatically, 
just put zero in that field for example.
Jun 25 2021
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 25 June 2021 at 17:37:13 UTC, IGotD- wrote:

 You cannot use the most significant bit as it will not work 
 with some 32-bit systems. Linux with a 3G kernel position for 
 example. Better to use the least significant bit as all 
 allocated memory is guaranteed to be aligned. Regardless this 
 requires compiler support for masking off this bit.


Hm. Not sure if I follow, I think we are talking about stuffing 
bits into the counter and not the address?


 Now where going into halfway fat pointer support. Then we can 
 just use fat pointers instead and have full freedom.


But fat pointers are 16 bytes, so quite expensive.
Jun 25 2021
parent reply IGotD- <nise nise.com> writes:
On Friday, 25 June 2021 at 20:22:24 UTC, Ola Fosheim Grøstad 
wrote:

 Hm. Not sure if I follow, I think we are talking about stuffing 
 bits into the counter and not the address?


Then I misunderstood. If it's a counter it should be fine.


 But fat pointers are 16 bytes, so quite expensive.


Yes, that's a tradeoff but one I'm willing to take. I'm thinking 
even bigger managed pointers of perhaps 32 bytes which has more 
metadata like the allocated size. Managed languages in general 
have fat pointers which we see everywhere and it is not a big 
deal.

If you are littering pointers you perhaps should refactor your 
code, use an array if loads of objects of the same type. Another 
thing which I'm not that satisfied with D is that there is no 
built in method of expanding member classes into the host class 
like C++ which creates pointer littering and memory fragmentation.
Jun 25 2021
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 25 June 2021 at 21:05:36 UTC, IGotD- wrote:

 Yes, that's a tradeoff but one I'm willing to take. I'm 
 thinking even bigger managed pointers of perhaps 32 bytes which 
 has more metadata like the allocated size. Managed languages in 
 general have fat pointers which we see everywhere and it is not 
 a big deal.


Which languages use fat pointers? C++ may use it (but is not 
required to).


 If you are littering pointers you perhaps should refactor your 
 code, use an array if loads of objects of the same type.


This is what I want to avoid as it makes refcounting more 
difficult. If D classes are reference types then they should 
always be referred to through a pointer. If you want to put it in 
an array, use a struct.


 Another thing which I'm not that satisfied with D is that there 
 is no built in method of expanding member classes into the host 
 class like C++ which creates pointer littering and memory 
 fragmentation.


Not sure what you mean by expanding? I never liked `alias this` 
for structs, inheritance would be simpler. Is this what you mean 
by expanding?

I think classes in C++ are usually used more like structs in D. 
C++ programmers tend to avoid using virtuals, so D-style classes 
(C++ classes with virtual members) tend to be a smaller part of 
C++ codebases (but it depends on the project, obviously).
Jun 27 2021
parent reply IGotD- <nise nise.com> writes:
On Sunday, 27 June 2021 at 07:48:22 UTC, Ola Fosheim Grøstad 
wrote:

 Which languages use fat pointers? C++ may use it (but is not 
 required to).


Probably about all managed languages. One common method is a that 
it is actually an identifier it is used in a hash table. Then you 
can find all sorts of meta data in there. My original point was 
that this consume more memory in managed languages but nobody 
seems to mind.


 Not sure what you mean by expanding? I never liked `alias this` 
 for structs, inheritance would be simpler. Is this what you 
 mean by expanding?


When you use a struct as a member variable in another struct the 
data will be expanded into the host struct. If the member struct 
is 16 bytes then the host struct will have grow 16 bytes to 
accommodate that member struct.

This is not the case in D with classes as classes always are 
allocated on the heap using dynamic allocation. This leads to 
more fragmentation and memory consumption.
Jun 27 2021
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Sunday, 27 June 2021 at 09:35:10 UTC, IGotD- wrote:

 Probably about all managed languages.


I am sceptical of this assumption. There are no reasons for a GC 
language to require the usage of fat pointers?


 When you use a struct as a member variable in another struct 
 the data will be expanded into the host struct. If the member 
 struct is 16 bytes then the host struct will have grow 16 bytes 
 to accommodate that member struct.

 This is not the case in D with classes as classes always are 
 allocated on the heap using dynamic allocation. This leads to 
 more fragmentation and memory consumption.


Ok, I understand what you mean, but classes tend to be used for 
"big objects". I don't think there is anything that prevents a 
private class reference to be replaced by an inline 
representation as an optimization if no references ever leak. If 
you use whole program optimizations such things could also be 
done for public members.

What is holdning D back here is the lack of a high level IR after 
the frontend where such global passes could improve the 
implementation quality.

So, this is at the core of good language design; keep the model 
simple, but enable and allow optimizations. Too much special 
casing and you end up with a language that is difficult to extend 
and a neverending discovery of corner cases and compiler bugs.
Jun 27 2021
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 25 June 2021 at 07:01:31 UTC, kinke wrote:

 Well AFAIK it's mandated by the language, so an RC scheme 
 replacing such allocations by heap ones seems like a definite 
 step backwards - it's a useful pattern, and as Stefan pointed 
 out, definitely in use. You could still stack-allocate but 
 accommodate for the counter prefix in the compiler.


Yes, but then I need to mark it as non-freeable.


 It's certainly possible as it's a library thing; some existing 
 code may assume the returned reference to point to the 
 beginning of the passed memory though (where there'd be your 
 counter). What you'd definitely need to adjust is 
 `__traits(classInstanceSize)`, accomodating for the extra 
 counter prefix.


Yes, if people don't make assumptions about where the class ends 
and overwrites some other object, but I suspect pointer 
arithmetics isn't all that common for classes in D code.


 There's very likely existing code out there which doesn't use 
 druntime's emplace[Initializer], but does it manually.


I guess, but the compiler could have a release note warning 
against this.


 ctor. It's probably easier to have the compiler put it into 
 static but writable memory, so that you can mess with the 
 counter.


Another reason to add the ability to mark it as non-freeable.


 All in all, I think a more interesting/feasible approach would 
 be abusing the monitor field of extern(D) classes for the 
 reference counter. It's the 2nd field (of pointer size) of each 
 class instance, directly after the vptr (pointer to vtable). I 
 think all monitor access goes through a druntime call, so you 
 could hook into there, disallowing any regular monitor access, 
 and put this (AFAIK, seldomly used) monitor field to some good 
 use.


Yes, if you don't want to support weak pointers. I think you need 
two counters if you want to enable the usage of weak pointers.

One reason to put it at a negative offset is that it makes it 
possible to make it fully compatible with shared_ptr. And then 
you can also have additional fields such as a weak counter or a 
deallocation function pointer.

I don't think maintaining the D ABI is important, so one could 
add additional fields to the class. Maintaining core language 
semantics shouldn't require ABI support I think.
Jun 25 2021
parent reply kinke <noone nowhere.com> writes:
On Friday, 25 June 2021 at 17:05:41 UTC, Ola Fosheim Grøstad 
wrote:

 Yes, if you don't want to support weak pointers. I think you 
 need two counters if you want to enable the usage of weak 
 pointers.


I cannot imagine how weak pointers would work without an ugly 
extra indirection layer. If we're on the same page, we're talking 
about embedding the reference counter *directly* in the class 
instance, and the class ref still pointing directly to the 
instance.

Weak pointers aren't in the language, so I don't see why they 
would matter here. I thought you were after replacing 
GC-allocated class instances by a simple RC scheme.


 One reason to put it at a negative offset is that it makes it 
 possible to make it fully compatible with shared_ptr.


In modern C++ code I've been looking at so far, shared_ptr was 
used very rarely (and unique_ptr everywhere). AFAIK, the main 
reason being poor performance due to the extra indirection of 
shared_ptr. So replacing every D class ref by a 
shared_ptr-analogon for interop reasons would seem very backwards 
to me.
Jun 25 2021
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 25 June 2021 at 23:55:40 UTC, kinke wrote:

 I cannot imagine how weak pointers would work without an ugly 
 extra indirection layer. If we're on the same page, we're 
 talking about embedding the reference counter *directly* in the 
 class instance, and the class ref still pointing directly to 
 the instance.


So, my understanding is that C++ `make_shared` may allocate the 
reference chunk and the object in the same memory area, so that 
the reference count/weak counter is at a negative offset. That 
way you can free up the object while retaining the counter. (At 
some fragmentation cost, if the counter isn't freed.) This may 
give some cache advantages over separate allocation.

Maybe  weak could be an annotation, but I haven't given this much 
thought. If we think about it; You don't have to pass around weak 
pointers, so there is no reason for parameters to be marked weak? 
Only pointer-fields in structs or classes? In function bodies you 
would typically not want to use a weak pointer as you want to 
extend the lifetime of the object until the function returns.

Also you could mark a class as non-weak, to save the weak counter.


 Weak pointers aren't in the language, so I don't see why they 
 would matter here. I thought you were after replacing 
 GC-allocated class instances by a simple RC scheme.


One goal could be to make a class compatible with C++ or Swift on 
request. Both support weak pointers. You could have multiple 
ref-count layout schemes as long as they all are on negative 
offsets. Just don't mix class hierarchies. So you could mix a  D 
class hierarchy, C++ class-hiearchy and Swift class-hierarchy in 
the same codebase?


 In modern C++ code I've been looking at so far, shared_ptr was 
 used very rarely (and unique_ptr everywhere). AFAIK, the main 
 reason being poor performance due to the extra indirection of 
 shared_ptr. So replacing every D class ref by a 
 shared_ptr-analogon for interop reasons would seem very 
 backwards to me.


In C++ ownership should in general be kept local and one should 
use borrowing pointers (raw pointers) for actual computations, so 
only use owned pointers for transfer of ownership. But that puts 
more of a burden on the developer.
Jun 26 2021
parent reply kinke <noone nowhere.com> writes:
On Saturday, 26 June 2021 at 07:00:37 UTC, Ola Fosheim Grøstad 
wrote:

 Weak pointers aren't in the language, so I don't see why they 
 would matter here. I thought you were after replacing 
 GC-allocated class instances by a simple RC scheme.


 One goal could be to make a class compatible with C++ or Swift 
 on request. Both support weak pointers. You could have multiple 
 ref-count layout schemes as long as they all are on negative 
 offsets. Just don't mix class hierarchies. So you could mix a  
 D class hierarchy, C++ class-hiearchy and Swift class-hierarchy 
 in the same codebase?


I'm pretty sure I haven't seen a weak pointer in C++ yet. I don't 
look at C++ much anymore, but I suspect they are even a lot rarer 
than shared_ptr. Wrt. mixed class hierarchies, being able to 
inherit from and instantiate C++ classes in D is of some priority 
and mostly works today. (Let's not get into discussing multiple 
inheritance here, it's hardly a show-stopper for most use cases.)

Is Swift a thing outside the Apple universe (I admittedly despise 
;))? It's surely much better than their Objective-C crap, but 
still...

So for rare use cases like shared_ptr/weak pointer interop, a 
library solution (just like they are in C++) is IMO enough.
Jun 26 2021
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Saturday, 26 June 2021 at 07:39:44 UTC, kinke wrote:

 I'm pretty sure I haven't seen a weak pointer in C++ yet. I 
 don't look at C++ much anymore, but I suspect they are even a 
 lot rarer than shared_ptr.


Weak pointers are usually not needed, but sometimes you do need 
them and then not having them becomes a big weakness. Usually 
ones strives to have ownership defined in a way that makes it 
possible to dismantle a graph in a structured way from a single 
thread, and then weak pointers are not needed at all.


 Wrt. mixed class hierarchies, being able to inherit from and 
 instantiate C++ classes in D is of some priority and mostly 
 works today. (Let's not get into discussing multiple 
 inheritance here, it's hardly a show-stopper for most use 
 cases.)


Is it possible to inherit from a C++ class and get a D subclass, 
and is it possible to inherit from a D class and get a C++ class?


 Is Swift a thing outside the Apple universe (I admittedly 
 despise ;))? It's surely much better than their Objective-C 
 crap, but still...


The Apple universe is pretty big, we can dislike that, but that 
does not change the market…

So the goal would be to put the ref-count on a negative offset so 
that the layout can match up with C++ or Swift. I don't think 
that implies any big disadvantages, but I could be wrong. The 
only way to find out is to hash out the alternatives.


 So for rare use cases like shared_ptr/weak pointer interop, a 
 library solution (just like they are in C++) is IMO enough.


But the best solution is to get to a place where you can hand 
D-objects to other languages with ease without doing a runtime 
conversion from one layout to another.
Jun 26 2021
parent reply kinke <noone nowhere.com> writes:
On Saturday, 26 June 2021 at 13:49:25 UTC, Ola Fosheim Grøstad 
wrote:

 Is it possible to inherit from a C++ class and get a D 
 subclass, and is it possible to inherit from a D class and get 
 a C++ class?


Sure thing, with `extern(C++) class` of course.


 But the best solution is to get to a place where you can hand 
 D-objects to other languages with ease without doing a runtime 
 conversion from one layout to another.


With C++, you can today, an `extern(C++) class C` is equivalent 
to and mangled as C++ `C*`. You can't pass it directly to some 
`unique_ptr<C>` or `shared_ptr<T>` of course; an according D 
wrapper reflecting the C++ implementation (library-dependent) 
would be needed anyway for correct mangling. It'd be implemented 
as a templated D struct, similar to how `std::vector` works today 
for the MSVC runtime: 
https://github.com/dlang/druntime/blob/master/src/core/stdcpp/vector.d
Jun 26 2021
next sibling parent kinke <noone nowhere.com> writes:
On Saturday, 26 June 2021 at 20:03:01 UTC, kinke wrote:

 With C++, you can today, an `extern(C++) class C` is equivalent 
 to and mangled as C++ `C*`. You can't pass it directly to some 
 `unique_ptr<C>` or `shared_ptr<T>` of course; an according D 
 wrapper reflecting the C++ implementation (library-dependent) 
 would be needed anyway for correct mangling. It'd be 
 implemented as a templated D struct, similar to how 
 `std::vector` works today for the MSVC runtime: 
 https://github.com/dlang/druntime/blob/master/src/core/stdcpp/vector.d


Oh, I've just seen that unique_ptr is already implemented in 
https://github.com/dlang/druntime/blob/master/src/core/stdcpp/memory.d, and
according to the tests, working for the MSVC++ runtime, see
https://github.com/dlang/druntime/pull/2723/files.
Jun 26 2021
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Saturday, 26 June 2021 at 20:03:01 UTC, kinke wrote:

 On Saturday, 26 June 2021 at 13:49:25 UTC, Ola Fosheim Grøstad 
 wrote:

 Is it possible to inherit from a C++ class and get a D 
 subclass, and is it possible to inherit from a D class and get 
 a C++ class?


 Sure thing, with `extern(C++) class` of course.


That is all good, but it will lead to `extern(C++) class` 
replacing D classes. So why not unify right away? Why wait for 
the inevitable?


 With C++, you can today, an `extern(C++) class C` is equivalent 
 to and mangled as C++ `C*`. You can't pass it directly to some 
 `unique_ptr<C>` or `shared_ptr<T>` of course; an according D 
 wrapper reflecting the C++ implementation (library-dependent) 
 would be needed anyway for correct mangling. It'd be 
 implemented as a templated D struct


Yes, this is all good. But if you unify the layout of C++ and D 
classes and use the same layout as C++ shared_ptr for reference 
counted D classes then you can easily move back and forth between 
the languages. I think the presumption that development only 
happens in D and you only use other people's C++ code is ill 
advised. One may use a framework in C++ that one also extend in 
C++, but maybe want to use another language for the high level 
stuff.
Jun 27 2021
parent reply kinke <noone nowhere.com> writes:
On Sunday, 27 June 2021 at 07:54:38 UTC, Ola Fosheim Grøstad 
wrote:

 That is all good, but it will lead to `extern(C++) class` 
 replacing D classes. So why not unify right away? Why wait for 
 the inevitable?


The assumption that all D code and all classes therein are or 
need to be designed for C++ interop is daring. The DMD frontend 
uses extern(C++) classes almost exclusively, but that's just for 
LDC and GDC.

Getting rid of the monitor field was discussed multiple times. 
The other major and not so trivial difference is that extern(C++) 
classes have no TypeInfo pointer (in the first vtable slot for 
extern(D) classes), which also means that dynamic casts don't 
work, neither in D nor in C++ (for the latter, only for instances 
instiantiated on the D side). [AFAIK, most C++ implementations 
put the - of course totally incompatible - *C++* TypeInfo into 
vtable slot -1.]
Jun 27 2021
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Sunday, 27 June 2021 at 08:41:27 UTC, kinke wrote:

 Getting rid of the monitor field was discussed multiple times.


You don't have to get rid of it, just implicitly declare it for 
classes that use monitors? I don't think it has to be at a 
specific offset?


 The other major and not so trivial difference is that 
 extern(C++) classes have no TypeInfo pointer (in the first 
 vtable slot for extern(D) classes), which also means that 
 dynamic casts don't work, neither in D nor in C++ (for the 
 latter, only for instances instiantiated on the D side). 
 [AFAIK, most C++ implementations put the - of course totally 
 incompatible - *C++* TypeInfo into vtable slot -1.]


But D could just extend C++ typeinfo?
Jun 27 2021
parent reply kinke <noone nowhere.com> writes:
On Sunday, 27 June 2021 at 09:46:45 UTC, Ola Fosheim Grøstad 
wrote:

 On Sunday, 27 June 2021 at 08:41:27 UTC, kinke wrote:

 Getting rid of the monitor field was discussed multiple times.


 You don't have to get rid of it, just implicitly declare it for 
 classes that use monitors? I don't think it has to be at a 
 specific offset?


It's not about classes using monitors themselves, it's about 
people potentially using `synchronized (obj)` for some arbitrary 
class reference `obj`. Embedding it in the instance directly 
prevents having to use a hashtable or similar. But I'm certainly 
not fond of that field as it's a rare use case.


 The other major and not so trivial difference is that 
 extern(C++) classes have no TypeInfo pointer (in the first 
 vtable slot for extern(D) classes), which also means that 
 dynamic casts don't work, neither in D nor in C++ (for the 
 latter, only for instances instiantiated on the D side). 
 [AFAIK, most C++ implementations put the - of course totally 
 incompatible - *C++* TypeInfo into vtable slot -1.]


 But D could just extend C++ typeinfo?


Sure, 'just' :D - as it 'just' takes someone to implement it (for 
all supported C++ runtimes). It's always the same problem, lots 
of talk and blabla in the forum, but hardly any real action 
coming out of it.
Jun 27 2021
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Sunday, 27 June 2021 at 10:11:44 UTC, kinke wrote:

 It's not about classes using monitors themselves, it's about 
 people potentially using `synchronized (obj)` for some 
 arbitrary class reference `obj`.


I wasn't aware this was a thing. If people want this they can 
just embed a mutex in the class themselves. No point in having it 
everywhere.  You usually don't want to coordinate over one 
instance anyway.


 Sure, 'just' :D - as it 'just' takes someone to implement it 
 (for all supported C++ runtimes). It's always the same problem,


Right, but what does all supported C++ runtimes mean? I thought 
LDC was tied to clang, which I guess means two runtimes? If C++ 
doesn't use arbitrary negative offsets, then D could use those?
Jun 27 2021
parent kinke <noone nowhere.com> writes:
On Sunday, 27 June 2021 at 12:00:41 UTC, Ola Fosheim Grøstad 
wrote:

 On Sunday, 27 June 2021 at 10:11:44 UTC, kinke wrote:
 Right, but what does all supported C++ runtimes mean? I thought 
 LDC was tied to clang, which I guess means two runtimes? If C++ 
 doesn't use arbitrary negative offsets, then D could use those?


LDC isn't tied to clang at all, we just use the same backend. - 
It's libstdc++ and libc++ in the POSIX world (most likely 
including MinGW), and MSVC++ for 'native' Windows. Another 
difficulty is that TypeInfo_Class instances are generated by the 
compiler, and not in the frontend either, meaning that DMD, GDC 
and LDC would all need to be adapted.
Jun 27 2021
prev sibling parent zjh <fqbqrr 163.com> writes:
On Sunday, 27 June 2021 at 10:11:44 UTC, kinke wrote:

 On Sunday, 27 June 2021 at 09:46:45 UTC, Ola Fosheim Grøstad


We need organized action,Instead of doing it blindly.

I always say organize.
Jun 27 2021
prev sibling parent reply Mathias LANG <geod24 gmail.com> writes:
On Sunday, 27 June 2021 at 08:41:27 UTC, kinke wrote:

 On Sunday, 27 June 2021 at 07:54:38 UTC, Ola Fosheim Grøstad 
 wrote: [AFAIK, most C++ implementations put the - of course 
 totally incompatible - *C++* TypeInfo into vtable slot -1.]


Actually my understanding is that it's part of the ABI:

 Each virtual table address in the VTT is the address to be 
 assigned to the respective virtual pointer, i.e. the address 
 past the end of the typeinfo pointer (the address of the first 
 virtual function pointer, if there are any), not of the first 
 vcall offset.


[Source: Itanium ABI, 2.6.2 VTT 
Order](https://itanium-cxx-abi.github.io/cxx-abi/abi.html#vtable-ctor-vtt)

Regarding replacing D's ABI with C++, it *might* be possible 
today, but:
- It takes someone to care about it and want it;
- It's going to be a lot of work, so it should come with lots of 
benefit - I don't see much as of now;
- It locks us in a position where we depend on an external 
committee / implementation to define our ABI.

In any case, if you feel like it's worth it  Ola, you could start 
to look into what it takes (druntime/dmd wise) and start to 
submit PR. For dlang devs questions, I recommend to use the dlang 
slack.
Jun 27 2021
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Sunday, 27 June 2021 at 23:20:38 UTC, Mathias LANG wrote:

 - It locks us in a position where we depend on an external 
 committee / implementation to define our ABI.


Well, that could be an issue, but it is not likely to change fast 
or frequently so I don't think it is a high risk approach.


 In any case, if you feel like it's worth it  Ola, you could 
 start to look into what it takes (druntime/dmd wise) and start 
 to submit PR.


There is no point unless there is consensus. You first need to 
get consensus otherwise it is means throwing time into the 
garbage bin.


 For dlang devs questions, I recommend to use the dlang slack.


I don't have much time for chat, I prefer async communication.
Jun 27 2021
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Monday, 28 June 2021 at 06:13:06 UTC, Ola Fosheim Grøstad 
wrote:


 I don't have much time for chat, I prefer async communication.


Slack isn't like our IRC or Discord channels. It's more async 
like the forums.
Jun 27 2021
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 28 June 2021 at 06:25:39 UTC, Mike Parker wrote:

 Slack isn't like our IRC or Discord channels. It's more async 
 like the forums.


Thanks for the info, I might look into Slack if it doesn't 
require me to install anything. If it is an official channel you 
might consider adding it to the community menu or at least:

https://wiki.dlang.org/Get_involved
Jun 27 2021
prev sibling parent reply Mathias LANG <geod24 gmail.com> writes:
On Monday, 28 June 2021 at 06:13:06 UTC, Ola Fosheim Grøstad 
wrote:

 On Sunday, 27 June 2021 at 23:20:38 UTC, Mathias LANG wrote:

 - It locks us in a position where we depend on an external 
 committee / implementation to define our ABI.


 Well, that could be an issue, but it is not likely to change 
 fast or frequently so I don't think it is a high risk approach.


The issue is twofold: it requires us to follow upstream changes 
(the case you are thinking of), but also provides us from making 
non backward-compatible downstream changes (meaning we can't 
change it as we see fit if we realize there is potential for 
optimization).


 In any case, if you feel like it's worth it  Ola, you could 
 start to look into what it takes (druntime/dmd wise) and start 
 to submit PR.


 There is no point unless there is consensus. You first need to 
 get consensus otherwise it is means throwing time into the 
 garbage bin.


Waiting on "consensus" is an easy way to avoid doing any kind of 
work :)
I'm fairly sure most large achievements that have been undertaken 
by people in this community (that were not W&A) have been done 
without their (W&A's) blessing. People just went ahead and did 
it. But obviously those people cared more about getting things 
done than spending time discussing it on the forums.
Jun 28 2021
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 28 June 2021 at 07:44:25 UTC, Mathias LANG wrote:

 (the case you are thinking of), but also provides us from 
 making non backward-compatible downstream changes (meaning we 
 can't change it as we see fit if we realize there is potential 
 for optimization).


Has this ever happened?


 Waiting on "consensus" is an easy way to avoid doing any kind 
 of work :)


You don't need strict consensus, but you need at least one 
compiler team to agree that it is a worthwhile.


 I'm fairly sure most large achievements that have been 
 undertaken by people in this community (that were not W&A) have 
 been done without their (W&A's) blessing. People just went 
 ahead and did it. But obviously those people cared more about 
 getting things done than spending time discussing it on the 
 forums.


Was that a snide comment? Totally uncalled for, I certainly don't 
depend on anyones blessing to play with my own fork, but it does 
not affect anything outside it.

Making a PR for a repo without acceptance is utterly pointless 
and a waste of effort. Nobody should do it. They will just end up 
feeling miserable about what they could instead have spent their 
time on (including kids and family).

I am 100% confident that there has been a massive waste of effort 
in the D history that is due to a lack of coordination. Ranging 
from libraries that went nowhere to PRs that dried up and died.

Individual PRs won't fix the whole. The whole can only be fixed 
with a plan. To get to a place where you can plan you need to 
form a vision. To form a vision you need to work towards 
consensus.

You cannot fix poor organization with PRs. The PR-demanding crowd 
is off the rails irrational. Cut down on the excuses, start 
planning!

(What large achievements are you speaking of, by the way?)
Jun 28 2021