www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - What would be the consequence of implementing interfaces as fat

reply "deadalnix" <deadalnix gmail.com> writes:
All is in the title.

This is becoming increasingly the norm in new languages. It is 
much better in term of performances (it avoid cascaded loads to 
call methods, which can be really costly on modern CPUs), make 
object themselves smaller.

Would implementing them that way break D code ? What would be the 
extent of the breakage ?
Mar 29 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/29/2014 5:57 PM, deadalnix wrote:
 All is in the title.

 This is becoming increasingly the norm in new languages. It is much better in
 term of performances (it avoid cascaded loads to call methods, which can be
 really costly on modern CPUs), make object themselves smaller.

 Would implementing them that way break D code ? What would be the extent of the
 breakage ?
D already has phat pointers - that's what delegates are. I'm curious how the scheme you propose is different?
Mar 29 2014
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 30 March 2014 at 01:04:27 UTC, Walter Bright wrote:
 On 3/29/2014 5:57 PM, deadalnix wrote:
 All is in the title.

 This is becoming increasingly the norm in new languages. It is 
 much better in
 term of performances (it avoid cascaded loads to call methods, 
 which can be
 really costly on modern CPUs), make object themselves smaller.

 Would implementing them that way break D code ? What would be 
 the extent of the
 breakage ?
D already has phat pointers - that's what delegates are. I'm curious how the scheme you propose is different?
I'm talking about interface here. The way they are implemented in most new language is via a struct that contains: - pointer to the object - pointer to vtable That way to don't make object bigger when they implement an interface, and you don't need cascaded load to call methods.
Mar 29 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/29/2014 6:11 PM, deadalnix wrote:
 I'm talking about interface here. The way they are implemented in most new
 language is via a struct that contains:
   - pointer to the object
   - pointer to vtable

 That way to don't make object bigger when they implement an interface,
True, but why is this a problem?
 and you don't need cascaded load to call methods.
True, but on the other hand, it takes up 2 registers rather than one, costing twice as much to copy around, store, pass/return to functions, etc.
Mar 29 2014
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 30 March 2014 at 01:42:24 UTC, Walter Bright wrote:
 On 3/29/2014 6:11 PM, deadalnix wrote:
 I'm talking about interface here. The way they are implemented 
 in most new
 language is via a struct that contains:
  - pointer to the object
  - pointer to vtable

 That way to don't make object bigger when they implement an 
 interface,
True, but why is this a problem?
Higher memory consumption, less objects fitting in cache, more scanning to do for the GC.
 and you don't need cascaded load to call methods.
True, but on the other hand, it takes up 2 registers rather than one, costing twice as much to copy around, store, pass/return to functions, etc.
Two pointers structs are passed in register, which is fast. If that spill, that spill on stack, which is hot, and prefetcher friendly. On the other hand, the double indirection is very cache unfriendly.
Mar 29 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/29/2014 8:06 PM, deadalnix wrote:
 On Sunday, 30 March 2014 at 01:42:24 UTC, Walter Bright wrote:
 True, but why is this a problem?
Higher memory consumption, less objects fitting in cache, more scanning to do for the GC.
Debatable. All fields that are interface references would double in size.
 and you don't need cascaded load to call methods.
True, but on the other hand, it takes up 2 registers rather than one, costing twice as much to copy around, store, pass/return to functions, etc.
Two pointers structs are passed in register, which is fast. If that spill, that spill on stack, which is hot, and prefetcher friendly.
That underestimates how precious register real estate is on the x86.
 On the other hand, the double indirection is very cache unfriendly.
I suspect that the results of all this will be some use cases go faster, other use cases go slower, a decidedly mixed result.
Mar 29 2014
parent reply Manu <turkeyman gmail.com> writes:
On 30 March 2014 13:39, Walter Bright <newshound2 digitalmars.com> wrote:

 On 3/29/2014 8:06 PM, deadalnix wrote:

 On Sunday, 30 March 2014 at 01:42:24 UTC, Walter Bright wrote:

 True, but why is this a problem?
Higher memory consumption, less objects fitting in cache, more scanning to do for the GC.
Debatable. All fields that are interface references would double in size. and you don't need cascaded load to call methods.

 True, but on the other hand, it takes up 2 registers rather than one,
 costing
 twice as much to copy around, store, pass/return to functions, etc.
Two pointers structs are passed in register, which is fast. If that spill, that spill on stack, which is hot, and prefetcher friendly.
That underestimates how precious register real estate is on the x86.
This is only a concern when passing args. x86 has huge internal register files and uses aggressive register renaming, for the last decade or so. The only time when the number of named registers is significant these days is the fastcall calling convention (standard on x64), since functions need to expect it's arg in an explicitly named register. x64 doubled the number of argument registers to help with this (still fewer than other arch's). On the other hand, the double indirection is very cache unfriendly.

 I suspect that the results of all this will be some use cases go faster,
 other use cases go slower, a decidedly mixed result.
The most interesting result of the change for me would be that it wouldn't break alignment. But wrt performance, in my experience, i'm frequently worried about the indirection and potential cache miss. I can't imagine I'd often be so concerned about using an additional arg register. And that's more easily mitigated.
Mar 30 2014
parent reply "dajones" <dajones hotmail.com> writes:
"Manu" <turkeyman gmail.com> wrote in message 
news:mailman.122.1396231817.25518.digitalmars-d puremagic.com...
 On 30 March 2014 13:39, Walter Bright <newshound2 digitalmars.com> wrote:
 Two pointers structs are passed in register, which is fast. If that
 spill, that
 spill on stack, which is hot, and prefetcher friendly.
That underestimates how precious register real estate is on the x86.
This is only a concern when passing args. x86 has huge internal register files and uses aggressive register renaming,
If we could use them that would be great but we cant. We have to store/load to memory, and that means aprox 3 cycle latency each way. The cpu cant guess that we're only saving it for later, it has to do the memory write, and even with the store to load forwarding mechanism, spilling and reloading is expensive.
Apr 01 2014
parent reply Manu <turkeyman gmail.com> writes:
On 1 April 2014 18:33, dajones <dajones hotmail.com> wrote:

 "Manu" <turkeyman gmail.com> wrote in message
 news:mailman.122.1396231817.25518.digitalmars-d puremagic.com...
 On 30 March 2014 13:39, Walter Bright <newshound2 digitalmars.com>
wrote:
 Two pointers structs are passed in register, which is fast. If that
 spill, that
 spill on stack, which is hot, and prefetcher friendly.
That underestimates how precious register real estate is on the x86.
This is only a concern when passing args. x86 has huge internal register files and uses aggressive register renaming,
If we could use them that would be great but we cant. We have to store/load to memory, and that means aprox 3 cycle latency each way. The cpu cant guess that we're only saving it for later, it has to do the memory write, and even with the store to load forwarding mechanism, spilling and reloading is expensive.
Can you detail this more? Obviously it must perform the store to maintain memory coherency, but I was under the impression that the typical implementation would also keep the value around in a renamed register, and when it pops up again at a later time, it would use the register directly, rather than load from memory. The store shouldn't take any significant time since there's no dependency on the stored value, it should only take as long as issuing the store instruction; latency is irrelevant, since it's never read back, there's nothing waiting on it. Not sure what you mean by 'each way', since stored values shouldn't be read back if gets the value from a stashed register. I'm not an expert on the topic, but I read about it some years back, and haven't given it much thought since.
Apr 01 2014
next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 1 April 2014 at 09:38:09 UTC, Manu wrote:
 under the impression that the typical implementation would also 
 keep the
 value around in a renamed register, and when it pops up again 
 at a later
 time, it would use the register directly, rather than load from 
 memory.
Not sure how that would work, the memory-page/cache-line would have to be marked as read-only Section 10.8 in this document only talks about elimination of register-to-register moves: http://www.agner.org/optimize/microarchitecture.pdf But new x86s have a cache for decoded instructions and special looping optimizations for tight inner loops that bypasses decoding (microop-cache). Anyway, I think the best solution to multiple inheritance and interfaces is whole program optimization either in the compiler or the linker. The cost of long vtables is probably quite low on todays desktop.
Apr 01 2014
prev sibling parent reply "dajones" <dajones hotmail.com> writes:
"Manu" <turkeyman gmail.com> wrote in message 
news:mailman.9.1396345088.19942.digitalmars-d puremagic.com...
 On 1 April 2014 18:33, dajones <dajones hotmail.com> wrote:

 "Manu" <turkeyman gmail.com> wrote in message
 news:mailman.122.1396231817.25518.digitalmars-d puremagic.com...
 On 30 March 2014 13:39, Walter Bright <newshound2 digitalmars.com>
wrote:
 Two pointers structs are passed in register, which is fast. If that
 spill, that
 spill on stack, which is hot, and prefetcher friendly.
That underestimates how precious register real estate is on the x86.
This is only a concern when passing args. x86 has huge internal register files and uses aggressive register renaming,
If we could use them that would be great but we cant. We have to store/load to memory, and that means aprox 3 cycle latency each way. The cpu cant guess that we're only saving it for later, it has to do the memory write, and even with the store to load forwarding mechanism, spilling and reloading is expensive.
Can you detail this more?
x86 uses something called (IIRC) a "store forwarding buffer". Essentialy it keeps track of stores untill they have been completed. Any time you read from an address the store forwrding buffer is checked first, then caches and main memory. If it cant do that you have to wait for the store to finalize, and that can be a lot slower again. If there's no pending store it comes from the cache. either way memory stores/loads generaly have at best a 3 cycle latency.
 Obviously it must perform the store to maintain memory coherency, but I 
 was
 under the impression that the typical implementation would also keep the
 value around in a renamed register, and when it pops up again at a later
 time, it would use the register directly, rather than load from memory.
I've never read of any x86 doing what you describe. But I'm not too well up on the latest CPUs.
 The store shouldn't take any significant time since there's no dependency
 on the stored value, it should only take as long as issuing the store
 instruction; latency is irrelevant, since it's never read back, there's
 nothing waiting on it.
True.
 Not sure what you mean by 'each way', since stored values shouldn't be 
 read
 back if gets the value from a stashed register.

 I'm not an expert on the topic, but I read about it some years back, and
 haven't given it much thought since.
Check out the agnor fog microarchitechre and instruction timings pdfs. That's pretty much the holy scripture when it comes to this stuff. It may even be that reducing contention on the memroy unit helps, modern x86 tend to have multiple ALUs but only 1 memory unit. So instructions with memory operands cant be done in paralell as often.
Apr 01 2014
next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 1 April 2014 at 23:05:55 UTC, dajones wrote:
 x86 uses something called (IIRC) a "store forwarding buffer". 
 Essentialy it
 keeps track of stores untill they have been completed. Any time 
 you read
 from an address the store forwrding buffer is checked first, 
 then caches and
 main memory. If it cant do that you have to wait for the store 
 to finalize,
 and that can be a lot slower again. If there's no pending store 
 it comes
 from the cache.
It is commonly called a store buffer? Most CPU have it these days. Indeed, store are put in the store buffer until realized (which can take some time as you have to acquire the cache line from another core or memory). When you load, the CPU snoop in the store buffer in parallel as L1 cache for a value.
Apr 01 2014
prev sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 1 April 2014 at 23:05:55 UTC, dajones wrote:
 x86 uses something called (IIRC) a "store forwarding buffer". 
 Essentialy it
 keeps track of stores untill they have been completed. Any time 
 you read
 from an address the store forwrding buffer is checked first, 
 then caches and main memory.
Store forwarding is probably important for passing parameters on the stack (where you have frequent subsequent writes/reads to the same memory location), but optimizing for it seems like very CPU dependent PITA and you are usually better off using SIMD registers IMO. After all store forwarding is only relevant until the store hits the L1 cache of the core.
 either way memory stores/loads generaly have at best a 3 cycle 
 latency.
Because the CPU has to check the dirty flag of the L3 cacheline in case another core have a dirty L1 from a store to the same memory?
Apr 02 2014
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 2 April 2014 at 10:21:50 UTC, Ola Fosheim Grøstad
wrote:
 either way memory stores/loads generaly have at best a 3 cycle 
 latency.
Because the CPU has to check the dirty flag of the L3 cacheline in case another core have a dirty L1 from a store to the same memory?
You don't even come close to L3 in 3 cycles. Propagating signal takes time. You end up with 2 constraint in tension: the bigger your cache, the longer the round trip. That is why we have L1 cache of 32kb for ages now. Making it bigger would require to increase the response time, which lower the performances.
Apr 02 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 2 April 2014 at 18:21:26 UTC, deadalnix wrote:
 You don't even come close to L3 in 3 cycles. Propagating signal
 takes time. You end up with 2 constraint in tension: the bigger
 your cache, the longer the round trip.
I was thinking about it the wrong way, I guess it does not matter if a read is getting the wrong value if there are concurrent writes to the same location when there is no synchronization. It's feels weird, but speculative out-of-order execution etc is not-very-intuitive in the first place...
Apr 02 2014
parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 2 April 2014 at 19:29:29 UTC, Ola Fosheim Grøstad
wrote:
 On Wednesday, 2 April 2014 at 18:21:26 UTC, deadalnix wrote:
 You don't even come close to L3 in 3 cycles. Propagating signal
 takes time. You end up with 2 constraint in tension: the bigger
 your cache, the longer the round trip.
I was thinking about it the wrong way, I guess it does not matter if a read is getting the wrong value if there are concurrent writes to the same location when there is no synchronization. It's feels weird, but speculative out-of-order execution etc is not-very-intuitive in the first place...
Yes this mechanism can cause memory operation to be seen out of order by other cores.
Apr 02 2014
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On 30 March 2014 11:42, Walter Bright <newshound2 digitalmars.com> wrote:

 On 3/29/2014 6:11 PM, deadalnix wrote:

 I'm talking about interface here. The way they are implemented in most new
 language is via a struct that contains:
   - pointer to the object
   - pointer to vtable

 That way to don't make object bigger when they implement an interface,
True, but why is this a problem?
The most annoying thing about a hidden vtable pointer is it breaks alignment. If there is any SIMD (vector, matrix, etc) or other aligned value in the class, you have to start worrying and compensating for the hidden member. and you don't need cascaded load to call methods.

 True, but on the other hand, it takes up 2 registers rather than one,
 costing twice as much to copy around, store, pass/return to functions, etc.
2 registers are used anyway; it will just populate the second register when it fetches the vtable pointer. There is an additional register used when passing a class pointer to a function, but I'd like to see statistics on function args that include a class pointer. In my experience, functions that receive a class as an argument are the sort of functions that rarely receive many arguments. I notice that classes tend to EITHER receive a class pointer, or receive many arguments to do some work (and not a class pointers). I don't imagine it affecting the arg register availability significantly in practise. Slices certainly have a much greater impact on arg register availability, and they haven't shown to be a problem. This is an interesting idea. Something I never thought of, and I think I like it!
Mar 30 2014
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Manu:

 The most annoying thing about a hidden vtable pointer is it 
 breaks alignment.
D class instances have two hidden fields. Bye, bearophile
Mar 30 2014
parent reply Manu <turkeyman gmail.com> writes:
On 31 March 2014 11:39, bearophile <bearophileHUGS lycos.com> wrote:

 Manu:


  The most annoying thing about a hidden vtable pointer is it breaks
 alignment.
D class instances have two hidden fields.
Oh yeah... well that loses a lot of appeal from me in that case ;)
Mar 30 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/30/2014 7:10 PM, Manu wrote:
 On 31 March 2014 11:39, bearophile <bearophileHUGS lycos.com
 <mailto:bearophileHUGS lycos.com>> wrote:

     Manu:


         The most annoying thing about a hidden vtable pointer is it breaks
         alignment.


     D class instances have two hidden fields.


 Oh yeah... well that loses a lot of appeal from me in that case ;)
On 64 bits, the two fields means the rest starts at 16 byte alignment - ideal for SIMD!
Mar 31 2014
parent reply Manu <turkeyman gmail.com> writes:
On 31 March 2014 18:18, Walter Bright <newshound2 digitalmars.com> wrote:

 On 3/30/2014 7:10 PM, Manu wrote:

 On 31 March 2014 11:39, bearophile <bearophileHUGS lycos.com
 <mailto:bearophileHUGS lycos.com>> wrote:

     Manu:


         The most annoying thing about a hidden vtable pointer is it breaks
         alignment.


     D class instances have two hidden fields.


 Oh yeah... well that loses a lot of appeal from me in that case ;)
On 64 bits, the two fields means the rest starts at 16 byte alignment - ideal for SIMD!
Most computers aren't 64bit though. The fact that the pointers change in size between platforms is a huge nuisance. I often find I end up with some very annoying #ifdef's at the start of tightly packed classes to deal with this issue in C++.
Mar 31 2014
parent reply "w0rp" <devw0rp gmail.com> writes:
On Monday, 31 March 2014 at 09:32:20 UTC, Manu wrote:
 Most computers aren't 64bit though.
This isn't accurate. The most popular Steam OS is Windows 7, 64bit. http://store.steampowered.com/hwsurvey/ Steam usage data shows the the overwhelming majority of Windows 8 installs are 64-bit, so newer Windows installs are 64-bit almost as a rule. Old Microsoft posts show a surge in 64-bit installs. http://blogs.windows.com/windows/b/bloggingwindows/archive/2010/07/08/64-bit-momentum-surges-with-windows-7.aspx Modern Microsoft and Sony consoles use 64-bit processors. Smartphones and tablets are a notable exception, probably due to their lower memory requirements, but this won't last forever. So I don't think it's fair to say "most computers aren't 64-bit." A good fair chunk of computers are.
Mar 31 2014
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/31/2014 4:44 AM, w0rp wrote:
 So I don't think it's fair to say "most computers aren't 64-bit." A good fair
 chunk of computers are.
At least as far as desktops go, 32 bits is dead.
Mar 31 2014
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On 31 March 2014 21:44, w0rp <devw0rp gmail.com> wrote:

 On Monday, 31 March 2014 at 09:32:20 UTC, Manu wrote:

 Most computers aren't 64bit though.
This isn't accurate. The most popular Steam OS is Windows 7, 64bit. http://store.steampowered.com/hwsurvey/ Steam usage data shows the the overwhelming majority of Windows 8 installs are 64-bit, so newer Windows installs are 64-bit almost as a rule. Old Microsoft posts show a surge in 64-bit installs. http://blogs.windows.com/windows/b/bloggingwindows/ archive/2010/07/08/64-bit-momentum-surges-with-windows-7.aspx
Desktop computers are a relatively small fraction of computers in the world today, and losing market share rapidly. Modern Microsoft and Sony consoles use 64-bit processors.

Finally, we have plenty of ram! Huzzah! :)


Smartphones and tablets are a notable exception, probably due to their
 lower memory requirements, but this won't last forever.

 So I don't think it's fair to say "most computers aren't 64-bit." A good
 fair chunk of computers are.
It's completely fair; it's fact that 'most' (literally) computers today are 32 bit, and the current trend is away from 64 bit, although that should change as mobile adopts 64 bit too. I suspect it'll be quite a while yet before developers can forget about 32 bit devices. I think there's only one 64 bit mobile device on the market so far? The point is, it's impossible to bank on pointers being either 32 or 64 bit. This leads to #ifdef's at the top of classes in my experience. D is not exempt.
Mar 31 2014
parent reply Michel Fortin <michel.fortin michelf.ca> writes:
On 2014-04-01 05:39:04 +0000, Manu <turkeyman gmail.com> said:

 The point is, it's impossible to bank on pointers being either 32 or 64
 bit. This leads to #ifdef's at the top of classes in my experience. D is
 not exempt.
Doesn't align(n) work for class members? If it does not, it doesn't seem it'd be hard to implement. -- Michel Fortin michel.fortin michelf.ca http://michelf.ca
Apr 01 2014
parent Manu <turkeyman gmail.com> writes:
On 1 April 2014 21:50, Michel Fortin <michel.fortin michelf.ca> wrote:

 On 2014-04-01 05:39:04 +0000, Manu <turkeyman gmail.com> said:

  The point is, it's impossible to bank on pointers being either 32 or 64
 bit. This leads to #ifdef's at the top of classes in my experience. D is
 not exempt.
Doesn't align(n) work for class members? If it does not, it doesn't seem it'd be hard to implement.
The point is to eliminate the wasted padding by rearranging structure members appropriately. Since the amount of padding may be different between arch's, the layout often needs tweaking.
Apr 01 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/30/2014 6:33 PM, Manu wrote:
 This is an interesting idea. Something I never thought of, and I think I like
it!
Frankly, I don't know why you use classes at all. Just use structs.
Mar 30 2014
parent reply Manu <turkeyman gmail.com> writes:
On 31 March 2014 12:21, Walter Bright <newshound2 digitalmars.com> wrote:

 On 3/30/2014 6:33 PM, Manu wrote:

 This is an interesting idea. Something I never thought of, and I think I
 like it!
Frankly, I don't know why you use classes at all. Just use structs.
Reference types are very useful. Most programmers are familiar with this workflow, and it's a convenient way of modelling lots of problems. I do find myself using a lot more struct's in D though, but that doesn't void the traditional approach. And I also maintain that these things are important particularly as a bridge for new D users. I also feel quite dirty using pointers in D where there is a dedicated reference type available. I don't want * and & to appear everywhere in my D code.
Mar 30 2014
next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 31 March 2014 at 03:25:11 UTC, Manu wrote:
 I also feel quite dirty using pointers in D where there is a 
 dedicated reference type available. I don't want * and & to 
 appear everywhere in my D code.
structs can pretty easily be reference types too: struct RefType { struct Impl { // put all the stuff in here } Impl* impl; alias impl this; // add ctors and stuff that new the impl }
Mar 30 2014
next sibling parent reply Manu <turkeyman gmail.com> writes:
On 31 March 2014 13:32, Adam D. Ruppe <destructionator gmail.com> wrote:

 On Monday, 31 March 2014 at 03:25:11 UTC, Manu wrote:

 I also feel quite dirty using pointers in D where there is a dedicated
 reference type available. I don't want * and & to appear everywhere in my D
 code.
structs can pretty easily be reference types too: struct RefType { struct Impl { // put all the stuff in here } Impl* impl; alias impl this; // add ctors and stuff that new the impl }
And you think this is 'cool'? The amount of boilerplate required makes C++ look neat and tidy. You've also truncated it significantly. class RefType { // put all the stuff in here } Why would anyone want to do all that crap? The reason is to overcome the limitations/restrictions of class... so just fix class? Or maybe improve struct, so that boilerplate can disappear. Perhaps add a distinct ref type like MS did with '^' pointers in WinRT and managed C++? Either way, for my money, that code might appeal to a D nerd (because you 'can'!), but I find it acutely distasteful code otherwise. No junior programmer would/should understand all that intuitively, and I would be embarrassed to show that to a non-D-user that I was trying to convince. If this pattern is recurring (it seems that it is), then I think it's clear sign of a chronic deficiency in D. It should probably be studied and addressed. I'm seeing it appear a lot.
Mar 30 2014
parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 31 March 2014 at 04:48:20 UTC, Manu wrote:
 On 31 March 2014 13:32, Adam D. Ruppe 
 <destructionator gmail.com> wrote:

 On Monday, 31 March 2014 at 03:25:11 UTC, Manu wrote:

 I also feel quite dirty using pointers in D where there is a 
 dedicated
 reference type available. I don't want * and & to appear 
 everywhere in my D
 code.
structs can pretty easily be reference types too: struct RefType { struct Impl { // put all the stuff in here } Impl* impl; alias impl this; // add ctors and stuff that new the impl }
And you think this is 'cool'? The amount of boilerplate required makes C++ look neat and tidy. You've also truncated it significantly. class RefType { // put all the stuff in here } Why would anyone want to do all that crap? The reason is to overcome the limitations/restrictions of class... so just fix class? Or maybe improve struct, so that boilerplate can disappear. Perhaps add a distinct ref type like MS did with '^' pointers in WinRT and managed C++? Either way, for my money, that code might appeal to a D nerd (because you 'can'!), but I find it acutely distasteful code otherwise. No junior programmer would/should understand all that intuitively, and I would be embarrassed to show that to a non-D-user that I was trying to convince. If this pattern is recurring (it seems that it is), then I think it's clear sign of a chronic deficiency in D. It should probably be studied and addressed. I'm seeing it appear a lot.
I think you're missing the point. D is able to create structs that work as reference types, as a generic library type. No novice programmer or new adopter has to understand how they work in order to use them. I think there should be a more vanilla reference type (than NullableRef) in std.typecons in order to address this.
Mar 31 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/30/2014 8:32 PM, Adam D. Ruppe wrote:
 On Monday, 31 March 2014 at 03:25:11 UTC, Manu wrote:
 I also feel quite dirty using pointers in D where there is a dedicated
 reference type available. I don't want * and & to appear everywhere in my D
code.
structs can pretty easily be reference types too:
Or just: alias S* C; Voila! Use C as the type instead of S*. The reason this works out so well in D is because C.member works (no need to use -> )
Mar 31 2014
next sibling parent reply Manu <turkeyman gmail.com> writes:
On 31 March 2014 17:28, Walter Bright <newshound2 digitalmars.com> wrote:

 On 3/30/2014 8:32 PM, Adam D. Ruppe wrote:

 On Monday, 31 March 2014 at 03:25:11 UTC, Manu wrote:

 I also feel quite dirty using pointers in D where there is a dedicated
 reference type available. I don't want * and & to appear everywhere in
 my D code.
structs can pretty easily be reference types too:
Or just: alias S* C; Voila! Use C as the type instead of S*. The reason this works out so well in D is because C.member works (no need to use -> )
Now it's deceptive that it's a pointer, and the pointer semantics are not suppressed. It might be surprising to find that a type that doesn't look like a pointer behaves like a pointer. You lose access to the operators, indexing/slicing etc, etc. I don't see how this is a reasonable comparison to 'class' as a reference type by definition.
Mar 31 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/31/2014 12:51 AM, Manu wrote:
 Now it's deceptive that it's a pointer, and the pointer semantics are not
 suppressed. It might be surprising to find that a type that doesn't look like a
 pointer behaves like a pointer.
 You lose access to the operators, indexing/slicing etc, etc.
 I don't see how this is a reasonable comparison to 'class' as a reference type
 by definition.
Of course it's reasonable - not many classes overload operators. The point is, there are numerous solutions available, you aren't stuck with one solution for every problem. And, you can use 'alias this' as Adam showed to create a type with fully customized behavior - you don't have to change the language to prove your ideas.
Mar 31 2014
parent reply Manu <turkeyman gmail.com> writes:
On 31 March 2014 18:16, Walter Bright <newshound2 digitalmars.com> wrote:

 On 3/31/2014 12:51 AM, Manu wrote:

 Now it's deceptive that it's a pointer, and the pointer semantics are not
 suppressed. It might be surprising to find that a type that doesn't look
 like a
 pointer behaves like a pointer.
 You lose access to the operators, indexing/slicing etc, etc.
 I don't see how this is a reasonable comparison to 'class' as a reference
 type
 by definition.
Of course it's reasonable - not many classes overload operators.
Is there a way to disable indexed dereferencing? Slicing? The point is, there are numerous solutions available, you aren't stuck with
 one solution for every problem.
I just wouldn't go so far as to call these alternatives 'solution's. A pointer is a pointer. Calling it a reference type is a stretch. While it can be indexed, sliced, and operators don't work, I don't think this is a compelling solution in very many contexts, and certainly not a general solution. And, you can use 'alias this' as Adam showed to create a type with fully
 customized behavior - you don't have to change the language to prove your
 ideas.
I haven't made any suggestion to change the language, I just said that adam's idea (well, not necessarily his idea, it seems to be an established pattern) is a rather elaborate hack; in many cases the boilerplate exceeds the volume of the useful code. The whole pointer-to-Impl + getter property + alias this + etc pattern is quite a lot of boilerplate. It's a really common pattern, it's obviously very useful, but it's surprising to me that people think that it's like, 'cool'. I get why it's done, and it's cool that D can do this (I use it a lot in my code), but I don't feel it's particularly elegant.
Mar 31 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/31/2014 10:02 PM, Manu wrote:
 It's a really common pattern, it's obviously very useful, but it's surprising
to
 me that people think that it's like, 'cool'.
 I get why it's done, and it's cool that D can do this (I use it a lot in my
 code), but I don't feel it's particularly elegant.
'alias this' is inelegant (sorry Andrei) but it was designed for precisely this purpose - being able to use a struct to wrap any other type, and forward to and override behaviors of that type. Nobody has found a better way. Fortunately, the inelegance can be encapsulated within that type, and the user of the type need not be even aware of it. Remember my halffloat implementation? It relied on 'alias this' to work. Just try doing that in C++ <g>.
Mar 31 2014
next sibling parent reply Manu <turkeyman gmail.com> writes:
On 1 April 2014 15:53, Walter Bright <newshound2 digitalmars.com> wrote:

 On 3/31/2014 10:02 PM, Manu wrote:

 It's a really common pattern, it's obviously very useful, but it's
 surprising to
 me that people think that it's like, 'cool'.
 I get why it's done, and it's cool that D can do this (I use it a lot in
 my
 code), but I don't feel it's particularly elegant.
'alias this' is inelegant (sorry Andrei) but it was designed for precisely this purpose - being able to use a struct to wrap any other type, and forward to and override behaviors of that type. Nobody has found a better way.
I don't think this is the only instance where alias this is useful though. It's one of those features that's unintuitive at first, but I find it pops up surprisingly often. This is a super common pattern however, and my point was, that language already has a well defined reference type that is convenient and familiar, but it comes with some baggage that's not always desired. It's kind of irrelevant though; the leading point was that one advantage might be that carrying the vtable beside the instance pointer would eliminate the hidden fiends in the class instance, but there's the classinfo pointer too. Incidentally, why did you go with a dedicated classinfo pointer rather than use the 1st slot of the vtable like c++? Fortunately, the inelegance can be encapsulated within that type, and the
 user of the type need not be even aware of it.
Sure, but my point was that it seems to be _really_ frequently occurring. It's relatively unprecedented in D to be happy with such a commitment to boilerplate like that. I can imagine there's a strong temptation to just reach for a class even though it's not appropriate in all situations. It's a lot less cognitive load; inexperienced programmers won't have trouble with the boilerplate implementation, or what to do when they hit edge cases. Remember my halffloat implementation? It relied on 'alias this' to work.
 Just try doing that in C++ <g>.
Of course, I use alias this all the time too for various stuff. I said before, it's a useful tool and it's great D *can* do this stuff, but I'm talking about this particular super common use case where it's used to hack together nothing more than a class without a vtable, ie, a basic ref type. I'd say that's worth serious consideration as a 1st-class concept?
Apr 01 2014
next sibling parent reply Michel Fortin <michel.fortin michelf.ca> writes:
On 2014-04-01 07:11:51 +0000, Manu <turkeyman gmail.com> said:

 Of course, I use alias this all the time too for various stuff. I said
 before, it's a useful tool and it's great D *can* do this stuff, but I'm
 talking about this particular super common use case where it's used to hack
 together nothing more than a class without a vtable, ie, a basic ref type.
 I'd say that's worth serious consideration as a 1st-class concept?
You don't need it as a 1st-class D concept though. Just implement the basics of the C++ object model in D, similar to what I did for Objective-C, and let people define their own extern(C++) classes with no base class. Bonus if it's binary compatible with the equivalent C++ class. Hasn't someone done that already? -- Michel Fortin michel.fortin michelf.ca http://michelf.ca
Apr 01 2014
next sibling parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Michel Fortin"  wrote in message news:lhe9v8$28lp$1 digitalmars.com...

 You don't need it as a 1st-class D concept though. Just implement the 
 basics of the C++ object model in D, similar to what I did for 
 Objective-C, and let people define their own extern(C++) classes with no 
 base class. Bonus if it's binary compatible with the equivalent C++ class. 
 Hasn't someone done that already?
Mostly, but it will put out a vtbl even for classes with no virtual member functions.
Apr 01 2014
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On 1 April 2014 22:03, Michel Fortin <michel.fortin michelf.ca> wrote:

 On 2014-04-01 07:11:51 +0000, Manu <turkeyman gmail.com> said:

  Of course, I use alias this all the time too for various stuff. I said
 before, it's a useful tool and it's great D *can* do this stuff, but I'm

 talking about this particular super common use case where it's used to
 hack
 together nothing more than a class without a vtable, ie, a basic ref type.
 I'd say that's worth serious consideration as a 1st-class concept?
You don't need it as a 1st-class D concept though. Just implement the basics of the C++ object model in D, similar to what I did for Objective-C, and let people define their own extern(C++) classes with no base class. Bonus if it's binary compatible with the equivalent C++ class. Hasn't someone done that already?
I don't think the right conceptual solution to a general ref-type intended for use throughout D code is to mark it extern C++... That makes no sense.
Apr 01 2014
parent Michel Fortin <michel.fortin michelf.ca> writes:
On 2014-04-01 14:17:33 +0000, Manu <turkeyman gmail.com> said:

 On 1 April 2014 22:03, Michel Fortin <michel.fortin michelf.ca> wrote:
 
 On 2014-04-01 07:11:51 +0000, Manu <turkeyman gmail.com> said:
 
 Of course, I use alias this all the time too for various stuff. I said
 before, it's a useful tool and it's great D *can* do this stuff, but I'm
 
 talking about this particular super common use case where it's used to
 hack
 together nothing more than a class without a vtable, ie, a basic ref type.
 I'd say that's worth serious consideration as a 1st-class concept?
 
You don't need it as a 1st-class D concept though. Just implement the basics of the C++ object model in D, similar to what I did for Objective-C, and let people define their own extern(C++) classes with no base class. Bonus if it's binary compatible with the equivalent C++ class. Hasn't someone done that already?
I don't think the right conceptual solution to a general ref-type intended for use throughout D code is to mark it extern C++... That makes no sense.
I was thinking of having classes that'd be semantically equivalent to those in D but would follow the C++ ABI, hence the extern(C++). It doesn't have to support all of C++, just the parts that intersect with what you can express in D. For instance, those classes would be reference types, just like D classes; if you need value-type behaviour, use a struct. But maybe that doesn't make sense. -- Michel Fortin michel.fortin michelf.ca http://michelf.ca
Apr 01 2014
prev sibling parent "Dicebot" <public dicebot.lv> writes:
I agree that current problems with defining performant reference 
types are very nasty. I disagree with conclusion that classes 
need to be made more lightweight though. What is really desired 
is good solution for defining own full-featured reference types 
instead. D classes and "performance" simply don't connect in my 
mind in a single use case.
Apr 01 2014
prev sibling next sibling parent reply "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Tuesday, 1 April 2014 at 05:53:15 UTC, Walter Bright wrote:
 On 3/31/2014 10:02 PM, Manu wrote:
 It's a really common pattern, it's obviously very useful, but 
 it's surprising to
 me that people think that it's like, 'cool'.
 I get why it's done, and it's cool that D can do this (I use 
 it a lot in my
 code), but I don't feel it's particularly elegant.
'alias this' is inelegant (sorry Andrei) but it was designed for precisely this purpose - being able to use a struct to wrap any other type, and forward to and override behaviors of that type. Nobody has found a better way.
There's any plan for implementing the multiple alias this? --- Paolo
Apr 01 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/1/2014 12:22 AM, Paolo Invernizzi wrote:
 There's any plan for implementing the multiple alias this?
It's a good idea, but it's a bit far down the list of what we simply must get done soon.
Apr 01 2014
prev sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 01/04/14 07:53, Walter Bright wrote:
 'alias this' is inelegant (sorry Andrei) but it was designed for precisely this
 purpose - being able to use a struct to wrap any other type, and forward to and
 override behaviors of that type. Nobody has found a better way.
There are currently some unfortunate problems with it. Unless I've missed a recent fix, the examples in TDPL pp.230-233 don't work as they should, because of problems with protection attributes. :-(
 Fortunately, the inelegance can be encapsulated within that type, and the user
 of the type need not be even aware of it.

 Remember my halffloat implementation? It relied on 'alias this' to work. Just
 try doing that in C++ <g>.
It's fantastic that we can do this, but I have to say, having spent over a year exploring different reference-type solutions for a successor to std.random, it was striking how nice and simple it felt to just use classes.
Apr 01 2014
prev sibling parent reply Artur Skawina <art.08.09 gmail.com> writes:
On 03/31/14 09:28, Walter Bright wrote:
 On 3/30/2014 8:32 PM, Adam D. Ruppe wrote:
 structs can pretty easily be reference types too:
Or just: alias S* C; Voila! Use C as the type instead of S*.
C c = ...; auto a = c[42]; Boom! This is a catastrophic failure mode. Op overloads may be added to the pseudo-C after it's already written and working, and then a) the bug might go unnoticed, and b) fixing it requires API changes. See also my other reply. artur
Mar 31 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/31/2014 3:09 AM, Artur Skawina wrote:
 This is a catastrophic failure mode. Op overloads may be added to the pseudo-C
 after it's already written and working, and then a) the bug might go unnoticed,
 and b) fixing it requires API changes. See also my other reply.
Manu brought up the same issue, see my reply to him.
Mar 31 2014
prev sibling parent reply Artur Skawina <art.08.09 gmail.com> writes:
On 03/31/14 05:32, Adam D. Ruppe wrote:
 On Monday, 31 March 2014 at 03:25:11 UTC, Manu wrote:
 I also feel quite dirty using pointers in D where there is a dedicated
reference type available. I don't want * and & to appear everywhere in my D
code.
structs can pretty easily be reference types too: struct RefType { struct Impl { // put all the stuff in here } Impl* impl; alias impl this; // add ctors and stuff that new the impl }
No. This will bite you once you decide to overload Impl's ops, such as indexing and slicing. And it's quite nasty, as the code will then keep compiling but return bogus results. Debugging this will be an "interesting" process, unless you're already aware of what's happening or get lucky and the program crashes... A little safer hack would be: struct RefType { struct Impl { // put all the stuff in here disable this(this); } Impl* impl; ref Impl _get() property { return *impl; } alias _get this; // add ctors and stuff that new the impl } but this is still a rather ugly workaround. artur
Mar 31 2014
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 31 March 2014 at 10:09:06 UTC, Artur Skawina wrote:
 No. This will bite you once you decide to overload Impl's ops, 
 such as indexing and slicing.
I agree with you that this is a problem that can bite you, but there's a fairly easy solution: you shouldn't overload Impl's ops, nor should it have constructors, postblits, or destructors since they won't run when you expect them to either. Putting them on the outer struct works right and opens the door to new things like refcounting too. But yeah, you're right that it would still compile if you did it wrong and that's potentially ugly.
    struct RefType {
       struct Impl {
            // put all the stuff in here

             disable this(this);
       }
       Impl* impl;
       ref Impl _get()  property { return *impl; }
       alias _get this;

       // add ctors and stuff that new the impl
    }
not bad to my eyes,
Mar 31 2014
parent Artur Skawina <art.08.09 gmail.com> writes:
On 03/31/14 15:16, Adam D. Ruppe wrote:
 On Monday, 31 March 2014 at 10:09:06 UTC, Artur Skawina wrote:
 No. This will bite you once you decide to overload Impl's ops, such as
indexing and slicing.
I agree with you that this is a problem that can bite you, but there's a fairly easy solution: you shouldn't overload Impl's ops, [...]
In real code that pseudo-ref implementation will not necessarily be anywhere near the payload, and will often be factored out (it makes no sense to duplicate this functionality in every type that needs it -- so it will be done as 'Ref!T'). Both value- and reference-semantics are very clear and intuitive, but an amalgamate of reference semantics and pointer arithmetic is a mine waiting to explode; you only need to forget that your not dealing with a "true reference" for a second. Which is more likely to happen than not -- when you're dealing with a reference type 'C' everywhere, it is natural to assume that 'C[10]' works -- that 'C[10]' expression does not /look/ wrong, so the bug is very hard to spot.
 [...] nor should it have constructors, postblits, or destructors since they
won't run when you expect them to either. Putting them on the outer struct
works right and opens the door to new things like refcounting too.
Been there, done that. Don't try at home. (Immediately ran into several language and compiler issues, after working around quite a few ended up with code that caused data corruption. A compiler that crashes on a block of code, but accepts that very same block copy&pasted twice, is not fun to deal with... Maybe things improved in the last two years or so since I did that, but I doubt it.)
    struct RefType {
       struct Impl {
            // put all the stuff in here

             disable this(this);
       }
       Impl* impl;
       ref Impl _get()  property { return *impl; }
       alias _get this;

       // add ctors and stuff that new the impl
    }
not bad to my eyes,
I think this is as good as it gets, in a language without proper refs. Still you have ABI issues (passing a struct around can be hadled differently from the bare-pointer case on some platforms) and the 'Ref!T' syntax is less than ideal. artur
Mar 31 2014
prev sibling parent "Chris" <wendlec tcd.ie> writes:
On Monday, 31 March 2014 at 03:25:11 UTC, Manu wrote:
 On 31 March 2014 12:21, Walter Bright 
 <newshound2 digitalmars.com> wrote:

 On 3/30/2014 6:33 PM, Manu wrote:

 This is an interesting idea. Something I never thought of, 
 and I think I
 like it!
Frankly, I don't know why you use classes at all. Just use structs.
Reference types are very useful. Most programmers are familiar with this workflow, and it's a convenient way of modelling lots of problems. I do find myself using a lot more struct's in D though, but that doesn't void the traditional approach. And I also maintain that these things are important particularly as a bridge for new D users. I also feel quite dirty using pointers in D where there is a dedicated reference type available. I don't want * and & to appear everywhere in my D code.
Again and again I find myself reluctantly turning a struct into a class simply to get the reference semantics. Sure I could find work arounds and use * and & etc., but it just does not feel right, because hacks should only be the last resort, not something that is all over the place in your code. As has been mentioned earlier in this thread, these things often come back and bite you and all of a sudden it doesn't seem "so clever" anymore. On the other hand, I don't think that we should change the language, because of random annoyances that might partly be down to our design decisions taken earlier in the code.
Mar 31 2014
prev sibling next sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"deadalnix"  wrote in message news:jizqsnuigdsvxhtcsshl forum.dlang.org...

 All is in the title.

 This is becoming increasingly the norm in new languages. It is much better 
 in term of performances (it avoid cascaded loads to call methods, which 
 can be really costly on modern CPUs), make object themselves smaller.

 Would implementing them that way break D code ? What would be the extent 
 of the breakage ?
It could, but we don't have a stable ABI and we don't allow directly casting from interface references to classes. I would be interested to see how the performance changes on some interface-heavy D code - and honestly without an implementation that shows a big speed bump I don't think this will ever happen.
Mar 29 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/29/2014 8:26 PM, Daniel Murphy wrote:
 It could, but we don't have a stable ABI
? The ABI for interfaces hasn't changed in many years. In fact I don't even remember changes in it.
Mar 29 2014
parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Walter Bright"  wrote in message news:lh83dr$6m1$1 digitalmars.com...

 On 3/29/2014 8:26 PM, Daniel Murphy wrote:
 It could, but we don't have a stable ABI
? The ABI for interfaces hasn't changed in many years. In fact I don't even remember changes in it.
I meant stable in the sense that we _can't_ break it, not that we _haven't_ broken it recently. ie we don't guarantee binary compatibility across releases.
Mar 29 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/29/2014 8:38 PM, Daniel Murphy wrote:
 "Walter Bright"  wrote in message news:lh83dr$6m1$1 digitalmars.com...

 On 3/29/2014 8:26 PM, Daniel Murphy wrote:
 It could, but we don't have a stable ABI
? The ABI for interfaces hasn't changed in many years. In fact I don't even remember changes in it.
I meant stable in the sense that we _can't_ break it, not that we _haven't_ broken it recently. ie we don't guarantee binary compatibility across releases.
ok.
Mar 29 2014
parent reply "Orvid King" <blah38621 gmail.com> writes:
On Sun, 30 Mar 2014 00:11:50 -0500, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 3/29/2014 8:38 PM, Daniel Murphy wrote:
 "Walter Bright"  wrote in message news:lh83dr$6m1$1 digitalmars.com...

 On 3/29/2014 8:26 PM, Daniel Murphy wrote:
 It could, but we don't have a stable ABI
? The ABI for interfaces hasn't changed in many years. In fact I don't even remember changes in it.
I meant stable in the sense that we _can't_ break it, not that we _haven't_ broken it recently. ie we don't guarantee binary compatibility across releases.
ok.
Actually, there is one big thing that this would break; C++ interop. With the way interfaces currently work, you can define an interface with virtual methods, and you can call those methods on that interface, and, if you've properly overlayed it over a C++ class instance, you will be calling a virtual method defined on that C++ class. Fat interfaces would break this capability, and would actually break some of my existing code, due to the fact I use that exact capability. I do however mark my interfaces as extern(C++), so perhaps they would have to be an exception to the ABI if this change were made?
Mar 30 2014
parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Orvid King"  wrote in message 
news:mailman.124.1396235867.25518.digitalmars-d puremagic.com...

 Actually, there is one big thing that this would break; C++ interop. With 
 the way interfaces currently work, you can define an interface with 
 virtual methods, and you can call those methods on that interface, and, if 
 you've properly overlayed it over a C++ class instance, you will be 
 calling a virtual method defined on that C++ class. Fat interfaces would 
 break this capability, and would actually break some of my existing code, 
 due to the fact I use that exact capability. I do however mark my 
 interfaces as extern(C++), so perhaps they would have to be an exception 
 to the ABI if this change were made?
We wouldn't be changing the C++ ABI, just the D ABI. Only D interfaces would be affected.
Mar 30 2014
prev sibling next sibling parent Benjamin Thaut <code benjamin-thaut.de> writes:
Am 30.03.2014 01:57, schrieb deadalnix:
 All is in the title.

 This is becoming increasingly the norm in new languages. It is much
 better in term of performances (it avoid cascaded loads to call methods,
 which can be really costly on modern CPUs), make object themselves smaller.

 Would implementing them that way break D code ? What would be the extent
 of the breakage ?
As a nice side effect, debuggers could directly use the object pointer inside the fat interface pointer and don't have to reimplement D's casting routines. Kind Regards Benjamin Thaut
Mar 30 2014
prev sibling parent "QAston" <qaston gmail.com> writes:
On Sunday, 30 March 2014 at 00:57:25 UTC, deadalnix wrote:
 All is in the title.

 This is becoming increasingly the norm in new languages. It is 
 much better in term of performances (it avoid cascaded loads to 
 call methods, which can be really costly on modern CPUs), make 
 object themselves smaller.

 Would implementing them that way break D code ? What would be 
 the extent of the breakage ?
How hard would that be to implement? Hard data would be much better than performance guesses all over this thread.
Mar 31 2014