www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Vtable for virtual functions in D

reply Henrik <henrik nothing.com> writes:
Does anyone know if D is using the vtable implementation for 
virtual functions just like most C++ compilers? If yes, can 
someone explain the advantages of this strategy? A function 
pointer in C is regarded as expensive because of missing 
inlining, but a double indirection through a vtable just looks 
insane - if there aren't really good reasons for such an 
implementation. Does it make class inheritance or class 
polymorphism much simpler to implement or what is the reason?

I have worked with C in embedded systems for many years now, and 
for our modern Linux systems we are using a combination of C and 
Java today. Java for parts where memory safety is more important 
than speed/determinism, and C for the critical real time parts. 
There should exist a language between these worlds, where we can 
achieve memory safety at relatively small costs. C++ is not 
really an alternative, and D looks much more pleasant for us C 
programmers than for example Rust.
Mar 06 2018
next sibling parent sarn <sarn theartofmachinery.com> writes:
On Tuesday, 6 March 2018 at 21:20:22 UTC, Henrik wrote:

 Does anyone know if D is using the vtable implementation for 
 virtual functions just like most C++ compilers? If yes, can 
 someone explain the advantages of this strategy? A function 
 pointer in C is regarded as expensive because of missing 
 inlining, but a double indirection through a vtable just looks 
 insane - if there aren't really good reasons for such an 
 implementation. Does it make class inheritance or class 
 polymorphism much simpler to implement or what is the reason?

 I have worked with C in embedded systems for many years now, 
 and for our modern Linux systems we are using a combination of 
 C and Java today. Java for parts where memory safety is more 
 important than speed/determinism, and C for the critical real 
 time parts. There should exist a language between these worlds, 
 where we can achieve memory safety at relatively small costs. 
 C++ is not really an alternative, and D looks much more 
 pleasant for us C programmers than for example Rust.


D uses vtables.

It's a tradeoff between having double indirection and bloating 
each instance with the function pointers.  In cases where 
bloating isn't a problem, I just explicitly add normal function 
pointer members.
Mar 06 2018
prev sibling next sibling parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Tuesday, 6 March 2018 at 21:20:22 UTC, Henrik wrote:

 Does anyone know if D is using the vtable implementation for 
 virtual functions just like most C++ compilers? If yes, can 
 someone explain the advantages of this strategy? A function 
 pointer in C is regarded as expensive because of missing 
 inlining, but a double indirection through a vtable just looks 
 insane - if there aren't really good reasons for such an 
 implementation. Does it make class inheritance or class 
 polymorphism much simpler to implement or what is the reason?

 I have worked with C in embedded systems for many years now, 
 and for our modern Linux systems we are using a combination of 
 C and Java today. Java for parts where memory safety is more 
 important than speed/determinism, and C for the critical real 
 time parts. There should exist a language between these worlds, 
 where we can achieve memory safety at relatively small costs. 
 C++ is not really an alternative, and D looks much more 
 pleasant for us C programmers than for example Rust.


p0nce[1] has a link to an excellent article explaining in detail 
how the implementation of the vtable, single inheritance with 
interfaces work. The article isn't about D per se but D classes 
use the same mechanism.

[1]: https://p0nce.github.io/d-idioms/#Inside-the-D-Object-Model
Mar 07 2018
prev sibling next sibling parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Tuesday, 6 March 2018 at 21:20:22 UTC, Henrik wrote:

 I have worked with C in embedded systems for many years now, 
 and for our modern Linux systems we are using a combination of 
 C and Java today. Java for parts where memory safety is more 
 important than speed/determinism, and C for the critical real 
 time parts. There should exist a language between these worlds, 
 where we can achieve memory safety at relatively small costs. 
 C++ is not really an alternative, and D looks much more 
 pleasant for us C programmers than for example Rust.


D is the perfect fit for your description. Converting java to D 
can be surprisingly easy, unless one used a lot of libraries 
which are quite different.
As for C, I'm of the opinion that D is much more respectful of 
its C heritage than even C++ is.
Mar 07 2018
prev sibling parent reply Guillaume Piolat <notthat email.com> writes:
On Tuesday, 6 March 2018 at 21:20:22 UTC, Henrik wrote:

 Does anyone know if D is using the vtable implementation for 
 virtual functions just like most C++ compilers?


Yes, except without multiple inheritance / virtual inheritance.


 If yes, can someone explain the advantages of this strategy? A 
 function pointer in C is regarded as expensive because of 
 missing inlining, but a double indirection through a vtable 
 just looks insane - if there aren't really good reasons for 
 such an implementation.


Without measuring, I'd tend to agree with you that it should be 
more expensive (let's not use the arguments that "this part of 
memory will be hot" which means it will take a share of cache 
that could go to something else).

I guess the reason is simple: low overhead with regards to layout 
(only a pointer gets added to each instance). The v-table point 
itself to TypeInfo so you hide this pointer too.
TypeInfo is a bit like RTTI in the C++ world.

Note that even in C++ if a virtual call is expensive in the 
profiler you can switch on a type tag and devirtualize by casting 
(virtual functions are only expensive when the runtime type is 
unknown to the compiler).

The alternative layout is to embed the v-table in each object 
when priming a new object, and having the TypeInfo pointer in 
each class instance in addition to a pointer for each virtual 
method.


 Does it make class inheritance or class polymorphism much 
 simpler to implement or what is the reason?


I don't know. Intuitively it doesn't seem much easier with 
v-table.
COM objects are expected to have such a layout too.


 I have worked with C in embedded systems for many years now, 
 and for our modern Linux systems we are using a combination of 
 C and Java today. Java for parts where memory safety is more 
 important than speed/determinism, and C for the critical real 
 time parts. There should exist a language between these worlds, 
 where we can achieve memory safety at relatively small costs. 
 C++ is not really an alternative, and D looks much more 
 pleasant for us C programmers than for example Rust.


If you know enough D maybe you can implement your own virtual 
functions on top of D structs. It seems no one has made it yet.
Mar 07 2018
next sibling parent reply Henrik <henrik nothing.com> writes:
Hi everyone,

thank you all for your great answers. I'm playing around with 
 nogc right now, and it looks really promising. Strings and 
static arrays all seem to be located on the stack, which is so 
much better compared to std::string and std::vector in C++. The 
double indirection for virtual functions bother me, and it isn't 
getting better with all methods being virtual by default - I 
guess I'll be writing the keyword "final" very intensively in all 
my programs. I would also gladly pay some bytes in pointer bloat 
to have my virtual functions speed up. Strange that D kept the 
vtable approach when breaking with C++.

I know that D concentrated much on C++ compatibility a while ago. 
Are there any plans to support the direct inclusion on C header 
files? Many important libraries like ICU have C interfaces even 
when written in C++. The direct support of C headers would be 
very convenient in migrating parts of C/C++ projects to D. It 
would also open up POSIX which is used extensively in our work.

This looks great:
https://github.com/D-Programming-Deimos/
but it must be tedious to keep such files up to date.

Despite the point where I'm complaining, I must say that D looks 
very impressive. Beautiful syntax, large standard library, and 
standardized inline assembler (Thank you!). I'll definitely try 
to find a suitable project to try out D under serious conditions.
Mar 07 2018
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Mar 07, 2018 at 09:14:12PM +0000, Henrik via Digitalmars-d wrote:
[...]

 The direct support of C headers would be very convenient in migrating
 parts of C/C++ projects to D. It would also open up POSIX which is
 used extensively in our work.


[...]

There's already core.sys.posix.*, which are POSIX headers translated
into D, and which you can import and use right away. E.g.:

	import core.sys.posix.unistd;
	char[1024] buf;
	int fd = open("/tmp/file", O_RDONLY);
	auto len = read(fd, buf.ptr, buf.length);
	...
	close(fd);

Similarly, C standard library functions are directly available via
core.stdc.*:

	import core.stdc.stdlib;
	void* buf = malloc(1024);
	... // use buf here
	free(buf);

In principle, although direct support of C headers is not supported
(yet?), it's pretty easy to declare C functions in D and then just call
it directly, e.g.:

	// Declare C function here
	extern(C) size_t strlen(const(char)* s);

	void main()
	{
		import std.string : toStringz;

		// Call here
		auto len = strlen("blah blah".toStringz);
	}

In the above example I purposely used std.string to showcase how easy it
is to interoperate with C API functions, with toStringz and fromStringz
providing convenient conversions to/from C's char* strings.  The only
wrinkle here is that toStringz may allocate (because D strings are not
null-terminated in general), so cannot be used in  nogc code.

However, D string *literals* are always null-terminated, so the above
code could actually be replaced with just:

	auto len = strlen("blah blah".ptr);

and it will work in  nogc as well. Just be careful not to do that with
non-literal strings (or append a null manually and then it should be
safe).

The trouble with general C headers is that sometimes complicated macros
are used, and D does not come with a C macro processor, so it is tricky
to interface with those kinds of headers.  Still, if your code is not
directly dependent on the macros, you could just run the header through
a C preprocessor and then translate the C prototypes into D.  (There are
some cases for which this may not be so straightforward, but generally
you should be able to get quite far this way.)


T

-- 
Caffeine underflow. Brain dumped.
Mar 07 2018
prev sibling parent Laeeth Isharc <Laeeth laeeth.com> writes:
On Wednesday, 7 March 2018 at 21:14:12 UTC, Henrik wrote:


Many important libraries like ICU have C

 interfaces even when written in C++. The direct support of C 
 headers would be very convenient in migrating parts of C/C++ 
 projects to D. It would also open up POSIX which is used 
 extensively in our work.

 This looks great:
 https://github.com/D-Programming-Deimos/
 but it must be tedious to keep such files up to date.



"Are there any plans to support the direct inclusion [of] C 
header files?"

Yes, very shortly, as an extra build step.  Watch this space.



 Apr 02 2018



prev sibling  parent reply sarn <sarn theartofmachinery.com>  writes:


On Wednesday, 7 March 2018 at 12:49:40 UTC, Guillaume Piolat 
wrote:

 If you know enough D maybe you can implement your own virtual 
 functions on top of D structs. It seems no one has made it yet.


When I wrote Xanthe a year ago, I rolled my own classes using 
alias this and explicit vtables:
https://gitlab.com/sarneaud/xanthe/blob/master/src/game/rigid_body.d#L15
(I did this because normal D classes use the druntime library, 
and Xanthe was an experiment in not using the D runtime at all.)

Henrik: you might be interested in this post I wrote about making 
that:
https://theartofmachinery.com/2017/02/28/bare_metal_d.html
NB: things are moving fast and some things have already improved 
since then.



 Mar 07 2018



 next sibling parent reply Mike Franklin <slavo5150 yahoo.com>  writes:


On Wednesday, 7 March 2018 at 22:02:17 UTC, sarn wrote:


 When I wrote Xanthe a year ago, I rolled my own classes using 
 alias this and explicit vtables:
 https://gitlab.com/sarneaud/xanthe/blob/master/src/game/rigid_body.d#L15
 (I did this because normal D classes use the druntime library, 
 and Xanthe was an experiment in not using the D runtime at all.)


Nice!  I was thinking about something almost exactly like this 
recently since 2.079.0 has features to further decouple the 
language from the runtime.  It would be nice to read a blog post 
about this technique.

Mike



 Mar 07 2018



  parent reply sarn <sarn theartofmachinery.com>  writes:


On Thursday, 8 March 2018 at 04:37:08 UTC, Mike Franklin wrote:

 Nice!  I was thinking about something almost exactly like this 
 recently since 2.079.0 has features to further decouple the 
 language from the runtime.  It would be nice to read a blog 
 post about this technique.

 Mike


I didn't realise there was any interest.  Sure, I'll try to make 
it one of my next few posts.

And, yeah, the new 2.079 release has made some very significant 
improvements.  I'm still experimenting to find out what's 
possible now.



 Mar 08 2018



  parent reply sarn <sarn theartofmachinery.com>  writes:


On Thursday, 8 March 2018 at 22:07:24 UTC, sarn wrote:

 On Thursday, 8 March 2018 at 04:37:08 UTC, Mike Franklin wrote:

 Nice!  I was thinking about something almost exactly like this 
 recently since 2.079.0 has features to further decouple the 
 language from the runtime.  It would be nice to read a blog 
 post about this technique.

 Mike


 I didn't realise there was any interest.  Sure, I'll try to 
 make it one of my next few posts.


I decided to pull some basic background info about vtables, etc, 
into its own post.  I'll write about taking advantage of alias 
this and template metaprogramming in a later post.
https://theartofmachinery.com/2018/04/02/inheritance_and_polymorphism.html



 Apr 02 2018



 next sibling parent  Mike Franklin <slavo5150 yahoo.com>  writes:


On Monday, 2 April 2018 at 07:02:07 UTC, sarn wrote:

 On Thursday, 8 March 2018 at 22:07:24 UTC, sarn wrote:

 On Thursday, 8 March 2018 at 04:37:08 UTC, Mike Franklin wrote:

 Nice!  I was thinking about something almost exactly like 
 this recently since 2.079.0 has features to further decouple 
 the language from the runtime.  It would be nice to read a 
 blog post about this technique.

 Mike


 I didn't realise there was any interest.  Sure, I'll try to 
 make it one of my next few posts.


 I decided to pull some basic background info about vtables, 
 etc, into its own post.  I'll write about taking advantage of 
 alias this and template metaprogramming in a later post.
 https://theartofmachinery.com/2018/04/02/inheritance_and_polymorphism.html


Nice!  Looking forward to part 2.

Mike



 Apr 02 2018



prev sibling  parent  Kagamin <spam here.lot>  writes:


On Monday, 2 April 2018 at 07:02:07 UTC, sarn wrote:

 I decided to pull some basic background info about vtables, 
 etc, into its own post.  I'll write about taking advantage of 
 alias this and template metaprogramming in a later post.
 https://theartofmachinery.com/2018/04/02/inheritance_and_polymorphism.html


You can use COM interfaces. You'll still need to build vtable by 
hand, but at least you'll have some syntax sugar at the call site 
if you don't have multiple inheritance.



 Apr 03 2018



prev sibling next sibling parent  Guillaume Piolat <notthat email.com>  writes:


On Wednesday, 7 March 2018 at 22:02:17 UTC, sarn wrote:

 On Wednesday, 7 March 2018 at 12:49:40 UTC, Guillaume Piolat 
 wrote:

 If you know enough D maybe you can implement your own virtual 
 functions on top of D structs. It seems no one has made it yet.


 When I wrote Xanthe a year ago, I rolled my own classes using 
 alias this and explicit vtables:
 https://gitlab.com/sarneaud/xanthe/blob/master/src/game/rigid_body.d#L15
 (I did this because normal D classes use the druntime library, 
 and Xanthe was an experiment in not using the D runtime at all.)

 Henrik: you might be interested in this post I wrote about 
 making that:
 https://theartofmachinery.com/2017/02/28/bare_metal_d.html
 NB: things are moving fast and some things have already 
 improved since then.


Very interesting



 Mar 08 2018



prev sibling next sibling parent reply Henrik <henrik nothing.com>  writes:


On Wednesday, 7 March 2018 at 22:02:17 UTC, sarn wrote:

 On Wednesday, 7 March 2018 at 12:49:40 UTC, Guillaume Piolat 
 wrote:

 If you know enough D maybe you can implement your own virtual 
 functions on top of D structs. It seems no one has made it yet.


 When I wrote Xanthe a year ago, I rolled my own classes using 
 alias this and explicit vtables:
 https://gitlab.com/sarneaud/xanthe/blob/master/src/game/rigid_body.d#L15
 (I did this because normal D classes use the druntime library, 
 and Xanthe was an experiment in not using the D runtime at all.)

 Henrik: you might be interested in this post I wrote about 
 making that:
 https://theartofmachinery.com/2017/02/28/bare_metal_d.html
 NB: things are moving fast and some things have already 
 improved since then.


This is great, it is exactly what I was hoping for! The 
core.sys.posix is also pure gold. I'm obviously not the first 
person here to have an interest in this topic. Really pleasant 
reading!

In my view; the evolution of programming languages skipped at 

and similar languages. If D could form an natural next step after 
C, with a favorable tradeoff between memory safety and 
performance, it could really change the embedded software 
industry. C11 + Valgrind/AdressSanitizer etc are great to develop 
fast reliable software, and most universities here in Europe - 
from my experience at least - are teaching C, but not as much 
C++. I like C, but it really only dominates because everyone else 
performs so poorly in embedded/system environments.

I should probably move to the learn category for the next part, 
but I can post one example at least. It is my first naive step of 
creating a RAII struct dummy, and check my program for memory 
corruptions; two things that C cannot do. It all works good, but 
why do I have to put the  nogc on the constructor and destructor 
separately?

import std.stdio;
import core.exception;

 nogc:

struct Nice
{
     int a;
     this(int a)  nogc
     {
         this.a = a;
         puts("A");
     }

     ~this()  nogc
     {
         puts("B");
     }
}

void TestStruct()
{
     //char *c = new char();
     Nice n = Nice(55);
}

void TestIndexOutOfBounds(int i)
{
     int a[100] = void;
     a[0] = a[i];
}

void main(string[ ] args)
{
     try {
         TestStruct();
         TestIndexOutOfBounds(100);
     } catch(core.exception.RangeError e) {
         puts("Sloppy work! Enter safety mode!");
     }
}



 Mar 08 2018



 next sibling parent  sarn <sarn theartofmachinery.com>  writes:


On Thursday, 8 March 2018 at 22:56:27 UTC, Henrik wrote:

why do I have to put the  nogc on the constructor and
 destructor separately?


You can make things slightly better by putting  nogc in the 
struct itself:

struct S
{
    nogc:

   void member1()
   {
   }

   void member2()
   {
   }
}

But, yeah, this is a current pain point: attributes aren't 
"deep".  Maybe one day we'll have a syntax for applying 
attributes recursively, but I think it's blocking on the lack of 
a syntax for inverting an attribute.  (I.e., a way to declare an 
impure/gc/whatever thing inside a region that's declared 
pure/nogc/whatever.)



 Mar 08 2018



prev sibling  parent  Manu <turkeyman gmail.com>  writes:


On 8 March 2018 at 14:56, Henrik via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 It all works good, but why do I have to put the  nogc on the constructor and
destructor separately?


 nogc in the global scope does not propagate inside the class (this
could lead to serious problems).
You can use ` nogc:` at the top of your class to make all class
contents  nogc too.



 Mar 09 2018



prev sibling next sibling parent reply Mike Franklin <slavo5150 yahoo.com>  writes:


On Wednesday, 7 March 2018 at 22:02:17 UTC, sarn wrote:


 When I wrote Xanthe a year ago, I rolled my own classes using 
 alias this and explicit vtables:
 https://gitlab.com/sarneaud/xanthe/blob/master/src/game/rigid_body.d#L15
 (I did this because normal D classes use the druntime library, 
 and Xanthe was an experiment in not using the D runtime at all.)


I'm curious about this comment in the code:


 Unfortunately, "protected" doesn't work, so a lot of members 
 end up being public.  This seems to just be an oversight in the 
 language, so maybe it will change in future versions of D.


What specifically do you think should be changed in the language 
to support your idea?

Thanks,
Mike



 Apr 02 2018



  parent  sarn <sarn theartofmachinery.com>  writes:


On Tuesday, 3 April 2018 at 00:22:52 UTC, Mike Franklin wrote:

 I'm curious about this comment in the code:


 Unfortunately, "protected" doesn't work, so a lot of members 
 end up being public.  This seems to just be an oversight in 
 the language, so maybe it will change in future versions of D.


 What specifically do you think should be changed in the 
 language to support your idea?

 Thanks,
 Mike


Sorry, I'll go into the details in my blog post, but what that 
comment is talking about is this: D has a "protected" specifier 
for classes, just like C++, but not one for structs - struct 
members can only be private or public.  This is because D structs 
don't natively support inheritance.



 Apr 03 2018



prev sibling  parent  Kagamin <spam here.lot>  writes:


On Wednesday, 7 March 2018 at 22:02:17 UTC, sarn wrote:

 When I wrote Xanthe a year ago, I rolled my own classes using 
 alias this and explicit vtables:
 https://gitlab.com/sarneaud/xanthe/blob/master/src/game/rigid_body.d#L15
 (I did this because normal D classes use the druntime library, 
 and Xanthe was an experiment in not using the D runtime at all.)


You can use address of vtable as type id. Unique and doesn't need 
CTFE.

alias TypeId=immutable(VTable*);
alias type_id=_vtable;



 Apr 03 2018