www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Struct default constructor - need some kind of solution for C++

reply Ethan Watson <gooberman gmail.com> writes:
Alright, so now I've definitely come up across something with 
Binderoo that has no easy solution.

For the sake of this example, I'm going to use the class I'm 
binary-matching with a C++ class and importing functionality with 
C++ function pointers to create a 100% functional match - our 
Mutex class. It doesn't have to be a mutex, it just needs to be 
any C++ class where a default constructor is non-trivial.

In C++, it looks much like what you'd expect:

class Mutex
{
public:
   Mutex();
   ~Mutex();
   void lock();
   bool tryLock();
   void unlock();

private:
   CRITICAL_SECTION  m_criticalSection;
};

Cool. Those functions call the exact library functions you'd 
expect, the constructor does an InitializeCriticalSection and the 
destructor does a DeleteCriticalSection.

Now, with Binderoo aiming to provide complete C++ matching to the 
point where it doesn't matter whether a class was allocated in 
C++ or D, this means I've chosen to make every C++-matching class 
a value type rather than a reference type. The reasoning is 
pretty simple:

class SomeOtherClass
{
private:
   SomeVitalObject m_object;
   Mutex           m_mutex;
};

This is a pretty common pattern. Other C++ classes will embed 
mutex instances inside them. A reference type for matching in 
this case is right out of the question. Which then leads to a 
major conundrum - default constructing this object in D.

D structs have initialisers, but you're only allowed constructors 
if you pass arguments. With a Binderoo matching struct 
declaration, it would basically look like this:

struct Mutex
{
    BindConstructor void __ctor();
    BindDestructor void __dtor();

    BindMethod void lock();
    BindMethod bool tryLock();
    BindMethod void unlock();

   private CRITICAL_SECTION m_criticalSection;
}

After mixin expansion, it would look come out looking something 
like this:

struct Mutex
{
   pragma( inline ) this() { __methodTable.function0(); }
   pragma( inline ) ~this() { __methodTable.function1(); }

   pragma( inline ) void lock() { __methodTable.function2(); }
   pragma( inline ) bool tryLock() { return 
__methodTable.function3(); }
   pragma( inline ) void unlock() { __methodTable.function4(); }

   private CRITICAL_SECTION m_criticalSection;
}

(Imagine __methodTable is a gshared object with the relevant 
function pointers imported from C++.)

Of course, it won't compile. this() is not allowed for obvious 
reasons. But in this case, we need to call a corresponding 
non-trivial constructor in C++ code to get the functionality 
match.

Of course, given the simplicity of the class, I don't need to 
import C++ code to provide exact functionality at all. But I 
still need to call InitializeCriticalSection somehow whenever 
it's instantiated anywhere. This pattern of non-trivial default 
constructors is certainly not limited to mutexes, not in our 
codebase or wider C++ practices at all.

So now I'm in a bind. This is one struct I need to construct 
uniquely every time. And I also need to keep the usability up to 
not require calling some other function since this is matching a 
C++ class's functionality, including its ability to instantiate 
anywhere.

Suggestions?
Sep 06 2016
next sibling parent reply Lodovico Giaretta <lodovico giaretart.net> writes:
On Tuesday, 6 September 2016 at 13:44:37 UTC, Ethan Watson wrote:

 [...]
 Suggestions?


Of course I don't know which level of usability you want to 
achieve, but I think that in this case your bind system, when 
binding a default ctor, could use  disable this() and define a 
factory method (do static opCall work?) that calls the C++ ctor.

It's not as good-looking as a true default ctor, but it doesn't 
provide any way to introduce bugs and it's not that bad (just a 
couple key strokes).
Sep 06 2016
next sibling parent Lodovico Giaretta <lodovico giaretart.net> writes:
On Tuesday, 6 September 2016 at 13:57:27 UTC, Lodovico Giaretta 
wrote:

 On Tuesday, 6 September 2016 at 13:44:37 UTC, Ethan Watson 
 wrote:

 [...]
 Suggestions?


 Of course I don't know which level of usability you want to 
 achieve, but I think that in this case your bind system, when 
 binding a default ctor, could use  disable this() and define a 
 factory method (do static opCall work?) that calls the C++ ctor.

 It's not as good-looking as a true default ctor, but it doesn't 
 provide any way to introduce bugs and it's not that bad (just a 
 couple key strokes).


Correcting my answer. The following code compiles fine:

struct S
{
     static S opCall()
     {
	S res = void;
	// call C++ ctor
	return res;
     }
}

void main()
{
     S s = S();
}

But introduces the possibility of using the default ctor 
inadvertitely.
Sadly, the following does not compile:

struct S
{
      disable this();
     static S opCall()
     {
	S res = void;
	// call C++ ctor
	return res;
     }
}

Making this compile would solve your issues.
Sep 06 2016
prev sibling parent reply Ethan Watson <gooberman gmail.com> writes:
On Tuesday, 6 September 2016 at 13:57:27 UTC, Lodovico Giaretta 
wrote:

 Of course I don't know which level of usability you want to 
 achieve, but I think that in this case your bind system, when 
 binding a default ctor, could use  disable this() and define a 
 factory method (do static opCall work?) that calls the C++ ctor.


static opCall doesn't work for the SomeOtherClass example listed 
in OP.  disable this() will hide the static opCall and the 
compiler will throw an error.

Somewhat related: googling "factory method dlang" doesn't provide 
any kind of clarity on what exactly is a factory method. 
Documentation for factory methods/functions could probably be 
improved on this front.
Sep 06 2016
parent reply Lodovico Giaretta <lodovico giaretart.net> writes:
On Tuesday, 6 September 2016 at 14:10:43 UTC, Ethan Watson wrote:

  disable this() will hide the static opCall and the compiler 
 will throw an error.


Yes, I realized that. My bad.

As  disable this is not actually defining a ctor, it should not 
be signaled as hiding the opCall. To me, this looks like an 
oversight in the frontend that should be fixed.


 static opCall doesn't work for the SomeOtherClass example 
 listed in OP.


That's because it doesn't initialize (with static opCall) the 
fields of SomeOtherClass, right? I guess that could be solved 
once and for all with some template magic of the binding system.
Sep 06 2016
parent Ethan Watson <gooberman gmail.com> writes:
On Tuesday, 6 September 2016 at 14:27:49 UTC, Lodovico Giaretta 
wrote:

 That's because it doesn't initialize (with static opCall) the 
 fields of SomeOtherClass, right? I guess that could be solved 
 once and for all with some template magic of the binding system.


Correct for the first part. The second part... not so much. Being 
all value types, there's nothing stopping you instantiating the 
example Mutex on the stack in a function in D - and no way of 
enforcing the user to go through a custom construction path 
either.
Sep 06 2016
prev sibling next sibling parent reply Ethan Watson <gooberman gmail.com> writes:
On Tuesday, 6 September 2016 at 13:44:37 UTC, Ethan Watson wrote:

 Suggestions?


Forgot to mention in OP that I had tried this( void* pArg = null 
); to no avail:

mutex.d(19): Deprecation: constructor mutex.Mutex.this all 
parameters have default arguments, but structs cannot have 
default constructors.

It's deprecated and the constructor doesn't get called. So no 
egregious sploits for me.
Sep 06 2016
parent reply Ethan Watson <gooberman gmail.com> writes:
On Tuesday, 6 September 2016 at 14:49:20 UTC, Ethan Watson wrote:

 this( void* pArg = null );


Also doesn't work: this( Args... )( Args args ) if( Args.length 
== 0 )

Just for funsies I tried making my Mutex a class for the purpose 
of embedding it manually in a struct. But thanks to all classes 
inheriting from Object there's 16 bytes at the front of the class 
that I don't want (64-bit build, it's 8 bytes in 32-bit builds 
but we're never going back to 32-bit). So that's very definitely 
out of the question.

static opCall() seems to be the only way to do this then. I can 
autogenerate it for any C++ bound class. But it's inadequate. It 
leaves room for user error when instantiating any C++ object in 
D. It's also another thing that C++ programmers need to be 
thoroughly educated about as Type() in C++11 calls the zero 
initializer, but in D it's effectively the opposite semantics.
Sep 07 2016
next sibling parent reply Lodovico Giaretta <lodovico giaretart.net> writes:
On Wednesday, 7 September 2016 at 11:16:20 UTC, Ethan Watson 
wrote:

 On Tuesday, 6 September 2016 at 14:49:20 UTC, Ethan Watson 
 wrote:

 this( void* pArg = null );


 Also doesn't work: this( Args... )( Args args ) if( Args.length 
 == 0 )

 Just for funsies I tried making my Mutex a class for the 
 purpose of embedding it manually in a struct. But thanks to all 
 classes inheriting from Object there's 16 bytes at the front of 
 the class that I don't want (64-bit build, it's 8 bytes in 
 32-bit builds but we're never going back to 32-bit). So that's 
 very definitely out of the question.

 static opCall() seems to be the only way to do this then. I can 
 autogenerate it for any C++ bound class. But it's inadequate. 
 It leaves room for user error when instantiating any C++ object 
 in D. It's also another thing that C++ programmers need to be 
 thoroughly educated about as Type() in C++11 calls the zero 
 initializer, but in D it's effectively the opposite semantics.


I guess the only thing you can ask and obtain here (I mean, with 
a bug report) is that  disable this() should be allowed to 
coexist with static opCall(). That would prevent your users from 
instantiating structs the wrong way (both nested in other structs 
and on the stack).
Sep 07 2016
parent reply Nick Treleaven <nick geany.org> writes:
On Wednesday, 7 September 2016 at 11:20:18 UTC, Lodovico Giaretta 
wrote:

 I guess the only thing you can ask and obtain here (I mean, 
 with a bug report) is that  disable this() should be allowed to 
 coexist with static opCall(). That would prevent your users 
 from instantiating structs the wrong way (both nested in other 
 structs and on the stack).


Or perhaps explicit this() could be allowed if we have ` disable 
S init` defined for S:

S s; // error, S.init is disabled
S s = S(); // OK

So how would this deal with Walter's 4 axioms:


 1. So S.init is a valid initializer


No, S.init is disallowed. Any code that needs it should fail to 
compile.


 2. So all instances of S can be guaranteed to contain a valid 
 instance


Yes, because the only instances that exist are those explicitly 
constructed by constructor call.


 3. So default initialization is guaranteed to succeed


No, this is disallowed (see 1).


 4. So any struct constructor starts with a valid state


This can be handled by initializing fields to their respective 
.init values before any constructor of S is called.
Sep 10 2016
parent reply Dicebot <public dicebot.lv> writes:
Presence of compile-time valid T.init for any type T is 
absolutely critical for generic programming and must not be 
compromised.
Sep 10 2016
parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 September 2016 at 06:09:01 UTC, Dicebot wrote:

 Presence of compile-time valid T.init for any type T is 
 absolutely critical for generic programming and must not be 
 compromised.


WAT ?
Sep 11 2016
parent reply Dicebot <public dicebot.lv> writes:
On Sunday, 11 September 2016 at 08:37:56 UTC, deadalnix wrote:

 On Sunday, 11 September 2016 at 06:09:01 UTC, Dicebot wrote:

 Presence of compile-time valid T.init for any type T is 
 absolutely critical for generic programming and must not be 
 compromised.


 WAT ?


Vast amount of traits operate on is(typeof(do stuff with T.init)) 
- all of them will return false negative for types with init 
disabled.
Sep 11 2016
parent reply David Nadlinger <code klickverbot.at> writes:
On Sunday, 11 September 2016 at 11:33:14 UTC, Dicebot wrote:

 On Sunday, 11 September 2016 at 08:37:56 UTC, deadalnix wrote:

 On Sunday, 11 September 2016 at 06:09:01 UTC, Dicebot wrote:

 Presence of compile-time valid T.init for any type T is 
 absolutely critical for generic programming and must not be 
 compromised.


 WAT ?


 Vast amount of traits operate on is(typeof(do stuff with 
 T.init)) - all of them will return false negative for types 
 with init disabled.


That's confusing an arbitrary implementation with necessity. 
lvalueOf!T/rvalueOf!T would be more than enough for these traits.

  — David
Sep 11 2016
parent deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 September 2016 at 12:11:43 UTC, David Nadlinger 
wrote:

 On Sunday, 11 September 2016 at 11:33:14 UTC, Dicebot wrote:

 On Sunday, 11 September 2016 at 08:37:56 UTC, deadalnix wrote:

 On Sunday, 11 September 2016 at 06:09:01 UTC, Dicebot wrote:

 Presence of compile-time valid T.init for any type T is 
 absolutely critical for generic programming and must not be 
 compromised.


 WAT ?


 Vast amount of traits operate on is(typeof(do stuff with 
 T.init)) - all of them will return false negative for types 
 with init disabled.


 That's confusing an arbitrary implementation with necessity. 
 lvalueOf!T/rvalueOf!T would be more than enough for these 
 traits.

  — David


This.
Sep 11 2016
prev sibling next sibling parent reply Daniel Kozak via Digitalmars-d <digitalmars-d puremagic.com> writes:
Dne 7.9.2016 v 13:16 Ethan Watson via Digitalmars-d napsal(a):


 On Tuesday, 6 September 2016 at 14:49:20 UTC, Ethan Watson wrote:

 this( void* pArg = null );


 Also doesn't work: this( Args... )( Args args ) if( Args.length == 0 )

 Just for funsies I tried making my Mutex a class for the purpose of 
 embedding it manually in a struct. But thanks to all classes 
 inheriting from Object there's 16 bytes at the front of the class that 
 I don't want (64-bit build, it's 8 bytes in 32-bit builds but we're 
 never going back to 32-bit). So that's very definitely out of the 
 question.


Even extern(C++) class ?

https://dlang.org/spec/cpp_interface.html#classes
Sep 07 2016
parent reply Dicebot <public dicebot.lv> writes:
On Wednesday, 7 September 2016 at 11:35:45 UTC, Daniel Kozak 
wrote:

 Dne 7.9.2016 v 13:16 Ethan Watson via Digitalmars-d napsal(a):


 On Tuesday, 6 September 2016 at 14:49:20 UTC, Ethan Watson 
 wrote:

 this( void* pArg = null );


 Also doesn't work: this( Args... )( Args args ) if( 
 Args.length == 0 )

 Just for funsies I tried making my Mutex a class for the 
 purpose of embedding it manually in a struct. But thanks to 
 all classes inheriting from Object there's 16 bytes at the 
 front of the class that I don't want (64-bit build, it's 8 
 bytes in 32-bit builds but we're never going back to 32-bit). 
 So that's very definitely out of the question.


 Even extern(C++) class ?

 https://dlang.org/spec/cpp_interface.html#classes


If it is so, I'd call it a major extern(c++) bug.
Sep 07 2016
next sibling parent reply Ethan Watson <gooberman gmail.com> writes:
On Wednesday, 7 September 2016 at 11:42:40 UTC, Dicebot wrote:

 If it is so, I'd call it a major extern(c++) bug.


The documentation seems to be correct. I can't extern( C++, class 
) or extern( C++, struct ) on an object, even in DMD 
2.071.2-beta3.

But ignoring that. My first member is offset by 8 bytes, even in 
an extern( C++ ) class. I assume it's just blindly sticking a 
vtable in there regardless of if I actually define virtual 
functions or not.

But regardless. Making it a class is still a bad idea since in 
this exact example it needs to exist on the stack/within an 
objects scope, which means you then need to further hack around 
with emplacement and wrappers and blah.

Binary matching, non-trivial constructors, and treating C++ 
objects like the value types they are will be required to make 
Binderoo work effortlessly. I've got two out of three of those. 
Not having any one of those is something of a deal breaker unless 
I get an effective workaround.
Sep 07 2016
next sibling parent reply Johan Engelen <j j.nl> writes:
On Wednesday, 7 September 2016 at 12:07:56 UTC, Ethan Watson 
wrote:

 The documentation seems to be correct. I can't extern( C++, 
 class ) or extern( C++, struct ) on an object, even in DMD 
 2.071.2-beta3.


`extern( C++, class/struct )` is supported by DMD master and LDC 
1.1.0-beta*.
Afaik, there is no versioning of the spec, and so the spec was 
already updated to prevent the spec going out-of-sync with the 
implementation.
Sep 07 2016
parent Johan Engelen <j j.nl> writes:
On Wednesday, 7 September 2016 at 12:26:25 UTC, Johan Engelen 
wrote:

 On Wednesday, 7 September 2016 at 12:07:56 UTC, Ethan Watson 
 wrote:

 The documentation seems to be correct. I can't extern( C++, 
 class ) or extern( C++, struct ) on an object, even in DMD 
 2.071.2-beta3.


 `extern( C++, class/struct )` is supported by DMD master and 
 LDC 1.1.0-beta*.


But it will only change mangling, it doesn't do what you want to 
do with it I think.
Sep 07 2016
prev sibling next sibling parent Daniel Kozak via Digitalmars-d <digitalmars-d puremagic.com> writes:
Dne 7.9.2016 v 14:07 Ethan Watson via Digitalmars-d napsal(a):


 On Wednesday, 7 September 2016 at 11:42:40 UTC, Dicebot wrote:

 If it is so, I'd call it a major extern(c++) bug.


 The documentation seems to be correct. I can't extern( C++, class ) or 
 extern( C++, struct ) on an object, even in DMD 2.071.2-beta3.

 But ignoring that. My first member is offset by 8 bytes, even in an 
 extern( C++ ) class. I assume it's just blindly sticking a vtable in 
 there regardless of if I actually define virtual functions or not.

 But regardless. Making it a class is still a bad idea since in this 
 exact example it needs to exist on the stack/within an objects scope, 
 which means you then need to further hack around with emplacement and 
 wrappers and blah.

 Binary matching, non-trivial constructors, and treating C++ objects 
 like the value types they are will be required to make Binderoo work 
 effortlessly. I've got two out of three of those. Not having any one 
 of those is something of a deal breaker unless I get an effective 
 workaround.


I belive there is no way how to achive what you want, maybe it could be 
possible to extent
extern (C++) syntax for structs, so it will be possible define this() 
for this specific structs
Sep 07 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/7/2016 5:07 AM, Ethan Watson wrote:

 But ignoring that. My first member is offset by 8 bytes, even in an extern( C++
 ) class. I assume it's just blindly sticking a vtable in there regardless of if
 I actually define virtual functions or not.


This came up before with Manu's desire to match complex C++ hierarchies that 
are, in essence, multiple inheritance even if they didn't look like multiple 
inheritance. D only supports multiple inheritance as interfaces. Having C++ 
classes with no _vptr means multiple inheritance (I know this isn't obvious
why, 
but if the details are worked through it is inevitable.)

I suggested to Manu some ways to get a workalike, but I don't recall what they
were.

Supporting MI is not impossible as a D enhancement, but it would entail a fair 
amount of rework as the compiler internals just don't work that way. If and 
until such a time as that is done, having classes with no _vptr isn't going to
work.
Sep 07 2016
parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 7 September 2016 at 20:55:52 UTC, Walter Bright 
wrote:

 On 9/7/2016 5:07 AM, Ethan Watson wrote:

 But ignoring that. My first member is offset by 8 bytes, even 
 in an extern( C++
 ) class. I assume it's just blindly sticking a vtable in there 
 regardless of if
 I actually define virtual functions or not.


 This came up before with Manu's desire to match complex C++ 
 hierarchies that are, in essence, multiple inheritance even if 
 they didn't look like multiple inheritance. D only supports 
 multiple inheritance as interfaces. Having C++ classes with no 
 _vptr means multiple inheritance (I know this isn't obvious 
 why, but if the details are worked through it is inevitable.)


He is not the only one. I raised that has the n°1 problem I had 
when interfacing with C++ years ago. Even made a proposal.

It is clear at this point that structures with obligatory 
initialization are necessary. For C++ but not only. Right now, 
all dtors need to make sure that the .init state is valid, which 
can be a performance problem (you need to add runtime checks to 
know if you actually need to destroy a resource).
Sep 07 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/7/2016 2:08 PM, deadalnix wrote:

 It is clear at this point that structures with obligatory initialization are
 necessary. For C++ but not only.


If not interfacing to C++, why?


 Right now, all dtors need to make sure that the
 .init state is valid, which can be a performance problem (you need to add
 runtime checks to know if you actually need to destroy a resource).


Is:

     if (resource != null)
         resource.destroy();

v.s.:

     resource.destroy();

so onerous? It's one TST/JNE pair for a value loaded into a register anyway.
And 
with a default constructor, there's all that code added to deal with the 
constructor failing and throwing.

Besides, you can still write:

     struct S {
         Resource resource;

         Resource builder() {
             S s;
             s.resource = new Resource();
             return s;
         }

         ~this() {
             assert(resource);  // ensure user used builder()
             resource.destroy();
         }
     }
Sep 07 2016
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 7 September 2016 at 22:52:04 UTC, Walter Bright 
wrote:

 On 9/7/2016 2:08 PM, deadalnix wrote:

 It is clear at this point that structures with obligatory 
 initialization are
 necessary. For C++ but not only.


 If not interfacing to C++, why?


I stated why.


 Is:

     if (resource != null)
         resource.destroy();

 v.s.:

     resource.destroy();


In some cases yes. Consider reference counting for instance.


 so onerous? It's one TST/JNE pair for a value loaded into a 
 register anyway. And with a default constructor, there's all 
 that code added to deal with the constructor failing and 
 throwing.


One needs to construct anyway.
Sep 07 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/7/2016 4:05 PM, deadalnix wrote:

 Consider reference counting for instance.


Andrei's scheme for RC doesn't have that issue.
Sep 07 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/8/16 1:37 AM, Walter Bright wrote:

 On 9/7/2016 4:05 PM, deadalnix wrote:

 Consider reference counting for instance.


 Andrei's scheme for RC doesn't have that issue.


Default constructors would help RC design. Raising the roof. -- Andrei
Sep 08 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/7/2016 4:05 PM, deadalnix wrote:

 And with a default constructor, there's all that code added to deal with the
 constructor failing and throwing.


 One needs to construct anyway.


For constructor failures, one has little choice but to throw an exception. 
Despite all the great work at making exceptions zero-cost for the non-throwing 
path, they aren't zero-cost.

With a 'builder()', there are options other than throwing an exception, if 
performance is the top priority.
Sep 07 2016
prev sibling parent reply Ethan Watson <gooberman gmail.com> writes:
On Wednesday, 7 September 2016 at 22:52:04 UTC, Walter Bright 
wrote:

 Is:

     if (resource != null)
         resource.destroy();

 v.s.:

     resource.destroy();

 so onerous? It's one TST/JNE pair for a value loaded into a 
 register anyway.


This one has performance implications for game developers. The 
branch predictor in the CPU used for the Xbox One and the PS4 
isn't the greatest. If, for example, that destructor gets inlined 
and you're iterating over a range of resources and the destroy 
method is virtual, there's a good chance you will invoke the 
wrath of the dense branch predictor. You don't want to deal with 
the dense branch predictor.

http://www.agner.org/optimize/microarchitecture.pdf section 3.13 
has a bit more info on the branch predictor. Desktop Intel CPUs 
tend to hide performance problems like this thanks to their 
far-higher-quality branch predictors. Both chips gain benefits 
from sorting to how you expect the branch predictor to work, but 
there's a lot of code in a game codebase that isn't that low 
level.
Sep 08 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/8/2016 1:10 AM, Ethan Watson wrote:

 On Wednesday, 7 September 2016 at 22:52:04 UTC, Walter Bright wrote:

 Is:

     if (resource != null)
         resource.destroy();

 v.s.:

     resource.destroy();

 so onerous? It's one TST/JNE pair for a value loaded into a register anyway.


 This one has performance implications for game developers. The branch predictor
 in the CPU used for the Xbox One and the PS4 isn't the greatest. If, for
 example, that destructor gets inlined and you're iterating over a range of
 resources and the destroy method is virtual, there's a good chance you will
 invoke the wrath of the dense branch predictor. You don't want to deal with the
 dense branch predictor.


The thing is, the 'destroy()' function is going to swamp any extra clock cycle, 
as will a virtual lookup and dereference.



 http://www.agner.org/optimize/microarchitecture.pdf section 3.13 has a bit more
 info on the branch predictor. Desktop Intel CPUs tend to hide performance
 problems like this thanks to their far-higher-quality branch predictors. Both
 chips gain benefits from sorting to how you expect the branch predictor to
work,
 but there's a lot of code in a game codebase that isn't that low level.


There's another way. Just don't worry about it being null, if you're going to 
ensure it is initialized regardless. It'll seg fault if it is dereferenced but 
not initialized.
Sep 08 2016
next sibling parent reply Ethan Watson <gooberman gmail.com> writes:
On Thursday, 8 September 2016 at 11:18:12 UTC, Walter Bright 
wrote:

 The thing is, the 'destroy()' function is going to swamp any 
 extra clock cycle, as will a virtual lookup and dereference.


Assume destroy() is a more trivial function then. The point is 
that if you put more than two branches in a 64-byte cacheline on 
that processor, things get significantly slower and the loop 
iteration itself becomes a hotspot.

Being D though. Destructors can be contracted, yeah? Because the 
way we operate is that we compile out all those validation checks 
for a retail release and assume everything works. Checking for 
the validity of the pointer in an in block would be perfect for 
that.
Sep 08 2016
next sibling parent Ethan Watson <gooberman gmail.com> writes:
On Thursday, 8 September 2016 at 11:26:22 UTC, Ethan Watson wrote:

 Being D though. Destructors can be contracted, yeah?


Tested. Confirmed.
Sep 08 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/8/2016 4:26 AM, Ethan Watson wrote:

 On Thursday, 8 September 2016 at 11:18:12 UTC, Walter Bright wrote:

 The thing is, the 'destroy()' function is going to swamp any extra clock
 cycle, as will a virtual lookup and dereference.


 Assume destroy() is a more trivial function then. The point is that if you put
 more than two branches in a 64-byte cacheline on that processor, things get
 significantly slower and the loop iteration itself becomes a hotspot.


Putting a destructor call in a tight loop is probably not a good pattern. Hoist 
it out of the loop.



 Being D though. Destructors can be contracted, yeah? Because the way we operate
 is that we compile out all those validation checks for a retail release and
 assume everything works. Checking for the validity of the pointer in an in
block
 would be perfect for that.


Sounds good!
Sep 08 2016
prev sibling parent Yuxuan Shui <yshuiv7 gmail.com> writes:
On Thursday, 8 September 2016 at 11:18:12 UTC, Walter Bright 
wrote:

 http://www.agner.org/optimize/microarchitecture.pdf section 
 3.13 has a bit more
 info on the branch predictor. Desktop Intel CPUs tend to hide 
 performance
 problems like this thanks to their far-higher-quality branch 
 predictors. Both
 chips gain benefits from sorting to how you expect the branch 
 predictor to work,
 but there's a lot of code in a game codebase that isn't that 
 low level.


 There's another way. Just don't worry about it being null, if 
 you're going to ensure it is initialized regardless. It'll seg 
 fault if it is dereferenced but not initialized.


Doesn't this seem to contradict the ".init is always a valid 
state" ideal?
Sep 08 2016
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Wednesday, 7 September 2016 at 11:42:40 UTC, Dicebot wrote:

 If it is so, I'd call it a major extern(c++) bug.


It is not surprising. Making it work require flow analysis much 
more powerful than what DMD has now.
Sep 07 2016
prev sibling parent Dicebot <public dicebot.lv> writes:
 static opCall() seems to be the only way to do this then. I can 
 autogenerate it for any C++ bound class. But it's inadequate. 
 It leaves room for user error when instantiating any C++ object 
 in D. It's also another thing that C++ programmers need to be 
 thoroughly educated about as Type() in C++11 calls the zero 
 initializer, but in D it's effectively the opposite semantics.


Please try out everything possible instead of resorting to static 
opCall. It is a terrible hack, both confusing and inconsistent 
(won't apply if you allocate struct on heap) that got popular 
simply because some devs were unhappy with language decision to 
prohibit default ctors. It never makes sense to use it, literally 
never.
Sep 07 2016
prev sibling next sibling parent reply Dicebot <public dicebot.lv> writes:
Is using svope class out of the question?
Sep 07 2016
parent reply Ethan Watson <gooberman gmail.com> writes:
On Wednesday, 7 September 2016 at 11:19:46 UTC, Dicebot wrote:

 Is using svope class out of the question?


This might actually get me what I want. I'll have to play around 
with it and see.
Sep 07 2016
parent reply Ethan Watson <gooberman gmail.com> writes:
On Wednesday, 7 September 2016 at 12:09:21 UTC, Ethan Watson 
wrote:

 This might actually get me what I want. I'll have to play 
 around with it and see.


"Scope classes have been recommended for deprecation."

"A scope class reference can only appear as a function local 
variable."

So that's two nopes right there.
Sep 07 2016
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 08/09/2016 12:10 AM, Ethan Watson wrote:

 On Wednesday, 7 September 2016 at 12:09:21 UTC, Ethan Watson wrote:

 This might actually get me what I want. I'll have to play around with
 it and see.


 "Scope classes have been recommended for deprecation."

 "A scope class reference can only appear as a function local variable."

 So that's two nopes right there.
Sep 07 2016
parent reply Ethan Watson <gooberman gmail.com> writes:
On Wednesday, 7 September 2016 at 12:14:46 UTC, rikki cattermole 
wrote:




This is the kind of hackaround I'd need to do if it were a 
class... And it would require more hacking around than the 
standard library supports. And it's a spiraling-out-of-control 
hack, which would effectively mean every C++ matching class will 
need to define a class and then an alias with the scoped type, 
and then that means the pattern matching I've been creating for 
function linkups won't work any more...

static opCall() and a static alloc function for allocating on the 
heap are still looking like the simplest options here.
Sep 07 2016
parent reply Dicebot <public dicebot.lv> writes:
On Wednesday, 7 September 2016 at 12:28:41 UTC, Ethan Watson 
wrote:

 On Wednesday, 7 September 2016 at 12:14:46 UTC, rikki 
 cattermole wrote:




 This is the kind of hackaround I'd need to do if it were a 
 class... And it would require more hacking around than the 
 standard library supports. And it's a spiraling-out-of-control 
 hack, which would effectively mean every C++ matching class 
 will need to define a class and then an alias with the scoped 
 type, and then that means the pattern matching I've been 
 creating for function linkups won't work any more...

 static opCall() and a static alloc function for allocating on 
 the heap are still looking like the simplest options here.


They aren't. It won't also be used if you declare new instance on 
stack via `S s`. Staic opCall is 100% identical to any other 
static factory method in its semantics (i.e. static S create()).

What I am trying to say is thta prohibiting struct default 
constructors is an intentional language design decision in D, 
with a lot of consequences. There is literally no chance it can 
be tricked.

Instead, it would be much more constructive (pun unintended) to 
focus on language changes to extern(c++) class bindings to make 
them suitable for the task - those won't affect anyone but C++ 
interop users.
Sep 08 2016
parent reply Ethan Watson <gooberman gmail.com> writes:
On Thursday, 8 September 2016 at 09:33:01 UTC, Dicebot wrote:

 Instead, it would be much more constructive (pun unintended) to 
 focus on language changes to extern(c++) class bindings to make 
 them suitable for the task - those won't affect anyone but C++ 
 interop users.


I agree in principle, but it doesn't help me right now. It's 
holding up my work, which means it's costing someone money. 
Workarounds will have to suffice until the language can be 
updated.
Sep 08 2016
parent reply Dicebot <public dicebot.lv> writes:
On Thursday, 8 September 2016 at 09:54:51 UTC, Ethan Watson wrote:

 On Thursday, 8 September 2016 at 09:33:01 UTC, Dicebot wrote:

 Instead, it would be much more constructive (pun unintended) 
 to focus on language changes to extern(c++) class bindings to 
 make them suitable for the task - those won't affect anyone 
 but C++ interop users.


 I agree in principle, but it doesn't help me right now. It's 
 holding up my work, which means it's costing someone money. 
 Workarounds will have to suffice until the language can be 
 updated.


As a workaround I sincerely believe explicit 'create' (with 
forged mangling if needed) is better. It provides exactly the 
same functionality without tricking the developet into expecting 
more by confusion of the syntax similarity.
Sep 08 2016
parent Ethan Watson <gooberman gmail.com> writes:
On Thursday, 8 September 2016 at 10:36:22 UTC, Dicebot wrote:

 As a workaround I sincerely believe explicit 'create' (with 
 forged mangling if needed) is better. It provides exactly the 
 same functionality without tricking the developet into 
 expecting more by confusion of the syntax similarity.


If I was to enforce a programming standard with static opCall(). 
The code for instantiating the Mutex example would look like:

Mutex foo = Mutex();

Later on down the track, behind the scenes when default 
constructors work for C++ types I remove the static opCall() 
implementation and replace it with default constructors. Right 
now, Mutex() without static opCall() just gives me the .init. 
With the static opCall(), I can construct it. With a default 
constructor?

I suppose that'd depend on future decisions that haven't been 
made yet. In C++ Mutex() is meant to invoke the zero initialiser. 
It's effectively the opposite in D when using static opCall(). 
Which one would be the correct way to default construct a class? 
We'll find out I suppose.

Either way, assuming the default constructor will be called 
regardless of if it's foo = Mutex; or foo = Mutex();, using 
static opCall() will cut down on future maintenance work.

We're going to disagree on this one, basically. I'm designing 
this system for people who don't want to have to remember to call 
fancy create functions.
Sep 08 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/7/2016 5:10 AM, Ethan Watson wrote:

 "Scope classes have been recommended for deprecation."


That decision will be revisited.
Sep 07 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
The reasons D structs don't have a default constructor:

1. So S.init is a valid initializer
2. So all instances of S can be guaranteed to contain a valid instance
3. So default initialization is guaranteed to succeed
4. So any struct constructor starts with a valid state
5. In my not-so-humble opinion, construction should never fail and all 
constructors should be nothrow, but I understand that is a minority viewpoint

Assumptions 1..4 are pervasive in D and the logic of the compiler.
Sep 07 2016
next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Wednesday, 7 September 2016 at 21:05:32 UTC, Walter Bright 
wrote:

 The reasons D structs don't have a default constructor:

 1. So S.init is a valid initializer
 2. So all instances of S can be guaranteed to contain a valid 
 instance
 3. So default initialization is guaranteed to succeed
 4. So any struct constructor starts with a valid state
 5. In my not-so-humble opinion, construction should never fail 
 and all constructors should be nothrow, but I understand that 
 is a minority viewpoint

 Assumptions 1..4 are pervasive in D and the logic of the 
 compiler.


What, precisely, does "valid" mean in the above?
Sep 07 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/7/2016 3:24 PM, John Colvin wrote:

 What, precisely, does "valid" mean in the above?


S is initialized to a valid state, meaning the fields are not filled with 
garbage, and are in a state expected by the member functions.

But if there's a default constructor,

     S s = S.init;
     S s;

which is correct?
Sep 07 2016
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 7 September 2016 at 22:31:17 UTC, Walter Bright 
wrote:

 On 9/7/2016 3:24 PM, John Colvin wrote:

 What, precisely, does "valid" mean in the above?


 S is initialized to a valid state, meaning the fields are not 
 filled with garbage, and are in a state expected by the member 
 functions.

 But if there's a default constructor,

     S s = S.init;
     S s;

 which is correct?


Potentially naive question, but is there any reason why, if a 
default constructor exists, S.init shouldn't just be the same as 
the result of calling the default constructor?
Sep 07 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/7/2016 4:17 PM, Joseph Rushton Wakeling wrote:

 Potentially naive question, but is there any reason why, if a default
 constructor exists, S.init shouldn't just be the same as the result of calling
 the default constructor?


So every instance of Mutex shares the same mutex?
Sep 07 2016
prev sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Wednesday, 7 September 2016 at 22:31:17 UTC, Walter Bright 
wrote:

 S is initialized to a valid state, meaning the fields are not 
 filled with garbage, and are in a state expected by the member 
 functions.


We can write member functions that require a state other than the 
initial state. I don't see what's special about this init state.

There are arguably also types that don't have any valid init 
state, I think mutexes fall in to this category.


 But if there's a default constructor,

     S s = S.init;
     S s;

 which is correct?


They are different, one has the initial state (pre-construction) 
and the other has the state post-default-construction.

I think it's too late for this stuff now for D anyway. There are 
workarounds that make life acceptable without default 
constructors, I can't see how we could add them without getting 
into a real mess.
Sep 08 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/8/16 12:06 PM, John Colvin wrote:

 I think it's too late for this stuff now for D anyway. There are
 workarounds that make life acceptable without default constructors, I
 can't see how we could add them without getting into a real mess.


A good point. I should mention, however, that the lack of a default 
constructor made at least one reference counting (in fact reference 
linking IIRC) impossible in D.

One thing we could look at is allow only CTFEable default constructors.


Andrei
Sep 08 2016
parent Ethan Watson <gooberman gmail.com> writes:
On Thursday, 8 September 2016 at 12:32:44 UTC, Andrei 
Alexandrescu wrote:

 One thing we could look at is allow only CTFEable default 
 constructors.


Wouldn't work in my case where I need to call a dynamically 
imported extern( C++ ) function to correctly construct an object.
Sep 08 2016
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 8 September 2016 at 10:06:16 UTC, John Colvin wrote:

 I think it's too late for this stuff now for D anyway. There 
 are workarounds that make life acceptable without default 
 constructors, I can't see how we could add them without getting 
 into a real mess.


The kind of flow analysis required to ensure something is 
constructed before use is roughly the same as the one required to 
do non nullable reference and/or lifetime analysis (in fact that 
last one require a superset of what is required for the first 2).

If adding it simply for the default constructor is probably 
hitting the wrong tradeof in term of bang for the buck, adding to 
get the 3 above seems like it would be worth it.

As s side note, this is why I think issue based language 
discussion is not a good way to proceed, as we are missing the 
kind of bigger picture insight as this one. When going issue 
based we are like, "Do we want to add this for default ctors ? 
No, it doesn't pay for itself." Then later on "Do we want to add 
this for non nullable references ? No, it doesn't pay for 
itself." and so, whereas the rational question would be "Do we 
want to add this for default ctors, non nullable references and 
lifetime analysis ? Well now, probably yes." Thing is, 
considering each question individually wouldn't yield the best 
choice.
Sep 08 2016
prev sibling parent reply Ethan Watson <gooberman gmail.com> writes:
On Wednesday, 7 September 2016 at 21:05:32 UTC, Walter Bright 
wrote:

 5. In my not-so-humble opinion, construction should never fail 
 and all constructors should be nothrow, but I understand that 
 is a minority viewpoint


100% agree there. I can't think of any class in our C++ codebase 
that fails construction, and it's a pretty common rule in the 
games industry to not assert/throw exceptions during construction.

Of course, with Binderoo being open sourced, I can't guarantee 
any of my end users will be just as disciplined.
Sep 08 2016
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Thu, 08 Sep 2016 07:52:58 +0000
schrieb Ethan Watson <gooberman gmail.com>:


 On Wednesday, 7 September 2016 at 21:05:32 UTC, Walter Bright 
 wrote:

 5. In my not-so-humble opinion, construction should never fail 
 and all constructors should be nothrow, but I understand that 
 is a minority viewpoint  


 
 100% agree there. I can't think of any class in our C++ codebase 
 that fails construction, and it's a pretty common rule in the 
 games industry to not assert/throw exceptions during construction.
 
 Of course, with Binderoo being open sourced, I can't guarantee 
 any of my end users will be just as disciplined.


So when you have an object that reads state from a file, you
first construct it and then call a member function
"loadFromFile()" that may throw? For argument's sake let's
take a *.bmp class. That one would not have a constructor with
a filename? Or do you have such constructors and I/O
exceptions are just logged and swallowed?

I'd like to understand how it would behave, since obviously
both you and Walter have written large software products and
personally I prefer to attempt construction and rollback on
errors until I'm back in the state where the user initiated the
action. What is the benefit? Well, one benefit is that you are
forced to write error concealment code and make the best out
of partially broken input. Others?

Out-of-memory and invalid arguments would be other examples of
exceptions, but I guess the Errors thrown from asserts don't
count as exceptions in the regular sense, since by definition
they are non-recoverable.

-- 
Marco
Sep 09 2016
parent reply Ethan Watson <gooberman gmail.com> writes:
On Friday, 9 September 2016 at 12:16:00 UTC, Marco Leise wrote:

 So when you have an object that reads state from a file, you 
 first construct it and then call a member function 
 "loadFromFile()" that may throw? For argument's sake let's take 
 a *.bmp class. That one would not have a constructor with a 
 filename? Or do you have such constructors and I/O exceptions 
 are just logged and swallowed?


Remedy's Northlight engine is a streaming engine (that actually 
supports an open world, the legacy of Alan Wake development lives 
on). Thus, you need to follow some important rules. These are 
also pretty standard rules for game engines in general.

First and foremost, resources are processed offline to match the 
ideal binary format for the target platform. The industry has 
been using DXT textures for over a decade now, and they've been 
supported on consoles. The overwhelming majority of textures are 
thus baked in to such a format. Whichever format is chosen, on 
disk the file will essentially represent the resource's final 
layout in memory.

Second, file loading. You can't just go loading files any old 
time you want in a streaming-based, or even just straight up 
multithreaded, engine if you expect to keep within performance 
targets and not lock up every thread you've created. They need 
scheduling. Thus, resource creation needs to go through several 
steps:

* Something requests a resource, goes to sleep
* File loader schedules appropriately, notifies on load complete
* Object gets resource load notification, does work to hook it up 
to whatever API needs it

Anything that can assert/throw an exception is not in a 
constructor in these phases. And as mentioned elsewhere, asserts 
and exceptions are defined out for a retail build. If there's a 
problem with the data, we expect to find it in development and 
ship a product that doesn't require constant validation.
Sep 09 2016
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Fri, 09 Sep 2016 14:46:31 +0000
schrieb Ethan Watson <gooberman gmail.com>:


 [=E2=80=A6]
=20
 First and foremost, resources are processed offline to match the=20
 ideal binary format for the target platform. The industry has=20
 been using DXT textures for over a decade now, and they've been=20
 supported on consoles. The overwhelming majority of textures are=20
 thus baked in to such a format. Whichever format is chosen, on=20
 disk the file will essentially represent the resource's final=20
 layout in memory.


I understand that.


 Second, file loading. You can't just go loading files any old=20
 time you want in a streaming-based, or even just straight up=20
 multithreaded, engine if you expect to keep within performance=20
 targets and not lock up every thread you've created. They need=20
 scheduling. Thus, resource creation needs to go through several=20
 steps:
=20
 * Something requests a resource, goes to sleep
 * File loader schedules appropriately, notifies on load complete
 * Object gets resource load notification, does work to hook it up=20
   to whatever API needs it


...and the objects are probably created ahead of time in a
pool, to avoid allocations? In such a scheme it is only
natural to not have I/O in ctors. But what about the parts of
the code that handle the game initialization before streaming
starts? Is there no

  config =3D new GameConfig("settings.ini");

or

  db =3D new AssetDatabase("menu.pkg");

that perform I/O during construction and potentially display
an exception error messages ?

--=20
Marco
Sep 09 2016
parent reply Ethan Watson <gooberman gmail.com> writes:
On Saturday, 10 September 2016 at 05:56:55 UTC, Marco Leise wrote:

 But what about the parts of the code that handle the game 
 initialization before streaming starts? Is there no

   config = new GameConfig("settings.ini");

 or

   db = new AssetDatabase("menu.pkg");

 that perform I/O during construction and potentially display an 
 exception error messages ?


Everything streams. No exceptions. The only file operations 
provided to game code at run time are asynchronous operations.

And if you exploit that correctly, this is one of those things 
that can increase boot times, actually. Create your 
config/database/whatever objects, request files, instead of 
initialising them all in order and slowing down because of 
synchronous IO they all go to sleep and streaming system can 
serve files up as quick as it gets them from disk.
Sep 10 2016
next sibling parent Ethan Watson <gooberman gmail.com> writes:
On Saturday, 10 September 2016 at 08:22:59 UTC, Ethan Watson 
wrote:

 And if you exploit that correctly, this is one of those things 
 that can increase boot times, actually.


*decrease boot times. Take that, edit button.
Sep 10 2016
prev sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 10/09/2016 8:22 PM, Ethan Watson wrote:

 On Saturday, 10 September 2016 at 05:56:55 UTC, Marco Leise wrote:

 But what about the parts of the code that handle the game
 initialization before streaming starts? Is there no

   config = new GameConfig("settings.ini");

 or

   db = new AssetDatabase("menu.pkg");

 that perform I/O during construction and potentially display an
 exception error messages ?


 Everything streams. No exceptions. The only file operations provided to
 game code at run time are asynchronous operations.

 And if you exploit that correctly, this is one of those things that can
 increase boot times, actually. Create your config/database/whatever
 objects, request files, instead of initialising them all in order and
 slowing down because of synchronous IO they all go to sleep and
 streaming system can serve files up as quick as it gets them from disk.


Is there a good example library for this that does not involve a full 
blown (game)framework?
Sep 10 2016
parent Ethan Watson <gooberman gmail.com> writes:
On Saturday, 10 September 2016 at 08:26:44 UTC, rikki cattermole 
wrote:

 Is there a good example library for this that does not involve 
 a full blown (game)framework?


Not that I'm aware of.
Sep 10 2016
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/6/2016 6:44 AM, Ethan Watson wrote:

 Suggestions?


Provide a "default" constructor that has a dummy (i.e. unused) parameter.

     struct _Unused { }
     alias Unused = immutable(_Unused);
     Unused unused;

     ...

     struct S {
         this(Unused) { ... }
         ...
     }

     ...

     S s = S(unused);

Auto-generated such a constructor when S is used as a field.
Sep 07 2016
prev sibling next sibling parent Jerry <Kickupx gmail.com> writes:
Just as a random jumpin.
Couldn't this be worked around with something like this:

struct Foo {
    disable this();
   private this(int x) { /* init */ }
}

auto foo() {
   return Foo(0);
}

You basicly just hides the weird int x constructor and still 
disallows default construction. I guess would not be desirable 
with mutexes being locked and unlocked but for most else it 
should be good enough I guess.
Sep 07 2016
prev sibling parent John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 6 September 2016 at 13:44:37 UTC, Ethan Watson wrote:

 So now I'm in a bind. This is one struct I need to construct 
 uniquely every time. And I also need to keep the usability up 
 to not require calling some other function since this is 
 matching a C++ class's functionality, including its ability to 
 instantiate anywhere.

 Suggestions?


I know you don't like the explicit function-calling way, but I 
think it is actually quite good, if you consider what the 
compiler does and doesn't allow

e.g.

struct S
{
	int a;
	 disable this();
	private this(int a) // just for ease of use in `create`
	{
		this.a = a;
	}
	static create() // could be free function instead
	{
		auto x = runtimeGetMySuperImportantUniqueValue();
		return(S(x));
	}
}

struct S1
{
	S s; // fine despite  disable this()

	// blah blah other state
	
	this(int argForOtherState)
	{
		// if you comment this out, it doesn't compile,
		// the compiler enforces initialisation for s
		// because of  disable this()
		s = S.create();
		
		// blah blah other construction
	}
}

void main()
{
//	S1 s1; // doesn't compile, would bypass initialisation of s1.s
	S1 s1 = S1(-1); // OK
}
Sep 08 2016