www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - order of static constructor execution

reply Walter Bright <newshound1 digitalmars.com> writes:
Currently, it is performed as a strictly "depth-first" traversal of the 
graph defined by the import statements. As we've been discussing here, 
this works great until one has circular imports, meaning the depth-first 
graph has a loop in it.

The current behavior on detecting a loop is to quit with an error message.

The problems are:

1. The cycles are not easily gotten rid of when they are the result of 
template mixins.

2. Trying to analyze the static constructors to see what the 
dependencies actually are is fraught with unsolvable problems.


So, I propose the following:

1. Attempt the depth-first traversal of the static constructors.

2. If a loop is detected, rather than issuing an error message, simply 
arbitrarily pick one order and continue constructing.


The mitigating rationale is that modules that import each other are 
presumably written by the same person or team, and so that person is in 
the best place to explicitly control dependencies themselves.


I'm not happy with this solution, but it seems to be the best compromise 
I can come up with.

What do you think?
Mar 11 2010
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:

 1. Attempt the depth-first traversal of the static constructors.
 2. If a loop is detected, rather than issuing an error message, simply 
 arbitrarily pick one order and continue constructing.


Some alternative possibilities:
- Issue a warning too?
- Or allow that only if the code is compiled with "-allowloops"?
- Or accept one specific loop only if the programmer somewhere states
explicitly that this loop is OK, and issue an error otherwise?

Bye,
bearophile
Mar 11 2010
prev sibling next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 03/11/2010 08:42 PM, Walter Bright wrote:

 2. Trying to analyze the static constructors to see what the
 dependencies actually are is fraught with unsolvable problems.


could you elucidate on this point? Is it definitely impossible to get a 
hold of a complete dag, even if at runtime?
Mar 11 2010
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Ellery Newcomer wrote:

 On 03/11/2010 08:42 PM, Walter Bright wrote:

 2. Trying to analyze the static constructors to see what the
 dependencies actually are is fraught with unsolvable problems.


 
 could you elucidate on this point? Is it definitely impossible to get a 
 hold of a complete dag, even if at runtime?



Suppose the static constructor calls foo(), the implementation of which 
is hidden to the compiler.
Mar 11 2010
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 03/11/2010 10:15 PM, Walter Bright wrote:

 Ellery Newcomer wrote:

 On 03/11/2010 08:42 PM, Walter Bright wrote:

 2. Trying to analyze the static constructors to see what the
 dependencies actually are is fraught with unsolvable problems.


 could you elucidate on this point? Is it definitely impossible to get
 a hold of a complete dag, even if at runtime?



 Suppose the static constructor calls foo(), the implementation of which
 is hidden to the compiler.


Oh, you mean the true dependencies. I was just going to say that 
printing out the cycle that causes bombout would be better than nothing.
Mar 11 2010
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 03/11/2010 08:42 PM, Walter Bright wrote:

 Currently, it is performed as a strictly "depth-first" traversal of the
 graph defined by the import statements. As we've been discussing here,
 this works great until one has circular imports, meaning the depth-first
 graph has a loop in it.

 The current behavior on detecting a loop is to quit with an error message.

 The problems are:

 1. The cycles are not easily gotten rid of when they are the result of
 template mixins.

 2. Trying to analyze the static constructors to see what the
 dependencies actually are is fraught with unsolvable problems.


 So, I propose the following:

 1. Attempt the depth-first traversal of the static constructors.

 2. If a loop is detected, rather than issuing an error message, simply
 arbitrarily pick one order and continue constructing.


 The mitigating rationale is that modules that import each other are
 presumably written by the same person or team, and so that person is in
 the best place to explicitly control dependencies themselves.


 I'm not happy with this solution, but it seems to be the best compromise
 I can come up with.

 What do you think?


It looks like a step backwards to me.

Andrei
Mar 11 2010
parent Max Samukha <spambox d-coding.com> writes:
Andrei Alexandrescu wrote:


 
 It looks like a step backwards to me.
 
 Andrei


What solution do you propose for problem 1?
Mar 12 2010
prev sibling next sibling parent reply BCS <none anon.com> writes:
Hello Walter,


 2. If a loop is detected, rather than issuing an error message, simply
 arbitrarily pick one order and continue constructing.


How about a way to explicitly cut edges in the graph (tagging imports with 
"pragma(nodep)" or " nodep" for instance)? That has the same end effect but 
for only a little more work, removes any non-determinism and allows for easy 
control of how things are resolved.

-- 
... <IXOYE><
Mar 11 2010
next sibling parent reply Don <nospam nospam.com> writes:
BCS wrote:

 Hello Walter,
 

 2. If a loop is detected, rather than issuing an error message, simply
 arbitrarily pick one order and continue constructing.


 
 How about a way to explicitly cut edges in the graph (tagging imports 
 with "pragma(nodep)" or " nodep" for instance)? That has the same end 
 effect but for only a little more work, removes any non-determinism and 
 allows for easy control of how things are resolved.


I agree. Seems to me that if a circular import exists, it's really 
something that the programmer needs to think about, and so it's 
reasonable for it to be explicit.

A really harsh solution would be:

pragma(nodependency, somepackage.somemodule);

The compiler could check that somepackage.somemodule actually defines a 
static constructor. And it could even check that a circular import 
situation actually exists. This would force the pragma to be maintained 
correctly.
Mar 12 2010
parent reply Fawzi Mohamed <fawzi gmx.ch> writes:
On 12-mar-10, at 09:59, Don wrote:


 BCS wrote:

 Hello Walter,

 2. If a loop is detected, rather than issuing an error message,  
 simply
 arbitrarily pick one order and continue constructing.


 How about a way to explicitly cut edges in the graph (tagging  
 imports with "pragma(nodep)" or " nodep" for instance)? That has  
 the same end effect but for only a little more work, removes any  
 non-determinism and allows for easy control of how things are  
 resolved.


 I agree. Seems to me that if a circular import exists, it's really  
 something that the programmer needs to think about, and so it's  
 reasonable for it to be explicit.

 A really harsh solution would be:

 pragma(nodependency, somepackage.somemodule);

 The compiler could check that somepackage.somemodule actually  
 defines a static constructor. And it could even check that a  
 circular import situation actually exists. This would force the  
 pragma to be maintained correctly.


I think that the main problem comes from static initializers that are  
mixed in and by themselves have no circular dependency. So annotations  
have to be at the static initializer level, not at the module level.

One way to solve this would be to add
 dependOnly(module1,...) static this(){ }

then each module would have 2 dependencies:

* indirectDeps
   imported modules + all dependOnly modules

* directDeps
   look at all static initializers in the module:
   - plain static this() -> add all imported modules as dependency
   - annotated static this -> add all dependOnly dependencies

* allDeps
   the list of module that have to be initialized before that module  
and is build from the previous ones as:
   direct deps + all indirect deps of those modules

more explicitly

allDeps(a){
   deps=[]
   foreach(m in a.directDeps){
     if (m in deps) continue;
     deps~=m;
     addDeps2(m,deps);
   }
   return deps;
}

addDeps2(a, ref deps){
   if (a in deps) return;
   foreach(m in a.indirectDeps){
     if (m in deps) continue;
     deps~=m;
     addDeps2(m,deps);
   }
}

then you can sort the modules that have static initializers (and only  
those) using:

compare(a,b){
   if (a in b.allDeps){
    if (b in a.allDeps){
      error("non comparable, circular dep between ",a,b);
    }
    return 1;
   }
   if (b in a.allDeps){
     return -1;
   }
   return 0;
}

this is more work, but would be perfectly defined.

conflicting ordering would still be disallowed, but circularly  
dependent modules can have initializers if all their initializers  
depend only on modules that are not circularly dependent.

Fawzi
Mar 12 2010
parent reply Don <nospam nospam.com> writes:
Fawzi Mohamed wrote:

 
 On 12-mar-10, at 09:59, Don wrote:
 

 BCS wrote:

 Hello Walter,

 2. If a loop is detected, rather than issuing an error message, simply
 arbitrarily pick one order and continue constructing.


 How about a way to explicitly cut edges in the graph (tagging imports 
 with "pragma(nodep)" or " nodep" for instance)? That has the same end 
 effect but for only a little more work, removes any non-determinism 
 and allows for easy control of how things are resolved.


 I agree. Seems to me that if a circular import exists, it's really 
 something that the programmer needs to think about, and so it's 
 reasonable for it to be explicit.

 A really harsh solution would be:

 pragma(nodependency, somepackage.somemodule);

 The compiler could check that somepackage.somemodule actually defines 
 a static constructor. And it could even check that a circular import 
 situation actually exists. This would force the pragma to be 
 maintained correctly.


 
 I think that the main problem comes from static initializers that are 
 mixed in and by themselves have no circular dependency. 


If that's true, then perhaps a more restricted solution is better.
Eg, use Walter's proposal ONLY if the static initializer is mixed in.
Mar 12 2010
next sibling parent reply BCS <none anon.com> writes:
Hello Don,


 If that's true, then perhaps a more restricted solution is better. Eg,
 use Walter's proposal ONLY if the static initializer is mixed in.


Random thought: Aside from making a new corner case, why not make static 
constructors in mixins only depend on imports from where they are defined 
rather than where they are used?
 
-- 
... <IXOYE><
Mar 12 2010
parent reply Walter Bright <newshound1 digitalmars.com> writes:
BCS wrote:

 Random thought: Aside from making a new corner case, why not make static 
 constructors in mixins only depend on imports from where they are 
 defined rather than where they are used?



Template mixins are intended to behave like macros - they are 
instantiated in the context of where they are used, not where they are 
defined.

Regular templates are instantiated in the context of where they are 
defined, not used.
Mar 12 2010
parent reply Max Samukha <spambox d-coding.com> writes:
On 12.03.2010 23:03, Walter Bright wrote:

 BCS wrote:

 Random thought: Aside from making a new corner case, why not make
 static constructors in mixins only depend on imports from where they
 are defined rather than where they are used?



 Template mixins are intended to behave like macros - they are
 instantiated in the context of where they are used, not where they are
 defined.

 Regular templates are instantiated in the context of where they are
 defined, not used.


But then:

module a;

static this() {} // 1

template Foo(int i)
{
     static this() // 2
     {
     }
}
----

module b;
import a;

static this() // 3
{
}

alias Foo!(1) foo;
----

Currently the construction order is 1, 3, 2 instead of the expected 1, 
2, 3. In other words, the constructor in the template instance (2) 
should be combined with constructor 1, not 3.

If static constructors in templates ran during the initialization of the 
module where they are defined, the problem with mixed-in code could be 
solved like this:

module a;

template StaticCtor(alias ctor)
{
     static this()
     {
         ctor();
     }
}

mixin template Foo(int i)
{
     void construct() { writeln("Constructing for ", i); }
     alias StaticCtor!(construct) ctor;
}

----
module b;

import a;
import c;

mixin Foo!(1);

----
module c;

import a;
import b;

mixin Foo!(2);

So, though Foo is mixed into modules 'b' and 'c', its instances would be 
constructed in module 'a' via the regular StaticCtor template.

Still a hack but the code at least makes it quite explicit for 
maintainers to stay alert.
Mar 12 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-03-12 19:10:10 -0500, Max Samukha <spambox d-coding.com> said:


 If static constructors in templates ran during the initialization of 
 the module where they are defined, the problem with mixed-in code could 
 be solved like this:


But templates can take function and types as argument. If the 'static 
this' of a template calls something through one of its argument, the 
module this function or type resides in needs to be initialized first.

So you can't always call a template's static this at the same time as 
the module the template is defined in.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Mar 12 2010
parent Max Samukha <spambox d-coding.com> writes:
On 13.03.2010 2:27, Michel Fortin wrote:

 On 2010-03-12 19:10:10 -0500, Max Samukha <spambox d-coding.com> said:


 If static constructors in templates ran during the initialization of
 the module where they are defined, the problem with mixed-in code
 could be solved like this:


 But templates can take function and types as argument. If the 'static
 this' of a template calls something through one of its argument, the
 module this function or type resides in needs to be initialized first.

 So you can't always call a template's static this at the same time as
 the module the template is defined in.


Yes that is the point and purpose.

The advantage is that the programmer is in control of when the static 
constructors in templated code are run - if you want the constructor to 
run in the context of the imported module use a regular template, 
otherwise, mixin.

I'd definitely preferred it over the current semantics. Others may disagree.
Mar 13 2010
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Don wrote:

 If that's true, then perhaps a more restricted solution is better.
 Eg, use Walter's proposal ONLY if the static initializer is mixed in.


I should note that all static constructors in a module are combined into 
one function. So, the mixin static initializers are combined with any 
other static initializers.
Mar 12 2010
parent reply BCS <none anon.com> writes:
Hello Walter,


 Don wrote:
 

 If that's true, then perhaps a more restricted solution is better.
 Eg, use Walter's proposal ONLY if the static initializer is mixed in.
 


 I should note that all static constructors in a module are combined
 into one function. So, the mixin static initializers are combined with
 any other static initializers.
 


Does it have to run that way or could you lump them into several pieces and 
have them call each other as needed?

-- 
... <IXOYE><
Mar 12 2010
parent Walter Bright <newshound1 digitalmars.com> writes:
BCS wrote:

 Hello Walter,
 

 Don wrote:


 If that's true, then perhaps a more restricted solution is better.
 Eg, use Walter's proposal ONLY if the static initializer is mixed in.


 I should note that all static constructors in a module are combined
 into one function. So, the mixin static initializers are combined with
 any other static initializers.


 
 Does it have to run that way or could you lump them into several pieces 
 and have them call each other as needed?
 


Right now, they are defined to run in lexical order. Regrouping them in 
arbitrary orders will mess up intra-module ordering dependencies.
Mar 12 2010
prev sibling parent Leandro Lucarella <llucax gmail.com> writes:
BCS, el 12 de marzo a las 04:43 me escribiste:

 Hello Walter,
 

2. If a loop is detected, rather than issuing an error message, simply
arbitrarily pick one order and continue constructing.


 
 How about a way to explicitly cut edges in the graph (tagging
 imports with "pragma(nodep)" or " nodep" for instance)? That has the
 same end effect but for only a little more work, removes any
 non-determinism and allows for easy control of how things are
 resolved.


Yei! Weak imports (like weak references =P)

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
Mar 14 2010
prev sibling next sibling parent "Bob Jones" <me not.com> writes:
"Walter Bright" <newshound1 digitalmars.com> wrote in message 
news:hnc9n2$2tkj$1 digitalmars.com...

 Currently, it is performed as a strictly "depth-first" traversal of the 
 graph defined by the import statements. As we've been discussing here, 
 this works great until one has circular imports, meaning the depth-first 
 graph has a loop in it.

 The current behavior on detecting a loop is to quit with an error message.

 The problems are:

 1. The cycles are not easily gotten rid of when they are the result of 
 template mixins.

 2. Trying to analyze the static constructors to see what the dependencies 
 actually are is fraught with unsolvable problems.


Surely there must be some low hanging fruit that can be exploited? Say a 
module has no static constructors then cant that be considered a break in 
the dependency cycle?
Mar 11 2010
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 3/12/10 03:42, Walter Bright wrote:

 Currently, it is performed as a strictly "depth-first" traversal of the
 graph defined by the import statements. As we've been discussing here,
 this works great until one has circular imports, meaning the depth-first
 graph has a loop in it.

 The current behavior on detecting a loop is to quit with an error message.

 The problems are:

 1. The cycles are not easily gotten rid of when they are the result of
 template mixins.

 2. Trying to analyze the static constructors to see what the
 dependencies actually are is fraught with unsolvable problems.


 So, I propose the following:

 1. Attempt the depth-first traversal of the static constructors.

 2. If a loop is detected, rather than issuing an error message, simply
 arbitrarily pick one order and continue constructing.


 The mitigating rationale is that modules that import each other are
 presumably written by the same person or team, and so that person is in
 the best place to explicitly control dependencies themselves.


 I'm not happy with this solution, but it seems to be the best compromise
 I can come up with.

 What do you think?


Yes please. As other have suggested a warning might be a good idea.
Mar 12 2010
prev sibling next sibling parent reply Regan Heath <regan netmail.co.nz> writes:
Walter Bright Wrote:

 I'm not happy with this solution, but it seems to be the best compromise 
 I can come up with.
 
 What do you think?


The first thought that occurred to me was "how this was handled in other


does not have modules, but it seems to me that a class with only static members
and a static constructor is roughly analogous.

I'm not sure how similar a test case this really is, to be honest it's been a
while since I wrote some D and I haven't used mixin and static construction in
depth but perhaps someone can take my example and improve it.  I can at least

about how the D implementation works.



http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=107373

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace StaticConstruct
{
	class Program
	{
		static void Main(string[] args)
		{
			System.Console.Out.WriteLine("A = {0}", A.a);
			System.Console.Out.WriteLine("B = {0}", B.b);
		}
	}

	class A
	{
		public static int a;

		static A()
		{
			a = 1;
		}
	}

	class B
	{
		public static int b;

		static B()
		{
			b = 2;
		}
	}
}


do with the above.  It's actually pretty simple, when it executes the first
WriteLine it evaluates A.a, when it does that it calls the static constructor
for A.  Likewise for B.b the same thing occurs.  So, it's effectively
performing lazy construction/evaluation.  Does D do this, or does D attempt to
construct _all_ modules on program start?  Could this be the solution, to make
module construction lazy?

Lets try a few more complex examples...

1)
Change "public static int a;" to "public static int a = B.b;" (create
dependency on B)
Change "public static int b;" to "public static int b = A.a;" (create
dependency on A)
This creates a circular dependency.

What happens:
01.When it executes the A.a WriteLine it needs to evaluate A.a, so it needs to
initialize A.
02.To initialise A it must initialize the 'globals' i.e. "public static int a =
B.b;"
03.This causes it to evaluate B.b, causing initialization of B
04.To initialise B it must initialize the 'globals' i.e. public static int b =
A.a;".
05.At this point it assigns 0 to b, as A.a is currently 0 and as it has already
entered initialization of A (preventing infinite loop)
06.It then calls the static constructor for B, assigning b = 2;
07.It then completes "public static int a = B.b;" assigning a = 2;
08.It then calls the static constructor for A, assigning a = 1;
09.It returns control to the WriteLine outputting "A = 1"
10.The call to the 2nd WriteLine simply outputs "B = 2"

So, here we see that the lazy construction/evaluation, paired with a flag for
detecting re-entrant initialization resolves the circular dependency.

2)
Add a static function to 'module' A, eg.

	class A
	{
		public static int a = B.b;

		static A()
		{
			a = 42;
		}

		public static int Foo()
		{
			return A.a;
		}
	}

and a call to main:

			System.Console.Out.WriteLine("A.Foo = {0}", A.Foo());
			System.Console.Out.WriteLine("A = {0}", A.a);
			System.Console.Out.WriteLine("B = {0}", B.b);

So, what happens now:

When it executes the Foo() WriteLine it triggers initialization of A (because
foo is a static member of A), this triggers the process shown in 1) above,
lines 02 thru 08 before returning control to the WriteLine.  As a consequence
all the initialization is done by the next WriteLine call, so it simply
completes etc.

Thoughts?
Mar 12 2010
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Doing lazy initialization certainly works, but it would require all 
static member access to go through a check for initialization, first. 
The cost of this check persists for a statically compiled language; for
Mar 12 2010
next sibling parent BCS <none anon.com> writes:
Hello Walter,


 Doing lazy initialization certainly works, but it would require all
 static member access to go through a check for initialization, first.
 The cost of this check persists for a statically compiled language;

 the check.


That and I have some patterns I like using where I use "static this()" to 
inject results without any change to the code base. For that to work, they 
need to run before things get referenced.

void delegate(string)[string] args; // static this injects into here.

void main(string[] argv)
{
    foreach(string s; argv)
         args[GetBefor('=',s)](GetAfter('=',s));
}

-- 
... <IXOYE><
Mar 12 2010
prev sibling next sibling parent Fawzi Mohamed <fawzi gmx.ch> writes:
On 12-mar-10, at 19:17, Walter Bright wrote:


 Doing lazy initialization certainly works, but it would require all  
 static member access to go through a check for initialization,  
 first. The cost of this check persists for a statically compiled  

 to remove the check.


well if one tracks what gets initialized and what is accessed then one  
could associate a lazy constructor with each static value, and call  
the static initializer explicitly before access in the static methods,  
so that the cost is paid only during startup.
But that is quite some work, because static initializers in D are so  
flexible, and so tracking what they initialize and what they access is  
some work (even if theoretically the compiler can know it).
Also invalid initializations could be catched (but only at runtime)  
having 3 initialization states: on initialized, during initialization,  
initialized.
I still think that something like my proposal  dependOnly(modules) for  
static initializers is a valid alternative.

Fawzi
Mar 12 2010
prev sibling parent reply Regan Heath <regan netmail.co.nz> writes:
Walter Bright wrote:

 Doing lazy initialization certainly works, but it would require all 
 static member access to go through a check for initialization, first. 
 The cost of this check persists for a statically compiled language; for 



Yeah, I figured that was going to be the stumbling block.

Is it possible for the compiler to figure out an order of execution 
during compile instead?  Then it could simply use that to order 
construction instead of import/lexical ordering and there would be no 
runtime cost.

I expect this either isn't possible or falls into the 'possible up to a 
point' category.

R
Mar 15 2010
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Regan Heath wrote:

 I expect this either isn't possible or falls into the 'possible up to a 
 point' category.



It's possible up to a point, that point being the parts of the program 
the compiler does not see.
Mar 15 2010
parent reply Regan Heath <regan netmail.co.nz> writes:
Walter Bright wrote:

 Regan Heath wrote:

 I expect this either isn't possible or falls into the 'possible up to 
 a point' category.


 
 
 It's possible up to a point, that point being the parts of the program 
 the compiler does not see.


So.. isn't that the same point the current import dependency method 
stops at?  And if so, does that mean this method would create a 
different possbily loop free tree for static construction?

R
Mar 15 2010
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Regan Heath wrote:

 Walter Bright wrote:

 Regan Heath wrote:

 I expect this either isn't possible or falls into the 'possible up to 
 a point' category.



 It's possible up to a point, that point being the parts of the program 
 the compiler does not see.


 
 So.. isn't that the same point the current import dependency method 
 stops at?


No, because that's a runtime check.
Mar 15 2010
parent reply Regan Heath <regan netmail.co.nz> writes:
Walter Bright wrote:

 Regan Heath wrote:

 Walter Bright wrote:

 Regan Heath wrote:

 I expect this either isn't possible or falls into the 'possible up 
 to a point' category.



 It's possible up to a point, that point being the parts of the 
 program the compiler does not see.


 So.. isn't that the same point the current import dependency method 
 stops at?


 
 No, because that's a runtime check.


No, I'm no longer suggesting a runtime solution.

How does 'unreachable code' detection in C/C++ compilers work?  Can you 
use something similar to figure out the order module static data is used?

R
Mar 15 2010
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Regan Heath wrote:

 Walter Bright wrote:

 Regan Heath wrote:

 Walter Bright wrote:

 Regan Heath wrote:

 I expect this either isn't possible or falls into the 'possible up 
 to a point' category.



 It's possible up to a point, that point being the parts of the 
 program the compiler does not see.


 So.. isn't that the same point the current import dependency method 
 stops at?


 No, because that's a runtime check.


 
 No, I'm no longer suggesting a runtime solution.
 
 How does 'unreachable code' detection in C/C++ compilers work?


It requires the source code so the compiler can look at it.


 Can you use something similar to figure out the order module static data is
used?


Yes, but that fails when the compiler doesn't have the source code to look at.
Mar 15 2010
parent reply Regan Heath <regan netmail.co.nz> writes:
Walter Bright wrote:

 Regan Heath wrote:

 Walter Bright wrote:

 Regan Heath wrote:

 Walter Bright wrote:

 Regan Heath wrote:

 I expect this either isn't possible or falls into the 'possible up 
 to a point' category.



 It's possible up to a point, that point being the parts of the 
 program the compiler does not see.


 So.. isn't that the same point the current import dependency method 
 stops at?


 No, because that's a runtime check.


 No, I'm no longer suggesting a runtime solution.

 How does 'unreachable code' detection in C/C++ compilers work?


 
 It requires the source code so the compiler can look at it.
 

 Can you use something similar to figure out the order module static 
 data is used?


 
 Yes, but that fails when the compiler doesn't have the source code to 
 look at.


Ahh.. I was naively assuming you were compiling all the source at the 
same time, but of course that's not necessarily true.

So, this needs to work when compiling modules seperately and then 
linking, likewise libraries, right?  Is that the problem you're 
referring to?

Couldn't you store a list of dependencies in usage order in the output 
of the compile (the .o[bj] file) and use these lists when linking to 
resolve module init order.  You would need to know the 'main' module for 
a starting point, but from there you should be able to create an 
ordering.  You'd probably want to treat a static library as a single 
dependency, likewise C libraries etc.

R
Mar 16 2010
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Regan Heath wrote:

 So, this needs to work when compiling modules seperately and then 
 linking, likewise libraries, right?  Is that the problem you're 
 referring to?


Yes.


 Couldn't you store a list of dependencies in usage order in the output 
 of the compile (the .o[bj] file) and use these lists when linking to 
 resolve module init order.  You would need to know the 'main' module for 
 a starting point, but from there you should be able to create an 
 ordering.  You'd probably want to treat a static library as a single 
 dependency, likewise C libraries etc.


Then you'd have to essentially build the linker into the compiler.
Mar 16 2010
parent Regan Heath <regan netmail.co.nz> writes:
Walter Bright wrote:

 Regan Heath wrote:

 So, this needs to work when compiling modules seperately and then 
 linking, likewise libraries, right?  Is that the problem you're 
 referring to?


 
 Yes.
 

 Couldn't you store a list of dependencies in usage order in the output 
 of the compile (the .o[bj] file) and use these lists when linking to 
 resolve module init order.  You would need to know the 'main' module 
 for a starting point, but from there you should be able to create an 
 ordering.  You'd probably want to treat a static library as a single 
 dependency, likewise C libraries etc.


 
 Then you'd have to essentially build the linker into the compiler.


Ah.. I'd have thought using 'import' as you currently do was the same 
sort of thing, but I guess I don't know enough about how compilers work :)

The thing that suddenly seemed odd to me, and the reason I wanted to see 

whether/when something needs to be initialised, when in fact 'import' 
doesn't guarantee the thing is actually used at all.

I can import something erroneously into a module, then never actually 
use anything in that module, or, use only one pure function that doesn't 
use the static needs-to-be-initialised data.

Using 'import' is always going to produce more false positives, but I 
guess it also cannot produce a false negative.

R
Mar 17 2010
prev sibling next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-03-11 21:42:47 -0500, Walter Bright <newshound1 digitalmars.com> said:


 Currently, it is performed as a strictly "depth-first" traversal of the 
 graph defined by the import statements. As we've been discussing here, 
 this works great until one has circular imports, meaning the 
 depth-first graph has a loop in it.
 
 The current behavior on detecting a loop is to quit with an error message.
 
 The problems are:
 
 1. The cycles are not easily gotten rid of when they are the result of 
 template mixins.
 
 2. Trying to analyze the static constructors to see what the 
 dependencies actually are is fraught with unsolvable problems.
 
 
 So, I propose the following:
 
 1. Attempt the depth-first traversal of the static constructors.
 
 2. If a loop is detected, rather than issuing an error message, simply 
 arbitrarily pick one order and continue constructing.
 
 
 The mitigating rationale is that modules that import each other are 
 presumably written by the same person or team, and so that person is in 
 the best place to explicitly control dependencies themselves.
 
 
 I'm not happy with this solution, but it seems to be the best 
 compromise I can come up with.
 
 What do you think?


I think it'd be better if it was explicit. Perhaps better would be an 
opt-in using an attribute on the static constructor. For instance:

module a;
import b;

int a;

static this() {
	a = b;
}


module b;
import a;

int b;

 cyclehead static this() {
	b = 10;
}


Here, module 'a' initialization depends on 'b'. Module 'b' 
initialization does not depend on a or anything, so you can make it 
 cyclehead.  cyclehead means that this module, when part of a cycle, 
can be initialized before the others. If all 'static this' in a module 
are  cyclehead, then the module gets the cyclehead flag and is 
initialized first in the cycle.

This is better than just picking one module at random to break the 
cycle. If no module have the cyclehead flag, then having a cycle is an 
error. The programmer will have to consciously choose one module to 
break the cycle.

Most mixins static constructors will probably have to be  cyclehead.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Mar 12 2010
parent Fawzi Mohamed <fmohamed mac.com> writes:
On 2010-03-12 15:05:25 +0100, Michel Fortin <michel.fortin michelf.com> said:


 [...]
 Most mixins static constructors will probably have to be  cyclehead.


Here is an example of a reasonable use case to clarify why I think that 
my proposal, or something like it is the correct solution.
Often the static initializers like those in module b or c would be 
mixed in (for example to give serialization support, as for example the 
class name in template classes is not unique).

{{{
module a;
// imports modules but does not create a circular dep

class Register {
  void register(...){...}
  ...
}

Register globalRegister;
static this(){
  globalRegister=new Register();
}

-------
module b;
import c;
import a;

class Bla {...}
 dependOnly(a) static this {
  globalRegister.register(...);
}
----
module c;
import b;
import a;

class Bla2 {...}
 dependOnly(a) static this {
  globalRegister.register(...);
}
}}}

My proposal would give the correct initialization order, and would 
still detect invalid cases (assuming the programmers does not lie about 
the dependOnly).

Fawzi
Mar 12 2010
prev sibling next sibling parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 12/03/10 02:42, Walter Bright wrote:

 Currently, it is performed as a strictly "depth-first" traversal of the
 graph defined by the import statements. As we've been discussing here,
 this works great until one has circular imports, meaning the depth-first
 graph has a loop in it.

 The current behavior on detecting a loop is to quit with an error message.

 The problems are:

 1. The cycles are not easily gotten rid of when they are the result of
 template mixins.

 2. Trying to analyze the static constructors to see what the
 dependencies actually are is fraught with unsolvable problems.


 So, I propose the following:

 1. Attempt the depth-first traversal of the static constructors.

 2. If a loop is detected, rather than issuing an error message, simply
 arbitrarily pick one order and continue constructing.


 The mitigating rationale is that modules that import each other are
 presumably written by the same person or team, and so that person is in
 the best place to explicitly control dependencies themselves.


 I'm not happy with this solution, but it seems to be the best compromise
 I can come up with.

 What do you think?


I don't know the full situation, or even if this will help, but maybe 
something like the following could help?

a.d:
----
module a;
import b;

Foo fooA;

static this()
{
    fooA = bar();
}
----
b.d:
----
module b;
import a;
class Foo { }

Foo bar()
{
   return new Foo;
}

Foo fooB;

pure static this()
{
    fooB = bar();
}
----

Here the pure notates that the static constructor doesn't not depend on 
any other modules (it's pure at module scope rather than function 
scope). If a module's static ctor is pure it doesn't matter what order 
it is executed in, so can be decided arbitrarily by the compiler. Doing 
this means you can have cyclic dependencies both with static 
constructors, without worrying about cyclic dependencies. It does 
however mean that you wouldn't be able to use functions from module C, 
it's better than the current situation though.
Mar 12 2010
parent Robert Clipsham <robert octarineparrot.com> writes:
On 12/03/10 16:44, Robert Clipsham wrote:

 I don't know the full situation, or even if this will help, but maybe
 something like the following could help?

 a.d:
 ----
 module a;
 import b;

 Foo fooA;

 static this()
 {
 fooA = bar();
 }
 ----
 b.d:
 ----
 module b;
 import a;
 class Foo { }

 Foo bar()
 {
 return new Foo;
 }

 Foo fooB;

 pure static this()
 {
 fooB = bar();
 }
 ----

 Here the pure notates that the static constructor doesn't not depend on
 any other modules (it's pure at module scope rather than function
 scope). If a module's static ctor is pure it doesn't matter what order
 it is executed in, so can be decided arbitrarily by the compiler. Doing
 this means you can have cyclic dependencies both with static
 constructors, without worrying about cyclic dependencies. It does
 however mean that you wouldn't be able to use functions from module C,
 it's better than the current situation though.


Another possible expansion of this to make it more flexible, but then 
pure might not be the correct keyword to use... The ctor could allow 
dependencies on other modules, as long as the other modules did not have 
cyclic dependencies/only had pure ctor's also. This would allow for 
something more complex like:

a.d:
----
module a;
import b;

Foo fooA;

static this()
{
    fooA = bar();
}
----
b.d:
----
module b;
import a;
import c;
class Foo : C { }

Foo bar()
{
   return new Foo;
}

Foo fooB;

pure static this()
{
    // This depends on module c, but c's does not have cyclic
    // dependencies, and its ctor is pure so it is safe to allow it
    fooB = bar();
}
----
c.d:
----
module c;
class C
{
}

C myC;

pure static this()
{
   myC = new C;
}
----
Mar 12 2010
prev sibling parent Russell Lewis <webmaster villagersonline.com> writes:
Walter Bright wrote:

 So, I propose the following:
 
 1. Attempt the depth-first traversal of the static constructors.
 
 2. If a loop is detected, rather than issuing an error message, simply 
 arbitrarily pick one order and continue constructing.


I'm of the camp that "if it's broken, the programmer needs to fix it." 
Why not just include this as an optional flag on the import statement?

import __dependency__ foo.bar.baz;

The __dependency__ flag means that if there exists a loop which involves 
both this module and foo.bar.baz, then foo.bar.baz should be initialized 
first.  (Contradictory flags would be an error.)

Russ
Mar 17 2010