www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Voldemort Types in D

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
http://www.reddit.com/r/programming/comments/telhj/voldemort_types_in_d/

Andrei
May 09 2012
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 09/05/2012 16:30, Andrei Alexandrescu a écrit :
 http://www.reddit.com/r/programming/comments/telhj/voldemort_types_in_d/

 Andrei

I read it. writeln(take(generator(5), 10)); => writeln(generator(5).take(10)); UFCS FTW ! Great reading BTW, and I love how ti is named.
May 09 2012
parent Sean Kelly <sean invisibleduck.org> writes:
On May 9, 2012, at 8:43 AM, deadalnix wrote:
 
 Great reading BTW, and I love how ti is named.

I'd prefer Lovecraftian types :-) Good article though.
May 09 2012
prev sibling next sibling parent reply "Nicolas Sicard" <dransic gmail.com> writes:
On Wednesday, 9 May 2012 at 14:30:33 UTC, Andrei Alexandrescu 
wrote:
 http://www.reddit.com/r/programming/comments/telhj/voldemort_types_in_d/

 Andrei

One drawback of Voldemort types, is that they are incompatible with the generation of .di files (option -H). See http://d.puremagic.com/issues/show_bug.cgi?id=5461. Nicolas
May 09 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-05-09 19:05, Adam Wilson wrote:

 This pull fixes this problem and a bunch of others:
 https://github.com/D-Programming-Language/dmd/pull/928. However, it
 currently fails to build on Linux and fails the unittests on Windows
 thanks to a problem with the dur template function in std.datetime. The
 solution used by this pull is to include the function body of the
 auto-function as that was needed to allow Phobos to build correctly
 using the DRT DI files. This is the only workable solution as DMD does
 not know the type of an auto-function when DI files are generated. No
 semantic analysis has been performed and since semantic analysis could
 change the layout of a module you wouldn't want it to be
 performed.

The correct way to solve this in the long run is to have the compiler replace "auto" with the inferred type. If the whole semantic analysis phase cannot be run the we need to have some kind of limited semantic analysis phase or some other phase to infer the types. -- /Jacob Carlborg
May 09 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 05/09/2012 10:09 PM, Jacob Carlborg wrote:
 On 2012-05-09 19:05, Adam Wilson wrote:

 This pull fixes this problem and a bunch of others:
 https://github.com/D-Programming-Language/dmd/pull/928. However, it
 currently fails to build on Linux and fails the unittests on Windows
 thanks to a problem with the dur template function in std.datetime. The
 solution used by this pull is to include the function body of the
 auto-function as that was needed to allow Phobos to build correctly
 using the DRT DI files. This is the only workable solution as DMD does
 not know the type of an auto-function when DI files are generated. No
 semantic analysis has been performed and since semantic analysis could
 change the layout of a module you wouldn't want it to be
 performed.

The correct way to solve this in the long run is to have the compiler replace "auto" with the inferred type. If the whole semantic analysis phase cannot be run the we need to have some kind of limited semantic analysis phase or some other phase to infer the types.

The type cannot be inferred without full semantic analysis in general. Furthermore, it could be different based on compilation options (or maybe based on in what order the modules were passed to DMD on the command line.)
May 09 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-05-09 23:03, Timon Gehr wrote:

 The type cannot be inferred without full semantic analysis in general.
 Furthermore, it could be different based on compilation options (or
 maybe based on in what order the modules were passed to DMD on the
 command line.)

That has to be solved, somehow. -- /Jacob Carlborg
May 09 2012
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Wed, 09 May 2012 08:58:56 -0700, Nicolas Sicard <dransic gmail.com>  
wrote:

 On Wednesday, 9 May 2012 at 14:30:33 UTC, Andrei Alexandrescu wrote:
 http://www.reddit.com/r/programming/comments/telhj/voldemort_types_in_d/

 Andrei

One drawback of Voldemort types, is that they are incompatible with the generation of .di files (option -H). See http://d.puremagic.com/issues/show_bug.cgi?id=5461. Nicolas

This pull fixes this problem and a bunch of others: https://github.com/D-Programming-Language/dmd/pull/928. However, it currently fails to build on Linux and fails the unittests on Windows thanks to a problem with the dur template function in std.datetime. The solution used by this pull is to include the function body of the auto-function as that was needed to allow Phobos to build correctly using the DRT DI files. This is the only workable solution as DMD does not know the type of an auto-function when DI files are generated. No semantic analysis has been performed and since semantic analysis could change the layout of a module you wouldn't want it to be performed. -- Adam Wilson IRC: LightBender Project Coordinator The Horizon Project http://www.thehorizonproject.org/
May 09 2012
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, May 09, 2012 10:05:36 Adam Wilson wrote:
 On Wed, 09 May 2012 08:58:56 -0700, Nicolas Sicard <dransic gmail.com>
 
 wrote:
 On Wednesday, 9 May 2012 at 14:30:33 UTC, Andrei Alexandrescu wrote:
 http://www.reddit.com/r/programming/comments/telhj/voldemort_types_in_d/
 
 Andrei

One drawback of Voldemort types, is that they are incompatible with the generation of .di files (option -H). See http://d.puremagic.com/issues/show_bug.cgi?id=5461. Nicolas

This pull fixes this problem and a bunch of others: https://github.com/D-Programming-Language/dmd/pull/928. However, it currently fails to build on Linux and fails the unittests on Windows thanks to a problem with the dur template function in std.datetime. The solution used by this pull is to include the function body of the auto-function as that was needed to allow Phobos to build correctly using the DRT DI files. This is the only workable solution as DMD does not know the type of an auto-function when DI files are generated. No semantic analysis has been performed and since semantic analysis could change the layout of a module you wouldn't want it to be performed.

Actually, dur is in core.time (which is publicly imported by std.datetime), but regardless, what's wrong with it? It's incredibly straightforward. - Jonathan M Davis
May 09 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-05-09 20:54, Adam Wilson wrote:

 Nuts. It's CTFE. Because the implementation source is being stripped
 out, that was *THE* main request of DI files and they are pointless
 without that.

 Essentially DI files and CTFE are mutually exclusive.

No, it's the same issue with templates and inline. C/C++ headers have the same issue. If you want to have a CTFE function you need to include it in the DI file. -- /Jacob Carlborg
May 09 2012
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2012 1:35 PM, Jonathan M Davis wrote:
 Yeah. You _can_ strip them out (except for templates), but doing so restricts
 what you can do (in this case, killing inlining and CTFE). Personally, I think
 that it makes .di files essentially useless except in specific cases where you
 can't use CTFE anyway and inlining isn't an issue (e.g. much of std.file
 wouldn't need its implementation available, but that would be fatal to a
 module like std.string).

Consider the garbage collector. There's no way you'd want to CTFE that. It makes perfect sense to only provide an interface to it (.di) rather than full source. And for many other things.
May 09 2012
prev sibling next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
On May 9, 2012, at 7:30 AM, Andrei Alexandrescu wrote:

 =

One thing: "and then use g. I know what you're thinking =97 use typeof and declare = another instance of RandomNumberGenerator: auto g =3D generator(4); typeof(g) h; Sorry, that won't work, the compiler will not allow a Voldemort Type to = be instantiated outside of its scope (the technical reason is it has no = reference to the seed local variable)." If the struct were made static and given a ctor that seed were passed = to, I imagine it would work, yes?=
May 09 2012
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2012 10:11 AM, Sean Kelly wrote:
 If the struct were made static and given a ctor that seed were passed to, I
imagine it would work, yes?

It should work. But you'll run into another issue - transmitting that typeof *up* out of its scope.
May 09 2012
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Wed, 09 May 2012 10:19:08 -0700, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Wednesday, May 09, 2012 10:05:36 Adam Wilson wrote:
 On Wed, 09 May 2012 08:58:56 -0700, Nicolas Sicard <dransic gmail.com>

 wrote:
 On Wednesday, 9 May 2012 at 14:30:33 UTC, Andrei Alexandrescu wrote:
  


 Andrei

One drawback of Voldemort types, is that they are incompatible with

 generation of .di files (option -H).

 See http://d.puremagic.com/issues/show_bug.cgi?id=5461.

 Nicolas

This pull fixes this problem and a bunch of others: https://github.com/D-Programming-Language/dmd/pull/928. However, it currently fails to build on Linux and fails the unittests on Windows thanks to a problem with the dur template function in std.datetime. The solution used by this pull is to include the function body of the auto-function as that was needed to allow Phobos to build correctly using the DRT DI files. This is the only workable solution as DMD does not know the type of an auto-function when DI files are generated. No semantic analysis has been performed and since semantic analysis could change the layout of a module you wouldn't want it to be performed.

Actually, dur is in core.time (which is publicly imported by std.datetime), but regardless, what's wrong with it? It's incredibly straightforward. - Jonathan M Davis

All of this is a result of using the above mentioned 928 pull. This from core.time.d: Duration dur(string units)(long length) safe pure nothrow if(units == "weeks" || units == "days" || units == "hours" || units == "minutes" || units == "seconds" || units == "msecs" || units == "usecs" || units == "hnsecs" || units == "nsecs") { return Duration(convert!(units, "hnsecs")(length)); } Gets translated to this in core.time.di: template dur(string units) if (units == "weeks" || units == "days" || units == "hours" || units == "minutes" || units == "seconds" || units == "msecs" || units == "usecs" || units == "hnsecs" || units == "nsecs") { pure nothrow safe Duration dur(long length) { return Duration(convert!(units,"hnsecs")(length)); } } Which results in this error from DMD (HEAD): ../druntime/import/core/time.di(218): Error: this cannot be interpreted at compile time, because it has no available source code std/net/curl.d(187): called from here: dur(2L) make[1]: *** [generated/linux/release/32/libphobos2.a] Error 1 make: *** [release] Error 2 But DMD, there *IS* source code to read! Even if it is just a function call... Note the above D->DI translation is the same without the patch, I checked on a vanilla DMD 2.059 install. Also, I don't know near enough about core.time to begin to guess what D is whining about here... It all looks good to me. -- Adam Wilson IRC: LightBender Project Coordinator The Horizon Project http://www.thehorizonproject.org/
May 09 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, May 09, 2012 10:41:31 Adam Wilson wrote:
 On Wed, 09 May 2012 10:19:08 -0700, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:
 On Wednesday, May 09, 2012 10:05:36 Adam Wilson wrote:
 On Wed, 09 May 2012 08:58:56 -0700, Nicolas Sicard <dransic gmail.com>
 
 wrote:
 On Wednesday, 9 May 2012 at 14:30:33 UTC, Andrei Alexandrescu wrote:

 Andrei

One drawback of Voldemort types, is that they are incompatible with

the
 generation of .di files (option -H).
 
 See http://d.puremagic.com/issues/show_bug.cgi?id=5461.
 
 Nicolas

This pull fixes this problem and a bunch of others: https://github.com/D-Programming-Language/dmd/pull/928. However, it currently fails to build on Linux and fails the unittests on Windows thanks to a problem with the dur template function in std.datetime. The solution used by this pull is to include the function body of the auto-function as that was needed to allow Phobos to build correctly using the DRT DI files. This is the only workable solution as DMD does not know the type of an auto-function when DI files are generated. No semantic analysis has been performed and since semantic analysis could change the layout of a module you wouldn't want it to be performed.

Actually, dur is in core.time (which is publicly imported by std.datetime), but regardless, what's wrong with it? It's incredibly straightforward. - Jonathan M Davis

All of this is a result of using the above mentioned 928 pull. This from core.time.d: Duration dur(string units)(long length) safe pure nothrow if(units == "weeks" || units == "days" || units == "hours" || units == "minutes" || units == "seconds" || units == "msecs" || units == "usecs" || units == "hnsecs" || units == "nsecs") { return Duration(convert!(units, "hnsecs")(length)); } Gets translated to this in core.time.di: template dur(string units) if (units == "weeks" || units == "days" || units == "hours" || units == "minutes" || units == "seconds" || units == "msecs" || units == "usecs" || units == "hnsecs" || units == "nsecs") { pure nothrow safe Duration dur(long length) { return Duration(convert!(units,"hnsecs")(length)); } } Which results in this error from DMD (HEAD): ../druntime/import/core/time.di(218): Error: this cannot be interpreted at compile time, because it has no available source code std/net/curl.d(187): called from here: dur(2L) make[1]: *** [generated/linux/release/32/libphobos2.a] Error 1 make: *** [release] Error 2 But DMD, there *IS* source code to read! Even if it is just a function call... Note the above D->DI translation is the same without the patch, I checked on a vanilla DMD 2.059 install. Also, I don't know near enough about core.time to begin to guess what D is whining about here... It all looks good to me.

Line 187 of std.net.curl is an enum: private enum _defaultDataTimeout = dur!"minutes"(2); So, dur!"minutes(2) will be evaluated at compile time. That means that the _full_ source of everything that's needed for dur!"minutes"(2) must be available. At minimum, that's going to be Duration's source code (including the full source for its constructor) and the source of convert. Duration's constructor is quite straightforward, so as long as its full body is included, you should be fine. convert requires the source of hnsecsPer in addition to its own source, but hnsecsPer doesn't rely on anything else and both of those are templates, so their full source should be in the .di file anyway. So, I would think that as long as the basic declaration for Duration is available along with the full source for Duration's constructor, std.time.convert, and std.time.hnsecsPer is available, then CTFE should be able to evaluate dur!"minutes"(2), and std.curl.net should be fine. So, if any of that source is being stripped out, then there's your problem. If it's all there, then I don't know what's wrong. Maybe there's another function in Duration that got stripped when it was needed (though I don't know what else in Duration besides the constructor would be required for dur!"minutes"(2) to be evaluated). Personally, I think that the fact that CTFE requires the full source to work is a prime example of why .di files tend to be a very bad idea, but you can still strip at least some of it for at least stuff that wouldn't be used in CTFE. - Jonathan M Davis
May 09 2012
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Wed, 09 May 2012 11:00:01 -0700, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Wednesday, May 09, 2012 10:41:31 Adam Wilson wrote:
 On Wed, 09 May 2012 10:19:08 -0700, Jonathan M Davis  
 <jmdavisProg gmx.com>

 wrote:
 On Wednesday, May 09, 2012 10:05:36 Adam Wilson wrote:
 On Wed, 09 May 2012 08:58:56 -0700, Nicolas Sicard  


 wrote:
 On Wednesday, 9 May 2012 at 14:30:33 UTC, Andrei Alexandrescu  



  


 Andrei

One drawback of Voldemort types, is that they are incompatible with

the
 generation of .di files (option -H).

 See http://d.puremagic.com/issues/show_bug.cgi?id=5461.

 Nicolas

This pull fixes this problem and a bunch of others: https://github.com/D-Programming-Language/dmd/pull/928. However, it currently fails to build on Linux and fails the unittests on Windows thanks to a problem with the dur template function in std.datetime.


 solution used by this pull is to include the function body of the
 auto-function as that was needed to allow Phobos to build correctly
 using
 the DRT DI files. This is the only workable solution as DMD does not
 know
 the type of an auto-function when DI files are generated. No semantic
 analysis has been performed and since semantic analysis could change  


 layout of a module you wouldn't want it to be
 performed.

Actually, dur is in core.time (which is publicly imported by std.datetime), but regardless, what's wrong with it? It's incredibly straightforward. - Jonathan M Davis

All of this is a result of using the above mentioned 928 pull. This from core.time.d: Duration dur(string units)(long length) safe pure nothrow if(units == "weeks" || units == "days" || units == "hours" || units == "minutes" || units == "seconds" || units == "msecs" || units == "usecs" || units == "hnsecs" || units == "nsecs") { return Duration(convert!(units, "hnsecs")(length)); } Gets translated to this in core.time.di: template dur(string units) if (units == "weeks" || units == "days" || units == "hours" || units == "minutes" || units == "seconds" || units == "msecs" || units == "usecs" || units == "hnsecs" || units == "nsecs") { pure nothrow safe Duration dur(long length) { return Duration(convert!(units,"hnsecs")(length)); } } Which results in this error from DMD (HEAD): ../druntime/import/core/time.di(218): Error: this cannot be interpreted at compile time, because it has no available source code std/net/curl.d(187): called from here: dur(2L) make[1]: *** [generated/linux/release/32/libphobos2.a] Error 1 make: *** [release] Error 2 But DMD, there *IS* source code to read! Even if it is just a function call... Note the above D->DI translation is the same without the patch, I checked on a vanilla DMD 2.059 install. Also, I don't know near enough about core.time to begin to guess what D is whining about here... It all looks good to me.

Line 187 of std.net.curl is an enum: private enum _defaultDataTimeout = dur!"minutes"(2); So, dur!"minutes(2) will be evaluated at compile time. That means that the _full_ source of everything that's needed for dur!"minutes"(2) must be available. At minimum, that's going to be Duration's source code (including the full source for its constructor) and the source of convert. Duration's constructor is quite straightforward, so as long as its full body is included, you should be fine. convert requires the source of hnsecsPer in addition to its own source, but hnsecsPer doesn't rely on anything else and both of those are templates, so their full source should be in the .di file anyway. So, I would think that as long as the basic declaration for Duration is available along with the full source for Duration's constructor, std.time.convert, and std.time.hnsecsPer is available, then CTFE should be able to evaluate dur!"minutes"(2), and std.curl.net should be fine. So, if any of that source is being stripped out, then there's your problem. If it's all there, then I don't know what's wrong. Maybe there's another function in Duration that got stripped when it was needed (though I don't know what else in Duration besides the constructor would be required for dur!"minutes"(2) to be evaluated). Personally, I think that the fact that CTFE requires the full source to work is a prime example of why .di files tend to be a very bad idea, but you can still strip at least some of it for at least stuff that wouldn't be used in CTFE. - Jonathan M Davis

Nuts. It's CTFE. Because the implementation source is being stripped out, that was *THE* main request of DI files and they are pointless without that. Essentially DI files and CTFE are mutually exclusive. I would say that Phobos' usage of CTFE breaks the cardinal rule of external code, never assume that it will always be available and won't change. I'll be posting a larger message on the main forums about this ... It's a big issue. -- Adam Wilson IRC: LightBender Project Coordinator The Horizon Project http://www.thehorizonproject.org/
May 09 2012
prev sibling next sibling parent reply "Bernard Helyer" <b.helyer gmail.com> writes:
You've been busy! Looks like you've hit your 2 article limit.

Log-in or register for a free account to get unlimited articles 
and full access to Dr. Dobb's.

Sorry doctor, but I don't care about that other article that much. My god, Dr. Dobb's is such a shit-hole. Walter's articles are their only redeeming feature.
May 09 2012
parent deadalnix <deadalnix gmail.com> writes:
Le 09/05/2012 21:23, Bernard Helyer a écrit :
 You've been busy! Looks like you've hit your 2 article limit.

 Log-in or register for a free account to get unlimited articles and
 full access to Dr. Dobb's.

Sorry doctor, but I don't care about that other article that much. My god, Dr. Dobb's is such a shit-hole. Walter's articles are their only redeeming feature.

Disable coockies. I did noticed this limitation only when I got a coockie enabled computer. This has been fixed quickly.
May 10 2012
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-05-09 16:30, Andrei Alexandrescu wrote:
 http://www.reddit.com/r/programming/comments/telhj/voldemort_types_in_d/

 Andrei

D also have some problems with voldemort types. As can be seen in a discussion in the main newsgroup, it's not possible to declare a function pointer which returns by reference. ref int function () bar; test.d(8): Error: variable test.bar only parameters or foreach declarations can be ref Then if we try place "ref" after "function" we get another error. int function ref () bar; test.d(8): found 'ref' when expecting '(' test.d(8): basic type expected, not ( test.d(8): function declaration without return type. (Note that constructors are always named 'this') test.d(8): found 'bar' when expecting ')' test.d(8): no identifier for declarator int function(int()) -- /Jacob Carlborg
May 09 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, May 09, 2012 22:11:20 Jacob Carlborg wrote:
 On 2012-05-09 20:54, Adam Wilson wrote:
 Nuts. It's CTFE. Because the implementation source is being stripped
 out, that was *THE* main request of DI files and they are pointless
 without that.
 
 Essentially DI files and CTFE are mutually exclusive.

No, it's the same issue with templates and inline. C/C++ headers have the same issue. If you want to have a CTFE function you need to include it in the DI file.

Yeah. You _can_ strip them out (except for templates), but doing so restricts what you can do (in this case, killing inlining and CTFE). Personally, I think that it makes .di files essentially useless except in specific cases where you can't use CTFE anyway and inlining isn't an issue (e.g. much of std.file wouldn't need its implementation available, but that would be fatal to a module like std.string). - Jonathan M Davis
May 09 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, May 09, 2012 14:06:10 Walter Bright wrote:
 On 5/9/2012 1:35 PM, Jonathan M Davis wrote:
 Yeah. You _can_ strip them out (except for templates), but doing so
 restricts what you can do (in this case, killing inlining and CTFE).
 Personally, I think that it makes .di files essentially useless except in
 specific cases where you can't use CTFE anyway and inlining isn't an
 issue (e.g. much of std.file wouldn't need its implementation available,
 but that would be fatal to a module like std.string).

Consider the garbage collector. There's no way you'd want to CTFE that. It makes perfect sense to only provide an interface to it (.di) rather than full source. And for many other things.

Yes. There _are_ cases where not providing the whole source makes sense, but I really think that that's the exception rather than the rule given the need for the full source for CTFE and inlining. I also suspect that in a lot of cases, automatically generating .di files makes no sense, because you need control over what is and isn't available. Using automatic generation as the base and then editing it makes a lot of sense, but blindly doing it on an entire project like we do with druntime is probably a mistake. And Adam's Wilson's attempt to make the .di generation strip out as much as possible makes that even worse. - Jonathan M Davis
May 09 2012
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Wed, 09 May 2012 23:20:47 -0700, Jacob Carlborg <doob me.com> wrote:

 On 2012-05-09 23:03, Timon Gehr wrote:

 The type cannot be inferred without full semantic analysis in general.
 Furthermore, it could be different based on compilation options (or
 maybe based on in what order the modules were passed to DMD on the
 command line.)

That has to be solved, somehow.

Pretty much the only option is to completely overhaul DI generation with a limited form of semantic analysis built-in, it would be work, but it's not rocket science. Although decyphering the DMD source can be ... tricky. -- Adam Wilson IRC: LightBender Project Coordinator The Horizon Project http://www.thehorizonproject.org/
May 09 2012
prev sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 5/9/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
 http://www.reddit.com/r/programming/comments/telhj/voldemort_types_in_d/

"Sorry, that won't work, the compiler will not allow a Voldemort Type to be instantiated outside of its scope (the technical reason is it has no reference to the seed local variable)." Actually you can, although this should probably be disallowed: import std.traits; alias ReturnType!(generator) Gen; Gen gen; void main() { writeln(gen.front); // access violation }
May 11 2012