digitalmars.D - CTFE bug causes null check to pass on null pointers (Issue 7602)
- H. S. Teoh (14/14) Mar 18 2012 While testing my AA implementation on existing AA-related bug, I came
- Don Clugston (9/20) Mar 19 2012 Yes. The existing D2 AA implementation is hopelessly broken.
- Daniel Murphy (3/5) Mar 19 2012 Walter didn't agree, so it didn't happen.
- H. S. Teoh (16/24) Mar 19 2012 I'm working on my AA implementation, hopefully to get it to the point it
- Don (25/46) Mar 19 2012 Yes, CTFE supports 'new'. The big issue for the runtime is supporting AA...
- H. S. Teoh (32/73) Mar 19 2012 I've thought about supporting literals. Currently what I have in mind is
- Don (5/76) Mar 19 2012 It may help you to know that inside the compiler, AA literals are
- Jens Mueller (16/42) Mar 21 2012 Interesting. How do I make use of this?
- David Nadlinger (5/10) Mar 21 2012 You can't do that right now (i.e. converting CTFE-allocated
- Jens Mueller (10/22) Mar 21 2012 Just for clarification:
- Don Clugston (12/37) Mar 21 2012 Any heap-allocated object is on the CTFE heap. At runtime, it needs to
- Jens Mueller (12/44) Mar 21 2012 That's fine with me. Just want to use new at compile-time.
- Jacob Carlborg (5/8) Mar 21 2012 What about anonymous nested classes, would that kind of like class liter...
- H. S. Teoh (13/24) Mar 21 2012 [...]
While testing my AA implementation on existing AA-related bug, I came across this issue: http://d.puremagic.com/issues/show_bug.cgi?id=7602 Upon playing around a bit with the sample code given in the bug, I managed to find a code snippet that would cause this code: if (impl !is null) { ... } to actually execute the code inside the if-statement body ***even when impl is null***, while running in CTFE. I've bumped the severity to critical because something is very very wrong with this. T -- Study gravitation, it's a field with a lot of potential.
Mar 18 2012
On 19/03/12 06:43, H. S. Teoh wrote:While testing my AA implementation on existing AA-related bug, I came across this issue: http://d.puremagic.com/issues/show_bug.cgi?id=7602 Upon playing around a bit with the sample code given in the bug, I managed to find a code snippet that would cause this code: if (impl !is null) { ... } to actually execute the code inside the if-statement body ***even when impl is null***, while running in CTFE. I've bumped the severity to critical because something is very very wrong with this. TYes. The existing D2 AA implementation is hopelessly broken. You have to understand that the whole implementation of AAs in D2 is a HACK. It is extremely complicated and the slightest change to any code in the compiler or the runtime can break it. Basically CTFE has to reverse-engineer the druntime code in order to make it to work. It's not an implementation issue, it's a fundamental design flaw. I do not understand why it still part of the compiler after we agreed to roll back to the D1 version.
Mar 19 2012
"Don Clugston" <dac nospam.com> wrote in message news:jk6ru4$1seu$1 digitalmars.com...I do not understand why it still part of the compiler after we agreed to roll back to the D1 version.Walter didn't agree, so it didn't happen.
Mar 19 2012
On Mon, Mar 19, 2012 at 09:49:07AM +0100, Don Clugston wrote: [...]Yes. The existing D2 AA implementation is hopelessly broken. You have to understand that the whole implementation of AAs in D2 is a HACK. It is extremely complicated and the slightest change to any code in the compiler or the runtime can break it. Basically CTFE has to reverse-engineer the druntime code in order to make it to work. It's not an implementation issue, it's a fundamental design flaw.I'm working on my AA implementation, hopefully to get it to the point it can replace the current mess. It already fixes a number of AA-related issues in the bug tracker. The main idea is to require a minimal number of lowerings from the compiler (effectively nothing more than syntactic sugar such as V[K] and AA literal syntax), and everything else will be done via existing operator overloading and templating mechanisms. Ideally, CTFE will "just work" with this implementation instead of requiring druntime-specific hacks in the compiler (but I'm not sure whether this will work, since it has to do memory allocations -- does CTFE support that?).I do not understand why it still part of the compiler after we agreed to roll back to the D1 version.I'm late to the game; how was the D1 version implemented? T -- A mathematician is a device for turning coffee into theorems. -- P. Erdos
Mar 19 2012
On 19.03.2012 18:25, H. S. Teoh wrote:On Mon, Mar 19, 2012 at 09:49:07AM +0100, Don Clugston wrote: [...]Yes, CTFE supports 'new'. The big issue for the runtime is supporting AA literals. CTFE needs to be able to take the output of the runtime functions, and pass it as an AA literal to the rest of the compiler.Yes. The existing D2 AA implementation is hopelessly broken. You have to understand that the whole implementation of AAs in D2 is a HACK. It is extremely complicated and the slightest change to any code in the compiler or the runtime can break it. Basically CTFE has to reverse-engineer the druntime code in order to make it to work. It's not an implementation issue, it's a fundamental design flaw.I'm working on my AA implementation, hopefully to get it to the point it can replace the current mess. It already fixes a number of AA-related issues in the bug tracker. The main idea is to require a minimal number of lowerings from the compiler (effectively nothing more than syntactic sugar such as V[K] and AA literal syntax), and everything else will be done via existing operator overloading and templating mechanisms. Ideally, CTFE will "just work" with this implementation instead of requiring druntime-specific hacks in the compiler (but I'm not sure whether this will work, since it has to do memory allocations -- does CTFE support that?).It was just extern(C) library functions. The D2 version is exactly the same thing (all of the D1 functions still exist in D2), except that it has an AssociativeArray!(Key, Value) wrapper around the extern(C) functions. Which sounds like a trivial intermediate step to a full library implementation, but it isn't. - it's a template, so it needs to be instantiated. What happens if it hasn't been instantiated yet? - what happens when AssociativeArray isn't a struct template? - what happens if there's an error while instantiating it? - what happens when all the functions are inlined away, and you're left with just void* pointers? - what happens when you something of type V[K] interacting with something of type AssociativeArray!(K, V)? This happens in things like template constraints, is() expressions, etc. - how is CTFE supposed to deal with this ruddy thing, that's fully of nasty casts to void *, and which may yet create AA literals themselves? - how are you supposed to get sensible error messages out of this beast? The answer to these questions is, hundreds of hours work, and the biggest implementation disaster in D's history. There can be no 'intermediate step'. The syntax sugar should be added last, not first.I do not understand why it still part of the compiler after we agreed to roll back to the D1 version.I'm late to the game; how was the D1 version implemented?
Mar 19 2012
On Mon, Mar 19, 2012 at 09:46:04PM +0100, Don wrote:On 19.03.2012 18:25, H. S. Teoh wrote:[...]I've thought about supporting literals. Currently what I have in mind is to build the AA in CTFE, then use mixins to explicitly create Slot structs and link them all up into an Impl struct. The address of the Impl struct (that contains the array of Slot*) is then readable at compile-time, so it can be assigned to const AA variables, .dup'd into mutable AA variables, etc. This only works for literals that only have compile-time known contents, of course. Things like: int x; int[string] aa = [ "abc": x ]; won't work with this scheme. To support literals that reference variables, some kind of runtime mechanism would be needed, perhaps something similar to the current implementation where the compiler passes an array of keys and an array of values to an AA factory function. [...]The main idea is to require a minimal number of lowerings from the compiler (effectively nothing more than syntactic sugar such as V[K] and AA literal syntax), and everything else will be done via existing operator overloading and templating mechanisms. Ideally, CTFE will "just work" with this implementation instead of requiring druntime-specific hacks in the compiler (but I'm not sure whether this will work, since it has to do memory allocations -- does CTFE support that?).Yes, CTFE supports 'new'. The big issue for the runtime is supporting AA literals. CTFE needs to be able to take the output of the runtime functions, and pass it as an AA literal to the rest of the compiler.Yeah, this "wrapper" is the source of a good number of issues currently on the bugtracker. It's also extremely ugly (the Range interface, for example, is essentially a copy-n-paste of the structs in aaA.d).It was just extern(C) library functions. The D2 version is exactly the same thing (all of the D1 functions still exist in D2), except that it has an AssociativeArray!(Key, Value) wrapper around the extern(C) functions. Which sounds like a trivial intermediate step to a full library implementation, but it isn't.I do not understand why it still part of the compiler after we agreed to roll back to the D1 version.I'm late to the game; how was the D1 version implemented?- it's a template, so it needs to be instantiated. What happens if it hasn't been instantiated yet? - what happens when AssociativeArray isn't a struct template? - what happens if there's an error while instantiating it? - what happens when all the functions are inlined away, and you're left with just void* pointers? - what happens when you something of type V[K] interacting with something of type AssociativeArray!(K, V)? This happens in things like template constraints, is() expressions, etc. - how is CTFE supposed to deal with this ruddy thing, that's fully of nasty casts to void *, and which may yet create AA literals themselves? - how are you supposed to get sensible error messages out of this beast?The source of most of these problems is the schizophrenic split between aaA.d and struct AssociativeArray. It has to be one or the other. There can be no intermediate. The compiler needs to treat V[K] as an alias for AssociativeArray!(K,V) (or vice versa, but regardless, the two must be *identical* in all respects). Having two different things for them (such as aaA.d returning void* and AssociativeArray!(K,V) being a struct wrapping a void*) only leads to pain and disaster.The answer to these questions is, hundreds of hours work, and the biggest implementation disaster in D's history. There can be no 'intermediate step'. The syntax sugar should be added last, not first.Agreed. T -- "Computer Science is no more about computers than astronomy is about telescopes." -- E.W. Dijkstra
Mar 19 2012
On 19.03.2012 22:14, H. S. Teoh wrote:On Mon, Mar 19, 2012 at 09:46:04PM +0100, Don wrote:It may help you to know that inside the compiler, AA literals are implemented as two arrays, one of keys, one of values. Those non-constant literals never participate in CTFE, so I think it would be best to treat them completely separately.On 19.03.2012 18:25, H. S. Teoh wrote:[...]I've thought about supporting literals. Currently what I have in mind is to build the AA in CTFE, then use mixins to explicitly create Slot structs and link them all up into an Impl struct. The address of the Impl struct (that contains the array of Slot*) is then readable at compile-time, so it can be assigned to const AA variables, .dup'd into mutable AA variables, etc. This only works for literals that only have compile-time known contents, of course. Things like: int x; int[string] aa = [ "abc": x ]; won't work with this scheme. To support literals that reference variables, some kind of runtime mechanism would be needed, perhaps something similar to the current implementation where the compiler passes an array of keys and an array of values to an AA factory function.The main idea is to require a minimal number of lowerings from the compiler (effectively nothing more than syntactic sugar such as V[K] and AA literal syntax), and everything else will be done via existing operator overloading and templating mechanisms. Ideally, CTFE will "just work" with this implementation instead of requiring druntime-specific hacks in the compiler (but I'm not sure whether this will work, since it has to do memory allocations -- does CTFE support that?).Yes, CTFE supports 'new'. The big issue for the runtime is supporting AA literals. CTFE needs to be able to take the output of the runtime functions, and pass it as an AA literal to the rest of the compiler.[...]Yeah, this "wrapper" is the source of a good number of issues currently on the bugtracker. It's also extremely ugly (the Range interface, for example, is essentially a copy-n-paste of the structs in aaA.d).It was just extern(C) library functions. The D2 version is exactly the same thing (all of the D1 functions still exist in D2), except that it has an AssociativeArray!(Key, Value) wrapper around the extern(C) functions. Which sounds like a trivial intermediate step to a full library implementation, but it isn't.I do not understand why it still part of the compiler after we agreed to roll back to the D1 version.I'm late to the game; how was the D1 version implemented?- it's a template, so it needs to be instantiated. What happens if it hasn't been instantiated yet? - what happens when AssociativeArray isn't a struct template? - what happens if there's an error while instantiating it? - what happens when all the functions are inlined away, and you're left with just void* pointers? - what happens when you something of type V[K] interacting with something of type AssociativeArray!(K, V)? This happens in things like template constraints, is() expressions, etc. - how is CTFE supposed to deal with this ruddy thing, that's fully of nasty casts to void *, and which may yet create AA literals themselves? - how are you supposed to get sensible error messages out of this beast?The source of most of these problems is the schizophrenic split between aaA.d and struct AssociativeArray. It has to be one or the other. There can be no intermediate. The compiler needs to treat V[K] as an alias for AssociativeArray!(K,V) (or vice versa, but regardless, the two must be *identical* in all respects). Having two different things for them (such as aaA.d returning void* and AssociativeArray!(K,V) being a struct wrapping a void*) only leads to pain and disaster.The answer to these questions is, hundreds of hours work, and the biggest implementation disaster in D's history. There can be no 'intermediate step'. The syntax sugar should be added last, not first.Agreed. T
Mar 19 2012
Don wrote:On 19.03.2012 18:25, H. S. Teoh wrote:Interesting. How do I make use of this? struct Foo {} Foo* foo() { auto a = new Foo; return a; } unittest { //enum b = foo(); // fails } => Error: cannot use non-constant CTFE pointer in an initializer What's the trick to use memory allocated in a CTFE. Say e.g. I want to build a tree at compile time using CTFE. JensOn Mon, Mar 19, 2012 at 09:49:07AM +0100, Don Clugston wrote: [...]Yes, CTFE supports 'new'. The big issue for the runtime is supporting AA literals. CTFE needs to be able to take the output of the runtime functions, and pass it as an AA literal to the rest of the compiler.Yes. The existing D2 AA implementation is hopelessly broken. You have to understand that the whole implementation of AAs in D2 is a HACK. It is extremely complicated and the slightest change to any code in the compiler or the runtime can break it. Basically CTFE has to reverse-engineer the druntime code in order to make it to work. It's not an implementation issue, it's a fundamental design flaw.I'm working on my AA implementation, hopefully to get it to the point it can replace the current mess. It already fixes a number of AA-related issues in the bug tracker. The main idea is to require a minimal number of lowerings from the compiler (effectively nothing more than syntactic sugar such as V[K] and AA literal syntax), and everything else will be done via existing operator overloading and templating mechanisms. Ideally, CTFE will "just work" with this implementation instead of requiring druntime-specific hacks in the compiler (but I'm not sure whether this will work, since it has to do memory allocations -- does CTFE support that?).
Mar 21 2012
On Wednesday, 21 March 2012 at 09:51:43 UTC, Jens Mueller wrote:Interesting. How do I make use of this? […] What's the trick to use memory allocated in a CTFE. Say e.g. I want to build a tree at compile time using CTFE.You can't do that right now (i.e. converting CTFE-allocated memory to initializers for run-time values), but it enables you to use classes, etc. _during_ CTFE. David
Mar 21 2012
David Nadlinger wrote:On Wednesday, 21 March 2012 at 09:51:43 UTC, Jens Mueller wrote:Just for clarification: Why do you say "initializers for _run-time_ values"? I believe I just want to allocate memory at compile-time. I just found out you are not allowed to return an instance. But you can use out parameters. That means I can achieve what I have in mind using classes. Even though it looks a bit clumsy and not like I do it in non-CTFE code. Many Thanks. JensInteresting. How do I make use of this? [$B!D(B] What's the trick to use memory allocated in a CTFE. Say e.g. I want to build a tree at compile time using CTFE.You can't do that right now (i.e. converting CTFE-allocated memory to initializers for run-time values), but it enables you to use classes, etc. _during_ CTFE.
Mar 21 2012
On 21/03/12 11:22, Jens Mueller wrote:David Nadlinger wrote:Any heap-allocated object is on the CTFE heap. At runtime, it needs to be on the runtime heap. There is currently no mechanism in the back-end for transferring data from the CTFE heap to the runtime heap, so it currently generates an error message. You can return structs and arrays, because we have struct literals and array literals, but there is no such thing as a class literal. CTFE will currently allow you to return a class, but only if it is null.On Wednesday, 21 March 2012 at 09:51:43 UTC, Jens Mueller wrote:Just for clarification: Why do you say "initializers for _run-time_ values"? I believe I just want to allocate memory at compile-time.Interesting. How do I make use of this? [$B!D(B] What's the trick to use memory allocated in a CTFE. Say e.g. I want to build a tree at compile time using CTFE.You can't do that right now (i.e. converting CTFE-allocated memory to initializers for run-time values), but it enables you to use classes, etc. _during_ CTFE.I just found out you are not allowed to return an instance. But you can use out parameters.Oh dear. That sounds like a bug. There is just no way you can automatically instantiate a class at runtime, using CTFE data. There would need to be code in the runtime to do it, and it just doesn't exist! That means I can achieve what I have in mind usingclasses. Even though it looks a bit clumsy and not like I do it in non-CTFE code. Many Thanks. Jens
Mar 21 2012
Don Clugston wrote:On 21/03/12 11:22, Jens Mueller wrote:That's fine with me. Just want to use new at compile-time. But being able to move memory from CTFE heap to runtime heap would be nice and probably finds its uses.David Nadlinger wrote:Any heap-allocated object is on the CTFE heap. At runtime, it needs to be on the runtime heap. There is currently no mechanism in the back-end for transferring data from the CTFE heap to the runtime heap, so it currently generates an error message.On Wednesday, 21 March 2012 at 09:51:43 UTC, Jens Mueller wrote:Just for clarification: Why do you say "initializers for _run-time_ values"? I believe I just want to allocate memory at compile-time.Interesting. How do I make use of this? [$B!D(B] What's the trick to use memory allocated in a CTFE. Say e.g. I want to build a tree at compile time using CTFE.You can't do that right now (i.e. converting CTFE-allocated memory to initializers for run-time values), but it enables you to use classes, etc. _during_ CTFE.You can return structs and arrays, because we have struct literals and array literals, but there is no such thing as a class literal. CTFE will currently allow you to return a class, but only if it is null.Aha.I'm doing it all at compile-time. I need to build some data structure (using pointers) at compile time to generate a string that gets mixed in. I think there is no bug. It behaves perfectly as you say. Many Thanks Don for bringing run-time to compile-time. Making compile-time programming less awkward. JensI just found out you are not allowed to return an instance. But you can use out parameters.Oh dear. That sounds like a bug. There is just no way you can automatically instantiate a class at runtime, using CTFE data. There would need to be code in the runtime to do it, and it just doesn't exist!
Mar 21 2012
On 2012-03-21 11:53, Don Clugston wrote:You can return structs and arrays, because we have struct literals and array literals, but there is no such thing as a class literal. CTFE will currently allow you to return a class, but only if it is null.What about anonymous nested classes, would that kind of like class literals? http://dlang.org/class.html#anonymous -- /Jacob Carlborg
Mar 21 2012
On Wed, Mar 21, 2012 at 10:55:05AM +0100, David Nadlinger wrote:On Wednesday, 21 March 2012 at 09:51:43 UTC, Jens Mueller wrote:[...] Actually you can... though it requires hacks using string mixins. Basically, you create the structure in CTFE, then walk the structure and construct a string containing declarations of the form: "struct node __node_1234 = { value1, value2, ... };" then use a string mixin to instantiate them. This will create the tree nodes as module globals, so they are compiled into the object file. They therefore also have runtime addresses, so the root of the tree can be assigned into a runtime tree pointer, for example. T -- Let X be the set not defined by this sentence...Interesting. How do I make use of this? […] What's the trick to use memory allocated in a CTFE. Say e.g. I want to build a tree at compile time using CTFE.You can't do that right now (i.e. converting CTFE-allocated memory to initializers for run-time values), but it enables you to use classes, etc. _during_ CTFE.
Mar 21 2012