www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Debugging CTFE

reply Matthias Walter <Matthias.Walter st.ovgu.de> writes:
Hello,

I have written some compile time executable functions with D 1.0 which work in
runtime but hang (and allocate memory without end) at compile-time. Is there a
way to debug this further? Can one print stuff out? (Don't know if writefln
works at compile-time, as I'm using Tango) Can I somehow get a stack trace of
the functions called?

best regards
Matthias Walter
Jun 17 2008
next sibling parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Matthias Walter" <Matthias.Walter st.ovgu.de> wrote in message 
news:g38na9$2gs7$1 digitalmars.com...
 Hello,

 I have written some compile time executable functions with D 1.0 which 
 work in runtime but hang (and allocate memory without end) at 
 compile-time. Is there a way to debug this further? Can one print stuff 
 out? (Don't know if writefln works at compile-time, as I'm using Tango) 
 Can I somehow get a stack trace of the functions called?

No, no, and no. CTFE support in the current frontend has some rather unworkable disadvantages. For one, it's terribly buggy. You're better off trying to convert it to templates in most cases. For two, CTFE is interpreting a garbage-collected language but is not itself garbage-collected, meaning that memory-unconscious code evaluated at compile time (i.e. a loop that appends data to the end of a string) will just leak like hell and cause the compiler to easily use up several GB of memory. That might be what's happening to your code. Or it could be a bug in CTFE.
Jun 17 2008
next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
Jarrett Billingsley a écrit :
 "Matthias Walter" <Matthias.Walter st.ovgu.de> wrote in message 
 news:g38na9$2gs7$1 digitalmars.com...
 Hello,

 I have written some compile time executable functions with D 1.0 which 
 work in runtime but hang (and allocate memory without end) at 
 compile-time. Is there a way to debug this further? Can one print stuff 
 out? (Don't know if writefln works at compile-time, as I'm using Tango) 
 Can I somehow get a stack trace of the functions called?

No, no, and no. CTFE support in the current frontend has some rather unworkable disadvantages. For one, it's terribly buggy. You're better off trying to convert it to templates in most cases. For two, CTFE is interpreting a garbage-collected language but is not itself garbage-collected, meaning that memory-unconscious code evaluated at compile time (i.e. a loop that appends data to the end of a string) will just leak like hell and cause the compiler to easily use up several GB of memory. That might be what's happening to your code. Or it could be a bug in CTFE.

After I finish making Descent support D2, I'll try to make templates and CTFE debuggable (i.e. you put a breakpoint, you right click on a template or a function and select "Debug at compile-time"). The advantage here is, it's java code, so the intepreted code does get garbage collected. ;-) I don't think it's hard to do, all the information is available in the front-end for this. Of course, any help will be appreciated.
Jun 17 2008
parent reply Extrawurst <spam extrawurst.org> writes:
Ary Borenszweig schrieb:
 After I finish making Descent support D2, I'll try to make templates 
 and CTFE debuggable (i.e. you put a breakpoint, you right click on a 
 template or a function and select "Debug at compile-time").

You are working on this at the very moment ? Thats great to hear, any idea when its gonna be released ? Thats what i am waiting for all the time ;)
Jun 18 2008
parent Ary Borenszweig <ary esperanto.org.ar> writes:
Extrawurst a écrit :
 Ary Borenszweig schrieb:
 After I finish making Descent support D2, I'll try to make templates 
 and CTFE debuggable (i.e. you put a breakpoint, you right click on a 
 template or a function and select "Debug at compile-time").

You are working on this at the very moment ? Thats great to hear, any idea when its gonna be released ? Thats what i am waiting for all the time ;)

I hope to have it done in about a month or less, I'm trying to port a little bit each day. And then after porting it's about moving the parser stuff up to the core stuff (public AST, signature) and then to the UI stuff (labels for const and invariant, etc.). But it's a race against Walter's releases... See, now after I finish porting to DMD 2.013, I'll have to port to DMD 2.015 :-P (but that should be much easier than porting from D1 to D2) It's great to hear there is interest in Descent for D2.
Jun 18 2008
prev sibling next sibling parent "Koroskin Denis" <2korden gmail.com> writes:
On Wed, 18 Jun 2008 15:20:23 +0400, Ary Borenszweig <ary esperanto.org.ar>  
wrote:

 Extrawurst a écrit :
 Ary Borenszweig schrieb:
 After I finish making Descent support D2, I'll try to make templates  
 and CTFE debuggable (i.e. you put a breakpoint, you right click on a  
 template or a function and select "Debug at compile-time").

idea when its gonna be released ? Thats what i am waiting for all the time ;)

I hope to have it done in about a month or less, I'm trying to port a little bit each day. And then after porting it's about moving the parser stuff up to the core stuff (public AST, signature) and then to the UI stuff (labels for const and invariant, etc.). But it's a race against Walter's releases... See, now after I finish porting to DMD 2.013, I'll have to port to DMD 2.015 :-P (but that should be much easier than porting from D1 to D2) It's great to hear there is interest in Descent for D2.

Of course there is!!!
Jun 18 2008
prev sibling next sibling parent reply Don <nospam nospam.com.au> writes:
Jarrett Billingsley wrote:
 "Matthias Walter" <Matthias.Walter st.ovgu.de> wrote in message 
 news:g38na9$2gs7$1 digitalmars.com...
 Hello,

 I have written some compile time executable functions with D 1.0 which 
 work in runtime but hang (and allocate memory without end) at 
 compile-time. Is there a way to debug this further? Can one print stuff 
 out? (Don't know if writefln works at compile-time, as I'm using Tango) 
 Can I somehow get a stack trace of the functions called?

No, no, and no. CTFE support in the current frontend has some rather unworkable disadvantages. For one, it's terribly buggy.You're better off trying to convert it to templates in most cases.

I disagree. I've not had much trouble with CTFE. The really, really nice thing about CTFE is that you can write it as a runtime function, and make sure it works before using it a compile time. The problem with CTFE is bugzilla issue #1382 (no memory release for CTFE functions)... For two, CTFE is
 interpreting a garbage-collected language but is not itself 
 garbage-collected, meaning that memory-unconscious code evaluated at compile 
 time (i.e. a loop that appends data to the end of a string) will just leak 
 like hell and cause the compiler to easily use up several GB of memory. 
 That might be what's happening to your code.

That's very likely. The maximum size of code you can write with CTFE is pretty small.
  Or it could be a bug in CTFE. 

Bugzilla issue #1382 is the killer. One of the most important bugs in bugzilla, I reckon.
Jun 19 2008
parent reply Matthias Walter <Matthias.Walter st.ovgu.de> writes:
Don Wrote:

 Jarrett Billingsley wrote:
 "Matthias Walter" <Matthias.Walter st.ovgu.de> wrote in message 
 news:g38na9$2gs7$1 digitalmars.com...
 Hello,

 I have written some compile time executable functions with D 1.0 which 
 work in runtime but hang (and allocate memory without end) at 
 compile-time. Is there a way to debug this further? Can one print stuff 
 out? (Don't know if writefln works at compile-time, as I'm using Tango) 
 Can I somehow get a stack trace of the functions called?

No, no, and no. CTFE support in the current frontend has some rather unworkable disadvantages. For one, it's terribly buggy.You're better off trying to convert it to templates in most cases.

I disagree. I've not had much trouble with CTFE. The really, really nice thing about CTFE is that you can write it as a runtime function, and make sure it works before using it a compile time. The problem with CTFE is bugzilla issue #1382 (no memory release for CTFE functions)... For two, CTFE is
 interpreting a garbage-collected language but is not itself 
 garbage-collected, meaning that memory-unconscious code evaluated at compile 
 time (i.e. a loop that appends data to the end of a string) will just leak 
 like hell and cause the compiler to easily use up several GB of memory. 
 That might be what's happening to your code.

That's very likely. The maximum size of code you can write with CTFE is pretty small.
  Or it could be a bug in CTFE. 

Bugzilla issue #1382 is the killer. One of the most important bugs in bugzilla, I reckon.

But in my case, the runtime-function needed some milliseconds, allocating not more than 1 MB data. the compiler then allocated 1 GB until I killed the process. I guess, there might be some other bugs. Unfortunatly, I was unable to make proofing code more easy to get the cause of the bug. Anyway, I now don't use functions for much stuff, but try to achieve my needs with templates, although template programming is sometimes a bit messier. But it works and debugging via pragmas is like printf-debugging in C and thus just a matter of time :) Training more functional thinking is also helpful for other things... best regards and thanks for the tips Matthias
Jun 19 2008
parent Don <nospam nospam.com.au> writes:
Matthias Walter wrote:
 Don Wrote:
 
 Jarrett Billingsley wrote:
 "Matthias Walter" <Matthias.Walter st.ovgu.de> wrote in message 
 news:g38na9$2gs7$1 digitalmars.com...
 Hello,

 I have written some compile time executable functions with D 1.0 which 
 work in runtime but hang (and allocate memory without end) at 
 compile-time. Is there a way to debug this further? Can one print stuff 
 out? (Don't know if writefln works at compile-time, as I'm using Tango) 
 Can I somehow get a stack trace of the functions called?

unworkable disadvantages. For one, it's terribly buggy.You're better off trying to convert it to templates in most cases.

thing about CTFE is that you can write it as a runtime function, and make sure it works before using it a compile time. The problem with CTFE is bugzilla issue #1382 (no memory release for CTFE functions)... For two, CTFE is
 interpreting a garbage-collected language but is not itself 
 garbage-collected, meaning that memory-unconscious code evaluated at compile 
 time (i.e. a loop that appends data to the end of a string) will just leak 
 like hell and cause the compiler to easily use up several GB of memory. 
 That might be what's happening to your code.

pretty small.
  Or it could be a bug in CTFE. 

bugzilla, I reckon.

But in my case, the runtime-function needed some milliseconds, allocating not more than 1 MB data. the compiler then allocated 1 GB until I killed the process. I guess, there might be some other bugs. Unfortunatly, I was unable to make proofing code more easy to get the cause of the bug.

That'll be issue #1382. You can only do a few thousand memory allocations (including ~) before CTFE dies. Basically, you can use CTFE for very small tasks, and for library development. Because of #1382, you can't use it large-scale stuff yet.
 
 Anyway, I now don't use functions for much stuff, but try to achieve my needs
with templates, although template programming is sometimes a bit messier. But
it works and debugging via pragmas is like printf-debugging in C and thus just
a matter of time :) Training more functional thinking is also helpful for other
things...
 
 best regards and thanks for the tips
 Matthias

Jun 19 2008
prev sibling parent "Koroskin Denis" <2korden gmail.com> writes:
On Thu, 19 Jun 2008 14:11:55 +0400, Don <nospam nospam.com.au> wrote:

 Matthias Walter wrote:
 Don Wrote:

 Jarrett Billingsley wrote:
 "Matthias Walter" <Matthias.Walter st.ovgu.de> wrote in message  
 news:g38na9$2gs7$1 digitalmars.com...
 Hello,

 I have written some compile time executable functions with D 1.0  
 which work in runtime but hang (and allocate memory without end) at  
 compile-time. Is there a way to debug this further? Can one print  
 stuff out? (Don't know if writefln works at compile-time, as I'm  
 using Tango) Can I somehow get a stack trace of the functions called?

unworkable disadvantages. For one, it's terribly buggy.You're better off trying to convert it to templates in most cases.

nice thing about CTFE is that you can write it as a runtime function, and make sure it works before using it a compile time. The problem with CTFE is bugzilla issue #1382 (no memory release for CTFE functions)... For two, CTFE is
 interpreting a garbage-collected language but is not itself  
 garbage-collected, meaning that memory-unconscious code evaluated at  
 compile time (i.e. a loop that appends data to the end of a string)  
 will just leak like hell and cause the compiler to easily use up  
 several GB of memory. That might be what's happening to your code.

is pretty small.
  Or it could be a bug in CTFE.

bugzilla, I reckon.

allocating not more than 1 MB data. the compiler then allocated 1 GB until I killed the process. I guess, there might be some other bugs. Unfortunatly, I was unable to make proofing code more easy to get the cause of the bug.

That'll be issue #1382. You can only do a few thousand memory allocations (including ~) before CTFE dies. Basically, you can use CTFE for very small tasks, and for library development. Because of #1382, you can't use it large-scale stuff yet.
  Anyway, I now don't use functions for much stuff, but try to achieve  
 my needs with templates, although template programming is sometimes a  
 bit messier. But it works and debugging via pragmas is like  
 printf-debugging in C and thus just a matter of time :) Training more  
 functional thinking is also helpful for other things...
  best regards and thanks for the tips
 Matthias


IIRC, there is a Hans Boehm garbage Collector available for MDC, could Walter integrate it into DMD and solve the problem? :)
Jun 19 2008
prev sibling next sibling parent reply "Koroskin Denis" <2korden gmail.com> writes:
On Tue, 17 Jun 2008 20:04:25 +0400, Matthias Walter  
<Matthias.Walter st.ovgu.de> wrote:

 Hello,

 I have written some compile time executable functions with D 1.0 which  
 work in runtime but hang (and allocate memory without end) at  
 compile-time. Is there a way to debug this further? Can one print stuff  
 out? (Don't know if writefln works at compile-time, as I'm using Tango)  
 Can I somehow get a stack trace of the functions called?

 best regards
 Matthias Walter

You can use pragma(msg, "hello there!") to output some text at compile time.
Jun 17 2008
next sibling parent BCS <ao pathlink.com> writes:
Reply to Koroskin,

 On Tue, 17 Jun 2008 20:04:25 +0400, Matthias Walter
 <Matthias.Walter st.ovgu.de> wrote:
 
 Hello,
 
 I have written some compile time executable functions with D 1.0
 which  work in runtime but hang (and allocate memory without end) at
 compile-time. Is there a way to debug this further? Can one print
 stuff  out? (Don't know if writefln works at compile-time, as I'm
 using Tango)  Can I somehow get a stack trace of the functions
 called?
 
 best regards
 Matthias Walter

time.

I don't /think/ that works for what is wanted: char[] CTFE() { char[] c = "hello world", for(;c.length >0; c=c[0..$-1]) pragma(msg, c); return c; } // not tested
Jun 17 2008
prev sibling next sibling parent "Koroskin Denis" <2korden gmail.com> writes:
On Tue, 17 Jun 2008 20:56:00 +0400, BCS <ao pathlink.com> wrote:

 Reply to Koroskin,

 On Tue, 17 Jun 2008 20:04:25 +0400, Matthias Walter
 <Matthias.Walter st.ovgu.de> wrote:

 Hello,
  I have written some compile time executable functions with D 1.0
 which  work in runtime but hang (and allocate memory without end) at
 compile-time. Is there a way to debug this further? Can one print
 stuff  out? (Don't know if writefln works at compile-time, as I'm
 using Tango)  Can I somehow get a stack trace of the functions
 called?
  best regards
 Matthias Walter

time.

I don't /think/ that works for what is wanted: char[] CTFE() { char[] c = "hello world", for(;c.length >0; c=c[0..$-1]) pragma(msg, c); return c; } // not tested

You are right. But although it doesn't work (currently) for CTFE, it does for templates: template itoa(uint i) { static if (i < 10) const char[] itoa = "" ~ cast(char)(i + '0'); else const char[] itoa = itoa!(i / 10) ~ itoa!(i % 10); } template factorial(int i) { pragma(msg, "Calculating factorial of " ~ itoa!(i)); static if (i < 2) { const int factorial = 1; } else { const int factorial = factorial!(i-1)*i; } } int main() { const int t = factorial!(100); return 0; }
Jun 17 2008
prev sibling parent Max Samukha <samukha voliacable.com.removethis> writes:
On Tue, 17 Jun 2008 20:45:48 +0400, "Koroskin Denis"
<2korden gmail.com> wrote:

On Tue, 17 Jun 2008 20:04:25 +0400, Matthias Walter  
<Matthias.Walter st.ovgu.de> wrote:

 Hello,

 I have written some compile time executable functions with D 1.0 which  
 work in runtime but hang (and allocate memory without end) at  
 compile-time. Is there a way to debug this further? Can one print stuff  
 out? (Don't know if writefln works at compile-time, as I'm using Tango)  
 Can I somehow get a stack trace of the functions called?

 best regards
 Matthias Walter

You can use pragma(msg, "hello there!") to output some text at compile time.

The only way to print a debugging message from the inside of compile-time function is to use assert(false, "hallo"); but it breaks execution.
Jun 17 2008
prev sibling parent reply BCS <ao pathlink.com> writes:
Reply to Matthias,

 Hello,
 
 I have written some compile time executable functions with D 1.0 which
 work in runtime but hang (and allocate memory without end) at
 compile-time. Is there a way to debug this further? Can one print
 stuff out? (Don't know if writefln works at compile-time, as I'm using
 Tango) Can I somehow get a stack trace of the functions called?
 
 best regards
 Matthias Walter

Walter,can we plese get some debugging hooks in the CTFE stuff?
Jun 17 2008
parent Fawzi Mohamed <fmohamed mac.com> writes:
On 2008-06-17 19:42:33 +0200, BCS <ao pathlink.com> said:

 Reply to Matthias,
 
 Hello,
 
 I have written some compile time executable functions with D 1.0 which
 work in runtime but hang (and allocate memory without end) at
 compile-time. Is there a way to debug this further? Can one print
 stuff out? (Don't know if writefln works at compile-time, as I'm using
 Tango) Can I somehow get a stack trace of the functions called?
 
 best regards
 Matthias Walter

Walter,can we plese get some debugging hooks in the CTFE stuff?

would be nice, actually a compile-time stacktrace upon failure is all I would need :) anyway I had once the same problem, in my case it was something like this class A(T,int i){ A!(T,i+1) growI(int b){} } that forced the instantiation of A!(T,i+1) which in turn forced the instantiation of ... moving the function out of the class fixed the things A!(T,i+1) growI(T,int i)(A!(T,i)a,int b){} In general I have found some very weird bugs (absence of this, getting tuck in some dependencies,...) with template member functions (that seem to be there in dmd since long), so when possible I use external templates, and limit the use of mixins. Doing it it worked out quite well. Fawzi
Jun 18 2008