www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Debugging CTFE

reply Matthias Walter <Matthias.Walter st.ovgu.de> writes:
Hello,

I have written some compile time executable functions with D 1.0 which work in
runtime but hang (and allocate memory without end) at compile-time. Is there a
way to debug this further? Can one print stuff out? (Don't know if writefln
works at compile-time, as I'm using Tango) Can I somehow get a stack trace of
the functions called?

best regards
Matthias Walter
Jun 17 2008
next sibling parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Matthias Walter" <Matthias.Walter st.ovgu.de> wrote in message 
news:g38na9$2gs7$1 digitalmars.com...

 Hello,

 I have written some compile time executable functions with D 1.0 which 
 work in runtime but hang (and allocate memory without end) at 
 compile-time. Is there a way to debug this further? Can one print stuff 
 out? (Don't know if writefln works at compile-time, as I'm using Tango) 
 Can I somehow get a stack trace of the functions called?



No, no, and no.  CTFE support in the current frontend has some rather 
unworkable disadvantages.  For one, it's terribly buggy.  You're better off 
trying to convert it to templates in most cases.  For two, CTFE is 
interpreting a garbage-collected language but is not itself 
garbage-collected, meaning that memory-unconscious code evaluated at compile 
time (i.e. a loop that appends data to the end of a string) will just leak 
like hell and cause the compiler to easily use up several GB of memory. 
That might be what's happening to your code.  Or it could be a bug in CTFE.
Jun 17 2008
next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
Jarrett Billingsley a écrit :

 "Matthias Walter" <Matthias.Walter st.ovgu.de> wrote in message 
 news:g38na9$2gs7$1 digitalmars.com...

 Hello,

 I have written some compile time executable functions with D 1.0 which 
 work in runtime but hang (and allocate memory without end) at 
 compile-time. Is there a way to debug this further? Can one print stuff 
 out? (Don't know if writefln works at compile-time, as I'm using Tango) 
 Can I somehow get a stack trace of the functions called?



 No, no, and no.  CTFE support in the current frontend has some rather 
 unworkable disadvantages.  For one, it's terribly buggy.  You're better off 
 trying to convert it to templates in most cases.  For two, CTFE is 
 interpreting a garbage-collected language but is not itself 
 garbage-collected, meaning that memory-unconscious code evaluated at compile 
 time (i.e. a loop that appends data to the end of a string) will just leak 
 like hell and cause the compiler to easily use up several GB of memory. 
 That might be what's happening to your code.  Or it could be a bug in CTFE. 



After I finish making Descent support D2, I'll try to make templates and 
CTFE debuggable (i.e. you put a breakpoint, you right click on a 
template or a function and select "Debug at compile-time"). The 
advantage here is, it's java code, so the intepreted code does get 
garbage collected. ;-)

I don't think it's hard to do, all the information is available in the 
front-end for this. Of course, any help will be appreciated.
Jun 17 2008
parent reply Extrawurst <spam extrawurst.org> writes:
Ary Borenszweig schrieb:

 After I finish making Descent support D2, I'll try to make templates 
 and CTFE debuggable (i.e. you put a breakpoint, you right click on a 
 template or a function and select "Debug at compile-time").



You are working on this at the very moment ? Thats great to hear, any 
idea when its gonna be released ? Thats what i am waiting for all the 
time ;)
Jun 18 2008
parent Ary Borenszweig <ary esperanto.org.ar> writes:
Extrawurst a écrit :

 Ary Borenszweig schrieb:

 After I finish making Descent support D2, I'll try to make templates 
 and CTFE debuggable (i.e. you put a breakpoint, you right click on a 
 template or a function and select "Debug at compile-time").



 You are working on this at the very moment ? Thats great to hear, any 
 idea when its gonna be released ? Thats what i am waiting for all the 
 time ;)



I hope to have it done in about a month or less, I'm trying to port a 
little bit each day. And then after porting it's about moving the parser 
stuff up to the core stuff (public AST, signature) and then to the UI 
stuff (labels for const and invariant, etc.). But it's a race against 
Walter's releases... See, now after I finish porting to DMD 2.013, I'll 
have to port to DMD 2.015 :-P

(but that should be much easier than porting from D1 to D2)

It's great to hear there is interest in Descent for D2.
Jun 18 2008
prev sibling next sibling parent "Koroskin Denis" <2korden gmail.com> writes:
On Wed, 18 Jun 2008 15:20:23 +0400, Ary Borenszweig <ary esperanto.org.ar>  
wrote:


 Extrawurst a écrit :

 Ary Borenszweig schrieb:

 After I finish making Descent support D2, I'll try to make templates  
 and CTFE debuggable (i.e. you put a breakpoint, you right click on a  
 template or a function and select "Debug at compile-time").



 idea when its gonna be released ? Thats what i am waiting for all the  
 time ;)



 I hope to have it done in about a month or less, I'm trying to port a  
 little bit each day. And then after porting it's about moving the parser  
 stuff up to the core stuff (public AST, signature) and then to the UI  
 stuff (labels for const and invariant, etc.). But it's a race against  
 Walter's releases... See, now after I finish porting to DMD 2.013, I'll  
 have to port to DMD 2.015 :-P

 (but that should be much easier than porting from D1 to D2)

 It's great to hear there is interest in Descent for D2.




Of course there is!!!
Jun 18 2008
prev sibling next sibling parent reply Don <nospam nospam.com.au> writes:
Jarrett Billingsley wrote:

 "Matthias Walter" <Matthias.Walter st.ovgu.de> wrote in message 
 news:g38na9$2gs7$1 digitalmars.com...

 Hello,

 I have written some compile time executable functions with D 1.0 which 
 work in runtime but hang (and allocate memory without end) at 
 compile-time. Is there a way to debug this further? Can one print stuff 
 out? (Don't know if writefln works at compile-time, as I'm using Tango) 
 Can I somehow get a stack trace of the functions called?



 No, no, and no.  CTFE support in the current frontend has some rather 
 unworkable disadvantages.  For one, it's terribly buggy.You're better off 
 trying to convert it to templates in most cases.



I disagree. I've not had much trouble with CTFE. The really, really nice 
thing about CTFE is that you can write it as a runtime function, and 
make sure it works before using it a compile time.

The problem with CTFE is bugzilla issue #1382 (no memory release for 
CTFE functions)...

For two, CTFE is

 interpreting a garbage-collected language but is not itself 
 garbage-collected, meaning that memory-unconscious code evaluated at compile 
 time (i.e. a loop that appends data to the end of a string) will just leak 
 like hell and cause the compiler to easily use up several GB of memory. 
 That might be what's happening to your code.



That's very likely. The maximum size of code you can write with CTFE is 
pretty small.


  Or it could be a bug in CTFE. 



Bugzilla issue #1382 is the killer. One of the most important bugs in 
bugzilla, I reckon.
Jun 19 2008
parent reply Matthias Walter <Matthias.Walter st.ovgu.de> writes:
Don Wrote:


 Jarrett Billingsley wrote:

 "Matthias Walter" <Matthias.Walter st.ovgu.de> wrote in message 
 news:g38na9$2gs7$1 digitalmars.com...

 Hello,

 I have written some compile time executable functions with D 1.0 which 
 work in runtime but hang (and allocate memory without end) at 
 compile-time. Is there a way to debug this further? Can one print stuff 
 out? (Don't know if writefln works at compile-time, as I'm using Tango) 
 Can I somehow get a stack trace of the functions called?



 No, no, and no.  CTFE support in the current frontend has some rather 
 unworkable disadvantages.  For one, it's terribly buggy.You're better off 
 trying to convert it to templates in most cases.



 I disagree. I've not had much trouble with CTFE. The really, really nice 
 thing about CTFE is that you can write it as a runtime function, and 
 make sure it works before using it a compile time.
 
 The problem with CTFE is bugzilla issue #1382 (no memory release for 
 CTFE functions)...
 
 For two, CTFE is

 interpreting a garbage-collected language but is not itself 
 garbage-collected, meaning that memory-unconscious code evaluated at compile 
 time (i.e. a loop that appends data to the end of a string) will just leak 
 like hell and cause the compiler to easily use up several GB of memory. 
 That might be what's happening to your code.



 That's very likely. The maximum size of code you can write with CTFE is 
 pretty small.
 

  Or it could be a bug in CTFE. 



 Bugzilla issue #1382 is the killer. One of the most important bugs in 
 bugzilla, I reckon.



But in my case, the runtime-function needed some milliseconds, allocating not
more than 1 MB data. the compiler then allocated 1 GB until I killed the
process. I guess, there might be some other bugs. Unfortunatly, I was unable to
make proofing code more easy to get the cause of the bug.

Anyway, I now don't use functions for much stuff, but try to achieve my needs
with templates, although template programming is sometimes a bit messier. But
it works and debugging via pragmas is like printf-debugging in C and thus just
a matter of time :) Training more functional thinking is also helpful for other
things...

best regards and thanks for the tips
Matthias
Jun 19 2008
parent Don <nospam nospam.com.au> writes:
Matthias Walter wrote:

 Don Wrote:
 

 Jarrett Billingsley wrote:

 "Matthias Walter" <Matthias.Walter st.ovgu.de> wrote in message 
 news:g38na9$2gs7$1 digitalmars.com...

 Hello,

 I have written some compile time executable functions with D 1.0 which 
 work in runtime but hang (and allocate memory without end) at 
 compile-time. Is there a way to debug this further? Can one print stuff 
 out? (Don't know if writefln works at compile-time, as I'm using Tango) 
 Can I somehow get a stack trace of the functions called?



 unworkable disadvantages.  For one, it's terribly buggy.You're better off 
 trying to convert it to templates in most cases.



 thing about CTFE is that you can write it as a runtime function, and 
 make sure it works before using it a compile time.

 The problem with CTFE is bugzilla issue #1382 (no memory release for 
 CTFE functions)...

 For two, CTFE is

 interpreting a garbage-collected language but is not itself 
 garbage-collected, meaning that memory-unconscious code evaluated at compile 
 time (i.e. a loop that appends data to the end of a string) will just leak 
 like hell and cause the compiler to easily use up several GB of memory. 
 That might be what's happening to your code.



 pretty small.


  Or it could be a bug in CTFE. 



 bugzilla, I reckon.



 But in my case, the runtime-function needed some milliseconds, allocating not
more than 1 MB data. the compiler then allocated 1 GB until I killed the
process. I guess, there might be some other bugs. Unfortunatly, I was unable to
make proofing code more easy to get the cause of the bug.



That'll be issue #1382. You can only do a few thousand memory 
allocations (including ~) before CTFE dies.

Basically, you can use CTFE for very small tasks, and for library 
development. Because of #1382, you can't use it large-scale stuff yet.


 
 Anyway, I now don't use functions for much stuff, but try to achieve my needs
with templates, although template programming is sometimes a bit messier. But
it works and debugging via pragmas is like printf-debugging in C and thus just
a matter of time :) Training more functional thinking is also helpful for other
things...
 
 best regards and thanks for the tips
 Matthias
Jun 19 2008
prev sibling parent "Koroskin Denis" <2korden gmail.com> writes:
On Thu, 19 Jun 2008 14:11:55 +0400, Don <nospam nospam.com.au> wrote:


 Matthias Walter wrote:

 Don Wrote:


 Jarrett Billingsley wrote:

 "Matthias Walter" <Matthias.Walter st.ovgu.de> wrote in message  
 news:g38na9$2gs7$1 digitalmars.com...

 Hello,

 I have written some compile time executable functions with D 1.0  
 which work in runtime but hang (and allocate memory without end) at  
 compile-time. Is there a way to debug this further? Can one print  
 stuff out? (Don't know if writefln works at compile-time, as I'm  
 using Tango) Can I somehow get a stack trace of the functions called?



 unworkable disadvantages.  For one, it's terribly buggy.You're better  
 off trying to convert it to templates in most cases.



 nice thing about CTFE is that you can write it as a runtime function,  
 and make sure it works before using it a compile time.

 The problem with CTFE is bugzilla issue #1382 (no memory release for  
 CTFE functions)...

 For two, CTFE is

 interpreting a garbage-collected language but is not itself  
 garbage-collected, meaning that memory-unconscious code evaluated at  
 compile time (i.e. a loop that appends data to the end of a string)  
 will just leak like hell and cause the compiler to easily use up  
 several GB of memory. That might be what's happening to your code.



 is pretty small.


  Or it could be a bug in CTFE.



 bugzilla, I reckon.



 allocating not more than 1 MB data. the compiler then allocated 1 GB  
 until I killed the process. I guess, there might be some other bugs.  
 Unfortunatly, I was unable to make proofing code more easy to get the  
 cause of the bug.



 That'll be issue #1382. You can only do a few thousand memory  
 allocations (including ~) before CTFE dies.

 Basically, you can use CTFE for very small tasks, and for library  
 development. Because of #1382, you can't use it large-scale stuff yet.


  Anyway, I now don't use functions for much stuff, but try to achieve  
 my needs with templates, although template programming is sometimes a  
 bit messier. But it works and debugging via pragmas is like  
 printf-debugging in C and thus just a matter of time :) Training more  
 functional thinking is also helpful for other things...
  best regards and thanks for the tips
 Matthias






IIRC, there is a Hans Boehm garbage Collector available for MDC, could  
Walter integrate it into DMD and solve the problem? :)
Jun 19 2008
prev sibling next sibling parent reply "Koroskin Denis" <2korden gmail.com> writes:
On Tue, 17 Jun 2008 20:04:25 +0400, Matthias Walter  
<Matthias.Walter st.ovgu.de> wrote:


 Hello,

 I have written some compile time executable functions with D 1.0 which  
 work in runtime but hang (and allocate memory without end) at  
 compile-time. Is there a way to debug this further? Can one print stuff  
 out? (Don't know if writefln works at compile-time, as I'm using Tango)  
 Can I somehow get a stack trace of the functions called?

 best regards
 Matthias Walter



You can use pragma(msg, "hello there!") to output some text at compile  
time.
Jun 17 2008
next sibling parent BCS <ao pathlink.com> writes:
Reply to Koroskin,


 On Tue, 17 Jun 2008 20:04:25 +0400, Matthias Walter
 <Matthias.Walter st.ovgu.de> wrote:
 

 Hello,
 
 I have written some compile time executable functions with D 1.0
 which  work in runtime but hang (and allocate memory without end) at
 compile-time. Is there a way to debug this further? Can one print
 stuff  out? (Don't know if writefln works at compile-time, as I'm
 using Tango)  Can I somehow get a stack trace of the functions
 called?
 
 best regards
 Matthias Walter



 time.
 



I don't /think/ that works for what is wanted:

char[] CTFE()
{
   char[] c = "hello world", 
   for(;c.length >0; c=c[0..$-1])
      pragma(msg, c);

   return c;
}


// not tested
Jun 17 2008
prev sibling next sibling parent "Koroskin Denis" <2korden gmail.com> writes:
On Tue, 17 Jun 2008 20:56:00 +0400, BCS <ao pathlink.com> wrote:


 Reply to Koroskin,


 On Tue, 17 Jun 2008 20:04:25 +0400, Matthias Walter
 <Matthias.Walter st.ovgu.de> wrote:


 Hello,
  I have written some compile time executable functions with D 1.0
 which  work in runtime but hang (and allocate memory without end) at
 compile-time. Is there a way to debug this further? Can one print
 stuff  out? (Don't know if writefln works at compile-time, as I'm
 using Tango)  Can I somehow get a stack trace of the functions
 called?
  best regards
 Matthias Walter



 time.



 I don't /think/ that works for what is wanted:

 char[] CTFE()
 {
    char[] c = "hello world",    for(;c.length >0; c=c[0..$-1])
       pragma(msg, c);

    return c;
 }


 // not tested



You are right. But although it doesn't work (currently) for CTFE, it does  
for templates:

template itoa(uint i)
{
     static if (i < 10)
         const char[] itoa = "" ~ cast(char)(i + '0');
     else
         const char[] itoa = itoa!(i / 10) ~ itoa!(i % 10);
}

template factorial(int i)
{
     pragma(msg, "Calculating factorial of " ~ itoa!(i));

     static if (i < 2) {
         const int factorial = 1;
     } else {
         const int factorial = factorial!(i-1)*i;
     }
}

int main()
{
     const int t = factorial!(100);
     return 0;
}
Jun 17 2008
prev sibling parent Max Samukha <samukha voliacable.com.removethis> writes:
On Tue, 17 Jun 2008 20:45:48 +0400, "Koroskin Denis"
<2korden gmail.com> wrote:


On Tue, 17 Jun 2008 20:04:25 +0400, Matthias Walter  
<Matthias.Walter st.ovgu.de> wrote:


 Hello,

 I have written some compile time executable functions with D 1.0 which  
 work in runtime but hang (and allocate memory without end) at  
 compile-time. Is there a way to debug this further? Can one print stuff  
 out? (Don't know if writefln works at compile-time, as I'm using Tango)  
 Can I somehow get a stack trace of the functions called?

 best regards
 Matthias Walter



You can use pragma(msg, "hello there!") to output some text at compile  
time.



The only way to print a debugging message from the inside of
compile-time function is to use assert(false, "hallo"); but it breaks
execution.
Jun 17 2008
prev sibling parent reply BCS <ao pathlink.com> writes:
Reply to Matthias,


 Hello,
 
 I have written some compile time executable functions with D 1.0 which
 work in runtime but hang (and allocate memory without end) at
 compile-time. Is there a way to debug this further? Can one print
 stuff out? (Don't know if writefln works at compile-time, as I'm using
 Tango) Can I somehow get a stack trace of the functions called?
 
 best regards
 Matthias Walter



Walter,can we plese get some debugging hooks in the CTFE stuff?
Jun 17 2008
parent Fawzi Mohamed <fmohamed mac.com> writes:
On 2008-06-17 19:42:33 +0200, BCS <ao pathlink.com> said:


 Reply to Matthias,
 

 Hello,
 
 I have written some compile time executable functions with D 1.0 which
 work in runtime but hang (and allocate memory without end) at
 compile-time. Is there a way to debug this further? Can one print
 stuff out? (Don't know if writefln works at compile-time, as I'm using
 Tango) Can I somehow get a stack trace of the functions called?
 
 best regards
 Matthias Walter



 Walter,can we plese get some debugging hooks in the CTFE stuff?



would be nice, actually a compile-time stacktrace upon failure is all I 
would need :)

anyway I had once the same problem, in my case it was something like this

class A(T,int i){
  A!(T,i+1) growI(int b){}
}

that forced the instantiation of A!(T,i+1) which in turn forced the 
instantiation of ...

moving the  function out of the class fixed the things
	A!(T,i+1) growI(T,int i)(A!(T,i)a,int b){}

In general I have found some very weird bugs (absence of this, getting 
tuck in some dependencies,...) with template member functions (that 
seem to be there in dmd since long), so when possible I use external 
templates, and limit the use of mixins.
Doing it it worked out quite well.

Fawzi
Jun 18 2008