www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Any ideas for lazy evaluation on varaible argument functions?

reply Darryl B <darrylbleau gmail.com> writes:
Tango has this great log library in the tradition on log4j (Or Phyton, if you
prefer). It provides loggers configured in a hierarchy, gives output functions
for differing levels, automatically finds infinite loops in code, makes
coffee, and general tells you that you are a great person and that you're just
a stellar example of your race.

Well, it does some of those things anyway. In the end, it's a pretty decent
log module, other than, it assumes what you're logging is a 'string' of some
sort, but that's a topic for another time. You want to log, and it's strings
you want in there, it pretty much covers all the bases.

With that love fest out of the way, it's a great big cup of cold yummy milk
(and you like milk, for the purposes of reading this post, even if you really
don't), but there's a fly in it. (Also for post reading, you don't like flies).

Yeah, you can work around the fly, take it out with a spoon, whatever, but
it's sort of ruined the otherwise great milk you had there, and you can still
drink it, but you sort of don't want to, though there's really nothing wrong
with it.

So, here's the fly in my cup of milk: I really want to log formatted output. I
can log normal strings without restraint, calling

'log.info("grab milk from the store tonight");'
or
'log.warn("mother in law is coming");'

and that's all great.

But I want to put some variable data in there (as I suspect is what is mostly
done while logging). What I _want_, is to do something like:

'log.info.format("grab {} from the {} {}", "beer", "pub", "right now");'
or
'log.warn.format("{} is {}", "Phobos Developer", "coming this way.");'

However, this can't work _and_ take advantage of anonymous delegates (ala
'lazy'), because the 'log.warn' is going to have to evaluate the format (to
call it).

The current workaround is to call it via:

'log.info(log.format("{} is really {}", "this", "ugly"))'

Which is absolutely optimal from a 'omg your code runs like a racehorse' point
of view. Unfortunately, it fails the 'your code looks like someone got
horribly sick somewhere, could only find a bag to put it in, and so decided to
stash it right there where you format your info log output' test.

I've been studying over the forum archives back from about a year ago, spoke
with some people on IRC, and played with every strange and off-the-wall
thoughts I had , and I can't see a solution (without macros) to call this
oft-used function with less risk of carpal tunnel and the slight increase in
blood pressure everytime I type it out completely.

So, here I am, telling my tale of sadness, hoping someone might have an
insight that I have not. Otherwise, I guess I'll be forced to take this
arg.info(arg.format()) girl to the prom, but I sure hope none of my friends
see me.
Oct 18 2007
next sibling parent reply BCS <ao pathlink.com> writes:
Reply to Darryl,


 I've been studying over the forum archives back from about a year ago,
 spoke with some people on IRC, and played with every strange and
 off-the-wall thoughts I had , and I can't see a solution (without
 macros) to call this oft-used function with less risk of carpal tunnel
 and the slight increase in blood pressure everytime I type it out
 completely.
 
 So, here I am, telling my tale of sadness, hoping someone might have
 an insight that I have not. Otherwise, I guess I'll be forced to take
 this arg.info(arg.format()) girl to the prom, but I sure hope none of
 my friends see me.
 



make .info look like this

infoV(A...)(lazy A a){
   this.info(arg.format(a));
}

that (or somthing like it) should work
Oct 18 2007
parent reply Darryl B <darrylbleau gmail.com> writes:
== Quote from BCS (ao pathlink.com)'s article

 make .info look like this
 infoV(A...)(lazy A a){
    this.info(arg.format(a));
 }
 that (or somthing like it) should work


Hmm, I think that works to preserve the laziness, but it's not really any
cleaner
to do:

t.infoV!(char[], char[])("{}", expensiveFunction());
or
t.infoV!(char[], int, float, char[])("{}, {}, {}", intFunc(), floatFunc(),
charFunc());

Than it is to do:

t.info(t.format("{}", expensiveFunction()));
or
t.info(t.format("{}, {}, {}", intFunc(), floatFunc(), charFunc()));

Unless I'm misunderstanding what you're saying?
Oct 19 2007
parent reply BCS <BCS pathlink.com> writes:
Darryl B wrote:

 == Quote from BCS (ao pathlink.com)'s article
 

make .info look like this
infoV(A...)(lazy A a){
   this.info(arg.format(a));
}
that (or somthing like it) should work


 
 
 Hmm, I think that works to preserve the laziness, but it's not really any
cleaner
 to do:
 
 t.infoV!(char[], char[])("{}", expensiveFunction());
 or
 t.infoV!(char[], int, float, char[])("{}, {}, {}", intFunc(), floatFunc(),
 charFunc());
 
 Than it is to do:
 
 t.info(t.format("{}", expensiveFunction()));
 or
 t.info(t.format("{}, {}, {}", intFunc(), floatFunc(), charFunc()));
 
 Unless I'm misunderstanding what you're saying?


it helps in the less typing area.


also this should work

t.infoV("{}", expensiveFunction());
or
t.infoV("{}, {}, {}", intFn(), floatFn(), charFn());
Oct 19 2007
parent reply Darryl B <darrylbleau gmail.com> writes:
== Quote from BCS (BCS pathlink.com)'s article

 also this should work
 t.infoV("{}", expensiveFunction());
 or
 t.infoV("{}, {}, {}", intFn(), floatFn(), charFn());


Hmm, I can't seem to get that to work. Here's what I was using (using Tango):

module TLog;
import tango.io.Stdout;

class TLog
{
        bool print = false;

        void info(lazy char[] msg)
        {
                if (print)
                        Stdout(msg).newline;
        }
        void infoF(A...)(lazy A a)
        {
                this.info(this.format(a));
        }
        private char[] format(lazy char[] fmt, ...)
        {
                return Stdout.layout().convert(_arguments, _argptr, fmt);
        }
}

char[] expensiveFunction(char[] msg)
{
        Stdout.format("Expensive! ({})", msg).newline;
        return msg;
}

void main()
{
        auto t = new TLog;
        t.info(expensiveFunction("info"));
        t.infoF!(char[], char[])("{}", expensiveFunction("infoF"));
        t.print = true;
        t.info(expensiveFunction("info2"));
        t.infoF!(char[], char[])("{}", expensiveFunction("infoF2"));
        //t.infoF("{}", "infoF3");
}

The output I get is as expected:

Expensive! (info2)
info2
Expensive! (infoF2)
infoF2

But if I uncomment that last line as you suggest, I get:

Error: functions cannot return static array char[2u]
Error: functions cannot return static array char[6u]
test.d:15: function TLog.TLog.format (char[],...) does not match parameter types
(int,int)
Error: cannot implicitly convert expression (_param_0()) of type int to char[]
test.d:37: template instance TLog.TLog.infoF!(char[2u],char[6u]) error
instantiating
test.d:37: Error: functions cannot return static array char[2u]
test.d:37: Error: functions cannot return static array char[6u]

Which is sort of what I would expect would happen. Am I missing something?
Oct 19 2007
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Darryl B wrote:

 ...
 
 But if I uncomment that last line as you suggest, I get:
 
 Error: functions cannot return static array char[2u]
 Error: functions cannot return static array char[6u]
 test.d:15: function TLog.TLog.format (char[],...) does not match parameter
types
 (int,int)
 Error: cannot implicitly convert expression (_param_0()) of type int to char[]
 test.d:37: template instance TLog.TLog.infoF!(char[2u],char[6u]) error
instantiating
 test.d:37: Error: functions cannot return static array char[2u]
 test.d:37: Error: functions cannot return static array char[6u]
 
 Which is sort of what I would expect would happen. Am I missing something?


Aaah.  It's those bloody static arrays again!

Try this instead of that last line: t.infoF("{}"[], "infoF3"[]);

One of the absolute banes of template metaprogramming in D are
statically sized arrays.  For instance, the type of "{}" is *not*
char[]; it's char[2u].

This is a problem for you because what "lazy" does, is it converts each
argument into a delegate that returns that type.  So in effect, what
that call looks like on the inside is:

void infoF(char[2u] delegate() a0, char[6u] delegate() a1)
{
    ...
}

In D, you *cannot* return statically sized arrays from functions.  So
the compiler borks.

Sadly, I don't think there's any way to fix this.  The problem is that
if you've got IFTI (which is what allows you to omit the explicit
template instantiation), the arguments can't be very complex.  What you
would need is something like this:

void infoF(A...)(lazy DecayStaticArrays!(A) a) { ... }

Where "DecayStaticArrays" turns all statically-sized arrays into
dynamically-sized ones: so (char[2u], char[6u]) would become (char[],
char[]).

So yeah: you *can* do this, you just need to put [] after any
statically-sized array variables or array literals.

	-- Daniel
Oct 19 2007
next sibling parent Darryl B <darrylbleau gmail.com> writes:
== Quote from Daniel Keep (daniel.keep.lists gmail.com)'s article

 Aaah.  It's those bloody static arrays again!


Yar, and they be tormenting me code, they is!


 Try this instead of that last line: t.infoF("{}"[], "infoF3"[]);


This does indeed work, and the infoF call this way seems to properly handle the
laziness (not running expensiveFunction() when print is false). This is _almost_
prettier than t.infoF(t.format()), even, though a tad strange to type out. :)


 One of the absolute banes of template metaprogramming in D are
 statically sized arrays.  For instance, the type of "{}" is *not*
 char[]; it's char[2u].


Aha! I see.


 So yeah: you *can* do this, you just need to put [] after any
 statically-sized array variables or array literals.


I also tried just plain old:

infoF(lazy char[] fmt, ...)

But it seems that all the variable _arguments are ran regardless of whether
infoF
uses any of them or not. (ie, infoF("{}", expensiveFunction()); runs
expensiveFunction no matter what). It would be nice to be able to do infoF(lazy
char fmt, lazy ...);, or similar. I was also trying something like
infoF(A...)(lazy A a) { foreach(t; a) { this.info(a); } } but that also has the
same effect (expensiveFunction() still evaluated even if this.info doesn't call
the delegate).
Oct 19 2007
prev sibling parent reply BCS <ao pathlink.com> writes:
Reply to Daniel,



 Sadly, I don't think there's any way to fix this.  The problem is that
 if you've got IFTI (which is what allows you to omit the explicit
 template instantiation), the arguments can't be very complex.  What
 you would need is something like this:
 


What is needed is some way to get at the list of type used and munge them 
before they are pushed into the args list.

I don't think this will work but something like this could be scabbed in 
somehow:

void TFunc(A...)(lazy A : ConvertStatic2Dynamic!(A) a) {...}

somehow the syntax* would have to say that A is the types that the function 
*would* be IFTIed on but just use that for the tuple and then I'll tell you 
what to really use. This would also solve a number of other problems like 
this:

void TFn(T)(T a, T b){...}
char[] world;
TFn("hello", world); // woops!

you could use this:

void TFn(T)(T a, T : T b){...}

to say that only the second arg should be used for ITFI, or even have the 
type derived like in the first case.

* That is what the "A : ..." is supposed to convey, but I'm open to better 
suggestions.
Oct 19 2007
parent Reiner Pope <some address.com> writes:
BCS wrote:

 Reply to Daniel,
 
 

 Sadly, I don't think there's any way to fix this.  The problem is that
 if you've got IFTI (which is what allows you to omit the explicit
 template instantiation), the arguments can't be very complex.  What
 you would need is something like this:


 
 What is needed is some way to get at the list of type used and munge 
 them before they are pushed into the args list.
 
 I don't think this will work but something like this could be scabbed in 
 somehow:
 
 void TFunc(A...)(lazy A : ConvertStatic2Dynamic!(A) a) {...}
 
 somehow the syntax* would have to say that A is the types that the 
 function *would* be IFTIed on but just use that for the tuple and then 
 I'll tell you what to really use. 


I like this, although I don't like the : syntax, since : means 
specialization to me, which we aren't doing. I also find that the 'a' 
seems to get lost at the end, but I don't have any better suggestions 
for this syntax. My alternative syntax which I suggested in "Tuple IFTI 
with implicit conversions" has a tuple as the params list, which I think 
looks nicer in the above case, but it doesn't work so well for specific 
parameters, as it requires ugly indexing by parameter position:

void foo(arglist A)(Wrapper!(A[0]) a) {...}

as opposed to

void foo(A)(A : Wrapper!(A) a) {...}

On second thoughts, I don't mind so much the : syntax.

So I like it.

This would also solve a number of

 other problems like this:
 
 void TFn(T)(T a, T b){...}
 char[] world;
 TFn("hello", world); // woops!


I was not aware of this problem until I tried it out just then. The docs 
don't say what it should do in this case. My initial thought was that D 
does this wrong: T should simply be expressed as typeof(true ? "hello" : 
world), which is char[] in D1.


 
 you could use this:
 
 void TFn(T)(T a, T : T b){...}
 
 to say that only the second arg should be used for ITFI, or even have 
 the type derived like in the first case.


This doesn't really solve the problem, as you get stuck when you write

TFn(world, "hello");

If my above solution isn't going to be used, then the following would 
also work (with your change)

void TFn(A, B)(A : CommonType!(A, B) a, B : CommonType!(A, B) b) {...}

template CommonType(A, B)
{
     alias typeof(true ? A : B) CommonType;
}


    -- Reiner
Oct 20 2007
prev sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
Darryl B wrote:

 Tango has this great log library in the tradition on log4j (Or Phyton, if you
 prefer). It provides loggers configured in a hierarchy, gives output functions
 for differing levels, automatically finds infinite loops in code, makes
 coffee, and general tells you that you are a great person and that you're just
 a stellar example of your race.
 
 Well, it does some of those things anyway. In the end, it's a pretty decent
 log module, other than, it assumes what you're logging is a 'string' of some
 sort, but that's a topic for another time. You want to log, and it's strings
 you want in there, it pretty much covers all the bases.
 
 With that love fest out of the way, it's a great big cup of cold yummy milk
 (and you like milk, for the purposes of reading this post, even if you really
 don't), but there's a fly in it. (Also for post reading, you don't like flies).
 
 Yeah, you can work around the fly, take it out with a spoon, whatever, but
 it's sort of ruined the otherwise great milk you had there, and you can still
 drink it, but you sort of don't want to, though there's really nothing wrong
 with it.
 
 So, here's the fly in my cup of milk: I really want to log formatted output. I
 can log normal strings without restraint, calling
 
 'log.info("grab milk from the store tonight");'
 or
 'log.warn("mother in law is coming");'
 
 and that's all great.
 
 But I want to put some variable data in there (as I suspect is what is mostly
 done while logging). What I _want_, is to do something like:
 
 'log.info.format("grab {} from the {} {}", "beer", "pub", "right now");'
 or
 'log.warn.format("{} is {}", "Phobos Developer", "coming this way.");'
 
 However, this can't work _and_ take advantage of anonymous delegates (ala
 'lazy'), because the 'log.warn' is going to have to evaluate the format (to
 call it).
 
 The current workaround is to call it via:
 
 'log.info(log.format("{} is really {}", "this", "ugly"))'
 
 Which is absolutely optimal from a 'omg your code runs like a racehorse' point
 of view. Unfortunately, it fails the 'your code looks like someone got
 horribly sick somewhere, could only find a bag to put it in, and so decided to
 stash it right there where you format your info log output' test.
 
 I've been studying over the forum archives back from about a year ago, spoke
 with some people on IRC, and played with every strange and off-the-wall
 thoughts I had , and I can't see a solution (without macros) to call this
 oft-used function with less risk of carpal tunnel and the slight increase in
 blood pressure everytime I type it out completely.
 
 So, here I am, telling my tale of sadness, hoping someone might have an
 insight that I have not. Otherwise, I guess I'll be forced to take this
 arg.info(arg.format()) girl to the prom, but I sure hope none of my friends
 see me.


It can be done; just replace the current functions with structs equipped 
with opCall and format.

private struct LogCaller {
         Level level;
         Logger opCall (string msg) {
                 return append (level, msg);
         }

         void format (char[] buffer, char[] formatStr, ...) {
                 level.format (buffer, formatStr, _arguments, _argptr);
         }

         static LogCaller opCall (Level level) {
                 LogCaller caller;
                 caller.level = level;
                 return caller;
         }
}

The remainder of the work is a bit less than obvious. Maybe a template 
taking an alias parameter -- the code uses static fields on Logger to 
determine what to append to.
Oct 18 2007
parent Darryl B <darrylbleau gmail.com> writes:
== Quote from Christopher Wright (dhasenan gmail.com)'s article

 It can be done; just replace the current functions with structs equipped
 with opCall and format.
 private struct LogCaller {
          Level level;
          Logger opCall (string msg) {
                  return append (level, msg);
          }
          void format (char[] buffer, char[] formatStr, ...) {
                  level.format (buffer, formatStr, _arguments, _argptr);
          }
          static LogCaller opCall (Level level) {
                  LogCaller caller;
                  caller.level = level;
                  return caller;
          }
 }
 The remainder of the work is a bit less than obvious. Maybe a template
 taking an alias parameter -- the code uses static fields on Logger to
 determine what to append to.


Ok, I think I sort of see where you're going here, but isn't this also going to
become non-thread safe (keeping state between calls?). I don't think it would
be a
good idea to have to manage thread safety in the logging module. It would also
be
optimal to not have to pass a buffer each time you want to do some output. :)
Oct 19 2007