www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Tips from the compiler

reply bearophile <bearophileHUGS lycos.com> writes:
In Bugzilla I have some compiler messages that don't fit in the set of error
messages nor in the set of normal warnings. Like:
http://d.puremagic.com/issues/show_bug.cgi?id=5056

So I have proposed a third kind of compiler message, that are tips/suggestions
to avoid possible bugs, improve the shape or performance of the code, and so
on. The compiler shows those tips only on request.

My usage of C lints has shown me that such tips sometimes are very useful to
avoid coding pitfalls, etc.

Here you may see some simple example of compiler tips from the CommonLips, that
the compiler shows only when the higher optimization levels are used:
http://shootout.alioth.debian.org/u32/program.php?test=mandelbrot&lang=sbcl&id=1

Bye,
bearophile
Oct 17 2010
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday 17 October 2010 19:40:36 bearophile wrote:
 In Bugzilla I have some compiler messages that don't fit in the set of
 error messages nor in the set of normal warnings. Like:
 http://d.puremagic.com/issues/show_bug.cgi?id=5056
 
 So I have proposed a third kind of compiler message, that are
 tips/suggestions to avoid possible bugs, improve the shape or performance
 of the code, and so on. The compiler shows those tips only on request.
 
 My usage of C lints has shown me that such tips sometimes are very useful
 to avoid coding pitfalls, etc.
 
 Here you may see some simple example of compiler tips from the CommonLips,
 that the compiler shows only when the higher optimization levels are used:
 http://shootout.alioth.debian.org/u32/program.php?test=mandelbrot&lang=sbc
 l&id=1
 
 Bye,
 bearophile

I think that what we should really look at doing is to have either dmd or some other tool built on dmd's front-end be able to do lint-like examination of source code (if it were dmd itself rather than an external tool, it would be a flag which presumably did that only rather than compile your code). That way, all of the various warning and error messages that you and others keep coming up with but will never end up in dmd can be put in there. That tool could provide a way to find all of the weird, stray cases that could result in errors, and anyone who wants to examine their code on that level could use it. Then we wouldn't have all of these potential warnings sitting around which Walter will never put into dmd. And honestly, I think that things like dangling else, unused variables, and redundant code make a lot more sense it a tool that is looking for potential programming errors rather than the compiler. It's the compiler's job to correctly generate the code that it's given, not worry about all of the possible ways that the programmer could have screwed up when they wrote the code. - Jonathan M Davis
Oct 17 2010
parent reply "Nick Sabalausky" <a a.a> writes:
-- 

-------------------------------
Not sent from an iPhone.

"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.685.1287377033.858.digitalmars-d puremagic.com...
 On Sunday 17 October 2010 19:40:36 bearophile wrote:
 In Bugzilla I have some compiler messages that don't fit in the set of
 error messages nor in the set of normal warnings. Like:
 http://d.puremagic.com/issues/show_bug.cgi?id=5056

 So I have proposed a third kind of compiler message, that are
 tips/suggestions to avoid possible bugs, improve the shape or performance
 of the code, and so on. The compiler shows those tips only on request.

 My usage of C lints has shown me that such tips sometimes are very useful
 to avoid coding pitfalls, etc.

 Here you may see some simple example of compiler tips from the 
 CommonLips,
 that the compiler shows only when the higher optimization levels are 
 used:
 http://shootout.alioth.debian.org/u32/program.php?test=mandelbrot&lang=sbc
 l&id=1

 Bye,
 bearophile

I think that what we should really look at doing is to have either dmd or some other tool built on dmd's front-end be able to do lint-like examination of source code (if it were dmd itself rather than an external tool, it would be a flag which presumably did that only rather than compile your code). That way, all of the various warning and error messages that you and others keep coming up with but will never end up in dmd can be put in there. That tool could provide a way to find all of the weird, stray cases that could result in errors, and anyone who wants to examine their code on that level could use it. Then we wouldn't have all of these potential warnings sitting around which Walter will never put into dmd. And honestly, I think that things like dangling else, unused variables, and redundant code make a lot more sense it a tool that is looking for potential programming errors rather than the compiler. It's the compiler's job to correctly generate the code that it's given, not worry about all of the possible ways that the programmer could have screwed up when they wrote the code. - Jonathan M Davis

The problem I've always seen with lint tools is you have two choices: A. Run them occasionally, or B. Set up the build system to run them on every compile. So: If you do A, you minimize the usefulness because you're not informed of potential issues until likely well after they've been introduced into the code. If you do B, then you may as well just have them built right into the compiler. And, in fact, outside of planet Mars, that's what's done: We call them "warnings".
Oct 17 2010
parent bearophile <bearophileHUGS lycos.com> writes:
Jonathan M Davis:

 The problem is that there are a lot of things that you can warn about which 
 aren't necessarily a problem, and it can be really annoying to have to fix a 
 bunch of useless warnings. If it really matters, then it should be an error.
And 
 if it doesn't really matter, then do we need to warn about?

This is a quite manichaean view of the programming world, but life isn't like that, it comes in shades of grey. I suggest you to read the book "Fuzzy Thinking: The New Science of Fuzzy Logic" by Bart Kosko, to help your mind grow up a bit. And sometimes even when the situation is clear for a human, a less smart compiler may be unsure, so it has to give a probability weighting of the situation. Some lint tools literally partition their messages in classes based on their truth probability. And then, keep in mind I was not talking about warnings in this post, but more about compiler tips. A performance tip, even when it's 100% certain, is not something you must listen to. It may be useful if you are performance-tuning a function, otherwise you may ignore it. This is why I have said they are a third kind of compiler message.
 I'm not entirely against warnings, but I do think that there are a lot of
things 
 that keep getting proposed as warnings (particularly by Bearophile) which are 
 far more likely to be annoying to fix non-problems than to actually be a
problem 
 and that such warnings would be far better suited to a lint-like tool than the 
 compiler.

There are two things to say about lint tools: - They are really really useful if you write C code. And I think they may be useful for D code too. - Lot of people aren't using them. I know older programmers that didn't even know what a lint is. In my opinion one way to avoid this is to put some lint-capabilities inside the compiler. Putting a whole lint inside the compiler is probably not a good idea, because they a good lint is complex. So I think it's a good thing to put some simple lint functionality inside the dmd compiler. In Bugzilla I have added some enhancement requests about this, that have received many comments from people. Bye, bearophile
Oct 18 2010
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday 17 October 2010 22:00:34 Nick Sabalausky wrote:
 And honestly, I think that things like dangling else, unused variables,
 and
 redundant code make a lot more sense it a tool that is looking for
 potential
 programming errors rather than the compiler. It's the compiler's job to
 correctly generate the code that it's given, not worry about all of the
 possible
 ways that the programmer could have screwed up when they wrote the code.
 
 - Jonathan M Davis

The problem I've always seen with lint tools is you have two choices: A. Run them occasionally, or B. Set up the build system to run them on every compile. So: If you do A, you minimize the usefulness because you're not informed of potential issues until likely well after they've been introduced into the code. If you do B, then you may as well just have them built right into the compiler. And, in fact, outside of planet Mars, that's what's done: We call them "warnings".

The problem is that there are a lot of things that you can warn about which aren't necessarily a problem, and it can be really annoying to have to fix a bunch of useless warnings. If it really matters, then it should be an error. And if it doesn't really matter, then do we need to warn about? And ultimately, any good programmer is going to fix any warnings, so having much in the way of useless warnings is going to either be really annoying or cause warnings to be outright ignored. I'm not entirely against warnings, but I do think that there are a lot of things that keep getting proposed as warnings (particularly by Bearophile) which are far more likely to be annoying to fix non-problems than to actually be a problem and that such warnings would be far better suited to a lint-like tool than the compiler. - Jonathan M Davis
Oct 17 2010
parent reply "Nick Sabalausky" <a a.a> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.686.1287379375.858.digitalmars-d puremagic.com...
 On Sunday 17 October 2010 22:00:34 Nick Sabalausky wrote:
 And honestly, I think that things like dangling else, unused variables,
 and
 redundant code make a lot more sense it a tool that is looking for
 potential
 programming errors rather than the compiler. It's the compiler's job to
 correctly generate the code that it's given, not worry about all of the
 possible
 ways that the programmer could have screwed up when they wrote the 
 code.

 - Jonathan M Davis

The problem I've always seen with lint tools is you have two choices: A. Run them occasionally, or B. Set up the build system to run them on every compile. So: If you do A, you minimize the usefulness because you're not informed of potential issues until likely well after they've been introduced into the code. If you do B, then you may as well just have them built right into the compiler. And, in fact, outside of planet Mars, that's what's done: We call them "warnings".

The problem is that there are a lot of things that you can warn about which aren't necessarily a problem, and it can be really annoying to have to fix a bunch of useless warnings.

Yea, maybe I oversimplified (I was in a hurry), but there are a lot of things out there that just simply don't fit into the "ok"/"bad" dichotomy. Fortunately, that's already been solved with multiple warning levels: - Errors: For things that are just plain wrong: Always on, fatal. - Warnings A: For things that are likely problems, but are acceptable temporarily (and only temporarily), ie, until you're ready to make a commit: Always on, nonfatal. - Warnings B: For things that may indicate a problem (and therefore are useful to be notified of), but may also be perfectly correct: Normally off, but occasionally turn on to check. - Warnings C: For things that may indicate a problem, but take a significant amount of processing time: Normally off, but occasionally turn on to check, possibly as a background cron job. Often, lint tools are used to handle "B" and "C". But, and this is my other point, it's not as if there're really any difference between (pure examples, not proposed syntax):
 dlint --pedantic foo.d    # <-- "B"

 dmd --pedantic foo.d    # <-- "B"

or:
 dlint --deep-analysis foo.d    # <-- "C"

 dmd --deep-analysis foo.d    # <-- "C"

But then, I've never really been one to buy into any of those unixy "purity" ideals.
Oct 18 2010
parent Rainer Deyke <rainerd eldwood.com> writes:
On 10/18/2010 02:44, Nick Sabalausky wrote:
 - Warnings A: For things that are likely problems, but are acceptable 
 temporarily (and only temporarily), ie, until you're ready to make a commit: 
 Always on, nonfatal.
 
 - Warnings B: For things that may indicate a problem (and therefore are 
 useful to be notified of), but may also be perfectly correct: Normally off, 
 but occasionally turn on to check.
 
 - Warnings C: For things that may indicate a problem, but take a significant 
 amount of processing time: Normally off, but occasionally turn on to check, 
 possibly as a background cron job.

I'd really like to see a system that's not based on turning warnings on and off by type, but on hiding known warnings and only showing new warnings. Warnings of type B are only a problem because you see the same set of warnings each time you recompile, and this obscures the real (new) warnings. If you could explicitly mark individual type B warnings as expected, there wouldn't be any need to ever turn the entire category off. -- Rainer Deyke - rainerd eldwood.com
Oct 18 2010
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday 18 October 2010 04:39:00 bearophile wrote:
 Jonathan M Davis:
 The problem is that there are a lot of things that you can warn about
 which aren't necessarily a problem, and it can be really annoying to
 have to fix a bunch of useless warnings. If it really matters, then it
 should be an error. And if it doesn't really matter, then do we need to
 warn about?

This is a quite manichaean view of the programming world, but life isn't like that, it comes in shades of grey. I suggest you to read the book "Fuzzy Thinking: The New Science of Fuzzy Logic" by Bart Kosko, to help your mind grow up a bit. And sometimes even when the situation is clear for a human, a less smart compiler may be unsure, so it has to give a probability weighting of the situation. Some lint tools literally partition their messages in classes based on their truth probability. And then, keep in mind I was not talking about warnings in this post, but more about compiler tips. A performance tip, even when it's 100% certain, is not something you must listen to. It may be useful if you are performance-tuning a function, otherwise you may ignore it. This is why I have said they are a third kind of compiler message.
 I'm not entirely against warnings, but I do think that there are a lot of
 things that keep getting proposed as warnings (particularly by
 Bearophile) which are far more likely to be annoying to fix non-problems
 than to actually be a problem and that such warnings would be far better
 suited to a lint-like tool than the compiler.

There are two things to say about lint tools: - They are really really useful if you write C code. And I think they may be useful for D code too. - Lot of people aren't using them. I know older programmers that didn't even know what a lint is. In my opinion one way to avoid this is to put some lint-capabilities inside the compiler. Putting a whole lint inside the compiler is probably not a good idea, because they a good lint is complex. So I think it's a good thing to put some simple lint functionality inside the dmd compiler. In Bugzilla I have added some enhancement requests about this, that have received many comments from people. Bye, bearophile

I, for one, want the compiler to tell you about things that are either absolutely guaranteed to be a problem or things which are _highly_ likely to be a problem. Anything less that doesn't belong in the compiler IMHO. If it's in the compiler, then it's going to be bugging me every time that I compile. If it's in a separate tool (or in a totally separate mode used by the compiler which isn't normal compilation), then I can use it when I want to be more thorough, but I do _not_ want to have the compiler throwing warnings at me left and right for things that _might_ be a problem. I much prefer it will the compiler restricts itself to things that are guaranteed to be problems (and perhaps some things that are _almost_ guaranteed). A lot of the kinds of warnings (or tips) that you like to suggest are exactly the kind of things that I don't think that compiler should be complaining about. Obviously, you don't agree, but that's how I see things. - Jonathan M Davis
Oct 18 2010
parent reply "Nick Sabalausky" <a a.a> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.693.1287403175.858.digitalmars-d puremagic.com...
 I, for one, want the compiler to tell you about things that are either
 absolutely guaranteed to be a problem or things which are _highly_ likely 
 to be
 a problem. Anything less that doesn't belong in the compiler IMHO. If it's 
 in
 the compiler, then it's going to be bugging me every time that I compile.

There are these things called "command line options", maybe you've heard of them?
Oct 18 2010
parent reply Don <nospam nospam.com> writes:
Nick Sabalausky wrote:
 "Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
 news:mailman.693.1287403175.858.digitalmars-d puremagic.com...
 I, for one, want the compiler to tell you about things that are either
 absolutely guaranteed to be a problem or things which are _highly_ likely 
 to be
 a problem. Anything less that doesn't belong in the compiler IMHO. If it's 
 in
 the compiler, then it's going to be bugging me every time that I compile.

There are these things called "command line options", maybe you've heard of them?

Maybe you've not heard of what a problem they are in C++? Ever had to turn individual warnings on and off just to get some different libraries to compile? The problem is, once you have an "optional warning" in a compiler, they are NOT optional. All standard or pseudo-standard libraries MUST comply with them. And if you have an idiotic warning that keeps complaining about perfectly valid code (VC++ for example has many such warnings), what you've done is reduce the quality of everyone's code everywhere. IMHO, it's extremely unprofessional for the compiler to cry wolf all the time, rather than to clearly identify the symptoms of genuine bugs.
Oct 18 2010
next sibling parent "Nick Sabalausky" <a a.a> writes:
"Don" <nospam nospam.com> wrote in message 
news:i9ic4k$eh4$1 digitalmars.com...
 Nick Sabalausky wrote:
 "Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
 news:mailman.693.1287403175.858.digitalmars-d puremagic.com...
 I, for one, want the compiler to tell you about things that are either
 absolutely guaranteed to be a problem or things which are _highly_ 
 likely to be
 a problem. Anything less that doesn't belong in the compiler IMHO. If 
 it's in
 the compiler, then it's going to be bugging me every time that I 
 compile.

There are these things called "command line options", maybe you've heard of them?

Maybe you've not heard of what a problem they are in C++? Ever had to turn individual warnings on and off just to get some different libraries to compile? The problem is, once you have an "optional warning" in a compiler, they are NOT optional. All standard or pseudo-standard libraries MUST comply with them. And if you have an idiotic warning that keeps complaining about perfectly valid code (VC++ for example has many such warnings), what you've done is reduce the quality of everyone's code everywhere. IMHO, it's extremely unprofessional for the compiler to cry wolf all the time, rather than to clearly identify the symptoms of genuine bugs.

See my earlier post on warning levels. There's no reason for library compliance, even on a std lib, to be mandatory on "B" and "C" types. And std lib compliance on "A" types isn't problematic.
Oct 18 2010
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Don:

 The problem is, once you have an "optional warning" in a compiler, they 
 are NOT optional. All standard or pseudo-standard libraries MUST comply 
 with them.

The topic of this thread is about tips, compiler comments that are meant to be kept usually disabled that are useful to avoid bugs or improve code performance, etc. When you include other D libraries you keep them disabled, because you don't care to know that in your code there are 35 functions where using an Appender may improve code performance a tiny bit. Such tips are useful only in some situations, like when you look for bugs in a function or when you need to tune its performance. Bye, bearophile
Oct 18 2010
parent Don <nospam nospam.com> writes:
bearophile wrote:
 Don:
 
 The problem is, once you have an "optional warning" in a compiler, they 
 are NOT optional. All standard or pseudo-standard libraries MUST comply 
 with them.

The topic of this thread is about tips, compiler comments that are meant to be kept usually disabled that are useful to avoid bugs or improve code performance, etc. When you include other D libraries you keep them disabled, because you don't care to know that in your code there are 35 functions where using an Appender may improve code performance a tiny bit. Such tips are useful only in some situations, like when you look for bugs in a function or when you need to tune its performance.

I know. But such features are usable only if the libraries cooperate.
Oct 19 2010
prev sibling parent reply Rainer Deyke <rainerd eldwood.com> writes:
On 10/18/2010 14:53, Don wrote:
 The problem is, once you have an "optional warning" in a compiler, they
 are NOT optional. All standard or pseudo-standard libraries MUST comply
 with them.

Libraries are assumed to be fully debugged, therefore they should always be compiled with all warnings turned off. Therefore libraries don't have to comply with ANY warnings. (And if your compiler doesn't allow you to specify which files are library files, then the problem is with the compiler.) -- Rainer Deyke - rainerd eldwood.com
Oct 18 2010
parent reply Don <nospam nospam.com> writes:
Rainer Deyke wrote:
 On 10/18/2010 14:53, Don wrote:
 The problem is, once you have an "optional warning" in a compiler, they
 are NOT optional. All standard or pseudo-standard libraries MUST comply
 with them.

Libraries are assumed to be fully debugged, therefore they should always be compiled with all warnings turned off. Therefore libraries don't have to comply with ANY warnings. (And if your compiler doesn't allow you to specify which files are library files, then the problem is with the compiler.)

It's not obvious to me how that can be done in the presence of templates. Although it's easy to distinguish between library and non-library *files*, it's not at all easy to distinguish between library and non-library *code*.
Oct 19 2010
parent reply Rainer Deyke <rainerd eldwood.com> writes:
On 10/19/2010 01:31, Don wrote:
 It's not obvious to me how that can be done in the presence of
 templates. Although it's easy to distinguish between library and
 non-library *files*, it's not at all easy to distinguish between library
 and non-library *code*.

Simple. If the template is in a library file, it's library code, regardless of where it was instantiated. Rationale: - If the warning is triggered on every instantiation of the template, then it's obviously the library's problem. - If the warning is triggered on only some template instantiations, the warning is probably spurious. Template instantiations often generate code that would look wrong in isolation, but is actually correct. - If you really want to catch those extra warnings, you can always turn on warnings for library code and shift through the results. You could add a compiler option to enable warning on templates in library code if the template was instantiated in user code, but I personally don't see the point. -- Rainer Deyke - rainerd eldwood.com
Oct 19 2010
parent reply Don <nospam nospam.com> writes:
Rainer Deyke wrote:
 On 10/19/2010 01:31, Don wrote:
 It's not obvious to me how that can be done in the presence of
 templates. Although it's easy to distinguish between library and
 non-library *files*, it's not at all easy to distinguish between library
 and non-library *code*.

Simple. If the template is in a library file, it's library code, regardless of where it was instantiated.

The separation isn't clean. User code instantiates library code which instantiates user code. Look at std.algorithm, for example. Mixins and template alias parameters blur things even further.
Oct 19 2010
parent reply Rainer Deyke <rainerd eldwood.com> writes:
On 10/19/2010 05:44, Don wrote:
 Rainer Deyke wrote:
 Simple.  If the template is in a library file, it's library code,
 regardless of where it was instantiated.

The separation isn't clean. User code instantiates library code which instantiates user code. Look at std.algorithm, for example. Mixins and template alias parameters blur things even further.

There are exactly four possible situations: - Template in library code instantiated by library code. - Template in library code instantiated by user code. - Template in user code instantiated by library code. - Template in user code instantiated by user code. These cases can be nested arbitrarily deep, but that doesn't add additional cases. If we categorically ignore where a template is instantiated, the four cases reduce to just two: - Template in library code. - Template in user code. -- Rainer Deyke - rainerd eldwood.com
Oct 19 2010
parent reply Don <nospam nospam.com> writes:
Rainer Deyke wrote:
 On 10/19/2010 05:44, Don wrote:
 Rainer Deyke wrote:
 Simple.  If the template is in a library file, it's library code,
 regardless of where it was instantiated.

instantiates user code. Look at std.algorithm, for example. Mixins and template alias parameters blur things even further.

There are exactly four possible situations: - Template in library code instantiated by library code. - Template in library code instantiated by user code. - Template in user code instantiated by library code. - Template in user code instantiated by user code. These cases can be nested arbitrarily deep, but that doesn't add additional cases. If we categorically ignore where a template is instantiated, the four cases reduce to just two: - Template in library code. - Template in user code.

Don't forget that delegate literals can be passed as template alias parameters; so can expressions. Problems get even more extreme when mixins are involved. Example: Suppose we have warnings for possible numeric overflow. Then consider: sort!"a*5<b"(x); We should have a warning that 'a*5' may cause a numeric overflow. How can that be done?
Oct 19 2010
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Don:

 We should have a warning that 'a*5' may cause a numeric overflow. How 
 can that be done?

Once the D2 compiler is able to test for (separated) signed/unsigned overflows both at compile and run-time, then the compile-time ones are hard errors, and the run-time ones are exceptions (in CTFE they may be CT exceptions). After that basic safety net functionality is present, and D smells less of bugs than before, you may want the compiler, when it's unsure if an overflow may occur, to raise a warning instead, but I think this kind of warnings are not so important. Bye, bearophile
Oct 19 2010
prev sibling parent Rainer Deyke <rainerd eldwood.com> writes:
On 10/19/2010 18:20, Don wrote:
 Example: Suppose we have warnings for possible numeric overflow. Then
 consider:
 sort!"a*5<b"(x);
 We should have a warning that 'a*5' may cause a numeric overflow. How
 can that be done?

I'm not convinced that we should have a warning here. Validating template arguments is the 'sort' template's job. If the argument validates, then the compiler should accept it as valid unless it leads to an actual error during compilation. If this causes some valid warnings to be lost, so be it. It beats the alternative of forcing library authors to write warning-free code. If you are really paranoid, there's always the option of generating warnings on all template instantiations triggered (directly or indirectly) by user code. It can and will lead to spurious warnings, but that's the price you pay for paranoia. -- Rainer Deyke - rainerd eldwood.com
Oct 19 2010