www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Article: Fuzzing D code with LDC

reply Johan Engelen <j j.nl> writes:
It's been a work-in-progress for half a year, but finished now:

http://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html

"A not-so-well-written article about the fuzzing capability 
recently added to LDC, using LLVM’s libFuzzer. Compiling code 
with -fsanitize=fuzzer adds control-flow instrumentation used to 
guide the fuzzing and links-in the libFuzzer library that drives 
the fuzz testing (same as Clang). -fsanitize=fuzzer is available 
from LDC 1.4.0, not on Windows. LDC 1.6.0 was used for the 
examples in this article."

Hope the article gives you enough information to start fuzz 
testing your own projects.

cheers,
   Johan

(per Ali's request, the article has a nice fuzzy font for some of 
you :P)
Jan 13 2018
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Saturday, 13 January 2018 at 23:59:52 UTC, Johan Engelen wrote:
 It's been a work-in-progress for half a year, but finished now:

 http://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html

 "A not-so-well-written article about the fuzzing capability 
 recently added to LDC, using LLVM’s libFuzzer. Compiling code 
 with -fsanitize=fuzzer adds control-flow instrumentation used 
 to guide the fuzzing and links-in the libFuzzer library that 
 drives the fuzz testing (same as Clang). -fsanitize=fuzzer is 
 available from LDC 1.4.0, not on Windows. LDC 1.6.0 was used 
 for the examples in this article."

 Hope the article gives you enough information to start fuzz 
 testing your own projects.

 cheers,
   Johan

 (per Ali's request, the article has a nice fuzzy font for some 
 of you :P)
Nice post, enjoyed reading it. I need to look into trying out the shift sanitizer sometime.
Jan 14 2018
parent Johan Engelen <j j.nl> writes:
On Sunday, 14 January 2018 at 17:33:45 UTC, Joakim wrote:
 Nice post, enjoyed reading it.  I need to look into trying out 
 the shift sanitizer sometime.
Definitely would be *great* to have `-fsanitize=undefined` ! - Johan
Jan 14 2018
prev sibling next sibling parent Johan Engelen <j j.nl> writes:
On Saturday, 13 January 2018 at 23:59:52 UTC, Johan Engelen wrote:
 It's been a work-in-progress for half a year, but finished now:

 http://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html

 "A not-so-well-written article about the fuzzing capability 
 recently added to LDC
Argh, and just now I find out about: - https://www.youtube.com/watch?v=k-Cv8Q3zWNQ - https://github.com/google/oss-fuzz Anybody interested in looking at OSS Fuzz? "OSS-Fuzz - Continuous Fuzzing for Open Source Software Status: Beta. We are now accepting applications from widely-used open source projects." Cheers, Johan
Jan 14 2018
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/13/2018 3:59 PM, Johan Engelen wrote:
 It's been a work-in-progress for half a year, but finished now:
It's now on Reddit and Hackernews: https://www.reddit.com/r/programming/comments/7qbpa3/fuzzing_d_code_with_ldc/ https://news.ycombinator.com/item?id=16144860 (Don't click on the specific ycombinator link, or your upvotes will not be counted. Click on: https://news.ycombinator.com/news instead and look for it.) Also, such postings do a LOT better when they: 1. Include a synopsis of what the article is about. (I've added them.) 2. Include an "Ask me anything" posting from the author. Not doing these things means the impact of the article is severely blunted from what it could be.
Jan 14 2018
prev sibling next sibling parent reply Temtaime <temtaime gmail.com> writes:
On Saturday, 13 January 2018 at 23:59:52 UTC, Johan Engelen wrote:
 It's been a work-in-progress for half a year, but finished now:

 http://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html

 "A not-so-well-written article about the fuzzing capability 
 recently added to LDC, using LLVM’s libFuzzer. Compiling code 
 with -fsanitize=fuzzer adds control-flow instrumentation used 
 to guide the fuzzing and links-in the libFuzzer library that 
 drives the fuzz testing (same as Clang). -fsanitize=fuzzer is 
 available from LDC 1.4.0, not on Windows. LDC 1.6.0 was used 
 for the examples in this article."

 Hope the article gives you enough information to start fuzz 
 testing your own projects.

 cheers,
   Johan

 (per Ali's request, the article has a nice fuzzy font for some 
 of you :P)
// `Throwable`s thrown are not bugs (in contrast to `Errors`). They _can_ be bugs. class Error : Throwable class Exception : Throwable
Jan 15 2018
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, January 15, 2018 09:49:24 Temtaime via Digitalmars-d-announce 
wrote:
 On Saturday, 13 January 2018 at 23:59:52 UTC, Johan Engelen wrote:
 It's been a work-in-progress for half a year, but finished now:

 http://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html

 "A not-so-well-written article about the fuzzing capability
 recently added to LDC, using LLVM’s libFuzzer. Compiling code
 with -fsanitize=fuzzer adds control-flow instrumentation used
 to guide the fuzzing and links-in the libFuzzer library that
 drives the fuzz testing (same as Clang). -fsanitize=fuzzer is
 available from LDC 1.4.0, not on Windows. LDC 1.6.0 was used
 for the examples in this article."

 Hope the article gives you enough information to start fuzz
 testing your own projects.

 cheers,

   Johan

 (per Ali's request, the article has a nice fuzzy font for some
 of you :P)
// `Throwable`s thrown are not bugs (in contrast to `Errors`). They _can_ be bugs. class Error : Throwable class Exception : Throwable
Talking about Throwables always gets a bit weird. Exceptions aren't bugs, whereas Errors are, and we usually talk about Exceptions and Errors rather than Throwables, and talking about Throwables implies both, but you _can_ also throw Throwables that aren't derived from Exception or Error (be it a naked Throwable or a class derived from Throwable). You're asking for it if you do, since there really isn't much difference between a Throwable and an Error in terms of how the compiler and runtime deal with clean-up code and nothrow, so trying to use a Throwable that isn't an Exception or Error can work, but it's basically going to be treated like an Error. Really, as far as the language and compiler are concerned, you have Exceptions, and then you have all other Throwables. nothrow has to do with Exceptions, and whether clean-up code is guaranteed to run or not has to do with Exceptions. Error really doesn't come into it except that we derive from Error rather than Throwable to more easily distinguish them from Exception. Ultimately, Errors and other non-Exception Throwables are really treated the same. So, all-in-all, things just get weird when you try and talk about Throwables, and I'd generally advise against anyone trying to use any Throwables that aren't Exceptions or Errors. Certainly, if you do, you had better know what you're doing and be _very_ careful about it. - Jonathan M Davis
Jan 15 2018
prev sibling parent Johan Engelen <j j.nl> writes:
On Monday, 15 January 2018 at 09:49:24 UTC, Temtaime wrote:
 // `Throwable`s thrown are not bugs (in contrast to `Errors`).

 They _can_ be bugs.
 class Error : Throwable
 class Exception : Throwable
Good catch ( ;-) ) thanks. -Johan
Jan 15 2018
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, Jan 13, 2018 at 11:59:52PM +0000, Johan Engelen via
Digitalmars-d-announce wrote:
 It's been a work-in-progress for half a year, but finished now:
 
 http://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html
[...] It would be good to program custom fuzzers for Phobos modules, especially the ones with generic code like std.algorithm and std.range, a lot of which I suspect have unhandled edge cases and other such overlooked flaws. Fuzzing at this level will be harder than merely fuzzing with random data; as input we'd need higher-level objects like ranges of various types and by-reference / by-value semantics. And detecting problem cases may not be so easy (how to detect if, say, reduce() returned the wrong answer when the input is randomized?). T -- Beware of bugs in the above code; I have only proved it correct, not tried it. -- Donald Knuth
Jan 15 2018