www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 9584] New: Exceptions in D are ludicrously slow (far worse than Java)

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9584

           Summary: Exceptions in D are ludicrously slow (far worse than
                    Java)
           Product: D
           Version: D2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: druntime
        AssignedTo: nobody puremagic.com
        ReportedBy: jmdavisProg gmx.com



PST ---
Take this D program

------
import std.conv;
import std.datetime;
import std.stdio;

void main()
{
    auto sw = StopWatch(AutoStart.yes);
    try
        throw new Exception("blah");
    catch(Exception e)
    {
        auto diff = to!Duration(sw.peek());
        writeln(diff.total!"hnsecs"());
        writeln(diff);
    }
}
------

and this Java program

------
class j
{
    public static final void main(String[] args)
    {
        long before = System.nanoTime();
        try
        {
            throw new Exception("blah");
        }
        catch(Exception e)
        {
            long diff = System.nanoTime() - before;
            long totalHNSecs = diff / 100;
            long msecs = diff / 1000000;
            diff -= msecs * 1000000;
            long usecs = diff / 1000;
            diff -= usecs * 1000;
            long hnsecs = diff / 100;
            System.out.println(totalHNSecs);
            System.out.printf("%sms %sus %shnsecs\n", msecs, usecs, hnsecs);
        }
    }
}
------

They are roughly equivalent, but their performance is drastically different
(and Java wins by a _long_ shot). These are typical outputs for the D version

13962
1 ms, 396 μs, and 2 hnsecs

13658
1 ms, 365 μs, and 8 hnsecs

13745
1 ms, 374 μs, and 5 hnsecs

whereas tese are typical outputs for the Java version

126
0ms 12us 6hnsecs

126
0ms 12us 6hnsecs

140
0ms 14us 0hnsecs

It varies a bit from execution to execution (particularly for D), but in
general, I'm seeing that D is taking 90x - 110x longer than Java is. Oddly,
once in a blue moon, D seems to take more like 20x longer, but even that's a
huge difference (and not in D's favor). In general, Java shouldn't be beating D
for performance, since D is a systems language, and a magnitute of 100x is
pathetic.

Granted, code should not be constantly throwing and catching exeptions, but
this sort of bad performance can make it so that exceptions have far more of a
negative impact then they should. The place that this is particularly negative
is in unit tests. Good unit tests will test that functions act appropriately
when given bad input, which generally means that they throw exceptions, and the
tests catch them, and right now, if you do much of that, unit tests start
taking a very long time to execute - unacceptably long. This makes it so that
code must be tested less in order for the tests to take a reasonable amount of
time, which will lead to worse code quality.

So, I think that it's clear that we need to take steps to improve the
performance of throwing and catching exceptions. As I understand it, the
stacktrace is currently put into string format when the exception is
constructed, which is unnecessary overhead if the stack trace never gets
printed, so that's one improvement that could be made. But I have no idea how
the internals of any of this currently work, so I don't know what other
improvements could be made. Regardless, we need to take steps to significantly
improve the situation.

Note that these tests were done on Linux. I don't know how the times compare on
Windows, though I expect that the situation is similar there.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 24 2013
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9584


bearophile_hugs eml.cc changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bearophile_hugs eml.cc




 So, I think that it's clear that we need to take steps to improve the
 performance of throwing and catching exceptions.


I agree.

My timings on Windows 32 bit:

D (latest DMD, -O -release -inline -noboundscheck):

1542590
154 ms and 259 us

69839
6 ms, 983 us, and 9 hnsecs

69741
6 ms, 974 us, and 1 hnsec

70468
7 ms, 46 us, and 8 hnsecs

70793
7 ms, 79 us, and 3 hnsecs

70539
7 ms, 53 us, and 9 hnsecs

69626
6 ms, 962 us, and 6 hnsecs


Java (version 1.7.0_13), same system:

260
0ms 26us 0hnsecs

243
0ms 24us 3hnsecs

241
0ms 24us 1hnsecs

241
0ms 24us 1hnsecs

235
0ms 23us 5hnsecs

231
0ms 23us 1hnsecs

250
0ms 25us 0hnsecs

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 24 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9584




Created an attachment (id=1194)
An old exception benchmark

In the attach there is an old exception benchmark I have kept, it compares the
same code in Java, C++, D, Python, C. On my system even Python is quite faster
than D, not just Java.

The C code uses longjmp/setjmp, it's not directly comparable, but it's useful
for a baseline data point.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 24 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9584




Exception handling is not intrinsically very slow. Basically there are two
function calls per stack frame, each of which only executes a handful of
instructions. All of the data that is accessed lies on the stack, so it should
be in cache. On Windows, there are a few system calls during stack unwinding,
but they are also extremely quick.

But even so, I'm a bit suspicious of that Java code. Is it really throwing an
exception? If it contains any exception handling optimisations at all, it would
optimize the throw/catch away.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 25 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9584







 I'm a bit suspicious of that Java code. Is it really throwing an
 exception? If it contains any exception handling optimisations at all, it would
 optimize the throw/catch away.


Maybe the benchmark in attach doesn't have this problem.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 25 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9584




PST ---

 But even so, I'm a bit suspicious of that Java code. Is it really throwing an
 exception? If it contains any exception handling optimisations at all, it
 would optimize the throw/catch away.


Hmm. A valid point. I hadn't thought of that. However, even just adding a line
to make it print the exception makes it take only about 250 - 350 microseconds
(which includes whatever cost there is to actually printing) on a machine
that's taking around 15 microseconds with the code in the bug, and if I make it
save the string from the exception before the end of the timing code but put
the print statement for the exception (now printing the saved string rather
than the exception) it's taking about 35 microseconds. Wrapping the throw in
another function doesn't seem to have much effect on performance either.

So, it may be that there's some optimizing going on with regards to the
exception in Java, but even with actually using it (which I would think would
force it to not be optimzed away), the D implementation is still taking orders
of magnitude longer than the Java implementation.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 25 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9584




Java does indeed do a lot of optimization of exceptions.

http://www.javaspecialists.eu/archive/Issue187.html

D does none. It's possible that this is what you're seeing. I can't yet tell if
there is a bug here or not.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 27 2013
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9584




PST ---

 D does none. It's possible that this is what you're seeing. I can't yet tell
 if there is a bug here or not.


There's a bug in the sense that D's exception performance sucks. If Java's
doing enough fancy optimizations, it may be that we can't match its performance
(especially if it's taking advantage of JVM stuff to do that), but unless there
is a fundamental reason why D's exceptions cannot be made faster, then they
should be. As it stands, it's ludicrously expensive to even just write unit
tests which verify that a function throws when it's supposed to, because the
cost of those exceptions quickly dwarfs everything else going on in the tests,
making them take orders of magnitude longer.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 27 2013