www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Nothrow, pure in druntime

reply dsimcha <dsimcha yahoo.com> writes:
Regarding ticket 14 in druntime:

http://www.dsource.org/projects/druntime/ticket/14

I've started looking at the best way to annotate all the pure/nothrow
functions in druntime.  A lot of druntime functionality calls functions from
the C standard library or the Windows/POSIX API.  I figure the best way to get
started is to deal w/ these first, and then worry about the higher level stuff
that's built on to of it.  These functions are obviously nothrow, since C
doesn't even support exceptions.  First of all, is there a decent automated
way to fix all of the declarations for these functions so that they're nothrow?

Secondly, since we don't have the sources to the C libraries for which the
function prototypes are declared, we can't be absolutely sure about their
purity.  A good portion of the ones that seem like they could be pure, mostly
math stuff like sqrt, log, etc., are probably messing w/ errno, and
theoretically they could be memoizing stuff, etc.  What's a reasonable course
of action for these?  It seems like pure is most useful for mathematical
functions, , so it would really be a shame not to use it here.
Jan 25 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
dsimcha wrote:
 I've started looking at the best way to annotate all the pure/nothrow
 functions in druntime.  A lot of druntime functionality calls functions from
 the C standard library or the Windows/POSIX API.  I figure the best way to get
 started is to deal w/ these first, and then worry about the higher level stuff
 that's built on to of it.  These functions are obviously nothrow, since C
 doesn't even support exceptions.  First of all, is there a decent automated
 way to fix all of the declarations for these functions so that they're nothrow?

I did think of making all functions that are extern(C) automatically nothrow, but was concerned that it would result in a lot of bugs and broken code from ones that did throw.
 Secondly, since we don't have the sources to the C libraries for which the
 function prototypes are declared, we can't be absolutely sure about their
 purity.  A good portion of the ones that seem like they could be pure, mostly
 math stuff like sqrt, log, etc., are probably messing w/ errno, and
 theoretically they could be memoizing stuff, etc.  What's a reasonable course
 of action for these?  It seems like pure is most useful for mathematical
 functions, , so it would really be a shame not to use it here.

Functions that modify errno cannot be pure. What needs to be done is go through the definition of each. For example, strlen, strcmp, etc. can be marked as pure. printf and strtok cannot. The math functions can be a problem since earlier incarnations often messed with global state, but with the advent of thread safe C runtime libraries, this shouldn't be an issue anymore.
Jan 25 2009
next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Walter Bright (newshound1 digitalmars.com)'s article
 dsimcha wrote:
 I've started looking at the best way to annotate all the pure/nothrow
 functions in druntime.  A lot of druntime functionality calls functions from
 the C standard library or the Windows/POSIX API.  I figure the best way to get
 started is to deal w/ these first, and then worry about the higher level stuff
 that's built on to of it.  These functions are obviously nothrow, since C
 doesn't even support exceptions.  First of all, is there a decent automated
 way to fix all of the declarations for these functions so that they're nothrow?

nothrow, but was concerned that it would result in a lot of bugs and broken code from ones that did throw.

I assume, when referring to the ones that do throw, you mean functions written in C++ or D, but declared w/ C linkage. If so, you could make this a per-module setting that defaults to not assuming nothrow. For example, let's say you made this pragma(Linkage, nothrow). Then, if this statement is seen at the top of a module, everything declared with extern(Linkage) in that module is assumed to be nothrow. For standard C, Windows and POSIX API functions and for any library written in pure C, I believe (correct me if I'm wrong) this would be a safe assumption. At any rate, it would make nothrow a heck of a lot more usable.
 Functions that modify errno cannot be pure. What needs to be done is go
 through the definition of each. For example, strlen, strcmp, etc. can be
 marked as pure. printf and strtok cannot.
 The math functions can be a problem since earlier incarnations often
 messed with global state, but with the advent of thread safe C runtime
 libraries, this shouldn't be an issue anymore.

Isn't errno defined in some implementations to be thread-local? If so, I guess we still have a problem. Then again, in the long run it probably makes sense to reimplement a lot of the math stuff that still uses the C std lib in pure D anyhow so that things like CTFE work on it, but in the sort run I'm sure that's not anyone's top priority.
Jan 25 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
dsimcha wrote:
 I assume, when referring to the ones that do throw, you mean functions written
in
 C++ or D, but declared w/ C linkage.  If so, you could make this a per-module
 setting that defaults to not assuming nothrow.  For example, let's say you made
 this pragma(Linkage, nothrow).  Then, if this statement is seen at the top of a
 module, everything declared with extern(Linkage) in that module is assumed to
be
 nothrow.  For standard C, Windows and POSIX API functions and for any library
 written in pure C, I believe (correct me if I'm wrong) this would be a safe
 assumption.  At any rate, it would make nothrow a heck of a lot more usable.

This will do it: nothrow: ... rest of module ...
 Isn't errno defined in some implementations to be thread-local?

Yes, but thread local doesn't mean pure.
 If so, I guess we
 still have a problem.  Then again, in the long run it probably makes sense to
 reimplement a lot of the math stuff that still uses the C std lib in pure D
anyhow
 so that things like CTFE work on it, but in the sort run I'm sure that's not
 anyone's top priority.

Don has already reimplemented most of them in D, this was done to: 1. ensure a minimum level of performance and accuracy; some C ones are crappily done 2. properly support all the D floating point types and overloading rules 3. support NAN and INFINITY correctly
Jan 25 2009
parent reply Don <nospam nospam.com> writes:
Walter Bright wrote:
 dsimcha wrote:
 I assume, when referring to the ones that do throw, you mean functions 
 written in
 C++ or D, but declared w/ C linkage.  If so, you could make this a 
 per-module
 setting that defaults to not assuming nothrow.  For example, let's say 
 you made
 this pragma(Linkage, nothrow).  Then, if this statement is seen at the 
 top of a
 module, everything declared with extern(Linkage) in that module is 
 assumed to be
 nothrow.  For standard C, Windows and POSIX API functions and for any 
 library
 written in pure C, I believe (correct me if I'm wrong) this would be a 
 safe
 assumption.  At any rate, it would make nothrow a heck of a lot more 
 usable.

This will do it: nothrow: ... rest of module ...
 Isn't errno defined in some implementations to be thread-local?

Yes, but thread local doesn't mean pure.
 If so, I guess we
 still have a problem.  Then again, in the long run it probably makes 
 sense to
 reimplement a lot of the math stuff that still uses the C std lib in 
 pure D anyhow
 so that things like CTFE work on it, but in the sort run I'm sure 
 that's not
 anyone's top priority.

Don has already reimplemented most of them in D, this was done to: 1. ensure a minimum level of performance and accuracy; some C ones are crappily done 2. properly support all the D floating point types and overloading rules 3. support NAN and INFINITY correctly

tango.math doesn't use the C library at all, except when inline asm is unavailable. Of they differ from the C functions, in that none of them set errno! One really annoying issue still remains, though -- the floating point flags in the CPU. They are entirely deterministic, but are they considered to be part of the return value of the function? Or would we allow them to be ignored? A compiler could check the exception flags before allowing memoisation. But one could also do the same thing for 'errno'. Likewise, floating point rounding modes. Essentially, the floating point status register is a hidden global variable, read from# and written to during every floating point operation. # - only the rounding mode and truncation affect the return value. We could deal with it by regarding that as a whole-program setting. But (depending on the CPU), the old exception flags generally get ORed with the new exception flags. Also, you can set the flags to allow any floating point function to throw a hardware exception. It's difficult for any function using floating point to claim to be nothrow under ANY circumstances; but that's a horrible limitation.
Jan 26 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Don wrote:
 tango.math doesn't use the C library at all, except when inline asm is 
 unavailable. Of they differ from the C functions, in that none of them 
 set errno!
 One really annoying issue still remains, though -- the floating point 
 flags in the CPU. They are entirely deterministic, but are they 
 considered to be part of the return value of the function? Or would we 
 allow them to be ignored?
 A compiler could check the exception flags before allowing memoisation. 
 But one could also do the same thing for 'errno'.
 
 Likewise, floating point rounding modes. Essentially, the floating point 
   status register is a hidden global variable, read from# and written to 
 during every floating point operation.
 
 # - only the rounding mode and truncation affect the return value. We 
 could deal with it by regarding that as a whole-program setting. But 
 (depending on the CPU), the old exception flags generally get ORed with 
 the new exception flags.

Those are good points. I don't know what the answer is. My inclination is to say if your program relies on changing the rounding mode or fiddles with the exception flags, it's undefined behavior.
 Also, you can set the flags to allow any floating point function to 
 throw a hardware exception. It's difficult for any function using 
 floating point to claim to be nothrow under ANY circumstances; but 
 that's a horrible limitation.

I would say that is not supported by D. I've never heard of a use for them.
Jan 26 2009
parent reply Don <nospam nospam.com> writes:
Walter Bright wrote:
 Don wrote:
 tango.math doesn't use the C library at all, except when inline asm is 
 unavailable. Of they differ from the C functions, in that none of them 
 set errno!
 One really annoying issue still remains, though -- the floating point 
 flags in the CPU. They are entirely deterministic, but are they 
 considered to be part of the return value of the function? Or would we 
 allow them to be ignored?
 A compiler could check the exception flags before allowing 
 memoisation. But one could also do the same thing for 'errno'.

 Likewise, floating point rounding modes. Essentially, the floating 
 point   status register is a hidden global variable, read from# and 
 written to during every floating point operation.

 # - only the rounding mode and truncation affect the return value. We 
 could deal with it by regarding that as a whole-program setting. But 
 (depending on the CPU), the old exception flags generally get ORed 
 with the new exception flags.

Those are good points. I don't know what the answer is. My inclination is to say if your program relies on changing the rounding mode or fiddles with the exception flags, it's undefined behavior.

One form of error analysis is to run the program with different rounding modes, and compare the results. You can also use rounding modes to implement interval arithmetic, but this would normally be restricted to low-level functions. The rounding mode would not escape from those functions. I normally use the exception flags for debugging.
 Also, you can set the flags to allow any floating point function to 
 throw a hardware exception. It's difficult for any function using 
 floating point to claim to be nothrow under ANY circumstances; but 
 that's a horrible limitation.

I would say that is not supported by D. I've never heard of a use for them.

What happens if a nothrow function throws an exception? IMHO a satisfactory response would be to abort the program with an error message/ drop you into a debugger -- anyway, that's the only thing I use when running with FP exceptions enabled. I guess it's reasonable to argue that using the floating-point flags is sufficiently hard-core that pure and nothrow should pretend that they don't exist. Still, some functions (especially correctly-rounded floating-point i/o) go to a lot of trouble to support them. I have a suspicion that it's not worth the effort.
Jan 26 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Don wrote:
 Walter Bright wrote:
 Don wrote:
 tango.math doesn't use the C library at all, except when inline asm 
 is unavailable. Of they differ from the C functions, in that none of 
 them set errno!
 One really annoying issue still remains, though -- the floating point 
 flags in the CPU. They are entirely deterministic, but are they 
 considered to be part of the return value of the function? Or would 
 we allow them to be ignored?
 A compiler could check the exception flags before allowing 
 memoisation. But one could also do the same thing for 'errno'.

 Likewise, floating point rounding modes. Essentially, the floating 
 point   status register is a hidden global variable, read from# and 
 written to during every floating point operation.

 # - only the rounding mode and truncation affect the return value. We 
 could deal with it by regarding that as a whole-program setting. But 
 (depending on the CPU), the old exception flags generally get ORed 
 with the new exception flags.

Those are good points. I don't know what the answer is. My inclination is to say if your program relies on changing the rounding mode or fiddles with the exception flags, it's undefined behavior.

One form of error analysis is to run the program with different rounding modes, and compare the results. You can also use rounding modes to implement interval arithmetic, but this would normally be restricted to low-level functions. The rounding mode would not escape from those functions. I normally use the exception flags for debugging.
 Also, you can set the flags to allow any floating point function to 
 throw a hardware exception. It's difficult for any function using 
 floating point to claim to be nothrow under ANY circumstances; but 
 that's a horrible limitation.

I would say that is not supported by D. I've never heard of a use for them.

What happens if a nothrow function throws an exception? IMHO a satisfactory response would be to abort the program with an error message/ drop you into a debugger -- anyway, that's the only thing I use when running with FP exceptions enabled.

I'd be ok with saying throwing fp exceptions is a non-recoverable error, like a seg fault or stack overflow, and is acceptable in a nothrow function.
 I guess it's reasonable to argue that using the floating-point flags is 
 sufficiently hard-core that pure and nothrow should pretend that they 
 don't exist.
 
 Still, some functions (especially correctly-rounded floating-point i/o) 
 go to a lot of trouble to support them.  I have a suspicion that it's 
 not worth the effort.

So we have two options. One is to say that floating point arithmetic cannot be made pure. The other is to ignore the problem (saying it's undefined behavior).
Jan 26 2009
next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2009-01-26 14:21:18 -0500, Walter Bright <newshound1 digitalmars.com> said:

 I guess it's reasonable to argue that using the floating-point flags is 
 sufficiently hard-core that pure and nothrow should pretend that they 
 don't exist.
 
 Still, some functions (especially correctly-rounded floating-point i/o) 
 go to a lot of trouble to support them.  I have a suspicion that it's 
 not worth the effort.

So we have two options. One is to say that floating point arithmetic cannot be made pure. The other is to ignore the problem (saying it's undefined behavior).

I see another: the compiler keeps track of the state of floating point flags inside a function, and prevent pure optimisations across those boundaries. Of course, the language would have to expose a mean to change these flags in order for it to work... and prevent any such change from leaking after the lifetime of a function. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Jan 26 2009
prev sibling parent Don <nospam nospam.com> writes:
Walter Bright wrote:
 Don wrote:
 Walter Bright wrote:
 Don wrote:
 tango.math doesn't use the C library at all, except when inline asm 
 is unavailable. Of they differ from the C functions, in that none of 
 them set errno!
 One really annoying issue still remains, though -- the floating 
 point flags in the CPU. They are entirely deterministic, but are 
 they considered to be part of the return value of the function? Or 
 would we allow them to be ignored?
 A compiler could check the exception flags before allowing 
 memoisation. But one could also do the same thing for 'errno'.

 Likewise, floating point rounding modes. Essentially, the floating 
 point   status register is a hidden global variable, read from# and 
 written to during every floating point operation.

 # - only the rounding mode and truncation affect the return value. 
 We could deal with it by regarding that as a whole-program setting. 
 But (depending on the CPU), the old exception flags generally get 
 ORed with the new exception flags.

Those are good points. I don't know what the answer is. My inclination is to say if your program relies on changing the rounding mode or fiddles with the exception flags, it's undefined behavior.

One form of error analysis is to run the program with different rounding modes, and compare the results. You can also use rounding modes to implement interval arithmetic, but this would normally be restricted to low-level functions. The rounding mode would not escape from those functions. I normally use the exception flags for debugging.
 Also, you can set the flags to allow any floating point function to 
 throw a hardware exception. It's difficult for any function using 
 floating point to claim to be nothrow under ANY circumstances; but 
 that's a horrible limitation.

I would say that is not supported by D. I've never heard of a use for them.

What happens if a nothrow function throws an exception? IMHO a satisfactory response would be to abort the program with an error message/ drop you into a debugger -- anyway, that's the only thing I use when running with FP exceptions enabled.

I'd be ok with saying throwing fp exceptions is a non-recoverable error, like a seg fault or stack overflow, and is acceptable in a nothrow function.

There are some important uses for catching fp exceptions, including one which in the revised IEEE standard: the product of an array of doubles. eg real.max * real.max * real.min * real.min == 1.0, but you get overflows during the calculation. What you do is enable floating point overflow and underflow as an exception, and set up an exception handler inside that function; the handler is essentially an inner function. Then you have a seperate counter for how many times it has overflowed. The thing to note about this is that although FP exceptions are used, they never leave the function which created them, so there's no stack unwinding, no destructors are called, etc. All functionality would be preserved if it is a non-recoverable error to transmit an fp exception across a function boundary (but possible to catch the fp exception in the function which generated it). But possibly such situations are so small in number that they can all be put in the standard library.
 I guess it's reasonable to argue that using the floating-point flags 
 is sufficiently hard-core that pure and nothrow should pretend that 
 they don't exist.

 Still, some functions (especially correctly-rounded floating-point 
 i/o) go to a lot of trouble to support them.  I have a suspicion that 
 it's not worth the effort.

So we have two options. One is to say that floating point arithmetic cannot be made pure. The other is to ignore the problem (saying it's undefined behavior).

It might not be a difficulty to ignore the problem. The flags are getting very hard to use these days, since the x87 flags are different from the SSE flags. I would recommend stating that when calling a pure function, the state of the FPU flags is implementation-dependent. Even better would be if we can state that the built-in math functions will respect the flags, despite being pure. They are very unlikely to be memoized since they are so simple.
Jan 27 2009
prev sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2009-01-25 14:39:52 -0500, Walter Bright <newshound1 digitalmars.com> said:

 I did think of making all functions that are extern(C) automatically 
 nothrow, but was concerned that it would result in a lot of bugs and 
 broken code from ones that did throw.

It's better that way, because it allows you to write proper wrappers for C++ functions that may throw exceptions. Like this: extern "C" int _doSomething_wrapper(int value) { try { return doSomething(value); } catch (const std::exception & e) { _d_throw_exception(e.what()); } } with _d_throw_exception written in D and throwing an exception. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Jan 26 2009