www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - core.stdc.stdlib._compare_fp_t and qsort

reply Joe <jma freedomcircle.com> writes:
I'm getting a compiler error in a qsort() call as follows:

     qsort(recs, num_recs, (Record *).sizeof, compar);

Record is a struct, recs is a fixed array of pointers to Record's 
and num_recs is a size_t that holds the number of valid records.

compar is this:

int compar(const void *p1, const void *p2)
{
     import core.stdc.string : strcmp;
     const Record **rp1 = cast(Record **)p1;
     const Record **rp2 = cast(Record **)p2;

     return strcmp((*rp1).name.ptr, (*rp2).name.ptr);
}

The error is: Error: function testd.compar (const(void*) p1, 
const(void*) p2) is not callable using argument types ()

I don't quite understand what those parentheses mean: is it 
implying "no arguments" and if so, where would one provide 
arguments?

Joe
Mar 11 2018
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 11 March 2018 at 23:12:30 UTC, Joe wrote:

 I'm getting a compiler error in a qsort() call as follows:

     qsort(recs, num_recs, (Record *).sizeof, compar);

 Record is a struct, recs is a fixed array of pointers to 
 Record's and num_recs is a size_t that holds the number of 
 valid records.

 compar is this:

 int compar(const void *p1, const void *p2)
 {
     import core.stdc.string : strcmp;
     const Record **rp1 = cast(Record **)p1;
     const Record **rp2 = cast(Record **)p2;

     return strcmp((*rp1).name.ptr, (*rp2).name.ptr);
 }

 The error is: Error: function testd.compar (const(void*) p1, 
 const(void*) p2) is not callable using argument types ()

 I don't quite understand what those parentheses mean: is it 
 implying "no arguments" and if so, where would one provide 
 arguments?

 Joe


You have to pass a pointer to the function.
Otherwise it'll be a parenthsis-less call.
use :  qsort(recs, num_recs, (Record *).sizeof, &compar);
Mar 11 2018
parent reply Joe <jma freedomcircle.com> writes:
On Sunday, 11 March 2018 at 23:26:04 UTC, Stefan Koch wrote:

 You have to pass a pointer to the function.
 Otherwise it'll be a parenthsis-less call.
 use :  qsort(recs, num_recs, (Record *).sizeof, &compar);


After passing a pointer, getting some other error messages, I 
changed the call to

     qsort(cast(void *)recs, num_recs, cast(size_t)(Record 
*).sizeof, &compar);

and the latest error is:

Error: function core.stdc.stdlib.qsort (scope void* base, ulong 
nmemb, ulong size, extern (C) int function(scope const(void*), 
scope const(void*))  system compar) is not callable using 
argument types (void*, ulong, ulong, int function(const(void*) 
p1, const(void*) p2))

I fail to see which argument is causing the problem now.

Joe
Mar 11 2018
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 12 March 2018 at 01:04:06 UTC, Joe wrote:

 and the latest error is:


D's error messages are so bad and shouldn't be hard to fix. It 
kills me that basic every-day functionality like this isn't a 
priority to the core devs. I even wrote a patch myself that would 
call this out but it didn't fit the code style so rejected. Ugh. 
But behold:

(scope void* base,
ulong nmemb,
ulong size,
extern (C) int function(scope const(void*), scope const(void*)) 
 system compar)

(void*,
ulong,
ulong,
int function(const(void*) p1, const(void*) p2))



I just reformatted it but now the difference should be visible: 
`extern(C)` is missing on your callback.

The scope things might make a difference too, but I know for sure 
extern(C) is necessary on your callback function.
Mar 11 2018
parent reply Joe <jma freedomcircle.com> writes:
On Monday, 12 March 2018 at 01:45:54 UTC, Adam D. Ruppe wrote:

 I just reformatted it but now the difference should be visible: 
 `extern(C)` is missing on your callback.

 The scope things might make a difference too, but I know for 
 sure extern(C) is necessary on your callback function.


I saw the extern(C) and I believe I tried it before my previous 
post, but dismissed it because I saw no difference in compiler 
behavior.  In any case declaring

extern (C) int compar(const (void *) p1, const (void *) p2) {
    ... // as before
}

still gives the error:

function core.stdc.stdlib.qsort (scope void* base, ulong nmemb, 
ulong size, extern (C) int function(scope const(void*), scope 
const(void*))  system compar) is not callable using argument 
types (void*, ulong, ulong, extern (C) int function(const(void*) 
p1, const(void*) p2))

It only went away with

extern (C) int compar(scope const (void *) p1, scope const (void 
*) p2)

Could you explain or direct me to something that elucidates why 
the "scope" qualifiers are needed?  And is there something else 
that is necessary inside compar() or are the "scope"s just for 
decorative purposes?

Joe
Mar 11 2018
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 12 March 2018 at 02:44:17 UTC, Joe wrote:

 I saw the extern(C) and I believe I tried it before my previous 
 post, but dismissed it because I saw no difference in compiler 
 behavior.


Yeah, the compiler should just tell you what specifically it is 
complaining about instead of making you hunt. These "no function 

and as a regular D tech support rep, I see there are very 
frequent problems for everyone else too.


 Could you explain or direct me to something that elucidates why 
 the "scope" qualifiers are needed?  And is there something else 
 that is necessary inside compar() or are the "scope"s just for 
 decorative purposes?


So `scope` means you promise not to escape the reference outside 
your scope; that you won't store the pointers somewhere else for 
future use.

It is defined here: 
https://dlang.org/spec/function.html#parameters tho that's pretty 
light.

The compiler only checks it if you throw some random 
-dipsomething switch. The idea is something similar to Rust's 
borrow checker. It aims to make it a compile error to sneak out 
pointers to temporaries through our callback. You wouldn't do 
that anyway with a compare function, but now it is an error to 
try ( if the strict enforcement thing is thrown. and i don't 
remember the name, they are all random dip numbers on the 
switches)

I'm actually mildly against adding these directly to C functions, 
but this attribute spam is being added to all the druntime 
definitions. It requires stuff like nothrow and nogc on a bunch 
of handlers too, ugh, but the idea behind it is to make sure you 
don't break the C function's documented rules, but here checked 
by the D compiler without needing any runtime wrappers/checks.



But until recently it wasn't checked at all which is why I wasn't 
sure if that was causing your error. I guess it is required to 
match the function call even if the strict enforcement isn't 
thrown now.
Mar 11 2018
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Mar 12, 2018 at 01:04:06AM +0000, Joe via Digitalmars-d-learn wrote:

 On Sunday, 11 March 2018 at 23:26:04 UTC, Stefan Koch wrote:

 You have to pass a pointer to the function.
 Otherwise it'll be a parenthsis-less call.
 use :  qsort(recs, num_recs, (Record *).sizeof, &compar);


 
 After passing a pointer, getting some other error messages, I changed
 the call to
 
     qsort(cast(void *)recs, num_recs, cast(size_t)(Record *).sizeof,
 &compar);
 
 and the latest error is:
 
 Error: function core.stdc.stdlib.qsort (scope void* base, ulong nmemb, ulong
 size, extern (C) int function(scope const(void*), scope const(void*))
  system compar) is not callable using argument types (void*, ulong, ulong,
 int function(const(void*) p1, const(void*) p2))
 
 I fail to see which argument is causing the problem now.


[...]

I wonder if the problem is because qsort expects a C linkage function
(`extern (C) int function(...)`), but you're passing a D linkage (i.e.,
native) function to it.  You can't do that, because the order of
arguments in a D linkage function may differ from a C linkage function,
and could potentially cause a subtle bug if you're not expecting the
arguments in a different order. (I actually ran into this myself, in a
different context where there was no type enforcement like there is
here, and it took me a long time to track down the problem.)

If indeed this is the problem, try adding `extern (C)` to your D
function to make it compatible with C linkage, and that should do the
trick.  I think.


T

-- 
If you look at a thing nine hundred and ninety-nine times, you are perfectly
safe; if you look at it the thousandth time, you are in frightful danger of
seeing it for the first time. -- G. K. Chesterton
Mar 11 2018
prev sibling parent reply Seb <seb wilzba.ch> writes:
On Sunday, 11 March 2018 at 23:12:30 UTC, Joe wrote:

 I'm getting a compiler error in a qsort() call as follows:

     qsort(recs, num_recs, (Record *).sizeof, compar);

 Record is a struct, recs is a fixed array of pointers to 
 Record's and num_recs is a size_t that holds the number of 
 valid records.

 compar is this:

 int compar(const void *p1, const void *p2)
 {
     import core.stdc.string : strcmp;
     const Record **rp1 = cast(Record **)p1;
     const Record **rp2 = cast(Record **)p2;

     return strcmp((*rp1).name.ptr, (*rp2).name.ptr);
 }

 The error is: Error: function testd.compar (const(void*) p1, 
 const(void*) p2) is not callable using argument types ()

 I don't quite understand what those parentheses mean: is it 
 implying "no arguments" and if so, where would one provide 
 arguments?

 Joe


Out of interest: I wonder what's your usecase for using qsort. Or 
in other words: why you can't use the high-level 
std.algorithm.sorting.sort?
Mar 11 2018
parent reply Joe <jma freedomcircle.com> writes:
On Monday, 12 March 2018 at 03:13:08 UTC, Seb wrote:

 Out of interest: I wonder what's your usecase for using qsort. 
 Or in other words: why you can't use the high-level 
 std.algorithm.sorting.sort?


This is only temporary. I will be using 
std.algorithm.sorting.sort. I was converting a C program and it 
annoyed me that I couldn't get the qsort invocation past the D 
compiler.
Mar 11 2018
parent reply Joe <jma freedomcircle.com> writes:
On Monday, 12 March 2018 at 03:50:42 UTC, Joe wrote:

 On Monday, 12 March 2018 at 03:13:08 UTC, Seb wrote:

 Out of interest: I wonder what's your usecase for using qsort. 
 Or in other words: why you can't use the high-level 
 std.algorithm.sorting.sort?


 This is only temporary. I will be using 
 std.algorithm.sorting.sort. I was converting a C program and it 
 annoyed me that I couldn't get the qsort invocation past the D 
 compiler.


Now that I'm trying to use std.algorithm.sorting, I'm again 
puzzled by what I need to use for the "less" predicate. My first 
try was:

    sort!((a, b) => to!string((*a).name) < 
to!string((*b).name))(recs);

This results in the error:

Error: template std.algorithm.sorting.sort cannot deduce function 
from argument types !((a, b) => to!string((*a).name) < 
to!string((*b).name))(Record*[10]), candidates are:
/usr/lib/ldc/x86_64-linux-gnu/include/d/std/algorithm/sorting.d(1851):       
std.algorithm.sorting.sort(alias less = "a < b", SwapStrategy ss =
SwapStrategy.unstable, Range)(Range r) if ((ss == SwapStrategy.unstable &&
(hasSwappableElements!Range || hasAssignableElements!Range) || ss !=
SwapStrategy.unstable && hasAssignableElements!Range) &&
isRandomAccessRange!Range && hasSlicing!Range && hasLength!Range)

which is not very helpful. It's a bit different from the "no 
function match" that Adam complains about, but I presume it's 
because now we're dealing with templates, and although the 
compiler finds a single candidate, it's not satisfactory but it 
can't tell the user anything further.

I've tried using fromStringz((*x).name.ptr) instead of to!string 
(I'm still unclear to what extent can templates be used within 
templates).  I also tried using an explicit cast(Record *)x 
because I'm also unsure that type information is passed down.  
Neither change helped.
Mar 18 2018
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On Sunday, 18 March 2018 at 11:29:47 UTC, Joe wrote:

 On Monday, 12 March 2018 at 03:50:42 UTC, Joe wrote:

 On Monday, 12 March 2018 at 03:13:08 UTC, Seb wrote:

 Out of interest: I wonder what's your usecase for using 
 qsort. Or in other words: why you can't use the high-level 
 std.algorithm.sorting.sort?


 This is only temporary. I will be using 
 std.algorithm.sorting.sort. I was converting a C program and 
 it annoyed me that I couldn't get the qsort invocation past 
 the D compiler.


 Now that I'm trying to use std.algorithm.sorting, I'm again 
 puzzled by what I need to use for the "less" predicate. My 
 first try was:

    sort!((a, b) => to!string((*a).name) < 
 to!string((*b).name))(recs);

 This results in the error:

 Error: template std.algorithm.sorting.sort cannot deduce 
 function from argument types !((a, b) => to!string((*a).name) < 
 to!string((*b).name))(Record*[10]),


This basically says you have fixed size array, in D they don’t 
decay to slices/pointers for safety reasons (it’s stack memory 
that is easy to leak out of scope with disasterous consequences).

Do this to get the usual ptr + length:

sort!((a, b) => to!string((*a).name) < 
to!string((*b).name))(recs[]);

Also to!string would be computed on each compare anew. May want 
to use schwartzSort to avoid that, on 10 elements there is no 
real difference though.
Mar 18 2018
parent reply Joe <jma freedomcircle.com> writes:
On Sunday, 18 March 2018 at 13:10:08 UTC, Dmitry Olshansky wrote:

 Do this to get the usual ptr + length:

 sort!((a, b) => to!string((*a).name) < 
 to!string((*b).name))(recs[]);

 Also to!string would be computed on each compare anew. May want 
 to use schwartzSort to avoid that, on 10 elements there is no 
 real difference though.


The 10 elements are just because it's a small test program.

What does changing "recs" to "recs[]" as the argument actually 
do?  Does it duplicate the fixed array on the fly? [I guess I 
have to study more!]

The change does pass the compiler, but at runtime it causes a 
segfault.  The next to last frame in the backtrace shows (having 
changed from to!string to fromStringz and using a string template 
instead of a lambda):


std.algorithm.sorting.sort!("fromStringz((*a).name.ptr) < 
fromStringz((*b).name.ptr)", 0, 
testd.Record*[]).sort(testd.Record*[]) (r=...)

Then it goes through quickSortImpl, shortSort and sort5, moving 
on to either std.functional.binaryFun or processing of the 
lambda, with a and b equal to 0, ending with a segfault in a ?? 
call from fromStringz or in memcpy called from object._dup (in 
the to!string case).
Mar 18 2018
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On Sunday, 18 March 2018 at 16:45:16 UTC, Joe wrote:

 On Sunday, 18 March 2018 at 13:10:08 UTC, Dmitry Olshansky 
 wrote:

 Do this to get the usual ptr + length:

 sort!((a, b) => to!string((*a).name) < 
 to!string((*b).name))(recs[]);

 Also to!string would be computed on each compare anew. May 
 want to use schwartzSort to avoid that, on 10 elements there 
 is no real difference though.


 The 10 elements are just because it's a small test program.

 What does changing "recs" to "recs[]" as the argument actually 
 do?  Does it duplicate the fixed array on the fly? [I guess I 
 have to study more!]


No it just creates a pair of pointer to recs[0] + length of recs, 
like this:

struct Array
{
   size_t length;
   Record* ptr;
}

In D it’s typed as Record[] and has a number of nice properties, 
like not losing its length;)


 The change does pass the compiler, but at runtime it causes a 
 segfault.  The next to last frame in the backtrace shows 
 (having changed from to!string to fromStringz and using a 
 string template instead of a lambda):


Well since recs is array of pointers this looks like a null 
pointer in your data.

The usual ways to fix that is either print stuff or poke around 
in debugger to see if a Record* is null or .name is null.



 std.algorithm.sorting.sort!("fromStringz((*a).name.ptr) < 
 fromStringz((*b).name.ptr)", 0, 
 testd.Record*[]).sort(testd.Record*[]) (r=...)


Also you don’t need (*a).name, D understands that . can access 
data through pointer (what would be an error in C and require ->).


 Then it goes through quickSortImpl, shortSort and sort5, moving 
 on to either std.functional.binaryFun or processing of the 
 lambda, with a and b equal to 0, ending with a segfault in a ?? 
 call from fromStringz or in memcpy called from object._dup (in 
 the to!string case).
Mar 18 2018
next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On Sunday, 18 March 2018 at 18:11:02 UTC, Dmitry Olshansky wrote:

 On Sunday, 18 March 2018 at 16:45:16 UTC, Joe wrote:

 [...]


 No it just creates a pair of pointer to recs[0] + length of 
 recs, like this:

 struct Array
 {
   size_t length;
   Record* ptr;
 }

 In D it’s typed as Record[] and has a number of nice 
 properties, like not losing its length;)


 [...]


 Well since recs is array of pointers this looks like a null 
 pointer in your data.


Or rather if 10 is capacity and num_recs is actual length, then:

recs[0..num_recs] is the slice with data that you want to sort.
Mar 18 2018
prev sibling parent reply Joe <jma freedomcircle.com> writes:
On Sunday, 18 March 2018 at 18:11:02 UTC, Dmitry Olshansky wrote:

 Well since recs is array of pointers this looks like a null 
 pointer in your data.

 The usual ways to fix that is either print stuff or poke around 
 in debugger to see if a Record* is null or .name is null.


The problem is that although the "recs" array is declared as 
having 10 pointers, not all pointers are used.  In the qsort 
case, the second argument limited the sort to just the first 
n_recs (valid) pointers. There doesn't seem to be a way to tell 
std.algorithm.sorting.sort to only look at part of the fixed 
array.

I managed to get it working by declaring a D dynamic array, 
appending n_recs pointers to it and using it as argument to sort. 
Unfortunately, I then had to copy from the dynamic array to the 
fixed array in order to continue using the latter. Any shortcuts 
around this?
Mar 18 2018
parent Joe <jma freedomcircle.com> writes:
On Sunday, 18 March 2018 at 19:01:11 UTC, Joe wrote:

 I managed to get it working by declaring a D dynamic array, 
 appending n_recs pointers to it and using it as argument to 
 sort. Unfortunately, I then had to copy from the dynamic array 
 to the fixed array in order to continue using the latter. Any 
 shortcuts around this?


I just saw your other reply.  Passing recs[0..n_recs] does the 
trick. Thanks.
Mar 18 2018