www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - I cannot understand problem with argument of the function

reply "mrd" <denis.feklushkin gmail.com> writes:
// Simple function called for unsigned integers:
static
ubyte[] packVarint(T)( T value )
if( isIntegral!T && isUnsigned!T )
out( arr )
{
     T d;
     size_t size = d.unpackVarint( &arr[0] );

     import std.stdio;
     import std.conv;

     writeln( "out contract, type=", typeid(T), " isUnsigned=", 
isUnsigned!T, " arg=", value, " result=", arr );
     stdout.flush;

     assert( size == arr.length );
     assert( d == value );
}
body
{
     import std.stdio;
     import std.conv;
     writeln( "value inside of body: ", value );
     stdout.flush;

     ubyte[] res;

     immutable ubyte maximal = 0b_1000_0000;

     while( value >= maximal )
     {
         res ~= cast( ubyte )( value | maximal );
         value >>= 7;
     }

     res ~= cast( ubyte ) value;

     return res;
}
unittest
{
     auto v = packVarint!ulong( 300 );
     assert( v.length == 2 );
     assert( v == [ 0b_10101100, 0b_00000010 ] );
}


output:
value inside of body: 1 // arg=1
out contract, type=uint isUnsigned=true arg=1 result=[1] // works 
for arg=1 !
value inside of body: 2
out contract, type=uint isUnsigned=true arg=2 result=[2]
value inside of body: 1
out contract, type=uint isUnsigned=true arg=1 result=[1]
value inside of body: 2
out contract, type=uint isUnsigned=true arg=2 result=[2]
value inside of body: 12
out contract, type=ulong isUnsigned=true arg=12 result=[12]
value inside of body: 30
out contract, type=ulong isUnsigned=true arg=30 result=[30]
value inside of body: 14
out contract, type=ulong isUnsigned=true arg=14 result=[14]
value inside of body: 30
out contract, type=ulong isUnsigned=true arg=30 result=[30]
value inside of body: 12
out contract, type=ulong isUnsigned=true arg=12 result=[12]
value inside of body: 30
out contract, type=ulong isUnsigned=true arg=30 result=[30]
value inside of body: 16
out contract, type=ulong isUnsigned=true arg=16 result=[16]
value inside of body: 32
out contract, type=ulong isUnsigned=true arg=32 result=[32]
value inside of body: 0
out contract, type=uint isUnsigned=true arg=0 result=[0]
value inside of body: 5
out contract, type=uint isUnsigned=true arg=5 result=[5]
value inside of body: 2
out contract, type=uint isUnsigned=true arg=2 result=[2]
value inside of body: 1
out contract, type=uint isUnsigned=true arg=1 result=[1]
value inside of body: 10
out contract, type=ulong isUnsigned=true arg=10 result=[10]
value inside of body: 30
out contract, type=ulong isUnsigned=true arg=30 result=[30]
value inside of body: 2
out contract, type=uint isUnsigned=true arg=2 result=[2]
value inside of body: 3
out contract, type=ulong isUnsigned=true arg=3 result=[3]
value inside of body: 30
out contract, type=ulong isUnsigned=true arg=30 result=[30]
value inside of body: 1 // argument=1
out contract, type=ulong isUnsigned=true arg=1 result=[1] // also 
successful result
value inside of body: 30
out contract, type=ulong isUnsigned=true arg=30 result=[30]
value inside of body: 3
out contract, type=ulong isUnsigned=true arg=3 result=[3]
value inside of body: 30
out contract, type=ulong isUnsigned=true arg=30 result=[30]
value inside of body: 18446744073709551615 // WTF???!!!
out contract, type=ulong isUnsigned=true arg=1 result=[255, 255, 
255, 255, 255, 255, 255, 255, 255, 1] // unsuccessful packing 
with previously many times used arg=1
core.exception.AssertError compression.pb_encoding(105): 
Assertion failure
----------------
./main(_d_assertm+0x16) [0x81b4316]
./main() [0x8179d47]
./main(ubyte[] 
compression.pb_encoding.packVarint!(ulong).packVarint(ulong)+0x149) 
[0x8179899]
./main(ubyte[] 
compression.pb_encoding.packVarint!(long).packVarint(inout(long))+0x18) 
[0x81796a8]
Sep 17 2013
parent reply "mrd" <denis.feklushkin gmail.com> writes:
(question is not about function body realisation - body just 
don't receives right argument value)
Sep 17 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Wednesday, 18 September 2013 at 04:17:41 UTC, mrd wrote:

 (question is not about function body realisation - body just 
 don't receives right argument value)


Looks like data corruption. Can't help you much without a full 
piece of code that *reproduces* the issue (reducing it helps too).

For what it's worth, your "weird" value, it appears to be a 
bitmask:
18446744073709551615
0x1999999999999999
0001_1001 1001_1001 1001_1001 1001_1001 1001_1001 1001_1001 
1001_1001 1001_1001

This is usually tell tale invalid memory reads and/or stack 
corruption.

TIP: Try marking your function  safe, if possible. It will limit 
the amount of unsafe things you can do, or at least, point you to 
the unsafe things that could be the source of the problem.
Sep 18 2013
parent reply "mrd" <denis.feklushkin gmail.com> writes:
On Wednesday, 18 September 2013 at 08:25:02 UTC, monarch_dodra 
wrote:

 On Wednesday, 18 September 2013 at 04:17:41 UTC, mrd wrote:

 (question is not about function body realisation - body just 
 don't receives right argument value)


 Looks like data corruption. Can't help you much without a full 
 piece of code that *reproduces* the issue (reducing it helps 
 too).


I will try to do it later. (I tried to copy these functions in a 
separate file - the problem is not reproduced.)


 For what it's worth, your "weird" value, it appears to be a 
 bitmask:
 18446744073709551615
 0x1999999999999999
 0001_1001 1001_1001 1001_1001 1001_1001 1001_1001 1001_1001 
 1001_1001 1001_1001

 This is usually tell tale invalid memory reads and/or stack 
 corruption.


In some other functions of project?


 TIP: Try marking your function  safe, if possible.


Isn't possible because active pointers usage in it
Sep 18 2013
parent reply "mrd" <denis.feklushkin gmail.com> writes:
On Wednesday, 18 September 2013 at 09:07:09 UTC, mrd wrote:


 Looks like data corruption. Can't help you much without a full 
 piece of code that *reproduces* the issue (reducing it helps 
 too).


 I will try to do it later. (I tried to copy these functions in 
 a separate file - the problem is not reproduced.)


this file:
http://pastebin.com/MCm5Yu7K
Sep 18 2013
next sibling parent "mrd" <denis.feklushkin gmail.com> writes:
On Wednesday, 18 September 2013 at 09:09:29 UTC, mrd wrote:

 On Wednesday, 18 September 2013 at 09:07:09 UTC, mrd wrote:


 Looks like data corruption.






I can not imagine what corruption was going on here: the fact 
that is contract gets the correct value, but the function itself 
- not. But they both should use same value source, yep?


 Can't help you much without a full piece of code that 
 *reproduces* the issue (reducing it helps too).
Sep 18 2013
prev sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Wednesday, 18 September 2013 at 09:09:29 UTC, mrd wrote:

 On Wednesday, 18 September 2013 at 09:07:09 UTC, mrd wrote:


 Looks like data corruption. Can't help you much without a 
 full piece of code that *reproduces* the issue (reducing it 
 helps too).


 I will try to do it later. (I tried to copy these functions in 
 a separate file - the problem is not reproduced.)


 this file:
 http://pastebin.com/MCm5Yu7K


I had to write "isMsbSet" myself. Also, you "main" doesn't 
compile:

//----
for( long i = -100; i < 100; i++ )
     packVarint( i );
//----

since "pack" only accepts unsigned.

After changing it to packVariant!ulong, it compiles. But it 
*could* explain the "weird" values you are seeing: -100 => 
18446744073709551516. I was wrong about the mask thing, that was 
just my windows calculator playing tricks on me.

Your out contracts also fail, because the body of your function 
modifies "value", but then, you use it in your contract. I had to 
change the body to use a copy of the passed in arguments, to 
verify them.

 From there, it *still* fails, because your unpackVarint is wrong. 
You store the result in a size_t, which immediatly overflows. The 
irony is you put a comment that says "big sized type used also 
for overflow checking". Why not use a T, or result directly?

Even, then, a problem remains that "data[i] & 0b_0111_1111" is of 
type int, causing overflow and sign mismatch for anything larger 
than int.max
Sep 18 2013
parent reply "mrd" <denis.feklushkin gmail.com> writes:
On Wednesday, 18 September 2013 at 09:56:26 UTC, monarch_dodra 
wrote:

 On Wednesday, 18 September 2013 at 09:09:29 UTC, mrd wrote:

 On Wednesday, 18 September 2013 at 09:07:09 UTC, mrd wrote:


 Looks like data corruption. Can't help you much without a 
 full piece of code that *reproduces* the issue (reducing it 
 helps too).


 I will try to do it later. (I tried to copy these functions 
 in a separate file - the problem is not reproduced.)


 this file:
 http://pastebin.com/MCm5Yu7K


 I had to write "isMsbSet" myself.


Omg, I am send wrong version of the file! Sorry!
Here is it: http://pastebin.com/h950BTyN


 Your out contracts also fail, because the body of your function 
 modifies "value", but then, you use it in your contract.


Is the contract does not uses its own version of arguments?
What is the point of passing earlier changed arguments into 
contract block?


 I had to change the body to use a copy of the passed in 
 arguments, to verify them.

 From there, it *still* fails, because your unpackVarint is 
 wrong. You store the result in a size_t, which immediatly 
 overflows. The irony is you put a comment that says "big sized 
 type used also for overflow checking". Why not use a T, or 
 result directly?

 Even, then, a problem remains that "data[i] & 0b_0111_1111" is 
 of type int, causing overflow and sign mismatch for anything 
 larger than int.max


Above that I'll think later, thanks! (It's amazing that this code 
passed all higher-level unit tests for several months.)
Sep 18 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Wednesday, 18 September 2013 at 10:37:29 UTC, mrd wrote:

 Is the contract does not uses its own version of arguments?
 What is the point of passing earlier changed arguments into 
 contract block?


That's a good question. I'll raise it on the main boards.
Sep 18 2013
parent reply "mrd" <denis.feklushkin gmail.com> writes:
On Wednesday, 18 September 2013 at 10:59:11 UTC, monarch_dodra 
wrote:

 On Wednesday, 18 September 2013 at 10:37:29 UTC, mrd wrote:

 Is the contract does not uses its own version of arguments?
 What is the point of passing earlier changed arguments into 
 contract block?


 That's a good question. I'll raise it on the main boards.


Here is simplified file that reproduces this bug (or not bug):

http://pastebin.com/0QRZcts8

$ ./bug
Inside of body: value=300 type=ulong
Inside of out contract: value=2 type=ulong result=[172, 2]
Sep 18 2013
parent reply "mrd" <denis.feklushkin gmail.com> writes:
This bug reproducible also without a template:

http://pastebin.com/QPvCFYL1

$ ./bug
works:
Inside of body: value=1 type=ulong
Inside of out contract: value=1 type=ulong result=[1]
not works:
Inside of body: value=300 type=ulong
Inside of out contract: value=2 type=ulong result=[172, 2]
Sep 18 2013
parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Wednesday, 18 September 2013 at 21:13:27 UTC, mrd wrote:

 This bug reproducible also without a template:

 http://pastebin.com/QPvCFYL1

 $ ./bug
 works:
 Inside of body: value=1 type=ulong
 Inside of out contract: value=1 type=ulong result=[1]
 not works:
 Inside of body: value=300 type=ulong
 Inside of out contract: value=2 type=ulong result=[172, 2]


Well, as we have already concluded, you can't mutate your input 
arguments if you wish to use them again in your out contract.
Sep 19 2013