www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - std.file.read returns void[] why?

reply "Spacen Jasset" <spacenjasset mailrazer.com> writes:
Why does the read function return void[] and not byte[]

void[] read(in char[] name, size_t upTo = size_t.max);
Apr 16 2014
parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Wed, 16 Apr 2014 14:36:20 +0100, Spacen Jasset  
<spacenjasset mailrazer.com> wrote:


 Why does the read function return void[] and not byte[]

 void[] read(in char[] name, size_t upTo = size_t.max);


One one hand the data is always /actually/ going to be a load of (u)bytes,  
but /conceptually/ it might be structs or something else and using void[]  
therefore doesn't /imply/ anything about what the data really is.

I also thought that void[] was implicitly cast.. but it seems this either  
has never been the case or was changed at some point:

import std.stdio;

void main(string[] args)
{
	byte[] barr = new byte[10];
	foreach(i, ref b; barr)
		b = cast(byte)('a' + i);
	
	void[] varr = barr;
	char[] carr;
	
	//carr = barr; // Error: cannot implicitly convert expression (barr) of  
type byte[] to char[]
	carr = cast(char[])barr;
	
	//carr = varr; // Error: cannot implicitly convert expression (varr) of  
type void[] to char[]
	carr = cast(char[])varr;
	
	writefln("%d,%s", carr.length, carr);
}

I am curious, was it ever possible, was it changed?  why?  It's always  
"safe" - as the compiler knows how much data the void[] contains, and  
void[] is "untyped" so it sorta makes sense to allow it..

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Apr 17 2014
next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
You can implicitly cast /to/ void[], but I don't think you could 
ever implicitly cast from it.

Casting from it needs a bit more thought because you should 
ensure that it actually is what you are saying it is and the 
compiler can't help with that, so by prohibiting the automatic 
cast it at least draws your attention to it.
Apr 17 2014
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 17 Apr 2014 07:57:35 -0400, Regan Heath <regan netmail.co.nz>  
wrote:


 On Wed, 16 Apr 2014 14:36:20 +0100, Spacen Jasset  
 <spacenjasset mailrazer.com> wrote:


 Why does the read function return void[] and not byte[]

 void[] read(in char[] name, size_t upTo = size_t.max);


 One one hand the data is always /actually/ going to be a load of  
 (u)bytes, but /conceptually/ it might be structs or something else and  
 using void[] therefore doesn't /imply/ anything about what the data  
 really is.

 I also thought that void[] was implicitly cast.. but it seems this  
 either has never been the case or was changed at some point:

 import std.stdio;

 void main(string[] args)
 {
 	byte[] barr = new byte[10];
 	foreach(i, ref b; barr)
 		b = cast(byte)('a' + i);
 	
 	void[] varr = barr;
 	char[] carr;
 	
 	//carr = barr; // Error: cannot implicitly convert expression (barr) of  
 type byte[] to char[]
 	carr = cast(char[])barr;
 	
 	//carr = varr; // Error: cannot implicitly convert expression (varr) of  
 type void[] to char[]
 	carr = cast(char[])varr;
 	
 	writefln("%d,%s", carr.length, carr);
 }

 I am curious, was it ever possible, was it changed?  why?  It's always  
 "safe" - as the compiler knows how much data the void[] contains, and  
 void[] is "untyped" so it sorta makes sense to allow it..


It was never possible. You must explicitly cast to void[].

void[] makes actually little sense as the result of whole-file read that  
allocates. byte[] is at least usable and more accurate. In fact, it's a  
little dangerous to use void[], since you could assign pointer-containing  
values to the void[] and it should be marked as NOSCAN (no pointers inside  
file data).

However, when using the more conventional read(void[]) makes a LOT of  
sense, since any T[] implicitly casts to void[].

-Steve
Apr 17 2014
next sibling parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Thu, 17 Apr 2014 13:59:20 +0100, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 It was never possible. You must explicitly cast to void[].


to -> from?


 void[] makes actually little sense as the result of whole-file read that  
 allocates. byte[] is at least usable and more accurate. In fact, it's a  
 little dangerous to use void[], since you could assign  
 pointer-containing values to the void[] and it should be marked as  
 NOSCAN (no pointers inside file data).


I see what you're saying, byte[] is what *is* allocated.. but my point is  
that it's not what those bytes actually represent.

Are you saying void[] *is* currently marked NOSCAN?


 However, when using the more conventional read(void[]) makes a LOT of  
 sense, since any T[] implicitly casts to void[].


Indeed. :)

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Apr 17 2014
next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 17 Apr 2014 10:05:49 -0400, Regan Heath <regan netmail.co.nz>  
wrote:


 On Thu, 17 Apr 2014 13:59:20 +0100, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:

 It was never possible. You must explicitly cast to void[].


 to -> from?


Yes, sorry :)


 void[] makes actually little sense as the result of whole-file read  
 that allocates. byte[] is at least usable and more accurate. In fact,  
 it's a little dangerous to use void[], since you could assign  
 pointer-containing values to the void[] and it should be marked as  
 NOSCAN (no pointers inside file data).


 I see what you're saying, byte[] is what *is* allocated.. but my point  
 is that it's not what those bytes actually represent.

 Are you saying void[] *is* currently marked NOSCAN?


No, I mean the return value from read, since it's newly allocated general  
data, should be marked NOSCAN.

Casting the type does not change how the block is marked, only the  
allocation type makes that distinction. When you *allocate* a void[]  
buffer, it's marked no scan. But when you allocate a byte[] buffer and  
implicitly cast it to void[], it's not marked NOSCAN.

TL;DR, IMO read should return byte[].

-Steve
Apr 17 2014
prev sibling parent reply "sclytrack" <sclytrack fake.com> writes:
On Thursday, 17 April 2014 at 14:05:50 UTC, Regan Heath wrote:

 On Thu, 17 Apr 2014 13:59:20 +0100, Steven Schveighoffer 
 <schveiguy yahoo.com> wrote:

 It was never possible. You must explicitly cast to void[].


 to -> from?


 void[] makes actually little sense as the result of whole-file 
 read that allocates. byte[] is at least usable and more 
 accurate. In fact, it's a little dangerous to use void[], 
 since you could assign pointer-containing values to the void[] 
 and it should be marked as NOSCAN (no pointers inside file 
 data).


 I see what you're saying, byte[] is what *is* allocated.. but 
 my point is that it's not what those bytes actually represent.

 Are you saying void[] *is* currently marked NOSCAN?


 However, when using the more conventional read(void[]) makes a 
 LOT of sense, since any T[] implicitly casts to void[].


 Indeed. :)

 R


auto a1 = new ubyte[10];  //NO_SCAN set
auto a2 = new ubyte*[10]; // NO_SCAN not set
auto a3 = new void[10];   //NO_SCAN not set
auto a4 = new void *[10];  //NO_SCAN not set

void [] retains = a1;    //NO_SCAN REMAINS SET from the ubyte [] 
at creation time.

Since read comes straight from the file. It contains no memory 
pointers
and the NO_SCAN can be set.
Apr 17 2014
parent "sclytrack" <sclytrack fake.com> writes:
 Are you saying void[] *is* currently marked NOSCAN?




import std.stdio;
import core.memory;

writeln("int* []", GC.query(cast(void *) arr2).attr);
Apr 17 2014
prev sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 17 April 2014 at 12:59:20 UTC, Steven Schveighoffer 
wrote:

 It was never possible. You must explicitly cast to void[].

 void[] makes actually little sense as the result of whole-file 
 read that allocates. byte[] is at least usable and more 
 accurate. In fact, it's a little dangerous to use void[], since 
 you could assign pointer-containing values to the void[] and it 
 should be marked as NOSCAN (no pointers inside file data).

 However, when using the more conventional read(void[]) makes a 
 LOT of sense, since any T[] implicitly casts to void[].

 -Steve


void[] will only make sense once you've accepted that 
"void.sizeof == 1".

Well, I guess "void[]" is C++'s "char*" for indiscriminate 
buffers. Speaking of which, does "void*" trigger strict aliasing 
in D? This subject seems like a hot potato no-one wants to touch.
Apr 17 2014
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 17 Apr 2014 17:04:25 -0400, monarch_dodra <monarchdodra gmail.com>  
wrote:


 On Thursday, 17 April 2014 at 12:59:20 UTC, Steven Schveighoffer wrote:

 It was never possible. You must explicitly cast to void[].

 void[] makes actually little sense as the result of whole-file read  
 that allocates. byte[] is at least usable and more accurate. In fact,  
 it's a little dangerous to use void[], since you could assign  
 pointer-containing values to the void[] and it should be marked as  
 NOSCAN (no pointers inside file data).

 However, when using the more conventional read(void[]) makes a LOT of  
 sense, since any T[] implicitly casts to void[].

 -Steve


 void[] will only make sense once you've accepted that "void.sizeof == 1".


It is already accepted that when we talk about length in a void[], it's  
the number of bytes. But the data has no formal type.

But any array implicitly casts to void[]. This is why it makes a good  
parameter for read or write (when reading or writing the binary data).


 Well, I guess "void[]" is C++'s "char*" for indiscriminate buffers.  
 Speaking of which, does "void*" trigger strict aliasing in D? This  
 subject seems like a hot potato no-one wants to touch.


No, it's equivalent to void *, not char *.

in D, ubyte[] would be the equivalent of C's char *.

-Steve
Apr 17 2014
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 17 April 2014 at 21:27:44 UTC, Steven Schveighoffer 
wrote:

 On Thu, 17 Apr 2014 17:04:25 -0400, monarch_dodra 
 <monarchdodra gmail.com> wrote:

 void[] will only make sense once you've accepted that 
 "void.sizeof == 1".


 It is already accepted that when we talk about length in a 
 void[], it's the number of bytes. But the data has no formal 
 type.


Well, I always thought that "void[] slice" meant "there are 
slice.length items, starting at slice.ptr. I don't know the size 
of the individual items".

For example, in C, a lot of functions take "void* first, size_t 
num, size_t width".

In fact, most of druntime functions take "void[]" buffers that 
work that way. There's an associated typeid, so that you can now 
how large each individual items are.


 But any array implicitly casts to void[]. This is why it makes 
 a good parameter for read or write (when reading or writing the 
 binary data).


I guess. I just find it kind of strange that a type "that has no 
type" would have an actual sizeof. Then again, I thought void had 
no sizeof in C, but I just checked, and I was wrong.


 Well, I guess "void[]" is C++'s "char*" for indiscriminate 
 buffers. Speaking of which, does "void*" trigger strict 
 aliasing in D? This subject seems like a hot potato no-one 
 wants to touch.


 No, it's equivalent to void *, not char *.

 in D, ubyte[] would be the equivalent of C's char *.

 -Steve


Correct.
Apr 17 2014
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 18 Apr 2014 02:04:16 -0400, monarch_dodra <monarchdodra gmail.com>  
wrote:


 On Thursday, 17 April 2014 at 21:27:44 UTC, Steven Schveighoffer wrote:

 On Thu, 17 Apr 2014 17:04:25 -0400, monarch_dodra  
 <monarchdodra gmail.com> wrote:

 void[] will only make sense once you've accepted that "void.sizeof ==  
 1".


 It is already accepted that when we talk about length in a void[], it's  
 the number of bytes. But the data has no formal type.


 Well, I always thought that "void[] slice" meant "there are slice.length  
 items, starting at slice.ptr. I don't know the size of the individual  
 items".

 For example, in C, a lot of functions take "void* first, size_t num,  
 size_t width".

 In fact, most of druntime functions take "void[]" buffers that work that  
 way. There's an associated typeid, so that you can now how large each  
 individual items are.


import std.stdio;

void main()
{
    int[] x = new int[5];
    void[] y = x;
    writeln(y.length); // 20
}

When a function takes a typeid, that usually is because the translation is  
not made. In druntine cases, the compiler is removing the type, and  
sticking it into the typeid instead. But the length has not been  
translated to bytes! It's still in terms of the original type.

In those cases, it's equivalent to:

void[] y = *cast(void[]*)&x;

which would make y.length == 5.


 But any array implicitly casts to void[]. This is why it makes a good  
 parameter for read or write (when reading or writing the binary data).


 I guess. I just find it kind of strange that a type "that has no type"  
 would have an actual sizeof. Then again, I thought void had no sizeof in  
 C, but I just checked, and I was wrong.


It's a little strange, but while void has no size, void[] *does* have a  
size. The size is in bytes. You can think of an array as "starts at this  
address, and ends at that address". Because addresses are in terms of  
bytes, so is the length of that array.

I admit, I didn't think C's void had a size ;) I'm pretty sure it doesn't  
in D, but then again...

-Steve
Apr 18 2014
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 18 April 2014 at 13:08:04 UTC, Steven Schveighoffer 
wrote:

 I admit, I didn't think C's void had a size ;) I'm pretty sure 
 it doesn't in D, but then again...

 -Steve


Yeah... "static assert(void.sizeof == 1);" passes :/

So in any case, long story short:
"void[]": This is an un-typed buffer, pointing to a memory 
location that starts at .ptr, and is .length bytes in length.

Also, as far as the GC is concerned, "void" is a type that should 
be scanned (whether or not the data originally allocated was 
marked as such is another issue).
Apr 18 2014
parent Andrej Mitrovic via Digitalmars-d-learn writes:
On 4/18/14, monarch_dodra via Digitalmars-d-learn
<digitalmars-d-learn puremagic.com> wrote:

 Yeah... "static assert(void.sizeof == 1);" passes :/


Note that you can even have static void arrays. E.g.:

https://issues.dlang.org/show_bug.cgi?id=9691

I'm not sure whether this is an oversight (accepts-invalid) or
something else. But it needs to be properly documented.
Apr 20 2014