www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - GC: no-pointer areas

reply Thomas Kuehne <thomas-dloop kuehne.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

With all that Phobos talk going on ... here is a small task:

Implement GC areas that are guaranteed to contain no pointers
and are subsequently not searched for pointers during GC-collections.

Basic Idea:

void *malloc(size_t size);
void *calloc(size_t nmemb, size_t size);

If less than sizeof(size_t) bytes are allocated per element, the
allocated memory is guaranteed to contain no pointers.

There are three implementation levels:

1) Basic (dmd/src/phobos/internal/gc/gcx.d)

Implement the basic idea(malloc, calloc, realloc, ...) and provide
the functions malloc_no_ptr, calloc_no_ptr, realloc_no_ptr.

2) Complete (dmd/src/phobos/internal/gc/gc.d)

Update the code to take advantage of gcx.d's *_no_ptr.

3) Extended (compiler and dmd/src/phobos/internal/gc/gc.d)

By adding compiler support, elements with sizes larger than or
equal to sizeof(size_t) that contain no pointers (e.g. the data
portion of float arrays and some structs)can use the no-ptr pools.

As a result, the GC for applications using lots of data arrays should
be noticeably faster while other applications shouldn't experience any
measurable speed decreases.

Thomas


-----BEGIN PGP SIGNATURE-----

iD8DBQFFPTf+LK5blCcjpWoRAkZKAJ4r05W8qpr9ZwXMewKZdHLOoE23vQCbB8ll
Zu2I/IkMgSbW+CO81Ljk/jI=
=RvEo
-----END PGP SIGNATURE-----
Oct 23 2006
next sibling parent reply "Craig Black" <cblack ara.com> writes:
Overall a very, very good idea.
However, why does the bytes allocated per element have to be the 
qualification?
This seems a bit convoluted to me. Why not just add a parameter with a 
default value?

void *malloc(size_t size, int hasPtr = 1);

-Craig

"Thomas Kuehne" <thomas-dloop kuehne.cn> wrote in message 
news:slrnejqe00.7a5.thomas-dloop birke.kuehne.cn...
 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1

 With all that Phobos talk going on ... here is a small task:

 Implement GC areas that are guaranteed to contain no pointers
 and are subsequently not searched for pointers during GC-collections.

 Basic Idea:

 void *malloc(size_t size);
 void *calloc(size_t nmemb, size_t size);

 If less than sizeof(size_t) bytes are allocated per element, the
 allocated memory is guaranteed to contain no pointers.

 There are three implementation levels:

 1) Basic (dmd/src/phobos/internal/gc/gcx.d)

 Implement the basic idea(malloc, calloc, realloc, ...) and provide
 the functions malloc_no_ptr, calloc_no_ptr, realloc_no_ptr.

 2) Complete (dmd/src/phobos/internal/gc/gc.d)

 Update the code to take advantage of gcx.d's *_no_ptr.

 3) Extended (compiler and dmd/src/phobos/internal/gc/gc.d)

 By adding compiler support, elements with sizes larger than or
 equal to sizeof(size_t) that contain no pointers (e.g. the data
 portion of float arrays and some structs)can use the no-ptr pools.

 As a result, the GC for applications using lots of data arrays should
 be noticeably faster while other applications shouldn't experience any
 measurable speed decreases.

 Thomas


 -----BEGIN PGP SIGNATURE-----

 iD8DBQFFPTf+LK5blCcjpWoRAkZKAJ4r05W8qpr9ZwXMewKZdHLOoE23vQCbB8ll
 Zu2I/IkMgSbW+CO81Ljk/jI=
 =RvEo
 -----END PGP SIGNATURE----- 

Oct 23 2006
parent reply Thomas Kuehne <thomas-dloop kuehne.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Craig Black schrieb am 2006-10-23:
 Overall a very, very good idea.
 However, why does the bytes allocated per element have to be the 
 qualification?
 This seems a bit convoluted to me. Why not just add a parameter with a 
 default value?

 void *malloc(size_t size, int hasPtr = 1);

That's what malloc_no_ptr was meant for, using a default parameter might be more useful though. Using int seems strange, how about: void *malloc(size_t size, bool hasPtr = true); Bytes per element doesn't require any cooperation by the compiler. Naturally the compiler could take advantage later on and use the no-pointer functions. Thomas
 "Thomas Kuehne" <thomas-dloop kuehne.cn> wrote in message 
 With all that Phobos talk going on ... here is a small task:

 Implement GC areas that are guaranteed to contain no pointers
 and are subsequently not searched for pointers during GC-collections.

 Basic Idea:

 void *malloc(size_t size);
 void *calloc(size_t nmemb, size_t size);

 If less than sizeof(size_t) bytes are allocated per element, the
 allocated memory is guaranteed to contain no pointers.

 There are three implementation levels:

 1) Basic (dmd/src/phobos/internal/gc/gcx.d)

 Implement the basic idea(malloc, calloc, realloc, ...) and provide
 the functions malloc_no_ptr, calloc_no_ptr, realloc_no_ptr.

 2) Complete (dmd/src/phobos/internal/gc/gc.d)

 Update the code to take advantage of gcx.d's *_no_ptr.

 3) Extended (compiler and dmd/src/phobos/internal/gc/gc.d)

 By adding compiler support, elements with sizes larger than or
 equal to sizeof(size_t) that contain no pointers (e.g. the data
 portion of float arrays and some structs)can use the no-ptr pools.

 As a result, the GC for applications using lots of data arrays should
 be noticeably faster while other applications shouldn't experience any
 measurable speed decreases.

 Thomas


-----BEGIN PGP SIGNATURE----- iD8DBQFFPa/rLK5blCcjpWoRApScAJ9gQnpjFFwr1glgOjngylTPipjBxgCdGm4K 9UXiWoNxIWEvdB+lpoYFfLA= =Z5M4 -----END PGP SIGNATURE-----
Oct 23 2006
next sibling parent "Craig Black" <cblack ara.com> writes:
"Thomas Kuehne" <thomas-dloop kuehne.cn> wrote in message 
news:slrnejrbvd.79u.thomas-dloop birke.kuehne.cn...
 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1

 Craig Black schrieb am 2006-10-23:
 Overall a very, very good idea.
 However, why does the bytes allocated per element have to be the
 qualification?
 This seems a bit convoluted to me. Why not just add a parameter with a
 default value?

 void *malloc(size_t size, int hasPtr = 1);

That's what malloc_no_ptr was meant for, using a default parameter might be more useful though. Using int seems strange, how about: void *malloc(size_t size, bool hasPtr = true);

Yeah bool is more appropriate. However, bool isn't a C type is it? I don't know if we are limited to using C stuff for the C runtime in order to maintain C compatibility. Anyway, if not this exact approach, something like this really needs to be done to improve GC performance. Anything we can do to optimize it is crucial, since GC is D's biggest performance bottleneck. -Craig
Oct 24 2006
prev sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Thomas Kuehne wrote:
 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1
 
 Craig Black schrieb am 2006-10-23:
 Overall a very, very good idea.
 However, why does the bytes allocated per element have to be the 
 qualification?
 This seems a bit convoluted to me. Why not just add a parameter with a 
 default value?

 void *malloc(size_t size, int hasPtr = 1);

That's what malloc_no_ptr was meant for, using a default parameter might be more useful though. Using int seems strange, how about: void *malloc(size_t size, bool hasPtr = true); Bytes per element doesn't require any cooperation by the compiler. Naturally the compiler could take advantage later on and use the no-pointer functions. Thomas

Huh? Isn't it so that calls to malloc/calloc are already not scanned by the GC (besides also not being GC managed), and only when addRoots/addRange is used on such segments are they scanned by the GC? -- Bruno Medeiros - MSc in CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Oct 24 2006
parent reply "Craig Black" <cblack ara.com> writes:
 Huh? Isn't it so that calls to malloc/calloc are already not scanned by 
 the GC (besides also not being GC managed), and only when 
 addRoots/addRange is used on such segments are they scanned by the GC?

You are right. This is still a good idea. addRoot and addRange could have an additional parameter that would indicate if the allocation unit contains pointers or not. -Craig
Oct 25 2006
parent Thomas Kuehne <thomas-dloop kuehne.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Craig Black schrieb am 2006-10-25:
 Huh? Isn't it so that calls to malloc/calloc are already not scanned by 
 the GC (besides also not being GC managed), and only when 
 addRoots/addRange is used on such segments are they scanned by the GC?

You are right. This is still a good idea. addRoot and addRange could have an additional parameter that would indicate if the allocation unit contains pointers or not.

Sloopy writing ... I was talking about gcx.GC.malloc(-> internal/gc/gcx.d) instead of std.c.stdlib.malloc(-> std/c/stdlib.d). Thomas -----BEGIN PGP SIGNATURE----- iD8DBQFFP4bfLK5blCcjpWoRAicKAKCOWwlCr2WRV8xmvtuBQPIulZTkbgCfT1/x KSqlJyuG5JK5oxA8uDU/mEA= =736L -----END PGP SIGNATURE-----
Oct 25 2006
prev sibling parent Alexander Panek <a.panek brainsware.org> writes:
I agree - a good idea.

Though, what about an explicit function that allocates memory for *data
only*? Theoretically, every kind of data and arrays without references
would then use this function, whereas every reference types use the
normal allocation methods.

Please correct me, if I understood something wrong there. :)

Alex

On Mon, 2006-10-23 at 16:59 -0500, Craig Black wrote:
 Overall a very, very good idea.
 However, why does the bytes allocated per element have to be the 
 qualification?
 This seems a bit convoluted to me. Why not just add a parameter with a 
 default value?
 
 void *malloc(size_t size, int hasPtr = 1);
 
 -Craig
 
 "Thomas Kuehne" <thomas-dloop kuehne.cn> wrote in message 
 news:slrnejqe00.7a5.thomas-dloop birke.kuehne.cn...
 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1

 With all that Phobos talk going on ... here is a small task:

 Implement GC areas that are guaranteed to contain no pointers
 and are subsequently not searched for pointers during GC-collections.

 Basic Idea:

 void *malloc(size_t size);
 void *calloc(size_t nmemb, size_t size);

 If less than sizeof(size_t) bytes are allocated per element, the
 allocated memory is guaranteed to contain no pointers.

 There are three implementation levels:

 1) Basic (dmd/src/phobos/internal/gc/gcx.d)

 Implement the basic idea(malloc, calloc, realloc, ...) and provide
 the functions malloc_no_ptr, calloc_no_ptr, realloc_no_ptr.

 2) Complete (dmd/src/phobos/internal/gc/gc.d)

 Update the code to take advantage of gcx.d's *_no_ptr.

 3) Extended (compiler and dmd/src/phobos/internal/gc/gc.d)

 By adding compiler support, elements with sizes larger than or
 equal to sizeof(size_t) that contain no pointers (e.g. the data
 portion of float arrays and some structs)can use the no-ptr pools.

 As a result, the GC for applications using lots of data arrays should
 be noticeably faster while other applications shouldn't experience any
 measurable speed decreases.

 Thomas


 -----BEGIN PGP SIGNATURE-----

 iD8DBQFFPTf+LK5blCcjpWoRAkZKAJ4r05W8qpr9ZwXMewKZdHLOoE23vQCbB8ll
 Zu2I/IkMgSbW+CO81Ljk/jI=
 =RvEo
 -----END PGP SIGNATURE----- 


Oct 23 2006