www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - nogc and NSAutoReleasePool-style regions.

reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
Seems to me that most of the utility beyond maintaining graph
like structures can be covered by making the compiler aware of
region allocator semantics. Assuming that the use of GC is
constrained locally and does not consume too much space. I think
it would work out ok for loading of smaller files etc with the
following kind of semantics:

Something_ptr someglobalvar;

process(A a) {
   int x;

    regionalloc(1024,256) { // 1 MiB default size, 256KiB
increments

	auto y = new ...; // gc new() is hijacked by  regionalloc
	
	// assignments must be restricted so that new() ptrs and y ptrs
	// are not assigned to memory that can escape the  regionalloc
	
	// and no writes to global ptrs of region allocated objects
allowed

	auto ptr = compute_with_constrained_gc_semantics(a,y)
	
	someglobalvar = make_malloc_copy(ptr); // optimization: remove
temporary
	x = obtain_some_value(ptr);
   }

   // region is dead here, including y and ptr objects

}

The real challenge is in putting the right constraints on the
 regionalloc block, but I guess weakly pure functions are kind of
close in their constraints. So it should be possible?

The goal would be to be able to link to libraries that aren't
 nogc aware without having the GC runtime available. For loading,
initialization etc.
Jul 08 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
A custom allocator can also be specified. The nice thing is, that 
this doesn't even need new syntax:

process(A a) {
     int x;
     {
         MyCustomAllocator alloc;
         GC.pushAllocator(alloc);
         scope(exit) GC.popAllocator();

	auto y = new ...;
         // alloc's destructor is called, which calls destructors 
of
         // allocated objects
     }
}

But as you already noted, there needs to be a mechanism to 
restrict escaping of pointers. Do you have some concrete idea how 
that could be solved? A more or less straight-forward way would 
be to encode the ownership/lifetime information somewhere in the 
types of the pointers (similar to how we use const and the like 
today). But this cannot be done, because it could not apply to 
external (non-template) functions.
Jul 08 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 8 July 2014 at 20:38:59 UTC, Marc Schütz wrote:

 But as you already noted, there needs to be a mechanism to 
 restrict escaping of pointers.


Yes, there ought to be, at least for  safe code.


 Do you have some concrete idea how that could be solved?


For my use, yes. Since I am considering a D-version for game 
server development that does whole program analysis and might put 
additional restrictions on the language. I'd like to restrict the 
language to the extent that I can generate C code for portability 
(server/client code sharing)…

However, it should be possible with more local annotations too, 
but maybe the existing ones that D provides are too weak or too 
strong?

One needs to prevent escaping, that means:

1. prevent writing through global variables (but allow reading 
values, but not reading references for writing?)

2. prevent establishing new global contexts (threads and fibers)

3. putting limitations on the in-parameters to functions called 
within the region-allocator block so they don't allow writing to 
global contexts

4. ???

Sounds a lot like "pure", but not quite matching up?


 A more or less straight-forward way would be to encode the 
 ownership/lifetime information somewhere in the types of the 
 pointers (similar to how we use const and the like today). But 
 this cannot be done, because it could not apply to external 
 (non-template) functions.


It would be nice if it could fit into the current infrastructure… 
but all suggestions are interesting. I think it could be a nice 
way to get "gc" level convinience during spin up of a server when 
you have plenty of memory available (you can afford to be 
wasteful before you load in the main data).
Jul 08 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 8 July 2014 at 21:17:50 UTC, Ola Fosheim Grøstad 
wrote:

 On Tuesday, 8 July 2014 at 20:38:59 UTC, Marc Schütz wrote:

 But as you already noted, there needs to be a mechanism to 
 restrict escaping of pointers.


 Yes, there ought to be, at least for  safe code.


 Do you have some concrete idea how that could be solved?


 For my use, yes. Since I am considering a D-version for game 
 server development that does whole program analysis and might 
 put additional restrictions on the language. I'd like to 
 restrict the language to the extent that I can generate C code 
 for portability (server/client code sharing)…

 However, it should be possible with more local annotations too, 
 but maybe the existing ones that D provides are too weak or too 
 strong?

 One needs to prevent escaping, that means:

 1. prevent writing through global variables (but allow reading 
 values, but not reading references for writing?)

 2. prevent establishing new global contexts (threads and fibers)

 3. putting limitations on the in-parameters to functions called 
 within the region-allocator block so they don't allow writing 
 to global contexts

 4. ???

 Sounds a lot like "pure", but not quite matching up?


Well, `scope` was supposed to be that, but it's unimplemented and 
not completely thought through.


 A more or less straight-forward way would be to encode the 
 ownership/lifetime information somewhere in the types of the 
 pointers (similar to how we use const and the like today). But 
 this cannot be done, because it could not apply to external 
 (non-template) functions.


 It would be nice if it could fit into the current 
 infrastructure… but all suggestions are interesting. I think it 
 could be a nice way to get "gc" level convinience during spin 
 up of a server when you have plenty of memory available (you 
 can afford to be wasteful before you load in the main data).


Thinking aloud: If we could distinguish GC pointers from non-GC 
pointers, we could apply the ownership information to them "after 
the fact". Any GC pointer that comes out from your library 
functions inside an auto-release block can be confined to inside 
that block, or to the lifetime of the allocator. (That is, only 
_newly allocated_ GC pointers inside those blocks...)
Jul 09 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 9 July 2014 at 12:01:12 UTC, Marc Schütz wrote:

 Sounds a lot like "pure", but not quite matching up?


 Well, `scope` was supposed to be that, but it's unimplemented 
 and not completely thought through.


*nods*


 Thinking aloud: If we could distinguish GC pointers from non-GC 
 pointers, we could apply the ownership information to them 
 "after the fact". Any GC pointer that comes out from your 
 library functions inside an auto-release block can be confined 
 to inside that block, or to the lifetime of the allocator. 
 (That is, only _newly allocated_ GC pointers inside those 
 blocks...)


For the  nogc+region allocator GC pointers could be helpeful. And 
as I tried to point out in another thread on GC performance, it 
could also provide precise GC collection performance benefits if 
you also clustered the GC pointers in objects at a negative 
offset (thus getting them onto the same cachelines even though 
they originate in deep sub-classes as well as super.classes).

Another benefit with explicit GC pointers is that you could ban 
GC pointers to classes that provide destructors, sanitizing the 
constructor/destructor situation.
Jul 09 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Wednesday, 9 July 2014 at 12:07:27 UTC, Ola Fosheim Grøstad 
wrote:

 Another benefit with explicit GC pointers is that you could ban 
 GC pointers to classes that provide destructors, sanitizing the 
 constructor/destructor situation.


This could also be done at the allocator level, though. An 
allocator presumably knows whether it's objects are garbage 
collected (hmm... maybe not necessarily?), and it can inspect 
types to find out whether they have destructors.
Jul 09 2014
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 9 July 2014 at 14:36:37 UTC, Marc Schütz wrote:

 This could also be done at the allocator level, though.
 An allocator presumably knows whether it's objects are garbage 
 collected (hmm... maybe not necessarily?), and it can inspect 
 types to find out whether they have destructors.


I think the advantage of using a region allocator would be to 
allocate and clean up quickly without any extra overhead beyond 
moving a pointer, but I guess new() could globally be restricted 
to classes that does not provide a destructor…
Jul 09 2014