www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Syntax for pinning

reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cfit27$snc$1 digitaldaemon.com>, Walter says...


I know how to build a gc that will compact
memory to deal with fragmentation when it does arise, so although this is
not in the current D it is not a technical problem to do it.



 That's probably good news, but:

 Will it be possible to mark specific arrays on the heap as "immovable"?




 say yes. I'm sure it is not a technical problem).



Yes. It's called "pinning". And I can guess why you need this feature <g>.




Walter, hi.

Listen - while the prospect of a compacting GC may be a long way away, I'd like
to be able to mark specific arrays on the heap as immovable right now, for
forward compatibility, so that when the new GC comes along everything will still
behave as intended.

Could you perhaps reserve a keyword and/or some syntax for pinning? It only has
to do nothing for now, but would swing into action when recompiled with the
future DMD which uses the future GC. Here are some suggested syntaxes:

// given uint[] a;, which could be a member variable

(1)    a = new pinned uint[n];

(2)    a = new uint[n];
#      pin a;
#      // and later
#      unpin a;

What think you?
Jill
Aug 22 2004
next sibling parent reply Martin <Martin_member pathlink.com> writes:
I am in favor, because I use sometimes char * and other 'hacks' to access
memory, so it wouldn't be very nice when memory suddenly would just run away. 
And I agree, that we need it now, pecause otherwise we need to do a lot of code
rewriting when new version comes.
Keywords 'pinned' and 'unpin' are ok, I think.  'pin', maybe not beacouse it is
a 3 letter word, maybe 'pinmem'?
Or whatever, but I agree that we need this and we need it now. 



In article <cg9ocj$9ce$1 digitaldaemon.com>, Arcane Jill says...

In article <cfit27$snc$1 digitaldaemon.com>, Walter says...


I know how to build a gc that will compact
memory to deal with fragmentation when it does arise, so although this is
not in the current D it is not a technical problem to do it.



 That's probably good news, but:

 Will it be possible to mark specific arrays on the heap as "immovable"?




 say yes. I'm sure it is not a technical problem).



Yes. It's called "pinning". And I can guess why you need this feature <g>.




Walter, hi.

Listen - while the prospect of a compacting GC may be a long way away, I'd like
to be able to mark specific arrays on the heap as immovable right now, for
forward compatibility, so that when the new GC comes along everything will still
behave as intended.

Could you perhaps reserve a keyword and/or some syntax for pinning? It only has
to do nothing for now, but would swing into action when recompiled with the
future DMD which uses the future GC. Here are some suggested syntaxes:

// given uint[] a;, which could be a member variable

(1)    a = new pinned uint[n];

(2)    a = new uint[n];
#      pin a;
#      // and later
#      unpin a;

What think you?
Jill
Aug 22 2004
next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
Martin wrote:


 I am in favor, because I use sometimes char * and other 'hacks' to access
 memory, so it wouldn't be very nice when memory suddenly would just run
 away. And I agree, that we need it now, pecause otherwise we need to do a
 lot of code rewriting when new version comes.
 Keywords 'pinned' and 'unpin' are ok, I think.  'pin', maybe not beacouse
 it is a 3 letter word, maybe 'pinmem'?
 Or whatever, but I agree that we need this and we need it now.
 
 
 
 In article <cg9ocj$9ce$1 digitaldaemon.com>, Arcane Jill says...

In article <cfit27$snc$1 digitaldaemon.com>, Walter says...


I know how to build a gc that will compact
memory to deal with fragmentation when it does arise, so although this
is not in the current D it is not a technical problem to do it.



 That's probably good news, but:

 Will it be possible to mark specific arrays on the heap as "immovable"?




 say yes. I'm sure it is not a technical problem).



Yes. It's called "pinning". And I can guess why you need this feature
<g>.




Walter, hi.

Listen - while the prospect of a compacting GC may be a long way away, I'd
like to be able to mark specific arrays on the heap as immovable right
now, for forward compatibility, so that when the new GC comes along
everything will still behave as intended.

Could you perhaps reserve a keyword and/or some syntax for pinning? It
only has to do nothing for now, but would swing into action when
recompiled with the future DMD which uses the future GC. Here are some
suggested syntaxes:

// given uint[] a;, which could be a member variable

(1)    a = new pinned uint[n];

(2)    a = new uint[n];
#      pin a;
#      // and later
#      unpin a;

What think you?
Jill






I'd prefer something like 2 since pinning/unpinning can be temporary. This
brings back memories of locking handles on the Mac (HLock and friends) -
something you always forget about until you spend hours trying to debug a
nasty seg-v as the memory manager moved a block at just the right time.
Also
1) it should be possible to query the pinned state.
2) can these go into gc.d instead of creating new keywords?
Aug 22 2004
parent Arcane Jill <Arcane_member pathlink.com> writes:
In article <cga6hv$l00$1 digitaldaemon.com>, Ben Hinkle says...


I'd prefer something like 2 since pinning/unpinning can be temporary.



I'd have no objection either way, but you can get by without unpinning, if you
need to - just create a new (unpinned) array, copy the old stuff into the new,
and then delete (or at least, lose all references to) the first. I'm not saying
this is ideal, but since I don't know how the future GC is going to work, I
don't know what's possible and what isn't.

That said, an unpinning function which did nothing would be completely harmless,
so we might as well have the syntax for it, for if and when it does become
possible.



This
brings back memories of locking handles on the Mac (HLock and friends) -
something you always forget about until you spend hours trying to debug a
nasty seg-v as the memory manager moved a block at just the right time.
Also
2) can these go into gc.d instead of creating new keywords?



I thought that after I posted. Functions would do fine. Maybe:

#    std.gc.pin(a);
#    std.gc.unpin(a);


1) it should be possible to query the pinned state.



Again, I don't want to tie Walter's hands here. I don't know whether this will
be possible or easy to do. But I'd certainly have no objection to such a
function.

Arcane Jill
Aug 22 2004
prev sibling next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cga5a4$jjl$1 digitaldaemon.com>, Martin says...

I am in favor, because I use sometimes char * and other 'hacks' to access
memory, so it wouldn't be very nice when memory suddenly would just run away.



Actually, I don't think that would char*s would be a problem. (I mean, D would
just break if that were true!)  So you wouldn't need pinning just to keep a
char* working. If you had a char* pointer pointing to a string, and the GC moved
the string during compaction, you can rest assured that the GC would also modify
your char* and everything would continue to work for you. As for your "other
'hacks'", read "Pointers and the Garbage Collector" on page
http://www.digitalmars.com/d/garbage.html for details about what you are and are
not allowed to do.

Pinning is only needed for sophisticated memory management stuff (or in my case,
security). Rest assured you won't need it just to keep pointers valid.

Arcane Jill
Aug 22 2004
parent Stewart Gordon <smjg_1998 yahoo.com> writes:
Arcane Jill wrote:

<snip>

 Pinning is only needed for sophisticated memory management stuff (or
 in my case, security). Rest assured you won't need it just to keep
 pointers valid.



You forgot interfacing with foreign code, crocky legacy APIs and the 
like....

D/26273

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on 
the 'group where everyone may benefit.
Sep 01 2004
prev sibling parent reply Andy Friesen <andy ikagames.com> writes:
Martin wrote:

 I am in favor, because I use sometimes char * and other 'hacks' to access
 memory, so it wouldn't be very nice when memory suddenly would just run away. 
 And I agree, that we need it now, pecause otherwise we need to do a lot of code
 rewriting when new version comes.
 Keywords 'pinned' and 'unpin' are ok, I think.  'pin', maybe not beacouse it is
 a 3 letter word, maybe 'pinmem'?
 Or whatever, but I agree that we need this and we need it now. 



'pin' is terrible because it's a noun.  'unpin' is bad because it's a 
verb.  These are both perfectly reasonable variable/method names, and it 
would be a crime to steal them from the programmer.

'pinned', is more or less perfect: it's short and, since it's an 
adjective, it wouldn't be a very good variable or method name.  'pinned' 
also fits in nicely with the other attributes:

     protected static pinned char* ptr;

I like the attribute style.  Unlike separate pin/unpin statements, it 
leaves no opportunity for absent-mindedness to wreak havoc. :)

  -- andy
Aug 22 2004
next sibling parent Deja Augustine <deja scratch-ware.net> writes:
I would love to see a pinned attribute.  I agree that it's less prone to 
mistakes.

-Deja

Andy Friesen wrote:


 Martin wrote:
 

 I am in favor, because I use sometimes char * and other 'hacks' to access
 memory, so it wouldn't be very nice when memory suddenly would just 
 run away. And I agree, that we need it now, pecause otherwise we need 
 to do a lot of code
 rewriting when new version comes.
 Keywords 'pinned' and 'unpin' are ok, I think.  'pin', maybe not 
 beacouse it is
 a 3 letter word, maybe 'pinmem'?
 Or whatever, but I agree that we need this and we need it now. 



 
 'pin' is terrible because it's a noun.  'unpin' is bad because it's a 
 verb.  These are both perfectly reasonable variable/method names, and it 
 would be a crime to steal them from the programmer.
 
 'pinned', is more or less perfect: it's short and, since it's an 
 adjective, it wouldn't be a very good variable or method name.  'pinned' 
 also fits in nicely with the other attributes:
 
     protected static pinned char* ptr;
 
 I like the attribute style.  Unlike separate pin/unpin statements, it 
 leaves no opportunity for absent-mindedness to wreak havoc. :)
 
  -- andy
Aug 22 2004
prev sibling parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
"Andy Friesen" <andy ikagames.com> wrote in message
news:cgadrn$qf8$1 digitaldaemon.com...

 Martin wrote:

 I am in favor, because I use sometimes char * and other 'hacks' to access
 memory, so it wouldn't be very nice when memory suddenly would just run away.
 And I agree, that we need it now, pecause otherwise we need to do a lot of code
 rewriting when new version comes.
 Keywords 'pinned' and 'unpin' are ok, I think.  'pin', maybe not beacouse it is
 a 3 letter word, maybe 'pinmem'?
 Or whatever, but I agree that we need this and we need it now.



 'pin' is terrible because it's a noun.



"pin" is both noun and verb


  'unpin' is bad because it's a
 verb.  These are both perfectly reasonable variable/method names, and it
 would be a crime to steal them from the programmer.



                      heavy


 'pinned', is more or less perfect: it's short and, since it's an
 adjective, it wouldn't be a very good variable or method name.  'pinned'
 also fits in nicely with the other attributes:

      protected static pinned char* ptr;

 I like the attribute style.  Unlike separate pin/unpin statements, it
 leaves no opportunity for absent-mindedness to wreak havoc. :)

   -- andy
Aug 22 2004
parent Andy Friesen <andy ikagames.com> writes:
Matthew wrote:

 "Andy Friesen" <andy ikagames.com> wrote in message
news:cgadrn$qf8$1 digitaldaemon.com...
 

Martin wrote:

'pin' is terrible because it's a noun.



 "pin" is both noun and verb



errr right.  Brainfart.

That makes it a pretty bad identifier, actually. (I still have 
nightmares about attributes and classes named 'query')


 'unpin' is bad because it's a
verb.  These are both perfectly reasonable variable/method names, and it
would be a crime to steal them from the programmer.



                          ^
                       heavy



Whoops!

  -- Aspiring student of the English language
Aug 22 2004
prev sibling next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cg9ocj$9ce$1 digitaldaemon.com>, Arcane Jill says...


// given uint[] a;, which could be a member variable

(1)    a = new pinned uint[n];

(2)    a = new uint[n];
#      pin a;
#      // and later
#      unpin a;

What think you?




Oh, and here's yet another possibility:

#    class A
#    {
#        pinned uint[] a;
#    }

That is, "pinned" is an attribute. Anything subsequently assigned to A.a is
pinned.

I'm just throwing ideas around here, because I don't know how the future GC will
work, and I don't really care about the syntax. All I'm asking is that some
syntax or other be supported in a near-future version of DMD, so that I can then
add it to my code and relax.

This is quite amusing in a way - I'm asking for a function that does nothing!
(Well, we got opPos() so there's precedent for that :) ).

Jill
Aug 22 2004
parent reply Matthias Becker <Matthias_member pathlink.com> writes:
// given uint[] a;, which could be a member variable

(1)    a = new pinned uint[n];

(2)    a = new uint[n];
#      pin a;
#      // and later
#      unpin a;

What think you?




Oh, and here's yet another possibility:

#    class A
#    {
#        pinned uint[] a;
#    }

That is, "pinned" is an attribute. Anything subsequently assigned to A.a is
pinned.

I'm just throwing ideas around here, because I don't know how the future GC will
work, and I don't really care about the syntax. All I'm asking is that some
syntax or other be supported in a near-future version of DMD, so that I can then
add it to my code and relax.

This is quite amusing in a way - I'm asking for a function that does nothing!
(Well, we got opPos() so there's precedent for that :) ).

Jill



Does it need to be a keyword? Perhaps normal functions in the library (in
std.gc) could do the work.
I'd like something like this:


a = new uint[n];

{
auto Pin pin = new Pin(a);
..
}

-- Matthias Becker
Aug 22 2004
parent Stewart Gordon <smjg_1998 yahoo.com> writes:
Matthias Becker wrote:

<snip>

 Does it need to be a keyword? Perhaps normal functions in the library (in
 std.gc) could do the work.
 I'd like something like this:
 
 a = new uint[n];
 
 {
 auto Pin pin = new Pin(a);
 ..
 }



Makes sense.  As long as the class itself isn't auto, as that would 
probably break the ability to store such crocky things as Windows 
handles as class members without further dirty tricks....

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on 
the 'group where everyone may benefit.
Sep 01 2004
prev sibling next sibling parent reply "antiAlias" <fu bar.com> writes:
Don't wish to be a party-pooper Jill, but asking Walter to reserve syntax
for a 'feature' that's not implemented is inviting trouble for D; simply
because it's typically rather difficult to "pin-down" exactly how pinning
will work until the requisite time and effort has been spent to figure out
how the whole thing will operate. It may turn out that Walter /does/ know
exactly how it will operate, but if not then your asking him to tie his own
hands over something that provides perhaps zero value right now.

Is it possible for you to 'mark' these arrays using an alias, typedef, or
template instead? It would then be trivial to patch them later, without
having to remember which ones should be pinned.



"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:cg9ocj$9ce$1 digitaldaemon.com...

 In article <cfit27$snc$1 digitaldaemon.com>, Walter says...


I know how to build a gc that will compact
memory to deal with fragmentation when it does arise, so although this













not in the current D it is not a technical problem to do it.



 That's probably good news, but:

 Will it be possible to mark specific arrays on the heap as "immovable"?




 say yes. I'm sure it is not a technical problem).



Yes. It's called "pinning". And I can guess why you need this feature







 Walter, hi.

 Listen - while the prospect of a compacting GC may be a long way away, I'd




 to be able to mark specific arrays on the heap as immovable right now, for
 forward compatibility, so that when the new GC comes along everything will




 behave as intended.

 Could you perhaps reserve a keyword and/or some syntax for pinning? It




 to do nothing for now, but would swing into action when recompiled with




 future DMD which uses the future GC. Here are some suggested syntaxes:

 // given uint[] a;, which could be a member variable

 (1)    a = new pinned uint[n];

 (2)    a = new uint[n];
 #      pin a;
 #      // and later
 #      unpin a;

 What think you?
 Jill
Aug 22 2004
parent Arcane Jill <Arcane_member pathlink.com> writes:
In article <cgajm4$tas$1 digitaldaemon.com>, antiAlias says...

Don't wish to be a party-pooper Jill, but asking Walter to reserve syntax
for a 'feature' that's not implemented is inviting trouble for D; simply
because it's typically rather difficult to "pin-down" exactly how pinning
will work until the requisite time and effort has been spent to figure out
how the whole thing will operate.



You have a very good point. I was assuming that Walter already had the
implementation in mind, but I guess I shouldn't have assumed that.



It may turn out that Walter /does/ know
exactly how it will operate, but if not then your asking him to tie his own
hands over something that provides perhaps zero value right now.



Yes, you're right aren't you. Well yeah - if the mechanism isn't know, we can't
add the syntax yet. Totally fair enough.



Is it possible for you to 'mark' these arrays using an alias, typedef, or
template instead? It would then be trivial to patch them later, without
having to remember which ones should be pinned.



Actually, there's really only one place in my code that needs it, so I won't
have any trouble remembering it. Thanks for the suggestion though.

Arcane Jill
Aug 22 2004
prev sibling parent Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
How about just a function call in the GC?  Then we don't have to add new 
syntax.  Walter could implement the following function (which would be a 
NOP in the current GC):

   void gc.pin(byte[] range);
Aug 23 2004