www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Reference counting

reply "Regan Heath" <regan netwin.co.nz> writes:
I got little or no comment on this idea/code I posted to another (rather  
large) thread here, I figured I'd post it again and see what people  
thought.

After reading this thread (the one I mention above) and also this one  
Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

It appears you need a type that:

1 - is stack based.
2 - has deterministic destruction.
3 - has opAssign.

It seems (from recent posts) that Walter plans to put auto classes on the  
stack, so they'd then fulfil requirements 1 and 2.

But what about 3? To me, opAssign isn't generally a good idea for classes  
(assigning a value to a reference seems illogical) but would be a good for  
structs (they're value types, assignment assigns a value).

Can you do reference counting without opAssign?

The idea for D (or so I've read) is that opAssign isn't required, instead  
you use a copy constructor. Now, I've little experience with ref counting,  
or with writing an implementation of one, but, does this work?

If not, why not? What's missing, what else is required? Overload of '.'  
perhaps? The ability to pass a RefPtr as the object being counted?

import std.c.windows.windows;
import std.random;
import std.thread;
import std.stdio;

class RefPtr
{
	RefPtr parent;
	Object resource = null;
	int refs = 0;

	this(Object res)
	{
		resource = res;
		writefln("ThreadID=(",Thread.getThis().id,") Initial RefPtr for  
resource=(",resource,")");
		increment();		
	}

	this(RefPtr rhs)
	{
		parent = rhs;
		parent.increment();
		writefln("ThreadID=(",Thread.getThis().id,") Ref=(",parent.refs,") for  
resource=(",parent.resource,")");
	}
	
	~this()
	{
		int r;
		
		if ((r = decrement()) == 0)
		{
			writefln("ThreadID=(",Thread.getThis().id,") release last ref  
Ref=(",r,")");
			if (parent) parent = null;
			else if (resource)
			{
				writefln("ThreadID=(",Thread.getThis().id,") delete  
resource=(",resource,")");
				delete resource;
				resource = null;
			}
		}
		writefln("ThreadID=(",Thread.getThis().id,") release Ref=(",r,")");
	}
protected:
	int increment()
	{
		int ret;
		
		if (parent) ret = parent.increment();
		else {
			synchronized(this)
			{
				ret = ++refs;
				writefln("ThreadID=(",Thread.getThis().id,") increment to  
Ref=(",refs,")");
			}
		}
		
		return ret;
	}
	int decrement()
	{
		int ret;
		
		if (parent) ret = parent.decrement();
		else {
			synchronized(this)
			{
				ret = --refs;
				writefln("ThreadID=(",Thread.getThis().id,") decrement to  
Ref=(",refs,")");
			}
		}
		
		return ret;
	}
}

class Resource
{
	char[] name = "11 was a racehorse";
	char[] toString() { return name; }
}

RefPtr pbob;

static this()
{
	pbob = new RefPtr(new Resource());
}

static ~this()
{
	delete pbob;
}

void main()
{
	Thread[] threads;
	int i;
	
	writefln("ThreadID=(",Thread.getThis().id,") is the main thread");
	
	for(i = 0; i < 10; i++)
	{
		threads ~= new Thread(&thread_function,null);
		threads[$-1].start();
	}
	
	while(true)
	{
		i = 0;
		foreach(Thread t; threads)
		{
			if (t.getState() == Thread.TS.TERMINATED) i++;
		}
		if (i == 10) break;
		Sleep(100);
	}
	
	writefln("Main exiting");
}

int thread_function(void* isnull)
{
	auto RefPtr p = new RefPtr(pbob);
	Sleep(1000+rand()%1000);	
	return 0;
}

Regan
Jun 26 2005
next sibling parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
Reagan, let's take a look on more simple case - implementation
of 'auto' pointer (this is simplifed version)

  struct Auto(X)
  {

    // constructor.
    this( X _ptr = 0 ) { ptr = _ptr; owner = true; }

    // the destructor deletes ptr only if we are the owner.
    ~this() { if( owner && ptr ) delete ptr; }

    // the assignment operator.
    void opAssign( X r )
    {
        if( owner && ptr ) delete ptr;
        ptr = r;
        owner = true;
    }

    // the assignment operator. transfer
    // ownership. (not full implementation, sic)
    void opAssign( Auto r )
    {
        if( owner && ptr ) delete ptr;
        ptr = r.release();
        owner = true;
    }

    X value() { return  ptr; }

    // release returns the ptr value and releases ownership
    // if we were previously the owner.

    X release()
    {
      owner = false;
      return value();
    }
    private X ptr;
    private bool owner;
  }

-------------------------

{
    Auto!(Object) autop = new Object(); // first instance
    autop = new Object(); // first instance will be deleted;
                                           // and autop contains second 
instance.
} // second instance will be deleted here
  // even in case of exception.

This is basic auto_ptr used in C++
and it simplfies life a lot.
opAssign and dtors are crucial here.
And such Auto(X) must be a struct.
Classes do not work here.

Andrew.


"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss0boj0r23k2f5 nrage.netwin.co.nz...
I got little or no comment on this idea/code I posted to another (rather 
large) thread here, I figured I'd post it again and see what people 
thought.

 After reading this thread (the one I mention above) and also this one 
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on the 
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for classes 
 (assigning a value to a reference seems illogical) but would be a good for 
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?

 The idea for D (or so I've read) is that opAssign isn't required, instead 
 you use a copy constructor. Now, I've little experience with ref counting, 
 or with writing an implementation of one, but, does this work?

 If not, why not? What's missing, what else is required? Overload of '.' 
 perhaps? The ability to pass a RefPtr as the object being counted?

 import std.c.windows.windows;
 import std.random;
 import std.thread;
 import std.stdio;

 class RefPtr
 {
 RefPtr parent;
 Object resource = null;
 int refs = 0;

 this(Object res)
 {
 resource = res;
 writefln("ThreadID=(",Thread.getThis().id,") Initial RefPtr for 
 resource=(",resource,")");
 increment(); }

 this(RefPtr rhs)
 {
 parent = rhs;
 parent.increment();
 writefln("ThreadID=(",Thread.getThis().id,") Ref=(",parent.refs,") for 
 resource=(",parent.resource,")");
 }

 ~this()
 {
 int r;

 if ((r = decrement()) == 0)
 {
 writefln("ThreadID=(",Thread.getThis().id,") release last ref 
 Ref=(",r,")");
 if (parent) parent = null;
 else if (resource)
 {
 writefln("ThreadID=(",Thread.getThis().id,") delete 
 resource=(",resource,")");
 delete resource;
 resource = null;
 }
 }
 writefln("ThreadID=(",Thread.getThis().id,") release Ref=(",r,")");
 }
 protected:
 int increment()
 {
 int ret;

 if (parent) ret = parent.increment();
 else {
 synchronized(this)
 {
 ret = ++refs;
 writefln("ThreadID=(",Thread.getThis().id,") increment to 
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 int decrement()
 {
 int ret;

 if (parent) ret = parent.decrement();
 else {
 synchronized(this)
 {
 ret = --refs;
 writefln("ThreadID=(",Thread.getThis().id,") decrement to 
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 }

 class Resource
 {
 char[] name = "11 was a racehorse";
 char[] toString() { return name; }
 }

 RefPtr pbob;

 static this()
 {
 pbob = new RefPtr(new Resource());
 }

 static ~this()
 {
 delete pbob;
 }

 void main()
 {
 Thread[] threads;
 int i;

 writefln("ThreadID=(",Thread.getThis().id,") is the main thread");

 for(i = 0; i < 10; i++)
 {
 threads ~= new Thread(&thread_function,null);
 threads[$-1].start();
 }

 while(true)
 {
 i = 0;
 foreach(Thread t; threads)
 {
 if (t.getState() == Thread.TS.TERMINATED) i++;
 }
 if (i == 10) break;
 Sleep(100);
 }

 writefln("Main exiting");
 }

 int thread_function(void* isnull)
 {
 auto RefPtr p = new RefPtr(pbob);
 Sleep(1000+rand()%1000); return 0;
 }

 Regan 

Jun 26 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
Looking at the code given I believe my code sample can be modified to  
achieve it. opAssign isn't required, you simply use a copy constructor (as  
I have done). Why can't it be a class? Specifically why can't it be an  
'auto' class reference (which will exist on the stack in future)?

Regan

p.s. It's "Regan" not "Reagan" ;)

On Sun, 26 Jun 2005 21:39:48 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:
 Reagan, let's take a look on more simple case - implementation
 of 'auto' pointer (this is simplifed version)

   struct Auto(X)
   {

     // constructor.
     this( X _ptr = 0 ) { ptr = _ptr; owner = true; }

     // the destructor deletes ptr only if we are the owner.
     ~this() { if( owner && ptr ) delete ptr; }

     // the assignment operator.
     void opAssign( X r )
     {
         if( owner && ptr ) delete ptr;
         ptr = r;
         owner = true;
     }

     // the assignment operator. transfer
     // ownership. (not full implementation, sic)
     void opAssign( Auto r )
     {
         if( owner && ptr ) delete ptr;
         ptr = r.release();
         owner = true;
     }

     X value() { return  ptr; }

     // release returns the ptr value and releases ownership
     // if we were previously the owner.

     X release()
     {
       owner = false;
       return value();
     }
     private X ptr;
     private bool owner;
   }

 -------------------------

 {
     Auto!(Object) autop = new Object(); // first instance
     autop = new Object(); // first instance will be deleted;
                                            // and autop contains second
 instance.
 } // second instance will be deleted here
   // even in case of exception.

 This is basic auto_ptr used in C++
 and it simplfies life a lot.
 opAssign and dtors are crucial here.
 And such Auto(X) must be a struct.
 Classes do not work here.

 Andrew.


 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0boj0r23k2f5 nrage.netwin.co.nz...
 I got little or no comment on this idea/code I posted to another (rather
 large) thread here, I figured I'd post it again and see what people
 thought.

 After reading this thread (the one I mention above) and also this one
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on  
 the
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for  
 classes
 (assigning a value to a reference seems illogical) but would be a good  
 for
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?

 The idea for D (or so I've read) is that opAssign isn't required,  
 instead
 you use a copy constructor. Now, I've little experience with ref  
 counting,
 or with writing an implementation of one, but, does this work?

 If not, why not? What's missing, what else is required? Overload of '.'
 perhaps? The ability to pass a RefPtr as the object being counted?

 import std.c.windows.windows;
 import std.random;
 import std.thread;
 import std.stdio;

 class RefPtr
 {
 RefPtr parent;
 Object resource = null;
 int refs = 0;

 this(Object res)
 {
 resource = res;
 writefln("ThreadID=(",Thread.getThis().id,") Initial RefPtr for
 resource=(",resource,")");
 increment(); }

 this(RefPtr rhs)
 {
 parent = rhs;
 parent.increment();
 writefln("ThreadID=(",Thread.getThis().id,") Ref=(",parent.refs,") for
 resource=(",parent.resource,")");
 }

 ~this()
 {
 int r;

 if ((r = decrement()) == 0)
 {
 writefln("ThreadID=(",Thread.getThis().id,") release last ref
 Ref=(",r,")");
 if (parent) parent = null;
 else if (resource)
 {
 writefln("ThreadID=(",Thread.getThis().id,") delete
 resource=(",resource,")");
 delete resource;
 resource = null;
 }
 }
 writefln("ThreadID=(",Thread.getThis().id,") release Ref=(",r,")");
 }
 protected:
 int increment()
 {
 int ret;

 if (parent) ret = parent.increment();
 else {
 synchronized(this)
 {
 ret = ++refs;
 writefln("ThreadID=(",Thread.getThis().id,") increment to
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 int decrement()
 {
 int ret;

 if (parent) ret = parent.decrement();
 else {
 synchronized(this)
 {
 ret = --refs;
 writefln("ThreadID=(",Thread.getThis().id,") decrement to
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 }

 class Resource
 {
 char[] name = "11 was a racehorse";
 char[] toString() { return name; }
 }

 RefPtr pbob;

 static this()
 {
 pbob = new RefPtr(new Resource());
 }

 static ~this()
 {
 delete pbob;
 }

 void main()
 {
 Thread[] threads;
 int i;

 writefln("ThreadID=(",Thread.getThis().id,") is the main thread");

 for(i = 0; i < 10; i++)
 {
 threads ~= new Thread(&thread_function,null);
 threads[$-1].start();
 }

 while(true)
 {
 i = 0;
 foreach(Thread t; threads)
 {
 if (t.getState() == Thread.TS.TERMINATED) i++;
 }
 if (i == 10) break;
 Sleep(100);
 }

 writefln("Main exiting");
 }

 int thread_function(void* isnull)
 {
 auto RefPtr p = new RefPtr(pbob);
 Sleep(1000+rand()%1000); return 0;
 }

 Regan


Jun 26 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
AutoPtr in D...

import std.c.windows.windows;
import std.random;
import std.thread;
import std.stdio;

class AutoPtr
{
	Object resource = null;
	bool owner;

	this(Object rhs)
	{
		resource = rhs;
		owner = true;
		writefln("%x constructed for %x",this,resource);
	}

	this(AutoPtr rhs)
	{		
		resource = rhs.release();
		owner = true;
		writefln("%x obtaining %x from %x",this,resource,rhs);
	}
	
	~this()
	{
		if (resource && owner)
		{
			writefln("%x deleting %x",this,resource);
			delete resource;
			resource = null;
		}		
	}
protected:
	Object value()
	{
		return resource;
	}
	
	Object release()
	{
		writefln("%x releasing %x",this,resource);
		owner = false;
		return value();
	}
}

class Resource
{
	char[] name = "11 was a racehorse";
	char[] toString() { return name; }
}

void main()
{
	Resource Bob = new Resource();
	auto AutoPtr p = new AutoPtr(Bob);
	auto AutoPtr q = new AutoPtr(p);
	writefln("end of main");
}

On Mon, 27 Jun 2005 16:52:09 +1200, Regan Heath <regan netwin.co.nz> wrote:
 Looking at the code given I believe my code sample can be modified to  
 achieve it. opAssign isn't required, you simply use a copy constructor  
 (as I have done). Why can't it be a class? Specifically why can't it be  
 an 'auto' class reference (which will exist on the stack in future)?

 Regan

 p.s. It's "Regan" not "Reagan" ;)

 On Sun, 26 Jun 2005 21:39:48 -0700, Andrew Fedoniouk  
 <news terrainformatica.com> wrote:
 Reagan, let's take a look on more simple case - implementation
 of 'auto' pointer (this is simplifed version)

   struct Auto(X)
   {

     // constructor.
     this( X _ptr = 0 ) { ptr = _ptr; owner = true; }

     // the destructor deletes ptr only if we are the owner.
     ~this() { if( owner && ptr ) delete ptr; }

     // the assignment operator.
     void opAssign( X r )
     {
         if( owner && ptr ) delete ptr;
         ptr = r;
         owner = true;
     }

     // the assignment operator. transfer
     // ownership. (not full implementation, sic)
     void opAssign( Auto r )
     {
         if( owner && ptr ) delete ptr;
         ptr = r.release();
         owner = true;
     }

     X value() { return  ptr; }

     // release returns the ptr value and releases ownership
     // if we were previously the owner.

     X release()
     {
       owner = false;
       return value();
     }
     private X ptr;
     private bool owner;
   }

 -------------------------

 {
     Auto!(Object) autop = new Object(); // first instance
     autop = new Object(); // first instance will be deleted;
                                            // and autop contains second
 instance.
 } // second instance will be deleted here
   // even in case of exception.

 This is basic auto_ptr used in C++
 and it simplfies life a lot.
 opAssign and dtors are crucial here.
 And such Auto(X) must be a struct.
 Classes do not work here.

 Andrew.


 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0boj0r23k2f5 nrage.netwin.co.nz...
 I got little or no comment on this idea/code I posted to another  
 (rather
 large) thread here, I figured I'd post it again and see what people
 thought.

 After reading this thread (the one I mention above) and also this one
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on  
 the
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for  
 classes
 (assigning a value to a reference seems illogical) but would be a good  
 for
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?

 The idea for D (or so I've read) is that opAssign isn't required,  
 instead
 you use a copy constructor. Now, I've little experience with ref  
 counting,
 or with writing an implementation of one, but, does this work?

 If not, why not? What's missing, what else is required? Overload of '.'
 perhaps? The ability to pass a RefPtr as the object being counted?

 import std.c.windows.windows;
 import std.random;
 import std.thread;
 import std.stdio;

 class RefPtr
 {
 RefPtr parent;
 Object resource = null;
 int refs = 0;

 this(Object res)
 {
 resource = res;
 writefln("ThreadID=(",Thread.getThis().id,") Initial RefPtr for
 resource=(",resource,")");
 increment(); }

 this(RefPtr rhs)
 {
 parent = rhs;
 parent.increment();
 writefln("ThreadID=(",Thread.getThis().id,") Ref=(",parent.refs,") for
 resource=(",parent.resource,")");
 }

 ~this()
 {
 int r;

 if ((r = decrement()) == 0)
 {
 writefln("ThreadID=(",Thread.getThis().id,") release last ref
 Ref=(",r,")");
 if (parent) parent = null;
 else if (resource)
 {
 writefln("ThreadID=(",Thread.getThis().id,") delete
 resource=(",resource,")");
 delete resource;
 resource = null;
 }
 }
 writefln("ThreadID=(",Thread.getThis().id,") release Ref=(",r,")");
 }
 protected:
 int increment()
 {
 int ret;

 if (parent) ret = parent.increment();
 else {
 synchronized(this)
 {
 ret = ++refs;
 writefln("ThreadID=(",Thread.getThis().id,") increment to
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 int decrement()
 {
 int ret;

 if (parent) ret = parent.decrement();
 else {
 synchronized(this)
 {
 ret = --refs;
 writefln("ThreadID=(",Thread.getThis().id,") decrement to
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 }

 class Resource
 {
 char[] name = "11 was a racehorse";
 char[] toString() { return name; }
 }

 RefPtr pbob;

 static this()
 {
 pbob = new RefPtr(new Resource());
 }

 static ~this()
 {
 delete pbob;
 }

 void main()
 {
 Thread[] threads;
 int i;

 writefln("ThreadID=(",Thread.getThis().id,") is the main thread");

 for(i = 0; i < 10; i++)
 {
 threads ~= new Thread(&thread_function,null);
 threads[$-1].start();
 }

 while(true)
 {
 i = 0;
 foreach(Thread t; threads)
 {
 if (t.getState() == Thread.TS.TERMINATED) i++;
 }
 if (i == 10) break;
 Sleep(100);
 }

 writefln("Main exiting");
 }

 int thread_function(void* isnull)
 {
 auto RefPtr p = new RefPtr(pbob);
 Sleep(1000+rand()%1000); return 0;
 }

 Regan



Jun 26 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss0is0r223k2f5 nrage.netwin.co.nz...
 AutoPtr in D...

 import std.c.windows.windows;
 import std.random;
 import std.thread;
 import std.stdio;

 class AutoPtr
 {
 Object resource = null;
 bool owner;

 this(Object rhs)
 {
 resource = rhs;
 owner = true;
 writefln("%x constructed for %x",this,resource);
 }

 this(AutoPtr rhs)
 { resource = rhs.release();
 owner = true;
 writefln("%x obtaining %x from %x",this,resource,rhs);
 }

 ~this()
 {
 if (resource && owner)
 {
 writefln("%x deleting %x",this,resource);
 delete resource;
 resource = null;
 } }
 protected:
 Object value()
 {
 return resource;
 }

 Object release()
 {
 writefln("%x releasing %x",this,resource);
 owner = false;
 return value();
 }
 }

 class Resource
 {
 char[] name = "11 was a racehorse";
 char[] toString() { return name; }
 }

 void main()
 {
 Resource Bob = new Resource();
 auto AutoPtr p = new AutoPtr(Bob);
 auto AutoPtr q = new AutoPtr(p);
 writefln("end of main");
 }

void main() { Resource Bob = new Resource(); Resource Mary = new Resource(); auto AutoPtr p = new AutoPtr(Bob); auto AutoPtr q; p = Mary; // What will happen here? q = p; // And here? } In D, for classes, q = p means assignment of reference and not value this is why classes do not work here. Having opAssign implemented allows to implement control of assigning values - to catch moment when *variable* is getting new value. And there are many useful implications of this feature. The Auto is just one example.
 On Mon, 27 Jun 2005 16:52:09 +1200, Regan Heath <regan netwin.co.nz> 
 wrote:
 Looking at the code given I believe my code sample can be modified to 
 achieve it. opAssign isn't required, you simply use a copy constructor 
 (as I have done). Why can't it be a class? Specifically why can't it be 
 an 'auto' class reference (which will exist on the stack in future)?

 Regan

 p.s. It's "Regan" not "Reagan" ;)


Ooops, terribly sorry :)))
 On Sun, 26 Jun 2005 21:39:48 -0700, Andrew Fedoniouk 
 <news terrainformatica.com> wrote:
 Reagan, let's take a look on more simple case - implementation
 of 'auto' pointer (this is simplifed version)

   struct Auto(X)
   {

     // constructor.
     this( X _ptr = 0 ) { ptr = _ptr; owner = true; }

     // the destructor deletes ptr only if we are the owner.
     ~this() { if( owner && ptr ) delete ptr; }

     // the assignment operator.
     void opAssign( X r )
     {
         if( owner && ptr ) delete ptr;
         ptr = r;
         owner = true;
     }

     // the assignment operator. transfer
     // ownership. (not full implementation, sic)
     void opAssign( Auto r )
     {
         if( owner && ptr ) delete ptr;
         ptr = r.release();
         owner = true;
     }

     X value() { return  ptr; }

     // release returns the ptr value and releases ownership
     // if we were previously the owner.

     X release()
     {
       owner = false;
       return value();
     }
     private X ptr;
     private bool owner;
   }

 -------------------------

 {
     Auto!(Object) autop = new Object(); // first instance
     autop = new Object(); // first instance will be deleted;
                                            // and autop contains second
 instance.
 } // second instance will be deleted here
   // even in case of exception.

 This is basic auto_ptr used in C++
 and it simplfies life a lot.
 opAssign and dtors are crucial here.
 And such Auto(X) must be a struct.
 Classes do not work here.

 Andrew.


 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0boj0r23k2f5 nrage.netwin.co.nz...
 I got little or no comment on this idea/code I posted to another 
 (rather
 large) thread here, I figured I'd post it again and see what people
 thought.

 After reading this thread (the one I mention above) and also this one
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on 
 the
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for 
 classes
 (assigning a value to a reference seems illogical) but would be a good 
 for
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?

 The idea for D (or so I've read) is that opAssign isn't required, 
 instead
 you use a copy constructor. Now, I've little experience with ref 
 counting,
 or with writing an implementation of one, but, does this work?

 If not, why not? What's missing, what else is required? Overload of '.'
 perhaps? The ability to pass a RefPtr as the object being counted?

 import std.c.windows.windows;
 import std.random;
 import std.thread;
 import std.stdio;

 class RefPtr
 {
 RefPtr parent;
 Object resource = null;
 int refs = 0;

 this(Object res)
 {
 resource = res;
 writefln("ThreadID=(",Thread.getThis().id,") Initial RefPtr for
 resource=(",resource,")");
 increment(); }

 this(RefPtr rhs)
 {
 parent = rhs;
 parent.increment();
 writefln("ThreadID=(",Thread.getThis().id,") Ref=(",parent.refs,") for
 resource=(",parent.resource,")");
 }

 ~this()
 {
 int r;

 if ((r = decrement()) == 0)
 {
 writefln("ThreadID=(",Thread.getThis().id,") release last ref
 Ref=(",r,")");
 if (parent) parent = null;
 else if (resource)
 {
 writefln("ThreadID=(",Thread.getThis().id,") delete
 resource=(",resource,")");
 delete resource;
 resource = null;
 }
 }
 writefln("ThreadID=(",Thread.getThis().id,") release Ref=(",r,")");
 }
 protected:
 int increment()
 {
 int ret;

 if (parent) ret = parent.increment();
 else {
 synchronized(this)
 {
 ret = ++refs;
 writefln("ThreadID=(",Thread.getThis().id,") increment to
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 int decrement()
 {
 int ret;

 if (parent) ret = parent.decrement();
 else {
 synchronized(this)
 {
 ret = --refs;
 writefln("ThreadID=(",Thread.getThis().id,") decrement to
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 }

 class Resource
 {
 char[] name = "11 was a racehorse";
 char[] toString() { return name; }
 }

 RefPtr pbob;

 static this()
 {
 pbob = new RefPtr(new Resource());
 }

 static ~this()
 {
 delete pbob;
 }

 void main()
 {
 Thread[] threads;
 int i;

 writefln("ThreadID=(",Thread.getThis().id,") is the main thread");

 for(i = 0; i < 10; i++)
 {
 threads ~= new Thread(&thread_function,null);
 threads[$-1].start();
 }

 while(true)
 {
 i = 0;
 foreach(Thread t; threads)
 {
 if (t.getState() == Thread.TS.TERMINATED) i++;
 }
 if (i == 10) break;
 Sleep(100);
 }

 writefln("Main exiting");
 }

 int thread_function(void* isnull)
 {
 auto RefPtr p = new RefPtr(pbob);
 Sleep(1000+rand()%1000); return 0;
 }

 Regan




Jun 26 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Sun, 26 Jun 2005 22:40:03 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message
 void main()
 {
    Resource Bob = new Resource();
    Resource Mary = new Resource();

    auto AutoPtr p = new AutoPtr(Bob);
    auto AutoPtr q;

    p = Mary; // What will happen here?

ERROR: cannot implicitly convert expression (Mary) of type autoptr.Resource to autoptr.AutoPtr (note "autoptr" was the name of the source file in my case)
    q = p; // And here?

The reference 'q' will now equal the reference 'p'.
 }

 In D, for classes,

       q = p

 means assignment of reference and not value

Correct.
 this is why classes do not work here.

I disagree. They *work* fine. If you meant to create a new AutoPtr and transfer ownership you should have used the copy constructor, eg. q = new AutoPtr(p); What you did instead was make a 2nd reference to the same AutoPtr. This is part of D's design goals (as I read them on the web site) classes shall not have opAssign instead you shall use a copy constructor. I can however see/understand the desire to make "q = p" an error, in order to prevent incorrect usage (as above). Have you a suggestion as to how to do that? (one that does not involve adding opAssign to classes) The way forwards, as I see it, is to solve the question above. Asking for opAssign for classes will likely get us nowhere. Of course the other option is to use a struct, the thread you posted earlier: http://www.digitalmars.com/d/archives/7988.html tends to suggest to me that struct will not be modified to solve this issue.
 Having opAssign implemented allows
 to implement control of assigning values -
 to catch moment when *variable*
 is getting new value.

Notice you say "value" above. You do not assign values to reference, you assign references to references, thus opAssign has no place in classes. (I believe we've already agreed to this point elsewhere). Regan
Jun 27 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss0r4ry023k2f5 nrage.netwin.co.nz...
 On Sun, 26 Jun 2005 22:40:03 -0700, Andrew Fedoniouk 
 <news terrainformatica.com> wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message
 void main()
 {
    Resource Bob = new Resource();
    Resource Mary = new Resource();

    auto AutoPtr p = new AutoPtr(Bob);
    auto AutoPtr q;

    p = Mary; // What will happen here?

ERROR: cannot implicitly convert expression (Mary) of type autoptr.Resource to autoptr.AutoPtr (note "autoptr" was the name of the source file in my case)
    q = p; // And here?

The reference 'q' will now equal the reference 'p'.
 }

 In D, for classes,

       q = p

 means assignment of reference and not value

Correct.
 this is why classes do not work here.

I disagree. They *work* fine. If you meant to create a new AutoPtr and transfer ownership you should have used the copy constructor, eg. q = new AutoPtr(p);

Intention for q = p here is: 1) to free object q is holding. 2) to transfer ownership from p to q. If p and q will be deleted then only owner will delete owned object - not both of them.
 What you did instead was make a 2nd reference to the same AutoPtr.

 This is part of D's design goals (as I read them on the web site) classes 
 shall not have opAssign instead you shall use a copy constructor.

 I can however see/understand the desire to make "q = p" an error, in order 
 to prevent incorrect usage (as above). Have you a suggestion as to how to 
 do that? (one that does not involve adding opAssign to classes)

 The way forwards, as I see it, is to solve the question above. Asking for 
 opAssign for classes will likely get us nowhere.

 Of course the other option is to use a struct, the thread you posted 
 earlier:
   http://www.digitalmars.com/d/archives/7988.html

 tends to suggest to me that struct will not be modified to solve this 
 issue.

 Having opAssign implemented allows
 to implement control of assigning values -
 to catch moment when *variable*
 is getting new value.

Notice you say "value" above. You do not assign values to reference, you assign references to references, thus opAssign has no place in classes. (I believe we've already agreed to this point elsewhere).

Yes. This is why I am using struct in my example. For classes '=' is copying references. For sructures it is copying values. The problem is that for structures '=' silently overrides values without any control from your side.
Jun 27 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 27 Jun 2005 08:55:50 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0r4ry023k2f5 nrage.netwin.co.nz..
 On Sun, 26 Jun 2005 22:40:03 -0700, Andrew Fedoniouk
 <news terrainformatica.com> wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message
 void main()
 {
    Resource Bob = new Resource();
    Resource Mary = new Resource();

    auto AutoPtr p = new AutoPtr(Bob);
    auto AutoPtr q;

    p = Mary; // What will happen here?

ERROR: cannot implicitly convert expression (Mary) of type autoptr.Resource to autoptr.AutoPtr (note "autoptr" was the name of the source file in my case)
    q = p; // And here?

The reference 'q' will now equal the reference 'p'.
 }

 In D, for classes,

       q = p

 means assignment of reference and not value

Correct.
 this is why classes do not work here.

I disagree. They *work* fine. If you meant to create a new AutoPtr and transfer ownership you should have used the copy constructor, eg. q = new AutoPtr(p);

Intention for q = p here is: 1) to free object q is holding. 2) to transfer ownership from p to q. If p and q will be deleted then only owner will delete owned object - not both of them.

I know what the intention is. My code does that, provided you do not write "q = p" but instead write: q = new AutoPtr(p); You could also write "p = null;" to signal the GC may collect (tho if it's auto it probably doesn't?) Stop thinking in C++ and start thinking in D. I repeat. D does not have opAssign, where you want opAssign use a copy constructor. The _only_ problem I can see with the code I posted is that you cannot prevent someone accidently writing "q = p" when they meant to write "q = new AutoPtr(p);". I'm not sure what the solution is, but it is the way forward with this concept IMO.
 Notice you say "value" above. You do not assign values to reference, you
 assign references to references, thus opAssign has no place in classes.  
 (I
 believe we've already agreed to this point elsewhere).

Yes. This is why I am using struct in my example. For classes '=' is copying references. For sructures it is copying values. The problem is that for structures '=' silently overrides values without any control from your side.

Structs also do not have ctor/dtor. Classes are the only option for AutoPtr IMO. Regan
Jun 27 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss1p7dkp23k2f5 nrage.netwin.co.nz...
 On Mon, 27 Jun 2005 08:55:50 -0700, Andrew Fedoniouk 
 <news terrainformatica.com> wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0r4ry023k2f5 nrage.netwin.co.nz..
 On Sun, 26 Jun 2005 22:40:03 -0700, Andrew Fedoniouk
 <news terrainformatica.com> wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message
 void main()
 {
    Resource Bob = new Resource();
    Resource Mary = new Resource();

    auto AutoPtr p = new AutoPtr(Bob);
    auto AutoPtr q;

    p = Mary; // What will happen here?

ERROR: cannot implicitly convert expression (Mary) of type autoptr.Resource to autoptr.AutoPtr (note "autoptr" was the name of the source file in my case)
    q = p; // And here?

The reference 'q' will now equal the reference 'p'.
 }

 In D, for classes,

       q = p

 means assignment of reference and not value

Correct.
 this is why classes do not work here.

I disagree. They *work* fine. If you meant to create a new AutoPtr and transfer ownership you should have used the copy constructor, eg. q = new AutoPtr(p);

Intention for q = p here is: 1) to free object q is holding. 2) to transfer ownership from p to q. If p and q will be deleted then only owner will delete owned object - not both of them.

I know what the intention is. My code does that, provided you do not write "q = p" but instead write: q = new AutoPtr(p); You could also write "p = null;" to signal the GC may collect (tho if it's auto it probably doesn't?) Stop thinking in C++ and start thinking in D. I repeat. D does not have opAssign, where you want opAssign use a copy constructor.

If I have C++ why do I need degradation? And about yours q = new AutoPtr(p); It does not call destructor of q on assignment so what is the point of this AutoPtr(p). 'Auto' means that it manages pointer for me.
 The _only_ problem I can see with the code I posted is that you cannot 
 prevent someone accidently writing "q = p" when they meant to write "q = 
 new AutoPtr(p);". I'm not sure what the solution is, but it is the way 
 forward with this concept IMO.

 Notice you say "value" above. You do not assign values to reference, you
 assign references to references, thus opAssign has no place in classes. 
 (I
 believe we've already agreed to this point elsewhere).

Yes. This is why I am using struct in my example. For classes '=' is copying references. For sructures it is copying values. The problem is that for structures '=' silently overrides values without any control from your side.

Structs also do not have ctor/dtor. Classes are the only option for AutoPtr IMO.

Without control of assignment and ctor/dtor it is impossible to implement the whole class of smart pointers and such thing as COW string. We agreed that opAssign in classes makes no sense. So only structs left. About real COW strings and motivation: http://www.wxwidgets.org/manuals/2.6.0/wx_wxstringoverview.html#wxstringrefcount It is impossible to reproduce in D such class and any other class based on pointer management paradigm. I mean to define string class which will behave as a string type and not just a set of C style functions dealing with struct slice { char* ptr; size_t length; }. Which is fine if we agree that D is a next letter after C and not after C++. Again, it depends on author motivation. As far as I can see there are three features that C++ has and D doesn't have: struct(opAssign/ctor/dtor) and const. If they will be there then D may be considered as a successor of C++. With all possible implications. IMHO. Andrew.
Jun 27 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 27 Jun 2005 21:31:18 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:
 And about yours q = new AutoPtr(p);
 It does not call destructor of q on assignment

True, In D the destructor is only called when the object is collected, or you call "delete".
 so what is the point of this AutoPtr(p).
 'Auto' means that it manages pointer for
 me.

Well, it seems to me that one solution would be for an 'auto' class to be deleted the moment it is no longer referenced. (rather than waiting till the end of the scope). Not sure if that is possible, but it makes sense. So, that just leaves the problem of preventing a normal reference assignment, got any ideas?
 Notice you say "value" above. You do not assign values to reference,  
 you
 assign references to references, thus opAssign has no place in  
 classes.
 (I
 believe we've already agreed to this point elsewhere).

Yes. This is why I am using struct in my example. For classes '=' is copying references. For sructures it is copying values. The problem is that for structures '=' silently overrides values without any control from your side.

Structs also do not have ctor/dtor. Classes are the only option for AutoPtr IMO.

Without control of assignment and ctor/dtor it is impossible to implement the whole class of smart pointers and such thing as COW string.

Perhaps, but that does not mean we require opAssign. There may be another way.
 We agreed that opAssign in classes makes no
 sense. So only structs left.

We could still use classes without using opAssign. This is where I have concentrated my efforts since reading the thread you found from back in 2002 where they decide on using 'auto'.
 About real COW strings and motivation:
 http://www.wxwidgets.org/manuals/2.6.0/wx_wxstringoverview.html#wxstringrefcount

 It is impossible to reproduce in D such class
 and any other class based on pointer management
 paradigm.

It's currently impossible to reproduce classes that work in 'exactly' the same way. The D solution does not have to be exactly the same, provided it still achieve the goals.
 I mean to define string class which
 will behave as a string type and not just a set
 of C style functions dealing with
 struct slice { char* ptr; size_t length; }.

What would this string class do that char[] cannot? If your only motivation is to provide a way of passing 'const' string data then I think enforcing readonly for "const" and "in" is much more productive and would also work for other types in D.
 Again, it depends on author motivation.

 As far as I can see there are three features
 that C++ has and D doesn't have:
 struct(opAssign/ctor/dtor) and const.

 If they will be there then D may be considered
 as a successor of C++. With all possible
 implications.

You have to remember that D is different to C++ and will therefore solve the same problems in different ways. I'm looking for a D solution. You seem to be asking for D to become C++. I agree D cannot solve these problems, yet, but the best way to fix that is to think of a way it could solve them *without* changing some of the fundamental design decisions that make D what it is. IMO. Regan
Jun 27 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss2fh7pw23k2f5 nrage.netwin.co.nz...
 On Mon, 27 Jun 2005 21:31:18 -0700, Andrew Fedoniouk 
 <news terrainformatica.com> wrote:
 And about yours q = new AutoPtr(p);
 It does not call destructor of q on assignment

True, In D the destructor is only called when the object is collected, or you call "delete".
 so what is the point of this AutoPtr(p).
 'Auto' means that it manages pointer for
 me.

Well, it seems to me that one solution would be for an 'auto' class to be deleted the moment it is no longer referenced. (rather than waiting till the end of the scope). Not sure if that is possible, but it makes sense. So, that just leaves the problem of preventing a normal reference assignment, got any ideas?
 Notice you say "value" above. You do not assign values to reference, 
 you
 assign references to references, thus opAssign has no place in 
 classes.
 (I
 believe we've already agreed to this point elsewhere).

Yes. This is why I am using struct in my example. For classes '=' is copying references. For sructures it is copying values. The problem is that for structures '=' silently overrides values without any control from your side.

Structs also do not have ctor/dtor. Classes are the only option for AutoPtr IMO.

Without control of assignment and ctor/dtor it is impossible to implement the whole class of smart pointers and such thing as COW string.

Perhaps, but that does not mean we require opAssign. There may be another way.
 We agreed that opAssign in classes makes no
 sense. So only structs left.

We could still use classes without using opAssign. This is where I have concentrated my efforts since reading the thread you found from back in 2002 where they decide on using 'auto'.
 About real COW strings and motivation:
 http://www.wxwidgets.org/manuals/2.6.0/wx_wxstringoverview.html#wxstringrefcount

 It is impossible to reproduce in D such class
 and any other class based on pointer management
 paradigm.

It's currently impossible to reproduce classes that work in 'exactly' the same way. The D solution does not have to be exactly the same, provided it still achieve the goals.
 I mean to define string class which
 will behave as a string type and not just a set
 of C style functions dealing with
 struct slice { char* ptr; size_t length; }.

What would this string class do that char[] cannot? If your only motivation is to provide a way of passing 'const' string data then I think enforcing readonly for "const" and "in" is much more productive and would also work for other types in D.
 Again, it depends on author motivation.

 As far as I can see there are three features
 that C++ has and D doesn't have:
 struct(opAssign/ctor/dtor) and const.

 If they will be there then D may be considered
 as a successor of C++. With all possible
 implications.

You have to remember that D is different to C++ and will therefore solve the same problems in different ways. I'm looking for a D solution. You seem to be asking for D to become C++. I agree D cannot solve these problems, yet, but the best way to fix that is to think of a way it could solve them *without* changing some of the fundamental design decisions that make D what it is.

Sigh... Round came to the end. The cycle is closed. I've started screaming about opAssign after I realized that const arrays/pointers are persons non grata in D for some ideological reasons. So idea was to have features in language allowing to implement const types by myself. That is why opAssign and co. Again, currently there are no solutions in D for strings and smart pointers. I mean real and natural solutions. -------------------- And about string and char[] char[] is a slice - fragment of character sequence and not a string. String variable represent/manage storage of textual data. In contrary char[] is just a poitner bundled with length. It is not more than char* - it does not have a concept of ownership - who and when can or cannot modify a string. BTW: char* and char[] are different in principle: in char* length of string is part of data (zero terminated). and in char[] it is part of particular variable. This is also why char[] is not a string. char[] can be used as a string field in some structures only if it *always* points to some *unique* memory location - owns it. So you can *safely* manipulate owning text. By itself char[] is convenient way to define and pass slices to functions - temporary (short living) structures. But here usability is limited by lack of const too. Sigh... string class is a COW manager of character data. Using it you don't need to invent COW algorithms each time when you are dealing with text and to account what source this text came from - it significantly reduces complexity of code, increases robustness of system in general and uses memory in most optimal way. Andrew.
Jun 28 2005
next sibling parent reply Brad Beveridge <brad somewhere.net> writes:
<SNIP>
 
 Sigh... Round came to the end. The cycle is closed.
 
 I've started screaming about opAssign after I realized
 that const arrays/pointers are persons non grata in D for
 some ideological reasons. So idea was to have features in language
 allowing to implement const types by myself. That is why
 opAssign and co.
 

<snip>
 Andrew. 
 
 

This topic, and many like them, have been kicked around for quite a while now. I would like to hear what Walter has to say on all this. Walter - do you have particular reasons why opAssign, or "const" can't/won't/shouldn't be implemented in D? To me it looks like Regan and Andrew (and many others) both want the same thing in the end - Regan is trying to do it within the confines of the current language (and lets face it, not having an easy time of it, which should tell us something), and Andrew has concluded that it is impossible without changes to D. So, what do you say? Are Andrew and Regan trying to do something that isn't required in D? Or is D lacking important functionality? Cheers Brad
Jun 28 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Tue, 28 Jun 2005 11:20:36 -0700, Brad Beveridge <brad somewhere.net>  
wrote:
 <SNIP>
  Sigh... Round came to the end. The cycle is closed.
  I've started screaming about opAssign after I realized
 that const arrays/pointers are persons non grata in D for
 some ideological reasons. So idea was to have features in language
 allowing to implement const types by myself. That is why
 opAssign and co.

<snip>
 Andrew.

This topic, and many like them, have been kicked around for quite a while now. I would like to hear what Walter has to say on all this. Walter - do you have particular reasons why opAssign, or "const" can't/won't/shouldn't be implemented in D? To me it looks like Regan and Andrew (and many others) both want the same thing in the end - Regan is trying to do it within the confines of the current language (and lets face it, not having an easy time of it, which should tell us something), and Andrew has concluded that it is impossible without changes to D. So, what do you say? Are Andrew and Regan trying to do something that isn't required in D? Or is D lacking important functionality?

Good post. Regan
Jun 28 2005
prev sibling parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Brad Beveridge wrote:

 So, what do you say?
 
 Are Andrew and Regan trying to do something that isn't required in D?

Maybe isn't required if there was some other way to protect the strings.
 Or is D lacking important functionality?

Yes, as most have pointed out by now: D is sorely lacking a "readonly" ! This makes everyone look for workarounds like unsafe character arrays (just like in old C) or string wrapper classes (just like in Java)... Instead of having a "readonly char[]" parameter ? (old name "const" changed, for political reasons) My 2 öre, --anders
Jun 29 2005
prev sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Tue, 28 Jun 2005 11:08:49 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:
 You have to remember that D is different to C++ and will therefore solve
 the same problems in different ways. I'm looking for a D solution. You
 seem to be asking for D to become C++. I agree D cannot solve these
 problems, yet, but the best way to fix that is to think of a way it  
 could
 solve them *without* changing some of the fundamental design decisions
 that make D what it is.

Sigh... Round came to the end. The cycle is closed. I've started screaming about opAssign after I realized that const arrays/pointers are persons non grata in D for some ideological reasons. So idea was to have features in language allowing to implement const types by myself. That is why opAssign and co. Again, currently there are no solutions in D for strings and smart pointers. I mean real and natural solutions.

I agree.
 --------------------
 And about string and char[]
 char[] is a slice - fragment of character sequence and not a string.
 String variable represent/manage storage of textual data.

I still don't see the difference. A char[] represents textual data, a char manages storage of textual data (it has a length you can set, it handles concatenation, with const/in protection it would handle ownership)
 In contrary char[] is just a poitner bundled with length.
 It is not more than char* - it does not have a concept
 of ownership - who and when can or cannot modify a string.

It would if we had readonly protection for 'const' and 'in' thought, right?
 BTW: char* and char[] are different in principle:
 in char* length of string is part of data (zero terminated).

I know, I have a C background.
 and in char[] it is part of particular variable. This
 is also why char[] is not a string.

So char* is a string but char[] isn't? That statement seems odd to me.
 char[] can be used as a string field in some structures only
 if it *always* points to some *unique* memory location -
 owns it. So you can *safely* manipulate owning text.

You are supposed to use COW. I agree without some sort of const/in protection you're left guessing whether you own it. With const/in protection you *know* whether you own a string, if it's const or passed as 'in' you dont, otherwise you do, right?
 By itself char[] is convenient way to define and pass slices
 to functions - temporary (short living) structures.
 But here usability is limited by lack of const too. Sigh...

Sure, we need const/in protection. We both agree there.
 string class is a COW manager of character data.

Aha! So this is the definition you're using. I believe with const/in protection char[] can be used with COW safely and with surety (no guessing).
 Using it you
 don't need to invent COW algorithms each time when you are dealing
 with text and to account what source this text came from - it  
 significantly
 reduces complexity of code, increases robustness of system in general
 and uses memory in most optimal way.

Agreed. We need something to help us use COW without guessing or adding complicated code to handle it. In short, if char[] had const/in protection do you agree that it would be the 'string class' you require? If not, what other factors are missing? And would those factors be handled if we could design and use an AutoPtr to a char[]? Regan.
Jun 28 2005
prev sibling next sibling parent reply David Medlock <noone nowhere.com> writes:
Regan Heath wrote:

 I got little or no comment on this idea/code I posted to another 
 (rather  large) thread here, I figured I'd post it again and see what 
 people  thought.
 
 After reading this thread (the one I mention above) and also this one  
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html
 
 It appears you need a type that:
 
 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.
 
 It seems (from recent posts) that Walter plans to put auto classes on 
 the  stack, so they'd then fulfil requirements 1 and 2.
 
 But what about 3? To me, opAssign isn't generally a good idea for 
 classes  (assigning a value to a reference seems illogical) but would be 
 a good for  structs (they're value types, assignment assigns a value).
 
 Can you do reference counting without opAssign?
 

 Regan

I thought the stack based types were to be useful for speed gains, not managing memory. Why do you need reference counting? Why not just let the gc deallocator handle it? Are you running into a problem running out of memory? Deallocation should not be such a central concept, otherwise we might as well ditch the gc. In your threads example, you probably would use a manager to watch the threads, not rely on the runtime to tell you when to free one. -DavidM
Jun 29 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Wed, 29 Jun 2005 09:41:01 -0400, David Medlock <noone nowhere.com>  
wrote:
 Regan Heath wrote:

 I got little or no comment on this idea/code I posted to another  
 (rather  large) thread here, I figured I'd post it again and see what  
 people  thought.
  After reading this thread (the one I mention above) and also this one   
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html
  It appears you need a type that:
  1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.
  It seems (from recent posts) that Walter plans to put auto classes on  
 the  stack, so they'd then fulfil requirements 1 and 2.
  But what about 3? To me, opAssign isn't generally a good idea for  
 classes  (assigning a value to a reference seems illogical) but would  
 be a good for  structs (they're value types, assignment assigns a  
 value).
  Can you do reference counting without opAssign?

 Regan

I thought the stack based types were to be useful for speed gains, not managing memory.

Correct. We need/want the reference counter or auto pointer to be as fast as possible because they're typically instantiated and destroyed with abandon. Andrew posted a good example of this, where his graphics library needs to obtain ownership of a graphics context on every call to the 'paint' method, it needs to get it, use it and release it, this function is called very often.
 Why do you need reference counting?

I don't, yet. Andrew does, now.
 Why not just let the gc deallocator handle it?

Because the problem we're trying to solve is 'ownership' and 'efficiency of memory'. In other words knowing when you need to copy a char[] not just guessing. At the moment people are likely copying to be safe, meaning a bunch of memory allocations you dont actually need. IMO we can solve these issues by enforcing readonly for 'const/readonly/whatever' and 'in parameters', eg. //this class returns readonly access to a string it owns class Foo { char[] ownedByFoo; readonly char[] getIt() { return ownedByFoo[0..5]; } } void mutate(char[] arr) { arr[0] = 'b'; //err 'arr' is readonly } void mutate2(inout char[] arr) { arr[0] = 'b'; } Foo f = new Foo(); char[] p = f.getIt(); //'readonly' not required** char[] s = p.dup; //dup does not copy readonly flag p[0] = 'a'; //error p is readonly s[0] = 'a'; //ok mutate(p); //ok, function def is actually in error mutate(s); //ok mutate2(p); //error p is readonly mutate2(s); //ok ** on variable decls (it's _not_ a seperate type, it's the _same_ type with a readonly bit flagged). All types could have a readonly bit. Ideally only during compile, i.e. the compiler does a pass where it verifies no variable it has previously flagged as readonly (does the flagging during it's normal existing passes) has been passed as a mutable variable, or modified with inbuilt operator etc.
 Are you running into a problem running out of memory?

No. It's all about ownership and not having to guess when COW should be actioned. A reference counted string knows it's the only reference thus does not need to dup. An autopointer knows whether it's the owner and if not does a dup. Currently char[] has no such knowledge.
 Deallocation should not be such a central concept, otherwise we might as  
 well ditch the gc.

Of course not, however for certain types you need to action something as they die, refcnt, autoptr and classes with resources to release. 'auto' seems to handle these cases. A remaining problem is that refcnt and autoptr need to fast, so we want them on the stack. Walter has mentioned 'auto' classes will eventually be on the stack, yay. The last issue is that it need to be illegal to assign an RefCnt to another RefCnt without increasing the reference count, i.e. RefCnt p = new RefCnt(object); RefCnt s = p; //should either be illegal, or should increment refcnt. This is currently impossible in D.
 In your threads example, you probably would use a manager to watch the  
 threads, not rely on the runtime to tell you when to free one.

A manager? you mean a whole other thread just to watch the first 10 threads? I've since decided it's just simpler to store the whole lot of them in an array, at least then I know the references cant be null (which was the problem with initial code - not handling null case). You can add null handling, and assume null means terminated and the initial code functions correctly. All in all, if you read the entire thread, you'll come to Brad's post which sums the situation up nicely. We need some direction from Walter, either: - You dont need those in D (and reasoning) - Heres how we do it in D (example) - I see, I will think about soln - Ok you're right I will add x,y and z Regan
Jun 29 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss5jwkvj23k2f5 nrage.netwin.co.nz...
 On Wed, 29 Jun 2005 09:41:01 -0400, David Medlock <noone nowhere.com> 
 wrote:
 Regan Heath wrote:

 I got little or no comment on this idea/code I posted to another 
 (rather  large) thread here, I figured I'd post it again and see what 
 people  thought.
  After reading this thread (the one I mention above) and also this one 
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html
  It appears you need a type that:
  1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.
  It seems (from recent posts) that Walter plans to put auto classes on 
 the  stack, so they'd then fulfil requirements 1 and 2.
  But what about 3? To me, opAssign isn't generally a good idea for 
 classes  (assigning a value to a reference seems illogical) but would 
 be a good for  structs (they're value types, assignment assigns a 
 value).
  Can you do reference counting without opAssign?

 Regan

I thought the stack based types were to be useful for speed gains, not managing memory.

Correct. We need/want the reference counter or auto pointer to be as fast as possible because they're typically instantiated and destroyed with abandon. Andrew posted a good example of this, where his graphics library needs to obtain ownership of a graphics context on every call to the 'paint' method, it needs to get it, use it and release it, this function is called very often.
 Why do you need reference counting?

I don't, yet. Andrew does, now.

Reference counting is one of most famous design patterns. RC as a memory management has its own limitations though ( e.g. cyclic refs ). Its power in those areas where GC is weak. And vice versa. So be able to use both will benefit the whole system twice. Good example of RC is in implementations of COW strings and shared_ptr in C++.
 Why not just let the gc deallocator handle it?

Because the problem we're trying to solve is 'ownership' and 'efficiency of memory'. In other words knowing when you need to copy a char[] not just guessing. At the moment people are likely copying to be safe, meaning a bunch of memory allocations you dont actually need. IMO we can solve these issues by enforcing readonly for 'const/readonly/whatever' and 'in parameters', eg.

Pretty much yes.
 //this class returns readonly access to a string it owns
 class Foo {
   char[] ownedByFoo;
   readonly char[] getIt() {
     return ownedByFoo[0..5];
   }
 }

 void mutate(char[] arr) {
   arr[0] = 'b'; //err 'arr' is readonly
 }

 void mutate2(inout char[] arr) {
   arr[0] = 'b';
 }

 Foo f = new Foo();
 char[] p = f.getIt(); //'readonly' not required**
 char[] s = p.dup; //dup does not copy readonly flag
 p[0] = 'a'; //error p is readonly
 s[0] = 'a'; //ok
 mutate(p);  //ok, function def is actually in error
 mutate(s);  //ok
 mutate2(p); //error p is readonly
 mutate2(s); //ok

 ** on variable decls (it's _not_ a seperate type, it's the _same_ type 
 with a readonly bit flagged).

 All types could have a readonly bit. Ideally only during compile, i.e. the 
 compiler does a pass where it verifies no variable it has previously 
 flagged as readonly (does the flagging during it's normal existing passes) 
 has been passed as a mutable variable, or modified with inbuilt operator 
 etc.

 Are you running into a problem running out of memory?

No. It's all about ownership and not having to guess when COW should be actioned. A reference counted string knows it's the only reference thus does not need to dup. An autopointer knows whether it's the owner and if not does a dup. Currently char[] has no such knowledge.
 Deallocation should not be such a central concept, otherwise we might as 
 well ditch the gc.

Of course not, however for certain types you need to action something as they die, refcnt, autoptr and classes with resources to release. 'auto' seems to handle these cases. A remaining problem is that refcnt and autoptr need to fast, so we want them on the stack. Walter has mentioned 'auto' classes will eventually be on the stack, yay.

Exactly!
 The last issue is that it need to be illegal to assign an RefCnt to 
 another RefCnt without increasing the reference count, i.e.

 RefCnt p = new RefCnt(object);
 RefCnt s = p; //should either be illegal, or should increment refcnt.

 This is currently impossible in D.

Sigh....
 In your threads example, you probably would use a manager to watch the 
 threads, not rely on the runtime to tell you when to free one.

A manager? you mean a whole other thread just to watch the first 10 threads? I've since decided it's just simpler to store the whole lot of them in an array, at least then I know the references cant be null (which was the problem with initial code - not handling null case). You can add null handling, and assume null means terminated and the initial code functions correctly. All in all, if you read the entire thread, you'll come to Brad's post which sums the situation up nicely. We need some direction from Walter, either: - You dont need those in D (and reasoning) - Heres how we do it in D (example) - I see, I will think about soln - Ok you're right I will add x,y and z Regan

Jun 29 2005
parent reply David Medlock <noone nowhere.com> writes:
Andrew Fedoniouk wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message 
 news:opss5jwkvj23k2f5 nrage.netwin.co.nz...
 
On Wed, 29 Jun 2005 09:41:01 -0400, David Medlock <noone nowhere.com> 
wrote:

Regan Heath wrote:


Regan

I thought the stack based types were to be useful for speed gains, not managing memory.

Correct. We need/want the reference counter or auto pointer to be as fast as possible because they're typically instantiated and destroyed with abandon. Andrew posted a good example of this, where his graphics library needs to obtain ownership of a graphics context on every call to the 'paint' method, it needs to get it, use it and release it, this function is called very often.


second) and draw all the needed objects? Dont pass the context to the objects, pass the objects to the context owner. I've written code of this type (managing texture memory in OpenGL for one) and its much easier managed on a per-frame basis than per object. (Sorry if I am not understanding this correctly, couldnt find the example you mention)
Why do you need reference counting?

I don't, yet. Andrew does, now.

Reference counting is one of most famous design patterns. RC as a memory management has its own limitations though ( e.g. cyclic refs ). Its power in those areas where GC is weak. And vice versa. So be able to use both will benefit the whole system twice. Good example of RC is in implementations of COW strings and shared_ptr in C++.

Copy on write is a programming discipline/design. Its still up to the programmer though, const just helps to keep the programmers honest. With (non-transactional)concurrent programming, there is no way to use reference counting without locks at each assignment/deref. Consider: A resource we call A has 2 references to it, owned by a variable in both Thread T and U. Thread T calls deref, but is blocked before it can check the reference count of A. Thread U calls deref as well, sees the reference count is zero, and deallocates the reference. Thread T awakens, sees the reference count is zero, calls destructor....OOPS. Not to mention modern architectures are very cache-sensitive. When you call destructors throughout your code, you 'thrash the cache' jumping around in memory a lot more than a gc calling the same destructors several times in sequence. I reject the assertion that managing references at each object is superior to an external collector(of any type, not simply memory). All those AddRef/Decref/Ref checks are not free. Managed resources should be managed by the code, not the runtime. I'm sure some will disagree with some(or all) of this, but I have yet to see evidence to the contrary. -DavidM
Jun 30 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 09:29:19 -0400, David Medlock <noone nowhere.com>  
wrote:
 Andrew Fedoniouk wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message  
 news:opss5jwkvj23k2f5 nrage.netwin.co.nz...

 On Wed, 29 Jun 2005 09:41:01 -0400, David Medlock <noone nowhere.com>  
 wrote:

 Regan Heath wrote:


 Regan

I thought the stack based types were to be useful for speed gains, not managing memory.

Correct. We need/want the reference counter or auto pointer to be as fast as possible because they're typically instantiated and destroyed with abandon. Andrew posted a good example of this, where his graphics library needs to obtain ownership of a graphics context on every call to the 'paint' method, it needs to get it, use it and release it, this function is called very often.


second) and draw all the needed objects? Dont pass the context to the objects, pass the objects to the context owner. I've written code of this type (managing texture memory in OpenGL for one) and its much easier managed on a per-frame basis than per object. (Sorry if I am not understanding this correctly, couldnt find the example you mention)

I'll let Andrew pick this up if he wants.
 Why do you need reference counting?

I don't, yet. Andrew does, now.

RC as a memory management has its own limitations though ( e.g. cyclic refs ). Its power in those areas where GC is weak. And vice versa. So be able to use both will benefit the whole system twice. Good example of RC is in implementations of COW strings and shared_ptr in C++.

Copy on write is a programming discipline/design. Its still up to the programmer though, const just helps to keep the programmers honest.

Yes, "const" helps make it bug tolerant.
 With (non-transactional)concurrent programming, there is no way to use  
 reference counting without locks at each assignment/deref.

 Consider:
 A resource we call A has 2 references to it, owned by a variable in both  
 Thread T and U.

 Thread T calls deref, but is blocked before it can check the reference  
 count of A.

 Thread U calls deref as well, sees the reference count is zero, and  
 deallocates the reference.

 Thread T awakens, sees the reference count is  zero, calls  
 destructor....OOPS.

Yes, the process needs to be sync'ed/locked.
 Not to mention modern architectures are very cache-sensitive. When you  
 call destructors throughout your code, you 'thrash the cache' jumping  
 around in memory a lot more than a gc calling the same destructors  
 several times in sequence.

 I reject the assertion that managing references at each object is  
 superior to an external collector(of any type, not simply memory). All  
 those AddRef/Decref/Ref checks are not free.

Oh I agree. But I'm not suggesting you'd want to use this for every object. Just certain objects. Being able to reply on the GC is great for the bulk of tasks, however certain objects, those that require explicit deterministic destruction are the ones you'd want to reference count.
 Managed resources should be managed by the code, not the runtime.  I'm  
 sure some will disagree with some(or all) of this, but I have yet to see  
 evidence to the contrary.

Reference counting, like GC, is not a panacea, it's not the perfect solution to all memory management problems. What it is, is a useful tool for certain occasions, a tool which can be bug tolerant and easy to use (if only it were possible to write an implementation which was these things in D). Regan
Jun 30 2005
prev sibling parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"David Medlock" <noone nowhere.com> wrote in message 
news:da0s3i$1309$1 digitaldaemon.com...
 Andrew Fedoniouk wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message 
 news:opss5jwkvj23k2f5 nrage.netwin.co.nz...

On Wed, 29 Jun 2005 09:41:01 -0400, David Medlock <noone nowhere.com> 
wrote:

Regan Heath wrote:


Regan

I thought the stack based types were to be useful for speed gains, not managing memory.

Correct. We need/want the reference counter or auto pointer to be as fast as possible because they're typically instantiated and destroyed with abandon. Andrew posted a good example of this, where his graphics library needs to obtain ownership of a graphics context on every call to the 'paint' method, it needs to get it, use it and release it, this function is called very often.


second) and draw all the needed objects? Dont pass the context to the objects, pass the objects to the context owner. I've written code of this type (managing texture memory in OpenGL for one) and its much easier managed on a per-frame basis than per object. (Sorry if I am not understanding this correctly, couldnt find the example you mention)

window::draw() { auto Graphics g = new Graphics( this ); draw_content(...); } Graphics is contains single memeber HDC (on win32). destructor contains : ReleaseDC(...). Heap allocation of Graphics here is far from optimal.
Why do you need reference counting?

I don't, yet. Andrew does, now.

Reference counting is one of most famous design patterns. RC as a memory management has its own limitations though ( e.g. cyclic refs ). Its power in those areas where GC is weak. And vice versa. So be able to use both will benefit the whole system twice. Good example of RC is in implementations of COW strings and shared_ptr in C++.

Copy on write is a programming discipline/design. Its still up to the programmer though, const just helps to keep the programmers honest. With (non-transactional)concurrent programming, there is no way to use reference counting without locks at each assignment/deref. Consider: A resource we call A has 2 references to it, owned by a variable in both Thread T and U. Thread T calls deref, but is blocked before it can check the reference count of A. Thread U calls deref as well, sees the reference count is zero, and deallocates the reference. Thread T awakens, sees the reference count is zero, calls destructor....OOPS. Not to mention modern architectures are very cache-sensitive. When you call destructors throughout your code, you 'thrash the cache' jumping around in memory a lot more than a gc calling the same destructors several times in sequence. I reject the assertion that managing references at each object is superior to an external collector(of any type, not simply memory). All those AddRef/Decref/Ref checks are not free.

Absolutely true. Refcounting as a sole method of managing memory is bad. The point is simple: Nor GC nor RC are silver bullets. Together they can be considered as a suboptimal solution. To be short - I would like to have a choice in each particular case. The thing is - in D there is no physical way to implement ligthweight RC.
 Managed resources should be managed by the code, not the runtime.  I'm 
 sure some will disagree with some(or all) of this, but I have yet to see 
 evidence to the contrary.

I agree.
 -DavidM 

Jul 01 2005
parent "Ben Hinkle" <bhinkle mathworks.com> writes:
"Andrew Fedoniouk" <news terrainformatica.com> wrote in message 
news:da40ce$1olo$1 digitaldaemon.com...
 "David Medlock" <noone nowhere.com> wrote in message 
 news:da0s3i$1309$1 digitaldaemon.com...
 Andrew Fedoniouk wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message 
 news:opss5jwkvj23k2f5 nrage.netwin.co.nz...

On Wed, 29 Jun 2005 09:41:01 -0400, David Medlock <noone nowhere.com> 
wrote:

Regan Heath wrote:


Regan

I thought the stack based types were to be useful for speed gains, not managing memory.

Correct. We need/want the reference counter or auto pointer to be as fast as possible because they're typically instantiated and destroyed with abandon. Andrew posted a good example of this, where his graphics library needs to obtain ownership of a graphics context on every call to the 'paint' method, it needs to get it, use it and release it, this function is called very often.


second) and draw all the needed objects? Dont pass the context to the objects, pass the objects to the context owner. I've written code of this type (managing texture memory in OpenGL for one) and its much easier managed on a per-frame basis than per object. (Sorry if I am not understanding this correctly, couldnt find the example you mention)

window::draw() { auto Graphics g = new Graphics( this ); draw_content(...); } Graphics is contains single memeber HDC (on win32). destructor contains : ReleaseDC(...). Heap allocation of Graphics here is far from optimal.

As another datapoint MinWin uses a free list to manage Graphics instances. The draw() method grabs a Graphics from the free list and returns it after finishing drawing. It's the only class in MinWin with custom memory management (so far).
Jul 01 2005
prev sibling next sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Regan Heath wrote:
 I got little or no comment on this idea/code I posted to another 
 (rather  large) thread here, I figured I'd post it again and see what 
 people  thought.
 
 After reading this thread (the one I mention above) and also this one  
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html
 
 It appears you need a type that:
 
 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

You can give a struct or class a method called opAssign if you want. Whether qwert = yuiop; will act as syntactic sugar for it is another matter.
 It seems (from recent posts) that Walter plans to put auto classes on 
 the  stack, so they'd then fulfil requirements 1 and 2.
 
 But what about 3? To me, opAssign isn't generally a good idea for 
 classes  (assigning a value to a reference seems illogical) but would be 
 a good for  structs (they're value types, assignment assigns a value).
 
 Can you do reference counting without opAssign?

Yes. Of the claims by various people that we "need" overloading of the assignment operator, I'm yet to see one that's true. AFAIC if you want something to the effect of custom assignment behaviour, you can create your own method with whatever name that does it. Can anyone supply an example, let alone a real-world one, that doesn't work unless the aforementioned syntactic sugar is available? Stewart. -- My e-mail is valid but not my primary mailbox. Please keep replies on the 'group where everyone may benefit.
Jun 30 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 11:19:04 +0100, Stewart Gordon <smjg_1998 yahoo.com>  
wrote:
 Regan Heath wrote:
 I got little or no comment on this idea/code I posted to another  
 (rather  large) thread here, I figured I'd post it again and see what  
 people  thought.
  After reading this thread (the one I mention above) and also this one   
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html
  It appears you need a type that:
  1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

You can give a struct or class a method called opAssign if you want. Whether qwert = yuiop; will act as syntactic sugar for it is another matter.

True, but not the problem.. read on :)
 It seems (from recent posts) that Walter plans to put auto classes on  
 the  stack, so they'd then fulfil requirements 1 and 2.
  But what about 3? To me, opAssign isn't generally a good idea for  
 classes  (assigning a value to a reference seems illogical) but would  
 be a good for  structs (they're value types, assignment assigns a  
 value).
  Can you do reference counting without opAssign?

Yes.

Then please enlighten me. I've tried, it 'almost' works but it suffers the fatal flaw shown below.
 Of the claims by various people that we "need" overloading of the  
 assignment operator, I'm yet to see one that's true. AFAIC if you want  
 something to the effect of custom assignment behaviour, you can create  
 your own method with whatever name that does it.  Can anyone supply an  
 example, let alone a real-world one, that doesn't work unless the  
 aforementioned syntactic sugar is available?

The problem, as I see it, is that it's not just syntactic sugar. You cannot prevent reference assignment in cases where the idea/technique requires it. As in these 2 cases. (reference counting, auto pointer) I've tried to write a reference counting implementation and also an auto pointer implementation using the recommended practices found on the D website. Both work, both use a "copy constructor" instead of opAssign (as suggested by the D spec/website). The problem is this: auto AutoPtr p = new AutoPtr(object); //create ptr passing object, p owns object. (or "pwns" for those of you..) auto AutoPtr s; s = new AutoPtr(p); //no problem, the copy constructor transfers ownership. s = p; //problem, s now refers to p, both 'own' "object". The same problem occurs with the reference counting implementation. s becomes a 2nd reference to object but the reference count does not increase. We can 'ask' people not to use a straight reference assignment but we cannot enforce it. Logically I agree, opAssign makes no sense for reference types, you assign references to reference types *not* values. opAssign would effectively change a reference assignment into a value assignment in a non obvious way. I agree with all this reasoning, however, the problem remains. Regan
Jun 30 2005
next sibling parent reply "Regan Heath" <regan netwin.co.nz> writes:
Oh, before you get the impression I think we *need* opAssign... I don't.

It all stems from a desire to protect data and/or be totally sure about  
when to COW and when not to COW. Guessing, which is where we are now is  
inefficient and seems like voodoo magic as opposed to computer science.

We might want to return a slice to some private data, eg.

class Foo {
   private char[] data;
   public char[] get() { return data[0..4]; }
}

Foo f = new Foo();
char[] p = f.get();
p[1] = 'd'; //oops they just nuked it.

So, we're left saying:

class Foo {
   private char[] data;
   public char[] get() { return data[0..4].dup; }
}

which works, but in cases where they didn't write to it, or also thought  
"I should dup this before I write to it" (COW) we have a wasted an  
allocation or 2.

The best soln I've read here / thought of is to enforce some sort of  
"readonly" (be it the oft maligned "const" or some other keyword) the  
result would be a *compile* time enforcement of "readonly", eg.

class Foo {
   private char[] data;
   public readonly char[] get() { return data[0..4]; }
}

Foo f = new Foo();
char[] p = f.get();
p[1] = 'd'; //error 'p' is readonly

(note, "readonly" is not required to be applied to the declaration of p.  
This is an important difference between this idea and 'const' from  
C++ "readonly char[]" is _not_ a seperate type to "char[]" it's simply  
achar[] with a readonly flagged to true during compile)

Further, 'in' parameters also require readonly enforcement, eg.

Foo f = new Foo();
mutate(f.get());

void mutate(char[] p) {
   p[0] = 'd'; //error p is in/readonly
}

if a function wants to modify a parameter, that parameter should be 'out'  
or 'inout', meaning, if we re-write mutate:

Foo f = new Foo();
mutate(f.get()); //error, f.get() is readonly

void mutate(inout char[] p) {
   p[0] = 'd'; //no error
}

It's compile time checking, all the compiler need do is flag variables as  
being readonly, and pass that flag on during assignment (as in the case of  
"char[] p = f.get();" above). It's not the "const hell" some see in C++.

With these abilities I believe the requirement for Auto pointers and  
reference counting is lessened (perhaps removed entirely)

Regan

p.s. This has got to be the 10th time I've posted this idea, I'm sorry if  
I sound like a broken record but I've yet to hear any opposition to it,  
 from anyone, Walter included.

On Thu, 30 Jun 2005 22:58:44 +1200, Regan Heath <regan netwin.co.nz> wrote:

 On Thu, 30 Jun 2005 11:19:04 +0100, Stewart Gordon <smjg_1998 yahoo.com>  
 wrote:
 Regan Heath wrote:
 I got little or no comment on this idea/code I posted to another  
 (rather  large) thread here, I figured I'd post it again and see what  
 people  thought.
  After reading this thread (the one I mention above) and also this  
 one  Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html
  It appears you need a type that:
  1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

You can give a struct or class a method called opAssign if you want. Whether qwert = yuiop; will act as syntactic sugar for it is another matter.

True, but not the problem.. read on :)
 It seems (from recent posts) that Walter plans to put auto classes on  
 the  stack, so they'd then fulfil requirements 1 and 2.
  But what about 3? To me, opAssign isn't generally a good idea for  
 classes  (assigning a value to a reference seems illogical) but would  
 be a good for  structs (they're value types, assignment assigns a  
 value).
  Can you do reference counting without opAssign?

Yes.

Then please enlighten me. I've tried, it 'almost' works but it suffers the fatal flaw shown below.
 Of the claims by various people that we "need" overloading of the  
 assignment operator, I'm yet to see one that's true. AFAIC if you want  
 something to the effect of custom assignment behaviour, you can create  
 your own method with whatever name that does it.  Can anyone supply an  
 example, let alone a real-world one, that doesn't work unless the  
 aforementioned syntactic sugar is available?

The problem, as I see it, is that it's not just syntactic sugar. You cannot prevent reference assignment in cases where the idea/technique requires it. As in these 2 cases. (reference counting, auto pointer) I've tried to write a reference counting implementation and also an auto pointer implementation using the recommended practices found on the D website. Both work, both use a "copy constructor" instead of opAssign (as suggested by the D spec/website). The problem is this: auto AutoPtr p = new AutoPtr(object); //create ptr passing object, p owns object. (or "pwns" for those of you..) auto AutoPtr s; s = new AutoPtr(p); //no problem, the copy constructor transfers ownership. s = p; //problem, s now refers to p, both 'own' "object". The same problem occurs with the reference counting implementation. s becomes a 2nd reference to object but the reference count does not increase. We can 'ask' people not to use a straight reference assignment but we cannot enforce it. Logically I agree, opAssign makes no sense for reference types, you assign references to reference types *not* values. opAssign would effectively change a reference assignment into a value assignment in a non obvious way. I agree with all this reasoning, however, the problem remains. Regan

Jun 30 2005
next sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 The best soln I've read here / thought of is to enforce some sort of 
 "readonly" (be it the oft maligned "const" or some other keyword) the 
 result would be a *compile* time enforcement of "readonly", eg.

 class Foo {
   private char[] data;
   public readonly char[] get() { return data[0..4]; }
 }

 Foo f = new Foo();
 char[] p = f.get();
 p[1] = 'd'; //error 'p' is readonly

 (note, "readonly" is not required to be applied to the declaration of p. 
 This is an important difference between this idea and 'const' from  C++ 
 "readonly char[]" is _not_ a seperate type to "char[]" it's simply 
 achar[] with a readonly flagged to true during compile)

Hmm. What about char[] p = f.get(); char[] q = p; q[1] = 'd'; //is 'q' readonly? or char[] p = test ? some_string : f.get(); p[1] = 'd'; //is 'p' readonly if some_string isn't? or void foo(char[] x){ x[1] = 'd'; } char[] p = f.get(); foo(p); I think it would need to be carried around as a different type.
 Further, 'in' parameters also require readonly enforcement, eg.

 Foo f = new Foo();
 mutate(f.get());

 void mutate(char[] p) {
   p[0] = 'd'; //error p is in/readonly
 }

 if a function wants to modify a parameter, that parameter should be 'out' 
 or 'inout', meaning, if we re-write mutate:

 Foo f = new Foo();
 mutate(f.get()); //error, f.get() is readonly

 void mutate(inout char[] p) {
   p[0] = 'd'; //no error
 }

I could see explicit 'in' array parameters are readonly. I don't think that would cause much pain. It wouldn't have any effect on non-array parameters. Making implicit 'in' array parameters readonly would be too much IMHO. It is non-trivial making an 'in' parameter into 'inout' since 'inout' requires an lvalue while 'in' simply requires an rvalue.
 It's compile time checking, all the compiler need do is flag variables as 
 being readonly, and pass that flag on during assignment (as in the case of 
 "char[] p = f.get();" above). It's not the "const hell" some see in C++.

 With these abilities I believe the requirement for Auto pointers and 
 reference counting is lessened (perhaps removed entirely)

 Regan

 p.s. This has got to be the 10th time I've posted this idea, I'm sorry if 
 I sound like a broken record but I've yet to hear any opposition to it, 
 from anyone, Walter included.

Jun 30 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 07:58:33 -0400, Ben Hinkle <ben.hinkle gmail.com>  
wrote:
 Hmm. What about
  char[] p = f.get();
  char[] q = p;
  q[1] = 'd'; //is 'q' readonly?

Yes. (as I said) The compile time readonly flag is passed on during assignment.
 or
  char[] p = test ? some_string : f.get();
  p[1] = 'd'; //is 'p' readonly if some_string isn't?

Ahh, this is the first example (anyones given me) where we require runtime checking also. The other half of my idea is that we have runtime flags also. They'd be in the same class as array bounds checking, i.e. disabled by -release.
 or
  void foo(char[] x){ x[1] = 'd'; }

error 'in' parameters are readonly.
  char[] p = f.get();
  foo(p);

 I think it would need to be carried around as a different type.

Most cases can be handled at compile time without that. The case above that cannot, why does it need to be a seperate type? I dont think it does, it's the same type with a readonly bit set to true (rather than false). This bit can be checked at runtime, and disabled by -release. One goal IMO is to remove the need for us to tell the compiler it's readonly when it can clearly tell all by itself (in many cases) and verify for itself in others (runtime readonly flag). So we don't need to use 'const' or 'readonly' everywhere eg. readonly char[] p = f.get(); char[] s; p = s; //error incompatible types
 Further, 'in' parameters also require readonly enforcement, eg.

 Foo f = new Foo();
 mutate(f.get());

 void mutate(char[] p) {
   p[0] = 'd'; //error p is in/readonly
 }

 if a function wants to modify a parameter, that parameter should be  
 'out'
 or 'inout', meaning, if we re-write mutate:

 Foo f = new Foo();
 mutate(f.get()); //error, f.get() is readonly

 void mutate(inout char[] p) {
   p[0] = 'd'; //no error
 }

I could see explicit 'in' array parameters are readonly. I don't think that would cause much pain. It wouldn't have any effect on non-array parameters. Making implicit 'in' array parameters readonly would be too much IMHO.

It has to be implicit to work. It's not bug tolerant otherwise.
 It is non-trivial making an 'in' parameter into 'inout' since 'inout'  
 requires an lvalue while 'in' simply requires an rvalue.

I can see how this would be painful for value types, but why is it painful for references (can I have an example?). Regan
Jun 30 2005
next sibling parent reply Brad Beveridge <brad somewhere.net> writes:
Regan Heath wrote:
 On Thu, 30 Jun 2005 07:58:33 -0400, Ben Hinkle <ben.hinkle gmail.com>  
 wrote:
 
 Hmm. What about
  char[] p = f.get();
  char[] q = p;
  q[1] = 'd'; //is 'q' readonly?

Yes. (as I said) The compile time readonly flag is passed on during assignment.

line char[] p = f.get(); flags an error if f.get() happens to return an immutable char[]. Ie, you cannot explicitly assign from an immutable char[] to a char[]. That leaves the problem solvable at compile time. The correct line would be immutable char[] p = f.get(); Now, for the sake of simplicity I do not think that the compiler should do run-time checking, nor should it prevent you from casting immutability away. IMHO all we want/need is a way to prevent accidental mistakes & be more descriptive in the way a variable is used. Yes, casting away const is bad - but any language with pointers can eventually do it. Brad
Jun 30 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 08:32:19 -0700, Brad Beveridge <brad somewhere.net>  
wrote:
 Regan Heath wrote:
 On Thu, 30 Jun 2005 07:58:33 -0400, Ben Hinkle <ben.hinkle gmail.com>   
 wrote:

 Hmm. What about
  char[] p = f.get();
  char[] q = p;
  q[1] = 'd'; //is 'q' readonly?

assignment.

line char[] p = f.get(); flags an error if f.get() happens to return an immutable char[]. Ie, you cannot explicitly assign from an immutable char[] to a char[].

Ahh, but that is half my point, they are not distinct types, nor do they need to be.
 That leaves the problem solvable at compile time.  The correct line  
 would be

It is still solvable at compile time. Every char[] has a compile time flag called readonly, which is flagged when 'in' or 'readonly' is used, and passed on every assignment/slice, and can be checked in cases like the one above.
 immutable char[] p = f.get();

I want to avoid this at all costs, otherwise we get: char[] s; ..s is a temp var used up here.. immutable char[] p = f.get(); s = p[0..2]; //error, p is immuable in other words, we cannot reuse the variable and we get all these really annoying errors which then lead us to cast the immutable away (as is done with C++).
 Now, for the sake of simplicity I do not think that the compiler should  
 do run-time checking, nor should it prevent you from casting  
 immutability away.  IMHO all we want/need is a way to prevent accidental  
 mistakes & be more descriptive in the way a variable is used.  Yes,  
 casting away const is bad - but any language with pointers can  
 eventually do it.

If it were implemented how I'm suggesting there would be no need to cast it away. If you did you would actually be blatantly and intentionally doing something wrong. Regan
Jun 30 2005
parent reply Brad Beveridge <brad somewhere.net> writes:
I didn't fully understand what you were describing before - I think I do 
now.  My initial thought is that your concept is superior, but may take 
more compiler effort.  I also don't like the idea of extra runtime 
overhead.

Once again though, we need Walter to put forth _something_ about what he 
is thinking before we hash out the details.
Jun 30 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 14:41:57 -0700, Brad Beveridge <brad somewhere.net>  
wrote:
 I didn't fully understand what you were describing before - I think I do  
 now.

Yay :)
 My initial thought is that your concept is superior, but may take more  
 compiler effort.

Indeed, but that's its job IMO ;) More effort once, less effort x times where x is the number of people writing in D for the forseeable future.
 I also don't like the idea of extra runtime overhead.

Neither, but I think most of it can be done at compile time, the little runtime checking that remains can be disabled with a compile time flag after developement and before release.
 Once again though, we need Walter to put forth _something_ about what he  
 is thinking before we hash out the details.

Yep, he holds the deciding vote (the only vote really). Regan
Jun 30 2005
prev sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 or
  void foo(char[] x){ x[1] = 'd'; }

error 'in' parameters are readonly.
  char[] p = f.get();
  foo(p);


OK. And I assume the readonly flag gets cleared if passed to an inout or out parameter?
 I think it would need to be carried around as a different type.

Most cases can be handled at compile time without that.

Is there a compile-time mechanism to check for this flag? As a code maintainer how can one tell if the flag is set or not?
 The case above that cannot, why does it need to be a seperate type? I dont 
 think it does, it's the same type with a readonly bit set to true (rather 
 than false). This bit can be checked at runtime, and disabled by -release.

I'm not sure how a compile-time bit can be checked at run-time. Where is the bit stored during runtime?
 One goal IMO is to remove the need for us to tell the compiler it's 
 readonly when it can clearly tell all by itself (in many cases) and verify 
 for itself in others (runtime readonly flag).

I like the idea that the compiler (and/or dlint) can warn (or possibly error if the situation is completely unambiguous) when it thinks a readonly value is being modified. I've suggested before that dlint include flow analysis to warn about possible COW violations and other stuff like that. Runtime checks should be avoided, though.
 So we don't need to use 'const' or 'readonly' everywhere eg.

 readonly char[] p = f.get();
 char[] s;
 p = s; //error incompatible types


 Further, 'in' parameters also require readonly enforcement, eg.

 Foo f = new Foo();
 mutate(f.get());

 void mutate(char[] p) {
   p[0] = 'd'; //error p is in/readonly
 }

 if a function wants to modify a parameter, that parameter should be 
 'out'
 or 'inout', meaning, if we re-write mutate:

 Foo f = new Foo();
 mutate(f.get()); //error, f.get() is readonly

 void mutate(inout char[] p) {
   p[0] = 'd'; //no error
 }

I could see explicit 'in' array parameters are readonly. I don't think that would cause much pain. It wouldn't have any effect on non-array parameters. Making implicit 'in' array parameters readonly would be too much IMHO.

It has to be implicit to work. It's not bug tolerant otherwise.

This is slightly OT but the phrase "bug tolerant" to me means the language tolerates bugs - which sounds like "bugs are ok". Of course I know this isn't what you mean by "bug tolerant" (I hope) but it's what I think of every time I read it.
 It is non-trivial making an 'in' parameter into 'inout' since 'inout' 
 requires an lvalue while 'in' simply requires an rvalue.

I can see how this would be painful for value types, but why is it painful for references (can I have an example?).

With void foo(inout char[] x) { ... } char[] bar() { ... } it is not legal to write foo(bar()); Instead one has to write char[] temp = bar(); foo(temp); The output of bar is not an lvalue.
Jun 30 2005
next sibling parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
 One goal IMO is to remove the need for us to tell the compiler it's 
 readonly when it can clearly tell all by itself (in many cases) and 
 verify for itself in others (runtime readonly flag).

I like the idea that the compiler (and/or dlint) can warn (or possibly error if the situation is completely unambiguous) when it thinks a readonly value is being modified. I've suggested before that dlint include flow analysis to warn about possible COW violations and other stuff like that. Runtime checks should be avoided, though.

For example I just hacked up a change to dlint with the following code in AssignExp::semantic() IdentifierExp* idexp = dynamic_cast<IdentifierExp*>(ae->e1); if (idexp != NULL) { Dsymbol* dsym = sc->search(idexp->ident,NULL); if (dsym != NULL) { Declaration* d = dsym->isDeclaration(); if (d != NULL && d->isParameter() && d->isIn()) error("Possible COW violation assigning through input parameter"); } } that will error (I will make that a warning next) if you perform indexed assignment on an input parameter. It's very basic and doesn't catch anything sneaky at all like void foo(char[] s) { char[] y = s; y[1] = 'd'; } but it will catch void foo(char[] s) { s[1] = 'd'; }
Jun 30 2005
parent reply Brad Beveridge <brad somewhere.net> writes:
Ben Hinkle wrote:
One goal IMO is to remove the need for us to tell the compiler it's 
readonly when it can clearly tell all by itself (in many cases) and 
verify for itself in others (runtime readonly flag).

I like the idea that the compiler (and/or dlint) can warn (or possibly error if the situation is completely unambiguous) when it thinks a readonly value is being modified. I've suggested before that dlint include flow analysis to warn about possible COW violations and other stuff like that. Runtime checks should be avoided, though.

For example I just hacked up a change to dlint with the following code in AssignExp::semantic() IdentifierExp* idexp = dynamic_cast<IdentifierExp*>(ae->e1); if (idexp != NULL) { Dsymbol* dsym = sc->search(idexp->ident,NULL); if (dsym != NULL) { Declaration* d = dsym->isDeclaration(); if (d != NULL && d->isParameter() && d->isIn()) error("Possible COW violation assigning through input parameter"); } } that will error (I will make that a warning next) if you perform indexed assignment on an input parameter. It's very basic and doesn't catch anything sneaky at all like void foo(char[] s) { char[] y = s; y[1] = 'd'; } but it will catch void foo(char[] s) { s[1] = 'd'; }

can tell the difference between a correct case of changing the input array, and a bug. Which means that for the times that you actually want to index into an array, you get dlint grumbling to you for no reason. Brad
Jun 30 2005
parent "Ben Hinkle" <bhinkle mathworks.com> writes:
"Brad Beveridge" <brad somewhere.net> wrote in message 
news:da1elm$1lp7$1 digitaldaemon.com...
 Ben Hinkle wrote:
One goal IMO is to remove the need for us to tell the compiler it's 
readonly when it can clearly tell all by itself (in many cases) and 
verify for itself in others (runtime readonly flag).

I like the idea that the compiler (and/or dlint) can warn (or possibly error if the situation is completely unambiguous) when it thinks a readonly value is being modified. I've suggested before that dlint include flow analysis to warn about possible COW violations and other stuff like that. Runtime checks should be avoided, though.

For example I just hacked up a change to dlint with the following code in AssignExp::semantic() IdentifierExp* idexp = dynamic_cast<IdentifierExp*>(ae->e1); if (idexp != NULL) { Dsymbol* dsym = sc->search(idexp->ident,NULL); if (dsym != NULL) { Declaration* d = dsym->isDeclaration(); if (d != NULL && d->isParameter() && d->isIn()) error("Possible COW violation assigning through input parameter"); } } that will error (I will make that a warning next) if you perform indexed assignment on an input parameter. It's very basic and doesn't catch anything sneaky at all like void foo(char[] s) { char[] y = s; y[1] = 'd'; } but it will catch void foo(char[] s) { s[1] = 'd'; }

can tell the difference between a correct case of changing the input array, and a bug. Which means that for the times that you actually want to index into an array, you get dlint grumbling to you for no reason.

Hence it should be a warning. Also note only indexed assignment through an 'in' parameter will warn. Arbitrary indexing is just fine and inout indexed assignment is fine so it won't grumble for most of the cases. Finally, in dlint there is a mechanism to tell dlint to squelch any warning on that line by putting (dlint) in a comment on the line. So something like void foo(char[] s) { s[1] = 'd'; //(dlint) } will shut up dlint if you know the assignment is fine.
Jun 30 2005
prev sibling parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 12:54:18 -0400, Ben Hinkle <ben.hinkle gmail.com>  
wrote:
 or
  void foo(char[] x){ x[1] = 'd'; }

error 'in' parameters are readonly.
  char[] p = f.get();
  foo(p);


OK. And I assume the readonly flag gets cleared if passed to an inout or out parameter?

As 'out' yes (it's initialised as false), as 'inout' no that would be an error (passing readonly as mutable).
 I think it would need to be carried around as a different type.

Most cases can be handled at compile time without that.

Is there a compile-time mechanism to check for this flag?As a code maintainer how can one tell if the flag is set or not?

That might be useful. I'm not sure whether it's required, after all you'll get errors from the compiler if you violate it (that's telling you right there).
 The case above that cannot, why does it need to be a seperate type? I  
 dont
 think it does, it's the same type with a readonly bit set to true  
 (rather
 than false). This bit can be checked at runtime, and disabled by  
 -release.

I'm not sure how a compile-time bit can be checked at run-time. Where is the bit stored during runtime?

If a runtime bit is required, it doesn't have to be the *same* bit as used during compile. That is entirely up to the implementors of the compiler. It does have to be set based on the compile time bit however. The runtime bit would be part of every single object. (yes, I hear you gasp, that's a lot of extra bits, which is why it's a design time feature and disabled by a flag or -release or whatever)
 One goal IMO is to remove the need for us to tell the compiler it's
 readonly when it can clearly tell all by itself (in many cases) and  
 verify
 for itself in others (runtime readonly flag).

I like the idea that the compiler (and/or dlint) can warn (or possibly error if the situation is completely unambiguous) when it thinks a readonly value is being modified. I've suggested before that dlint include flow analysis to warn about possible COW violations and other stuff like that. Runtime checks should be avoided, though.

I agree, but, if it's DBC system and disabled with -release and/or a seperate compiler flag then you can 'avoid' them. I see them as the same sort of thing as array bounds checking (tho they differ because readonly is permanant data for every object rather than a check that can be done when required).
 It has to be implicit to work. It's not bug tolerant otherwise.

This is slightly OT but the phrase "bug tolerant" to me means the language tolerates bugs - which sounds like "bugs are ok". Of course I know this isn't what you mean by "bug tolerant" (I hope) but it's what I think of every time I read it.

Ok, time to change my phrase. (I meant "bug tolerant" as in programming in the language was bug tolerant, not the language itself was bug tolerant) How about: "bug preventative"
 It is non-trivial making an 'in' parameter into 'inout' since 'inout'
 requires an lvalue while 'in' simply requires an rvalue.

I can see how this would be painful for value types, but why is it painful for references (can I have an example?).

With void foo(inout char[] x) { ... } char[] bar() { ... } it is not legal to write foo(bar());

True.
 Instead one has to write
  char[] temp = bar();
  foo(temp);
 The output of bar is not an lvalue.

Ok, I dont really think that is a problem, because: A. If you have a function: void foo(char[] x) { x[0] = 's'; } and someone is passing a rvalue into it, what is the point? I mean, they're not going to see the modification on the outside, in which case I assume you're using the change internally for something. If so, IMO the function should be written: void foo(char[] x) { char[] y = x.dup; } In order to be safe and to hold to it's design contract of only reading it's inputs. B. If you have the function: void foo(char[] x) { x[0] = 's'; } and the intention is to see the changes on the outside then it needs to be rewritten to: void foo(inout char[] x) { x[0] = 's'; } and you can't pass an lvalue to that, nor would you want to. Regan
Jun 30 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Fri, 01 Jul 2005 09:47:50 +1200, Regan Heath <regan netwin.co.nz> wrote:
 On Thu, 30 Jun 2005 12:54:18 -0400, Ben Hinkle <ben.hinkle gmail.com>  
 wrote:
 or
  void foo(char[] x){ x[1] = 'd'; }

error 'in' parameters are readonly.
  char[] p = f.get();
  foo(p);


OK. And I assume the readonly flag gets cleared if passed to an inout or out parameter?

As 'out' yes (it's initialised as false), as 'inout' no that would be an error (passing readonly as mutable).
 I think it would need to be carried around as a different type.

Most cases can be handled at compile time without that.

Is there a compile-time mechanism to check for this flag?As a code maintainer how can one tell if the flag is set or not?

That might be useful. I'm not sure whether it's required, after all you'll get errors from the compiler if you violate it (that's telling you right there).
 The case above that cannot, why does it need to be a seperate type? I  
 dont
 think it does, it's the same type with a readonly bit set to true  
 (rather
 than false). This bit can be checked at runtime, and disabled by  
 -release.

I'm not sure how a compile-time bit can be checked at run-time. Where is the bit stored during runtime?

If a runtime bit is required, it doesn't have to be the *same* bit as used during compile. That is entirely up to the implementors of the compiler. It does have to be set based on the compile time bit however. The runtime bit would be part of every single object. (yes, I hear you gasp, that's a lot of extra bits, which is why it's a design time feature and disabled by a flag or -release or whatever)
 One goal IMO is to remove the need for us to tell the compiler it's
 readonly when it can clearly tell all by itself (in many cases) and  
 verify
 for itself in others (runtime readonly flag).

I like the idea that the compiler (and/or dlint) can warn (or possibly error if the situation is completely unambiguous) when it thinks a readonly value is being modified. I've suggested before that dlint include flow analysis to warn about possible COW violations and other stuff like that. Runtime checks should be avoided, though.

I agree, but, if it's DBC system and disabled with -release and/or a seperate compiler flag then you can 'avoid' them. I see them as the same sort of thing as array bounds checking (tho they differ because readonly is permanant data for every object rather than a check that can be done when required).
 It has to be implicit to work. It's not bug tolerant otherwise.

This is slightly OT but the phrase "bug tolerant" to me means the language tolerates bugs - which sounds like "bugs are ok". Of course I know this isn't what you mean by "bug tolerant" (I hope) but it's what I think of every time I read it.

Ok, time to change my phrase. (I meant "bug tolerant" as in programming in the language was bug tolerant, not the language itself was bug tolerant) How about: "bug preventative"
 It is non-trivial making an 'in' parameter into 'inout' since 'inout'
 requires an lvalue while 'in' simply requires an rvalue.

I can see how this would be painful for value types, but why is it painful for references (can I have an example?).

With void foo(inout char[] x) { ... } char[] bar() { ... } it is not legal to write foo(bar());

True.
 Instead one has to write
  char[] temp = bar();
  foo(temp);
 The output of bar is not an lvalue.

Ok, I dont really think that is a problem, because: A. If you have a function: void foo(char[] x) { x[0] = 's'; } and someone is passing a rvalue into it, what is the point? I mean, they're not going to see the modification on the outside, in which case I assume you're using the change internally for something. If so, IMO the function should be written: void foo(char[] x) { char[] y = x.dup; } In order to be safe and to hold to it's design contract of only reading it's inputs. B. If you have the function: void foo(char[] x) { x[0] = 's'; } and the intention is to see the changes on the outside then it needs to be rewritten to: void foo(inout char[] x) { x[0] = 's'; } and you can't pass an lvalue to that, nor would you want to.

Ooops, meant rvalue here. Regan
Jun 30 2005
prev sibling parent Kramer <Kramer_member pathlink.com> writes:
Regan (and everyone) please keep posting.  Everytime you post an example, it
just further shows what a useful and necessary feature this is.  So please, keep
the posts coming!

-Kramer

P.S. - Let me just say this.  I enjoy reading these examples and posts because
other people who are smarter and better programmers explain these things in much
clearer ways than I ever could.  Having said that, without even knowing if this
feature existed or not in D, I went looking for const/readonly so I could use
it.  It seems like such a logical thing to have to make really robust software.
It helps the programmer better state what they want their code to do; the
compiler should help.  I'm not a compiler writer and don't know how difficult it
is, but it just seems right to have it.  I would feel better just knowing it's
coming even if we didn't get it for several releases.  But there's been no word
on it.  I think that's the frustrating part...

In article <opss6km00b23k2f5 nrage.netwin.co.nz>, Regan Heath says...
Oh, before you get the impression I think we *need* opAssign... I don't.

It all stems from a desire to protect data and/or be totally sure about  
when to COW and when not to COW. Guessing, which is where we are now is  
inefficient and seems like voodoo magic as opposed to computer science.

We might want to return a slice to some private data, eg.

class Foo {
   private char[] data;
   public char[] get() { return data[0..4]; }
}

Foo f = new Foo();
char[] p = f.get();
p[1] = 'd'; //oops they just nuked it.

So, we're left saying:

class Foo {
   private char[] data;
   public char[] get() { return data[0..4].dup; }
}

which works, but in cases where they didn't write to it, or also thought  
"I should dup this before I write to it" (COW) we have a wasted an  
allocation or 2.

The best soln I've read here / thought of is to enforce some sort of  
"readonly" (be it the oft maligned "const" or some other keyword) the  
result would be a *compile* time enforcement of "readonly", eg.

class Foo {
   private char[] data;
   public readonly char[] get() { return data[0..4]; }
}

Foo f = new Foo();
char[] p = f.get();
p[1] = 'd'; //error 'p' is readonly

(note, "readonly" is not required to be applied to the declaration of p.  
This is an important difference between this idea and 'const' from  
C++ "readonly char[]" is _not_ a seperate type to "char[]" it's simply  
achar[] with a readonly flagged to true during compile)

Further, 'in' parameters also require readonly enforcement, eg.

Foo f = new Foo();
mutate(f.get());

void mutate(char[] p) {
   p[0] = 'd'; //error p is in/readonly
}

if a function wants to modify a parameter, that parameter should be 'out'  
or 'inout', meaning, if we re-write mutate:

Foo f = new Foo();
mutate(f.get()); //error, f.get() is readonly

void mutate(inout char[] p) {
   p[0] = 'd'; //no error
}

It's compile time checking, all the compiler need do is flag variables as  
being readonly, and pass that flag on during assignment (as in the case of  
"char[] p = f.get();" above). It's not the "const hell" some see in C++.

With these abilities I believe the requirement for Auto pointers and  
reference counting is lessened (perhaps removed entirely)

Regan

p.s. This has got to be the 10th time I've posted this idea, I'm sorry if  
I sound like a broken record but I've yet to hear any opposition to it,  
 from anyone, Walter included.

On Thu, 30 Jun 2005 22:58:44 +1200, Regan Heath <regan netwin.co.nz> wrote:

 On Thu, 30 Jun 2005 11:19:04 +0100, Stewart Gordon <smjg_1998 yahoo.com>  
 wrote:
 Regan Heath wrote:
 I got little or no comment on this idea/code I posted to another  
 (rather  large) thread here, I figured I'd post it again and see what  
 people  thought.
  After reading this thread (the one I mention above) and also this  
 one  Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html
  It appears you need a type that:
  1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

You can give a struct or class a method called opAssign if you want. Whether qwert = yuiop; will act as syntactic sugar for it is another matter.

True, but not the problem.. read on :)
 It seems (from recent posts) that Walter plans to put auto classes on  
 the  stack, so they'd then fulfil requirements 1 and 2.
  But what about 3? To me, opAssign isn't generally a good idea for  
 classes  (assigning a value to a reference seems illogical) but would  
 be a good for  structs (they're value types, assignment assigns a  
 value).
  Can you do reference counting without opAssign?

Yes.

Then please enlighten me. I've tried, it 'almost' works but it suffers the fatal flaw shown below.
 Of the claims by various people that we "need" overloading of the  
 assignment operator, I'm yet to see one that's true. AFAIC if you want  
 something to the effect of custom assignment behaviour, you can create  
 your own method with whatever name that does it.  Can anyone supply an  
 example, let alone a real-world one, that doesn't work unless the  
 aforementioned syntactic sugar is available?

The problem, as I see it, is that it's not just syntactic sugar. You cannot prevent reference assignment in cases where the idea/technique requires it. As in these 2 cases. (reference counting, auto pointer) I've tried to write a reference counting implementation and also an auto pointer implementation using the recommended practices found on the D website. Both work, both use a "copy constructor" instead of opAssign (as suggested by the D spec/website). The problem is this: auto AutoPtr p = new AutoPtr(object); //create ptr passing object, p owns object. (or "pwns" for those of you..) auto AutoPtr s; s = new AutoPtr(p); //no problem, the copy constructor transfers ownership. s = p; //problem, s now refers to p, both 'own' "object". The same problem occurs with the reference counting implementation. s becomes a 2nd reference to object but the reference count does not increase. We can 'ask' people not to use a straight reference assignment but we cannot enforce it. Logically I agree, opAssign makes no sense for reference types, you assign references to reference types *not* values. opAssign would effectively change a reference assignment into a value assignment in a non obvious way. I agree with all this reasoning, however, the problem remains. Regan


Jun 30 2005
prev sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Regan Heath wrote:
 On Thu, 30 Jun 2005 11:19:04 +0100, Stewart Gordon 
 <smjg_1998 yahoo.com>  wrote:

 Of the claims by various people that we "need" overloading of the 
 assignment operator, I'm yet to see one that's true. AFAIC if you 
 want something to the effect of custom assignment behaviour, you can 
 create your own method with whatever name that does it.  Can anyone 
 supply an example, let alone a real-world one, that doesn't work 
 unless the aforementioned syntactic sugar is available?

The problem, as I see it, is that it's not just syntactic sugar. You cannot prevent reference assignment in cases where the idea/technique requires it. As in these 2 cases. (reference counting, auto pointer)

I guess that if you use either of these, you're supposed to know what you're doing and that you have to be careful. Remember that because of the way D is designed, most applications will be using the built-in GC rather than this. As such, a facility for disabling the assignment operator for select types would probably be seen as an unnecessary complexity.
 I've tried to write a reference counting implementation and also an 
 auto pointer implementation using the recommended practices found on 
 the D website. Both work, both use a "copy constructor" instead of 
 opAssign (as suggested by the D spec/website). The problem is this:
 
 auto AutoPtr p = new AutoPtr(object); //create ptr passing object, p 
 owns  object. (or "pwns" for those of you..)
 auto AutoPtr s;
 
 s = new AutoPtr(p); //no problem, the copy constructor transfers ownership.
 s = p; //problem, s now refers to p, both 'own' "object".

http://www.digitalmars.com/d/attribute.html#auto "Assignment to an auto, other than initialization, is not allowed." I just had a go at it with GDC 0.10. It appears from the destruction order that if an auto variable is initialised to another, then the new reference becomes the owning one. However, there seems to be a bug whereby it accepts assignments after initialisation; however, doing this doesn't seem to have any effect on what gets destructed when. Here's the program I just tried: ---------- import std.stream; import std.stdio; class Thing { int value; this(int v) { value = v; } ~this() { writefln("destructed %d", value); } } void main() { auto Thing o = new Thing(1); auto Thing p = new Thing(2); char[] line = std.stream.stdin.readLine(); auto Thing q = line[0] == 'y' ? o : new Thing(3); if (line[1] == 'y') o = q; writefln(q.value); writefln(o.value); } ----------
 The same problem occurs with the reference counting implementation. s 
 becomes a 2nd reference to object but the reference count does not 
 increase.
 
 We can 'ask' people not to use a straight reference assignment but we  
 cannot enforce it.

True. But I guess what we really need is a clear spec of what happens if one auto is assigned to another. If only by some means (not sure how easy or not it would be to implement) the auto stayed with the original owner, then such reference assignment could be OK (and indeed more efficient) as long as it isn't going to outlive the original. Stewart. -- My e-mail is valid but not my primary mailbox. Please keep replies on the 'group where everyone may benefit.
Jun 30 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 15:20:46 +0100, Stewart Gordon <smjg_1998 yahoo.com>  
wrote:
 Regan Heath wrote:
 On Thu, 30 Jun 2005 11:19:04 +0100, Stewart Gordon  
 <smjg_1998 yahoo.com>  wrote:

 Of the claims by various people that we "need" overloading of the  
 assignment operator, I'm yet to see one that's true. AFAIC if you want  
 something to the effect of custom assignment behaviour, you can create  
 your own method with whatever name that does it.  Can anyone supply an  
 example, let alone a real-world one, that doesn't work unless the  
 aforementioned syntactic sugar is available?

cannot prevent reference assignment in cases where the idea/technique requires it. As in these 2 cases. (reference counting, auto pointer)

I guess that if you use either of these, you're supposed to know what you're doing and that you have to be careful. Remember that because of the way D is designed, most applications will be using the built-in GC rather than this. As such, a facility for disabling the assignment operator for select types would probably be seen as an unnecessary complexity.

Just because you're being clever doesn't mean you can't use all the help you can get. Being able to write an idiot proof reference counting implementation would be an enormous boon for everybody. I don't care how it's done (to a degree), I'm not sure disabling assignment is the solution.
 I've tried to write a reference counting implementation and also an  
 auto pointer implementation using the recommended practices found on  
 the D website. Both work, both use a "copy constructor" instead of  
 opAssign (as suggested by the D spec/website). The problem is this:
  auto AutoPtr p = new AutoPtr(object); //create ptr passing object, p  
 owns  object. (or "pwns" for those of you..)
 auto AutoPtr s;
  s = new AutoPtr(p); //no problem, the copy constructor transfers  
 ownership.
 s = p; //problem, s now refers to p, both 'own' "object".

http://www.digitalmars.com/d/attribute.html#auto "Assignment to an auto, other than initialization, is not allowed." I just had a go at it with GDC 0.10. It appears from the destruction order that if an auto variable is initialised to another, then the new reference becomes the owning one. However, there seems to be a bug whereby it accepts assignments after initialisation; however, doing this doesn't seem to have any effect on what gets destructed when.

Indeed, it's broken then. I wasn't aware of the rule: "Assignment to an auto, other than initialization, is not allowed." in fact, I suggested it after trying to write the implementation in response to Andrew pointing out what I just showed you. Funny that.
 Here's the program I just tried:

 ----------
 import std.stream;
 import std.stdio;

 class Thing {
      int value;
      this(int v) { value = v; }
      ~this() {
          writefln("destructed %d", value);
      }
 }

 void main() {
      auto Thing o = new Thing(1);
      auto Thing p = new Thing(2);
 	
      char[] line = std.stream.stdin.readLine();
      auto Thing q = line[0] == 'y' ? o : new Thing(3);
 		
      if (line[1] == 'y') o = q;
      writefln(q.value);
      writefln(o.value);
 }
 ----------

 The same problem occurs with the reference counting implementation. s  
 becomes a 2nd reference to object but the reference count does not  
 increase.
  We can 'ask' people not to use a straight reference assignment but we   
 cannot enforce it.

True. But I guess what we really need is a clear spec of what happens if one auto is assigned to another.

That is step 1.
 If only by some means (not sure how easy or not it would be to  
 implement) the auto stayed with the original owner, then such reference  
 assignment could be OK (and indeed more efficient) as long as it isn't  
 going to outlive the original.

I think having 2 references to a reference counter without incrementing the counter is still illegal and should be prevented. Same goes for an auto pointer really. Preventing the assignment of an 'auto' and preventing assigning an auto to another reference would solve the problem, from what I can see. Regan
Jun 30 2005
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Regan Heath wrote:
 On Thu, 30 Jun 2005 15:20:46 +0100, Stewart Gordon 
 <smjg_1998 yahoo.com>  wrote:

 I guess that if you use either of these, you're supposed to know what  
 you're doing and that you have to be careful.  Remember that because 
 of the way D is designed, most applications will be using the 
 built-in GC  rather than this.  As such, a facility for disabling the 
 assignment operator for select types would probably be seen as an 
 unnecessary complexity.

Just because you're being clever doesn't mean you can't use all the help you can get.

True. But there's certainly a limit to the extent to which language features to protect you during with uncommon D&D tasks are worth it.
 Being able to write an idiot proof reference 
 counting implementation would be an enormous boon for everybody.

Everybody? Even those of us who are perfectly happy with D's own GC?
 I don't care how it's done (to a degree), I'm not sure disabling 
 assignment is the solution.

That makes a change.
 If only by some means (not sure how easy or not it would be to 
 implement) the auto stayed with the original owner, then such 
 reference assignment could be OK (and indeed more efficient) as long 
 as it isn't going to outlive the original.

I think having 2 references to a reference counter without incrementing the counter is still illegal and should be prevented. Same goes for an auto pointer really.

I refer you back to this: http://www.digitalmars.com/d/garbage.html "Reference counting is a common solution to solve explicit memory allocation problems. The code to implement the increment and decrement operations whenever assignments are made is one source of slowdown. Hiding it behind smart pointer classes doesn't help the speed." So you're saying it should be forbidden to partly get around this issue by circumventing the reference count even when you know it's safe?
 Preventing the assignment of an 'auto' and preventing assigning an auto 
 to another reference would solve the problem, from what I can see.

Up to a point. But autos can be passed into functions, which generally don't know that they're dealing with auto objects and so can arbitrarily create more references to them. But arguably this is a good thing, as such ability is useful for Windows GDI stuff and the like. Stewart. -- My e-mail is valid but not my primary mailbox. Please keep replies on the 'group where everyone may benefit.
Jul 01 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Fri, 01 Jul 2005 11:23:44 +0100, Stewart Gordon <smjg_1998 yahoo.com>  
wrote:
 Being able to write an idiot proof reference counting implementation  
 would be an enormous boon for everybody.

Everybody? Even those of us who are perfectly happy with D's own GC?

Reference counting is not an "alternative" to the GC, it's an "additional" tool. I can imagine using the GC for 80-90% of my memory management and reference counting for those remaining situations where it is beneficial.
 I don't care how it's done (to a degree), I'm not sure disabling  
 assignment is the solution.

That makes a change.

?
 If only by some means (not sure how easy or not it would be to  
 implement) the auto stayed with the original owner, then such  
 reference assignment could be OK (and indeed more efficient) as long  
 as it isn't going to outlive the original.

incrementing the counter is still illegal and should be prevented. Same goes for an auto pointer really.

I refer you back to this: http://www.digitalmars.com/d/garbage.html "Reference counting is a common solution to solve explicit memory allocation problems. The code to implement the increment and decrement operations whenever assignments are made is one source of slowdown. Hiding it behind smart pointer classes doesn't help the speed."

True.
 So you're saying it should be forbidden to partly get around this issue  
 by circumventing the reference count even when you know it's safe?

As long as the new reference does not outlive the original it's probably safe, but that sort of thing is not the general or common usage case. Meaning, ideally, it shouldn't be easy to do by accident. Currently it is: AutoPtr p = new AutoPtr(object); AutoPtr s; s = p; //oops
 Preventing the assignment of an 'auto' and preventing assigning an auto  
 to another reference would solve the problem, from what I can see.

Up to a point. But autos can be passed into functions, which generally don't know that they're dealing with auto objects and so can arbitrarily create more references to them. But arguably this is a good thing, as such ability is useful for Windows GDI stuff and the like.

But again, not a general or comman usage case. So ideally, shouldn't be easy to do accidentally. It sounds like it is currently. Regan
Jul 01 2005
prev sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss0boj0r23k2f5 nrage.netwin.co.nz...
I got little or no comment on this idea/code I posted to another (rather 
large) thread here, I figured I'd post it again and see what people 
thought.

 After reading this thread (the one I mention above) and also this one 
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on the 
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for classes 
 (assigning a value to a reference seems illogical) but would be a good for 
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?

See the section in the Memory Management doc about reference counting. It says the add/release must be explicit. Perhaps Walter could expound on that design decision somewhere to address people's concerns. Personally I have no beef with making non-trivial memory mangement explicit since it should be rarely needed. And in those cases having it explicit will alert code maintainers to what is going on. Also it avoids having to add special "give me a reference without manipulating the count for performance" issue. On the other hand by making it explicit one has to advertise the fact it uses fancy memory management so that users know to increment and decrement the count. Currently D errs on the side of simplicity and making code explicit - which is a fairly common tactic in D in general.
Jun 30 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 07:38:01 -0400, Ben Hinkle <ben.hinkle gmail.com>  
wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0boj0r23k2f5 nrage.netwin.co.nz...
 I got little or no comment on this idea/code I posted to another (rather
 large) thread here, I figured I'd post it again and see what people
 thought.

 After reading this thread (the one I mention above) and also this one
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on  
 the
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for  
 classes
 (assigning a value to a reference seems illogical) but would be a good  
 for
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?

See the section in the Memory Management doc about reference counting. It says the add/release must be explicit.

Indeed. This strikes me as being rather bug intolerant i.e. open for mistakes and bugs to creep in. I am fine with it being this explicit: RefCnt r = new RefCnt(object); //explicit RefCnt s; s = new RefCent(r); //explicit However the problem is actually this: s = r; //needs to be disallowed or to inc refcnt.
 Perhaps Walter could expound on that design decision somewhere to  
 address people's concerns.

Yes please.
 Personally I have no beef with making non-trivial memory mangement  
 explicit since it should be
 rarely needed. And in those cases having it explicit will alert code
 maintainers to what is going on.

I agree.
 Also it avoids having to add special "give me a reference without  
 manipulating the count forperformance" issue.

Ah, but that is the problem.
 On the other hand by making it explicit one has to advertise the fact it  
 uses fancy memory management so that users know to increment and  
 decrement the count.

So, you're saying it should be done: RefCnt r = new RefCnt(object); r.inc(); //perhaps optional?; RefCnt s = r; s.inc(); the room for error here is rather larger than I'd like.
 Currently D errs on the side of simplicity and making code explicit -  
 which is a fairly common tactic in D in general.

I am a big believer in simple code and making odd things explicit. What I want in this case is something simple, that is bug tolerant. I don't mind if it can be abused/broken with intent, it just needs to be accident(bug) tolerant. Regan
Jun 30 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss6mi5iy23k2f5 nrage.netwin.co.nz...
 On Thu, 30 Jun 2005 07:38:01 -0400, Ben Hinkle <ben.hinkle gmail.com> 
 wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0boj0r23k2f5 nrage.netwin.co.nz...
 I got little or no comment on this idea/code I posted to another (rather
 large) thread here, I figured I'd post it again and see what people
 thought.

 After reading this thread (the one I mention above) and also this one
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on 
 the
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for 
 classes
 (assigning a value to a reference seems illogical) but would be a good 
 for
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?

See the section in the Memory Management doc about reference counting. It says the add/release must be explicit.

Indeed. This strikes me as being rather bug intolerant i.e. open for mistakes and bugs to creep in. I am fine with it being this explicit: RefCnt r = new RefCnt(object); //explicit RefCnt s; s = new RefCent(r); //explicit However the problem is actually this: s = r; //needs to be disallowed or to inc refcnt.

People will want to be able to get a reference without changing the ref count and = does that. There are advantages and disadvantages to using =. To me the main advantage is that = does the right thing and matches the user's expectations of what = does and the main disadvantage is that people who don't know they need to care about ref counting will just use = and get into trouble. To me the advantages outweigh the disadvantages.
 Perhaps Walter could expound on that design decision somewhere to 
 address people's concerns.

Yes please.
 Personally I have no beef with making non-trivial memory mangement 
 explicit since it should be
 rarely needed. And in those cases having it explicit will alert code
 maintainers to what is going on.

I agree.
 Also it avoids having to add special "give me a reference without 
 manipulating the count forperformance" issue.

Ah, but that is the problem.

I'm not sure what you mean. I don't think it's unusual to have a ref-counting API that has a way to get a reference without changing the count.
 On the other hand by making it explicit one has to advertise the fact it 
 uses fancy memory management so that users know to increment and 
 decrement the count.

So, you're saying it should be done: RefCnt r = new RefCnt(object); r.inc(); //perhaps optional?; RefCnt s = r; s.inc(); the room for error here is rather larger than I'd like.

Actually I wouldn't even bother having a separate RefCnt class. The ref counting can be mixed into the class of object, much like it is for COM objects. So for example Foo f = new Foo(); // creates with refcount of 1 Foo g = f.incRef; // inc ref count and return this g.decRef; // dec ref count
 Currently D errs on the side of simplicity and making code explicit - 
 which is a fairly common tactic in D in general.

I am a big believer in simple code and making odd things explicit. What I want in this case is something simple, that is bug tolerant. I don't mind if it can be abused/broken with intent, it just needs to be accident(bug) tolerant.

I agree but I think in this case there are a number of factors that make the decision non-trivial. Sometimes it's best to keep everything simple and allow people to shoot themselves in the foot instead of making things complicated to prevent them from shooting themselves in the foot. The weights one gives to all the factors can tip the design in one way or another. The current situation is a reasonable set of compromises IMHO. Adding opAssign would have an impact beyond ref-counting so it's a pretty big step to take.
Jun 30 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 08:37:56 -0400, Ben Hinkle <ben.hinkle gmail.com>  
wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss6mi5iy23k2f5 nrage.netwin.co.nz...
 On Thu, 30 Jun 2005 07:38:01 -0400, Ben Hinkle <ben.hinkle gmail.com>
 wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0boj0r23k2f5 nrage.netwin.co.nz...
 I got little or no comment on this idea/code I posted to another  
 (rather
 large) thread here, I figured I'd post it again and see what people
 thought.

 After reading this thread (the one I mention above) and also this one
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on
 the
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for
 classes
 (assigning a value to a reference seems illogical) but would be a good
 for
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?

See the section in the Memory Management doc about reference counting. It says the add/release must be explicit.

Indeed. This strikes me as being rather bug intolerant i.e. open for mistakes and bugs to creep in. I am fine with it being this explicit: RefCnt r = new RefCnt(object); //explicit RefCnt s; s = new RefCent(r); //explicit However the problem is actually this: s = r; //needs to be disallowed or to inc refcnt.

People will want to be able to get a reference without changing the ref count and = does that. There are advantages and disadvantages to using =. To me the main advantage is that = does the right thing and matches the user's expectations of what = does and the main disadvantage is that people who don't know they need to care about ref counting will just use = and get into trouble. To me the advantages outweigh the disadvantages.

Do you mean get a reference to the "reference counting object", or a reference to the "object being counted"? = gives the former, a reference to the "reference counting object". Getting a reference the the "reference counting object" without incrementing the reference count is not dangerous, provided the new reference does not outlive the original (as the object could be free'd then). I can see how you might want one in order to log something or whatever, but the general case you want to increment the count. that suggests to me that the generally expected behaviour is to increment the count, and the specialist behaviour is not to, in which case the obvious thing = should do the general case.
 Perhaps Walter could expound on that design decision somewhere to
 address people's concerns.

Yes please.
 Personally I have no beef with making non-trivial memory mangement
 explicit since it should be
 rarely needed. And in those cases having it explicit will alert code
 maintainers to what is going on.

I agree.
 Also it avoids having to add special "give me a reference without
 manipulating the count forperformance" issue.

Ah, but that is the problem.

I'm not sure what you mean. I don't think it's unusual to have a ref-counting API that has a way to get a reference without changing the count.

See above.
 On the other hand by making it explicit one has to advertise the fact  
 it
 uses fancy memory management so that users know to increment and
 decrement the count.

So, you're saying it should be done: RefCnt r = new RefCnt(object); r.inc(); //perhaps optional?; RefCnt s = r; s.inc(); the room for error here is rather larger than I'd like.

Actually I wouldn't even bother having a separate RefCnt class. The ref counting can be mixed into the class of object, much like it is for COM objects. So for example Foo f = new Foo(); // creates with refcount of 1 Foo g = f.incRef; // inc ref count and return this g.decRef; // dec ref count

The point of a seperate object is that it does this leg work for you. Meaning, you cannot get it wrong, meaning it's more "bug preventative", robust, and in the end simply easier to use.
 Currently D errs on the side of simplicity and making code explicit -
 which is a fairly common tactic in D in general.

I am a big believer in simple code and making odd things explicit. What I want in this case is something simple, that is bug tolerant. I don't mind if it can be abused/broken with intent, it just needs to be accident(bug) tolerant.

I agree but I think in this case there are a number of factors that make the decision non-trivial. Sometimes it's best to keep everything simple and allow people to shoot themselves in the foot instead of making things complicated to prevent them from shooting themselves in the foot. The weights one gives to all the factors can tip the design in one way or another. The current situation is a reasonable set of compromises IMHO. Adding opAssign would have an impact beyond ref-counting so it's a pretty big step to take.

I'm not asking for opAssign!! I'm asking for a solution to the problem, everyone assumes the only solution is opAssign (which might be because it is, I don't know). I'm saying, here is the problem, find me a solution (because I've exhausted my abilities trying). I'm starting to understand Andrew's frustration. Everyone here seems incapable of exploring the landscape for a solution, they'd all much rather sit in their dirt huts and bandage their bleeding foot. (just ignore me here, this is frustration talking..) Regan
Jun 30 2005