www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - new D2.0 + C++ language

reply Weed <resume755 mail.ru> writes:
Hi!

I want to offer the dialect of the language D2.0, suitable for use where
are now used C/C++. Main goal of this is making language like D, but
corresponding "zero-overhead principle" like C++:

- Its does not contains garbage collection and
- allows active using of a stack for the objects (as in C++)
- Its uses syntax and a tree of objects taken from the D

The code on this language almost as dangerous as a code on C++ - it is a
necessary cost for increasing performance.

Compiler for that language does not exist! And it is unlikely that I
will be able to do it. In any case, I want to discuss before thinking
about compiler.

I just give you an example of using, without a description of pure
syntax because I do not propose anything new for those who remember C++
and D.

Ask the questions!


/*
Demonstration of a new dialect of the D language
It describes only what differs from D2.0

This language is compatible with C and (probably) D ABI, but not with C++.
*/


/*
Structures and classes are completely similar, except structs are having
controlled
alignment and the lack of polymorphism.

The structures are POD and fully compatible with the structures of the
language C.
*/
struct S {
    int var0;
    int var1;
    int var2;

    // Structs constructors are entirely same as in a classes:
    this() {
        var1 = 5;
    }
}

interface I {
    void incr();
}

// The structures is a POD.
// They supports inheritance without a polymorphism and they support
// interfaces too.
struct SD : S, I {
    int var3;
    int var4;

    void incr() { ++var3; ++var4; }

    /*
    Structs constructors are similar to the class constructors.
    Calling base constructor super () is required.
    */
    this() {
        super();
        var4 = 8;
    }
}

class C, I {
    int var;
    void incr() { ++var; }

    /*
    Instead of overloading the operator "=" for classes and structures,
    there is present constructor, same as the copy constructor in C++ -
    in the parameters it accepts only
    object of the same type.
    This is differs from D and the need to ensure that copy constructor
    can change the source object
    (for example, to copy objects linked to the linked-list).

    Unlike the C++ constructor, it first makes a copy of the bitwise
    the original object, then an additional postblit, the same way as
    occurs in D2.0. This allows increase performance of copying than in
    C++.

    And about references:
    When compiling with "-debug" option compiler builds binary with the
    reference-counting.
    This approach is criticized by Walter Bright there:
        http://www.digitalmars.com/d/2.0/faq.html#reference-counting

    But, if the language is not have GC, reference-counting is a good
    way to make sure that
    the object which it references exists. The cost - an additional
    pointer dereferencing and
    checking the counter (and this is only when compiling with option
    "-debug"!).
    */
    this( ref C src ) {
        var3 = src.var3;
        var4 = src.var4;
    }
}

class CD : C {
    real var2;
    void dumb_method() {};
}


void func()
{
    /*
    Classes to be addressed in the heap by pointer. "*" need to
    distinguish the classes in
    heap of classes in the stack. I.e., creating classes and structures
    takes place the same as creating
    them in the C++.
    */
    CD  cd_stack;         // Creates class in a stack
    CD* cd_heap = new CD; // Creates class in a heap, new returns
                          // pointer (same as in C++)
    C*  c_heap  = new C;
    C   c_stack;

    // Copying of a objects (same as in C++)
    cd_stack = *cd_heap;
    *cd_heap = cd_stack;

    /*
    Copying of a pointers to the objects

    "c_heap" pointer points to the object "cd_heap", with the object to
     which the previously pointed "c_heap" is not removed (as there is
    no GC and not used smartpointer template).
    This is memory leak!
    */
    c_heap = cd_heap;

    /*
    "Slicing" demo:

    As a parent object is copied from derived class with additional
    fields and methods. The "real var2" field data is not available in
    "c_stack" and not will be copied:
    */
    c_stack = *cd_heap;

    /*
    Attempt to place an object of type C into the derived object of type
    CD. Field real var2 is not filled by C object. There field now
    contains garbage:
    */
    cd_stack = c_stack;
    cd_stack.var2; // <- garbage data

}
Mar 18 2009
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Weed:
 I want to offer the dialect of the language D2.0, suitable for use where
 are now used C/C++. Main goal of this is making language like D, but
 corresponding "zero-overhead principle" like C++:
...
 The code on this language almost as dangerous as a code on C++ - it is a
 necessary cost for increasing performance.

No, thanks... And regarding performance, eventually it will come a lot from a good usage of multiprocessing, that in real-world programs may need pure functions and immutable data. That D2 has already, while C++ is less lucky. Bye, bearophile
Mar 18 2009
next sibling parent reply Weed <resume755 mail.ru> writes:
bearophile :
 Weed:
 I want to offer the dialect of the language D2.0, suitable for use where
 are now used C/C++. Main goal of this is making language like D, but
 corresponding "zero-overhead principle" like C++:
 ...
 The code on this language almost as dangerous as a code on C++ - it is a
 necessary cost for increasing performance.

No, thanks... And regarding performance, eventually it will come a lot from a good usage of multiprocessing,

The proposal will be able support multiprocessing - for it provided a references counting in the debug version of binaries. If you know the best way for language *without GC* guaranteeing the existence of an object without overhead - I have to listen!
 that in real-world programs may need pure
 functions and immutable data.

I do not see any problems with this
 That D2 has already, while C++ is less lucky.

Mar 18 2009
next sibling parent reply BCS <ao pathlink.com> writes:
Reply to Weed,

 If you know the
 best way for language *without GC* guaranteeing the existence of an
 object without overhead - I have to listen!
 

Never delete anything? One of the arguments for GC is that it might well have /less/ overhead than any other practical way of managing dynamic memory. Yes you can be very careful in keeping track of pointers (not practical) or use smart pointers and such (might end up costing more than GC) but neither is particularly nice.
Mar 18 2009
parent reply Weed <resume755 mail.ru> writes:
BCS :
 Reply to Weed,
 
 If you know the
 best way for language *without GC* guaranteeing the existence of an
 object without overhead - I have to listen!

Never delete anything? One of the arguments for GC is that it might well have /less/ overhead than any other practical way of managing dynamic memory.

Mmm When I say "overhead" I mean the cost of executions, and not cost of programming
 Yes you can be
 very careful in keeping track of pointers (not practical) or use smart
 pointers and such (might end up costing more than GC)

I am do not agree: GC overexpenditure CPU or memory. Typically, both.
 but neither is
 particularly nice.

Mar 19 2009
next sibling parent reply naryl <cy ngs.ru> writes:
Weed Wrote:
 BCS �����:
 Yes you can be
 very careful in keeping track of pointers (not practical) or use smart
 pointers and such (might end up costing more than GC)


I wouldn't be so sure about CPU: http://shootout.alioth.debian.org/debian/benchmark.php?test=all&lang=gdc&lang2=gpp&box=1
Mar 19 2009
parent reply Weed <resume755 mail.ru> writes:
naryl :
 Weed Wrote:
 BCS :
 Yes you can be
 very careful in keeping track of pointers (not practical) or use smart
 pointers and such (might end up costing more than GC)


I wouldn't be so sure about CPU: http://shootout.alioth.debian.org/debian/benchmark.php?test=all&lang=gdc&lang2=gpp&box=1

You should not compare benchmarks - they depend on the quality of the testing code. But it is important to provide an opportunity to write a program efficiently.
Mar 19 2009
parent reply naryl <cy ngs.ru> writes:
Weed Wrote:
 naryl �����:
 Weed Wrote:
 BCS ���������������:
 Yes you can be
 very careful in keeping track of pointers (not practical) or use smart
 pointers and such (might end up costing more than GC)


I wouldn't be so sure about CPU: http://shootout.alioth.debian.org/debian/benchmark.php?test=all&lang=gdc&lang2=gpp&box=1

You should not compare benchmarks - they depend on the quality of the testing code.

Then find a way to prove that GC costs more CPU time than explicit memory management and/or reference counting.
Mar 19 2009
parent reply Weed <resume755 mail.ru> writes:
naryl :
 Weed Wrote:
 naryl :
 Weed Wrote:
 BCS :
 Yes you can be
 very careful in keeping track of pointers (not practical) or use smart
 pointers and such (might end up costing more than GC)


http://shootout.alioth.debian.org/debian/benchmark.php?test=all&lang=gdc&lang2=gpp&box=1

testing code.

Then find a way to prove that GC costs more CPU time than explicit memory management and/or reference counting.

I suggest that reference counting for -debug. Yes, it slows down a bit. As invariant{}, in{}, out(){}, assert()
Mar 19 2009
parent Weed <resume755 mail.ru> writes:
Denis Koroskin :
 On Thu, 19 Mar 2009 19:54:10 +0300, Weed <resume755 mail.ru> wrote:
 
 naryl :
 Weed Wrote:
 naryl :
 Weed Wrote:
 BCS :
 Yes you can be
 very careful in keeping track of pointers (not practical) or use
 smart
 pointers and such (might end up costing more than GC)


http://shootout.alioth.debian.org/debian/benchmark.php?test=all&lang=gdc&lang2=gpp&box=1

testing code.

Then find a way to prove that GC costs more CPU time than explicit memory management and/or reference counting.

I suggest that reference counting for -debug. Yes, it slows down a bit. As invariant{}, in{}, out(){}, assert()

Yeah, ref-count your objects in debug and let the memory leak in release!

Not leak - that may be a reference to non-existent object. The design of "invariant{}" does not reveal all problems with the class in all cases which compiled to the release. So that they abandon invariant{}? Yes, this language is the same danger as the C++.
Mar 20 2009
prev sibling next sibling parent BCS <none anon.com> writes:
Hello Weed,

 BCS ?????:
 
 Reply to Weed,
 
 If you know the
 best way for language *without GC* guaranteeing the existence of an
 object without overhead - I have to listen!

One of the arguments for GC is that it might well have /less/ overhead than any other practical way of managing dynamic memory.

When I say "overhead" I mean the cost of executions, and not cost of programming

So do I. I figure unless it save me more times than it costs /all/ the users, run time cost trumps.
 Yes you can be
 very careful in keeping track of pointers (not practical) or use
 smart
 pointers and such (might end up costing more than GC)


ditto naryl on CPU As for memory, unless the thing overspends into swap and does so very quickly (many pages per second) I don't think that matters. This is because most of the extra will not be part of the resident set so the OS will start paging it out to keep some free pages. This is basically free until you have the CPU or HDD locked hard at 100%. The other half is that the overhead of reference counting and/or the like will cost in memory (you have to store the count somewhere) and might also have bad effects regarding cache misses.
 
 but neither is
 particularly nice.


Mar 19 2009
prev sibling parent reply BCS <none anon.com> writes:
Hello Weed,

 BCS ?????:
 
 Reply to Weed,
 
 If you know the
 best way for language *without GC* guaranteeing the existence of an
 object without overhead - I have to listen!

One of the arguments for GC is that it might well have /less/ overhead than any other practical way of managing dynamic memory.

When I say "overhead" I mean the cost of executions, and not cost of programming

So do I. I figure unless it save me more times than it costs /all/ the users, run time cost trumps.
 Yes you can be
 very careful in keeping track of pointers (not practical) or use
 smart
 pointers and such (might end up costing more than GC)


ditto naryl on CPU As for memory, unless the thing overspends into swap and does so very quickly (many pages per second) I don't think that matters. This is because most of the extra will not be part of the resident set so the OS will start paging it out to keep some free pages. This is basically free until you have the CPU or HDD locked hard at 100%. The other half is that the overhead of reference counting and/or the like will cost in memory (you have to store the count somewhere) and might also have bad effects regarding cache misses.
 
 but neither is
 particularly nice.


Mar 19 2009
parent reply Weed <resume755 mail.ru> writes:
BCS :
 Hello Weed,
 
 BCS ?????:

 Reply to Weed,

 If you know the
 best way for language *without GC* guaranteeing the existence of an
 object without overhead - I have to listen!

One of the arguments for GC is that it might well have /less/ overhead than any other practical way of managing dynamic memory.

When I say "overhead" I mean the cost of executions, and not cost of programming

So do I. I figure unless it save me more times than it costs /all/ the users, run time cost trumps.

This is a philosophical dispute. A good and frequently used code can be written once and then used 10 years in 50 applications in 10000 installations. Here, the costs of programming may be less than the cost of end-user's time and hardware.
 Yes you can be
 very careful in keeping track of pointers (not practical) or use
 smart
 pointers and such (might end up costing more than GC)


ditto naryl on CPU As for memory, unless the thing overspends into swap and does so very quickly (many pages per second) I don't think that matters. This is because most of the extra will not be part of the resident set so the OS will start paging it out to keep some free pages. This is basically free until you have the CPU or HDD locked hard at 100%. The other half is that the overhead of reference counting and/or the like will cost in memory (you have to store the count somewhere) and might also have bad effects regarding cache misses.

Once again I repeat: forget about reference counting - it is only for the debug purposes. I think this addition should be switchable by compiler option. It did not included into the resulting code. Ref-counting needed for multithreaded programs, when there is a risk to get and use a reference to an object that another process has already been killed. This situation needs to be recognized and issued to a run-time error. This is addition to synchronization of threads, which is realized in D.
Mar 20 2009
parent reply BCS <ao pathlink.com> writes:
Reply to Weed,

 BCS ?????:
 
 Hello Weed,
 
 
 Mmm
 When I say "overhead" I mean the cost of executions, and not cost of
 programming

I figure unless it save me more times than it costs /all/ the users, run time cost trumps.

A good and frequently used code can be written once and then used 10 years in 50 applications in 10000 installations. Here, the costs of programming may be less than the cost of end-user's time and hardware.

You are agreeing with me.
 
 As for memory, unless the thing overspends into swap and does so very
 quickly (many pages per second) I don't think that matters. This is
 because most of the extra will not be part of the resident set so the
 OS will start paging it out to keep some free pages. This is
 basically free until you have the CPU or HDD locked hard at 100%. The
 other half is that the overhead of reference counting and/or the like
 will cost in memory (you have to store the count somewhere) and might
 also have bad effects regarding cache misses.
 

Once again I repeat: forget about reference counting - it is only for the debug purposes. I think this addition should be switchable by compiler option. It did not included into the resulting code. Ref-counting needed for multithreaded programs, when there is a risk to get and use a reference to an object that another process has already been killed. This situation needs to be recognized and issued to a run-time error.

As I understand the concept "reference counting" is a form of GC. It has nothing to do with threading. The point is to keep track of how many references, in any thread, there are to a dynamic resource and to free it when there are no more. Normally (as in if you are not doing things wrong) you never release/free/delete a reference counted resource so it doesn't even check if it is delete. Also because the count is attached to the referenced resource, it can't do that check because the count is deleted right along with it. For that concept (the only meaning of the term "reference counting" I known of) the idea of turning it off for non-debug builds is silly. Are you referring to something else?
Mar 20 2009
parent reply Weed <resume755 mail.ru> writes:
BCS :
 I figure unless it save me more times than it costs /all/ the users,
 run time cost trumps.

A good and frequently used code can be written once and then used 10 years in 50 applications in 10000 installations. Here, the costs of programming may be less than the cost of end-user's time and hardware.

You are agreeing with me.
 As for memory, unless the thing overspends into swap and does so very
 quickly (many pages per second) I don't think that matters. This is
 because most of the extra will not be part of the resident set so the
 OS will start paging it out to keep some free pages. This is
 basically free until you have the CPU or HDD locked hard at 100%. The
 other half is that the overhead of reference counting and/or the like
 will cost in memory (you have to store the count somewhere) and might
 also have bad effects regarding cache misses.

Once again I repeat: forget about reference counting - it is only for the debug purposes. I think this addition should be switchable by compiler option. It did not included into the resulting code. Ref-counting needed for multithreaded programs, when there is a risk to get and use a reference to an object that another process has already been killed. This situation needs to be recognized and issued to a run-time error.

As I understand the concept "reference counting" is a form of GC.

In the proposed language is a way to learn mistake - deleting object in another thread that still have a reference. In other words, it is a way to provide proof that the reference refers to an object rather than the emptiness or garbage.
 It has
 nothing to do with threading. The point is to keep track of how many
 references, in any thread, there are to a dynamic resource and to free
 it when there are no more. Normally (as in if you are not doing things
 wrong) you never release/free/delete a reference counted resource so it
 doesn't even check if it is delete. Also because the count is attached
 to the referenced resource, it can't do that check because the count is
 deleted right along with it.
 
 For that concept (the only meaning of the term "reference counting" I
 known of) the idea of turning it off for non-debug builds is silly. Are
 you referring to something else?
 
 

Mar 20 2009
parent BCS <ao pathlink.com> writes:
Reply to Weed,

 BCS пишет:
 
 As I understand the concept "reference counting" is a form of GC.
 

in another thread that still have a reference. In other words, it is a way to provide proof that the reference refers to an object rather than the emptiness or garbage.

OK, than quit calling it reference counting because everyone will think of something else when you call it that. Also, what you are proposing would not be specific to threads. The problem of deleting stuff early is just as much a problem and looks exactly the same in non threaded code.
Mar 20 2009
prev sibling next sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
Weed wrote:
 bearophile :
 Weed:
 I want to offer the dialect of the language D2.0, suitable for use where
 are now used C/C++. Main goal of this is making language like D, but
 corresponding "zero-overhead principle" like C++:
 ...
 The code on this language almost as dangerous as a code on C++ - it is a
 necessary cost for increasing performance.

And regarding performance, eventually it will come a lot from a good usage of multiprocessing,

The proposal will be able support multiprocessing - for it provided a references counting in the debug version of binaries. If you know the best way for language *without GC* guaranteeing the existence of an object without overhead - I have to listen!

You cannot alter the reference count of an immutable variable.
Mar 18 2009
parent reply Weed <resume755 mail.ru> writes:
Christopher Wright :
 Weed wrote:
 bearophile :
 Weed:
 I want to offer the dialect of the language D2.0, suitable for use
 where
 are now used C/C++. Main goal of this is making language like D, but
 corresponding "zero-overhead principle" like C++:
 ...
 The code on this language almost as dangerous as a code on C++ - it
 is a
 necessary cost for increasing performance.

And regarding performance, eventually it will come a lot from a good usage of multiprocessing,

The proposal will be able support multiprocessing - for it provided a references counting in the debug version of binaries. If you know the best way for language *without GC* guaranteeing the existence of an object without overhead - I have to listen!

You cannot alter the reference count of an immutable variable.

Why?
Mar 19 2009
parent reply Christopher Wright <dhasenan gmail.com> writes:
Weed wrote:
 Christopher Wright :
 Weed wrote:
 bearophile :
 Weed:
 I want to offer the dialect of the language D2.0, suitable for use
 where
 are now used C/C++. Main goal of this is making language like D, but
 corresponding "zero-overhead principle" like C++:
 ...
 The code on this language almost as dangerous as a code on C++ - it
 is a
 necessary cost for increasing performance.

And regarding performance, eventually it will come a lot from a good usage of multiprocessing,

references counting in the debug version of binaries. If you know the best way for language *without GC* guaranteeing the existence of an object without overhead - I have to listen!


Why?

Because it's immutable! Unless you're storing a dictionary of objects to reference counts somewhere, that is. Which would be hideously slow and a pain to use. Not that reference counting is fun.
Mar 19 2009
parent reply Weed <resume755 mail.ru> writes:
Christopher Wright :


 And regarding performance, eventually it will come a lot from a good
 usage of multiprocessing,

references counting in the debug version of binaries. If you know the best way for language *without GC* guaranteeing the existence of an object without overhead - I have to listen!


Why?

Because it's immutable! Unless you're storing a dictionary of objects to reference counts somewhere, that is. Which would be hideously slow and a pain to use. Not that reference counting is fun.

Precisely. I wrote the cost for that: 1 dereferencing + inc/dec of counter. Ok, you can make reference counting is disabled by option. In any case, to release code that does not been included, it is only for -debug!
Mar 19 2009
parent reply Christopher Wright <dhasenan gmail.com> writes:
Weed wrote:
 Christopher Wright :
 
 
 And regarding performance, eventually it will come a lot from a good
 usage of multiprocessing,

references counting in the debug version of binaries. If you know the best way for language *without GC* guaranteeing the existence of an object without overhead - I have to listen!



Unless you're storing a dictionary of objects to reference counts somewhere, that is. Which would be hideously slow and a pain to use. Not that reference counting is fun.

Precisely. I wrote the cost for that: 1 dereferencing + inc/dec of counter.

It's more expensive than dereferencing. If your const object points to its reference count, then the reference count is also const, so you can't alter it. So the best you can possibly do is one hashtable lookup for every time you alter the reference count for a non-mutable variable. That is a huge overhead, much more so than garbage collection.
Mar 19 2009
parent reply Weed <resume755 mail.ru> writes:
Christopher Wright :

 And regarding performance, eventually it will come a lot from a good
 usage of multiprocessing,

references counting in the debug version of binaries. If you know the best way for language *without GC* guaranteeing the existence of an object without overhead - I have to listen!



Unless you're storing a dictionary of objects to reference counts somewhere, that is. Which would be hideously slow and a pain to use. Not that reference counting is fun.

Precisely. I wrote the cost for that: 1 dereferencing + inc/dec of counter.

It's more expensive than dereferencing. If your const object points to its reference count, then the reference count is also const, so you can't alter it.

It is designed not so. There will be a hidden dereferencing: const ref Obj object -> struct{ Obj* object; -> Obj object; int counter; }; For all objects will be created such structs. "this" inside the object returns ptr to struct. Any appeal by reference to an object would cause such dereferencing. Creating reference to object in a code block will cause an increase in the counter. Destruction of reference will cause an automatic decrease in the counter. By the way, upon arrival into the try{} it will save values of reference counters for the correct exit if exception will be generated.
 
 So the best you can possibly do is one hashtable lookup for every time
 you alter the reference count for a non-mutable variable. That is a huge
 overhead, much more so than garbage collection.

Mar 20 2009
next sibling parent Christopher Wright <dhasenan gmail.com> writes:
Weed wrote:
 Christopher Wright :
 It's more expensive than dereferencing. If your const object points to
 its reference count, then the reference count is also const, so you
 can't alter it.

It is designed not so. There will be a hidden dereferencing: const ref Obj object -> struct{ Obj* object; -> Obj object; int counter; };

Okay, a language level feature, or a wrapper struct. That would work. If it's a library level feature, there's a problem of usage.
Mar 20 2009
prev sibling parent reply BCS <ao pathlink.com> writes:
Reply to Weed,


 It is designed not so. There will be a hidden dereferencing:
 
 const ref Obj object -> struct{ Obj* object;    -> Obj object;
 int counter; };

Who deletes those structs and when?
Mar 20 2009
parent Weed <resume755 mail.ru> writes:
BCS :
 Reply to Weed,
 
 
 It is designed not so. There will be a hidden dereferencing:

 const ref Obj object -> struct{ Obj* object;    -> Obj object;
 int counter; };

Who deletes those structs and when?

When an object is deleted the struct also is deleted
Mar 20 2009
prev sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 19 Mar 2009 19:54:10 +0300, Weed <resume755 mail.ru> wrote:

 naryl пишет:
 Weed Wrote:
 naryl яПНяПНяПНяПНяПН:
 Weed Wrote:
 BCS яПНяПНяПНяПНяПНяПНяПНяПНяПНяПНяПНяПНяПНяПНяПН:
 Yes you can be
 very careful in keeping track of pointers (not practical) or use  
 smart
 pointers and such (might end up costing more than GC)


http://shootout.alioth.debian.org/debian/benchmark.php?test=all&lang=gdc&lang2=gpp&box=1

testing code.

Then find a way to prove that GC costs more CPU time than explicit memory management and/or reference counting.

I suggest that reference counting for -debug. Yes, it slows down a bit. As invariant{}, in{}, out(){}, assert()

Yeah, ref-count your objects in debug and let the memory leak in release!
Mar 19 2009
prev sibling next sibling parent reply Craig Black <cblack ara.com> writes:
bearophile Wrote:

 Weed:
 I want to offer the dialect of the language D2.0, suitable for use where
 are now used C/C++. Main goal of this is making language like D, but
 corresponding "zero-overhead principle" like C++:
...
 The code on this language almost as dangerous as a code on C++ - it is a
 necessary cost for increasing performance.

No, thanks... And regarding performance, eventually it will come a lot from a good usage of multiprocessing, that in real-world programs may need pure functions and immutable data. That D2 has already, while C++ is less lucky. Bye, bearophile

Multiprocessing can only improve performance for tasks that can run in parallel. So far, every attempt to do this with GC (that I know of) has ended up slower, not faster. Bottom line, if GC is the bottleneck, more CPU's won't help. For applications where GC performance is unacceptable, we either need a radically new way to do GC faster, rely less on the GC, or drop GC altogether. However, in D, we can't get rid of the GC altogether, since the compiler relies on it. But we can use explicit memory management where it makes sense to do so. -Craig
Mar 18 2009
next sibling parent Sergey Gromov <snake.scaly gmail.com> writes:
Wed, 18 Mar 2009 13:48:55 -0400, Craig Black wrote:

 bearophile Wrote:
 
 Weed:
 I want to offer the dialect of the language D2.0, suitable for use where
 are now used C/C++. Main goal of this is making language like D, but
 corresponding "zero-overhead principle" like C++:
...
 The code on this language almost as dangerous as a code on C++ - it is a
 necessary cost for increasing performance.

No, thanks... And regarding performance, eventually it will come a lot from a good usage of multiprocessing, that in real-world programs may need pure functions and immutable data. That D2 has already, while C++ is less lucky. Bye, bearophile

Multiprocessing can only improve performance for tasks that can run in parallel. So far, every attempt to do this with GC (that I know of) has ended up slower, not faster. Bottom line, if GC is the bottleneck, more CPU's won't help. For applications where GC performance is unacceptable, we either need a radically new way to do GC faster, rely less on the GC, or drop GC altogether. However, in D, we can't get rid of the GC altogether, since the compiler relies on it. But we can use explicit memory management where it makes sense to do so. -Craig

I think that the "shared" memory concept in D2 is introduced specifically to improve multi-processing GC performance. There going to be thread-local GC for every thread allocating memory, and, since thread-local will be the default allocation strategy, most memory will be GCed without synchronizing with other threads.
Mar 18 2009
prev sibling next sibling parent reply Weed <resume755 mail.ru> writes:
Robert Jacques :

 Multiprocessing can only improve performance for tasks that can run in
 parallel.  So far, every attempt to do this with GC (that I know of)
 has ended up slower, not faster.  Bottom line, if GC is the
 bottleneck, more CPU's won't help.

 For applications where GC performance is unacceptable, we either need
 a radically new way to do GC faster, rely less on the GC, or drop GC
 altogether.

 However, in D, we can't get rid of the GC altogether, since the
 compiler relies on it.  But we can use explicit memory management
 where it makes sense to do so.

 -Craig

*Sigh*, you do know people run cluster & multi-threaded Java apps all the time right? I'd recommend reading about concurrent GCs http://en.wikipedia.org/wiki/Garbage_collection_(computer_science)#Stop-the-world_vs._incremental_vs._concurrent. By the way, traditional malloc has rather horrible multi-threaded performance as 1) it creates lots of kernel calls

D2.0 with GC also creates lots of kernel calls!
 and 2) requires a
 global lock on access.

Who?
 Yes, there are several alternatives available
 now, but the same techniques work for enabling multi-threaded GCs. D's
 shared/local model should support thread local heaps, which would
 improve all of the above.

It does not prevent pre-create the objects, or to reserve memory for them in advance. (This is what makes the GC, but a programmer would do it better)
Mar 19 2009
parent reply Weed <resume755 mail.ru> writes:
Robert Jacques :

 *sigh* All memory allocation must make some kernel calls. D's GC makes
 fewer calls than a traditional malloc. Actually,  modern malloc
 replacements imitate the way GCs allocate memory since it's a lot
 faster. (Intel's threading building blocks mentions this as part of its
 marketing and performance numbers, so modern mallocs are probably not
 that common)
 
 and 2) requires a
 global lock on access.

Who?

Traditional malloc requires taking global lock (and as point 1, often a kernel lock. Again, fixing this issue is one of Intel's TBB's marketing/performance points)
 Yes, there are several alternatives available
 now, but the same techniques work for enabling multi-threaded GCs. D's
 shared/local model should support thread local heaps, which would
 improve all of the above.

It does not prevent pre-create the objects, or to reserve memory for them in advance. (This is what makes the GC, but a programmer would do it better)

I think the point you're trying to make is that a GC is more memory intensive.

+ Sometimes allocation and freeing of memory in an arbitrary unpredictable time unacceptable. (in game development or realtime software, for example. One hundred million times discussed about it there, I guess)
 Actually, since fast modern mallocs and GC share the same
 underlying allocation techniques, they have about the same memory usage,
 etc. Of course, a traditional malloc with aggressive manual control can
 often return memory to the kernel in a timely manner, so a program's
 memory allocation better tracks actual usage as opposed to the maximum.
 Doing so is very performance intensive and GCs can return memory to the
 system too (Tango's does if I remember correctly).
 

I think so: during the performance of malloc is controlled by the OS. And it is so does the optimization of memory allocation for programs. And OS has more facilities to do this. GC there will be just extra layer. A need language that does not contain a GC (or contains optional). Many C++ programmers do not affect the D only because of this.
Mar 19 2009
next sibling parent BCS <ao pathlink.com> writes:
Reply to Weed,

 I think so: during the performance of malloc is controlled by the OS.
 And it is so does the optimization of memory allocation for programs.
 And OS has more facilities to do this. GC there will be just extra
 layer.

malloc is not a system call. malloc make systems calls (IIRC mmap) once in a while to ask for more memory to be mapped in but I think that D's GC also works that way.
Mar 19 2009
prev sibling next sibling parent BCS <ao pathlink.com> writes:
Reply to Weed,

 + Sometimes allocation and freeing of memory in an arbitrary
 unpredictable time  unacceptable. (in game development or realtime
 software, for example. One hundred million times discussed about it
 there, I guess)

This issue is in no way special to GC systems, IIRC malloc has no upper limit on it's run time. Yes GC has some down sides, yes non GC has some down sides. Take your pick or use a language that lets you do both (like D).
Mar 19 2009
prev sibling next sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
Weed wrote:
 + Sometimes allocation and freeing of memory in an arbitrary
 unpredictable time  unacceptable. (in game development or realtime
 software, for example. One hundred million times discussed about it
 there, I guess)

So you are optimizing for the uncommon case?
Mar 19 2009
parent reply Weed <resume755 mail.ru> writes:
Christopher Wright :
 Weed wrote:
 + Sometimes allocation and freeing of memory in an arbitrary
 unpredictable time  unacceptable. (in game development or realtime
 software, for example. One hundred million times discussed about it
 there, I guess)

So you are optimizing for the uncommon case?

GC is an attempt of optimizing for the uncommon case )
Mar 20 2009
parent reply Christopher Wright <dhasenan gmail.com> writes:
Weed wrote:
 Christopher Wright :
 Weed wrote:
 + Sometimes allocation and freeing of memory in an arbitrary
 unpredictable time  unacceptable. (in game development or realtime
 software, for example. One hundred million times discussed about it
 there, I guess)


GC is an attempt of optimizing for the uncommon case )

I don't think so. Programmers have more important things to do than write memory management systems. My boss would not be happy if I produced an application that leaked memory at a prodigious rate, and he would not be happy if I spent much time at all on memory management. With the application I develop at work, we cache some things. These would have to be reference counted or deleted and recomputed every time. Reference counting is a lot of tedious developer effort. Recomputing is rather expensive. Deleting requires tedious developer effort and determining ownership of everything. This costs time and solves no problems for the customers. And the best manual memory management that I am likely to write would not be faster than a good garbage collector. What sort of applications do you develop? Have you used a garbage collector in a large application?
Mar 20 2009
parent reply Weed <resume755 mail.ru> writes:
Christopher Wright :

 + Sometimes allocation and freeing of memory in an arbitrary
 unpredictable time  unacceptable. (in game development or realtime
 software, for example. One hundred million times discussed about it
 there, I guess)


GC is an attempt of optimizing for the uncommon case )

I don't think so. Programmers have more important things to do than write memory management systems. My boss would not be happy if I produced an application that leaked memory at a prodigious rate, and he would not be happy if I spent much time at all on memory management.

You should use language with GC in this case.
 With the application I develop at work, we cache some things. These
 would have to be reference counted or deleted and recomputed every time.
 Reference counting is a lot of tedious developer effort. Recomputing is
 rather expensive. Deleting requires tedious developer effort and
 determining ownership of everything. This costs time and solves no
 problems for the customers.

I do not agree. I am quite easy to give tracking the creation and deletion of the objects on the stack and on the heap. I do not see problem there. Although there is an alternative - C++, but not D. And you do not need to do reference counting for all the objects in the program. Normally, objects in need not so much as objects do not need. (Therefore, I propose to extend the stack usage for storing and passing objects.) Unfortunately, not yet thought of another way to memory management
 
 And the best manual memory management that I am likely to write would
 not be faster than a good garbage collector.
 
 What sort of applications do you develop?

games, images processing
 Have you used a garbage
 collector in a large application?

I do not write really large applications
Mar 20 2009
next sibling parent BCS <ao pathlink.com> writes:
Reply to Weed,

 Christopher Wright ?????:
 
 + Sometimes allocation and freeing of memory in an arbitrary
 unpredictable time  unacceptable. (in game development or realtime
 software, for example. One hundred million times discussed about
 it there, I guess)
 



write memory management systems. My boss would not be happy if I produced an application that leaked memory at a prodigious rate, and he would not be happy if I spent much time at all on memory management.


that type of cases IS the normals case
 And the best manual memory management that I am likely to write would
 not be faster than a good garbage collector.
 
 What sort of applications do you develop?
 

 Have you used a garbage
 collector in a large application?

I do not write really large applications

Small applications are NOT the normal case. trying to design a language for large apps (one of the things D is targeted at) based on what works in small apps is like say "I know what works for bicycles so now I'll design a railroad train". Yes there are programs where manual memory management is easy, they are generally considered to be few and far between in real life. Many of them really have no need for memory management at all as they die before they would run out of ram anyway.
Mar 20 2009
prev sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
Weed wrote:
 Christopher Wright :
 What sort of applications do you develop?

games, images processing

Libraries will often have no need for data structures with complex lifestyles. There are exceptions, of course, but that's what I have generally found to be the case. For the bulk of image processing, you can just throw the memory management problem to the end user. Games have strict performance requirements that a stop-the-world type of garbage collector violates. Specifically, a full collection would cause an undue delay of hundreds of milliseconds on occasion. If this happens once every ten seconds, your game has performance problems. This is not true of pretty much any other type of application. Games usually have scripting languages that might make use of a garbage collector, though. And there is research into realtime garbage collectors that would be suitable for use in games.
Mar 20 2009
parent reply Rainer Deyke <rainerd eldwood.com> writes:
Christopher Wright wrote:
 Games have strict performance requirements that a stop-the-world type of
 garbage collector violates. Specifically, a full collection would cause
 an undue delay of hundreds of milliseconds on occasion. If this happens
 once every ten seconds, your game has performance problems. This is not
 true of pretty much any other type of application.

If you spend hundreds of milliseconds on garbage collection every ten second, you spend multiple percent of your total execution time on garbage collection. I wouldn't consider that acceptable anywhere. -- Rainer Deyke - rainerd eldwood.com
Mar 20 2009
next sibling parent BCS <none anon.com> writes:
Hello Rainer,

 Christopher Wright wrote:
 
 Games have strict performance requirements that a stop-the-world type
 of garbage collector violates. Specifically, a full collection would
 cause an undue delay of hundreds of milliseconds on occasion. If this
 happens once every ten seconds, your game has performance problems.
 This is not true of pretty much any other type of application.
 

second, you spend multiple percent of your total execution time on garbage collection. I wouldn't consider that acceptable anywhere.

If you spend a few 0.1ths of a ms every 10 ms on reference counting, smart pointers or passing around other meta data it's the just as bad. I have no data but even worse would be, as I expect is true, if non GC apps end up being architected different to make memory management easier (if so, you can bet money it won't be faster as result).
Mar 20 2009
prev sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
Rainer Deyke wrote:
 Christopher Wright wrote:
 Games have strict performance requirements that a stop-the-world type of
 garbage collector violates. Specifically, a full collection would cause
 an undue delay of hundreds of milliseconds on occasion. If this happens
 once every ten seconds, your game has performance problems. This is not
 true of pretty much any other type of application.

If you spend hundreds of milliseconds on garbage collection every ten second, you spend multiple percent of your total execution time on garbage collection. I wouldn't consider that acceptable anywhere.

I was pulling numbers out of my ass. If I wanted to do a proper job, I would have built a large application and modified druntime to get proper timings. 0.1 seconds out of every ten is a small amount to pay for the benefits of garbage collection in most situations. (Most GUI applications are idle most of the time anyway.) I did, however, specifically make the point that it's unacceptable in some situations. These situations may be your situations. Even so, the garbage collector might not be that slow. (And for what it's doing, that seems pretty fast to me.) It would be cool if the GC could watch for what pages have been written to since the last collection and only bother looking through them. That would require some additional accounting. On Windows, there's a system call GetWriteWatch that works in that regard, but on Linux, the only solution I've seen is marking the memory readonly and trapping SIGSEGV. That would be pretty expensive.
Mar 20 2009
next sibling parent BCS <none anon.com> writes:
Hello Christopher,

 
 It would be cool if the GC could watch for what pages have been
 written to since the last collection and only bother looking through
 them. That would require some additional accounting. On Windows,
 there's a system call GetWriteWatch that works in that regard, but on
 Linux, the only solution I've seen is marking the memory readonly and
 trapping SIGSEGV. That would be pretty expensive.
 

http://libsigsegv.sourceforge.net/ """ What is libsigsegv? This is a library for handling page faults in user mode. A page fault occurs when a program tries to access to a region of memory that is currently not available. Catching and handling a page fault is a useful technique for implementing: ... * generational garbage collectors, """
Mar 20 2009
prev sibling parent reply Rainer Deyke <rainerd eldwood.com> writes:
Christopher Wright wrote:
 I was pulling numbers out of my ass.

That's what I assumed. I'm a game developer. I use GC.
 0.1 seconds out of every ten is a small amount to pay for the benefits
 of garbage collection in most situations.

GC is useless for resource management. RAII solves the resource management problem, in C++ and D2. GC is a performance optimization on top of that. If the GC isn't faster than simple reference counting, then it serves no purpose, because you could use RAII with reference counting for the same effect. (No, I don't consider circular references a problem worth discussing.) -- Rainer Deyke - rainerd eldwood.com
Mar 20 2009
next sibling parent Christopher Wright <dhasenan gmail.com> writes:
Rainer Deyke wrote:
 Christopher Wright wrote:
 I was pulling numbers out of my ass.

That's what I assumed. I'm a game developer. I use GC.
 0.1 seconds out of every ten is a small amount to pay for the benefits
 of garbage collection in most situations.

GC is useless for resource management. RAII solves the resource management problem, in C++ and D2. GC is a performance optimization on top of that. If the GC isn't faster than simple reference counting, then it serves no purpose, because you could use RAII with reference counting for the same effect. (No, I don't consider circular references a problem worth discussing.)

I believe Python is using reference counting with a garbage collector, with the collector intended to solve the circular reference problem, so apparently Guido van Rossum thinks it's a problem worth discussing. And my opinion of reference counting is, if it requires no programmer intervention, it's just another garbage collector. Reference counting would probably be a win overall if a reference count going to zero would only optionally trigger a collection -- you're eliminating the 'mark' out of 'mark and sweep'. Though I would still demand a full mark-and-sweep, just not as often. Nontrivial data structures nearly always have circular references.
Mar 21 2009
prev sibling parent reply Sergey Gromov <snake.scaly gmail.com> writes:
Sat, 21 Mar 2009 00:59:22 -0600, Rainer Deyke wrote:

 GC is useless for resource management.

I think this is an overstatement. It's only abstract write buffers where GC really doesn't work, like std.stream.BufferedFile. In any other resource management case I can think of GC works fine.
Mar 21 2009
parent reply Rainer Deyke <rainerd eldwood.com> writes:
Sergey Gromov wrote:
 I think this is an overstatement.  It's only abstract write buffers
 where GC really doesn't work, like std.stream.BufferedFile.  In any
 other resource management case I can think of GC works fine.

OpenGL objects (textures/shader programs/display lists). SDL surfaces. Hardware sound buffers. Mutex locks. File handles. Any object with a non-trivial destructor. Any object that contains or manages one of the above. Many of the above need to be released in a timely manner. For example, it is a serious error to free a SDL surface after closing the SDL video subsystem, and closing the SDL video subsystem is the only way to close the application window under SDL. Non-deterministic garbage collection cannot work. Others don't strictly need to be released immediately after use, but should still be released as soon as reasonably possible to prevent resource hogging. The GC triggers when the program is low on system memory, not when the program is low on texture memory. By my estimate, in my current project (rewritten in C++ after abandoning D due to its poor resource management), about half of the classes manage resources (directly or indirectly) that need to be released in a timely manner. The other 50% does not need RAII, but also wouldn't benefit from GC in any area other than performance. -- Rainer Deyke - rainerd eldwood.com
Mar 21 2009
parent Sergey Gromov <snake.scaly gmail.com> writes:
Sat, 21 Mar 2009 20:16:07 -0600, Rainer Deyke wrote:

 Sergey Gromov wrote:
 I think this is an overstatement.  It's only abstract write buffers
 where GC really doesn't work, like std.stream.BufferedFile.  In any
 other resource management case I can think of GC works fine.

OpenGL objects (textures/shader programs/display lists). SDL surfaces. Hardware sound buffers. Mutex locks. File handles. Any object with a non-trivial destructor. Any object that contains or manages one of the above. Many of the above need to be released in a timely manner. For example, it is a serious error to free a SDL surface after closing the SDL video subsystem, and closing the SDL video subsystem is the only way to close the application window under SDL. Non-deterministic garbage collection cannot work. Others don't strictly need to be released immediately after use, but should still be released as soon as reasonably possible to prevent resource hogging. The GC triggers when the program is low on system memory, not when the program is low on texture memory. By my estimate, in my current project (rewritten in C++ after abandoning D due to its poor resource management), about half of the classes manage resources (directly or indirectly) that need to be released in a timely manner. The other 50% does not need RAII, but also wouldn't benefit from GC in any area other than performance.

Thanks for the explanation, it really helps to keep this picture in mind.
Mar 22 2009
prev sibling parent reply Weed <resume755 mail.ru> writes:
Simen Kjaeraas :
 Weed <resume755 mail.ru> wrote:
 
 I think the point you're trying to make is that a GC is more memory
 intensive.

+ Sometimes allocation and freeing of memory in an arbitrary unpredictable time unacceptable. (in game development or realtime software, for example. One hundred million times discussed about it there, I guess)

Then use the stub GC or disable the GC, then re-enable it when you have the time to run a sweep (yes, you can).

Then a memory overrun
 
 A need language that does not contain a GC (or contains optional). Many
 C++ programmers do not affect the D only because of this.

While GC in D is not optional, it can be stubbed out or disabled,

Then some part of the language will stop working (dynamic arrays, and possibly delegates)
 and malloc/free used in its place. What more is it you ask for?

I need an optional GC and complete freedom to use the stack.
Mar 20 2009
next sibling parent reply Weed <resume755 mail.ru> writes:
Simen Kjaeraas пишет:
 Weed <resume755 mail.ru> wrote:
 
 Simen Kjaeraas пишет:
 Weed <resume755 mail.ru> wrote:

 I think the point you're trying to make is that a GC is more memory
 intensive.

+ Sometimes allocation and freeing of memory in an arbitrary unpredictable time unacceptable. (in game development or realtime software, for example. One hundred million times discussed about it there, I guess)

Then use the stub GC or disable the GC, then re-enable it when you have the time to run a sweep (yes, you can).

Then a memory overrun

If so, you have allocated a lot of things you shouldn't have, or otherwise would have the same problem using manual allocation.

No, as far as I know that some pieces of language does not imply a manual release memory. (See below)
 A need language that does not contain a GC (or contains optional). Many
 C++ programmers do not affect the D only because of this.

While GC in D is not optional, it can be stubbed out or disabled,

Then some part of the language will stop working (dynamic arrays, and possibly delegates)

Yes. So don't use those parts,

It is not impossible without blocking at the compiler level (by CLI option) in the real world.
 or disable the GC and enable it
 when you have the time.

Again, the memory will be overrun)
Mar 20 2009
parent BCS <ao pathlink.com> writes:
Reply to Weed,

 Simen Kjaeraas ?????:
 
 or disable the GC and enable it
 when you have the time.

Again, the memory will be overrun)

Are you saying that you have a program with a time critical section that allocates 100s of MB of memory? If so, you have other problems to fix. If that is not the case, then disable the GC run your critical/RT section allocating a few kB/MB and when you exit that section re enable the GC and clean up. This won't create a memory overrun unless you allocate huge amounts of memory or forget to re enable the GC. Short version: The only places I can think of where having the GC run would cause problems are can't be allowed to run for very long or allocate hardly ram at all. To argue this point you will have to give a specific use case.
Mar 20 2009
prev sibling parent BCS <ao pathlink.com> writes:
Reply to Weed,

 Simen Kjaeraas ?????:
 
 Weed <resume755 mail.ru> wrote:
 
 I think the point you're trying to make is that a GC is more memory
 intensive.
 

unpredictable time unacceptable. (in game development or realtime software, for example. One hundred million times discussed about it there, I guess)

have the time to run a sweep (yes, you can).


I can't think of a case where having the GC running would be a problem where allocating memory at all would not (malloc/new/most any allocator is NOT cheap)
Mar 20 2009
prev sibling parent "Craig Black" <craigblack2 cox.net> writes:
"Robert Jacques" <sandford jhu.edu> wrote in message 
news:op.uq0ng1we26stm6 sandford.myhome.westell.com...
 On Wed, 18 Mar 2009 13:48:55 -0400, Craig Black <cblack ara.com> wrote:

 bearophile Wrote:

 Weed:
 I want to offer the dialect of the language D2.0, suitable for use

 are now used C/C++. Main goal of this is making language like D, but
 corresponding "zero-overhead principle" like C++:
...
 The code on this language almost as dangerous as a code on C++ - it

 necessary cost for increasing performance.

No, thanks... And regarding performance, eventually it will come a lot from a good usage of multiprocessing, that in real-world programs may need pure functions and immutable data. That D2 has already, while C++ is less lucky. Bye, bearophile

Multiprocessing can only improve performance for tasks that can run in parallel. So far, every attempt to do this with GC (that I know of) has ended up slower, not faster. Bottom line, if GC is the bottleneck, more CPU's won't help. For applications where GC performance is unacceptable, we either need a radically new way to do GC faster, rely less on the GC, or drop GC altogether. However, in D, we can't get rid of the GC altogether, since the compiler relies on it. But we can use explicit memory management where it makes sense to do so. -Craig

*Sigh*, you do know people run cluster & multi-threaded Java apps all the time right? I'd recommend reading about concurrent GCs http://en.wikipedia.org/wiki/Garbage_collection_(computer_science)#Stop-the-world_vs._increme tal_vs._concurrent. By the way, traditional malloc has rather horrible multi-threaded performance as 1) it creates lots of kernel calls and 2) requires a global lock on access. Yes, there are several alternatives available now, but the same techniques work for enabling multi-threaded GCs. D's shared/local model should support thread local heaps, which would improve all of the above.

I admit to knowing nothing about clusters, so my point does not apply to them. Also note that I didn't say GC was not useful. I said GC can be a bottleneck. If it is a bottleneck (on a single computer), throwing more CPU's at it doesn't help. Why? The big performance problem with GC is with large applications that allocate a lot of memory. In these apps, modern GC's are constantly causing page faults because they are touching too much memory. I look forward to the day where all the GC problems are solved, and I believe it will come. It would be really nice to have a faster GC in D. However, I don't see how each processor working on a separate heap will solve the problem of the GC causing page faults. But maybe I missed something. BTW, I don't use traditional malloc. I use nedmalloc and the performance is quite good. -Craig
Mar 20 2009
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Wed, 18 Mar 2009 13:48:55 -0400, Craig Black <cblack ara.com> wrote:

 bearophile Wrote:

 Weed:
 I want to offer the dialect of the language D2.0, suitable for use  

 are now used C/C++. Main goal of this is making language like D, but
 corresponding "zero-overhead principle" like C++:
...
 The code on this language almost as dangerous as a code on C++ - it  

 necessary cost for increasing performance.

No, thanks... And regarding performance, eventually it will come a lot from a good usage of multiprocessing, that in real-world programs may need pure functions and immutable data. That D2 has already, while C++ is less lucky. Bye, bearophile

Multiprocessing can only improve performance for tasks that can run in parallel. So far, every attempt to do this with GC (that I know of) has ended up slower, not faster. Bottom line, if GC is the bottleneck, more CPU's won't help. For applications where GC performance is unacceptable, we either need a radically new way to do GC faster, rely less on the GC, or drop GC altogether. However, in D, we can't get rid of the GC altogether, since the compiler relies on it. But we can use explicit memory management where it makes sense to do so. -Craig

*Sigh*, you do know people run cluster & multi-threaded Java apps all the time right? I'd recommend reading about concurrent GCs http://en.wikipedia.org/wiki/Garbage_collection_(computer_science)#Stop-the-world_vs._increme tal_vs._concurrent. By the way, traditional malloc has rather horrible multi-threaded performance as 1) it creates lots of kernel calls and 2) requires a global lock on access. Yes, there are several alternatives available now, but the same techniques work for enabling multi-threaded GCs. D's shared/local model should support thread local heaps, which would improve all of the above.
Mar 18 2009
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Thu, 19 Mar 2009 07:32:18 -0400, Weed <resume755 mail.ru> wrote:

 Robert Jacques пишет:

 Multiprocessing can only improve performance for tasks that can run in
 parallel.  So far, every attempt to do this with GC (that I know of)
 has ended up slower, not faster.  Bottom line, if GC is the
 bottleneck, more CPU's won't help.

 For applications where GC performance is unacceptable, we either need
 a radically new way to do GC faster, rely less on the GC, or drop GC
 altogether.

 However, in D, we can't get rid of the GC altogether, since the
 compiler relies on it.  But we can use explicit memory management
 where it makes sense to do so.

 -Craig

*Sigh*, you do know people run cluster & multi-threaded Java apps all the time right? I'd recommend reading about concurrent GCs http://en.wikipedia.org/wiki/Garbage_collection_(computer_science)#Stop-the-world_vs._incremental_vs._concurrent. By the way, traditional malloc has rather horrible multi-threaded performance as 1) it creates lots of kernel calls

D2.0 with GC also creates lots of kernel calls!

*sigh* All memory allocation must make some kernel calls. D's GC makes fewer calls than a traditional malloc. Actually, modern malloc replacements imitate the way GCs allocate memory since it's a lot faster. (Intel's threading building blocks mentions this as part of its marketing and performance numbers, so modern mallocs are probably not that common)
 and 2) requires a
 global lock on access.

Who?

Traditional malloc requires taking global lock (and as point 1, often a kernel lock. Again, fixing this issue is one of Intel's TBB's marketing/performance points)
 Yes, there are several alternatives available
 now, but the same techniques work for enabling multi-threaded GCs. D's
 shared/local model should support thread local heaps, which would
 improve all of the above.

It does not prevent pre-create the objects, or to reserve memory for them in advance. (This is what makes the GC, but a programmer would do it better)

I think the point you're trying to make is that a GC is more memory intensive. Actually, since fast modern mallocs and GC share the same underlying allocation techniques, they have about the same memory usage, etc. Of course, a traditional malloc with aggressive manual control can often return memory to the kernel in a timely manner, so a program's memory allocation better tracks actual usage as opposed to the maximum. Doing so is very performance intensive and GCs can return memory to the system too (Tango's does if I remember correctly).
Mar 19 2009
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
Weed <resume755 mail.ru> wrote:

 I think the point you're trying to make is that a GC is more memory
 intensive.

+ Sometimes allocation and freeing of memory in an arbitrary unpredictable time unacceptable. (in game development or realtime software, for example. One hundred million times discussed about it there, I guess)

Then use the stub GC or disable the GC, then re-enable it when you have the time to run a sweep (yes, you can).
 A need language that does not contain a GC (or contains optional). Many
 C++ programmers do not affect the D only because of this.

While GC in D is not optional, it can be stubbed out or disabled, and malloc/free used in its place. What more is it you ask for?
Mar 19 2009
prev sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
Weed <resume755 mail.ru> wrote:

 Simen Kjaeraas пишет:
 Weed <resume755 mail.ru> wrote:

 I think the point you're trying to make is that a GC is more memory
 intensive.

+ Sometimes allocation and freeing of memory in an arbitrary unpredictable time unacceptable. (in game development or realtime software, for example. One hundred million times discussed about it there, I guess)

Then use the stub GC or disable the GC, then re-enable it when you have the time to run a sweep (yes, you can).

Then a memory overrun

If so, you have allocated a lot of things you shouldn't have, or otherwise would have the same problem using manual allocation.
 A need language that does not contain a GC (or contains optional). Many
 C++ programmers do not affect the D only because of this.

While GC in D is not optional, it can be stubbed out or disabled,

Then some part of the language will stop working (dynamic arrays, and possibly delegates)

Yes. So don't use those parts, or disable the GC and enable it when you have the time.
Mar 20 2009
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
Weed Wrote:

 - Its does not contains garbage collection and
 - allows active using of a stack for the objects (as in C++)
 - Its uses syntax and a tree of objects taken from the D

Garbage collection can be turned off already, you can get rid of it just by minor modification of druntime, stack allocation is also a minor modification to the compiler, you just need to add syntactical support for it.
Mar 18 2009
parent reply Kagamin <spam here.lot> writes:
Kagamin Wrote:

 Weed Wrote:
 
 - Its does not contains garbage collection and
 - allows active using of a stack for the objects (as in C++)
 - Its uses syntax and a tree of objects taken from the D

you just need to add syntactical support for it.

...well, you already has it with structure constructors...
Mar 18 2009
parent Weed <resume755 mail.ru> writes:
Kagamin :
 Kagamin Wrote:
 
 Weed Wrote:

 - Its does not contains garbage collection and
 - allows active using of a stack for the objects (as in C++)
 - Its uses syntax and a tree of objects taken from the D



Yes
 ...well, you already has it with structure constructors...

Remember that we have already discussed this here several times, and came to the conclusion (?) that emulation of the "value semantic" by structs unreasonably difficult
Mar 18 2009
prev sibling next sibling parent Weed <resume755 mail.ru> writes:
Weed :
 Hi!
 

colorized example: http://paste.dprogramming.com/dpd6j5co
Mar 18 2009
prev sibling parent reply Weed <resume755 mail.ru> writes:
Weed :
 Hi!
 
 I want to offer the dialect of the language D2.0, suitable for use where
 are now used C/C++. Main goal of this is making language like D, but
 corresponding "zero-overhead principle" like C++:

at least to something like this idea? )
Mar 21 2009
parent reply Piotrek <starpit tlen.pl> writes:
Weed pisze:
 Weed :
 Hi!

 I want to offer the dialect of the language D2.0, suitable for use where
 are now used C/C++. Main goal of this is making language like D, but
 corresponding "zero-overhead principle" like C++:

at least to something like this idea? )

The idea could be ok but have you written a compiler or specification? Or is it wishful thinking like let's make the language that's productive to the skies while faster than asm? ;) Cheers
Mar 21 2009
parent reply Weed <resume755 mail.ru> writes:
Piotrek :
 Weed pisze:
 Weed :
 Hi!

 I want to offer the dialect of the language D2.0, suitable for use where
 are now used C/C++. Main goal of this is making language like D, but
 corresponding "zero-overhead principle" like C++:

at least to something like this idea? )

The idea could be ok but have you written a compiler or specification?

My experience in the creation of the compilers is reduced to half-read book "Compilers: Principles, Techniques, and Tools". It was easier to write the differences from D, as fully to specification - I hoped that receive point to some fundamental problems, but there seems all to be good (excluding holy war about GC, of course)
 Or is it wishful thinking like let's make the language that's productive
 to the skies while faster than asm? ;)

No. ) I'm not suggesting anything new, it is suggested that all the time-tested things.
Mar 21 2009
next sibling parent reply Piotrek <starpit tlen.pl> writes:
Weed pisze:
 No. )
 I'm not suggesting anything new, it is suggested that all the
 time-tested things.

OK. I tell you what I think. D is a well designed language. What you suggest is some kind of hack to that language. I don't think there's much interest in it. As you said you don't have much experience in writing compilers (neither do I) but you should know how hard is to keep the design points all the way a language works. Walter spent many years on it. Form my point of view he does the best (of course there are bugs but when I write something in D I'm so glad I don't have to to do it in something else). Cheers
Mar 21 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Piotrek:
 (of course there are bugs but  when I write something in D
 I'm so glad I don't have to to do it in something else).

D is surely not a puzzle language :-) http://prog21.dadgum.com/38.html Well, writing D templates in functional style and lot of string mixins is a puzzle (even if they are less puzzles than some things you have to do in Forth). AST macros too can become puzzles, but I think if well designed they can be more natural to use than the current templates & string mixins. Bye, bearophile
Mar 21 2009
parent reply Piotrek <starpit tlen.pl> writes:
bearophile pisze:
 Piotrek:
 (of course there are bugs but  when I write something in D
 I'm so glad I don't have to to do it in something else).

D is surely not a puzzle language :-) http://prog21.dadgum.com/38.html

Haha, I found good discussion on reddit http://www.reddit.com/r/programming/comments/7vnm0/puzzle_languages/ Just couple of nice citations: J language expample: meanf=: 1 : '(0>.255<.]) :<. (m (+/ % #) ,;.3 ])"2' boxf=: 1 : '(0>.255<.]) :<. ((2 2$m) (+/ % #) ,;.3 ])"2' pc=: &.:([: 2 0 1&|: 256 256 256&#:) inter=: ( [: , 2&(({. , (+/%#))\) )"1 inter2=: ([: (<. inter pc) |:)^:2 fit=: [: >.`<./ , fit1=: [: >.`<./ (,~ (,~ -))~ histogram=: <: (#/.~) (i. # [ , I.) Does it look like the Matrix? And the the one made me laugh the most: "CSS is the puzzliest puzzle that I ever puzzled" Every one should try CSS :D Cheers
Mar 21 2009
parent bearophile <bearophileHUGS lycos.com> writes:
Piotrek:

 Haha, I found good discussion on reddit<

On the other hand, D too for me becomes a puzzle language when I use many string mixins or templates in functional-style. D macros will possibly improve that situation some.
 J language expample:<

K language seems worse to me, this is a full raytracer that saves in pgm (in C++ it's about 120 lines of code written in normal style): http://www.nsl.com/k/ray/ray.k U:{x%_sqrt x _dot x} S:{[r;s]:[0>d:_sqr[s 1]+_sqr[b:v _dot r 1]-v _dot v:s[0]-*r;0i;0>t:b+e:_sqrt d;0i;0<u:b-e;u;t]} I:{[r;h;o]:[~4:*o;:[~S[r;*o]<*h;h;h _f[r]/o 1];~h[0]>l:S[r]o;h;(l;U r[0]-o[0]-l*r 1)]} T:{[r;o;d;z;l]:[0i=*h:I[r;z]o;0.;~0>g:h[1]_dot l;0.;0i=*I[(r[0]+(r[1]**h)+d*h 1;-l);z]o;-g;0.]} N:{[n;o;i]0{x+T[(0 0 -4.;U(i+(y%4)-n%2),n);o;_sqrt 2^-42;0i,,3#0.]U -1 -3 2.}/+4_vs!16} R:{[k;n]"P5\n",(5:n,n),"\n255\n",_ci _.5+15.9375*N[n*1.;C[k;0 -1 0.]1.]'+| [n _vs!n*n;0;|:]} C:{[k;c;r]:[k=1;(c;r);((c;r*3);(,(c;r)),C[k-1;;r%2]'+c+-3 3[2_vs 2 3 6 7]*r%_sqrt 12)]} \t q:R[3]32 "temp.pgm"6:q \"C:\\Program Files\\IrfanView\\i_view32.exe" temp.pgm APL-like languages are a dead-end... Bye, bearophile
Mar 21 2009
prev sibling parent Christopher Wright <dhasenan gmail.com> writes:
Weed wrote:
 Piotrek :
 Weed pisze:
 Weed :
 Hi!

 I want to offer the dialect of the language D2.0, suitable for use where
 are now used C/C++. Main goal of this is making language like D, but
 corresponding "zero-overhead principle" like C++:



My experience in the creation of the compilers is reduced to half-read book "Compilers: Principles, Techniques, and Tools". It was easier to write the differences from D, as fully to specification - I hoped that receive point to some fundamental problems, but there seems all to be good (excluding holy war about GC, of course)

I inferred from your original post that you had written such a compiler. I think probably the first thing to do, if you are serious about this, is to choose one essential feature of your dialect and implement that. Then there will be something concrete to discuss. That isn't the normal modus operandi around here, but there doesn't seem to be much support for your suggestions. I think this might be less to do with your ideas and more that I can't really envision what you're talking about and how code would look with your dialect.
Mar 21 2009