www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP18: Non-GC threads

reply Piotr Szturmaj <bncrbme jadamspam.pl> writes:
http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP18

The proposal is to create a base for safe, non garbage-collected 
threads. This is done by adding nogc attribute for functions. nogc 
functions cannot perform operations that may allocate garbage collected 
memory. They are covariant with gc ones.

Any comments are welcome.
Aug 31 2012
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Piotr Szturmaj:

 This is done by adding nogc attribute for functions. nogc 
 functions cannot perform operations that may allocate garbage 
 collected memory.

I suggest to call that nogc. Time ago I have suggested a " noheap" attribute for functions, that is similar to that "nogc", see the discussion there: http://d.puremagic.com/issues/show_bug.cgi?id=5219 The difference is that noheap disallows C functions like C malloc(), etc. Stack allocation like alloca() and variable length arrays is allowed in noheap functions. noheap/ nogc is probably inferred for templated functions, as pure/nothrow. The main disadvantage of an annotation like this is the proliferation of kinds of functions. The advantage is a more precise control of the effects of functions. The Koka language has a function annotation similar to noheap, I have discussed a little about Koka here: http://forum.dlang.org/thread/ocxmiolhlcysehbjtcop forum.dlang.org but Koka has type inference for effects, so often you don't need to add that annotation to functions. Bye, bearophile
Aug 31 2012
parent reply Piotr Szturmaj <bncrbme jadamspam.pl> writes:
bearophile wrote:
 Piotr Szturmaj:

 This is done by adding nogc attribute for functions. nogc functions
 cannot perform operations that may allocate garbage collected memory.

I suggest to call that nogc. Time ago I have suggested a " noheap" attribute for functions, that is similar to that "nogc", see the discussion there: http://d.puremagic.com/issues/show_bug.cgi?id=5219

Thanks, I didn't know that. There are some good arguments for nogc.
 The difference is that  noheap disallows C functions like C malloc(),
 etc. Stack allocation like alloca() and variable length arrays is
 allowed in  noheap functions.

  noheap/ nogc is probably inferred for templated functions, as
 pure/nothrow.

I would prefer nogc, because nogc functions can manually allocate. Not using malloc, or preallocating using it is a lot easier than checking for GC allocations and manually managing non-gc threads. The main reason under nogc is not non-deterministic allocation of GC or malloc. The whole idea is to guarantee that nogc threads are not suspended by the GC.
 The main disadvantage of an annotation like this is the proliferation of
 kinds of functions. The advantage is a more precise control of the
 effects of functions.

The advantage is that you'd get the static checks similar to nothrow and pure.
 The Koka language has a function annotation similar to  noheap, I have
 discussed a little about Koka here:
 http://forum.dlang.org/thread/ocxmiolhlcysehbjtcop forum.dlang.org
 but Koka has type inference for effects, so often you don't need to add
 that annotation to functions.

Well, __traits(hasGCallocations, func) will also do the trick.
Sep 01 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 01-Sep-12 15:47, Piotr Szturmaj wrote:
 bearophile wrote:

 The Koka language has a function annotation similar to  noheap, I have
 discussed a little about Koka here:
 http://forum.dlang.org/thread/ocxmiolhlcysehbjtcop forum.dlang.org
 but Koka has type inference for effects, so often you don't need to add
 that annotation to functions.

Well, __traits(hasGCallocations, func) will also do the trick.

Make it __traits(hasAttribute, "nogc", func) and that might be something. Actually, I'd love to see some proposal that a) clarifies how attributes are supposed to be used b) adds generic infrastructure to to use them via __traits or whatever -- Olshansky Dmitry
Sep 01 2012
parent deadalnix <deadalnix gmail.com> writes:
Le 01/09/2012 14:29, Dmitry Olshansky a écrit :
 On 01-Sep-12 15:47, Piotr Szturmaj wrote:
 bearophile wrote:

 The Koka language has a function annotation similar to  noheap, I have
 discussed a little about Koka here:
 http://forum.dlang.org/thread/ocxmiolhlcysehbjtcop forum.dlang.org
 but Koka has type inference for effects, so often you don't need to add
 that annotation to functions.

Well, __traits(hasGCallocations, func) will also do the trick.

Make it __traits(hasAttribute, "nogc", func) and that might be something. Actually, I'd love to see some proposal that a) clarifies how attributes are supposed to be used b) adds generic infrastructure to to use them via __traits or whatever

+1
Sep 01 2012
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
One more comment, a quotation from the DIP18:

Currently, all functions can use garbage collected memory. 
Threads that call those functions must be managed by the garbage 
collector, which may occasionally suspend them to perform 
collection. This can lead to unwanted pauses, that are not 
acceptable in some situations, mainly real-time audio/video 
processing, gaming, and others.<

C heap functions like malloc(), calloc(), realloc() have the same problem, their run-time is not deterministic, so if you don't want pauses a noheap is useful. noheap is not meant to forbid calling low-level system-specific memory allocation functions. Bye, bearophile
Aug 31 2012
prev sibling next sibling parent reply Michel Fortin <michel.fortin michelf.ca> writes:
 If some of the function's arguments take references (directly or 
 indirectly), then all of them are automatically added to the list of GC 
 roots (addRoot()).

You still have a problem: you might have added an object passed as an argument to the GC roots, preventing the GC from collecting it, but if any thread is allowed to mutate a pointer inside this object pointing to some other data, using this other data in your thread is not safe, unless you have added a root for this other pointer too. You'd have to recursively add roots for this to work. There's also the same issue with global variables, the ones which are pointers to other pointers. -- Michel Fortin michel.fortin michelf.ca http://michelf.ca/
Aug 31 2012
parent reply Piotr Szturmaj <bncrbme jadamspam.pl> writes:
Michel Fortin wrote:
 If some of the function's arguments take references (directly or
 indirectly), then all of them are automatically added to the list of
 GC roots (addRoot()).

You still have a problem: you might have added an object passed as an argument to the GC roots, preventing the GC from collecting it, but if any thread is allowed to mutate a pointer inside this object pointing to some other data, using this other data in your thread is not safe, unless you have added a root for this other pointer too. You'd have to recursively add roots for this to work. There's also the same issue with global variables, the ones which are pointers to other pointers.

Yes, there is no obvious solution, but I think it could be done. For example non-immutable references may be disallowed at all. The other solution is to copy the whole referenced object using malloc, and add it to the GC roots.
Sep 01 2012
parent Michel Fortin <michel.fortin michelf.ca> writes:
On 2012-09-01 11:54:10 +0000, Piotr Szturmaj <bncrbme jadamspam.pl> said:

 Michel Fortin wrote:
 If some of the function's arguments take references (directly or
 indirectly), then all of them are automatically added to the list of
 GC roots (addRoot()).

You still have a problem: you might have added an object passed as an argument to the GC roots, preventing the GC from collecting it, but if any thread is allowed to mutate a pointer inside this object pointing to some other data, using this other data in your thread is not safe, unless you have added a root for this other pointer too. You'd have to recursively add roots for this to work. There's also the same issue with global variables, the ones which are pointers to other pointers.

Yes, there is no obvious solution, but I think it could be done. For example non-immutable references may be disallowed at all.

That would be sufficient to enforce the guaranties, but it's a severe limitation that'll cause people to question the usefulness of the whole thing.
 The other solution is to copy the whole referenced object using malloc, 
 and add it to the GC roots.

Having to copy everything is also severe limitation I think. And even then it's not that simple to correctly copy a data structure if you want that to be done automatically. If you want it to be done manually, then how can you tell that the resulting pointer, and everything it points to, is not allocated from the GC heap? -- Michel Fortin michel.fortin michelf.ca http://michelf.ca/
Sep 01 2012
prev sibling next sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
I think this is an unnecessary addition.

I use D for game dev, and I avoid the GC + malloc like the 
plague, however I don't want to have to litter my code with 
distractions like nogc,  noheap or whatever keyword we choose.

It's easy enough to avoid these without expanding the type 
system. I add trace statements to the GC in druntime, and just 
avoid calling malloc. I think this is a more practical solution 
to this problem than extending the language and cluttering my 
code.
Sep 01 2012
parent reply Piotr Szturmaj <bncrbme jadamspam.pl> writes:
Peter Alexander wrote:
 I think this is an unnecessary addition.

 I use D for game dev, and I avoid the GC + malloc like the plague,
 however I don't want to have to litter my code with distractions like
 nogc,  noheap or whatever keyword we choose.

 It's easy enough to avoid these without expanding the type system. I add
 trace statements to the GC in druntime, and just avoid calling malloc. I
 think this is a more practical solution to this problem than extending
 the language and cluttering my code.

It's similar behavior to nothrow and pure. Instead of manually avoiding GC allocations, compiler does this checks for you. Imagine D doesn't have nothrow. You'd have to check every called function to see if it doesn't throw. In big programs throwing function may be left unnoticed and this is why we have static nothrow checks in D.
Sep 01 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 01-Sep-12 16:27, Peter Alexander wrote:
 On Saturday, 1 September 2012 at 11:37:39 UTC, Piotr Szturmaj wrote:
 It's similar behavior to nothrow and pure. Instead of manually
 avoiding GC allocations, compiler does this checks for you. Imagine D
 doesn't have nothrow. You'd have to check every called function to see
 if it doesn't throw. In big programs throwing function may be left
 unnoticed and this is why we have static nothrow checks in D.

I understand the benefit. However, there are at least two significant costs: 1. If I want my entire program to be GC free, I have to annotate every single function with 'nogc'. This is not something I want to do.

I'd say nogc: at the top and deal is sealed.
 2. It's a new language feature and has all the associated costs: initial
 implementation, bug fixing, marking up of functions in Phobos,
 documentation, etc.

 Yes, with my approach, a rare allocation may go unnoticed, and you end
 up with an undesirable GC collection sometime in the future. It's not
 great, but it's not the end of the world, and I'm willing to risk that
 to avoid the costs I mentioned above.

-- Olshansky Dmitry
Sep 01 2012
next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 01-Sep-12 19:58, Peter Alexander wrote:
 C:

 int mul(int x, int y);


 Future D:

 pure  safe nothrow nogc commutative associative distributive
  forceinline mayoverflow int mul(int x, int y);

No it's GNU C++ of today just replace 'pure' with __attribute__((pure)) and you are done :) And don't even get me started at MS extensions... Either way this is a kind of thing that only some users need but std library have to use all of them just so that those 'some users' can use it. In fact if pure safe nothrow could be combined in a user defined tag: alias pure safe nothrow nogc nice_func; nice_func int mul(int x, int y); and even C has it via macros, damn... -- Olshansky Dmitry
Sep 01 2012
parent Piotr Szturmaj <bncrbme jadamspam.pl> writes:
Dmitry Olshansky wrote:
 On 01-Sep-12 19:58, Peter Alexander wrote:
 C:

 int mul(int x, int y);


 Future D:

 pure  safe nothrow nogc commutative associative distributive
  forceinline mayoverflow int mul(int x, int y);

Either way this is a kind of thing that only some users need but std library have to use all of them just so that those 'some users' can use it.

The DIP states that nogc should be inferred by the compiler to avoid marking all existing functions. But, I know there are some problems with inferring non templated methods: http://www.digitalmars.com/d/archives/digitalmars/D/Why_isn_t_purity_co._inferred_for_all_functions_166946.html.
Sep 03 2012
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 01-Sep-12 17:01, Adam D. Ruppe wrote:
 On Saturday, 1 September 2012 at 12:39:41 UTC, Dmitry Olshansky wrote:
 I'd say
  nogc:
 at the top and deal is sealed.

BTW don't forget a yesgc would be good to counter it. We need a way to turn off each attribute to use this pattern best.

What I see with this nogc proposal is that that problem it tries to solve (and tool used to do so) is far more interesting and general. Namely the problem is to specify that some functions can call only specific subset of functions and able to use only specific subset of language features. It's more far reaching then just gc, one may want to get noblocking or async attribute to statically check e.g. that GUI thread can't ever block. (it would however require to identify API calls that don't block, not possible on every OS, might entail some wrappers etc. but is very desirable) To peek at what's more there that you can get this way see C++ AMP on why restricting a block of code or function is useful e.g. here by Herb Sutter (including nice demo!): http://www.youtube.com/watch?v=HK9rJFpX32Y BTW safe is in the same bucket, it does allow only to call safe & trusted (read as "subset of ") functions and use a subset of language features. Actually exactly the same goes for pure (noglobal whatever) you just state this piece of code can't use this language feature (global variables, static variables, you name it), same for nothrow. And not long ago David Nadlinger showed that function level trusted doesn't help much with templates that are trusted for their own code, but need normal safety guarantee from foreign code brought by parameters. Thus restriction has to be able to work on a block scope too. So what I want with this? I want to bring together C++ AMP restrict and safe/ trusted/ system/pure/nothrow machinery in one clean and user-extensible future-proof feature. The idea in a nutshell: - specify a way attach tags to function declarations (to "paint" functions) - allow specifying is-a relation between tags (supertags and subtags). - add restrict specification that allows user to enforce on a block of code the following: a) disallow use of certain language features b) allow(or disallow) calling function with specific tags - fit with existing world and provide clean migration path, specify defaults I claim that it's enough to implement pure/safe/trusted/nothrow/nogc and proposed nogc. The following is the draft of proposal. 1. Introduce a restrict keyword. Restrict applies to a block of code or a function: auto foo(...) restrict(<restriction spec>) { } auto bar(...) { restrict(<restriction spec>){ } } And the usual restrict(<restrict spec>): to affect all subsequent declarations in the module. Multiple restrict blocks on the same declaration are treated as single restrict with restrict specifications concatenated. The effect of such a clause is that declarations and statements inside of a block have to pass restriction check in order to compile. The kind of check is described in restrict specification. 2. Functions declared with restrict(<restrict spec>) can only be bound by function pointer of more permissive type w.r.t. spec: (assume A,B,C,D,E are elements of spec) void funcABCD() restrict(A,B,C,D); void funcACD() restrict(A,C,D); function void() restrict(A,B,D) fp; fp = &funcABCD; ///okay, fp is more permissive fp = &funcACD; //fail, B is allowed inside of funcACD but not by fp 3. Function tags. Tag is a special literal associated with function declaration and type of function pointer. Any FP or function declarations can have an arbitrary amount of tags attached: tag(mytag, safe, nothrow) void foobar(); function void () tag(default, nothrow) Every function without tags has one implicit 'default' tag (name clashes with 'default' keyword may require another name). To only add tags include 'default' among others and to override it just don't list default. User defined tags are specified as follows: tag tag_name; //new tag called tag_name tag tag_name : supertag; //tag_name is a subtag of super tag The following tags are declared in object.d: //pure is subtag of default, address of pure function can be assigned to default FP tag pure : default; tag system : default; //and system is subtag of default tag nothrow: default; tag trusted: system; tag safe: trusted; Any assignments of FP allows only compatible pointer types. Namely for the left hand side of assignment exp (LHS) and the right hand side (RHS): all of RHS tags must have super tag or direct match in LHS's set of tags. 4. Restrict specifications Introduce a list of language features that can be enabled/disabled with descriptive names, the list may be extended in the future. Common suspects are: - heap stack frames and implicit GC actions - ability to declare/access kinds of variables (immutable, shared, static ...) - pointers to functions - pointers to ... - virtual functions - labels & goto - RTTI - inline assembler - pointer arithmetic - unions & reinterpret casts - C and D style varargs - other casts (integer/float coercing, const, dynamic) - exceptions ... each of those should have an IDs akin to __traits like 'virtual_calls', 'gc' etc. Also all of the built-in names are reserved in a tags namespace. Restrict specification is a comma separated list of: 1) tag - allow to call only functions that have tag 'tag' or have supertag of 'tag' 2) language_feature_id - disable feature with name language_feature_id 5. Backward compatibility. safe, trusted, system, nothrow and pure work as aliases respectively: safe - tag(safe) restrict(safe) trusted - tag(trusted) restrict(trusted) system - tag(system) restrict(system) pure - tag(pure) restrict(globals, static, pure) nothrow - tag(nothrow) restrict(nothrow) Sometime later these could be deprecated or left as simple shortcuts. 6. Probably we need a way to alias a list of restictions to ease the use of frequent combinations. (No idea, but any sane syntax around alias will do) -- Olshansky Dmitry
Sep 01 2012
next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 02-Sep-12 01:13, foobar wrote:
[snip]
 I skipped the details but I do agree with the general sentiment.
 This long post begs the question - Why a general annotation
 mechanism isn't enough to justify adding so much complexity to
 the language?

It's quite short compared to all of the features it replaces and it had to allow for backwards compatibility. And it had to leave place for other annotations as I felt restricting blocks of code is not the only use case for annotations. Also sadly it reiterates a lot of trivia around how tagged functions and pointers interact with each another. About general annotation mechanism. Nobody yet put together a proper proposal (if any at all), all I see is people that keep saying: "it's simple - just add the general annotation mechanism!".
 Me thinks that simply tagging malloc/alloca/GC.allocate/etc..
 with a _user defined_ annotation should be sufficient without the
 need to introduce all of the above syntax to the language.

Simple tags have no much use on their own - need syntax to use them. The compiler can deduce their meaning so you also need a way tell the compiler: this function can only use "yellow" and "green" functions. Then some tags need to be compatible (or convertible with others), then there is a need to bundle tags together and so on. I agree that simply tagging functions with red/green is attractive and simple concept but to implement things like say safe you need more then that. You need to tweak what compiler puts in there for you (and forbid/allow as it suits your needs). In
 fact, I have a vague memory of reading about green/red marking of
 code via templates in plain old c++. No extra syntax required.

The method entails carrying around tag arguments apparently. It could be labeled as creative but a very backward way to do something simple. More specifically it shows that this has to be a language feature so that developers can focus on using it instead of being busy implementing it by hand.
 Here's the link:
 http://www.artima.com/cppsource/codefeaturesP.html

-- Olshansky Dmitry
Sep 01 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/02/2012 12:08 PM, foobar wrote:
 This basically reiterates the original post without answering my question.
 Regarding the general mechanism - Why do we have to re-invent the wheel?

No need to re-invent the wheel, but it needs to be tweaked because the vehicle it should be attached to has different features than other cars.
Sep 02 2012
prev sibling parent reply Piotr Szturmaj <bncrbme jadamspam.pl> writes:
Dmitry Olshansky wrote:
 On 01-Sep-12 17:01, Adam D. Ruppe wrote:
 On Saturday, 1 September 2012 at 12:39:41 UTC, Dmitry Olshansky wrote:
 I'd say
  nogc:
 at the top and deal is sealed.

BTW don't forget a yesgc would be good to counter it. We need a way to turn off each attribute to use this pattern best.

What I see with this nogc proposal is that that problem it tries to solve (and tool used to do so) is far more interesting and general. Namely the problem is to specify that some functions can call only specific subset of functions and able to use only specific subset of language features. It's more far reaching then just gc, one may want to get noblocking or async attribute to statically check e.g. that GUI thread can't ever block. (it would however require to identify API calls that don't block, not possible on every OS, might entail some wrappers etc. but is very desirable)

[snip]
 The idea in a nutshell:

[snip] How the attribute inferring works in this proposal?
Sep 03 2012
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 03-Sep-12 16:32, Piotr Szturmaj wrote:
 Dmitry Olshansky wrote:
 On 01-Sep-12 17:01, Adam D. Ruppe wrote:
 On Saturday, 1 September 2012 at 12:39:41 UTC, Dmitry Olshansky wrote:
 I'd say
  nogc:
 at the top and deal is sealed.

BTW don't forget a yesgc would be good to counter it. We need a way to turn off each attribute to use this pattern best.

What I see with this nogc proposal is that that problem it tries to solve (and tool used to do so) is far more interesting and general. Namely the problem is to specify that some functions can call only specific subset of functions and able to use only specific subset of language features. It's more far reaching then just gc, one may want to get noblocking or async attribute to statically check e.g. that GUI thread can't ever block. (it would however require to identify API calls that don't block, not possible on every OS, might entail some wrappers etc. but is very desirable)

[snip]
 The idea in a nutshell:

[snip] How the attribute inferring works in this proposal?

Interesting... and it seems like user-defined tags can't be inferred or that it would need more thought. However restrictions can be inferred by observing which things which template instance does and the tags of functions it calls. It's exactly the same as now but every restriction out of a set (say safe) now get inferred separately. -- Olshansky Dmitry
Sep 03 2012
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 1 September 2012 at 11:37:39 UTC, Piotr Szturmaj 
wrote:
 It's similar behavior to nothrow and pure. Instead of manually 
 avoiding GC allocations, compiler does this checks for you. 
 Imagine D doesn't have nothrow. You'd have to check every 
 called function to see if it doesn't throw. In big programs 
 throwing function may be left unnoticed and this is why we have 
 static nothrow checks in D.

I understand the benefit. However, there are at least two significant costs: 1. If I want my entire program to be GC free, I have to annotate every single function with 'nogc'. This is not something I want to do. 2. It's a new language feature and has all the associated costs: initial implementation, bug fixing, marking up of functions in Phobos, documentation, etc. Yes, with my approach, a rare allocation may go unnoticed, and you end up with an undesirable GC collection sometime in the future. It's not great, but it's not the end of the world, and I'm willing to risk that to avoid the costs I mentioned above.
Sep 01 2012
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 1 September 2012 at 12:39:41 UTC, Dmitry Olshansky 
wrote:
 I'd say
  nogc:
 at the top and deal is sealed.

BTW don't forget a yesgc would be good to counter it. We need a way to turn off each attribute to use this pattern best.
Sep 01 2012
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
C:

int mul(int x, int y);


Future D:

pure  safe nothrow nogc commutative associative distributive 
 forceinline mayoverflow int mul(int x, int y);
Sep 01 2012
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
C:

int mul(int x, int y);


Future D:

pure  safe nothrow nogc commutative associative distributive 
 forceinline mayoverflow int mul(int x, int y);
Sep 01 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Saturday, 1 September 2012 at 16:20:14 UTC, Dmitry Olshansky
wrote:
 On 01-Sep-12 17:01, Adam D. Ruppe wrote:
 On Saturday, 1 September 2012 at 12:39:41 UTC, Dmitry 
 Olshansky wrote:
 I'd say
  nogc:
 at the top and deal is sealed.

BTW don't forget a yesgc would be good to counter it. We need a way to turn off each attribute to use this pattern best.

What I see with this nogc proposal is that that problem it tries to solve (and tool used to do so) is far more interesting and general. Namely the problem is to specify that some functions can call only specific subset of functions and able to use only specific subset of language features. It's more far reaching then just gc, one may want to get noblocking or async attribute to statically check e.g. that GUI thread can't ever block. (it would however require to identify API calls that don't block, not possible on every OS, might entail some wrappers etc. but is very desirable) To peek at what's more there that you can get this way see C++ AMP on why restricting a block of code or function is useful e.g. here by Herb Sutter (including nice demo!): http://www.youtube.com/watch?v=HK9rJFpX32Y BTW safe is in the same bucket, it does allow only to call safe & trusted (read as "subset of ") functions and use a subset of language features. Actually exactly the same goes for pure (noglobal whatever) you just state this piece of code can't use this language feature (global variables, static variables, you name it), same for nothrow. And not long ago David Nadlinger showed that function level trusted doesn't help much with templates that are trusted for their own code, but need normal safety guarantee from foreign code brought by parameters. Thus restriction has to be able to work on a block scope too. So what I want with this? I want to bring together C++ AMP restrict and safe/ trusted/ system/pure/nothrow machinery in one clean and user-extensible future-proof feature. The idea in a nutshell:

I skipped the details but I do agree with the general sentiment. This long post begs the question - Why a general annotation mechanism isn't enough to justify adding so much complexity to the language? Me thinks that simply tagging malloc/alloca/GC.allocate/etc.. with a _user defined_ annotation should be sufficient without the need to introduce all of the above syntax to the language. In fact, I have a vague memory of reading about green/red marking of code via templates in plain old c++. No extra syntax required. Here's the link: http://www.artima.com/cppsource/codefeaturesP.html
Sep 01 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Saturday, 1 September 2012 at 21:50:33 UTC, Dmitry Olshansky 
wrote:
 On 02-Sep-12 01:13, foobar wrote:
 [snip]
 I skipped the details but I do agree with the general

 This long post begs the question - Why a general annotation
 mechanism isn't enough to justify adding so much complexity to
 the language?

It's quite short compared to all of the features it replaces and it had to allow for backwards compatibility. And it had to leave place for other annotations as I felt restricting blocks of code is not the only use case for annotations. Also sadly it reiterates a lot of trivia around how tagged functions and pointers interact with each another. About general annotation mechanism. Nobody yet put together a proper proposal (if any at all), all I see is people that keep saying: "it's simple - just add the general annotation mechanism!".

This basically reiterates the original post without answering my question. Regarding the general mechanism - Why do we have to re-invent the wheel? There already is a very successful model used by Java, C# and even C++11 (with slight variations). My "proper proposal" would than to simply look at the C# documentation!
 Me thinks that simply tagging malloc/alloca/GC.allocate/etc..
 with a _user defined_ annotation should be sufficient without

 need to introduce all of the above syntax to the language.

Simple tags have no much use on their own - need syntax to use them. The compiler can deduce their meaning so you also need a way tell the compiler: this function can only use "yellow" and "green" functions. Then some tags need to be compatible (or convertible with others), then there is a need to bundle tags together and so on. I agree that simply tagging functions with red/green is attractive and simple concept but to implement things like say safe you need more then that. You need to tweak what compiler puts in there for you (and forbid/allow as it suits your needs).

We already have at least part of the syntax in D. There is no need to "tweak the compiler", everything can be implemented in user code as shown in the link I provided. Including a proof of concept implementation in D, thanks to Bartosz.
  In
 fact, I have a vague memory of reading about green/red

 code via templates in plain old c++. No extra syntax required.

The method entails carrying around tag arguments apparently. It could be labeled as creative but a very backward way to do something simple. More specifically it shows that this has to be a language feature so that developers can focus on using it instead of being busy implementing it by hand.
 Here's the link:
 http://www.artima.com/cppsource/codefeaturesP.html


Again, this is a proof of concept that shows that even current D is sufficient to implement this sort of restrictions. Add annotations to the mix and you could implement this with a better syntax without the need to add extra parameters to functions (which is quite an elegant solution imo) No need for developers to implement this "by hand". Any language with a meta-data facility also provides standard annotations in its stdlib. If such restrictions are general purpose and useful enough they'll end up in Phobos anyway, allowing developers to concentrate on using them.
Sep 02 2012
prev sibling next sibling parent Maxim Fomin <maxim maxim-fomin.ru> writes:
2012/9/1 Dmitry Olshansky <dmitry.olsh gmail.com>:
 On 01-Sep-12 17:01, Adam D. Ruppe wrote:
 On Saturday, 1 September 2012 at 12:39:41 UTC, Dmitry Olshansky wrote:
 I'd say
  nogc:
 at the top and deal is sealed.

BTW don't forget a yesgc would be good to counter it. We need a way to turn off each attribute to use this pattern best.

What I see with this nogc proposal is that that problem it tries to solve (and tool used to do so) is far more interesting and general.

This reminds me custom attributes discussion which can be found in NG archive. Requested language additions relevant to attributes may be divided into three groups: 1 (tags). We want to tag some (likely function) declarations like nogc. This addition has several issues: - can tags carry additional information (for e.x tag(atrr1=opened))? - are they part of type or not (similar problems with default arguments)? - are they user-defined or entirely built in the language? - how to distinguish between restricted and permissive semantic (when tagged function care about whether they call non-tagged functions or not)? In this case possible solutions are to link user-defined enumeration types to existing syntax or to create entirely new syntax for purpose of holding additional information ( tag(atrr1=opened) where attr1 refers to enum attr1{} ). Simply tagged functions ( tag(name)) would have restrictive semantics. Another possible solution would be that construction " tag(attr1=value) type symbol;" is rewritten as struct __name {type symbol; enum att1 = value; alias type this; } - just not to add brand new features which are hard to implement and easy to introduce bugs. 2 (extending type). We want to define type (likely classes) and then extend its functionality. AFAIK C# attributes belong to this camp (correct me if I am wrong). In this case (likely class) type implicitly derives from attribute class. Obviously this also related to compile time because D being static typed language and has no way to be aware at run time that type functionality was extended. I think that it is not worth embedding in the language, because it is already possible to derive from base class, or interface, or instantiated template or to use mixin. 3 (extending instance). We want to make some instance of (extremely likely class) type to extend functionality while still remaining in original type without affecting type. Obviously this can be done only in run-time where actual instance of type is created. In this case possible solutions are to put associative array or some container in druntime (object.Object) which export attributes for e.x. through .attributeof property. I offer simple solutions because I estimate probability of introducing complex addition to the language now as extremely low.
Sep 02 2012
prev sibling next sibling parent "SomeDude" <lovelydear mailmetrash.com> writes:
On Saturday, 1 September 2012 at 02:52:23 UTC, bearophile wrote:
 One more comment, a quotation from the DIP18:

Currently, all functions can use garbage collected memory. 
Threads that call those functions must be managed by the 
garbage collector, which may occasionally suspend them to 
perform collection. This can lead to unwanted pauses, that are 
not acceptable in some situations, mainly real-time audio/video 
processing, gaming, and others.<

C heap functions like malloc(), calloc(), realloc() have the same problem, their run-time is not deterministic, so if you don't want pauses a noheap is useful. noheap is not meant to forbid calling low-level system-specific memory allocation functions. Bye, bearophile

It looks very much that several annotations are actually directed at the compiler or the runtime for specific optimisations. The could be used (or are used) as local compiler switches or runtime hints.
Sep 02 2012
prev sibling next sibling parent "SomeDude" <lovelydear mailmetrash.com> writes:
On Saturday, 1 September 2012 at 16:08:08 UTC, Dmitry Olshansky 
wrote:

 In fact if pure  safe nothrow could be combined in a user 
 defined tag:

 alias pure  safe nothrow nogc nice_func;

 nice_func int mul(int x, int y);

That would be awesome.
Sep 02 2012
prev sibling next sibling parent "SomeDude" <lovelydear mailmetrash.com> writes:
On Saturday, 1 September 2012 at 16:20:14 UTC, Dmitry Olshansky 
wrote:
....
 6. Probably we need a way to alias a list of restictions to 
 ease the use of frequent combinations. (No idea, but any sane 
 syntax around alias will do)

So you can tailor precisely the language to your own requirements, by basically inserting compiler switches in the code. That is a very smart idea, but I don't see how this could play well with most of the std lib.
Sep 02 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Sunday, 2 September 2012 at 16:59:19 UTC, Maxim Fomin wrote:

 This reminds me custom attributes discussion which can be found 
 in NG archive.
 Requested language additions relevant to attributes may be 
 divided
 into three groups:

 1 (tags). We want to tag some (likely function) declarations 
 like  nogc.

IMO, this is a poor choice.
 2 (extending type). We want to define type (likely classes) and 
 then
 extend its functionality.
 AFAIK C# attributes belong to this camp (correct me if I am 
 wrong). In
 this case (likely class) type implicitly derives
 from attribute class. Obviously this also related to compile 
 time
 because D being static typed language and has no
 way to be aware at run time that type functionality was 
 extended. I
 think that it is not worth embedding in the language, because
 it is already possible to derive from base class, or interface, 
 or
 instantiated template or to use mixin.

This is my preferred solution. This is also similar to Java. In fact there are only small differences between C# and Java semantics. This also includes other languages on the matching platforms, E.g. Nemerle is a .net language and provides the same attributes mechanism as C#. I assume that Scala does the same with Java annotations. The C# way (which I slightly prefer over the Java one): class Whatever : Annotation { // user defined annotation type } Java uses interface to define a user defined annotation. interface whatever { // user defined annotation type } I don't get your last point at all regarding inheritance - in both Java and C# it should be possible to use polymorphism with annotations. No need to add a new mechanism for it. class Foo : whatever {} // inheritance Really, the main things that are needed inside the compiler are a mechanism to attach custom attributes to various lingual parts, a way to mark such custom attributes by e.g. deriving from a special class (not unlike the IUnknown interface in D for COM) and standard compiler APIs for usage inside such an attribute. One extra step is defining an execution model for custom attributes: Java way is multi-pass - compiler executes the annotations which generate new version of the code, which is run through the compiler again, and so forth. Nemerle way - attributes can be macros which in turn can manipulate the AST. etc..
 3 (extending instance). We want to make some instance of 
 (extremely likely class) type to extend functionality
 while still remaining in original type without affecting type.

I don't get this at all. What's the purpose of having a single type?
 I offer simple solutions because I estimate probability of 
 introducing
 complex addition to the language now as
 extremely low.

Sep 02 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--bcaec54fb624b6cb6d04c8c8e72e
Content-Type: text/plain; charset=UTF-8

On 1 September 2012 19:08, Dmitry Olshansky <dmitry.olsh gmail.com> wrote:

 On 01-Sep-12 19:58, Peter Alexander wrote:

 C:

 int mul(int x, int y);


 Future D:

 pure  safe nothrow nogc commutative associative distributive
  forceinline mayoverflow int mul(int x, int y);

and you are done :) And don't even get me started at MS extensions... Either way this is a kind of thing that only some users need but std library have to use all of them just so that those 'some users' can use it. In fact if pure safe nothrow could be combined in a user defined tag: alias pure safe nothrow nogc nice_func; nice_func int mul(int x, int y); and even C has it via macros, damn...

Aside the context of this thread, this would actually be REALLY useful. I think I've asked about it a few times before. There is already a fairly annoying problem where GDC supports GNU attributes that DMD doesn't know about, which are basically impossible to use; there's no nice way to use compiler-specific attributes without some sort of alias like this. I think the only current way is to duplicate the definitions in different version blocks. An attribute alias would be super handy, particularly when abstracting support for multiple compilers. --bcaec54fb624b6cb6d04c8c8e72e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div class=3D"gmail_quote">On 1 September 2012 19:08, Dmitry Olshansky <spa= n dir=3D"ltr">&lt;<a href=3D"mailto:dmitry.olsh gmail.com" target=3D"_blank= ">dmitry.olsh gmail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail= _quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:= 1ex"> <div class=3D"HOEnZb"><div class=3D"h5">On 01-Sep-12 19:58, Peter Alexander= wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> C:<br> <br> int mul(int x, int y);<br> <br> <br> Future D:<br> <br> pure safe nothrow nogc commutative associative distributive<br> forceinline mayoverflow int mul(int x, int y);<br> <br> </blockquote> <br></div></div> No it&#39;s GNU C++ of today just replace &#39;pure&#39; with __attribute__= ((pure))<br> and you are done :)<br> And don&#39;t even get me started at MS extensions...<br> <br> Either way this is a kind of thing that only some users need but std librar= y have to use all of them just so that those &#39;some users&#39; can use i= t.<br> <br> In fact if pure safe nothrow could be combined in a user defined tag:<br> <br> alias pure safe nothrow nogc nice_func;<br> <br> nice_func int mul(int x, int y);<br> <br> and even C has it via macros, damn...</blockquote><div><br></div><div>Aside= the context of this thread, this would actually be REALLY useful. I think = I&#39;ve asked about it a few times before.</div><div>There is already a fa= irly annoying problem where GDC supports GNU attributes that DMD doesn&#39;= t know about, which are basically impossible to use; there&#39;s no nice wa= y to use compiler-specific attributes without some sort of alias like this.= </div> <div>I think the only current way is to duplicate the definitions in differ= ent version blocks.</div><div>An attribute alias would be super handy, part= icularly when abstracting support for multiple compilers.</div></div> --bcaec54fb624b6cb6d04c8c8e72e--
Sep 03 2012
prev sibling next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Monday, 3 September 2012 at 00:01:09 UTC, foobar wrote:
 2 (extending type). We want to define type (likely classes) 
 and then
 extend its functionality.
 AFAIK C# attributes belong to this camp (correct me if I am 
 wrong). In
 this case (likely class) type implicitly derives
 from attribute class. Obviously this also related to compile 
 time
 because D being static typed language and has no
 way to be aware at run time that type functionality was 
 extended. I
 think that it is not worth embedding in the language, because
 it is already possible to derive from base class, or 
 interface, or
 instantiated template or to use mixin.

This is my preferred solution. This is also similar to Java. In fact there are only small differences between C# and Java semantics. This also includes other languages on the matching platforms, E.g. Nemerle is a .net language and provides the same attributes mechanism as C#. I assume that Scala does the same with Java annotations. The C# way (which I slightly prefer over the Java one): class Whatever : Annotation { // user defined annotation type } Java uses interface to define a user defined annotation. interface whatever { // user defined annotation type } I don't get your last point at all regarding inheritance - in both Java and C# it should be possible to use polymorphism with annotations. No need to add a new mechanism for it. class Foo : whatever {} // inheritance Really, the main things that are needed inside the compiler are a mechanism to attach custom attributes to various lingual parts, a way to mark such custom attributes by e.g. deriving from a special class (not unlike the IUnknown interface in D for COM) and standard compiler APIs for usage inside such an attribute. One extra step is defining an execution model for custom attributes: Java way is multi-pass - compiler executes the annotations which generate new version of the code, which is run through the compiler again, and so forth. Nemerle way - attributes can be macros which in turn can manipulate the AST. etc..

The question with this approach is follows: if attributes are just another way of deriving why not use old syntax?
 3 (extending instance). We want to make some instance of 
 (extremely likely class) type to extend functionality
 while still remaining in original type without affecting type.

I don't get this at all. What's the purpose of having a single type?

It is not a single type, it is extending functionality of instance at run time which can be simulated by containers.
Sep 03 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Monday, 3 September 2012 at 16:21:08 UTC, Maxim Fomin wrote:
 The question with this approach is follows: if attributes are 
 just another way of deriving why not use old syntax?

I think you conflate two concepts - the custom attributes _themselves_ can use the old syntax for polymorphism and reuse. This is however completely orthogonal from the action of attaching a custom attribute to an object (function, variable, type, etc). This is close to AOP and cross cutting concerns. an example (perhaps a bit convoluted, just to illustrate a point): // I want to have all my logging related code in one place. class Log : Attribute {} // inheritance of attributes class LegalLog : Log {} // for legal department class DeveloperLog : Log {} // for developers // usage LegalLog(params) class Foo : Bar {} DeveloperLog(params) class Goo : Foo {} The idea is that all the specifics of logging are concentrated in one place and not spread throughout the application. Say I want to change logging file format, I can do it by simply changing the relevant attributes without touching the main business logic of my application. It make no sense in this scenario to derive my business object classes from the Logging classes.
Sep 03 2012
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
On Saturday, 1 September 2012 at 00:40:23 UTC, Piotr Szturmaj 
wrote:
 http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP18

How this nogc attribute will work with phobos and druntime?
Sep 05 2012
parent Piotr Szturmaj <bncrbme jadamspam.pl> writes:
Kagamin wrote:
 On Saturday, 1 September 2012 at 00:40:23 UTC, Piotr Szturmaj wrote:
 http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP18

How this nogc attribute will work with phobos and druntime?

I don't know what do you mean exactly. Some of the phobos and druntime functions will certainly not work, but some will do. Also, I think that nogc/pure/nothrow should be inferred, so they can be used without marking so many functions.
Sep 05 2012
prev sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On 2012-09-01, 18:20, Dmitry Olshansky wrote:

 The idea in a nutshell:

 - specify a way attach tags to function declarations (to "paint"  
 functions)

 - allow specifying is-a relation between tags (supertags and subtags).

 - add restrict specification that allows user to enforce on a block of  
 code the following:
 	a) disallow use of certain language features
 	b) allow(or disallow) calling function with specific tags

 - fit with existing world and provide clean migration path, specify  
 defaults

 I claim that it's enough to implement pure/safe/trusted/nothrow/nogc and  
 proposed nogc.

I'm sorry I have not gotten around to commenting on this before, but I *really* like this proposal. As foobar has said, a general annotation system would be real nice, but as far as I can see, it can be built on top of this, at a later point. Also, I cannot see that this has been turned into a proper DIP. :p http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs -- Simen
Sep 22 2012