www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Massive loss for D on Tiobe

reply Georg Wrede <georg.wrede iki.fi> writes:
D made the May headline on Tiobe: "Programming language D suffers sharp 
fall". You can say that again, D went down 5 places, to below languges 
like RPG(OS/400) and ABAP!



D's loss seems unbelievable. D now has a 0.628% share, which is even 
less than what it's lost (-0.82%) in the last 12 months. What could be 
the reasons for it? Is it even possible to figure out any reason??

Can this loss induce people to abandon D, and others to not take it up, 
leading to cumulating losses in the coming months? What do we have to do 
to prevent this?
May 06 2009
next sibling parent grauzone <none example.net> writes:
Georg Wrede wrote:
 
 
 D made the May headline on Tiobe: "Programming language D suffers sharp 
 fall". You can say that again, D went down 5 places, to below languges 
 like RPG(OS/400) and ABAP!
 
 
 
 D's loss seems unbelievable. D now has a 0.628% share, which is even 
 less than what it's lost (-0.82%) in the last 12 months. What could be 
 the reasons for it? Is it even possible to figure out any reason??
 
 Can this loss induce people to abandon D, and others to not take it up, 
 leading to cumulating losses in the coming months? What do we have to do 
 to prevent this?

D2.0. Now flame away.
May 06 2009
prev sibling next sibling parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Georg Wrede (georg.wrede iki.fi)'s article
 D made the May headline on Tiobe: "Programming language D suffers sharp
 fall". You can say that again, D went down 5 places, to below languges
 like RPG(OS/400) and ABAP!
 D's loss seems unbelievable. D now has a 0.628% share, which is even
 less than what it's lost (-0.82%) in the last 12 months. What could be
 the reasons for it? Is it even possible to figure out any reason??
 Can this loss induce people to abandon D, and others to not take it up,
 leading to cumulating losses in the coming months? What do we have to do
 to prevent this?

This fully convinces me that the Tiobe index should not be taken at face value. How does a language hit an all time high and a multiyear low within 2 months of each other? At best the Tiobe index is an unbiased but extremely high variance estimator of language popularity, and meaningful results can only be produced by averaging results over a period much longer than a month. At worst, it's so biased that it's just plain garbage.
May 06 2009
prev sibling next sibling parent reply "Carlos Smith" <carlos-smith sympatico.ca> writes:
"Georg Wrede" <georg.wrede iki.fi> a écrit

 Can this loss induce people to abandon D, and others to not take it 
 up, leading to cumulating losses in the coming months? What do we 
 have to do to prevent this?

Yes. Give D1 a future. Shift focus on D1 (D2 is experimental). Make D1 really usable in the workplace. Being called stable is not enough. Produce a grammar for the language. This will give it a definition on which every body will align. Fix any inconsistencies in the language. Choose one (1) license for all DigitalMars D related stuff. Go true Open Source. No strings attached. I was absolutely thrilled to read that D has gone Open Source. Then i was quite unthrilled after i have read the official license. This episode had a negative impact on D. Get rid of OMF... D has a future ...
May 06 2009
parent reply Eldar Insafutdinov <e.insafutdinov gmail.com> writes:
Carlos Smith Wrote:

 "Georg Wrede" <georg.wrede iki.fi> a écrit
 
 Can this loss induce people to abandon D, and others to not take it 
 up, leading to cumulating losses in the coming months? What do we 
 have to do to prevent this?

Yes. Give D1 a future. Shift focus on D1 (D2 is experimental). Make D1 really usable in the workplace. Being called stable is not enough.

D1 has complete open sources now, submit patches, make it more stable. That's what people are actually doing.
 Get rid of OMF...

Both hands raised for that...
 D has a future ...

Oh yeah!
May 06 2009
parent Vincenzo Ampolo <vincenzo.ampolo gmail.com> writes:
Eldar Insafutdinov wrote:

 D1 has complete open sources now, submit patches, make it more 

 That's what people are actually doing.

From dmd.2.029/dmd/src/dmd/backendlicense.txt "The Software is copyrighted and comes with a single user license, and may not be redistributed. If you wish to obtain a redistribution license, please contact Digital Mars." This is not Open Source neither Free Software IMHO. So D2, the "bleeding edge" which could be very interesting for development is not free software (at least backend) I remember that an open backend is needed to port a compiler into other platforms (x86_64, armel, sparc, powerpc, ecc ecc) Let's see D1. i don't see any "backend" (backend not released?) directory and in dmd.1.030/dmd/license.txt there is again: "The Software is copyrighted and comes with a single user license, and may not be redistributed. If you wish to obtain a redistribution license, please contact Digital Mars." inside dmd/src/dmd there are gpl.txt and artistic.txt. So dmd frontend is gplv1 (well, quite old version of gpl, but at least it's gpl!).
May 07 2009
prev sibling next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Georg Wrede wrote:
 D's loss seems unbelievable. D now has a 0.628% share, which is even 
 less than what it's lost (-0.82%) in the last 12 months. What could be 
 the reasons for it? Is it even possible to figure out any reason??

Of course it's unbelievable. This change didn't happen over a year's time, it happened in one month. This means that the methodology Tiobe uses changed, or the search portals changed their hit count algorithm. Notice http://www.tiobe.com/index.php/content/paperinfo/tpci/tpci_definition.htm where they automatically discount all D hits by 10%. They don't do that for C. For example, of the first 100 hits of "D programming" on google, I found only 6 that were not about D, two of which was already excluded by Tiobe's algorithm. That's 4%, not 10%. I found 3 non-C ones for "C programming" in the first 100. That's 3%, not 0%.
May 06 2009
next sibling parent reply superdan <super dan.org> writes:
Walter Bright Wrote:

 Georg Wrede wrote:
 D's loss seems unbelievable. D now has a 0.628% share, which is even 
 less than what it's lost (-0.82%) in the last 12 months. What could be 
 the reasons for it? Is it even possible to figure out any reason??

Of course it's unbelievable. This change didn't happen over a year's time, it happened in one month. This means that the methodology Tiobe uses changed, or the search portals changed their hit count algorithm. Notice http://www.tiobe.com/index.php/content/paperinfo/tpci/tpci_definition.htm where they automatically discount all D hits by 10%. They don't do that for C. For example, of the first 100 hits of "D programming" on google, I found only 6 that were not about D, two of which was already excluded by Tiobe's algorithm. That's 4%, not 10%. I found 3 non-C ones for "C programming" in the first 100. That's 3%, not 0%.

i sorta prefer grau douche zone's theory. makes no sense 'cept in da framework where he's a retarded dumbass & evil to boot. but u gotta respect the man. he's waited so patiently fer dis opportunity to suck collective cock. gotta give it 2 da man.
May 06 2009
parent reply grauzone <none example.net> writes:
You're offending me. Please stop this immediately.

Thank you.
May 06 2009
parent reply superdan <super dan.org> writes:
grauzone Wrote:

 You're offending me. Please stop this immediately.
 
 Thank you.

wut happened to `flame on', hercules? anyway just killfile me. i dun change handles. better yet. stop being an ass. we'd all be way better off. suit yerself.
May 06 2009
parent reply grauzone <none example.net> writes:
superdan wrote:
 grauzone Wrote:
 
 You're offending me. Please stop this immediately.

 Thank you.

wut happened to `flame on', hercules? <expletives deleted>

When it comes to "everyone being better off", what about stopping talking like a 16 year old rapper on a hormone trip? Grow up. Thank you.
May 07 2009
parent reply superdan <super dan.org> writes:
grauzone Wrote:

 superdan wrote:
 grauzone Wrote:
 
 You're offending me. Please stop this immediately.

 Thank you.

wut happened to `flame on', hercules? <expletives deleted>

When it comes to "everyone being better off", what about stopping talking like a 16 year old rapper on a hormone trip? Grow up.

expletives deleted? then u missed the subject. subject was yer bashing d2 more often than a teen has a boner. that is da problem, not my expletives. ok, yer highness, we fuckin' get it. u dun like d2. u made ur point several times now move on with life. if u r too hung up u say idiotic crap like this with tiobe & d2. better have a foul mouth & a clean mind than vice versa. so u grow up friend. til then, at least dun flamebait. dun do da crime if u can't do da time.
May 07 2009
parent reply grauzone <none example.net> writes:
superdan wrote:
 grauzone Wrote:
 
 superdan wrote:
 grauzone Wrote:

 You're offending me. Please stop this immediately.

 Thank you.

<expletives deleted>

talking like a 16 year old rapper on a hormone trip? Grow up.

expletives deleted? then u missed the subject. subject was yer bashing d2 more often than <expletives deleted>

I don't dislike D2. (OK, except for some parts like const & immutable.) I'm just thinking that what D actually needs, is a stable implementation, and not more features.
 better have a foul mouth & a clean mind than vice versa.

Sorry, the "I have a foul mouth, but what I'm really saying is highly intellectual and deep, and thus everyone criticizing me is actually dumb" turn doesn't work on me. Grow up.
May 07 2009
parent reply Vincenzo Ampolo <vincenzo.ampolo gmail.com> writes:
grauzone wrote:

 I'm just thinking that what D actually needs, is a stable
 implementation, and not more features.

+1 And... i think grauzone is right, Superdan, your language seems offensive to me too. Please stop a not needed flame.
May 07 2009
parent Don <nospam nospam.com> writes:
Vincenzo Ampolo wrote:
 grauzone wrote:
 
 I'm just thinking that what D actually needs, is a stable
 implementation, and not more features.

+1

I think you'll like the next DMD release <g>.
May 07 2009
prev sibling next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Ignore the Tiobe index. It's trash.

Bye,
bearophile
May 06 2009
prev sibling parent Jarrett Billingsley <jarrett.billingsley gmail.com> writes:
On Thu, May 7, 2009 at 10:30 AM, Don <nospam nospam.com> wrote:
 Vincenzo Ampolo wrote:
 grauzone wrote:

 I'm just thinking that what D actually needs, is a stable
 implementation, and not more features.

+1

I think you'll like the next DMD release <g>.

Oman, will it be another 0.178? I can only hope..
May 07 2009
prev sibling next sibling parent reply BCS <ao pathlink.com> writes:
Reply to Georg,

 D made the May headline on Tiobe: "Programming language D suffers
 sharp fall". You can say that again, D went down 5 places, to below
 languges like RPG(OS/400) and ABAP!
 
 D's loss seems unbelievable. D now has a 0.628% share, which is even
 less than what it's lost (-0.82%) in the last 12 months. What could be
 the reasons for it? Is it even possible to figure out any reason??
 
 Can this loss induce people to abandon D, and others to not take it
 up, leading to cumulating losses in the coming months? What do we have
 to do to prevent this?
 

take a look at the graph for RPG(OS/400) and D http://www.tiobe.com/index.php/paperinfo/tpci/RPG_(OS_400).html http://www.tiobe.com/index.php/paperinfo/tpci/D.html something/someone is gameing the system.
May 06 2009
next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
BCS wrote:
 something/someone is gameing the system.

Here's some more food for thought. Tiobe says they do a search for "xxx programming". "C programming" 2,000,000 19.537 "Pascal programming" 136,000 .776 "D programming" 187,000 .628 This doesn't add up. Also, I tried "D programming" an hour ago and got 437,000 hits. ???
May 06 2009
next sibling parent Daniel Keep <daniel.keep.lists gmail.com> writes:
Denis Koroskin wrote:
 On Thu, 07 May 2009 01:52:23 +0400, Walter Bright
 <newshound1 digitalmars.com> wrote:
 
 BCS wrote:
 something/someone is gameing the system.

Here's some more food for thought. Tiobe says they do a search for "xxx programming". "C programming" 2,000,000 19.537 "Pascal programming" 136,000 .776 "D programming" 187,000 .628 This doesn't add up. Also, I tried "D programming" an hour ago and got 437,000 hits. ???

I got just 184 000 for "D programming"

Just did a google search, got 186k. I think we should stop basing D's worth as a language on what Tiobe says. Sticks and stones may break D's bones, but Tiobe has to sleep eventually... -- Daniel
May 06 2009
prev sibling parent Georg Wrede <georg.wrede iki.fi> writes:
Walter Bright wrote:
 BCS wrote:
 something/someone is gameing the system.

Here's some more food for thought. Tiobe says they do a search for "xxx programming". "C programming" 2,000,000 19.537 "Pascal programming" 136,000 .776 "D programming" 187,000 .628 This doesn't add up. Also, I tried "D programming" an hour ago and got 437,000 hits. ???

Tried Google: 184,000 for "D programming" 166,000 for "3d programming" 1,590,000 for "C++ programming" 1,950,000 for "C programming" 125,000 for "Pascal programming" 292,000 for "Delphi programming" 2,920,000 for "Java programming" But 114,000 for "abap programming" 44,400 for "rpg programming"
May 06 2009
prev sibling next sibling parent Georg Wrede <georg.wrede iki.fi> writes:
BCS wrote:
 Reply to Georg,
 
 D made the May headline on Tiobe: "Programming language D suffers
 sharp fall". You can say that again, D went down 5 places, to below
 languges like RPG(OS/400) and ABAP!

 D's loss seems unbelievable. D now has a 0.628% share, which is even
 less than what it's lost (-0.82%) in the last 12 months. What could be
 the reasons for it? Is it even possible to figure out any reason??

 Can this loss induce people to abandon D, and others to not take it
 up, leading to cumulating losses in the coming months? What do we have
 to do to prevent this?

take a look at the graph for RPG(OS/400) and D http://www.tiobe.com/index.php/paperinfo/tpci/RPG_(OS_400).html http://www.tiobe.com/index.php/paperinfo/tpci/D.html something/someone is gameing the system.

http://en.wikipedia.org/wiki/IBM_RPG http://en.wikipedia.org/wiki/Abap Tears and grief, RPG(OS/400) is a language for *punch cards*. And ABAP is Gerry's answer to *COBOL*. And Finland just lost to the *USA* in hockey, right after we beat Canada. Jansen (of Tiobe) should probably not go wild adjusting the knobs. It may erode their credibility. But blaming him doesn't tidy our nest either. The D landscape in front of the programmer in search of a C++ replacement isn't what it should be. The language is ten years old, but you'd never guess. (Except from outdated stuff on each website.)
May 06 2009
prev sibling parent reply Derek Parnell <derek psych.ward> writes:
On Wed, 6 May 2009 21:02:58 +0000 (UTC), BCS wrote:

 Reply to Georg,
 
 D made the May headline on Tiobe: "Programming language D suffers
 sharp fall". You can say that again, D went down 5 places, to below
 languges like RPG(OS/400) and ABAP!
 
 D's loss seems unbelievable. D now has a 0.628% share, which is even
 less than what it's lost (-0.82%) in the last 12 months. What could be
 the reasons for it? Is it even possible to figure out any reason??
 
 Can this loss induce people to abandon D, and others to not take it
 up, leading to cumulating losses in the coming months? What do we have
 to do to prevent this?
 

take a look at the graph for RPG(OS/400) and D http://www.tiobe.com/index.php/paperinfo/tpci/RPG_(OS_400).html

I wonder if they have accidentally include "RolePlayingGame" programming in the RPG category? For "RPG Programming" I get I get 47,500 hits. For "RPG Programming" + OS/400 I get 9,840 hits. On the surface, it seems that the Tiobe figures a very "rubbery", if not outright dishonest. -- Derek Parnell Melbourne, Australia skype: derek.j.parnell
May 07 2009
parent Walter Bright <newshound1 digitalmars.com> writes:
Derek Parnell wrote:
 On the surface, it seems that the Tiobe figures a very "rubbery", if not
 outright dishonest.

I sent an email to Tiobe, and received a nice reply from Paul Jansen, who runs the index. He showed me his numbers, and the biggest factor in the drop in D's ranking was a large drop in hits from Yahoo's engine. Why that would be neither of us knows. He agreed that the 90% "adjustment" factor needed to be revisited. Anyhow, I don't believe there is anything dishonest going on. It's just the erratic nature of what search engines report as the "number of hits". Google's varies wildly all over the place. Who knows what is going on at Yahoo.
May 07 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 07 May 2009 01:52:23 +0400, Walter Bright <newshound1 digitalmars.com>
wrote:

 BCS wrote:
 something/someone is gameing the system.

Here's some more food for thought. Tiobe says they do a search for "xxx programming". "C programming" 2,000,000 19.537 "Pascal programming" 136,000 .776 "D programming" 187,000 .628 This doesn't add up. Also, I tried "D programming" an hour ago and got 437,000 hits. ???

I got just 184 000 for "D programming"
May 06 2009
prev sibling next sibling parent Vincenzo Ampolo <vincenzo.ampolo gmail.com> writes:
Georg Wrede wrote:

 D's loss seems unbelievable. D now has a 0.628% share, which is even
 less than what it's lost (-0.82%) in the last 12 months. What could 

 the reasons for it? Is it even possible to figure out any reason??

IMHO it's just marketing. Do you wanna still rise that 0.628% share? If yes: 1) Use D 2) Help into other projects and do not create one-man projects 3) Hope that the "big" projects merge into few useful ones well supported (like phobos and tango in D2)(maybe compilers should do it too?).
May 06 2009
prev sibling parent reply Alix Pexton <alix.DOT.pexton gmail.DOT.com> writes:
Georg Wrede wrote:
 
 
 D made the May headline on Tiobe: "Programming language D suffers sharp 
 fall". You can say that again, D went down 5 places, to below languges 
 like RPG(OS/400) and ABAP!
 
 
 
 D's loss seems unbelievable. D now has a 0.628% share, which is even 
 less than what it's lost (-0.82%) in the last 12 months. What could be 
 the reasons for it? Is it even possible to figure out any reason??
 
 Can this loss induce people to abandon D, and others to not take it up, 
 leading to cumulating losses in the coming months? What do we have to do 
 to prevent this?

There was a small drop last month, and a note saying that hits for DTrace had been eliminated as false positives for the D Programming Language. If it had been possible for DTrace to be such a false positive, I am curious about what other false positives could be effecting other languages, but I definitly think it is a blow to the index's credibility. A...
May 07 2009
next sibling parent reply Nick B <nick.barbalich gmail.com> writes:
Hi

It seems that Bartosz's  latest post, dated April 26 th is missing from 
his blog.

See :

http://bartoszmilewski.wordpress.com/


Nick B.
May 07 2009
parent reply Bartosz Milewski <bartosz-nospam relisoft.com> writes:
The post is back, rewritten and with some code teasers.

Nick B Wrote:

 Hi
 
 It seems that Bartosz's  latest post, dated April 26 th is missing from 
 his blog.
 
 See :
 
 http://bartoszmilewski.wordpress.com/
 
 
 Nick B.

May 26 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bartosz Milewski wrote:
 The post is back, rewritten and with some code teasers.

Has anyone reddit'ed it yet? Andrei
May 26 2009
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bartosz Milewski wrote:
 The post is back, rewritten and with some code teasers.

http://www.reddit.com/r/programming/comments/8ngwn/racefree_multithreading_in_a_hypothetical_language/ Vote up! Andrei
May 26 2009
prev sibling next sibling parent reply Jason House <jason.james.house gmail.com> writes:
Bartosz Milewski wrote:

 The post is back, rewritten and with some code teasers.

We've been teased for 6 months or more. I'm hoping the details will come out quickly now! Here's what I took away from the article: * Goal is to have minimal code changes for single threaded code * unique and lent are two new new transitive type constructors * lockfree is a new storage class (to guarantee sequential consistency) * The new := operator is used for move semantics (if appropriate for type) * Objects can be declared as self-owned I think a deep understanding of exactly what the final design is requires understanding the ownership scheme which isn't described yet. unique is to invariant as lent is to const. Function arguments can be declared as lent and accept both unique and non-unique types (just like const can accept immutable and non-immutable types). Lent basically means what I think scope was intended to mean for function arguments. I'm happy to finally see unique in the type system since it really felt like a gaping hole to me.
May 26 2009
parent reply Bartosz Milewski <bartosz-nospam relisoft.com> writes:
You pretty much nailed it. The ownership scheme will be explained in more
detail in the next two installments, which are almost ready.
May 27 2009
parent reply Tim Matthews <tim.matthews7 gmail.com> writes:
This may seem slightly OT but in your blog "I will use syntax similar to 
that of the D programming language, but C++ and Java programmers 
shouldn’t have problems following it."


class MVar<T> {
private:
     T    _msg;
     bool _full;
public:
     // put: asynchronous (non-blocking)
     // Precondition: MVar must be empty
     void put(T msg) {
         assert (!_full);
         _msg := msg; // move
         _full = true;
         notify();
     }
     // take: If empty, blocks until full.
     // Removes the message and switches state to empty
     T take() {
         while (!_full)
             wait();
         _full = false;
         return := _msg;
     }
}
auto mVar = new MVar<owner::self, int>;

Why not MVar!(owner::self, int)? Why go back to ambiguous templates? 
Apart from the move operator it looks like c++ to me. Sorry if this 
doesn't make sense but I've missed a few previous posts.
May 27 2009
next sibling parent Robert Fraser <fraserofthenight gmail.com> writes:
Tim Matthews wrote:
 
 This may seem slightly OT but in your blog "I will use syntax similar to 
 that of the D programming language, but C++ and Java programmers 
 shouldn’t have problems following it."
 
 
 class MVar<T> {
 private:
     T    _msg;
     bool _full;
 public:
     // put: asynchronous (non-blocking)
     // Precondition: MVar must be empty
     void put(T msg) {
         assert (!_full);
         _msg := msg; // move
         _full = true;
         notify();
     }
     // take: If empty, blocks until full.
     // Removes the message and switches state to empty
     T take() {
         while (!_full)
             wait();
         _full = false;
         return := _msg;
     }
 }
 auto mVar = new MVar<owner::self, int>;
 
 Why not MVar!(owner::self, int)? Why go back to ambiguous templates? 
 Apart from the move operator it looks like c++ to me. Sorry if this 
 doesn't make sense but I've missed a few previous posts.

I think most of Bartoz's readers are C++ users. The "I will use syntax similar to that of the D programming language" was probably put there in a first draft and after revision it was changed to more C++y example code, but the sentence wasn't removed.
May 27 2009
prev sibling parent reply Jason House <jason.james.house gmail.com> writes:
Tim Matthews Wrote:

 
 This may seem slightly OT but in your blog "I will use syntax similar to 
 that of the D programming language, but C++ and Java programmers 
 shouldn’t have problems following it."
 
 
 class MVar<T> {
 private:
      T    _msg;
      bool _full;
 public:
      // put: asynchronous (non-blocking)
      // Precondition: MVar must be empty
      void put(T msg) {
          assert (!_full);
          _msg := msg; // move
          _full = true;
          notify();
      }
      // take: If empty, blocks until full.
      // Removes the message and switches state to empty
      T take() {
          while (!_full)
              wait();
          _full = false;
          return := _msg;
      }
 }
 auto mVar = new MVar<owner::self, int>;
 
 Why not MVar!(owner::self, int)? Why go back to ambiguous templates? 
 Apart from the move operator it looks like c++ to me. Sorry if this 
 doesn't make sense but I've missed a few previous posts.

Don't read into it. I took it as being more readable for non-D users. Angle brackets show up in C++ templates, Java generics, and C# generics. IMHO, <> is more recognizable, even for those that don't code in any of the languages mentioned. D's syntax is good, just not wide spread. Notice the lack of a template<typename T> that's required for C++, instead, the template argument is after the class name. There's also no constrictor or initializers which would be bugs in C++. It still looks like tweaked D code.
May 27 2009
parent Tim Matthews <tim.matthews7 gmail.com> writes:
Jason House wrote:

 Don't read into it. I took it as being more readable for non-D users. Angle
brackets show up in C++ templates, Java generics, and C# generics. IMHO, <> is
more recognizable, even for those that don't code in any of the languages
mentioned. D's syntax is good, just not wide spread. 
 
 Notice the lack of a template<typename T> that's required for C++, instead,
the template argument is after the class name. There's also no constrictor or
initializers which would be bugs in C++. It still looks like tweaked D code.

Fuck that! I am very experienced in C++ , C# and some in java and read some of his articles a long time ago, I was just pointing out that design decision and that "Similar to D but C++/Java users will be OK" message.
May 27 2009
prev sibling parent reply Jason House <jason.james.house gmail.com> writes:
The article implies some level of flow analysis. Has Walter come around on this
topic?

As far as considering a variable moved, I believe the following should be
reasonable
• Any if statement (or else clause) containing a move
• Any switch statement containing a move for any case
• Any fall-through cases where the prior case moved the variable
• Any function call not using a lent argument for the variable
• Moving inside a loop should be illegal

An explicit is null check should be able to bypass these rules. There are
probably ways to loosen the looping rule such as if there is a way to guarantee
the moved variable won't be read from again.

Very similar rules can be used for detecting initialization of (unique)
variables. A variable can be considered initialized if:
• Both the if and else must initialize a variable
• All cases in a switch must initialize a variable
• Out parameter in a function call 
• Loops can't initialize variables
  relaxation: can init if guaranteed to run at least once

Those rules should be sufficiently simple to implement and extremely tolerable
for programmers.
Inevitably, I missed a case, but I hope the idea is clear, and that whatever I
overlooked does not add complexity.


Bartosz Milewski Wrote:

 The post is back, rewritten and with some code teasers.
 
 Nick B Wrote:
 
 Hi
 
 It seems that Bartosz's  latest post, dated April 26 th is missing from 
 his blog.
 
 See :
 
 http://bartoszmilewski.wordpress.com/
 
 
 Nick B.


May 27 2009
parent reply Jason House <jason.james.house gmail.com> writes:
I'm really surprised by the lack of design discussion in this thread. It's
amazing how there can be huge bursts of discussion on which keyword to use
(e.g. manifest constants), but then complete silence about major design
decisions like thread safety that defines new transitive states and a bunch of
new keywords. The description even made parallels to the (previously?)
unpopular const architecture. 

Maybe people are waiting for Walter to go through all the hard work of
implementing this stuff before complaining that it's crap and proclaiming
Walter should have done in the first place? 
This seems really unfair to Walter. Then again, I see no indication of Walter
wanting anything else.


Jason House Wrote:

 The article implies some level of flow analysis. Has Walter come around on
this topic?
 
 As far as considering a variable moved, I believe the following should be
reasonable
 • Any if statement (or else clause) containing a move
 • Any switch statement containing a move for any case
 • Any fall-through cases where the prior case moved the variable
 • Any function call not using a lent argument for the variable
 • Moving inside a loop should be illegal
 
 An explicit is null check should be able to bypass these rules. There are
probably ways to loosen the looping rule such as if there is a way to guarantee
the moved variable won't be read from again.
 
 Very similar rules can be used for detecting initialization of (unique)
variables. A variable can be considered initialized if:
 • Both the if and else must initialize a variable
 • All cases in a switch must initialize a variable
 • Out parameter in a function call 
 • Loops can't initialize variables
   relaxation: can init if guaranteed to run at least once
 
 Those rules should be sufficiently simple to implement and extremely tolerable
for programmers.
 Inevitably, I missed a case, but I hope the idea is clear, and that whatever I
overlooked does not add complexity.
 
 
 Bartosz Milewski Wrote:
 
 The post is back, rewritten and with some code teasers.
 
 Nick B Wrote:
 
 Hi
 
 It seems that Bartosz's  latest post, dated April 26 th is missing from 
 his blog.
 
 See :
 
 http://bartoszmilewski.wordpress.com/
 
 
 Nick B.



May 28 2009
next sibling parent reply grauzone <none example.net> writes:
1. Everyone agrees anyway, that emulating fork() is the best idea to 
deal with multithreading and synchronization.
2. We'll yet have to see how an implementation of the proposed design 
will work out. This means Walter has to implement it. Reading blog 
entries about it is almost a bigger waste of time than discussing in 
this newsgroup.
3. Not that many people are interested in D2.
4. Bikeshed colors
May 28 2009
parent dsimcha <dsimcha yahoo.com> writes:
== Quote from grauzone (none example.net)'s article
 1. Everyone agrees anyway, that emulating fork() is the best idea to
 deal with multithreading and synchronization.
 2. We'll yet have to see how an implementation of the proposed design
 will work out. This means Walter has to implement it. Reading blog
 entries about it is almost a bigger waste of time than discussing in
 this newsgroup.
 3. Not that many people are interested in D2.
 4. Bikeshed colors

Yeah, unfortunately for something as complex as what's being proposed, I have a hard time understanding/forming an opinion of it until I've gotten my hands dirty and actually tried to use it a little. Just reading about it in the abstract, it's hard to really form much of an opinion on it.
May 28 2009
prev sibling next sibling parent reply Tim Matthews <tim.matthews7 gmail.com> writes:
Jason House wrote:
 I'm really surprised by the lack of design discussion in this thread. It's
amazing how there can be huge bursts of discussion on which keyword to use
(e.g. manifest constants), but then complete silence about major design
decisions like thread safety that defines new transitive states and a bunch of
new keywords. The description even made parallels to the (previously?)
unpopular const architecture. 
 
 Maybe people are waiting for Walter to go through all the hard work of
implementing this stuff before complaining that it's crap and proclaiming
Walter should have done in the first place? 
 This seems really unfair to Walter. Then again, I see no indication of Walter
wanting anything else.
 
 

I have a few things I would like to discuss but I feel you are going to reply again with something like "do go there, the syntax is too dangerous for you" (you can really offend people with comments like that and you should get to know them first) I also feel you are going to keep top-replying unless someone tells you not to do so quit complaining and get to your points, design recommendations, ideas etc..
May 28 2009
parent Jason House <jason.james.house gmail.com> writes:
Tim Matthews Wrote:

 Jason House wrote:
 I'm really surprised by the lack of design discussion in this thread. It's
amazing how there can be huge bursts of discussion on which keyword to use
(e.g. manifest constants), but then complete silence about major design
decisions like thread safety that defines new transitive states and a bunch of
new keywords. The description even made parallels to the (previously?)
unpopular const architecture. 
 
 Maybe people are waiting for Walter to go through all the hard work of
implementing this stuff before complaining that it's crap and proclaiming
Walter should have done in the first place? 
 This seems really unfair to Walter. Then again, I see no indication of Walter
wanting anything else.
 
 

I have a few things I would like to discuss but I feel you are going to reply again with something like "do go there, the syntax is too dangerous for you" (you can really offend people with comments like that and you should get to know them first)

I won't bite your head off, or anyone else's. I'm sorry if a prior post on this NG came across that way. That wasn't my intent.
 I also feel you are going to keep top-replying unless someone tells you 
 not to do so quit complaining and get to your points, design 
 recommendations, ideas etc..

I top-posted because what I had to say had very little to do with the message I replied to. Many of my posts lately have aimed at trying to encourage collaboration. Maybe I'm going about it the wrong way. I know I'm a nobody, but I'm still trying in my own way to have a positive impact on D. So far, I think I'm just pissing people off.
May 28 2009
prev sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
Jason House, el 28 de mayo a las 08:45 me escribiste:
 I'm really surprised by the lack of design discussion in this thread.
 It's amazing how there can be huge bursts of discussion on which keyword
 to use (e.g. manifest constants), but then complete silence about major
 design decisions like thread safety that defines new transitive states
 and a bunch of new keywords. The description even made parallels to the
 (previously?) unpopular const architecture. 

I just find the new "thread-aware" design of D2 so complex, so twisted that I don't even know where to start. I think the solution is way worse than the problem here. That's why I don't comment at all. I think D duplicate functionality. For "safe" concurrency I use processes and IPC (I have even more guarantees that D could ever give me). That's all I need. I don't need a huge complexity in the language for that. And I think D2 concurrency model is still way too low level. I would like D2 better if it was focussed on macros for example.
 Maybe people are waiting for Walter to go through all the hard work of
 implementing this stuff before complaining that it's crap and
 proclaiming Walter should have done in the first place?

No, I don't see any point in saying what I said above, because I don't think anything will change. If I didn't like some little detail, that could worth discussing because it has any chance to change Walter/Bartoz mind, but saying "I think all the model is way too complex" don't help much IMHO =) -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ ---------------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ----------------------------------------------------------------------------
May 28 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Leandro Lucarella wrote:
 Jason House, el 28 de mayo a las 08:45 me escribiste:
 I'm really surprised by the lack of design discussion in this thread.
 It's amazing how there can be huge bursts of discussion on which keyword
 to use (e.g. manifest constants), but then complete silence about major
 design decisions like thread safety that defines new transitive states
 and a bunch of new keywords. The description even made parallels to the
 (previously?) unpopular const architecture. 

I just find the new "thread-aware" design of D2 so complex, so twisted that I don't even know where to start. I think the solution is way worse than the problem here. That's why I don't comment at all. I think D duplicate functionality. For "safe" concurrency I use processes and IPC (I have even more guarantees that D could ever give me). That's all I need. I don't need a huge complexity in the language for that. And I think D2 concurrency model is still way too low level. I would like D2 better if it was focussed on macros for example.
 Maybe people are waiting for Walter to go through all the hard work of
 implementing this stuff before complaining that it's crap and
 proclaiming Walter should have done in the first place?

No, I don't see any point in saying what I said above, because I don't think anything will change. If I didn't like some little detail, that could worth discussing because it has any chance to change Walter/Bartoz mind, but saying "I think all the model is way too complex" don't help much IMHO =)

On the contrary, we all (Bartosz, Walter, myself and probably other participants) think this would be valuable feedback. We'll always have some insecurity that we cut the pie the wrong way, and therefore we're continuously on lookout for well-argued positives or negatives. Those could lead to very useful quantitative discussions a la "X, Y, and Z together are way too complex, but X' and Z seems palatable and get 90% of the territory covered". I like Bartosz's design, it's sound (as far as I can tell) and puts the defaults in the right place so there's a nice pay-as-you-need quality to it. There are two details that are wanting. One is that I'm not sure we want high-level race-free so badly, we're prepared to pay that kind of price for it. Message passing is more likely to work well (better than lock-based concurrency) on contemporary and future processors. Then there's a design for solving low-level races that is much simpler and solves the nastiest part of the problem, so I wonder whether that would be more suitable. We also have immutable sharing that should help. Given this landscape, do we want to focus on high-level race elimination that badly? I'm not sure. Second, there is no regard to language integration. Bartosz says syntax doesn't matter and that he's flexible, but what that really means is that no attention has been paid to language integration. There is more to language integration than just syntax (and then even syntax is an important part of it). Andrei
May 28 2009
prev sibling next sibling parent reply BCS <none anon.com> writes:
Hello Leandro,

 Jason House, el 28 de mayo a las 08:45 me escribiste:
 
 I'm really surprised by the lack of design discussion in this thread.
 It's amazing how there can be huge bursts of discussion on which
 keyword to use (e.g. manifest constants), but then complete silence
 about major design decisions like thread safety that defines new
 transitive states and a bunch of new keywords. The description even
 made parallels to the (previously?) unpopular const architecture.
 

that I don't even know where to start. I think the solution is way worse than the problem here. That's why I don't comment at all.

I get the impression, from what little I known about threading, that it is likely you are under estimating the complexity of the threading problem. I get the feeling that *most* non-experts do (either that, or they just assume it more complex than they want to deal with).
 I think D duplicate functionality. For "safe" concurrency I use
 processes and IPC (I have even more guarantees that D could ever give
 me). That's all I need. I don't need a huge complexity in the language
 for that. And I think D2 concurrency model is still way too low level.

You are crazy! processes+IPC only works well if either the OS supports very fast IPC (IIRC none do aside from shared memory and now we are back where we started) or the processing between interaction is very long. Everything is indicating that shared memory multi-threading is where it's all going.
May 28 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
BCS wrote:
 Everything is indicating that shared memory multi-threading is where 
 it's all going.

That is correct, just that it's 40 years late. Right now everything is indicating that things are moving *away* from shared memory. Andrei
May 28 2009
next sibling parent reply BCS <ao pathlink.com> writes:
Reply to Andrei,

 BCS wrote:
 
 Everything is indicating that shared memory multi-threading is where
 it's all going.
 

indicating that things are moving *away* from shared memory. Andrei

I'm talking at the ASM level (not the language model level) and as opposed to each thread running in its own totally isolated address space. Am I wrong in assuming that most languages use user mode (not kernel mode) shared memory for inter thread communication?
May 28 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
BCS wrote:
 Reply to Andrei,
 
 BCS wrote:

 Everything is indicating that shared memory multi-threading is where
 it's all going.

indicating that things are moving *away* from shared memory. Andrei

I'm talking at the ASM level (not the language model level) and as opposed to each thread running in its own totally isolated address space. Am I wrong in assuming that most languages use user mode (not kernel mode) shared memory for inter thread communication?

What happens is that memory is less shared as cache hierarchies go deeper. It was a great model when there were a couple of processors hitting on the same memory because it was close to reality. Cache hierarchies reveal the hard reality that memory is shared to a decreasing extent and that each processor would rather deal with its own memory. Incidentally, message-passing-style protocols are prevalent in such architectures even at low level. It follows that message passing is not only an attractive model for programming at large, but also a model that's closer to machine than memory sharing. Andrei
May 28 2009
next sibling parent reply BCS <ao pathlink.com> writes:
Reply to Andrei,

 It follows that message passing is not only an attractive model

I'm thinking implementation not model. How is the message passing implemented? OS system calls (probably on top of kernel level shared memory)? user space shared memory? Special hardware? If you can't get #3, I'd rather a good standard implemntation using #2 (and a back door :).
 for programming at large, but also a model that's closer to
 machine than memory sharing.

I think I see what your getting at,.. even for shared memory on a deep cache; the cache invalidation system /is/ your message path.
 
 Andrei
 

May 28 2009
parent Sean Kelly <sean invisibleduck.org> writes:
BCS wrote:
 Reply to Andrei,
 
 It follows that message passing is not only an attractive model

I'm thinking implementation not model. How is the message passing implemented? OS system calls (probably on top of kernel level shared memory)? user space shared memory? Special hardware? If you can't get #3, I'd rather a good standard implemntation using #2 (and a back door :).

I think it depends on whether the message is intraprocess or interprocess. In the first case, I expect message passing would probably be done via user space shared memory if possible (things get a bit weird with per-thread heaps). In the latter case, a kernel api would probably be used if possible--perhaps TIPC or something related to MPI. It's the back door bit that's at issue right now. Should the language provide full explicit support for the intraprocess message passing? ie. move semantics, memory protection, etc?
 for programming at large, but also a model that's closer to
 machine than memory sharing.

I think I see what your getting at,.. even for shared memory on a deep cache; the cache invalidation system /is/ your message path.

Yeah kinda. Look at NUMA machines, for example (SPARC, etc). I expect that NUMA architectures will become increasingly common in the coming years, and it makes total sense to try and build a language that expects such a model.
May 29 2009
prev sibling next sibling parent Daniel Keep <daniel.keep.lists gmail.com> writes:
Andrei Alexandrescu wrote:
 BCS wrote:
 ...

 Am I wrong in assuming that most languages use user mode (not kernel
 mode) shared memory for inter thread communication?

What happens is that memory is less shared as cache hierarchies go deeper. It was a great model when there were a couple of processors hitting on the same memory because it was close to reality. Cache hierarchies reveal the hard reality that memory is shared to a decreasing extent and that each processor would rather deal with its own memory. Incidentally, message-passing-style protocols are prevalent in such architectures even at low level. It follows that message passing is not only an attractive model for programming at large, but also a model that's closer to machine than memory sharing.

This is all very interesting. I've recently been playing with a little toy language I'm designing. It's a postfix language, so I'm fairly certain no one will ever want to even look at it. :P But when I was designing it, I was adamant that it should do safe parallelism. I worked out that I could get everything other than deadlock safety by giving everything value semantics (using copy-on-write for anything larger than an atomic value.) Add in references that remember their "owner" thread and can only be dereferenced by that single thread, and then note that the global dict and stack are just values themselves and hence copied not referenced when you create a new thread. The only method of communication between threads is using message channels. This could be quite slow if you try to pass a very large data structure (since everything always gets copied), so you can create it on the heap via a reference, then "disown" the reference and assign it to another thread. That way you can get the efficiency of pass-by-reference without inter-thread aliasing. I also have a plan for making the language deadlock-free by either re-expressing all locks as blocking messages such that the interpreter knows who is blocking who, or by going all-out and just using transactions. But this is all just me stuffing about with a completely impractical language. What's being done for D is much more interesting. :)
May 28 2009
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2009-05-28 12:52:06 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:

 What happens is that memory is less shared as cache hierarchies go 
 deeper. It was a great model when there were a couple of processors 
 hitting on the same memory because it was close to reality. Cache 
 hierarchies reveal the hard reality that memory is shared to a 
 decreasing extent and that each processor would rather deal with its 
 own memory. Incidentally, message-passing-style protocols are prevalent 
 in such architectures even at low level. It follows that message 
 passing is not only an attractive model for programming at large, but 
 also a model that's closer to machine than memory sharing.

While message-passing might be useful for some applications, I have a hard time seeing how it could work for others. Try split processing of a 4 Gb array over 4 processors, or implement multi-threaded access to an in-memory database. Message passing by copying all the data might happen at the very low-level, but shared memory is more the right abstraction for these cases. There's a reason why various operating systems support shared memory between different processes: sometime it's easier to deal with shared memory than messaging, even with all the data races you have to deal with. Shared memory becoming more and more implemented as message passing at the very low level might indicate that some uses of shared memory will migrate to message passing at the application level and get some performance gains, but I don't think message passing will ever completely replace shared memory for dealing with large data sets. It's more likely that shared memory will become a scarse resource for some systems while it'll continue to grow for others. That said, I'm no expert in this domain. But I believe D should have good support both shared memory and message passing. I also take note that having a good shared memory model could prove very useful when writting on-disk databases or file systems, not just RAM-based data structures. You could have objects representing disk sectors or file segments, and the language's type system would help you handle the locking part of the equation. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
May 30 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Michel Fortin wrote:
 On 2009-05-28 12:52:06 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> said:
 
 What happens is that memory is less shared as cache hierarchies go 
 deeper. It was a great model when there were a couple of processors 
 hitting on the same memory because it was close to reality. Cache 
 hierarchies reveal the hard reality that memory is shared to a 
 decreasing extent and that each processor would rather deal with its 
 own memory. Incidentally, message-passing-style protocols are 
 prevalent in such architectures even at low level. It follows that 
 message passing is not only an attractive model for programming at 
 large, but also a model that's closer to machine than memory sharing.

While message-passing might be useful for some applications, I have a hard time seeing how it could work for others. Try split processing of a 4 Gb array over 4 processors, or implement multi-threaded access to an in-memory database. Message passing by copying all the data might happen at the very low-level, but shared memory is more the right abstraction for these cases.

Depends on what you want to do to those arrays. If concurrent writing is limited (e.g. quicksort) then there's no need to copy. Then many times you want to move (hand over) data from one thread to another. Here something like unique would help because you can safely pass pointers without worrying about subsequent contention.
 There's a reason why various operating systems support shared memory 
 between different processes: sometime it's easier to deal with shared 
 memory than messaging, even with all the data races you have to deal with.

Of course sometimes shared memory is a more natural fit. My argument is that that's the rare case.
 Shared memory becoming more and more implemented as message passing at 
 the very low level might indicate that some uses of shared memory will 
 migrate to message passing at the application level and get some 
 performance gains, but I don't think message passing will ever 
 completely replace shared memory for dealing with large data sets. It's 
 more likely that shared memory will become a scarse resource for some 
 systems while it'll continue to grow for others.
 
 That said, I'm no expert in this domain. But I believe D should have 
 good support both shared memory and message passing.
 
 I also take note that having a good shared memory model could prove very 
 useful when writting on-disk databases or file systems, not just 
 RAM-based data structures. You could have objects representing disk 
 sectors or file segments, and the language's type system would help you 
 handle the locking part of the equation.

I think shared files with interlocking can support such cases with ease. Andrei
May 30 2009
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2009-05-30 09:36:19 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:

 While message-passing might be useful for some applications, I have a 
 hard time seeing how it could work for others. Try split processing of 
 a 4 Gb array over 4 processors, or implement multi-threaded access to 
 an in-memory database. Message passing by copying all the data might 
 happen at the very low-level, but shared memory is more the right 
 abstraction for these cases.

Depends on what you want to do to those arrays. If concurrent writing is limited (e.g. quicksort) then there's no need to copy. Then many times you want to move (hand over) data from one thread to another. Here something like unique would help because you can safely pass pointers without worrying about subsequent contention.

If you include passing unique pointers to shared memory in your definition of "message passing", then yes, you can work with "message passing", and yes 'unique' would help a lot to ensure safety. But then you still need to have shared memory between threads: it's just that by making the pointer 'unique' we're ensuring that no more than one thread at a time is accessing that particular piece of data in the shared memory. I'm still convinced that we should offer a good way to access shared mutable data. It'll have to have limits to what the language can enforce and I'm not sure what they should be. For instance, while I see a need for 'lockfree', it's footprint in language complexity seems a little big for a half-unsafe manual performance enhancement capability, so I'm undecided on that one. Implicit synchronization of shared object member functions seems a good idea good however. I'll be waiting a little more to see what Bartosz has to say about expressing object ownership before making other comments.
 There's a reason why various operating systems support shared memory 
 between different processes: sometime it's easier to deal with shared 
 memory than messaging, even with all the data races you have to deal 
 with.

Of course sometimes shared memory is a more natural fit. My argument is that that's the rare case.

Shared memory is rare between processes, but not between threads. At least, in my experience. Shared memory is something you want to use whenever you can't afford copying data. Message passing is often implemented using shared memory, especially when between threads. That said, sometime you don't have the choice, you need to copy the data (to the GPU or to somewhere else on the network). Also, shared memory is something you want for storage systems you want to be accessible concurently by many threads like a database, a cache, a filesystem, etc. Those are shared storage systems and message passing doesn't help them really much since we're talking about storage here, not communication. Do you realy want to say that multithreaded access to stored data is rare? -- Michel Fortin michel.fortin michelf.com http://michelf.com/
May 30 2009
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
Michel Fortin wrote:
 On 2009-05-28 12:52:06 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> said:
 
 What happens is that memory is less shared as cache hierarchies go 
 deeper. It was a great model when there were a couple of processors 
 hitting on the same memory because it was close to reality. Cache 
 hierarchies reveal the hard reality that memory is shared to a 
 decreasing extent and that each processor would rather deal with its 
 own memory. Incidentally, message-passing-style protocols are 
 prevalent in such architectures even at low level. It follows that 
 message passing is not only an attractive model for programming at 
 large, but also a model that's closer to machine than memory sharing.

While message-passing might be useful for some applications, I have a hard time seeing how it could work for others. Try split processing of a 4 Gb array over 4 processors, or implement multi-threaded access to an in-memory database. Message passing by copying all the data might happen at the very low-level, but shared memory is more the right abstraction for these cases.

Perhaps at an implementation level in some instances, yes. But look at Folding home, etc. The approach is based on message passing. Just if you were Folding OnOnePCOnly then you might pass references to array regions around instead of copying the data. I suppose what I'm getting at is that an interface doesn't typically necessitate a particular implementation.
 There's a reason why various operating systems support shared memory 
 between different processes: sometime it's easier to deal with shared 
 memory than messaging, even with all the data races you have to deal with.
 
 Shared memory becoming more and more implemented as message passing at 
 the very low level might indicate that some uses of shared memory will 
 migrate to message passing at the application level and get some 
 performance gains, but I don't think message passing will ever 
 completely replace shared memory for dealing with large data sets. It's 
 more likely that shared memory will become a scarse resource for some 
 systems while it'll continue to grow for others.

Well sure. At some level, sharing is going to be happening even in message-passing oriented applications. The issue is more what the "encouraged" approach to solving problems that a language supports than what the language allows. D will always allow all sorts of wickedness because it's a systems language. But that doesn't mean this stuff has to be the central feature of the language.
May 30 2009
prev sibling parent reply Bartosz Milewski <bartosz-nospam relisoft.com> writes:
Andrei Alexandrescu Wrote:

 BCS wrote:
 Everything is indicating that shared memory multi-threading is where 
 it's all going.

That is correct, just that it's 40 years late. Right now everything is indicating that things are moving *away* from shared memory. Andrei

I understand where you stand. You are looking at where the state-of-the-art hardware is going and that makes perfect sense. There is, however, a co-evolution of hardware and software that is not a simple software-follow-hardware (remember the era of RISC processors?). I'm looking at programming languages and I don't see that away-from-shared-memory trend--neither in mainstream languages, nor in newer languages like Scala, nor in the operating systems. There are many interesting high-level paradigms like message passing, futures, actors, etc.; and I'm sure there will be more in the future. D has a choice of betting the store on one of these paradigms (like ML did on message passing or Erlang on actors) or to try, build solid foundations for a multi-paradigm language or do nothing. I am trying to build the foundations. The examples that I'm exploring in my posts are data structures that support higher level concurrency: channels, message queues, lock-free objects, etc. I want to build a system where high-level concurrency paradigms are reasonably easy to implement. Let's look at the alternatives. Nobody thinks seriously of making message passing a language feature in D. The Erlangization of D wouldn't work because D does not provide guarantees of address space separation (and providing such guarantees would cripple the language beyond repair). Another option we discussed was to provide specialized, well tested message queues in the library and count on programmers' discipline not to stray away from the message passing paradigm. Without type-system support, though, such queues would have to be either unsafe (no guarantee that the client doesn't have unprotected aliases to messages), or restrict messages to very simple data structures (pass-by-value and maybe some very clever unique pointer/array template). The latter approach introduces huge complexity into the library, essentially making user-defined extensions impossible (unless the user's name is Andrei ;-) ). Let's not forget that right now D is _designed_ to support shared-memory programming. Every D object has a lock, it supports synchronized methods, the keyword "shared" is being introduced, etc. It doesn't look like D is moving away from shared memory. It looks more like it's adding some window dressing to the pre-existing mess and bidding its time. I haven't seen a comprehensive plan for D to tackle concurrency and I'm afraid that if D doesn't take a solid stance right now, it will miss the train.
May 28 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bartosz Milewski wrote:
 Andrei Alexandrescu Wrote:
 
 BCS wrote:
 Everything is indicating that shared memory multi-threading is
 where it's all going.

is indicating that things are moving *away* from shared memory. Andrei

I understand where you stand. You are looking at where the state-of-the-art hardware is going and that makes perfect sense. There is, however, a co-evolution of hardware and software that is not a simple software-follow-hardware (remember the era of RISC processors?).

I understand that. However, I don't understand how the comment applies to the situation at hand.
 I'm looking at programming languages and I don't see that
 away-from-shared-memory trend--neither in mainstream languages, nor
 in newer languages like Scala, nor in the operating systems.

Scala doesn't know what to do about threads. The trend I'm seeing is that functional languages are getting increasing attention, and that's exactly because they never share mutable memory. As far as I can see, languages that are based on heavy shared mutation are pretty much dead in the water. We have the chance to not be so.
 There
 are many interesting high-level paradigms like message passing,
 futures, actors, etc.; and I'm sure there will be more in the future.
 D has a choice of betting the store on one of these paradigms (like
 ML did on message passing or Erlang on actors) or to try, build solid
 foundations for a multi-paradigm language or do nothing. I am trying
 to build the foundations.

Building foundations is great. What I'm seeing, however, is one very heavy strong pillar put in a place that might become the doghouse. I'm not at all sure the focus must be put on high-level race avoidance, particularly given that the cost in perceived complexity is this high.
 The examples that I'm exploring in my posts are data structures that
 support higher level concurrency: channels, message queues, lock-free
 objects, etc. I want to build a system where high-level concurrency
 paradigms are reasonably easy to implement.
 
 Let's look at the alternatives.
 
 Nobody thinks seriously of making message passing a language feature
 in D. The Erlangization of D wouldn't work because D does not provide
 guarantees of address space separation (and providing such guarantees
 would cripple the language beyond repair).

Why wouldn't we think of making message passing a language feature in D? Why does message passing need erlangization to support message passing?
 Another option we discussed was to provide specialized, well tested
 message queues in the library and count on programmers' discipline
 not to stray away from the message passing paradigm.

That's not what I discussed. I think there is is an interesting point that you've been missing, so please allow me to restate it. What I discussed was a holistic approach in which language + standard library provides a trusted computing base. Consider Java's new (for arrays) and C's malloc. The new function cannot be defined in Java because it would require unsafe manipulation underneath, so it is defined by its runtime support library. That runtime support is implemented in the likes of C. However, due to the fact that the runtime support is part of Java, in fact Java does have dynamic memory allocation - and nobody blinks an eye. C has famously had a mantra of self-sufficiency: its own support libraries have been written in C, which is pretty remarkable - and almost unprecedented at the time C was defined. For example, C's malloc is written in C, but (and here's an important detail) at some point it becomes _nonportable_ C. So even C has to cross a barrier of some sorts at some point. How is this related to the discussion at hand? You want to put all concurrency support in the language. That is, you want to put enough power into the language to be able to typecheck a variety of concurrent programming primitives and patterns. This approach is blindsided to the opportunity of defining some of these primitives in the standard library, in unsafe/unportable D, yet offering safe primitives to the user. In the process, the user is not hurt because she still has access to the primitives. What she can't do is define their own primitives in safe D. But I think that's as useless a pursuit as adding keywords to C to allow one to implement malloc() in safe C.
 Without
 type-system support, though, such queues would have to be either
 unsafe (no guarantee that the client doesn't have unprotected aliases
 to messages), or restrict messages to very simple data structures
 (pass-by-value and maybe some very clever unique pointer/array
 template). The latter approach introduces huge complexity into the
 library, essentially making user-defined extensions impossible
 (unless the user's name is Andrei ;-) ).

Complexity will be somewhere. The problem is, you want to put much of it in the language, and that will hit the language user too. Semantic checking will always be harder on everyone than a human being who sits down and implements a provably safe library in ways that the compiler can't prove.
 Let's not forget that right now D is _designed_ to support
 shared-memory programming. Every D object has a lock, it supports
 synchronized methods, the keyword "shared" is being introduced, etc.
 It doesn't look like D is moving away from shared memory. It looks
 more like it's adding some window dressing to the pre-existing mess
 and bidding its time. I haven't seen a comprehensive plan for D to
 tackle concurrency and I'm afraid that if D doesn't take a solid
 stance right now, it will miss the train.

I think we can safely ditch this argument. Walter had no idea what to do about threads when he defined D, so he put whatever he saw and understood in Java. He'd be the first to say that - actually, he wouldn't tell me so, he _told_ me so. To me, adding concurrency capabilities to D is nothing like adding window dressing on top of whatever crap is there. Java and C++ are in trouble, and doing what they do doesn't strike me as a good bet. You're right about missing the train, but I think you and I are talking about different trains. I don't want to embark on the steam-powered train. Andrei
May 28 2009
next sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el 28 de mayo a las 19:52 me escribiste:
 To me, adding concurrency capabilities to D is nothing like adding window 
 dressing on top of whatever crap is there. Java and C++ are in trouble, and 
 doing what they do doesn't strike me as a good bet. You're right about missing 
 the train, but I think you and I are talking about different trains. I don't 
 want to embark on the steam-powered train.

I agree. Maybe is just unjustified fear, but I see D2 being to concurrency what C++ was to templates. Great new idea, terrible hard to use and understand for mortals. For some time people used to think they were complex because they had to, but I think D could prove that wrong ;) -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ ---------------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ----------------------------------------------------------------------------
May 29 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Leandro Lucarella:
 I agree. Maybe is just unjustified fear, but I see D2 being to concurrency
 what C++ was to templates.

Sometimes you need lot of time to find what a simple implementation can be. Often someone has to pay the price of being the first one to implement something :-] This is bad if you mix it with the will of keeping backwards compatibility. Bye, bearophile
May 29 2009
parent reply Leandro Lucarella <llucax gmail.com> writes:
bearophile, el 29 de mayo a las 13:39 me escribiste:
 Leandro Lucarella:
 I agree. Maybe is just unjustified fear, but I see D2 being to concurrency
 what C++ was to templates.

Sometimes you need lot of time to find what a simple implementation can be. Often someone has to pay the price of being the first one to implement something :-] This is bad if you mix it with the will of keeping backwards compatibility.

Exactly. I think D had a good model of "steal good proven stuff that other languages got right". With this, I thinks it's taking a new path of being a pioneer, and chances are it get it wrong (I don't mean to be offensive with this, I'm just speaking statistically) and suffer the mistake for a long long time because of backward compatibility. -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ ---------------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ----------------------------------------------------------------------------
May 29 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Leandro Lucarella wrote:
 bearophile, el 29 de mayo a las 13:39 me escribiste:
 Leandro Lucarella:
 I agree. Maybe is just unjustified fear, but I see D2 being to concurrency
 what C++ was to templates.

be. Often someone has to pay the price of being the first one to implement something :-] This is bad if you mix it with the will of keeping backwards compatibility.

Exactly. I think D had a good model of "steal good proven stuff that other languages got right". With this, I thinks it's taking a new path of being a pioneer, and chances are it get it wrong (I don't mean to be offensive with this, I'm just speaking statistically) and suffer the mistake for a long long time because of backward compatibility.

With its staunch default isolation, I think D is already making a departure from the traditional imperative languages (which extend their imperative approach to concurrent programming). The difference is that it takes what I think is a sound model (interprocess isolation) and augment it with the likes of shared and Bartosz's work. So my perception is that it's less likely to get things regrettably wrong. But then you never know. Andrei
May 29 2009
prev sibling parent reply Bartosz Milewski <bartosz-nospam relisoft.com> writes:
Andrei Alexandrescu Wrote:

 Scala doesn't know what to do about threads. 

That's my impression too, although Scala's support for actors leaves D in the dust.
 The trend I'm seeing is 
 that functional languages are getting increasing attention, and that's 
 exactly because they never share mutable memory. As far as I can see, 
 languages that are based on heavy shared mutation are pretty much dead 
 in the water. We have the chance to not be so.
 

It's a very sweeping statement. I just looked at the TIOBE index and couldn't find _any_ functional languages in the top 20.
 I'm 
 not at all sure the focus must be put on high-level race avoidance, 
 particularly given that the cost in perceived complexity is this high.

The complexity argument is tenuous. It might look like my proposal is complex because I'm dropping the whole system in at once. But it's a solid, well thought-out system. What I see happening in D is the creeping complexity resulting from sloppy design. We've been talking for years about closing some gaping holes in the design of arrays, slices, immutable, qualifier polymorphism--the list goes on--and there's little progress. There is no solid semantics for scope and shared. A solid solution to those issues will look complex too.
 Why wouldn't we think of making message passing a language feature in D? 

Because we don't have even a tiniest proposal describing it, not to mention a design.
 Why does message passing need erlangization to support message passing?

Because the strength of the Erlang model is the isolation of processes. Take away isolation and it's no better than Scala or Java. Granted, having to explicitly mark objects for sharing in D is a big help. Here we agree.
 What I discussed was a holistic approach in which language + standard 
 library provides a trusted computing base. 

Have you thought about how to eliminate data races in your holistic approach? Will "shared" be forbidden in SafeD? Will library-based message-passing channels (and actors?) only accept simple value types and immutables? Andrei, you are the grand wizard of squeezing powerful abstractions out of a kludge of a language that is C++ with its ad-hoc support for generics. It's impressive and very useful, but it's also hermetic. By contrast, generic programming in D is relatively easy because of the right kind of support built into the language (compile-time interpreter). I trust that you could squeeze powerful multithreading abstraction out of D, even if the language/type system doesn't offer much of a support. But it will be hermetic. Prove me wrong by implementing a message queue using the current D2 (plus some things that are still in the pipeline).
 How is this related to the discussion at hand? You want to put all 
 concurrency support in the language. That is, you want to put enough 
 power into the language to be able to typecheck a variety of concurrent 
 programming primitives and patterns. This approach is blindsided to the 
 opportunity of defining some of these primitives in the standard 
 library, in unsafe/unportable D, yet offering safe primitives to the 
 user. In the process, the user is not hurt because she still has access 
 to the primitives. What she can't do is define their own primitives in 
 safe D. But I think that's as useless a pursuit as adding keywords to C 
 to allow one to implement malloc() in safe C.
 

That's a bad analogy. I'm proposing the tightening of the type system, not the implementation of weak atomics. A better analogy would be adding immutable/const to the language. Except that I don't think const-correctness is as important as the safety of shared-memory concurrency.
 Without
 type-system support, though, such queues would have to be either
 unsafe (no guarantee that the client doesn't have unprotected aliases
 to messages), or restrict messages to very simple data structures
 (pass-by-value and maybe some very clever unique pointer/array
 template). The latter approach introduces huge complexity into the
 library, essentially making user-defined extensions impossible
 (unless the user's name is Andrei ;-) ).

Complexity will be somewhere. The problem is, you want to put much of it in the language, and that will hit the language user too.

I'm being very careful not to hit the language user. You might have noticed that my primitive channel, the MVar, is less complex than a D2 implementation (it doesn't require "synchronized"). And the compiler will immediately tell you if you use it in an unsafe way.
 Semantic 
 checking will always be harder on everyone than a human being who sits 
 down and implements a provably safe library in ways that the compiler 
 can't prove.
 

Be careful with such arguments. Somebody might use them to discredit immutability.
 Let's not forget that right now D is _designed_ to support
 shared-memory programming. Every D object has a lock, it supports
 synchronized methods, the keyword "shared" is being introduced, etc.
 It doesn't look like D is moving away from shared memory. It looks
 more like it's adding some window dressing to the pre-existing mess
 and bidding its time. I haven't seen a comprehensive plan for D to
 tackle concurrency and I'm afraid that if D doesn't take a solid
 stance right now, it will miss the train.

I think we can safely ditch this argument. Walter had no idea what to do about threads when he defined D, so he put whatever he saw and understood in Java. He'd be the first to say that - actually, he wouldn't tell me so, he _told_ me so.

I realize that. Except that the "shared" concept was added very recently.
 To me, adding concurrency capabilities to D is nothing like adding 
 window dressing on top of whatever crap is there. Java and C++ are in 
 trouble, and doing what they do doesn't strike me as a good bet. 

So far D has been doing exactly what Java and C++ are doing. My proposal goes way beyond that. But if you mean D should not support shared-memory concurrency or give it only lip service, than you really have to come up with something revolutionary to take its place. This would obviously not make it into D2 or into your book. So essentially D2 would doomed in the concurrency department.
 You're 
 right about missing the train, but I think you and I are talking about 
 different trains. I don't want to embark on the steam-powered train.

Should we embark on a vapor-powered train then ;-)
May 29 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bartosz Milewski wrote:
 Andrei Alexandrescu Wrote:
 Scala doesn't know what to do about threads.

That's my impression too, although Scala's support for actors leaves D in the dust.

Scala actors are a library.
 The trend I'm seeing is that functional languages are getting
 increasing attention, and that's exactly because they never share
 mutable memory. As far as I can see, languages that are based on
 heavy shared mutation are pretty much dead in the water. We have
 the chance to not be so.
 

It's a very sweeping statement. I just looked at the TIOBE index and couldn't find _any_ functional languages in the top 20.

We can safely ditch Tiobe, but I agree that functional languages aren't mainstream. There are two trends though. One is that for most of its existence Haskell has had about 12 users. It has definitely turned an exponential elbow during the recent years. Similar trends are to be seen for ML, Ocaml, and friends. The other trend is that all of today's languages are scrambling to add support for pure functional programming.
 I'm not at all sure the focus must be put on high-level race
 avoidance, particularly given that the cost in perceived complexity
 is this high.

The complexity argument is tenuous. It might look like my proposal is complex because I'm dropping the whole system in at once. But it's a solid, well thought-out system. What I see happening in D is the creeping complexity resulting from sloppy design. We've been talking for years about closing some gaping holes in the design of arrays, slices, immutable, qualifier polymorphism--the list goes on--and there's little progress. There is no solid semantics for scope and shared. A solid solution to those issues will look complex too.

What do those holes have to do with the problem at hand? I'm seeing implementation bugs, not design holes. I'd love them to be fixed as much as the next guy, but I don't think we're looking at issues that would be complex (except for scope which sucks; I never claimed there was a solution to that). All of other features you mention have no holes I know of in their design.
 Why wouldn't we think of making message passing a language feature
 in D?

Because we don't have even a tiniest proposal describing it, not to mention a design.

That doesn't mean we shouldn't think of it.
 Why does message passing need erlangization to support message
 passing?

Because the strength of the Erlang model is the isolation of processes. Take away isolation and it's no better than Scala or Java. Granted, having to explicitly mark objects for sharing in D is a big help. Here we agree.

So we should be thinking about it, right?
 What I discussed was a holistic approach in which language +
 standard library provides a trusted computing base.

Have you thought about how to eliminate data races in your holistic approach? Will "shared" be forbidden in SafeD? Will library-based message-passing channels (and actors?) only accept simple value types and immutables?

Shared will be allowed in SafeD but it will be the responsibility of user code to ensure high-level race elimination. Shared will eliminate low-level races. I think it's worth contemplating a scenario in which message passing is restricted to certain types.
 Andrei, you are the grand wizard of squeezing powerful abstractions
 out of a kludge of a language that is C++ with its ad-hoc support for
 generics. It's impressive and very useful, but it's also hermetic. By
 contrast, generic programming in D is relatively easy because of the
 right kind of support built into the language (compile-time
 interpreter).

Please no ad hominem, flattering or not.
 I trust that you could squeeze powerful multithreading abstraction
 out of D, even if the language/type system doesn't offer much of a
 support. But it will be hermetic. Prove me wrong by implementing a
 message queue using the current D2 (plus some things that are still
 in the pipeline).

I don't have the time, but I think it's worth looking into what a message queue implementation should look like.
 How is this related to the discussion at hand? You want to put all
  concurrency support in the language. That is, you want to put
 enough power into the language to be able to typecheck a variety of
 concurrent programming primitives and patterns. This approach is
 blindsided to the opportunity of defining some of these primitives
 in the standard library, in unsafe/unportable D, yet offering safe
 primitives to the user. In the process, the user is not hurt
 because she still has access to the primitives. What she can't do
 is define their own primitives in safe D. But I think that's as
 useless a pursuit as adding keywords to C to allow one to implement
 malloc() in safe C.
 

That's a bad analogy. I'm proposing the tightening of the type system, not the implementation of weak atomics. A better analogy would be adding immutable/const to the language. Except that I don't think const-correctness is as important as the safety of shared-memory concurrency.

It's a good analogy because you want to make possible the implementation of threading primitives in portable D. I am debating whether that is a worthy goal. I doubt it is.
 Without type-system support, though, such queues would have to be
 either unsafe (no guarantee that the client doesn't have
 unprotected aliases to messages), or restrict messages to very
 simple data structures (pass-by-value and maybe some very clever
 unique pointer/array template). The latter approach introduces
 huge complexity into the library, essentially making user-defined
 extensions impossible (unless the user's name is Andrei ;-) ).

of it in the language, and that will hit the language user too.

I'm being very careful not to hit the language user. You might have noticed that my primitive channel, the MVar, is less complex than a D2 implementation (it doesn't require "synchronized"). And the compiler will immediately tell you if you use it in an unsafe way.
 Semantic checking will always be harder on everyone than a human
 being who sits down and implements a provably safe library in ways
 that the compiler can't prove.
 

Be careful with such arguments. Somebody might use them to discredit immutability.

Let them discredit it and we'll see how strong their argument is.
 Let's not forget that right now D is _designed_ to support 
 shared-memory programming. Every D object has a lock, it supports
  synchronized methods, the keyword "shared" is being introduced,
 etc. It doesn't look like D is moving away from shared memory. It
 looks more like it's adding some window dressing to the
 pre-existing mess and bidding its time. I haven't seen a
 comprehensive plan for D to tackle concurrency and I'm afraid
 that if D doesn't take a solid stance right now, it will miss the
 train.

to do about threads when he defined D, so he put whatever he saw and understood in Java. He'd be the first to say that - actually, he wouldn't tell me so, he _told_ me so.

I realize that. Except that the "shared" concept was added very recently.
 To me, adding concurrency capabilities to D is nothing like adding
  window dressing on top of whatever crap is there. Java and C++ are
 in trouble, and doing what they do doesn't strike me as a good bet.
 

So far D has been doing exactly what Java and C++ are doing. My proposal goes way beyond that.

My problem is that I think it goes way beyond that straight in the wrong directions. It goes on and on and on about how to make deadlock-oriented programming less susceptible to races. I don't care about deadlock-oriented programming. I want to stay away from deadlock-oriented programming. I don't understand why I need a hecatomb of concepts and notions that help me continue using a programming style that is unrecommended.
 But if you mean D should not support
 shared-memory concurrency or give it only lip service, than you
 really have to come up with something revolutionary to take its
 place. This would obviously not make it into D2 or into your book. So
 essentially D2 would doomed in the concurrency department.

Message passing and functional style have been around for a while and form an increasingly compelling alternative to mutable sharing. We can support deadlock-oriented programming in addition to these for those who want it, but I don't think it's an area where we need to pay an arm and a leg for eliminating high-level races. I just think it's the wrong problem to work on. Andrei
May 29 2009
next sibling parent reply Bartosz Milewski <bartosz-nospam relisoft.com> writes:
Can you believe it? I was convinced that my response was lost because the
stupid news reader on Digital Mars web site returned an error (twice, hence two
posts). I diligently rewrote the riposte from scratch and tried to post it. It
flunked again! Now I'm not sure if it won't appear in the newsgroup after an
hour. (By the way, I refined my arguments.)
May 29 2009
parent reply Bartosz Milewski <bartosz-nospam relisoft.com> writes:
This is the missing second reply to Andrei. I'm posting parts of it because it
my help understand my position better.

I wouldn't dismiss Scala out of hand. The main threading model in Scala is
(library-supported) actor model. Isn't that what you're proposing for D? Except
that Scala has much better support for functional programming.

The languages [C++ and Java] may be dead in the water (altough still the
overwhelming majority of programmers use them), but I don't see the idea of
shared-memory concurrency dying any time soon. I bet it will be the major
programming model for the next ten years. What will come after that, nobody
knows.

The complexity argument:  My proposal looks complex because I am dropping the
whole comprehensive solution on the D community all at once. I would be much
wearier of the kind of creeping complexity resulting from incremental ad-hoc
solutions. For instance, the whole complexity of immutability hasn't been
exposed yet. If it were, there would be a much larger insurgency among D users.
You know what I'm talking about--invariant constructors. My proposal goes into
nooks and crannies and, of course, it makes it look more complex than it really
is. Not to mention that there could be a lot of ideas that would lead to
simplifications. I sometimes discuss various options for discussion.

Take my proposal for unique objects. I could have punted the need for "lent".
Maybe nobody would ask for it? Compare "unique" with "scope"--nobody knows the
target semantics of "scope". It's a half-baked idea, but nobody's protesting. 

Try to define the semantics of array slices and you'll see eyes glazing. We
know we have to fix them, but we don't know how (array[new]?). Another
half-baked idea. Are slices simple or complex? 

Define the semantics of "shared". Or should we implement it first and hope that
the complexity won't creep in when we discover its shortcomings. 

 Why wouldn't we think of making message passing a language feature in D? 

Well, we could, but why? We don't have to add any new primitives to the language to implement message queues. We would have to *eliminate* some features to make message passing safe. For instance, we'd have to eliminate "shared". Is that an option?
 Why does message passing need erlangization to support message passing?

The power of Erlang lies in guaranteed process isolation. If we don't guarantee that, we are in the same league as Java or C++.
 What I discussed was a holistic approach in which language + standard 
 library provides a trusted computing base. 

I like that very much. But I see the library as enabling certain useful features, while the type system as disabling the dangerous ones. You can't disable features through a library.
May 30 2009
next sibling parent grauzone <none example.net> writes:
 For instance, the whole complexity of immutability hasn't been exposed yet.

What? I thought immutable was already quite complex.
 Compare "unique" with "scope"--nobody knows the target semantics of "scope".
It's a half-baked idea, but nobody's protesting. 

Everyone knows that D is full of half-baked ideas. We're not using D because it's a beautiful or elegant language - we use it because it makes life easier. Slices and arrays are half-baked, but they are much simpler and easier to use than corresponding C/C++ solutions. We're also using D because it's so C/C++ like. D is to C what C++ should have been to C. Other than that, there are already languages, which could have taken D's job: Delphi-Pascal, Ada, Modula... If D stops making life easier, it will be the death of D.
May 30 2009
prev sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2009-05-30 13:00:14 -0400, Bartosz Milewski 
<bartosz-nospam relisoft.com> said:

 The complexity argument:  My proposal looks complex because I am 
 dropping the whole comprehensive solution on the D community all at 
 once. I would be much wearier of the kind of creeping complexity 
 resulting from incremental ad-hoc solutions. For instance, the whole 
 complexity of immutability hasn't been exposed yet. If it were, there 
 would be a much larger insurgency among D users. You know what I'm 
 talking about--invariant constructors. My proposal goes into nooks and 
 crannies and, of course, it makes it look more complex than it really 
 is. Not to mention that there could be a lot of ideas that would lead 
 to simplifications. I sometimes discuss various options for discussion.
 
 Take my proposal for unique objects. I could have punted the need for 
 "lent". Maybe nobody would ask for it? Compare "unique" with 
 "scope"--nobody knows the target semantics of "scope". It's a 
 half-baked idea, but nobody's protesting.
 
 Try to define the semantics of array slices and you'll see eyes 
 glazing. We know we have to fix them, but we don't know how 
 (array[new]?). Another half-baked idea. Are slices simple or complex?

Bartosz, you're arguing that your proposal isn't that complex compared to strange semantics of other parts of the language and I agree... I should even say that what you propose offer a solution for fixing these other parts of the language. It's funny how what's needed to make multithreading safe is pretty much the same as what is needed to make safe immutable constructors and safe array slices, let's take a look: A constructor for a unique object is all you need to build an immutable one: move the unique pointer to an immutable pointer and you're sure no one has a mutable pointer to it. Of course, to implement unique constructors, you need 'lent' (or 'scope', whatever our prefered keyword) so you can call functions that will alter the unique object and its member without escaping a reference. As for slices, as long as your slice is 'unique', you can enlarge it without side effects (relocating the slice in memory won't affect any other slice because you're guarentied there aren't any), making a 'unique T[]' as good as an equivalent container... or should I say even safer since enlarging a non-unique container might be as bad as enlarging a slice (the container may realocate and disconnect from all its slices). You could also later transform 'unique T[]' to 'immutable T[]', or to a mutable 'T[]', but then you shouldn't be able to grow it without making a duplicate first to avoid undesirable side effects. So instead of fighting over what's too complex and what isn't by looking at each hole of the language in isolation, I think it's time to look at the various problems as a whole. I believe all those half-baked ideas point to the same underlying deficiency: the lack of a safe unique type (which then requires move semantics and 'lent'/'scope' constrains). C++ will get half of that soon (unique_ptr) but it will still be missing 'lent' though so it won't be so safe. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
May 30 2009
prev sibling next sibling parent reply Bartosz Milewski <bartosz-nospam relisoft.com> writes:
I don't think the item-by-item pingpong works well in the newsgroup. Let's
separate our discussion into separate threads. One philosophical, about the
future of concurrency. Another about the immediate future of concurrency in D2.
And a separate one about my proposed system in the parallel universe where we
all agree that for the next 10 years shared-memory concurrency will be the
dominating paradigm. 
May 29 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bartosz Milewski wrote:
 I don't think the item-by-item pingpong works well in the newsgroup.
 Let's separate our discussion into separate threads. One
 philosophical, about the future of concurrency. Another about the
 immediate future of concurrency in D2. And a separate one about my
 proposed system in the parallel universe where we all agree that for
 the next 10 years shared-memory concurrency will be the dominating
 paradigm.

I'm sure it's a good idea, particularly if others will participate as well. I warn that I'll be at a conference Sat-Thu and I don't have much time even apart from that. Andrei
May 29 2009
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:
I just think it's the wrong problem to work on.<

Beside multiprocessing (that I am ignorant to comment on still), I can see other purposes for having a way to tell the type system that it exists only one reference/pointer to mutable data and ways to safely change ownership of such pointer. It can also be used by the compiler to optimize in various situations. It can be good when the type system gives you a formal way to state a constraint that the programmer wants to put in the program anyway (often just stated in comments, if the language doesn't allow such higher level feature). For example a type system can give a big help avoiding using null object references in a program, saving lot of time of the programmer (eventually D will need this feature), in such situations a more refined type system reduces the time/complexity to write correct programs (and in 20-lines long programs you may just don't use such features). Bye, bearophile
May 30 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
bearophile wrote:
 Andrei Alexandrescu:
 I just think it's the wrong problem to work on.<

Beside multiprocessing (that I am ignorant to comment on still), I can see other purposes for having a way to tell the type system that it exists only one reference/pointer to mutable data and ways to safely change ownership of such pointer. It can also be used by the compiler to optimize in various situations.

Correct. We've been trying valiantly to introduce unique in the type system two times, first time in 2007. Our conclusion back then has been that unique brings more problems than it solves. Last meeting the same pattern ensued. Problems with unique cropped up faster than those solved. Andrei
May 30 2009
prev sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
BCS, el 28 de mayo a las 15:57 me escribiste:
 Hello Leandro,
 
Jason House, el 28 de mayo a las 08:45 me escribiste:
I'm really surprised by the lack of design discussion in this thread.
It's amazing how there can be huge bursts of discussion on which
keyword to use (e.g. manifest constants), but then complete silence
about major design decisions like thread safety that defines new
transitive states and a bunch of new keywords. The description even
made parallels to the (previously?) unpopular const architecture.

that I don't even know where to start. I think the solution is way worse than the problem here. That's why I don't comment at all.

I get the impression, from what little I known about threading, that it is likely you are under estimating the complexity of the threading problem. I get the feeling that *most* non-experts do (either that, or they just assume it more complex than they want to deal with).

I guess all depends on the kind of fine granularity you want. I work on a distributed application, so threading is not very tempting for me, I get to use the multi-cores by splitting the work among processes, not threads, because I need "location-transparency" (I don't care if the process I'm communicating with runs in the same computer or not, so things like "move semantics" are not interesting for me). Sometimes I need some threading support, for example to be able to receive queries and do some I/O intensive stuff in the same thread, but the thread-communication I need for that is so trivial, using simple mutexes works just fine. And I never needed so much performance to think about lock-free communication either (mutexes are really fast in Linux). I guess threading complexity is proportional to the complexity of the design. If your design is simple, concurrency is simple. Is really hard to get a deadlock in a simple design. Races are a little trickier, but they are inherent to some kind of problems, so there is not a lot to do about that, you have to see problem by problem and try to get a good design to handle them well.
I think D duplicate functionality. For "safe" concurrency I use
processes and IPC (I have even more guarantees that D could ever give
me). That's all I need. I don't need a huge complexity in the language
for that. And I think D2 concurrency model is still way too low level.

You are crazy! processes+IPC only works well if either the OS supports very fast IPC (IIRC none do aside from shared memory and now we are back where we started) or the processing between interaction is very long.

The later is my case =)
 Everything is indicating that shared memory multi-threading is where
 it's all going.

Maybe, I'm just saying why I don't comment on D2 concurrency model. I find it too complex for my needs (i.e. for what I know, I won't give my opinion about things I don't know/use). -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ ---------------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ----------------------------------------------------------------------------
May 28 2009
parent reply Bartosz Milewski <bartosz-nospam relisoft.com> writes:
Leandro Lucarella Wrote:

 BCS, el 28 de mayo a las 15:57 me escribiste:

 
 Maybe, I'm just saying why I don't comment on D2 concurrency model. I find
 it too complex for my needs (i.e. for what I know, I won't give my opinion
 about things I don't know/use).
 

Probably the majority of users either don't use multithreading (yet) or use it only for very simple tasks. My stated goal is not to force such users to learn the whole race-free type system. In most cases things "just work" by default, and the compiler catches any accidental race conditions. The complex part is for library writers who have very demanding needs. Unfortunately, I have to describe the whole shebang in my blog, otherwise people won't believe that the system is workable and that it satisfies their high expectations. I should probably write a simple tutorial that would show how to use my system for simple tasks.
May 28 2009
next sibling parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Bartosz Milewski (bartosz-nospam relisoft.com)'s article
 Leandro Lucarella Wrote:
 BCS, el 28 de mayo a las 15:57 me escribiste:

 Maybe, I'm just saying why I don't comment on D2 concurrency model. I find
 it too complex for my needs (i.e. for what I know, I won't give my opinion
 about things I don't know/use).


whole race-free type system. In most cases things "just work" by default, and the compiler catches any accidental race conditions.
 The complex part is for library writers who have very demanding needs.

won't believe that the system is workable and that it satisfies their high expectations.
 I should probably write a simple tutorial that would show how to use my system

This would be much appreciated. I try to read your blogs, which are geared toward hardcore multithreading people. I know just enough about multithreading to understand why it's a hard problem, so I usually get to about the second paragraph before I feel lost. I would love to see a version that offers simple examples of how the new multithreading might be useful to the kinds of people (like me) who understand the basics of multithreading and write multithreaded code in the very simple cases, but are not experts in concurrency, etc. For my purposes, I'm more interested in the mildly complicated things that are made simple, not the highly complicated things that are made possible.
May 28 2009
prev sibling parent reply Jason House <jason.james.house gmail.com> writes:
Bartosz Milewski Wrote:

 Leandro Lucarella Wrote:
 
 BCS, el 28 de mayo a las 15:57 me escribiste:

 
 Maybe, I'm just saying why I don't comment on D2 concurrency model. I find
 it too complex for my needs (i.e. for what I know, I won't give my opinion
 about things I don't know/use).
 

Probably the majority of users either don't use multithreading (yet) or use it only for very simple tasks. My stated goal is not to force such users to learn the whole race-free type system. In most cases things "just work" by default, and the compiler catches any accidental race conditions.

My hobby project is a multi-threaded game-playing AI. My current scheme uses a shared search tree using lockless updates with search results. Besides general ability to use your scheme for what I've already done, I'm also interested in how to overhaul the garbage collector and implementing lockless hashtables (see high-scale-lib on sf.net) I was interested in doing some of that infrastructure and contributing, but so far I've had no luck getting something as simple as a weak references into druntime :(
 The complex part is for library writers who have very demanding needs.
Unfortunately, I have to describe the whole shebang in my blog, otherwise
people won't believe that the system is workable and that it satisfies their
high expectations. 

Yeah, I'm waiting for more details like which fences are introduced by the lockless SC requirements. The high-scale-lib is virtually fence free.
 I should probably write a simple tutorial that would show how to use my system
for simple tasks. 
 

May 28 2009
next sibling parent reply Bartosz Milewski <bartosz-nospam relisoft.com> writes:
Jason House Wrote:

 Bartosz Milewski Wrote:
 
 My hobby project is a multi-threaded game-playing AI. My current scheme uses a
shared search tree using lockless updates with search results. Besides general
ability to use your scheme for what I've already done, I'm also interested in
how to overhaul the garbage collector and implementing lockless hashtables (see
high-scale-lib on sf.net)
 

I see, you're a hardcore lockfree programmer. All you can expect from D is Sequential Consistency--nothing fancy like C++ weak atomics. But that's for the better.
 The complex part is for library writers who have very demanding needs.
Unfortunately, I have to describe the whole shebang in my blog, otherwise
people won't believe that the system is workable and that it satisfies their
high expectations. 

Yeah, I'm waiting for more details like which fences are introduced by the lockless SC requirements. The high-scale-lib is virtually fence free.

I don't have much to say about that because it's a know problem and it has already been solved in Java. I can tell you what is required on an x86: use xchg for writes, and that's all. I think Walter has already implemented it, because he asked me the same question.
May 28 2009
parent reply Jason House <jason.james.house gmail.com> writes:
Bartosz Milewski Wrote:

 Jason House Wrote:
 
 Bartosz Milewski Wrote:
 
 My hobby project is a multi-threaded game-playing AI. My current scheme uses a
shared search tree using lockless updates with search results. Besides general
ability to use your scheme for what I've already done, I'm also interested in
how to overhaul the garbage collector and implementing lockless hashtables (see
high-scale-lib on sf.net)
 

I see, you're a hardcore lockfree programmer. All you can expect from D is Sequential Consistency--nothing fancy like C++ weak atomics. But that's for the better.

Far from it! I'm stumbling through in an attempt to teach myself the black art. I'm probably in my 3rd coding of the project. The first incarnation had no threads. The 2nd used message passing. The current one is lockless, but still a work in progress.
 The complex part is for library writers who have very demanding needs.
Unfortunately, I have to describe the whole shebang in my blog, otherwise
people won't believe that the system is workable and that it satisfies their
high expectations. 

Yeah, I'm waiting for more details like which fences are introduced by the lockless SC requirements. The high-scale-lib is virtually fence free.

I don't have much to say about that because it's a know problem and it has already been solved in Java. I can tell you what is required on an x86: use xchg for writes, and that's all. I think Walter has already implemented it, because he asked me the same question.

What about cmpchx (AKA compare and swap). It occurs in a lot of algorithms. Also, "lock inc" is fundamental to my use of lockless variables.
May 28 2009
parent Bartosz Milewski <bartosz-nospam relisoft.com> writes:
Jason House Wrote:


 I see, you're a hardcore lockfree programmer. All you can expect from D is
Sequential Consistency--nothing fancy like C++ weak atomics. But that's for the
better.

Far from it! I'm stumbling through in an attempt to teach myself the black art. I'm probably in my 3rd coding of the project. The first incarnation had no threads. The 2nd used message passing. The current one is lockless, but still a work in progress.

 What about cmpchx (AKA compare and swap). It occurs in a lot of algorithms.
Also, "lock inc" is fundamental to my use of lockless variables.

These will either be implemented in the library (inline assembly) or as compiler intrinsic. It's not hard.
May 30 2009
prev sibling parent reply BCS <ao pathlink.com> writes:
Reply to Jason,

 My hobby project is a multi-threaded game-playing AI. My current
 scheme uses a shared search tree using lockless updates with search
 results.
 

As in threaded min-max? Have you got anything working? I known from experience that this ones a cast iron SOB. http://arrayboundserror.blogspot.com/search/label/min%20max
May 28 2009
parent Jason House <jason.james.house gmail.com> writes:
BCS Wrote:

 Reply to Jason,
 
 My hobby project is a multi-threaded game-playing AI. My current
 scheme uses a shared search tree using lockless updates with search
 results.
 

As in threaded min-max? Have you got anything working? I known from experience that this ones a cast iron SOB. http://arrayboundserror.blogspot.com/search/label/min%20max

No. Min-max is only good for theory. I'm also not doing alpha-beta which is successful in chess. I'm doing UCT and mostly aim to play the game of "go". UCT uses statistical bounds instead of hard heuristics. It also has less uniform tree exploration.
May 28 2009
prev sibling next sibling parent reply Tim Matthews <tim.matthews7 gmail.com> writes:
Leandro Lucarella wrote:

 I would like D2 better if it was focussed on macros for example.
 

Can you elaborate on this? I think of the word macro as a C preprocessor feature which is no longer needed in D.
May 28 2009
next sibling parent BCS <none anon.com> writes:
Hello Tim,

 Leandro Lucarella wrote:
 
 I would like D2 better if it was focussed on macros for example.
 

preprocessor feature which is no longer needed in D.

AST macros. Look up Walter et al's talk from the D conference
May 28 2009
prev sibling parent Tim Matthews <tim.matthews7 gmail.com> writes:
Denis Koroskin wrote:
 On Thu, 28 May 2009 19:59:00 +0400, Tim Matthews <tim.matthews7 gmail.com>
wrote:
 
 Leandro Lucarella wrote:

 I would like D2 better if it was focussed on macros for example.

feature which is no longer needed in D.

I believe he is talking about AST macros that are postponed until D3 because current focus has shifted to concurrency.

OK thanks I see now because macros have that extra flexibility over templates/mixins. Very useful and I agree so is parallelism/concurrency.
May 28 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 28 May 2009 20:32:29 +0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

 BCS wrote:
 Everything is indicating that shared memory multi-threading is where  
 it's all going.

That is correct, just that it's 40 years late. Right now everything is indicating that things are moving *away* from shared memory. Andrei

That's true. For example, we develop for PS3, and its 7 SPU cores have 256KiB of TLS each (which is as fast as L2 cache) and no direct shared memory access. Shared memory needs to be requested via asynchronous memcpy requests, and this scheme doesn't work with OOP well: even after you transfer some object, its vtbl etc still point to shared memory. We had hard time re-arranging our data so that object and everything it owns (and points to) is stored sequencially in a single large block of memory. This also resulted in replacing most of the pointers with relative offsets. Parallelization is hard, but the result is worth the trouble.
May 28 2009
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Thu, 28 May 2009 12:45:41 -0400, Denis Koroskin <2korden gmail.com>  
wrote:

 On Thu, 28 May 2009 20:32:29 +0400, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:

 BCS wrote:
 Everything is indicating that shared memory multi-threading is where
 it's all going.

That is correct, just that it's 40 years late. Right now everything is indicating that things are moving *away* from shared memory. Andrei

That's true. For example, we develop for PS3, and its 7 SPU cores have 256KiB of TLS each (which is as fast as L2 cache) and no direct shared memory access. Shared memory needs to be requested via asynchronous memcpy requests, and this scheme doesn't work with OOP well: even after you transfer some object, its vtbl etc still point to shared memory. We had hard time re-arranging our data so that object and everything it owns (and points to) is stored sequencially in a single large block of memory. This also resulted in replacing most of the pointers with relative offsets. Parallelization is hard, but the result is worth the trouble.

I agree that Andrei's right, but your example is wrong. The Cell's SPU are a SIMD vector processors, not general CPUs. I also work with vector processors (NVIDIA's CUDA) but every software/hardware iteration gets further and further away from pure vector processing. Rumor has it that the NVIDIA's next chip will be MIMD, instead of SIMD.
May 28 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 28 May 2009 21:07:57 +0400, Robert Jacques <sandford jhu.edu> wrote:

 On Thu, 28 May 2009 12:45:41 -0400, Denis Koroskin <2korden gmail.com>  
 wrote:

 On Thu, 28 May 2009 20:32:29 +0400, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:

 BCS wrote:
 Everything is indicating that shared memory multi-threading is where
 it's all going.

That is correct, just that it's 40 years late. Right now everything is indicating that things are moving *away* from shared memory. Andrei

That's true. For example, we develop for PS3, and its 7 SPU cores have 256KiB of TLS each (which is as fast as L2 cache) and no direct shared memory access. Shared memory needs to be requested via asynchronous memcpy requests, and this scheme doesn't work with OOP well: even after you transfer some object, its vtbl etc still point to shared memory. We had hard time re-arranging our data so that object and everything it owns (and points to) is stored sequencially in a single large block of memory. This also resulted in replacing most of the pointers with relative offsets. Parallelization is hard, but the result is worth the trouble.

I agree that Andrei's right, but your example is wrong. The Cell's SPU are a SIMD vector processors, not general CPUs. I also work with vector processors (NVIDIA's CUDA) but every software/hardware iteration gets further and further away from pure vector processing. Rumor has it that the NVIDIA's next chip will be MIMD, instead of SIMD.

I wanted to stress that multicore PUs tent to have their own local memory (small but fast) and little or none global (shared) memory access (it is not efficient and error prone - race condition et al.) I believe SIMD/MIMD discussion is irrelevant here. It's all about Shared/Distributed Memory Model. MIMD devices can be both (http://en.wikipedia.org/wiki/MIMD)
May 28 2009
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Thu, 28 May 2009 13:36:28 -0400, Denis Koroskin <2korden gmail.com>  
wrote:

 On Thu, 28 May 2009 21:07:57 +0400, Robert Jacques <sandford jhu.edu>  
 wrote:

 On Thu, 28 May 2009 12:45:41 -0400, Denis Koroskin <2korden gmail.com>
 wrote:

 On Thu, 28 May 2009 20:32:29 +0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 BCS wrote:
 Everything is indicating that shared memory multi-threading is where
 it's all going.

That is correct, just that it's 40 years late. Right now everything is indicating that things are moving *away* from shared memory. Andrei

That's true. For example, we develop for PS3, and its 7 SPU cores have 256KiB of TLS each (which is as fast as L2 cache) and no direct shared memory access. Shared memory needs to be requested via asynchronous memcpy requests, and this scheme doesn't work with OOP well: even after you transfer some object, its vtbl etc still point to shared memory. We had hard time re-arranging our data so that object and everything it owns (and points to) is stored sequencially in a single large block of memory. This also resulted in replacing most of the pointers with relative offsets. Parallelization is hard, but the result is worth the trouble.

I agree that Andrei's right, but your example is wrong. The Cell's SPU are a SIMD vector processors, not general CPUs. I also work with vector processors (NVIDIA's CUDA) but every software/hardware iteration gets further and further away from pure vector processing. Rumor has it that the NVIDIA's next chip will be MIMD, instead of SIMD.

I wanted to stress that multicore PUs tent to have their own local memory (small but fast) and little or none global (shared) memory access (it is not efficient and error prone - race condition et al.) I believe SIMD/MIMD discussion is irrelevant here. It's all about Shared/Distributed Memory Model. MIMD devices can be both (http://en.wikipedia.org/wiki/MIMD)

Well, I thought you were making a different point. Really, the Cell SPU is the only current PU with the design you're talking about. All commercial CPUs and GPUs have very large global memory buses. Every blog and talk I've read/attended has painted the SPU in a very negative light, at least with regard to the programming model. (Which makes sense, since it's sorta like non-cache coherent NUMA, which pretty much all everyone decided is a bad idea.)
May 28 2009
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
== Quote from Leandro Lucarella (llucax gmail.com)'s article
 Jason House, el 28 de mayo a las 08:45 me escribiste:

 Maybe people are waiting for Walter to go through all the hard work of
 implementing this stuff before complaining that it's crap and
 proclaiming Walter should have done in the first place?

No, I don't see any point in saying what I said above, because I don't think anything will change. If I didn't like some little detail, that could worth discussing because it has any chance to change Walter/Bartoz mind, but saying "I think all the model is way too complex" don't help much IMHO =)

That was basically the complaint about the const design for D2, and it did end up being simplified. I also think it would have been simplified further if anyone knew how to do so without losing any required functionality. Regarding the shared proposal so far, I think D will always support sharing memory across processes so the issue is really where to post the sign that says "here be monsters." Bartosz has come up with a model that would provide complete (?) verifiable data integrity, and therefore makes the domain of "safe" shared-memory programming as large as possible (deadlocks aside, of course). However, the overarching question in my mind is whether we really want to build so much support into the language for something that is intended to be used sparingly at best. I can just see someone saying "so you have all these new keywords and all this stuff and you're saying that despite all this I'm really not supposed to use any of it?" This is an area where community feedback would be very valuable, I'd think.
May 28 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 28 May 2009 16:45:42 +0400, Jason House <jason.james.house gmail.com>
wrote:

 I'm really surprised by the lack of design discussion in this thread.  
 It's amazing how there can be huge bursts of discussion on which keyword  
 to use (e.g. manifest constants), but then complete silence about major  
 design decisions like thread safety that defines new transitive states  
 and a bunch of new keywords. The description even made parallels to the  
 (previously?) unpopular const architecture.

 Maybe people are waiting for Walter to go through all the hard work of  
 implementing this stuff before complaining that it's crap and  
 proclaiming Walter should have done in the first place?
 This seems really unfair to Walter. Then again, I see no indication of  
 Walter wanting anything else.

It's plain easier to discuss bycicle shed color, because everyone is expert in it.
May 28 2009
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Thu, 28 May 2009 08:45:42 -0400, Jason House  
<jason.james.house gmail.com> wrote:
 I'm really surprised by the lack of design discussion in this thread.  
 It's amazing how there can be huge bursts of discussion on which keyword  
 to use (e.g. manifest constants), but then complete silence about major  
 design decisions like thread safety that defines new transitive states  
 and a bunch of new keywords. The description even made parallels to the  
 (previously?) unpopular const architecture.

 Maybe people are waiting for Walter to go through all the hard work of  
 implementing this stuff before complaining that it's crap and  
 proclaiming Walter should have done in the first place?
 This seems really unfair to Walter. Then again, I see no indication of  
 Walter wanting anything else.

Well, there's been a fair amount of previous related discussion. I've placed a proposal up on Wiki4D (http://www.prowiki.org/wiki4d/wiki.cgi?OwnershipTypesInD), though since it was assembled from a bunch of personal notes on the subject and uses Walter's old suggestion of using 'scope' instead of Bartosz's 'lent' it's a bit confusing. I'm planning on re-working it, but other deadlines come first. There's also been a lot of talk about message passing/future/promise/task/actor/agent based concurrency, data parallel models such as bulk synchronous programming (BSP) or GPU programming and auto-parallelization of pure functions. About the only thing needed from the type system to implement either of these models is the ability for uniques/mobiles to do a do-si-do type move (which should supported by ref unique). And BSP/GPU stuff are way too bleeding edge to support in the language proper yet. Honestly, I think people are holding back in part because Bartosz has only started to reveal a threading scheme and so are waiting for him to complete it, before proverbially ripping it apart.
May 28 2009
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 28 May 2009 08:45:42 -0400, Jason House  
<jason.james.house gmail.com> wrote:

 I'm really surprised by the lack of design discussion in this thread.  
 It's amazing how there can be huge bursts of discussion on which keyword  
 to use (e.g. manifest constants), but then complete silence about major  
 design decisions like thread safety that defines new transitive states  
 and a bunch of new keywords. The description even made parallels to the  
 (previously?) unpopular const architecture.

For the most part, this really academic threading stuff is beyond me. It took me long enough to understand threading with mutex locks... In any case, it didn't seem from the post that this was coming to D. It seemed like it was for a language Bartosz was working on besides D, the syntax doesn't even look close. Is this planned for D2 or D3? Or not at all? I remember Walter saying he didn't want to add umpteen different type constructor keywords, even unique, because of the confusion it would cause. In any case, once I decided it wasn't D related, I ignored it just like I usually ignore bearophile's "look at what the obscureX language does" posts (no offense bearophile). -Steve
May 28 2009
prev sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 28 May 2009 19:59:00 +0400, Tim Matthews <tim.matthews7 gmail.com>
wrote:

 Leandro Lucarella wrote:

 I would like D2 better if it was focussed on macros for example.

Can you elaborate on this? I think of the word macro as a C preprocessor feature which is no longer needed in D.

I believe he is talking about AST macros that are postponed until D3 because current focus has shifted to concurrency.
May 28 2009