www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - =?windows-1252?Q?The_Next_Mainstream_Programming_Languag?=

reply Marcio <mqmnews123 sglebs.com> writes:
http://www.cs.princeton.edu/~dpw/popl/06/Tim-POPL.ppt
Aug 23 2006
next sibling parent reply John Demme <me teqdruid.com> writes:
Marcio wrote:


 http://www.cs.princeton.edu/~dpw/popl/06/Tim-POPL.ppt


This was a very, very interesting presentation.  Tim makes a lot of very
good points, and a very large portion of it applies far past game
development.  To that end, there are a lot of items in there which I hope
will be features for D 2.0.

In particular, I'm hoping that Walter's got some ideas concerning
parallelism.  I don't think there's any doubt that a language which excels
at allowing programmers to massively parallel code easily would be
absolutely invaluable.  Although I think that a lot of parallel-assist
mechanisms should and can be implemented in a library, but some compiler
assisted stuff would be great.

Some for instances:
-object versioning could help a library to implement non-locking
transactional writes (does this terminology make any sense?).
-If the compiler could somehow tell a library whether or not two functions
ever write to the same memory, the library could schedule one to run after
the other or in parallel. 
-What if the compiler could tell a library that a certain function call
never writes to and/or reads from existing heap space?
-Race condition checking and possible dead-lock checking are also obvious
assists.
-Some interface for a library to manage the code execution would also be
cool:
manager(ParallelComputation) {
        float foo() {
                float pi = calcPiLotsofAccuracy();
                long rects = getRectangleCount();
                return pi * rects;
        }
}
If the compiler can tell "ParallelComputation" manager that
calcPiLostofAccuracy() and getRectangleCount never write to the heap, it
will execute them in parallel.  Clearly, we could just introduce a keyword
to tell the compiler to do this, but allowing some interface for some other
component of the program to manage execution would allow for great
flexibilty, like functionally-specific thread-pools, or various
prioritization algorithms.

Here's another thought:
D's in and out function blocks are great, but could we get some
automagically generated stuff?  For instance,
void foo(notnull Object o){}
would automatically put an assert(o) in the in block.  Better yet, what
about some way to genericize that, so we could have something like in block
mixins?
template LessThan(A,B) {
        in {
                assert(A < B);
        }
}
template NotNull(O) {
        in {
                assert(O);
        }
}
void var(Object o, int i, int j) LessThan!(i,j), NotNull!(o) {
        //o won't be null, and i is less than j!
}


I dunno if any of these are decent ideas, but the above slides sparked some
ideas and real thought about some cool features, and I figured I might as
well throw some of them out there as food for thought....

Real point of the post, I guess:

Clearly some (err.. all) of these thoughts are quarter-baked, extreme ideas,
so I what I'm saying is that I'd like to get D 1.0 out there so we can move
on to some really innovative and extreme features....  D's pretty great as
is, but it's not the quantum leap forward needed to really assist
programmers with new challenges; it's got some really great potential to
get there, though.

-- 
~John Demme
me teqdruid.com
http://www.teqdruid.com/
Aug 23 2006
next sibling parent =?ISO-8859-1?Q?=22R=E9my_J=2E_A=2E_Mou=EBza=22?= writes:
John Demme a écrit :

 Here's another thought:
 D's in and out function blocks are great, but could we get some
 automagically generated stuff?  For instance,
 void foo(notnull Object o){}
 would automatically put an assert(o) in the in block.  Better yet, what
 about some way to genericize that, so we could have something like in block
 mixins?
 template LessThan(A,B) {
         in {
                 assert(A < B);
         }
 }
 template NotNull(O) {
         in {
                 assert(O);
         }
 }
 void var(Object o, int i, int j) LessThan!(i,j), NotNull!(o) {
         //o won't be null, and i is less than j!
 }
 
 
 I dunno if any of these are decent ideas, but the above slides sparked some
 ideas and real thought about some cool features, and I figured I might as
 well throw some of them out there as food for thought....


These are decent ideas.
In some ways, this looks like aspect oriented programming. LessThan is 
an aspect, the « in » block code is an advice of that aspect, void var ( 
Object, int, int ) is a joinpoint. What we're looking for is a way to 
weave the aspect in D code. However using aspect oriented programming 
for something no more complex than a contract definition is like using a 
rocket launcher to kill a fly.
In the presentation, Tim writes that sometimes developpers need to 
change a framework's base classes. In this situation, aspect oriented 
programming would allow to define totally and exclusively all the 
changes in a separate module and avoid to mess up with the framework's code.



 Real point of the post, I guess:
 
 Clearly some (err.. all) of these thoughts are quarter-baked, extreme ideas,
 so I what I'm saying is that I'd like to get D 1.0 out there so we can move
 on to some really innovative and extreme features....  D's pretty great as
 is, but it's not the quantum leap forward needed to really assist
 programmers with new challenges; it's got some really great potential to
 get there, though.
 


I agree.
Aug 24 2006
prev sibling parent reply kris <foo bar.com> writes:
John Demme wrote:

 Marcio wrote:
 
 

http://www.cs.princeton.edu/~dpw/popl/06/Tim-POPL.ppt


 
 
 This was a very, very interesting presentation.  Tim makes a lot of very
 good points, and a very large portion of it applies far past game
 development.  To that end, there are a lot of items in there which I hope
 will be features for D 2.0.
 
 In particular, I'm hoping that Walter's got some ideas concerning
 parallelism.  I don't think there's any doubt that a language which excels
 at allowing programmers to massively parallel code easily would be
 absolutely invaluable.  Although I think that a lot of parallel-assist
 mechanisms should and can be implemented in a library, but some compiler
 assisted stuff would be great.
 
 Some for instances:
 -object versioning could help a library to implement non-locking
 transactional writes (does this terminology make any sense?).
 -If the compiler could somehow tell a library whether or not two functions
 ever write to the same memory, the library could schedule one to run after
 the other or in parallel. 
 -What if the compiler could tell a library that a certain function call
 never writes to and/or reads from existing heap space?
 -Race condition checking and possible dead-lock checking are also obvious
 assists.
 -Some interface for a library to manage the code execution would also be
 cool:
 manager(ParallelComputation) {
         float foo() {
                 float pi = calcPiLotsofAccuracy();
                 long rects = getRectangleCount();
                 return pi * rects;
         }
 }
 If the compiler can tell "ParallelComputation" manager that
 calcPiLostofAccuracy() and getRectangleCount never write to the heap, it
 will execute them in parallel.  Clearly, we could just introduce a keyword
 to tell the compiler to do this, but allowing some interface for some other
 component of the program to manage execution would allow for great
 flexibilty, like functionally-specific thread-pools, or various
 prioritization algorithms.
 
 Here's another thought:
 D's in and out function blocks are great, but could we get some
 automagically generated stuff?  For instance,
 void foo(notnull Object o){}
 would automatically put an assert(o) in the in block.  Better yet, what
 about some way to genericize that, so we could have something like in block
 mixins?
 template LessThan(A,B) {
         in {
                 assert(A < B);
         }
 }
 template NotNull(O) {
         in {
                 assert(O);
         }
 }
 void var(Object o, int i, int j) LessThan!(i,j), NotNull!(o) {
         //o won't be null, and i is less than j!
 }
 
 
 I dunno if any of these are decent ideas, but the above slides sparked some
 ideas and real thought about some cool features, and I figured I might as
 well throw some of them out there as food for thought....
 
 Real point of the post, I guess:
 
 Clearly some (err.. all) of these thoughts are quarter-baked, extreme ideas,
 so I what I'm saying is that I'd like to get D 1.0 out there so we can move
 on to some really innovative and extreme features....  D's pretty great as
 is, but it's not the quantum leap forward needed to really assist
 programmers with new challenges; it's got some really great potential to
 get there, though.
 



CSP has been around for 30 years to address much of these concerns. 
Amusingly, certain somewhat prominent figures are 'rediscovering' CSP 
and calling it their own. You might be interested in reading up on Tony 
Hoare's CSP, and a principal implementation, occam ?
Aug 24 2006
next sibling parent John Demme <me teqdruid.com> writes:
kris wrote:

 
 
 CSP has been around for 30 years to address much of these concerns.
 Amusingly, certain somewhat prominent figures are 'rediscovering' CSP
 and calling it their own. You might be interested in reading up on Tony
 Hoare's CSP, and a principal implementation, occam ?


Thanks for the pointer, Kris.  I've downloaded Tony's book and will try to
digest some of it.  Looks very interesting.

-- 
~John Demme
me teqdruid.com
http://www.teqdruid.com/
Aug 24 2006
prev sibling parent reply Marcio <mqmnews123 sglebs.com> writes:
kris wrote:

 CSP has been around for 30 years to address much of these concerns. 
 Amusingly, certain somewhat prominent figures are 'rediscovering' CSP 
 and calling it their own. You might be interested in reading up on Tony 
 Hoare's CSP, and a principal implementation, occam ?



For a rediscovery of CSP channels but in C#, see:

"The Concurrency and Coordination Runtime (CCR) is a lightweight 
port-based concurrency library for C# 2.0 developed by George 
Chrysanthakopoulos in the Advanced Strategies group at Microsoft. Here 
http://channel9.msdn.com/ShowPost.aspx?PostID=143582 , we have a deep 
discussion about CCR with George, a Software Architect, and Satnam 
Singh, Architect. You can get more info about CCR on the CCR Wiki 
http://channel9.msdn.com/wiki/default.aspx/Channel9.ConcurrencyRuntime . 
This is super cool stuff and represents a really innovative approach to 
making managed threaded programming more readily understandable and 
predictable.

Please check out the OOPSLA/SCOOL paper on the CCR 
http://research.microsoft.com/~tharris/scool/papers/sing.pdf .

Click here http://channel9.msdn.com/Showpost.aspx?postid=206574 to see 
how the CCR is being used by the Microsoft Robotics Group."

CCR Programming http://channel9.msdn.com/ShowPost.aspx?PostID=219308



marcio
Aug 25 2006
parent kris <foo bar.com> writes:
Marcio wrote:

 kris wrote:
 

 CSP has been around for 30 years to address much of these concerns. 
 Amusingly, certain somewhat prominent figures are 'rediscovering' CSP 
 and calling it their own. You might be interested in reading up on 
 Tony Hoare's CSP, and a principal implementation, occam ?


 
 
 
 For a rediscovery of CSP channels but in C#, see:
 
 "The Concurrency and Coordination Runtime (CCR) is a lightweight 
 port-based concurrency library for C# 2.0 developed by George 
 Chrysanthakopoulos in the Advanced Strategies group at Microsoft. Here 
 http://channel9.msdn.com/ShowPost.aspx?PostID=143582 , we have a deep 
 discussion about CCR with George, a Software Architect, and Satnam 
 Singh, Architect. You can get more info about CCR on the CCR Wiki 
 http://channel9.msdn.com/wiki/default.aspx/Channel9.ConcurrencyRuntime . 
 This is super cool stuff and represents a really innovative approach to 
 making managed threaded programming more readily understandable and 
 predictable.
 
 Please check out the OOPSLA/SCOOL paper on the CCR 
 http://research.microsoft.com/~tharris/scool/papers/sing.pdf .
 
 Click here http://channel9.msdn.com/Showpost.aspx?postid=206574 to see 
 how the CCR is being used by the Microsoft Robotics Group."
 
 CCR Programming http://channel9.msdn.com/ShowPost.aspx?PostID=219308
 
 
 
 marcio




Nice! Thanks;
Aug 25 2006
prev sibling next sibling parent reply "Søren J. Løvborg" <web kwi.dk> writes:
This presentation seems familiar... :)

One interesting point, is the need to specify ranges of valid numbers. 
Pascal had decent support for this, allowing one to define integer types 
constrained to a given range:

type MyNumber : 1..15;

D could use something like this, though I'm not sure what to make of the 
syntax.

alias   1..15 MyNumber; // allows implicit conversion to (but not from) int.
typedef 1..15 MyNumber; // only allows explicit conversion.

One could imagine this extended to dynamic ranges:

void foo(int i, 0..i j)

But then again, maybe not.


 Software Frameworks The Problem: Users of a framework want to extend the 
 functionality
 of the framework's base classes!
 The workarounds:
 - Modify the source ... and modify it again with each new version
 - Add references to payload classes, and dynamically cast them at
   runtime to the appropriate types. - These are all error-prone: Can the 
 compiler help us here?


This is a problem I've only seen handled in the pretty obscure language 
TADS, which had a "modify" keyword, which allowed user code to replace 
entire methods of library classes, as well as add methods. Unlike the 
traditional approach -- overriding the methods in a subclass -- this 
affected all instances of the given class, also those instantiated in 
library code.

class LibraryClass
{
    void doStuff() { printf("foo\n"); }
}

modify LibraryClass
{
    void doStuff() { printf("bar\n"); }
    void entirelyNewFunction() { }
}

This was implemented by making an anonymous subclass of the LibraryClass, 
and then having the "LibraryClass" symbol refer to the subclass instead, and 
I guess the same could be done in D by screwing around with the linker and 
VMT.

I'm not saying this belongs in D; it's a horrible hack from an OOP 
standpoint (handy, though), and the "real" solution would be to design 
better libraries, with proper hooks.

Søren J. Løvborg
web kwi.dk
Aug 25 2006
parent reply Marcio <mqmnews123 sglebs.com> writes:
S=F8ren J. L=F8vborg wrote:

 Software Frameworks The Problem: Users of a framework want to extend t=




he=20

 functionality
 of the framework's base classes!
 The workarounds:
 - Modify the source ... and modify it again with each new version
 - Add references to payload classes, and dynamically cast them at
   runtime to the appropriate types. - These are all error-prone: Can t=




he=20

 compiler help us here?


=20
 This is a problem I've only seen handled in the pretty obscure language=


=20

 TADS, which had a "modify" keyword, which allowed user code to replace =



 entire methods of library classes, as well as add methods. Unlike the=20
 traditional approach -- overriding the methods in a subclass -- this=20
 affected all instances of the given class, also those instantiated in=20
 library code.




	Smalltalk has always had the ability to add methods to existing=20
classes, and even replace methods. This notion was organized better with =

more Software Engineering principles with ENVY/Developer from OTI (which =

was later bought by IBM and went on to produce VIsualAge for Smalltalk,=20
VisualAge for Java and later what is now called Eclipse).

	Anyway, ENVY/Developer brought the notion of modules (applications)=20
that could extend existing applications by adding 1) classes or 2)=20
methods to existing classes. So, ENVY/Swapper, for example, was an=20
application framework that added persistency. It pre-req'ed and extended =

the Kernel application and, among other things, added methods to class=20
Object (Object was defined in Kernel).

	I believe Ruby may have been inspired by Smalltalk here and allows=20
later code to add methods to existing classes a la Smalltalk. But I am=20
not a Ruby expert, so check the facts for yourself.

	Replacing an existing method is always dangerous and was flagged in=20
ENVY/Developer (if memory serves me well) as a conflict. This is more=20
common for runtime system - patching is a clear example. Another clear=20
example of runtime modification is AOP (Aspect Oriented Programming)=20
where people can plug code into running code and do things like logging, =

profiling, code coverage analysis etc.

	Obviously CLOS people will probably say "we've been doing this for=20
decades", even inserting a class C between A and B (inheritance-wise).

marcio
Aug 25 2006
parent =?windows-1252?Q?=22R=E9my_J=2E_A=2E_Mou=EBza=22?= writes:
Marcio a écrit :

     Replacing an existing method is always dangerous and was flagged in 
 ENVY/Developer (if memory serves me well) as a conflict. This is more 
 common for runtime system - patching is a clear example. Another clear 
 example of runtime modification is AOP (Aspect Oriented Programming) 
 where people can plug code into running code and do things like logging, 
 profiling, code coverage analysis etc.


AOP can also be done with a precompiler like AspectJ or AspectC++. 
However, compared to dynamic AOP ( using a dynamic language, with 
metaclases or reflexion ), it becomes more difficult to debug an weaved 
program because the aspects are scattered throughout the generated code 
thus it becomes less apparent what the programmer intended but tools 
exist, mainly for Eclipse ( I don't use it ).


     Obviously CLOS people will probably say "we've been doing this for 
 decades", even inserting a class C between A and B (inheritance-wise).


Thanks to the power of reflexion and metaprogramming. It seems that D's 
TypeInfo classes could be considered as metaclasses. Maybe there is a 
way to use them to extend existing classes, and replace methods or wrap 
them around new ones.
Aug 28 2006
prev sibling parent "news.digitalmars.com" <WorksOnMyMachine gmail.com> writes:
"Marcio" <mqmnews123 sglebs.com> wrote in message 
news:ecipfc$15v4$1 digitaldaemon.com...

 http://www.cs.princeton.edu/~dpw/popl/06/Tim-POPL.ppt



I'm new here but this somewhat old thread starting with this .ppt is very 
interesting to me since I work with the UE3 engine as a middleware.  Its 
pretty much dead on, as far as what the future is going to be, and that is 
in a massively parallel/threaded environment.  Those that do not make the 
transition are going to have very noticably 'rigid' game simulations that 
are far less capable.  Only the problem from my point of view is there isn't 
a single good language make the leap with.  Sure you can have a truely 
expert programmer get your threading right with the current tools available, 
but the reality is the current tools and languages available to make 
threaded programs are pretty bad.  In addition you end up with only 'one 
guy' who is responsible for far too much code since he has to wrangle 
everyone elses broken stuff into shape, and always on a rather riduclous 
timeline.

So how do games, threading, and D all tie into this?

Ok so I've been following D as a lurker off and on.  From the outside as a 
C++ programmer, D looks great to me.  I literally get more angry working 
with C++ every day.  And its all because doing anything 'cool' in C++, 
particularly templates, requires jumping through hoops.  Jumping through 
hoops is really the reality of having to deal with language deficiencies. 
So looking at D the initial impression is 'sweet' I want to write something 
in that.  Except from my world there are several huge problems:

Problem A : Garbage Collection is a dealbreaker.  But not because it exists 
or even that is is forced or a default.  We definitely want garbage 
collection.  It is a dealbreaker because of how it behaves.  There are 
several behaviors that make it such a strong negative as to be a 
dealbreaker, primarily the unknown frequency of collections, duration of 
collections, and the fact all our threads get suspended.  The more hardcore 
games run at a required 60 fps (gran turismo, god of war, etc).  This means 
all threads are executing a full cycle in 16.6 ms.  How much time do we want 
the GC to spend?  The answer is 0 ms.  Even spending 1ms for any kind of 
function in any game is pretty damn slow in a game engine.  If it starts 
pushing 5 or 10ms it starts impacting response of input, and noticebly 
hitches the rendering, since this hitch generally isn't every single frame. 
Consistency of the frame rate matters a lot.   In fact consistency matters 
so much that collecting could take 2ms of every 15 and we would be ok with 
it as long at was predictable so we can budget the game around it.  Even if 
it would only really need 10ms every 5 minutes, that is unacceptable, 
because a collector taking 10ms is a dealbreaker.

Problem B : Threading support.  The language of the future addresses 
threading head on, understanding that the number of cores on CPU processors 
is going to rapidly be in the tripple digits and beyond.  The chip makers 
have hit a wall, and the gains are going to come predominantly from core 
increases and memory I/O catching back up to the CPUs.  Eventually the line 
between CPU's and GPU's will blur quite a bit.  Which means we need to write 
threadable code safely, efficiently, without jumping through hoops, and not 
even really worrying about it a whole lot.  If our CPU based languages fail 
at this, we are going to be ending up writing game-physics raytracing code 
on the GPU instead via stuff like NVIDIA's GPGPU.  Which is essentially 
going to be stealing GPU performance to make up for the inability to take 
advantage of a massively parallel CPU core architecture.  Languages that are 
created, or are modified to make this leap cleanly will be the dominant 
players of the future.  I believe that if D makes this leap, it will be the 
one of them.  Judging by the progress of c++0x, I believe it has lost the 
agility necessary to make this transition and will be superceded by another 
language at this transition.


My gut feeling says that if the threading issues of are dealt with up front 
it would help a lot with garbage collection, since the requirements of 
moving the data efficiently in a massively threaded environment would drive 
evoloution in the GC system.  Even if the end-result is simply that you can 
construct isolated threads to get private GC heaps, and that data must be 
marshalled to them as if they were in another process space, it would be an 
improvement, because at least then the GC doesnt show down every single 
thread, and thin threads can collect very very fast.
Jul 14 2007