www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D.NET - An explanation

reply Déja Augustine <Déja_member pathlink.com> writes:
I have received a number of inquiries about my old D.NET project with people
wanting the source code or wondering when I will finish it.  I have decided that
it was time to put it all to rest and perhaps pass the torch, along with what
knowledge I have acquired during the project in hopes that someone else will do
it right.

First off, I no longer have the code.  I lost it in a HDD crash, so please do
not e-mail me for it anymore.

Secondly, I wouldn't give it even if I had it, because it so badly mangled the D
front-end, and was so poorly designed in general, that I could not advocate its
use, even for a learning tool.

I will, however, point anyone interested in the general direction for how to use
NET's Compiler namespace to generate assemblies.  This is far from a
comprehensive analysis, but it should at least give you enough to get started.
Much of the rest can be learned from the myriad MSDN articles available online.

The basic process for creating a .NET assembly is this:  (If you can't follow
this, then take a few programming classes or read some tutorials before you try
this, because you'll have no hope otherwise)

1. Instantiate the AssemblyBuilder class
2. Use it to create new ModuleBuilder instances
3. In each module, you can use the various other *Builder classes to create the
framework for the module (VariableBuilder, FunctionBuilder, ClassBuilder, etc)

Okay, so I don't remember enough of the details to help more than that.
Basically, start with the AssemblyBuilder and read the MSDN notes.  

The biggest advice I have to offer is not to build it directly in the front-end,
but rather implement the stubs in the front-end to build an input file that you
can then pass to the assembly generator code.

I hope this helps.  .NET is really too wonderful a concept for D to ignore, so I
hope someone decides to take this up again.

Best of luck,

Déja Augustine
Dec 03 2005
parent reply Chris <ctlajoie yahoo.com> writes:
I was actually working on my D compiler for the .NET framework just before 
I saw this message.  I have read a lot on the subject, and I have experience 
implimenting a .net compiler for a language I created, but I still feel
amazingly 
unqualified to take on this project. Eventually I'd like to get a large enough 
group of people to contibute 



some discussion on the NG about how it would be nice to add experimental 
functionality, I decided to write a grammar instead. That way the lexer/parser 
changes would be very simple, and the only thing we would need to is change 
the code generation to handle the changes.

I finally got the lexer to work for all of my test cases (catching all those 
number formats was a bitch) and I'm currently working on getting the parser 
to create a complete enough AST that I can start writing the code generator. 
My goal at this point is to get it to compile some basic arithmetic and
variable 
assignment. When I get it to that point, I'll add a liscense and put it up 
on dsource.org so anyone can track it's progress yourself and contribute 
if you want.



will have a few deviations from the D spec. I mentioned one of them here: 
http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/30653. No doubt 
there will be more differences I will figure out along the way...

Chris


 I have received a number of inquiries about my old D.NET project with
 people wanting the source code or wondering when I will finish it.  I
 have decided that it was time to put it all to rest and perhaps pass
 the torch, along with what knowledge I have acquired during the
 project in hopes that someone else will do it right.
 [...]
 
 Déja Augustine
Dec 04 2005
parent reply Ivan Senji <ivan.senji_REMOVE_ _THIS__gmail.com> writes:
Chris wrote:

 I was actually working on my D compiler for the .NET framework just 
 before I saw this message.  I have read a lot on the subject, and I have 
 experience implimenting a .net compiler for a language I created, but I 
 still feel amazingly unqualified to take on this project. Eventually I'd 
 like to get a large enough group of people to contibute


 after reading some discussion on the NG about how it would be nice to 
 add experimental functionality, I decided to write a grammar instead.


I like the way you are thinking. This way it will be much simpler to 
experiment with.


 That way the lexer/parser changes would be very simple, and the only 
 thing we would need to is change the code generation to handle the changes.
 
 I finally got the lexer to work for all of my test cases (catching all 
 those number formats was a bitch) and I'm currently working on getting 
 the parser to create a complete enough AST that I can start writing the 
 code generator. My goal at this point is to get it to compile some basic 
 arithmetic and variable assignment. When I get it to that point, I'll 
 add a liscense and put it up on dsource.org so anyone can track it's 
 progress yourself and contribute if you want.
 


Great.



 don't want it to be confused with Deja's compiler, which was called 

 of them here: 
 http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/30653. No doubt 
 there will be more differences I will figure out along the way...
 


I think Deja used a pragma directive to indicate to wich namespace the 
code belongs to. Maybe that is an idea to consider. It would be nice if 
language didn't have to be changed a lot.

One more question by the way: will you be generating .NET 1 or 2 code?
Dec 04 2005
parent reply Chris Lajoie <ctlajoie yahoo.com> writes:
Sorry Ivan, somehow I missed this post. Your questions are answered below.



 I don't want it to be confused with Deja's compiler, which was called

 one of them here:
 http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/30653. No
 doubt there will be more differences I will figure out along the
 way...
 


 I think Deja used a pragma directive to indicate to wich namespace the
 code belongs to. Maybe that is an idea to consider. It would be nice
 if language didn't have to be changed a lot.


The deeper I get into this the more I'm finding out that it's going to be 
impossible to avoid changing the language. I will need to make additions 
to the language as well, for things like metadata attributes. There are some 
fundamental differences in system languages vs managed languages. C++ got 
around it using a concept I have heard referred to as "IJW", It Just Works. 
They can also compile native code into a .NET assembly, but I don't know 
how to do something like that. 

using pragmas might make it somewhat compatible, but tbh, I enjoy using D 
because it's powerful yet elegant (usually). If I can't have that in a .NET 
version of D, then I really wouldn't bother trying to write it.


 One more question by the way: will you be generating .NET 1 or 2 code?


I thought about this for a while. I decided on 2.0, but for now the compiler 

System.Collections.Generic.KeyValuePair<TKey, 
TValue> for associated arrays.

Chris
Dec 06 2005
parent reply Ivan Senji <ivan.senji_REMOVE_ _THIS__gmail.com> writes:
Chris Lajoie wrote:

 Sorry Ivan, somehow I missed this post. Your questions are answered below.


No problem.


 
 The deeper I get into this the more I'm finding out that it's going to 
 be impossible to avoid changing the language. I will need to make 
 additions to the language as well, for things like metadata attributes. 


Naturally, but I hope that the changes aren't going to be that big.


 There are some fundamental differences in system languages vs managed 
 languages. C++ got around it using a concept I have heard referred to as 
 "IJW", It Just Works. They can also compile native code into a .NET 
 assembly, but I don't know how to do something like that.


C++/CLI is a really crazy language. Imagine having existing native D 
code + managed code in the same assembly.


 using pragmas might make it somewhat compatible, but tbh, I enjoy using 
 D because it's powerful yet elegant (usually). If I can't have that in a 
 .NET version of D, then I really wouldn't bother trying to write it.
 


That is the reason I like D too. Wouldn't it be possible to just use D 
modules as namespaces somehow?


 One more question by the way: will you be generating .NET 1 or 2 code?


 
 
 I thought about this for a while. I decided on 2.0, but for now the 

 to use System.Collections.Generic.KeyValuePair<TKey, TValue> for 
 associated arrays.
 


Great! I can't wait to see something on dsource :)
Dec 06 2005
parent reply Chris Lajoie <ctlajoie yahoo.com> writes:
 Chris Lajoie wrote:
 

 Sorry Ivan, somehow I missed this post. Your questions are answered
 below.
 


 No problem.
 

 The deeper I get into this the more I'm finding out that it's going
 to be impossible to avoid changing the language. I will need to make
 additions to the language as well, for things like metadata
 attributes.
 


 Naturally, but I hope that the changes aren't going to be that big.
 

 There are some fundamental differences in system languages vs managed
 languages. C++ got around it using a concept I have heard referred to
 as "IJW", It Just Works. They can also compile native code into a
 .NET assembly, but I don't know how to do something like that.
 


 C++/CLI is a really crazy language. Imagine having existing native D
 code + managed code in the same assembly.
 

 using pragmas might make it somewhat compatible, but tbh, I enjoy
 using D because it's powerful yet elegant (usually). If I can't have
 that in a .NET version of D, then I really wouldn't bother trying to
 write it.
 


 That is the reason I like D too. Wouldn't it be possible to just use D
 modules as namespaces somehow?


yes. last night I was trying to figure out whether I could do modules similar 
to the way they are now in D (without namespaces necessarily). CIL already 
uses a module concept that would work great for that purpose, but there's 
no example code for "importing" code only from one specific module (as opposed 
to a whole namespace. classes in different modules can be in the same
namespace. 
The framework of course does this. namespace System is in mscorlib.dll and 
in System.dll, and most other framework assemblies for that matter). in a 
.NET assembly, a "module" usually wraps around all the different namespaces. 
Anyway, it shouldn't be too hard to figure out from the docs.


 One more question by the way: will you be generating .NET 1 or 2
 code?
 


 I thought about this for a while. I decided on 2.0, but for now the

 want to use System.Collections.Generic.KeyValuePair<TKey, TValue> for
 associated arrays.
 


 Great! I can't wait to see something on dsource :)


Hopefully some real magic will start to happen when this semester's over. 
I've been doing papers and speeches and tests left and right, and finals 
are coming up next week. I find that programmers usually have *way* too much 
time on their hands, or they don't have nearly enough.

Chris
Dec 06 2005
parent reply Bruno Medeiros <daiphoenixNO SPAMlycos.com> writes:
Chris Lajoie wrote:

 Chris Lajoie wrote:
 That is the reason I like D too. Wouldn't it be possible to just use D
 modules as namespaces somehow?


 
 
 yes. last night I was trying to figure out whether I could do modules 
 similar to the way they are now in D (without namespaces necessarily). 
 CIL already uses a module concept that would work great for that 
 purpose, but there's no example code for "importing" code only from one 
 specific module (as opposed to a whole namespace. classes in different 
 modules can be in the same namespace. The framework of course does this. 
 namespace System is in mscorlib.dll and in System.dll, and most other 
 framework assemblies for that matter). in a ..NET assembly, a "module" 
 usually wraps around all the different namespaces. Anyway, it shouldn't 
 be too hard to figure out from the docs.
 


Why not just use the namespaces for the D modules implementation? I 
don't see any problem, it is the other way around that would not be 
possible.

-- 
Bruno Medeiros - CS/E student
"Certain aspects of D are a pathway to many abilities some consider to 
be... unnatural."
Dec 06 2005
parent reply Chris Lajoie <ctlajoie yahoo.com> writes:
 yes. last night I was trying to figure out whether I could do modules
 similar to the way they are now in D (without namespaces
 necessarily). CIL already uses a module concept that would work great
 for that purpose, but there's no example code for "importing" code
 only from one specific module (as opposed to a whole namespace.
 classes in different modules can be in the same namespace. The
 framework of course does this. namespace System is in mscorlib.dll
 and in System.dll, and most other framework assemblies for that
 matter). in a ..NET assembly, a "module" usually wraps around all the
 different namespaces. Anyway, it shouldn't be too hard to figure out
 from the docs.
 


 Why not just use the namespaces for the D modules implementation? I
 don't see any problem, it is the other way around that would not be
 possible.


I don't know if I should do that because in D, there is only one module per 
file. With namespaces, there can (should) be an infinite number of files 
per namespace. This could be confusing.

I've been considering a few different options:
1) do what Bruno said: just use the module directive to indicate the namespace. 
the import directive (naturally) would import namespaces.
2) a hybrid of namespaces and modules. -- I found this to be impossible
considering 
the heirarchy of a .net assembly has a 'module' encompassing namespaces within 
it. D modules are "narrow" and could not encompass a namespace.
I would say a D module is more analogous to a .net class than anything else. 
using that analogy, classes and other data structures within the D module 
would be nested classes (or nested structs, etc) in .net.
3) remove the module keyword altogether, and add a namespace directive. unlike 
the 'module' in D, you should be able to have multiple namespaces in a single 
file. it could be written like this:
namespace Foo
{...}

// or
namespace Foo:
...

Since I've ruled out 2), please comment on 1) and 3). I am leaning towards 
3) myself, but I want some input first.

Thanks,
Chris
Dec 09 2005
next sibling parent John Reimer <John_member pathlink.com> writes:
In article <a8deea613fda8c7ca88decd305a news.digitalmars.com>, Chris Lajoie
says...


I don't know if I should do that because in D, there is only one module per 
file. With namespaces, there can (should) be an infinite number of files 
per namespace. This could be confusing.

I've been considering a few different options:
1) do what Bruno said: just use the module directive to indicate the namespace. 
the import directive (naturally) would import namespaces.
2) a hybrid of namespaces and modules. -- I found this to be impossible
considering 
the heirarchy of a .net assembly has a 'module' encompassing namespaces within 
it. D modules are "narrow" and could not encompass a namespace.
I would say a D module is more analogous to a .net class than anything else. 
using that analogy, classes and other data structures within the D module 
would be nested classes (or nested structs, etc) in .net.
3) remove the module keyword altogether, and add a namespace directive. unlike 
the 'module' in D, you should be able to have multiple namespaces in a single 
file. it could be written like this:
namespace Foo
{...}

// or
namespace Foo:
...

Since I've ruled out 2), please comment on 1) and 3). I am leaning towards 
3) myself, but I want some input first.

Thanks,
Chris



appropriately and keep the old D "module" keyword for its original purposes.
That would probably be better for compatibility with regular D.  Of course, that



Also you might like to add a new pragma for your .NET compiler... pragma(MS_NET)
or something similar, so other D compilers can skip the new keyword:

pragma(MS_NET)
{
namespace whatever;
}
Dec 09 2005
prev sibling parent "John C" <johnch_atms hotmail.com> writes:
"Chris Lajoie" <ctlajoie yahoo.com> wrote in message 
news:a8deea613fda8c7ca88decd305a news.digitalmars.com...

 yes. last night I was trying to figure out whether I could do modules
 similar to the way they are now in D (without namespaces
 necessarily). CIL already uses a module concept that would work great
 for that purpose, but there's no example code for "importing" code
 only from one specific module (as opposed to a whole namespace.
 classes in different modules can be in the same namespace. The
 framework of course does this. namespace System is in mscorlib.dll
 and in System.dll, and most other framework assemblies for that
 matter). in a ..NET assembly, a "module" usually wraps around all the
 different namespaces. Anyway, it shouldn't be too hard to figure out
 from the docs.


 Why not just use the namespaces for the D modules implementation? I
 don't see any problem, it is the other way around that would not be
 possible.


 I don't know if I should do that because in D, there is only one module 
 per file. With namespaces, there can (should) be an infinite number of 
 files per namespace. This could be confusing.

 I've been considering a few different options:
 1) do what Bruno said: just use the module directive to indicate the 
 namespace. the import directive (naturally) would import namespaces.
 2) a hybrid of namespaces and modules. -- I found this to be impossible 
 considering the heirarchy of a .net assembly has a 'module' encompassing 
 namespaces within it. D modules are "narrow" and could not encompass a 
 namespace.
 I would say a D module is more analogous to a .net class than anything 
 else. using that analogy, classes and other data structures within the D 
 module would be nested classes (or nested structs, etc) in .net.
 3) remove the module keyword altogether, and add a namespace directive. 
 unlike the 'module' in D, you should be able to have multiple namespaces 
 in a single file. it could be written like this:
 namespace Foo
 {...}

 // or
 namespace Foo:
 ...

 Since I've ruled out 2), please comment on 1) and 3). I am leaning towards 
 3) myself, but I want some input first.

 Thanks,
 Chris


Have you looked at how other languages have handled this? Delphi's units are 
similar in concept to D's modules, and this article explains how Dephi.NET 
maps unit declarations to .NET namespaces:
http://bdn.borland.com/article/0,1410,32765,00.html
Dec 09 2005