www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D1 -> D2

reply Fawzi Mohamed <fawzi gmx.ch> writes:
Is there any "porting" guide around in a wiki?
If not a page where to share the best tricks would be nice "D1->D2  
conversion tricks"?

In the short term I don't think that going D2 only is really an option  
for me, so how feasible it is to keep the code base compatible to both  
D1 and D2?

I know that one can define some templates (for example Const(T),....),  
and maybe use mixins, but how much uglier does the code become as  
result?
I choose D to have cleaner code, I am not interested in loosing all  
that just to be D1 and D2, then I prefer to wait, and convert  
everything at once.

Well that is about it...

thanks

Fawzi
Nov 18 2010
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On 11/18/2010 7:51 PM, Fawzi Mohamed wrote:
 Is there any "porting" guide around in a wiki?
 If not a page where to share the best tricks would be nice "D1->D2
 conversion tricks"?

 In the short term I don't think that going D2 only is really an option
 for me, so how feasible it is to keep the code base compatible to both
 D1 and D2?

 I know that one can define some templates (for example Const(T),....),
 and maybe use mixins, but how much uglier does the code become as result?
 I choose D to have cleaner code, I am not interested in loosing all that
 just to be D1 and D2, then I prefer to wait, and convert everything at
 once.

 Well that is about it...

 thanks

 Fawzi

In maintaining Derelict, which is nothing more than a simple collection of bindings to C libraries, I have had headaches keeping compatibility between D1/D2. It's nothing that has been difficult to solve, just ugly. If something as simple as a C-binding is uglified, I cringe at the thought of maintaining something more complex. It's going to get very ugly, very quickly. My attitude is that any future D projects I make available will be D2 only. I just don't think it's worth being compatible with both versions from a code maintenance perspective.
Nov 18 2010
parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Mike Parker wrote:
 On 11/18/2010 7:51 PM, Fawzi Mohamed wrote:

 In the short term I don't think that going D2 only is really an option
 for me, so how feasible it is to keep the code base compatible to both
 D1 and D2?

 I know that one can define some templates (for example Const(T),....),
 and maybe use mixins, but how much uglier does the code become as result?
 I choose D to have cleaner code, I am not interested in loosing all that
 just to be D1 and D2, then I prefer to wait, and convert everything at
 once.


 In maintaining Derelict, which is nothing more than a simple collection
 of bindings to C libraries, I have had headaches keeping compatibility
 between D1/D2. It's nothing that has been difficult to solve, just ugly.

 If something as simple as a C-binding is uglified, I cringe at the
 thought of maintaining something more complex. It's going to get very
 ugly, very quickly. My attitude is that any future D projects I make
 available will be D2 only. I just don't think it's worth being
 compatible with both versions from a code maintenance perspective.

wxD had the same experience, with supporting Phobos/Tango and D1/D2. It's "possible", but leads to ugly/duplicated code and best avoided... For a C/C++-binding it's doable with liberal amounts of version(Tango) and version(D_Version2) but that quickly gets hideous and in the way. --anders PS. Adding three compilers and three build systems, it just exploded. It made even the two OS and the three UI look simple in comparison. OS: Windows, Unix (linux, darwin) UI: __WXMSW__, __WXGTK__, __WXMAC__
Nov 18 2010
prev sibling next sibling parent Fawzi Mohamed <fawzi gmx.ch> writes:
On 18-nov-10, at 12:07, Mike Parker wrote:

 On 11/18/2010 7:51 PM, Fawzi Mohamed wrote:
 Is there any "porting" guide around in a wiki?
 If not a page where to share the best tricks would be nice "D1->D2
 conversion tricks"?

 In the short term I don't think that going D2 only is really an  
 option
 for me, so how feasible it is to keep the code base compatible to  
 both
 D1 and D2?

 I know that one can define some templates (for example  
 Const(T),....),
 and maybe use mixins, but how much uglier does the code become as  
 result?
 I choose D to have cleaner code, I am not interested in loosing all  
 that
 just to be D1 and D2, then I prefer to wait, and convert everything  
 at
 once.

 Well that is about it...

 thanks

 Fawzi

In maintaining Derelict, which is nothing more than a simple collection of bindings to C libraries, I have had headaches keeping compatibility between D1/D2. It's nothing that has been difficult to solve, just ugly. If something as simple as a C-binding is uglified, I cringe at the thought of maintaining something more complex. It's going to get very ugly, very quickly. My attitude is that any future D projects I make available will be D2 only. I just don't think it's worth being compatible with both versions from a code maintenance perspective.

that was my feeling too, but I wanted some confirmation from people having actually done it. This just reinforces my choice of being D1 only for the moment...
Nov 18 2010
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 18 Nov 2010 05:51:32 -0500, Fawzi Mohamed <fawzi gmx.ch> wrote:

 Is there any "porting" guide around in a wiki?
 If not a page where to share the best tricks would be nice "D1->D2  
 conversion tricks"?

 In the short term I don't think that going D2 only is really an option  
 for me, so how feasible it is to keep the code base compatible to both  
 D1 and D2?

 I know that one can define some templates (for example Const(T),....),  
 and maybe use mixins, but how much uglier does the code become as result?
 I choose D to have cleaner code, I am not interested in loosing all that  
 just to be D1 and D2, then I prefer to wait, and convert everything at  
 once.

I have had two experiences moving code from D1 to D2. First, I tried to port Tango to D2, and found that backwards compatibility is not possible. So I was simply reorganizing the files. The hugest problem for going from D1 to D2 is string literals. Every place you use char[] has to be switched to const(char)[] or string. I came up with a set of rules for this: 1. if the 99% common case for string arguments will be string literals, then use string -- you never have to worry about duping. 2. if you intend to keep a copy of the string, but you aren't sure whether the string to be passed in will have mutable aliases, use const(char)[] and dup. 3. if you don't intend to keep a copy of the string, use const(char)[] to have the most flexibility. 4. if you intend on returning a portion of the string, well, currently you're SOL, but eventually inout will come into play here. For example, Tango's log package fell under rules 1 and 3 (most of the time you are creating log objects with string literals, and logging a message does not save the message anywhere). Many many cases fell under rule 4, which is one of the main reasons I gave up (inout still isn't ready). The second experience I had was porting dcollections to D2. Since dcollections has not many places for string literals, it was much easier to port. I don't think I made it backwards compatible, but it's very very similar to the original in terms of constancy. I've added a few features that only really make sense in D2 (ranges for one). Porting dcollections was quite easy, but with my experience trying to port Tango, I never tried to make it backwards compatible. My recommendation -- when you are ready, switch wholly to D2. Don't bother with compatibility, it's just not possible. -Steve
Nov 18 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Steven Schveighoffer wrote:
 My recommendation -- when you are ready, switch wholly to D2.  Don't 
 bother with compatibility, it's just not possible.

From what you wrote, it appears that most of the difficulties were in dealing with strings. Might I suggest: 1. Replace all occurrences of char[] with string. 2. Compile to find every place that mutable strings are used. 3. Refactor the code to clearly encapsulate where mutable strings are created and manipulated, and as the last step, cast them to string.
Nov 18 2010
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Denis Koroskin wrote:
  From my experience, a bigger issue is that
 
 version (D2) {
    /* some D2 only code involving const etc */
 }
 
 simply won't compile in D1, because even if it's D2-only the code needs 
 to be correct D1 code as well.

I agree that's an issue. Currently, the only way to deal with this is one of: 1. Minimize the differences, and maintain two copies of the source code. Using the (rather fabulous) meld tool (available on Linux), the merging is pretty easy. I use meld all the time to, for example, merge differences in the code base between dmd1 and dmd2. 2. Isolate the code that is different into different files, which minimizes the work involved in (1). 3. Use string mixins and the token string literal form. I've also found that using the string alias rather than char[] does account for an awful lot of the differences.
Nov 18 2010
next sibling parent Rainer Deyke <rainerd eldwood.com> writes:
On 11/18/2010 12:39, Walter Bright wrote:
 I agree that's an issue. Currently, the only way to deal with this is
 one of:
 
 1. Minimize the differences, and maintain two copies of the source code.
 Using the (rather fabulous) meld tool (available on Linux), the merging
 is pretty easy. I use meld all the time to, for example, merge
 differences in the code base between dmd1 and dmd2.
 
 2. Isolate the code that is different into different files, which
 minimizes the work involved in (1).
 
 3. Use string mixins and the token string literal form.

4. Use an external preprocessor to generate the D1 and D2 code from the same source file. -- Rainer Deyke - rainerd eldwood.com
Nov 18 2010
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Walter Bright wrote:
 I agree that's an issue. Currently, the only way to deal with this is 
 one of:

There's a 4th method I almost hesitate to mention, but you can use a common source code file, and a macro text preprocessor (such as m4) to generate the D1 and D2 versions from it.
Nov 18 2010
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Denis Koroskin wrote:
 All these are ugly.

I agree.
Nov 18 2010
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Steven Schveighoffer wrote:
 On Thu, 18 Nov 2010 14:19:12 -0500, Walter Bright 
 <newshound2 digitalmars.com> wrote:
 
 Steven Schveighoffer wrote:
 My recommendation -- when you are ready, switch wholly to D2.  Don't 
 bother with compatibility, it's just not possible.

From what you wrote, it appears that most of the difficulties were in dealing with strings. Might I suggest: 1. Replace all occurrences of char[] with string. 2. Compile to find every place that mutable strings are used. 3. Refactor the code to clearly encapsulate where mutable strings are created and manipulated, and as the last step, cast them to string.

But string is not always what is desired. It depends on the library.

Right.
 Tango uses mutable arrays everywhere because it prefers not to use the 
 heap as much as possible.  For example, something might take a char[] 
 buffer, and a function may pass in a stack-allocated array for the 
 buffer.  Changing this to string is useless.

That's where refactoring comes in. I haven't looked at the Tango code, but there's a big difference between "using mutable arrays everywhere" and actually needing to mutate them. Only the latter needs to be char[], and the boundaries between the two are bridged with a .idup or a cast(string). If Tango is relying on string literals to be char[], i.e. be mutable, this is a serious bug and will cause seg faults. So, this issue must have already been dealt with in Tango, and codifying string literal types as "string" rather than "char[]" should not negatively impact it. And lastly, if there are APIs in tango that accept char[] and leave the caller perplexed about whether the API will mutate the string or not, that is something that using string for the non-mutating ones nicely resolves.
Nov 18 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Steven Schveighoffer wrote:
 It's not impossible, but Tango's #1 rule is minimize heap allocations.  
 By peppering idup everywhere, you have created a conflict with the main 
 goal.  Refactoring in these cases isn't quite as simple as you say, and 
 it certainly won't be backwards compatible.

If you know the string is unique, you can use cast(string) rather than .idup, and no allocations will be done.
 In general, I tried to make Tango for D2 as const aware as possible 
 (it's a standard lib, and shouldn't require people to ignore const), and 
 found all sorts of issues with it (const).  The main one that made me 
 quit was the problem that inout is supposed to solve.

Right; inout is intended to fix that.
Nov 18 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-11-18 16:02:40 -0500, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 class obj
 {
     char[] name;
     this(char[] name) { this.name = name;}
 }
 
 What to do there?  If name is typically a string literal, then string 
 is  appropriate.  But if there are cases where a name is passed in via 
 a  char[] allocated on the heap, then const(char) makes sense.  
 However, you  really don't want the name to change at some other point, 
 so if it's not  immutable it should idup the name.
 
 It's not easy to make this code both safe and efficient.  In the end, I 
  think in cases like this I did something like:
 
 string name;
 this(const(char)[] name) { this.name = name.idup;}
 this(string name) {this.name = name;}

If you're only going to store the string as immutable, I think it's better to accept it only as immutable in the constructor. This way you pass the responsibility of iduping the string to the caller and this might reveal optimization opportunities. For instance: obj[100] versions; foreach (i; 1..100) versions[i] = new obj(name.idup); See how it the inefficiency obvious when the caller is the one calling idup? -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Nov 18 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/18/10 1:50 PM, Michel Fortin wrote:
 On 2010-11-18 16:02:40 -0500, "Steven Schveighoffer"
 <schveiguy yahoo.com> said:

 class obj
 {
 char[] name;
 this(char[] name) { this.name = name;}
 }

 What to do there? If name is typically a string literal, then string
 is appropriate. But if there are cases where a name is passed in via a
 char[] allocated on the heap, then const(char) makes sense. However,
 you really don't want the name to change at some other point, so if
 it's not immutable it should idup the name.

 It's not easy to make this code both safe and efficient. In the end, I
 think in cases like this I did something like:

 string name;
 this(const(char)[] name) { this.name = name.idup;}
 this(string name) {this.name = name;}

If you're only going to store the string as immutable, I think it's better to accept it only as immutable in the constructor. This way you pass the responsibility of iduping the string to the caller and this might reveal optimization opportunities. For instance: obj[100] versions; foreach (i; 1..100) versions[i] = new obj(name.idup); See how it the inefficiency obvious when the caller is the one calling idup?

Yes, I think that's the best idiom (I use it in Phobos in a number of places). It's safe and as efficient as it could. Andrei
Nov 18 2010
parent Walter Bright <newshound2 digitalmars.com> writes:
Andrei Alexandrescu wrote:
 On 11/18/10 1:50 PM, Michel Fortin wrote:
 On 2010-11-18 16:02:40 -0500, "Steven Schveighoffer"
 <schveiguy yahoo.com> said:

 class obj
 {
 char[] name;
 this(char[] name) { this.name = name;}
 }

 What to do there?



I agree that it just seems like good class design to have obj.name be immutable, and pass the responsibility for making it so up the call chain to the constructor.
Nov 18 2010
prev sibling next sibling parent reply %u <e ee.com> writes:
== Quote from Denis Koroskin (2korden gmail.com)'s article
 On Thu, 18 Nov 2010 22:19:12 +0300, Walter Bright
 <newshound2 digitalmars.com> wrote:
 Steven Schveighoffer wrote:
 My recommendation -- when you are ready, switch wholly to D2.  Don't
 bother with compatibility, it's just not possible.

From what you wrote, it appears that most of the difficulties were in dealing with strings.

version (D2) { /* some D2 only code involving const etc */ } simply won't compile in D1, because even if it's D2-only the code needs to be correct D1 code as well.

Wow, that sucks!! Can't this simply be mended with special D_DMD1 and D_DMD2 (D_DMD3 :) version identifiers? If the compiler version isn't the same then it won't look at the code.
Nov 18 2010
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Bernard Helyer wrote:
 No. The contents of a version block is still parsed. If the D2 code 
 doesn't _parse_ as valid D1, then the D1 compiler will reject it, even if 
 the version block will not be processed further than that.

Yes. Now, if the D1 compiler is supposed to successfully lex & parse (and then ignore) false version blocks, it becomes impossible for a D1 compiler to be stable. It will have to track every future D language variant, missteps and all.
Nov 19 2010
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Andrew Wiley wrote:
 I'm a fan of parsing false version blocks, but it seems like there 
 should be some sort of exception for version blocks for a different 
 version of the language. Could they just be skipped with a warning that 
 an unknown version of the language is in use? It's somewhat hackish, but 
 to me, that seems better than parsing it and giving an error because of 
 syntax changes. 

How can it skip it without knowing the grammar?
Nov 19 2010
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Denis Koroskin wrote:
 On Sat, 20 Nov 2010 04:14:05 +0300, Walter Bright 
 How can it skip it without knowing the grammar?

#ifdef D_Version2 ... #else ... #endif D has a precedence of having preprocessor macros (e.g #line) even if it's not a full-fledged preprocessor, and I don't see why it couldn't adapt the same solution. FWIW, C# has #if blocks, too.

Yes, I know it can be done with a text preprocessor. [[ Queue run away screaming ]]
Nov 19 2010
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Andrew Wiley wrote:
 On Fri, Nov 19, 2010 at 7:14 PM, Walter Bright 
 <newshound2 digitalmars.com <mailto:newshound2 digitalmars.com>> wrote:
 
     Andrew Wiley wrote:
 
         I'm a fan of parsing false version blocks, but it seems like
         there should be some sort of exception for version blocks for a
         different version of the language. Could they just be skipped
         with a warning that an unknown version of the language is in
         use? It's somewhat hackish, but to me, that seems better than
         parsing it and giving an error because of syntax changes.
 
 
 
     How can it skip it without knowing the grammar?
 
 
 It's probably too much work for too little gain, but would it be 
 possible to basically just scan for matching braces until the version 
 block ends? If they always come in matching pairs (or at least they 
 *should*), you could scan for them and ignore everything else until you 
 find the end of the block.

It would still have to know about all the new tokens.
Nov 19 2010
prev sibling parent reply torhu <no spam.invalid> writes:
On 20.11.2010 01:18, Walter Bright wrote:
 Bernard Helyer wrote:
  No. The contents of a version block is still parsed. If the D2 code
  doesn't _parse_ as valid D1, then the D1 compiler will reject it, even if
  the version block will not be processed further than that.

Yes. Now, if the D1 compiler is supposed to successfully lex& parse (and then ignore) false version blocks, it becomes impossible for a D1 compiler to be stable. It will have to track every future D language variant, missteps and all.

But current D 1 compilers are useless for most people anyway, until Tango catches up. And in any case, if you upgrade, you often get some nice new bugs in addition to the fixes, so upgrading is often not worth the hassle. But making it easier to go from D 1 to D 2 would still be a big gain, so why not do that? No one cares about "stability" of D 1 compilers. If you want "stability", it's very simple. You just don't upgrade. Tango is dying, and D 1 with it. We don't want "stability", we want a viable way to move to D 2.
Nov 19 2010
parent Walter Bright <newshound2 digitalmars.com> writes:
torhu wrote:
 But current D 1 compilers are useless for most people anyway, until 
 Tango catches up.  And in any case, if you upgrade, you often get some 
 nice new bugs in addition to the fixes, so upgrading is often not worth 
 the hassle.  But making it easier to go from D 1 to D 2 would still be a 
 big gain, so why not do that?  No one cares about "stability" of D 1 
 compilers.  If you want "stability", it's very simple.  You just don't 
 upgrade.
 
 Tango is dying, and D 1 with it.  We don't want "stability", we want a 
 viable way to move to D 2.

I understand. You can use a text preprocessor right now to do it, m4. It's quick, easy, and works.
Nov 19 2010
prev sibling parent %u <e ee.com> writes:
== Quote from Bernard Helyer (b.helyer gmail.com)'s article
 On Thu, 18 Nov 2010 19:58:42 +0000, %u wrote:
 == Quote from Denis Koroskin (2korden gmail.com)'s article
 On Thu, 18 Nov 2010 22:19:12 +0300, Walter Bright
 <newshound2 digitalmars.com> wrote:
 Steven Schveighoffer wrote:
 My recommendation -- when you are ready, switch wholly to D2.  Don't
 bother with compatibility, it's just not possible.

From what you wrote, it appears that most of the difficulties were in dealing with strings.

/* some D2 only code involving const etc */ } simply won't compile in D1, because even if it's D2-only the code needs to be correct D1 code as well.

Wow, that sucks!! Can't this simply be mended with special D_DMD1 and D_DMD2 (D_DMD3 :) version identifiers? If the compiler version isn't the same then it won't look at the code.

doesn't _parse_ as valid D1, then the D1 compiler will reject it, even if the version block will not be processed further than that.

Well, actually I said the compiler wouldn't look at it, so it wouldn't get parsed as well. What you should have said is that is very difficult to not parse a piece of code and still know its boundaries ;) So how can the compiler be sure a version block ends without actually parsing the text as code. A single line version would do :)
Nov 19 2010
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Fawzi Mohamed wrote:
  From the discussion it seems that defining something like:
 
 version(D_Version2){
     template Const(T){
         alias const(T) Const;
     }
     template Immutable(T){
         alias immutable(T) Immutable;
     }
     immutable(T) Idup(T)(T val){
         return val.idup;
     }
     alias const(char)[] cstring;
 } else {
     template Const(T){
         alias T Const;
     }
     template Immutable(T){
         alias T Immutable;
     }
     T Idup(T)(T val){
         return val.dup;
     }
     alias char[] string;
     alias char[] cstring;
 }
 
 could help a lot
 later one can simply replace Const! with const and Immutable! with 
 immutable, Idup replacement is more complicated, but doable

The problem with this approach is it requires the D1 compiler to be able to parse D2 syntax, including recognizing all of D2's keywords.
 What is not so clear is how to cope with scope, because I have lot of 
 delegates around that will need it.
 For it a preprocessing step might really be the best thing, or someone 
 knows a smart way to cope with that?

Not sure what the issue is.
Nov 18 2010
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Fawzi Mohamed wrote:
 I don't find a valid D1 expression to put in place of scope, or to 
 somehow hide it, i.e. how do you write something like
 module t;
 
 void f(scope void delegate() action){
     action();
 }
 
 void main(){
     f(scope delegate(){
         printf("bla\n");
     });
 }
 
 so that it is valid D1 and D2?

Just remove the 'scope'.
Nov 18 2010
prev sibling next sibling parent Fawzi Mohamed <fawzi gmx.ch> writes:
On 19-nov-10, at 08:44, Walter Bright wrote:

 Fawzi Mohamed wrote:
 I don't find a valid D1 expression to put in place of scope, or to  
 somehow hide it, i.e. how do you write something like
 module t;
 void f(scope void delegate() action){
    action();
 }
 void main(){
    f(scope delegate(){
        printf("bla\n");
    });
 }
 so that it is valid D1 and D2?

Just remove the 'scope'.

Still maybe you are right, I will use /+scope+/, so that one has something working, and easily go to the efficient D2 version
Nov 19 2010
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
Walter Bright <newshound2 digitalmars.com> wrote:
 Fawzi Mohamed wrote:
  From the discussion it seems that defining something like:
 version(D_Version2){

alias const(T) Const; } template Immutable(T){ alias immutable(T) Immutable; } immutable(T) Idup(T)(T val){ return val.idup; } alias const(char)[] cstring; } else { template Const(T){ alias T Const; } template Immutable(T){ alias T Immutable; } T Idup(T)(T val){ return val.dup; } alias char[] string; alias char[] cstring; }
 could help a lot

immutable, Idup replacement is more complicated, but doable

The problem with this approach is it requires the D1 compiler to be able to parse D2 syntax, including recognizing all of D2's keywords.

Make them string mixins. I had druntime working on both D1 and D2 for a while and did it with a mix of aliases, string mixins, and "in" for const args. It worked, but ended up being worse than simply maintaining two codebases.
Nov 21 2010
prev sibling parent Fawzi Mohamed <fawzi gmx.ch> writes:
On 19-nov-10, at 08:08, Walter Bright wrote:

 Fawzi Mohamed wrote:
 From the discussion it seems that defining something like:
 version(D_Version2){


    template Const(T){
        alias const(T) Const;
    }
    template Immutable(T){
        alias immutable(T) Immutable;
    }
    immutable(T) Idup(T)(T val){
        return val.idup;
    }
    alias const(char)[] cstring;


 } else {
    template Const(T){
        alias T Const;
    }
    template Immutable(T){
        alias T Immutable;
    }
    T Idup(T)(T val){
        return val.dup;
    }
    alias char[] string;
    alias char[] cstring;
 }
 could help a lot
 later one can simply replace Const! with const and Immutable! with  
 immutable, Idup replacement is more complicated, but doable

The problem with this approach is it requires the D1 compiler to be able to parse D2 syntax, including recognizing all of D2's keywords.

yes you are right, I meant to put a mixin ther but then I forgot it.
 What is not so clear is how to cope with scope, because I have lot  
 of delegates around that will need it.
 For it a preprocessing step might really be the best thing, or  
 someone knows a smart way to cope with that?

Not sure what the issue is.

I don't find a valid D1 expression to put in place of scope, or to somehow hide it, i.e. how do you write something like module t; void f(scope void delegate() action){ action(); } void main(){ f(scope delegate(){ printf("bla\n"); }); } so that it is valid D1 and D2?
Nov 18 2010
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2010-11-18 11:51, Fawzi Mohamed wrote:
 Is there any "porting" guide around in a wiki?
 If not a page where to share the best tricks would be nice "D1->D2
 conversion tricks"?

 In the short term I don't think that going D2 only is really an option
 for me, so how feasible it is to keep the code base compatible to both
 D1 and D2?

 I know that one can define some templates (for example Const(T),....),
 and maybe use mixins, but how much uglier does the code become as result?
 I choose D to have cleaner code, I am not interested in loosing all that
 just to be D1 and D2, then I prefer to wait, and convert everything at
 once.

 Well that is about it...

 thanks

 Fawzi

The experience I have with this is the Orange library which supports both D1 (Tango) and D2. I say this: it can sometimes be surprisingly easy and sometimes a PITA. Depending on what you do you can get a long way with aliases, I have for quite a long time now been using a string alias in D1 (I just like it better than char[]). For some things you will have to use string mixins and after some testing with Derelict2 we found out that it's better to use few string mixins containing a lot of code than many string mixins containing just some code. Otherwise the compile time will increase, sometimes quite a lot. -- /Jacob Carlborg
Nov 18 2010
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 18 Nov 2010 22:19:12 +0300, Walter Bright  
<newshound2 digitalmars.com> wrote:

 Steven Schveighoffer wrote:
 My recommendation -- when you are ready, switch wholly to D2.  Don't  
 bother with compatibility, it's just not possible.

From what you wrote, it appears that most of the difficulties were in dealing with strings.

No, it's not. From my experience, a bigger issue is that version (D2) { /* some D2 only code involving const etc */ } simply won't compile in D1, because even if it's D2-only the code needs to be correct D1 code as well.
Nov 18 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 18 Nov 2010 14:19:12 -0500, Walter Bright  
<newshound2 digitalmars.com> wrote:

 Steven Schveighoffer wrote:
 My recommendation -- when you are ready, switch wholly to D2.  Don't  
 bother with compatibility, it's just not possible.

From what you wrote, it appears that most of the difficulties were in dealing with strings. Might I suggest: 1. Replace all occurrences of char[] with string. 2. Compile to find every place that mutable strings are used. 3. Refactor the code to clearly encapsulate where mutable strings are created and manipulated, and as the last step, cast them to string.

But string is not always what is desired. It depends on the library. Tango uses mutable arrays everywhere because it prefers not to use the heap as much as possible. For example, something might take a char[] buffer, and a function may pass in a stack-allocated array for the buffer. Changing this to string is useless. -Steve
Nov 18 2010
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On Thu, 18 Nov 2010 20:07:38 +0900
Mike Parker <aldacron gmail.com> wrote:

 On 11/18/2010 7:51 PM, Fawzi Mohamed wrote:
 Is there any "porting" guide around in a wiki?
 If not a page where to share the best tricks would be nice "D1->D2
 conversion tricks"?

 In the short term I don't think that going D2 only is really an option
 for me, so how feasible it is to keep the code base compatible to both
 D1 and D2?

 I know that one can define some templates (for example Const(T),....),
 and maybe use mixins, but how much uglier does the code become as resul=


 I choose D to have cleaner code, I am not interested in loosing all that
 just to be D1 and D2, then I prefer to wait, and convert everything at
 once.

 Well that is about it...

 thanks

 Fawzi

In maintaining Derelict, which is nothing more than a simple collection=20 of bindings to C libraries, I have had headaches keeping compatibility=20 between D1/D2. It's nothing that has been difficult to solve, just ugly. =20 If something as simple as a C-binding is uglified, I cringe at the=20 thought of maintaining something more complex. It's going to get very=20 ugly, very quickly. My attitude is that any future D projects I make=20 available will be D2 only. I just don't think it's worth being=20 compatible with both versions from a code maintenance perspective.

I have read your blog post on the topic. This let me think that all true ne= eds for string mixins, like yours, are cases revealing plain language issue= s. denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 18 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 18 Nov 2010 14:50:06 -0500, Walter Bright  
<newshound2 digitalmars.com> wrote:

 Steven Schveighoffer wrote:
 On Thu, 18 Nov 2010 14:19:12 -0500, Walter Bright  
 <newshound2 digitalmars.com> wrote:

 Steven Schveighoffer wrote:
 My recommendation -- when you are ready, switch wholly to D2.  Don't  
 bother with compatibility, it's just not possible.

From what you wrote, it appears that most of the difficulties were in dealing with strings. Might I suggest: 1. Replace all occurrences of char[] with string. 2. Compile to find every place that mutable strings are used. 3. Refactor the code to clearly encapsulate where mutable strings are created and manipulated, and as the last step, cast them to string.


Right.
 Tango uses mutable arrays everywhere because it prefers not to use the  
 heap as much as possible.  For example, something might take a char[]  
 buffer, and a function may pass in a stack-allocated array for the  
 buffer.  Changing this to string is useless.

That's where refactoring comes in. I haven't looked at the Tango code, but there's a big difference between "using mutable arrays everywhere" and actually needing to mutate them. Only the latter needs to be char[], and the boundaries between the two are bridged with a .idup or a cast(string). If Tango is relying on string literals to be char[], i.e. be mutable, this is a serious bug and will cause seg faults. So, this issue must have already been dealt with in Tango, and codifying string literal types as "string" rather than "char[]" should not negatively impact it. And lastly, if there are APIs in tango that accept char[] and leave the caller perplexed about whether the API will mutate the string or not, that is something that using string for the non-mutating ones nicely resolves.

Yes, Tango obviously works on Linux, so it does not modify string literals (as much as it can, you can't really prevent a user from passing in a string literal to sort for example). However, the char[] type accepts both string literals and mutable buffers. In some cases, those strings are stored for later use, in some they are modified (obviously one has to avoid passing in literals for that), and in some they are just used temporarily. The differences require you to change the argument type to string or const(char)[], and possibly to overload the function. The rules I outlined were what I was going by in order to port D1 code. It's not impossible, but Tango's #1 rule is minimize heap allocations. By peppering idup everywhere, you have created a conflict with the main goal. Refactoring in these cases isn't quite as simple as you say, and it certainly won't be backwards compatible. In general, I tried to make Tango for D2 as const aware as possible (it's a standard lib, and shouldn't require people to ignore const), and found all sorts of issues with it (const). The main one that made me quit was the problem that inout is supposed to solve. For someone just trying to "get something working" on D2 from a D1 base, the biggest issue is to deal with string literals since you are forced to deal with const at that point. Other than that, you can usually safely leave everything non-const-ified. That's all I was saying. And I don't think it's worth the effort to try and achieve backwards compatibility with D1. -Steve
Nov 18 2010
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On Thu, 18 Nov 2010 07:34:01 -0500
"Steven Schveighoffer" <schveiguy yahoo.com> wrote:

 4. if you intend on returning a portion of the string, well, currently =20
 you're SOL, but eventually inout will come into play here.
=20
 For example, Tango's log package fell under rules 1 and 3 (most of the =20
 time you are creating log objects with string literals, and logging a =20
 message does not save the message anywhere).
 Many many cases fell under rule 4, which is one of the main reasons I gav=

 up (inout still isn't ready).

I don't understand the issue here. Why don't slices do the job? Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 18 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 18 Nov 2010 14:43:17 -0500, spir <denis.spir gmail.com> wrote:

 On Thu, 18 Nov 2010 07:34:01 -0500
 "Steven Schveighoffer" <schveiguy yahoo.com> wrote:

 4. if you intend on returning a portion of the string, well, currently
 you're SOL, but eventually inout will come into play here.

 For example, Tango's log package fell under rules 1 and 3 (most of the
 time you are creating log objects with string literals, and logging a
 message does not save the message anywhere).
 Many many cases fell under rule 4, which is one of the main reasons I  
 gave
 up (inout still isn't ready).

I don't understand the issue here. Why don't slices do the job?

It's the source of the problem for which inout was invented to solve. How do you specify a function that says "I won't molest the data" but returns a portion of that data? Simplest example is strstr: const(char)[] strstr(const(char)[] haystack, const(char)[] needle) Notice that the return value *must* be const(char)[]. But what if you have a char[]? All of a sudden, you can't modify that data which was returned to you. In essence, strstr modified your const contract with your data. The useful idiom: str = strstr(str, "hello"); doesn't work, because you can't reassign the data. A solution is to have 3 overloads, one for each constancy type, but the mutable version doesn't guarantee it won't modify the data, and you have to repeat the exact same code. Another solution is to use a template, but that also has the same problem that it doesn't prevent the mutable version from modifying the data, plus does not indicate that you can use const data. In addition, templates cannot be virtual functions. But inout solves this in a different way that does exactly what we want. If you have read TDPL, then it should explain how inout is *supposed* to work, or you can read DIP2 (http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP2). inout was chosen instead of vconst to allow reusing essentially a dead keyword. -Steve
Nov 18 2010
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 18 Nov 2010 22:39:31 +0300, Walter Bright  
<newshound2 digitalmars.com> wrote:

 Denis Koroskin wrote:
  From my experience, a bigger issue is that
  version (D2) {
    /* some D2 only code involving const etc */
 }
  simply won't compile in D1, because even if it's D2-only the code  
 needs to be correct D1 code as well.

I agree that's an issue. Currently, the only way to deal with this is one of: 1. Minimize the differences, and maintain two copies of the source code. Using the (rather fabulous) meld tool (available on Linux), the merging is pretty easy. I use meld all the time to, for example, merge differences in the code base between dmd1 and dmd2. 2. Isolate the code that is different into different files, which minimizes the work involved in (1). 3. Use string mixins and the token string literal form. I've also found that using the string alias rather than char[] does account for an awful lot of the differences.

All these are ugly.
Nov 18 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 18 Nov 2010 15:49:10 -0500, Walter Bright  
<newshound2 digitalmars.com> wrote:

 Steven Schveighoffer wrote:
 It's not impossible, but Tango's #1 rule is minimize heap allocations.   
 By peppering idup everywhere, you have created a conflict with the main  
 goal.  Refactoring in these cases isn't quite as simple as you say, and  
 it certainly won't be backwards compatible.

If you know the string is unique, you can use cast(string) rather than .idup, and no allocations will be done.

class obj { char[] name; this(char[] name) { this.name = name;} } What to do there? If name is typically a string literal, then string is appropriate. But if there are cases where a name is passed in via a char[] allocated on the heap, then const(char) makes sense. However, you really don't want the name to change at some other point, so if it's not immutable it should idup the name. It's not easy to make this code both safe and efficient. In the end, I think in cases like this I did something like: string name; this(const(char)[] name) { this.name = name.idup;} this(string name) {this.name = name;} These are the kinds of cases I was talking about. They require more than simple "search and replace" refactoring. In reality, the original code is unsafe and flawed, but you want to try and support any existing code that uses it as well as possible. There was also some anti-D2 sentiments to overcome when trying to get Tango to accept a D2 port, so I wanted to try and make the port as painless as possible. -Steve
Nov 18 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday 18 November 2010 13:02:40 Steven Schveighoffer wrote:
 On Thu, 18 Nov 2010 15:49:10 -0500, Walter Bright
 
 <newshound2 digitalmars.com> wrote:
 Steven Schveighoffer wrote:
 It's not impossible, but Tango's #1 rule is minimize heap allocations.
 By peppering idup everywhere, you have created a conflict with the main
 goal.  Refactoring in these cases isn't quite as simple as you say, and
 it certainly won't be backwards compatible.

If you know the string is unique, you can use cast(string) rather than .idup, and no allocations will be done.

class obj { char[] name; this(char[] name) { this.name = name;} } What to do there? If name is typically a string literal, then string is appropriate. But if there are cases where a name is passed in via a char[] allocated on the heap, then const(char) makes sense. However, you really don't want the name to change at some other point, so if it's not immutable it should idup the name. It's not easy to make this code both safe and efficient. In the end, I think in cases like this I did something like: string name; this(const(char)[] name) { this.name = name.idup;} this(string name) {this.name = name;} These are the kinds of cases I was talking about. They require more than simple "search and replace" refactoring. In reality, the original code is unsafe and flawed, but you want to try and support any existing code that uses it as well as possible. There was also some anti-D2 sentiments to overcome when trying to get Tango to accept a D2 port, so I wanted to try and make the port as painless as possible.

And, of course, the other issue is the ideally, most code which uses strings would work with all of the various string types, so if you want to go all the way, you have to templatize up the wazoo on string functions. - Jonathan M Davis
Nov 18 2010
prev sibling next sibling parent Fawzi Mohamed <fawzi gmx.ch> writes:
 From the discussion it seems that defining something like:

version(D_Version2){
	template Const(T){
		alias const(T) Const;
	}
	template Immutable(T){
		alias immutable(T) Immutable;
	}
	immutable(T) Idup(T)(T val){
		return val.idup;
	}
	alias const(char)[] cstring;
} else {
	template Const(T){
		alias T Const;
	}
	template Immutable(T){
		alias T Immutable;
	}
	T Idup(T)(T val){
		return val.dup;
	}
	alias char[] string;
	alias char[] cstring;
}

could help a lot
later one can simply replace Const! with const and Immutable! with  
immutable, Idup replacement is more complicated, but doable
What is not so clear is how to cope with scope, because I have lot of  
delegates around that will need it.
For it a preprocessing step might really be the best thing, or someone  
knows a smart way to cope with that?
Nov 18 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 19 Nov 2010 01:47:27 -0500, Fawzi Mohamed <fawzi gmx.ch> wrote:

  From the discussion it seems that defining something like:

 version(D_Version2){
 	template Const(T){
 		alias const(T) Const;
 	}
 	template Immutable(T){
 		alias immutable(T) Immutable;
 	}
 	immutable(T) Idup(T)(T val){
 		return val.idup;
 	}
 	alias const(char)[] cstring;
 } else {
 	template Const(T){
 		alias T Const;
 	}
 	template Immutable(T){
 		alias T Immutable;
 	}
 	T Idup(T)(T val){
 		return val.dup;
 	}
 	alias char[] string;
 	alias char[] cstring;
 }

 could help a lot
 later one can simply replace Const! with const and Immutable! with  
 immutable, Idup replacement is more complicated, but doable
 What is not so clear is how to cope with scope, because I have lot of  
 delegates around that will need it.
 For it a preprocessing step might really be the best thing, or someone  
 knows a smart way to cope with that?

BTW, this doesn't work as well as you would think. The problem is with IFTI. For example, IFTI can handle this: void foo(T)(const(T) arg) but cannot handle this: void foo(T)(Const!(T) arg) It was one of the things we were trying to do in Tango for D2, we had aliases for const/immutable/mutable strings templated on character type, but all of Tango's IFTI usage failed to compile. It's a really old bug I filed: http://d.puremagic.com/issues/show_bug.cgi?id=1653 which was superseded by: http://d.puremagic.com/issues/show_bug.cgi?id=1807 -Steve
Nov 19 2010
prev sibling next sibling parent Bernard Helyer <b.helyer gmail.com> writes:
On Thu, 18 Nov 2010 19:58:42 +0000, %u wrote:

 == Quote from Denis Koroskin (2korden gmail.com)'s article
 On Thu, 18 Nov 2010 22:19:12 +0300, Walter Bright
 <newshound2 digitalmars.com> wrote:
 Steven Schveighoffer wrote:
 My recommendation -- when you are ready, switch wholly to D2.  Don't
 bother with compatibility, it's just not possible.

From what you wrote, it appears that most of the difficulties were in dealing with strings.

/* some D2 only code involving const etc */ } simply won't compile in D1, because even if it's D2-only the code needs to be correct D1 code as well.

Wow, that sucks!! Can't this simply be mended with special D_DMD1 and D_DMD2 (D_DMD3 :) version identifiers? If the compiler version isn't the same then it won't look at the code.

No. The contents of a version block is still parsed. If the D2 code doesn't _parse_ as valid D1, then the D1 compiler will reject it, even if the version block will not be processed further than that.
Nov 19 2010
prev sibling next sibling parent Andrew Wiley <debio264 gmail.com> writes:
--001636c59a2e82cb850495710b8b
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Nov 19, 2010 at 6:18 PM, Walter Bright
<newshound2 digitalmars.com>wrote:

 Bernard Helyer wrote:

 No. The contents of a version block is still parsed. If the D2 code
 doesn't _parse_ as valid D1, then the D1 compiler will reject it, even if
 the version block will not be processed further than that.

Yes. Now, if the D1 compiler is supposed to successfully lex & parse (and then ignore) false version blocks, it becomes impossible for a D1 compiler to be stable. It will have to track every future D language variant, missteps and all.

I'm a fan of parsing false version blocks, but it seems like there should be some sort of exception for version blocks for a different version of the language. Could they just be skipped with a warning that an unknown version of the language is in use? It's somewhat hackish, but to me, that seems better than parsing it and giving an error because of syntax changes. --001636c59a2e82cb850495710b8b Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <div class=3D"gmail_quote">On Fri, Nov 19, 2010 at 6:18 PM, Walter Bright <= span dir=3D"ltr">&lt;<a href=3D"mailto:newshound2 digitalmars.com">newshoun= d2 digitalmars.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quot= e" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"=

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> No. The contents of a version block is still parsed. If the D2 code doesn&#= 39;t _parse_ as valid D1, then the D1 compiler will reject it, even if the = version block will not be processed further than that.<br> </blockquote> <br></div> Yes. Now, if the D1 compiler is supposed to successfully lex &amp; parse (a= nd then ignore) false version blocks, it becomes impossible for a D1 compil= er to be stable. It will have to track every future D language variant, mis= steps and all.<br> </blockquote><div><br></div><div>I&#39;m a fan of parsing false version blo= cks, but it seems like there should be some sort of exception for version b= locks for a different version of the language. Could they just be skipped w= ith a warning that an unknown version of the language is in use? It&#39;s s= omewhat hackish, but to me, that seems better than parsing it and giving an= error because of syntax changes.=A0</div> </div><br> --001636c59a2e82cb850495710b8b--
Nov 19 2010
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Sat, 20 Nov 2010 04:14:05 +0300, Walter Bright  
<newshound2 digitalmars.com> wrote:

 Andrew Wiley wrote:
 I'm a fan of parsing false version blocks, but it seems like there  
 should be some sort of exception for version blocks for a different  
 version of the language. Could they just be skipped with a warning that  
 an unknown version of the language is in use? It's somewhat hackish,  
 but to me, that seems better than parsing it and giving an error  
 because of syntax changes.

How can it skip it without knowing the grammar?

#ifdef D_Version2 ... #else ... #endif D has a precedence of having preprocessor macros (e.g #line) even if it's not a full-fledged preprocessor, and I don't see why it couldn't adapt the same solution. FWIW, C# has #if blocks, too.
Nov 19 2010
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Sat, 20 Nov 2010 04:33:36 +0300, Walter Bright  
<newshound2 digitalmars.com> wrote:

 Denis Koroskin wrote:
 On Sat, 20 Nov 2010 04:14:05 +0300, Walter Bright
 How can it skip it without knowing the grammar?

... #else ... #endif D has a precedence of having preprocessor macros (e.g #line) even if it's not a full-fledged preprocessor, and I don't see why it couldn't adapt the same solution. FWIW, C# has #if blocks, too.

Yes, I know it can be done with a text preprocessor. [[ Queue run away screaming ]]

Once again, it's NOT a text preprocessor. It's NOT a separate parsing phase (unlike C/C++). It's done at parsing level, similarly to comments.
Nov 19 2010
prev sibling parent Andrew Wiley <debio264 gmail.com> writes:
--20cf3054a69555143c0495744646
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Nov 19, 2010 at 7:14 PM, Walter Bright
<newshound2 digitalmars.com>wrote:

 Andrew Wiley wrote:

 I'm a fan of parsing false version blocks, but it seems like there should
 be some sort of exception for version blocks for a different version of the
 language. Could they just be skipped with a warning that an unknown version
 of the language is in use? It's somewhat hackish, but to me, that seems
 better than parsing it and giving an error because of syntax changes.

How can it skip it without knowing the grammar?

It's probably too much work for too little gain, but would it be possible to basically just scan for matching braces until the version block ends? If they always come in matching pairs (or at least they *should*), you could scan for them and ignore everything else until you find the end of the block. --20cf3054a69555143c0495744646 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <div class=3D"gmail_quote">On Fri, Nov 19, 2010 at 7:14 PM, Walter Bright <= span dir=3D"ltr">&lt;<a href=3D"mailto:newshound2 digitalmars.com">newshoun= d2 digitalmars.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quot= e" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"=

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I&#39;m a fan of parsing false version blocks, but it seems like there shou= ld be some sort of exception for version blocks for a different version of = the language. Could they just be skipped with a warning that an unknown ver= sion of the language is in use? It&#39;s somewhat hackish, but to me, that = seems better than parsing it and giving an error because of syntax changes.= <br> </blockquote> <br> <br></div> How can it skip it without knowing the grammar?<br> </blockquote></div><br><div>It&#39;s probably too much work for too little = gain, but would it be possible to basically just scan for matching braces u= ntil the version block ends? If they always come in matching pairs (or at l= east they *should*), you could scan for them and ignore everything else unt= il you find the end of the block.</div> --20cf3054a69555143c0495744646--
Nov 19 2010