www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Transitions to new language semantics

reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
This is something that should have been discussed already, but I can't 
remember whether that was actually the case, and it always bothers me 
every time there is friction with new DIP switches.

Right now, new language semantics are introduced using `-preview` and 
`-revert` switches, which unfortunately has a massive drawback:

Command line flags work on a per-compiler-invocation basis, so that, 
depending on the set of files passed to the compiler, they affect code 
that they shouldn't, or don't affect the code that they should. This can 
be worked around to some degree by grouping files appropriately, but 
breaks for many language changes as soon as imports between these groups 
are involved.

Now, what I've always thought to be the obvious solution for this is to 
instead annotate the code on a per-module basis, using UDA syntax, 
pragmas, or something similar. In addition to avoiding import issues 
with external code, that would also allow to upgrade large code bases on 
a per-module basis.


      semantic("dip1000")  semantic("default-to-safe")
      disableSemantic("dip999")
     module foo;


     module foo;
     pragma(semantic, "dip1000");
     pragma(semantic, "default-to-safe");
     pragma(disableSemantic, "dip999");


The deprecation path would look like this:

1. new semantic is optionally available
2. new semantic enabled by default, mention in error messages that
    disabling is still possible
3. deprecate disabling the new semantic
4. make disabling an error
5. deprecate explicitly enabling the semantic
6. make enabling an error, or just keep it indefinitely


Finally, I think that this would considerably impact the rate of 
adoption during the preview phase, which I guess currently is extremely 
low due to the friction with external code and the need to upgrade full 
code bases at once.

I'm aware that this will require a different approach for the 
implementation and probably only makes sense for new DIPs, but it would 
also make some simple but long overdue changes, such as changing 
defaults to ` safe nothrow` (`final`?), a lot easier.
Jun 11 2021
next sibling parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
This was the most recent example of the issue:

https://github.com/vibe-d/vibe-core/issues/284
Jun 11 2021
parent reply singingbush <singingbush hotmail.com> writes:
There have been so many breaking changes in D that I've kept a 
few useful version checks in this gist: 
https://gist.github.com/SingingBush/3a2a6d2a41a81c1bafb26e5f69f823c0

for things like

```
     static if (__VERSION__ < 2074) {
         import std.traits : FieldTypeTuple, Filter; // Filter 
used to be in std.traits
     } else {
         import std.traits : FieldTypeTuple;
         import std.meta : Filter;
     }
```

and

```
     // JSON_TYPE is now JSONType simply due to code style: 
https://issues.dlang.org/show_bug.cgi?id=19135
     static if(__VERSION__ >= 2082)
         import std.json : JSONType;
     else
         alias JSONType = std.json.JSON_TYPE;
```

this is one of my main problems with using D. There are breaking 
changes far to often. Generally with little to no overlap 
allowing time for developers to update code.

With GDC only supporting D 2.076 and the latest D release being 
2.097 there's a whole range of D versions that dub packages 
should support if they are to work across all compilers. Sadly 
though, very few will work with a range of D versions that wide.

While ever this statement remains true; "D code written with 
latest dmd today may not compile with latest dmd in 12 months 
time", then D cannot really be considered for use by most 
organisations. This is the biggest problem with the D ecosystem
Jun 11 2021
next sibling parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Friday, 11 June 2021 at 08:07:57 UTC, singingbush wrote:

 There have been so many breaking changes in D that I've kept a 
 few useful version checks in this gist: 
 https://gist.github.com/SingingBush/3a2a6d2a41a81c1bafb26e5f69f823c0

 [...]


Agreed. Stability/reliability is of most importance in any 
system. Without it you can't build anything of value.
Jun 11 2021
prev sibling parent reply Luis <Luis.panadero gmail.com> writes:
On Friday, 11 June 2021 at 08:07:57 UTC, singingbush wrote:

 With GDC only supporting D 2.076 and the latest D release being 
 2.097 there's a whole range of D versions that dub packages 
 should support if they are to work across all compilers. Sadly 
 though, very few will work with a range of D versions that wide.


GDC 10/11 isn't supporting D 2.081 ? 
https://wiki.dlang.org/GDC#Status
Jun 11 2021
parent mori <stigma disroot.org> writes:
On 11/6/21 6:31 pm, Luis wrote:

 GDC 10/11 isn't supporting D 2.081 ? https://wiki.dlang.org/GDC#Status
 


In the recent GCC 11.1 announcement, Iain said the baseline is still 
2.076.1:

   https://forum.dlang.org/post/yyblavikfluxsbtrdxry forum.dlang.org

In the same post, Iain also said that GCC 12 should support the latest 
DMD front-end at the time of release, which -- as someone who primarily 
uses GDC -- would be nice.
Jun 11 2021
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/11/2021 12:36 AM, Sönke Ludwig wrote:

 Now, what I've always thought to be the obvious solution for this is to
instead 
 annotate the code on a per-module basis, using UDA syntax, pragmas, or
something 
 similar. In addition to avoiding import issues with external code, that would 
 also allow to upgrade large code bases on a per-module basis.


Interestingly, this is how #ImportC is working.
Jun 11 2021
prev sibling next sibling parent WebFreak001 <d.forum webfreak.org> writes:
On Friday, 11 June 2021 at 07:36:47 UTC, Sönke Ludwig wrote:

 [...]


      semantic("dip1000")  semantic("default-to-safe")
      disableSemantic("dip999")
     module foo;


     module foo;
     pragma(semantic, "dip1000");
     pragma(semantic, "default-to-safe");
     pragma(disableSemantic, "dip999");


 [...]

 Finally, I think that this would considerably impact the rate 
 of adoption during the preview phase, which I guess currently 
 is extremely low due to the friction with external code and the 
 need to upgrade full code bases at once.

 I'm aware that this will require a different approach for the 
 implementation and probably only makes sense for new DIPs, but 
 it would also make some simple but long overdue changes, such 
 as changing defaults to ` safe nothrow` (`final`?), a lot 
 easier.


I would love this! I agree with your points and I think adoption 
(and bug finding) could be significantly improved by something 
like this.
Jun 11 2021
prev sibling next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 11 June 2021 at 07:36:47 UTC, Sönke Ludwig wrote:

 This is something that should have been discussed already, but 
 I can't remember whether that was actually the case, and it 
 always bothers me every time there is friction with new DIP 
 switches.

 Right now, new language semantics are introduced using 
 `-preview` and `-revert` switches, which unfortunately has a 
 massive drawback:


I am troubled in general by the implementation of incomplete 
solutions and making them gradually available.

I would find it much more reassuring if a comprehensive solution 
was developed as a completely separate compiler branch. Basically 
have a stable branch (as is), and then a future branch that is 
considered unstable until all the corner cases have been ironed 
out. This also allows more heavy restructuring of compiler 
internals, like introducing an appropriate IR (which is needed 
for things like borrowing or ARC, if you want something solid).

The cost of moving to a more complete solution after something 
incomplete has been made official could break the camel's back.

The piece-by-piece approach is a slippery slope.
Jun 11 2021
next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Friday, 11 June 2021 at 11:27:03 UTC, Ola Fosheim Grøstad 
wrote:

 I would find it much more reassuring if a comprehensive 
 solution was developed as a completely separate compiler branch.
 future branch that is considered unstable until all the corner 
 cases have been ironed out.
 This also allows more heavy restructuring of compiler 
 internals, like introducing an appropriate IR (which is needed 
 for things like borrowing or ARC, if you want something solid).


This would be short-sighted. It'd mean that the experimental 
feature developers would then have to backport all the compiler 
improvements that have been done while the feature was 
experimental. Its easier to account for the experimental features 
when doing the restructuring.

Second, if using an experimental feature requires compiling a 
separate compiler branch, not as many will use it. Hence, less 
real world testing, and even less issues ironed out.


 The cost of moving to a more complete solution after something 
 incomplete has been made official could break the camel's back.

 The piece-by-piece approach is a slippery slope.


We already have a three-round DIP process to catch issues before 
we make the new feature official. We probably need to be more 
explicit about what is still experimental and what is official, 
though. I think it wouldn't hurt to document that status to each 
of the preview switches in DMD.
Jun 11 2021
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 11 June 2021 at 11:48:22 UTC, Dukc wrote:

 This would be short-sighted. It'd mean that the experimental 
 feature developers would then have to backport all the compiler 
 improvements that have been done while the feature was 
 experimental. Its easier to account for the experimental 
 features when doing the restructuring.


I would go the opposite direction. Improvements in the unstable 
branch that has proven itself stable and with low semantic impact 
(like bugfixes and security fixes) would be backported as a 
revision of the stable branch.


 Second, if using an experimental feature requires compiling a 
 separate compiler branch, not as many will use it. Hence, less 
 real world testing, and even less issues ironed out.


It should be compiled and packaged as a nightly delivery. 
(Automated)
Jun 11 2021
parent reply zjh <fqbqrr 163.com> writes:
We have `enough features` in D.
But, one have to use `d` carefully in production,since it is not 
stable enough.
We need a stable D version. Then gradually experiment.
Rather than only the latest version one. Although the development 
is fast, there are a lot of bugs. This scares away beginners.
I think after D teams finished `ImportC`, we should first create 
a stable version.
This stable version lasts for 2 years, mainly fixes bugs.
And the new and latest version, `d team` can do `BIG change`. 
Anyway, we have a stable one available.
In my opinion, the `Big change` is to divide large files into 
small files, so that the granularity of source files is smaller 
and the dependency relationship is clearer. By the way,solving 
the problem of GC .So it's less likely to have fatal errors.
This way,many people would like to join `the d development`.
Jun 11 2021
next sibling parent zjh <fqbqrr 163.com> writes:
On Friday, 11 June 2021 at 13:10:23 UTC, zjh wrote:

 We have `enough features` in D.


Normal user using features when they are immature,resulting only 
increasing the burden of users.
A stable `d` build then preventing your from immature features.
Jun 11 2021
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 11 June 2021 at 13:10:23 UTC, zjh wrote:

 In my opinion, the `Big change` is to divide large files into 
 small files, so that the granularity of source files is smaller 
 and the dependency relationship is clearer.


There is really no need to start with a big change. The best 
approach is to:

1. First insulate independent parts. So, insulate the backend 
from the front end, by putting a layer between (most likely a 
high level IR).

2. Then move independent things like inlining out of the frontend 
and onto the middle layer.

3. Modularize.

4. Iterate between redesigning interfaces / refactoring frontend 
internals.

The frontend really only need to deal with type unification, 
templates and CRFE. Maybe I missed something, but roughly.

When you have reduced the complexity of the front end then you 
can start to modularize it. Otherwise you end up doing the job 
twice.
Jun 11 2021
parent reply zjh <fqbqrr 163.com> writes:
On Friday, 11 June 2021 at 13:36:27 UTC, Ola Fosheim Grøstad 
wrote:

 On Friday, 11 June 2021 at 13:10:23 1. First insulate 
 independent parts. So, insulate the backend from the front end, 
 by putting a layer between (most likely a high level IR).


D team members should have a listen to your advice.
Your suggestion are always very good.
Jun 11 2021
parent russhy <russhy gmail.com> writes:
i disagree, polluting code with compiler specific sheningans is 
the root of all evil

don't bloat code that is already hard to read for features people 
want to test

it's meant to be transitive, something temporary, not something 
that needs to live on people's code

let's just stop with the   bloat


other aspect of D needs love, it certainly not the -preview 
thing, that is a pure waste of time
Jun 11 2021
prev sibling next sibling parent WebFreak001 <d.forum webfreak.org> writes:
On Friday, 11 June 2021 at 11:27:03 UTC, Ola Fosheim Grøstad 
wrote:

 On Friday, 11 June 2021 at 07:36:47 UTC, Sönke Ludwig wrote:

 This is something that should have been discussed already, but 
 I can't remember whether that was actually the case, and it 
 always bothers me every time there is friction with new DIP 
 switches.

 Right now, new language semantics are introduced using 
 `-preview` and `-revert` switches, which unfortunately has a 
 massive drawback:


 I am troubled in general by the implementation of incomplete 
 solutions and making them gradually available.

 I would find it much more reassuring if a comprehensive 
 solution was developed as a completely separate compiler 
 branch. Basically have a stable branch (as is), and then a 
 future branch that is considered unstable until all the corner 
 cases have been ironed out. This also allows more heavy 
 restructuring of compiler internals, like introducing an 
 appropriate IR (which is needed for things like borrowing or 
 ARC, if you want something solid).

 The cost of moving to a more complete solution after something 
 incomplete has been made official could break the camel's back.

 The piece-by-piece approach is a slippery slope.


I think this is a good point.

Having an unstable compiler would allow removing things that 
don't work so well again, while having the advantage of having a 
big user base trying it out (and complaining about bugs) - though 
this could also be seen as inconvenience for users and authors 
e.g. when libraries are only working with an unstable branch or 
change in behavior with the unstable compiler.
Jun 11 2021
prev sibling next sibling parent reply surlymoor <surlymoor cock.li> writes:
On Friday, 11 June 2021 at 11:27:03 UTC, Ola Fosheim Grøstad 
wrote:

 On Friday, 11 June 2021 at 07:36:47 UTC, Sönke Ludwig wrote:

 [...]


 I am troubled in general by the implementation of incomplete 
 solutions and making them gradually available.

 I would find it much more reassuring if a comprehensive 
 solution was developed as a completely separate compiler 
 branch. Basically have a stable branch (as is), and then a 
 future branch that is considered unstable until all the corner 
 cases have been ironed out. This also allows more heavy 
 restructuring of compiler internals, like introducing an 
 appropriate IR (which is needed for things like borrowing or 
 ARC, if you want something solid).

 The cost of moving to a more complete solution after something 
 incomplete has been made official could break the camel's back.

 The piece-by-piece approach is a slippery slope.


Doesn't Rust do something like this? A problem I read concerning 
its ecosystem is the tendency to target nightly, and thus using a 
more stable branch of the compiler leaves one high and dry, so to 
speak.
Jun 11 2021
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 11 June 2021 at 11:57:03 UTC, surlymoor wrote:

 Doesn't Rust do something like this? A problem I read 
 concerning its ecosystem is the tendency to target nightly, and 
 thus using a more stable branch of the compiler leaves one high 
 and dry, so to speak.


I don't know. But I think some languages (Ada?) have different 
feature-profiles. D does also have a profile called "BetterC".

So, one could define a "maximal compatible" feature profile for D 
and use a linter to verify that a library stays within that 
profile.

Say, if you wanted to write a geometry library, then you might 
want to target BetterC as the minimum profile, but use 
conditional version statements to enable more features for other 
profiles.

D as a language have the mechanisms, I think, but it isn't used 
effectively as it requires someone to work out a "standard way" 
of writing libraries.
Jun 11 2021
prev sibling parent reply mw <mingwu gmail.com> writes:
 I would find it much more reassuring if a comprehensive 
 solution was developed as a completely separate compiler 
 branch. Basically have a stable branch (as is), and then a 
 future branch that is considered unstable until all the corner 
 cases have been ironed out.


Separating `stable` release vs `development` experiment branch is 
something that have been discussed on this forum for so many 
times, (last time I remember is `safe` vs `system` by default 
discussion).

I think what we need is action! I.e take it into action. 
Seriously.
Jun 11 2021
parent zjh <fqbqrr 163.com> writes:
On Friday, 11 June 2021 at 21:59:21 UTC, mw wrote:

 I would find it much more reassuring if a comprehensive




We first need orgnazition,random action is of no use.
D need organization,dispatch tasks,Repeate 3 times.
Jun 11 2021
prev sibling next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Friday, 11 June 2021 at 07:36:47 UTC, Sönke Ludwig wrote:

 Now, what I've always thought to be the obvious solution for 
 this is to instead annotate the code on a per-module basis, 
 using UDA syntax, pragmas, or something similar. In addition to 
 avoiding import issues with external code, that would also 
 allow to upgrade large code bases on a per-module basis.


Same here. Great idea IMO. However, we have to acknowledge that 
each feature that can be enabled per-module basis needs to 
consider how old and new semantics act together. For example, if 
we want to deprecate autodecoding:

```
 semanticDisable("implicitByCodeUnit") module a;

auto foo(const char(x))
{ import std;
   return x.map!(/*...*/);
}
```

```
 semantic("implicitByCodeUnit") module b;

void main()
{ import b;
   auto var = "50€".foo; //var iterated by code unit or by code 
point?
}
```
Jun 11 2021
parent Dukc <ajieskola gmail.com> writes:
On Friday, 11 June 2021 at 11:27:20 UTC, Dukc wrote:

 ```
  semantic("implicitByCodeUnit") module b;

 void main()
 { import b;
   auto var = "50€".foo; //var iterated by code unit or by code 
 point?
 }
 ```


should be

```
 semantic("implicitByCodeUnit") module b;

void main()
{ import a;
   auto var = "50€".foo; //"50€" iterated by code unit or by code
point?
}
```
Jun 11 2021
prev sibling next sibling parent Mathias LANG <geod24 gmail.com> writes:
On Friday, 11 June 2021 at 07:36:47 UTC, Sönke Ludwig wrote:

 This is something that should have been discussed already, but 
 I can't remember whether that was actually the case, and it 
 always bothers me every time there is friction with new DIP 
 switches.

 [...]


I agree per-module UDAs would be nice.

We'd have to be careful about templates though: Currently the 
emission strategy is that if a template can be found in a root 
module, it will not be codegened (which is good because codegen 
is slow). With per-module UDAs we'd probably have to emit 
defensively when different attributes are used, and have to 
mangle the attributes into the template (to avoid them being 
folded despite being different).
This could become quite impactful if e.g. the standard library is 
shipped with a different default (try to use `-allinst` and 
you'll see a massive slowdown).

Another thing is that `-preview` switch are, for the most part, 
not finished, nor are they made compatible with libraries. I 
think every `-preview` switch should come with its "Enable by 
default" draft PR to see how much breaks on Buildkite, and those 
failures should mostly be fixed, so that users have a much better 
experience.
Jun 11 2021
prev sibling parent sighoya <sighoya gmail.com> writes:
On Friday, 11 June 2021 at 07:36:47 UTC, Sönke Ludwig wrote:

```D
  semantic("dip1000")  semantic("default-to-safe")
  disableSemantic("dip999")
 module foo;
```


+1, but a possible problem with this is scoping. How does a 
variable interact with another being rule differently.
Jun 12 2021