www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DConf Hackathon Ideas

reply Mike Parker <aldacron gmail.com> writes:
This year, DConf has an extra day tacked on for problem solving 
in the form of a hackathon. The intent is to work on issues 
people find frustrating in the D ecosystem. While there will be 
time given at the event for proposals, and those involving 
third-party projects are welcome, it will help speed things along 
for the organizers to come in with a list of big issues to get 
the ball rolling.

To help in compiling the list, what are some major issues from 
the ecosystem that you'd like to see fixed?
Apr 27
next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:
 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?
* multi-threaded parsing in DMD * https://github.com/dlang/dub/issues/838 * finishing std.experimental.xml https://github.com/dlang/phobos/pull/4741 * Docs which allow people to go back and see docs for previous versions. Huge headache when using GDC or LDC
Apr 27
parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 27 April 2017 at 17:15, Jack Stouffer via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 * Docs which allow people to go back and see docs for previous versions.
 Huge headache when using GDC or LDC
Maybe you could submit a patch to add build hook to generate the documentation for GDC. It would be a welcome contribution to add to the current bulk of documentation I'm starting to build up. ;-)
Apr 27
prev sibling next sibling parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:
 This year, DConf has an extra day tacked on for problem solving 
 in the form of a hackathon. The intent is to work on issues 
 people find frustrating in the D ecosystem. While there will be 
 time given at the event for proposals, and those involving 
 third-party projects are welcome, it will help speed things 
 along for the organizers to come in with a list of big issues 
 to get the ball rolling.

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?
For starters: - https://github.com/dlang/dub/issues/104 - https://github.com/dlang/dub-registry/issues/117 I don't think the following ones are feasible to deal with in that context, but they are ones that irk me in the ecosystem: - ABI compatibility between D compilers with the same frontend version. - More than one official package description language (json and sdlang). I honestly don't care which it ends up being, but please pick *one* (I am aware of the previous discussions on the topic, but the current state of supporting both is just one more point of friction for newcomers)
Apr 27
parent reply singingbush <singingbush hotmail.com> writes:
 - More than one official package description language (json and 
 sdlang). I honestly don't care which it ends up being, but 
 please pick *one* (I am aware of the previous discussions on 
 the topic, but the current state of supporting both is just one 
 more point of friction for newcomers)
SDL should be dropped. for starters it's poorly supported. just go to https://sdlang.org/ and scroll to the links at the bottom. official homepage and reference are both broken urls (as http://sdl.ikayzo.org/ is offline) and needs mirrors. Then there's the fact that there's not very good support for SDL in other languages, sure the website says there are implementations in other languages but has anyone tried using them? I did recently, the Java version (actually written by ikayzo - https://github.com/ikayzo/SDL) will not even parse all the examples on the homepage! not to mention it being a dead project with no response to issues (https://github.com/ikayzo/SDL/issues). As far as I can tell the only reason dub defaults to sdl now is because s-ludwig maintains the sdlang.org website. at least json is universally known and can already be handled by just about any tool. If we really want to have an alternative it would at least make sense to use something like YAML that is also heavily used and supported by all IDE's. It would save me the hassle of writing a lexer for SDL - which of course has no support in my tool of choice because... NOBODY USES IT!
Apr 27
next sibling parent Moritz Maxeiner <moritz ucworks.org> writes:
On Thursday, 27 April 2017 at 16:33:02 UTC, singingbush wrote:
 As far as I can tell the only reason dub defaults to sdl now
It did in the past, but hasn't done so for several minor versions (it defaults to json).
Apr 27
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 27 April 2017 at 16:33:02 UTC, singingbush wrote:
 SDL should be dropped.
Deprecated, sure. But dropping it seems a bad idea given that various projects do still use it for their DUB package config.
 NOBODY USES IT!
Probably not true. Perhaps a hackathon project could be to create a little app to find which projects on GitHub (or at least code.dlang.org) still use a `dub.sdl`, and auto-submit a PR to fix that? :-)
May 01
parent reply Mike Parker <aldacron gmail.com> writes:
On Monday, 1 May 2017 at 14:38:11 UTC, Joseph Rushton Wakeling 
wrote:
 On Thursday, 27 April 2017 at 16:33:02 UTC, singingbush wrote:
 SDL should be dropped.
Deprecated, sure. But dropping it seems a bad idea given that various projects do still use it for their DUB package config.
 NOBODY USES IT!
Probably not true. Perhaps a hackathon project could be to create a little app to find which projects on GitHub (or at least code.dlang.org) still use a `dub.sdl`, and auto-submit a PR to fix that? :-)
I love SDL and much prefer it over JSON for DUB configs. Use it for all of my D projects. It looks cleaner and supports comments. I really would hate to see support dropped.
May 01
next sibling parent reply bachmeier <no spam.net> writes:
On Monday, 1 May 2017 at 14:51:19 UTC, Mike Parker wrote:

 I love SDL and much prefer it over JSON for DUB configs. Use it 
 for all of my D projects. It looks cleaner and supports 
 comments. I really would hate to see support dropped.
I'm not even sure what it would mean to "drop" support for SDL. As long as someone decides to support it, it will be supported. SDL to my knowledge is not an "official" D project.
May 01
parent Mike Parker <aldacron gmail.com> writes:
On Monday, 1 May 2017 at 15:36:16 UTC, bachmeier wrote:
 On Monday, 1 May 2017 at 14:51:19 UTC, Mike Parker wrote:

 I love SDL and much prefer it over JSON for DUB configs. Use 
 it for all of my D projects. It looks cleaner and supports 
 comments. I really would hate to see support dropped.
I'm not even sure what it would mean to "drop" support for SDL. As long as someone decides to support it, it will be supported. SDL to my knowledge is not an "official" D project.
Well, sure, but we were specifically talking about support in DUB. http://forum.dlang.org/post/eejeorhqgwxbduunmenh forum.dlang.org
May 01
prev sibling next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 May 2017 at 16:51, Mike Parker via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 1 May 2017 at 14:38:11 UTC, Joseph Rushton Wakeling wrote:
 On Thursday, 27 April 2017 at 16:33:02 UTC, singingbush wrote:
 SDL should be dropped.
Deprecated, sure. But dropping it seems a bad idea given that various projects do still use it for their DUB package config.
 NOBODY USES IT!
Probably not true. Perhaps a hackathon project could be to create a little app to find which projects on GitHub (or at least code.dlang.org) still use a `dub.sdl`, and auto-submit a PR to fix that? :-)
I love SDL and much prefer it over JSON for DUB configs. Use it for all of my D projects. It looks cleaner and supports comments. I really would hate to see support dropped.
We should make XML the default config format for DUB. <?xml version='1.0' encoding='utf-8'?> <dub xmlns="http://code.dlang.org/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <project name="gdcproject" description="The vibe.d server application running gdcproject.org." copyright="Copyright © 2014, Iain Buclaw"> <authors> <author name="Iain Buclaw"> </authors> <dependencies> <dependency name="vibe-d" version="0.7.22"> <dependency name="mustache-d" version="0.1.0"> </dependencies> <versions> <version name="VibeDefaultMain"> </versions> </project> </dub> /Runaway!
May 01
next sibling parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Monday, 1 May 2017 at 17:04:42 UTC, Iain Buclaw wrote:
 [...]

 We should make XML the default config format for DUB.

 [...]

 /Runaway!
Well, at least nearly everyone hates XML equally, which may be an indicator of a good compromise.
May 01
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Mon, May 01, 2017 at 06:02:53PM +0000, Moritz Maxeiner via Digitalmars-d
wrote:
 On Monday, 1 May 2017 at 17:04:42 UTC, Iain Buclaw wrote:
 [...]
 
 We should make XML the default config format for DUB.
 
 [...]
 
 /Runaway!
Well, at least nearly everyone hates XML equally, which may be an indicator of a good compromise.
Whoa. XML for DUB? That may just be the final nail in the coffin for me ever picking up DUB in this lifetime. ;-) T -- Dogs have owners ... cats have staff. -- Krista Casada
May 01
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 1 May 2017 at 17:04:42 UTC, Iain Buclaw wrote:
 On 1 May 2017 at 16:51, Mike Parker via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On Monday, 1 May 2017 at 14:38:11 UTC, Joseph Rushton Wakeling 
 wrote:
 On Thursday, 27 April 2017 at 16:33:02 UTC, singingbush wrote:
 SDL should be dropped.
Deprecated, sure. But dropping it seems a bad idea given that various projects do still use it for their DUB package config.
 NOBODY USES IT!
Probably not true. Perhaps a hackathon project could be to create a little app to find which projects on GitHub (or at least code.dlang.org) still use a `dub.sdl`, and auto-submit a PR to fix that? :-)
I love SDL and much prefer it over JSON for DUB configs. Use it for all of my D projects. It looks cleaner and supports comments. I really would hate to see support dropped.
We should make XML the default config format for DUB. <?xml version='1.0' encoding='utf-8'?> <dub xmlns="http://code.dlang.org/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <project name="gdcproject" description="The vibe.d server application running gdcproject.org." copyright="Copyright © 2014, Iain Buclaw"> <authors> <author name="Iain Buclaw"> </authors> <dependencies> <dependency name="vibe-d" version="0.7.22"> <dependency name="mustache-d" version="0.1.0"> </dependencies> <versions> <version name="VibeDefaultMain"> </versions> </project> </dub> /Runaway!
You forgot a few / there
May 01
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2017-05-01 16:51, Mike Parker wrote:

 I love SDL and much prefer it over JSON for DUB configs. Use it for all
 of my D projects. It looks cleaner and supports comments. I really would
 hate to see support dropped.
+1 I would be fine with YAML as well. -- /Jacob Carlborg
May 01
prev sibling next sibling parent reply Andre Pany <andre s-e-a-p.de> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:
 This year, DConf has an extra day tacked on for problem solving 
 in the form of a hackathon. The intent is to work on issues 
 people find frustrating in the D ecosystem. While there will be 
 time given at the event for proposals, and those involving 
 third-party projects are welcome, it will help speed things 
 along for the organizers to come in with a list of big issues 
 to get the ball rolling.

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?
I would vote for getting the std.decimal proposal from the review queue into a good shape for std.experimental. Another big issue for me is using dub in a company. Big companies do not want to use the official dub repository due to security issues. They want to run their own dub repository with dub packages they have done code checks. That means also the git projects are mirrored in git repositories within the company. First step into this direction is to have the possibility to specify the dub repository address within the dub.json. Kind regards Andre
Apr 27
next sibling parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Thursday, 27 April 2017 at 17:06:39 UTC, Andre Pany wrote:
 Another big issue for me is using dub in a company. Big 
 companies do not want to use the official dub repository due to 
 security issues. They want to run their own dub repository with 
 dub packages they have done code checks. That means also the 
 git projects are mirrored in git repositories within the 
 company. First step into this direction is to have the 
 possibility to specify the dub repository address within the 
 dub.json.
Is there a problem with just using the dub command line arguments for this? # cat > /usr/local/bin/company-dub #! /bin/sh /usr/bin/env dub --skip-registry=standard --registry=YOUR_COMPANYS_REGISTRY
Apr 27
parent reply Andre Pany <andre s-e-a-p.de> writes:
On Thursday, 27 April 2017 at 17:47:19 UTC, Moritz Maxeiner wrote:
 On Thursday, 27 April 2017 at 17:06:39 UTC, Andre Pany wrote:
 Another big issue for me is using dub in a company. Big 
 companies do not want to use the official dub repository due 
 to security issues. They want to run their own dub repository 
 with dub packages they have done code checks. That means also 
 the git projects are mirrored in git repositories within the 
 company. First step into this direction is to have the 
 possibility to specify the dub repository address within the 
 dub.json.
Is there a problem with just using the dub command line arguments for this? # cat > /usr/local/bin/company-dub #! /bin/sh /usr/bin/env dub --skip-registry=standard --registry=YOUR_COMPANYS_REGISTRY
As workaround this is possible. Every developer and in every continious integration build step, dub is used, you have to remember to use these arguments. Defining it one time in dub.json would be great. Kind regards Andre
Apr 27
parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Thursday, 27 April 2017 at 19:55:44 UTC, Andre Pany wrote:
 On Thursday, 27 April 2017 at 17:47:19 UTC, Moritz Maxeiner 
 wrote:
 On Thursday, 27 April 2017 at 17:06:39 UTC, Andre Pany wrote:
 Another big issue for me is using dub in a company. Big 
 companies do not want to use the official dub repository due 
 to security issues. They want to run their own dub repository 
 with dub packages they have done code checks. That means also 
 the git projects are mirrored in git repositories within the 
 company. First step into this direction is to have the 
 possibility to specify the dub repository address within the 
 dub.json.
Is there a problem with just using the dub command line arguments for this? # cat > /usr/local/bin/company-dub #! /bin/sh /usr/bin/env dub --skip-registry=standard --registry=YOUR_COMPANYS_REGISTRY
As workaround this is possible. Every developer and in every continious integration build step, dub is used, you have to remember to use these arguments. Defining it one time in dub.json would be great.
Alright, I take your point. Since the internal functionality itself is already there, this seems like an easy fix (just add a new field to the json/sdlang parsing of dub and bind it to what the CLI arguments already do). Have you opened a dub issue about this (first step to getting an issue fixed and so on), I wasn't able to find one on first glance?
Apr 27
parent Andre Pany <andre s-e-a-p.de> writes:
On Thursday, 27 April 2017 at 22:57:48 UTC, Moritz Maxeiner wrote:
 On Thursday, 27 April 2017 at 19:55:44 UTC, Andre Pany wrote:
 On Thursday, 27 April 2017 at 17:47:19 UTC, Moritz Maxeiner 
 wrote:
 [...]
As workaround this is possible. Every developer and in every continious integration build step, dub is used, you have to remember to use these arguments. Defining it one time in dub.json would be great.
Alright, I take your point. Since the internal functionality itself is already there, this seems like an easy fix (just add a new field to the json/sdlang parsing of dub and bind it to what the CLI arguments already do). Have you opened a dub issue about this (first step to getting an issue fixed and so on), I wasn't able to find one on first glance?
I created an issue in the dub github repository https://github.com/dlang/dub/issues/1119 Kind regards André
May 01
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2017-04-27 19:06, Andre Pany wrote:

 Another big issue for me is using dub in a company. Big companies do not
 want to use the official dub repository due to security issues.
That would be nice. -- /Jacob Carlborg
Apr 28
prev sibling next sibling parent Brad Roberts via Digitalmars-d <digitalmars-d puremagic.com> writes:
The pending pull requests.  In person is a great high-bandwidth way to 
work through the massive backlog.


On 4/27/2017 7:53 AM, Mike Parker via Digitalmars-d wrote:
 This year, DConf has an extra day tacked on for problem solving in the 
 form of a hackathon. The intent is to work on issues people find 
 frustrating in the D ecosystem. While there will be time given at the 
 event for proposals, and those involving third-party projects are 
 welcome, it will help speed things along for the organizers to come in 
 with a list of big issues to get the ball rolling.

 To help in compiling the list, what are some major issues from the 
 ecosystem that you'd like to see fixed?
Apr 27
prev sibling next sibling parent ag0aep6g <anonymous example.com> writes:
On 04/27/2017 04:53 PM, Mike Parker wrote:
 This year, DConf has an extra day tacked on for problem solving in the
 form of a hackathon. The intent is to work on issues people find
 frustrating in the D ecosystem. While there will be time given at the
 event for proposals, and those involving third-party projects are
 welcome, it will help speed things along for the organizers to come in
 with a list of big issues to get the ball rolling.

 To help in compiling the list, what are some major issues from the
 ecosystem that you'd like to see fixed?
I'm most frustrated by regressions and wrong-code bugs in dmd/druntime/phobos (with a focus on dmd). Regressions are annoying, because that stuff used to work, dammit! And wrong-code bugs are annoying, because they can be hard to track down and work around. I don't really have a list of them sorted by importance or impact, but I guess the one's I have reported are, on average, more dear to me than others [1][2]. I haven't reported this one, but it has a special place in my heart, because the miscompiled code is so simple: https://issues.dlang.org/show_bug.cgi?id=15538 Generally, just sorting the issue list by severity and going from the top would probably have a good impact: https://issues.dlang.org/buglist.cgi?bug_status=__open__&columnlist=component%2Cassigned_to%2Cbug_status%2Cbug_severity%2Cshort_desc&content=&no_redirect=1&order=bug_severity%2Cbug_status%2Cpriority%2Cassigned_to%2Cbug_id&product=&query_based_on=&query_format=specific Instead of trying to fix those bugs during DConf, maybe experienced devs could help interested folks get into it. Look at an issue together, lay out what needs to be done about it. I'm thinking about dmd in particular, because it seems to be understaffed. Only works if there are people who want to get into fixing compiler bugs, of course. [1] My regressions: https://issues.dlang.org/buglist.cgi?bug_severity=regression&email1=ag0aep6g&emailreporter1=1&emailtype1=substring&list_id=214636&query_format=advanced&resolution=--- [2] My wrong-code bugs: https://issues.dlang.org/buglist.cgi?email1=ag0aep6g&emailreporter1=1&emailtype1=substring&keywords=wrong-code&keywords_type=allwords&list_id=214637&query_format=advanced&resolution=---
Apr 27
prev sibling next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Apr 27, 2017 at 11:22:06AM -0700, Brad Roberts via Digitalmars-d wrote:
 The pending pull requests.  In person is a great high-bandwidth way to
 work through the massive backlog.
+1. Phobos at one time was down to about 30-40 PRs, but now it has clogged back up to around 100. We need to get it back under control so that valuable contributions aren't just sitting there bitrotting. T -- Не дорог подарок, дорога любовь.
Apr 27
prev sibling next sibling parent reply =?UTF-8?B?THXDrXM=?= Marques <luis luismarques.eu> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:
 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?
Backtraces with line information on macOS?
Apr 27
parent Jacob Carlborg <doob me.com> writes:
On 2017-04-28 04:12, Luís Marques wrote:

 Backtraces with line information on macOS?
That would be nice. Should be fairly trivial to look at the Linux implementation and to the same but read the data from different sections. -- /Jacob Carlborg
Apr 28
prev sibling next sibling parent Adrian Matoga <dlang.spam matoga.info> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:
 This year, DConf has an extra day tacked on for problem solving 
 in the form of a hackathon. The intent is to work on issues 
 people find frustrating in the D ecosystem. While there will be 
 time given at the event for proposals, and those involving 
 third-party projects are welcome, it will help speed things 
 along for the organizers to come in with a list of big issues 
 to get the ball rolling.

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?
It's probably not a "major issue", but I'd be interested in joining any efforts towards making dub more cross-compilation-friendly. "--compiler=" and "--arch=" aren't quite useful in any serious scenario with multiple target platforms that use different compiler settings and sysroots.
Apr 28
prev sibling next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:
 This year, DConf has an extra day tacked on for problem solving 
 in the form of a hackathon. The intent is to work on issues 
 people find frustrating in the D ecosystem. While there will be 
 time given at the event for proposals, and those involving 
 third-party projects are welcome, it will help speed things 
 along for the organizers to come in with a list of big issues 
 to get the ball rolling.

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?
Here's my wishlist: D's safe-ty story DMD as a library (both frontend and backend): * the backend usable as a JIT library. * unittests (!= integration tests like those in https://github.com/dlang/dmd/tree/master/test) * fixing the GC interoperability (AFAIK, dmd currently can't use the GC) * less coupling (e.g. https://github.com/dlang/dmd/pull/6625) * no use of global state, except in the compiler driver * task-based parallelism * better docs etc. D REPL based on dmd library. There were a couple of attempts (e.g. https://github.com/callumenator/dabble, https://github.com/MartinNowak/drepl), but as far I know, none of them is using dmd as a library and I'm not sure how close they are actually being useful. AST introspection - given a function definition (!= declaration, i.e. the body is available) f, the expression __traits(ast, f) should return an instance of FuncDeclaration (https://github.com/dlang/dmd/blob/197ff0fd84b7b101f47458ab06e5b6f95959941e/src/ dmd/astnull.d#L151) with all of its members filled. The class-es used to represent the AST should be plain old data types (i.e. should contain no functionality). The AST as produced by the parser should be returned, without any semantic analysis (which can modify it). No backwards compatibility guarantees should be made about __traits(ast, f) at least for a couple of years. DMD usable as a library can help the various IDE projects improve the user experience, which is major complaint of new comers (e.g. better auto-completion, refactoring, etc). AST introspection (in addition with the previous point) is needed for D to remain competitive on the metaprogramming front with other rising languages such as Jai or Nim. I would love to meet and work with other people interested in those areas at DConf.
Apr 28
parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Friday, 28 April 2017 at 09:52:29 UTC, Petar Kirov 
[ZombineDev] wrote:
 AST introspection - given a function definition (!= 
 declaration, i.e. the body is available) f, the expression 
 __traits(ast, f) should return an instance of FuncDeclaration 
 (https://github.com/dlang/dmd/blob/197ff0fd84b7b101f47458ab06e5b6f95959941e/src/
dmd/astnull.d#L151) with all of its members filled. The class-es used to
represent the AST should be plain old data types (i.e. should contain no
functionality). The AST as produced by the parser should be returned, without
any semantic analysis (which can modify it). No backwards compatibility
guarantees should be made about __traits(ast, f) at least for a couple of years.
This sounds interesting, but could you expand on what this enables/improves compared to CTFE+mixins?
Apr 28
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Friday, 28 April 2017 at 10:09:32 UTC, Moritz Maxeiner wrote:
 On Friday, 28 April 2017 at 09:52:29 UTC, Petar Kirov 
 [ZombineDev] wrote:
 AST introspection - given a function definition (!= 
 declaration, i.e. the body is available) f, the expression 
 __traits(ast, f) should return an instance of FuncDeclaration 
 (https://github.com/dlang/dmd/blob/197ff0fd84b7b101f47458ab06e5b6f95959941e/src/
dmd/astnull.d#L151) with all of its members filled. The class-es used to
represent the AST should be plain old data types (i.e. should contain no
functionality). The AST as produced by the parser should be returned, without
any semantic analysis (which can modify it). No backwards compatibility
guarantees should be made about __traits(ast, f) at least for a couple of years.
This sounds interesting, but could you expand on what this enables/improves compared to CTFE+mixins?
CTFE and mixins are building blocks, needed to for my idea to actually useful. Currently if you want to introspect a piece of code (function body) at compile time, you need to duplicate the code in a string and then pass this string to a CTFE-able D parser. As you can imagine, even with Stefan's new CTFE engine, this would be orders of magnitude slower than just reusing the work that parser inside the compiler *has already done* anyway. This makes such code introspection infeasible for large projects. Strings (at least until mixined) can contain uncompilable source (though lexically valid, in the case of q{}), further complicating the matter. Additionally, the fact that you need to keep the source code and the strings in sync is just stupid, violating DRY at a whole new level. In addition to AST introspection, AST transformation should be as easy as: mixin template profileFunctionStatements(alias func, string newFunctionName) { enum funcAst = __traits(ast, func); enum newAst = insertProfiling(funcAst, newFunctionName); mixin(newAst.toString()); // a further optimization would be AST mixins, which // could directly be used by the compiler, instead of // first converting the AST to string and then // parsing it after mixing: mixin(newAst); } void main() { int local = 42; void fun(int[] arr) { import std.conv : text; import std.file : remove, write; arr[] += local; string s = text(arr); "delete-me.txt".write(s); } mixin profileFunctionStatements!(print, `funInstrumented`); import std.array : array; import std.range : iota; funInstrumented(10000.iota.array); } Output: { arr[] += local; } took 0.002 miliseconds to execute. { string s = text(arr); } took 1.8052 miliseconds to execute. { "delete-me.txt".write(s); } took 7.746 miliseconds to execute. Where funInstrumented could be generated to something like this: void funInstrumented(int[] arr) { import std.datetime : __Sw = StopWatch, __to = to; // generated import std.stdio : __wfln = writefln; // generated import std.conv : text; import std.file : remove, write; __Sw __sw; // generated __sw.start(); // generated arr[] += local; __sw.stop(); // generated __wfln("{ %s } took %s miliseconds to execute.", q{ arr[] += local; }, __sw.peek().__to!("msecs", double)); // generated __sw.start(); // generated string s = text(arr); __sw.stop(); // generated __wfln("{ %s } took %s miliseconds to execute.", q{ string s = text(arr); }, __sw.peek().__to!("msecs", double)); // generated __sw.start(); // generated "delete-me.txt".write(s); __sw.stop(); // generated __wfln("{%s} took %s miliseconds to execute.", q{ "delete-me.txt".write(s); }, __sw.peek().__to!("msecs", double)); // generated } Other applications include: * compiling/transpiling D functions to targets like JS, SPIR-V, WebAssembly, etc. using CTFE. * CTFE-driven code diagnostics (linting) * Adding semantic to user defined attributes: E.g. asyncSafe attribute for use in libraries like vibe.d that allows calling only functions marked as asyncSafe from asyncSafe code. That way libraries can introduce *and enforce* correct use of UDAs without any need for language changes. * ...
Apr 28
next sibling parent Moritz Maxeiner <moritz ucworks.org> writes:
On Friday, 28 April 2017 at 13:31:33 UTC, Petar Kirov 
[ZombineDev] wrote:
 [...]

 Other applications include:
 * compiling/transpiling D functions to targets
 like JS, SPIR-V, WebAssembly, etc. using CTFE.
 * CTFE-driven code diagnostics (linting)
 * Adding semantic to user defined attributes:
   E.g.  asyncSafe attribute for use in libraries like vibe.d 
 that allows calling only
   functions marked as  asyncSafe from  asyncSafe code. That way 
 libraries can
   introduce *and enforce* correct use of UDAs without any need 
 for language changes.
 * ...
Thanks for the summary :)
Apr 28
prev sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 28 April 2017 at 13:31:33 UTC, Petar Kirov 
[ZombineDev] wrote:
 Other applications include:
 * compiling/transpiling D functions to targets
 like JS, SPIR-V,
I got you covered ;) (LDC not CTFE though. It would be fiendishly complicated to do at CTFE as a fair amount of compiler magic is necessary.)
Apr 28
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Saturday, 29 April 2017 at 00:54:59 UTC, Nicholas Wilson wrote:
 On Friday, 28 April 2017 at 13:31:33 UTC, Petar Kirov 
 [ZombineDev] wrote:
 Other applications include:
 * compiling/transpiling D functions to targets
 like JS, SPIR-V,
I got you covered ;)
I know, and I'm looking forward to using your GPU support in LDC :P
 (LDC not CTFE though. It would be fiendishly complicated to do 
 at CTFE as a fair amount of compiler magic is necessary.)
The way I see it, metaprogramming is a whole spectrum. Yes, you need a full compiler if you want to compile a whole program to e.g. JS or WebAsm, but there are many areas where even the smallest improvement would be highly beneficial. Take for example C#. It has zero CTFE capabilities (you can safely ignore constant folding, which works only with string and number literals), yet it has pretty powerful reflection and code-generation capabilities at run-time. Even though the performance difference between CT and RT reflection is orders of magnitude, those reflection capabilities are used pervasively throughout the whole ecosystem. The prime example being LINQ - probably the most widely used feature of .NET. Given a chain of operations (similar to D's ranges) like: dbContext.Persons .Where(p => p.Birthdate < DateTime.Now.Date.AddYears(-18)) .Where(p => p.Birthdate > DateTime.Now.Date.AddYears(-28)) .Select(p => new { Name = p.FirstName + " " + p.LastName, Birthdate = p.Birthdate }) It gets compiled to the following SQL: -- Region Parameters DECLARE p0 DateTime = '1989-05-01 00:00:00.000' DECLARE p1 DateTime = '1999-05-01 00:00:00.000' DECLARE p2 NVarChar(1000) = ' ' -- EndRegion SELECT ([t0].[FirstName] + p2) + [t0].[LastName] AS [Name], [t0].[Birthdate] FROM [Person] AS [t0] WHERE ([t0].[Birthdate] > p0) AND ([t0].[Birthdate] < p1) As you can see the mapping and filtering is done entirely on the database server side. The only magic need is to make the compiler to dump the AST. In C# that's accomplished by wrapping the function type F in to an Expression<F> [0]. For example C#'s analog of: InputRange!T filter(T)(InputRange!T source, bool delegate(T) predicate) is expressed as: IEnumerable<T> Where<T>(IEnumerable<T> source, Func<T, bool> predicate) In order to request the AST of the predicate function, it needs to be wrapped in Expression<T>: IQueryable<T> Where<T>( IQueryable<T> source, Expression<Func<T, bool>> predicate) (IQueryable<T> [1] is an extension of IEnumerable<T> [2] interface (which is similar to D's Input/ForwardRange-s) which adds the Expression property which represents the AST of the query against the IQueryable data source.) ---- In addition to compiling range operations to database queries, this would also be useful specializing on lambda's in range libraries (see [3]) and much more. [0]: https://msdn.microsoft.com/en-us/library/bb335710(v=vs.110).aspx [1]: https://msdn.microsoft.com/en-us/library/bb351562(v=vs.110).aspx [2]: https://msdn.microsoft.com/en-us/library/9eekhta0(v=vs.110).aspx [3]: https://github.com/dlang/phobos/pull/4265
May 01
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 1 May 2017 at 12:35:11 UTC, Petar Kirov [ZombineDev] 
wrote:
 On Saturday, 29 April 2017 at 00:54:59 UTC, Nicholas Wilson 
 wrote:
 On Friday, 28 April 2017 at 13:31:33 UTC, Petar Kirov 
 [ZombineDev] wrote:
 Other applications include:
 * compiling/transpiling D functions to targets
 like JS, SPIR-V,
I got you covered ;)
I know, and I'm looking forward to using your GPU support in LDC :P
 (LDC not CTFE though. It would be fiendishly complicated to do 
 at CTFE as a fair amount of compiler magic is necessary.)
The way I see it, metaprogramming is a whole spectrum. Yes, you need a full compiler if you want to compile a whole program to e.g. JS or WebAsm, but there are many areas where even the smallest improvement would be highly beneficial.
Oh I agree. For text to text (or lambda to text) this is comprehensible and not too difficult (perhaps even simple with Dmitry Olshansky's Pry) but SPIRV is rather complicated, as evidenced by a 203 page spec *(that doesn't even include the OpenCL or Vulkan specific stuff). Having said that it shouldn't be too difficult to port the tablegen tables I've been working on for LLVM to D (or write a D backend for tablegen) and get most of the really boring work out of the way. I won't stop you, but just letting you know the rabbit hole is quite deep :) The good news is the the runtime stuff will be all transferable. This approach for SPIRV won't automatically translate for NVPTX like it will for ldc though, which is one of the main draws for dcompute *Other languages probably have longer specs, but this is just for the format.
 Take for example C#. It has zero CTFE capabilities (you can
 safely ignore constant folding, which works only with string 
 and number literals),
 yet it has pretty powerful reflection and code-generation 
 capabilities
 at run-time. Even though the performance difference between CT 
 and RT
 reflection is orders of magnitude, those reflection 
 capabilities are used
 pervasively throughout the whole ecosystem. The prime example 
 being LINQ -
 probably the most widely used feature of .NET. Given a chain of 
 operations
 (similar to D's ranges) like:

 dbContext.Persons
   .Where(p => p.Birthdate < DateTime.Now.Date.AddYears(-18))
   .Where(p => p.Birthdate > DateTime.Now.Date.AddYears(-28))
   .Select(p => new { Name = p.FirstName + " " + p.LastName, 
 Birthdate = p.Birthdate })

 It gets compiled to the following SQL:
 -- Region Parameters
 DECLARE  p0 DateTime = '1989-05-01 00:00:00.000'
 DECLARE  p1 DateTime = '1999-05-01 00:00:00.000'
 DECLARE  p2 NVarChar(1000) = ' '
 -- EndRegion
 SELECT ([t0].[FirstName] +  p2) + [t0].[LastName] AS [Name], 
 [t0].[Birthdate]
 FROM [Person] AS [t0]
 WHERE ([t0].[Birthdate] >  p0) AND ([t0].[Birthdate] <  p1)

 As you can see the mapping and filtering is done entirely on the
 database server side. The only magic need is to make the 
 compiler to
 dump the AST. In C# that's accomplished by wrapping the function
 type F in to an Expression<F> [0]. For example C#'s analog of:

 InputRange!T filter(T)(InputRange!T source, bool delegate(T) 
 predicate)

 is expressed as:

 IEnumerable<T> Where<T>(IEnumerable<T> source, Func<T, bool> 
 predicate)

 In order to request the AST of the predicate function, it needs 
 to
 be wrapped in Expression<T>:

 IQueryable<T> Where<T>(
     IQueryable<T> source, Expression<Func<T, bool>> predicate)

 (IQueryable<T> [1] is an extension of IEnumerable<T> [2] 
 interface (which is
 similar to D's Input/ForwardRange-s) which adds the Expression 
 property
 which represents the AST of the query against the IQueryable 
 data source.)

 ----

 In addition to compiling range operations to database queries, 
 this would
 also be useful specializing on lambda's in range libraries (see 
 [3]) and
 much more.


 [0]: 
 https://msdn.microsoft.com/en-us/library/bb335710(v=vs.110).aspx
 [1]: 
 https://msdn.microsoft.com/en-us/library/bb351562(v=vs.110).aspx
 [2]: 
 https://msdn.microsoft.com/en-us/library/9eekhta0(v=vs.110).aspx
 [3]: https://github.com/dlang/phobos/pull/4265
May 01
prev sibling next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:
 This year, DConf has an extra day tacked on for problem solving 
 in the form of a hackathon. The intent is to work on issues 
 people find frustrating in the D ecosystem. While there will be 
 time given at the event for proposals, and those involving 
 third-party projects are welcome, it will help speed things 
 along for the organizers to come in with a list of big issues 
 to get the ball rolling.

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?
Not so much major issue but I would like to: * figure out a solution for https://github.com/ldc-developers/ldc/issues/2011 * consider the merits of standardising allocSize (https://github.com/ldc-developers/druntime/blob/ldc/src/ldc/attributes.d#L16) or equivalent among dmd/ldc/gdc * get half precision floating point into the language /or/ ability to create __vector's of user types (need for intrinsics for GPU et al targets of dcompute). I would also like to hold a mini hackathon/gauge interest in dcompute as we could benefit significantly from the ML craze.
Apr 28
prev sibling next sibling parent reply Daniel N <no public.email> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:
 This year, DConf has an extra day tacked on for problem solving 
 in the form of a hackathon. The intent is to work on issues 
 people find frustrating in the D ecosystem. While there will be 
 time given at the event for proposals, and those involving 
 third-party projects are welcome, it will help speed things 
 along for the organizers to come in with a list of big issues 
 to get the ball rolling.

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?
Wishlist 1) Make alias parameters work with buildin types, I bet anyone using D has fallen into trip trap. alias X(alias T) = T; X!int x;
Apr 29
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/29/17 5:42 AM, Daniel N wrote:
 On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:
 This year, DConf has an extra day tacked on for problem solving in the
 form of a hackathon. The intent is to work on issues people find
 frustrating in the D ecosystem. While there will be time given at the
 event for proposals, and those involving third-party projects are
 welcome, it will help speed things along for the organizers to come in
 with a list of big issues to get the ball rolling.

 To help in compiling the list, what are some major issues from the
 ecosystem that you'd like to see fixed?
Wishlist 1) Make alias parameters work with buildin types, I bet anyone using D has fallen into trip trap. alias X(alias T) = T; X!int x;
+1, I would love to see this fixed. I recall Walter said it should be fixed 2 years ago in Utah. -Steve
May 01
prev sibling parent Joakim <dlang joakim.fea.st> writes:
On Thursday, 27 April 2017 at 14:53:02 UTC, Mike Parker wrote:
 This year, DConf has an extra day tacked on for problem solving 
 in the form of a hackathon. The intent is to work on issues 
 people find frustrating in the D ecosystem. While there will be 
 time given at the event for proposals, and those involving 
 third-party projects are welcome, it will help speed things 
 along for the organizers to come in with a list of big issues 
 to get the ball rolling.

 To help in compiling the list, what are some major issues from 
 the ecosystem that you'd like to see fixed?
Probably the plan anyway, but my suggestion would be for the core team to not hack on anything, but spend the time discussing issues that have been discussed to death online but not resolved, such as some devs not agreeing with Walter and the DIP 1000 path he's taking. Use the in-person time to get some heavy bandwidth on those issues and try to make sure the differences are hashed out. There may not be a final agreement on the solution, but there certainly shouldn't be any more misunderstanding of the proposed options. For people not on the core team, they can hack on a lot of the stuff mentioned in this thread, perhaps after coordinating with the core team about what's really needed and how to go about doing it. Great idea, btw, to set aside a day just for this.
May 01