www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - D Language Foundation Monthly Meeting Summary for May 2023

reply Mike Parker <aldacron gmail.com> writes:
The monthly meeting for May 2023 took place on Friday the 5th at 
14:00 UTC. It lasted about an hour and a half. This was the last 
meeting before we started our new planning sessions.

The following people attended:

* Walter Bright
* Iain Buclaw
* Ali Çehreli
* Martin Kinkelin
* Dennis Korpel
* Mathias Lang
* Razvan Nitu
* Mike Parker
* Robert Schadek




I started with an update on DConf submissions, reporting the 
numbers and some of the details on who and what. At that point, 
we had a total of 11 submissions from 8 people. I suggested we 
could put a panel in one of the open slots if we needed to and 
asked everyone to think of some ideas. (We ended up with 39 
submissions from 21 people, or 24 if you count the additional 
three members of the one team that submitted, and had no room at 
all for a panel).

Next, I told everyone we had secured some [sponsorship funding 
from Ahrefs](https://ahrefs.com/) and noted that I planned to use 
it for BeerConf. I hoped to rent the same space we rented last 
year at The Fox, so I detailed how I was going to work out paying 
for it via our event planner. Last year, Funkwerk provided the 
funding to the DLF, Átila and Walter each paid one or more nights 
on their credit cards, and the DLF reimbursed them. 
(Unfortunately, pub hire rates this year have massively increased 
over last year, putting them well beyond our budget. We've fallen 
back to the old way of designating a hotel as the BeerConf 
location, this time Travelodge Central City Road. The sponsorship 
funding will go toward our speaker reimbursement budget.)

Next, I gave an update on registrations. At that point, only one 
person had registered. I then talked about some email exchanges 
I'd had with some D shops about possible sponsorships. (Our 
current headcount as of June 23 is 43. I'm anticipating a final 
number in the neighborhood of 60-70, which would be in the same 
ballpark as last year.)

Walter brought up [Sebastiaan Koppe's presentation from last 
year](https://youtu.be/hJhNhIeq29U) on structured concurrency. He 
said he'd like to see Sebastiaan there this year for an update on 
the project, preferably as a follow-up presentation. He was 
excited about the project and felt it was a big deal in general 
and a big deal for D. Even if Sebastiaan decided not to move the 
project forward, Walter felt we should find someone to take it 
over. He said the ideas behind it were big and would be a nice 
thing for us. (We subsequently confirmed with Sebastiaan that he 
didn't have enough material yet for a follow-up talk).

I summed up by saying that DConf planning was on track and there 
were no issues so far.

Later, I remembered to mention that I had Robert's GitHub to 
Bugzilla migration script in hand. Now it was just waiting on me 
to make time to test it out, so I was the blocker now rather than 
Robert. He warned me about a command line option I should leave 
off while testing. (You can read about the progress on that [in 
the General 
forum](https://forum.dlang.org/thread/pfpoeaoihsqpohumjvsi forum.dlang.org) and
track our progress [on our GitHub projects
page](https://github.com/orgs/dlang/projects/23?pane=issue&itemId=29572916.)).)

I asked about some issues that [had been migrated to the tools 
repository](https://github.com/dlang/tools/issues?q=is%3Aissue+is%3Aopen+
ort%3Aupdated-desc) from Bugzilla. Robert said that was not from him. Mathias
jumped in to say that he had done it a long time ago when he took a stab at
this. He had closed all of the Bugzilla issues he'd migrated, so there would be
no duplications when migrating the open ones.


Iain opened what turned into a 25-minute segment by saying that 
he hadn't gotten much done since the middle of March as he'd 
spent several long weekends on holiday. He'd released D 2.103.0 
in April and 2.103.1 on May 1. He had also pushed out the first 
beta of 2.104.0, and it was on track for release at the end of 
the month ([he announced it on June 
2](https://forum.dlang.org/thread/waigfuztqsqlhzoujcmz forum.dlang.org)).

He said that GCC 13.1 had been released on April 26. That 
included version 2.103 of the D frontend in GDC.

Walter thanked Iain for handling the releases for us. Martin 
asked how long the release process takes nowadays. Iain said it 
depends on if he hits an issue building the Windows release. The 
Windows box, and *only* the Windows box, seems to hit a file 
system sync issue. There's a D program, `copyimports.exe`, that 
had replaced several make recipes to copy files during the build. 
Sometimes when building the releases, `make` tries to call that 
program before the linker has created the executable. When that 
happens, it aborts and Iain has to run the release process from 
the start again. [He had filed an 
issue](https://issues.dlang.org/show_bug.cgi?id=23486) for it.

If everything goes smoothly, it's about a two-hour process. The 
biggest part of that is waiting for the builds to complete, and 
then waiting for the CI pipelines to finish. He had noticed that 
the CI pipelines take longer now. The Mac pipeline had broken a 
while back and Max had fixed it. Before the breakage, it took 
about 30 minutes to complete, but since the fix, it had been 
taking 50 minutes. He didn't know if the slowdown was due to the 
fix or because the actual runners had slowed down. Martin didn't 
think it had anything to do with the fix.

At a prompt from Martin, Iain went into quite a bit of detail 
about the Mac CI pipeline. The slowdown comes when testing the 
optimized build of dmd. It's two times slower than testing the 
debug build. It's also running on Rosetta, so there could be many 
potential reasons for the problem. He said it's a minor 
annoyance, but someone with an M1 machine should look into it to 
determine the cause.

Martin noted that they still hadn't found the root cause of the 
problem that prompted Max's fix. It appears to be due to some 
sort of change in Apple's libc. Things worked just fine on other 
platforms and had been working fine on Mac for years until that 
change.

Iain then talked about Cirrus CI's usage policy and how this 
issue affects us there. This was followed by a discussion about 
some of the different kinds of pipelines we have, the pros and 
cons of different CI providers we've used/are using, and the 
possibility of moving some of the pipelines to GitHub Actions. 
Mathias noted that we are severely underutilizing GH Actions.

This was followed by a discussion of Apple's libc, other possible 
causes of the slowdown, the cost of renting a dedicated M1 Mac, 
more details about our CI pipelines, and other related topics.

We wrapped up by agreeing that we need to look into making more 
use of GH Actions.


Ali reported he was still using D to prototype things at work. He 
then told us he had recently taken a trip to Germany and met up 
with several D programmers while he was there.


Martin said that unlike Iain, for whom everything had worked out 
with the most recent GDC release, the LDC side was having some 
issues. They had hit several regressions.

He described one example that had been reported where adding `-g` 
to the LDC command line caused compilation to take 10 times 
longer than it used to. At first, he wasn't sure if it was a 
major problem or an outlier, but then Robert had seen a 50-times 

regression to tackle. He said it was related to a change in LLVM 
15 that hadn't yet been accounted for in LDC, and it was blocking 
a new release. (He was able to resolve it [a few days after the 
meeting](https://github.com/ldc-developers/ldc/issues/4354) and 
[released the fix in LDC 
1.32.2](https://github.com/ldc-developers/ldc/releases/tag/v1.32.2).)

For D 2.103 support, all of the tests were green on Linux except 
on Windows. It was apparently related to a batch file generated 
either by CMake or Ninja exceeding the maximum command-line 
length.

Next, he brought up a topic that led us into a discussion about 
how to interact with contributors. The gist of it is that someone 
had submitted a PR that was an incorrect fix for a particular 
issue. Most likely, the submitter didn't have the big picture and 
was unable to see that the root problem was elsewhere. By 
submitting the PR, he did get attention to the issue and someone 
else solved it with the correct fix, but Martin felt he should 
have been told sooner that the fix was wrong and why, instead of 
being asked to, e.g., add missing tests.

Razvan explained why he had taken a neutral approach to that 
particular PR instead of coming right out and saying it was 
wrong. In short, he was trying not to upset the person who had 
submitted it. Martin said he understood the intent, but what if 
no one else had stepped in and mentioned it, and no one else had 
provided a more appropriate fix, until after the submitter had 
addressed any issues raised with the PR? He would surely have 
been upset in that case. We should always try to be as polite as 
possible, but if the approach seems to be wrong in the first 
place, stating that up front is better than wasting his time.

Walter thought Razvan had made a good effort at trying to resolve 
the issue without creating conflict. He said he often worries 
about how to get things on track when someone submits a PR that 
he thinks is the wrong approach. He cited an example where 
someone had submitted a PR for something that Walter thought was 
a good idea that had the wrong implementation. He had completely 
failed to get that point across and ended up upsetting the 
contributor. This is a tough problem that he knows he needs to do 
better with.

Robert said he'd found in conflict situations, text-based 
communication is bad. He said he'd had text discussions with 
coworkers that seemed negative and weren't working out. In that 
situation, just setting up a video call with the person changes 
everything. It can be uncomfortable and weird just talking with 
people you've never met, especially for techies like us, but 
you're doing that in text already, and something is lost in text. 
In doing that, he had found that the way he had read a coworker's 
words was not the way the coworker had written them.

I noted that Razvan, Dennis, Walter, and I had discussed this not 
long before. When a conflict arises with a contributor in a 
discussion thread, invite the contributor to have a call so that 
you aren't typing past each other. Iain brought up a quote (he 
couldn't recall the source): "On the internet, no one can read 
you speak subtle."

Robert said the invitation to the call should be done publicly in 
the discussion thread rather than privately by email so that the 
next person that comes along can see that ultimately there was a 
resolution. Otherwise, you might resolve the problem, but then 
you close the issue and, for whatever reason, don't leave a note 
describing why. If someone sees an accepted invitation to a call 
and the issue is subsequently closed, then it's safer to assume 
there was a discussion involved. Walter said that was good advice.

(I should take this opportunity to note that improving the 
environment around contributions is something we're aiming at as 
part of our ongoing organizational development. We want to 
encourage increased contributions while reducing the potential 
for people to waste their time. One of the early steps is to 
outline a policy on PR reviews, [a task you can track in one of 
our GH 
Projects](https://github.com/orgs/dlang/projects/24?pane=issue&itemId=29573012)
once Razvan and Dennis get started on it.)


Mathias said he had written a whole 20 lines of D over the past 
month. He still wanted to get back to dub but hadn't had much 
time. As such, he had nothing to report.


Robert talked about the 50-times slow down he'd run into at work, 
and thanked Martin for helping him track the problem down.

Next, he said he'd made the work project quicker this week by 
doing quite a bit of refactoring of [his graphqld 
library](https://github.com/burner/graphqld). He said it's still 
too slow, so the compiler is too slow. Martin disagreed that the 
compiler was at fault, noting how Robert's project takes longer 
to compile than a full repl (an internal Symmetry project) that 
takes 70 seconds. He doubts it's the compiler that's off. Robert 
said that he had seen a 100% speedup on his machine after 
refactoring, getting down to a one-minute compile from two. An 
incremental rebuild with Ninja was still 12 or 13 seconds.

I asked if he had run LDC's time trace on the project. He said he 
had. The most time was spent in Phobos, but they're calls that 
make sense. If he were to do those things himself, he'd just be 
bloating his code and recreating Phobos. The project is 70K lines 
of D at this point, and it's mostly templates. He feels D would 
be much more competitive in the language space if such a project 
took less than a second to incrementally compile. He wouldn't 
want to do something like this with Rust because it would take 
forever to compile. Go would be much faster, but then you 
wouldn't get the nice templates, and you'd have a much larger 
project.

Walter noted that some particular Phobos templates had been 
reported as taking too much time, and he'd been able to speed 
some of those up with new implementations. Sometimes when you 
profile, it may just be one template, or a handful of them, 
giving you speed problems.

Robert explained that graphqld takes so much compile time because 
it introspects all of the types it's given and creates an 
internal AST that's then used to generate the reflection API and 
a schema file that he needs for the front end. His refactoring 
got that down from happening three times to one time. That in 
itself is taking quite a while. He has to do it at least once, 
and unless he writes it all out by hand as the graphql libraries 
in other languages do, he doesn't see how he can make it faster. 
That's the lure of D's introspection: you just give it the entry 
point and it gets all of the links in the graph, and all the 
types, and gives you something else.

As a user of the language, he shouldn't have to care about the 
compile-time performance of a feature of the language. If he 
wants to write a 10,000-line template function, it shouldn't be 
much different from writing a 10,000-line normal function.

On the topic of compile times, Walter said that in trying to 
resolve a compiler bug, he'd discovered that CTFE was being 
called three times for each use of CTFE. He hadn't yet been able 
to track down why it was doing that, but we could potentially be 
looking at a situation where he's able to triple the speed of 
CTFE if he can resolve it.

Finally, Robert thanked Dennis [for his awesome 
video](https://youtu.be/b8wZqU5t9vs), saying he learned something 
from it.


Razvan had been focused on fixing regressions and compiler ICEs 
while managing the PR queues and going through Bugzilla issues. 
He had been looking at statistics on the issues and found the 
total number of open issues was on a clear downward trend. At the 
time of the meeting, we had gotten down to a total we last saw in 
2018.

He said the Phobos PR queue was pretty quiet, with only two or 
three PRs submitted per week. There was much more activity in the 
compiler repository, with around 20 or 30 PRs per week. Most of 
the dmd PRs in the past month had come from Razvan, Dennis, and 
Walter, and only a few from other people. But in the past week, 
there had been a sudden spike in activity from other people. He 
wondered who we could assign tasks to outside of the people in 
this meeting once we set up our task lists.



__Broken nightlies__

Dennis reported that the nightlies were broken because FreeBSD 
switched from LLVM 9 straight to 15 and the old package was no 
longer installed. Martin had been working on that for LDC, and 
Dennis asked if Iain had run into that problem with the last DMD 
release. Iain had not. The only thing he'd noticed was that LLVM 
1.32.1 was missing, so he'd had to use the .0 release to build 
DMD.

As for building on FreeBSD, Iain said he builds on his own box. 
He [linked the line from the release 
builder](https://github.com/dlang/installer/blob/d761b194c513fa9ab2814f1e4c84b1c56e7be380/create_dmd_release/b
ild_all.d#L19-L23). Martin noted the comments show that the VM this uses is a
version of FreeBSD from 2020, which is why it still works. Iain checked and
found the latest configuration available on Vagrant was from FreeBSD 12.4 from
January of this year, so he could switch to using that for building releases.
Or he could jump to 13. If he updates, maybe he'll hit the same issue that was
killing the nightlies.

Martin said it's always nice if the CI scripts use echoing. It 
comes in handy precisely in this kind of situation. Concerning 
FreeBSD, he had no idea what they'd changed. He didn't use it 
himself outside of LDC CI. Somehow they changed *extremely*, even 
on FreeBSD 12, a years-old version. They didn't just bump the LDC 
package from 9 to 15, but they also removed the LLVM 9 package. 
10+ are all available, but 9 is gone. There also was a major bump 
in the GCC package. And they changed the "official" LDC version 
to 1.32.0. He'd already encountered a problem with some sort of 
change they made in the GNU tools that forced him to flip the 
order of some switches for a trivial command. His experience with 
backward compatibility on FreeBSD was horrible. He reiterated 
something he'd said in the past: CI is a constant battle. Apple 
suddenly made changes in something that had worked for years, and 
FreeBSD has made changes in something that had worked for 
years... it's always like this.

Dennis had opened [a PR to update the 
installer](https://github.com/dlang/installer/pull/562). With 
that, building now succeeded, but he was getting an error from a 
later step in the process (see the discussion thread on the 
linked PR). He had no clue how to debug that and could use some 
assistance. Martin spent a little time looking at it but saw 
nothing obvious.

I asked if we should take the FreeBSD builds out of the 
nightlies. Dennis said he'd try that and see how it goes.

__More CI woes__

Dennis had been working on refactoring the backend, something for 
which Walter had given his approval in our April meeting. He'd 
been replacing C-style function prototypes with imports, and that 
had revealed some missing `const` and ` nogc` attributes and how 
incremental compilation is broken. He'd made nice progress with 
that. Unfortunately, replacing a prototype with an import 
sometimes causes the Azure CI to complain about being out of 
memory. Either it's at the brink of its resource limits, or 
replacing a prototype with an import somehow spikes memory usage. 
There's barely an impact on memory usage locally. The CI for 
Ubuntu sometimes times out.

I asked if we need to upgrade the Azure account. Dennis had no 
idea what the current resource limits were. Martin said it's 
mostly the same as GitHub Actions. They're built on the same 
infrastructure and resource limits are the same. It used to allow 
10 concurrent jobs at a time, but they doubled that to 20 last 
year. They're using the same extremely slow Azure VMs.

Dennis said he was surprised by the OOM. Martin said the VM 
probably had at least 8GB. He asked for [a link to the PR that 
triggered the OOM](https://github.com/dlang/dmd/pull/15188). 
Walter suggested switching to separate compilation and that 
should cut down on the memory usage. The backend was originally 
set up to be separately compiled. Dennis said currently it's all 
compiled in one go. The front end is compiled separately. He 
suspected that it wasn't a resource problem. Phobos should take 
more memory to compile, but never hits an OOM. Something weird 
was going on, but he didn't know what. Walter said the backend 
had significantly larger functions than Phobos. I reminded 
everyone that Dennis had said it was only happening on Azure and 
not locally.

As we were talking, Martin had been investigating. He said it 
wasn't only Azure, but also Cirrus, and only when using DMD as 
the host compiler. With LDC, they were passing. It appeared to be 
a DMD problem, but it made no sense to him at all based on the 
diffs. It was astonishing that replacing a prototype with an 
import could lead to this behavior.

No one had any ideas, so Dennis said he would work on something 
else and defer this to later.

(Apparently, it was fixed, but I don't know the details.)


Following on from his update in our April meeting, Walter 
reported that his efforts at Twitter outreach had shown some 
moderate success. Instead of tweeting once every few months as he 
had before, he'd been trying to tweet something every day about 
D, programming in general, and maybe some engineering anecdotes.

Next, he said he'd been working on ImportC. One frustrating thing 
was that he had no idea how successful it was among users. He had 
[posted in the 
forums](https://forum.dlang.org/thread/u2el6n$1tee$1 digitalmars.com) asking
people to try it and report any issues. That had resulted in a bunch of bug
reports that he was fixing. That was kind of an endless whack-a-mole. There's
always somebody using some silly C extension in their header files that's best
removed, but he can't remove it so he has to find a way to make it work.

He'd recently done a PR to implement C vector types, which had 
turned out to be a surprisingly small amount of code. That was 
nice. But he doesn't have a feel for where we are in the utility 
of ImportC.

He encouraged people to give ImportC a try and tell him how it 
breaks, then he would try to fix it. He'd added the vector 
instructions because of an issue Steven Schveighoffer ran into 
when he tried it. He wants people to report even the small bugs. 
Those are like a small rock in your shoe: it's easy to get out, 
but annoying while it's there. He wants to get ImportC polished 
even at that level.

Otherwise, he was just continuing the process of working on bug 
fixes rather than enhancements. That may be reflected in the 
statistics Razvan reported of the bug count going down.

Walter then brought up [the rejected enum inference 
DIP](https://forum.dlang.org/thread/fdxkixkcgzrxveayybel forum.dlang.org). He
thinks an implicit `with` on case statements when the switch variable is an
`enum` is a better alternative. He'd been thinking a lot about it, and it was
still in the back of his mind. The problem with it was, where do you stop? If
you have an expression, `a === b`, where `a` is an enum expression and `b` is
an enum value, does that automatically imply a `with` on the `b`? I noted that
Java stopped with the case. Walter thinks it's a good idea, he just worries
that if we do it we'll be opening the door to all kinds of whacky things.

There had been a lot of trouble with using `format` with BetterC 
and CTFE. It boils down to `format` allocating memory from the 
GC. So he'd been going through the source of `format` to find out 
where it's allocating memory and if it really needs to allocate. 
He thought he could make some progress on that. For example, 
converting an integer to a string always allocates, but there's 
no reason for it to allocate because the amount of memory it 
needs is always bounded.

Other than that, he had no momentous issues. He was just grinding 
along, fixing problems one by one. He hoped to get ImportC's 
problems into the background. As far as implementing C, it's 
doing very well. It's just all those "nutburger" extensions 
people use that cause problems.

Martin said he'd welcome getting rid of the C initializers in the 
AST. He and Walter had talked before about converting them to D 
initializers in the AST. Walter had said he'd do it, but we were 
still waiting on it. This got Walter going on a rant about how 
complex C initializers had become in C23. The spec on them had 
blown up into pages and pages of descriptions and, even as a 
language lawyer who's spent his whole career reading C specs, he 
can't figure out what it's saying. These days, it's a fraud to 
regard C as a simple language. And it's unnecessary complexity.


We started our new planning sessions the following week, the 
results of which I've already posted in the General forum. Our 
next monthly took place on June 2 at 15:00 UTC and was the first 
with expanded membership. I'll have the summary for that as soon 
as I can.

As always, if you have an issue or idea you'd like to discuss 
with us, please let me know and I'll bring you into the first 
monthly I can. I want to emphasize that these are *not* formal 
meetings. Our conversations are free, open, and casual. We're 
eager for input from diverse sources, so if you feel like you 
have something that can improve the D language, ecosystem, or 
community, please don't hesitate to bring it to us.
Jun 23 2023
next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On Friday, 23 June 2023 at 14:32:51 UTC, Mike Parker wrote:
 The monthly meeting for May 2023 took place on Friday the 5th 
 at 14:00 UTC. It lasted about an hour and a half. This was the 
 last meeting before we started our new planning sessions.
Nice to read on what you guys are doing! — Dmitry Olshansky CEO at Glow labs https://sponsr.ru/glow https://patreon.com/dmitry_glow_labs
Jun 23 2023
prev sibling next sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Friday, 23 June 2023 at 14:32:51 UTC, Mike Parker wrote:
 [...]

 Walter brought up [Sebastiaan Koppe's presentation from last 
 year](https://youtu.be/hJhNhIeq29U) on structured concurrency. 
 He said he'd like to see Sebastiaan there this year for an 
 update on the project, preferably as a follow-up presentation. 
 He was excited about the project and felt it was a big deal in 
 general and a big deal for D. Even if Sebastiaan decided not to 
 move the project forward, Walter felt we should find someone to 
 take it over. He said the ideas behind it were big and would be 
 a nice thing for us. (We subsequently confirmed with Sebastiaan 
 that he didn't have enough material yet for a follow-up talk).

 [...]
For anyone interested, this is still very much alive. In fact, the lovely folks at https://github.com/NVIDIA/stdexec have validated several ideas I had for improving things, and I have been using any spare time I have left - which isn't much sadly - to incorporate these concepts. I hope to find some time to do a lighting talk.
Jun 23 2023
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
If you feel up to doing a practice talk, you're welcome to join us on 
BeerConf!
Jun 23 2023
prev sibling parent reply Sergey <kornburn yandex.ru> writes:
On Friday, 23 June 2023 at 14:32:51 UTC, Mike Parker wrote:
 The monthly meeting for May 2023 took place on Friday the 5th 
 at 14:00 UTC. It lasted about an hour and a half.
Hi Mike. Does anyone consider some automatization and application of modern technologies for the process of meeting summarization? Like NN that: * record the sound from the call * make sound to text (with different speakers) * make text summarization * generate report After that real person could verify result and make small corrections. It should significantly reduce effort and decrease time-to-market :)
Jun 27 2023
parent reply Mike Parker <aldacron gmail.com> writes:
On Tuesday, 27 June 2023 at 12:36:20 UTC, Sergey wrote:

 After that real person could verify result and make small 
 corrections. It should significantly reduce effort and decrease 
 time-to-market :)
Yes. The next version of Davinci Resolve (which I use for video editing) will have automated transcriptions. Once it's released, I intend to start loading my meeting recordings into it and then derive my summaries from that. However, I still will write the summaries as summaries, rather than publishing the transcripts directly or otherwise quoting them verbatim. This will save me from having to go back and forth through the video, so it will certainly save time. I still will put at least three or four weeks between the meetings and the summaries. One benefit I've found in the delay is that it's an easy way for me to follow up on any actionable items. As I'm writing a summary, I can check on the status of issues, follow up with anyone who said they'd do something, etc. Then I don't have to worry about taking notes during the meeting, I can add the current status directly into the summary, and anything that's not done by the time I write it up I can stay on top of.
Jun 27 2023
parent harakim <harakim gmail.com> writes:
On Tuesday, 27 June 2023 at 14:18:52 UTC, Mike Parker wrote:
 I still will put at least three or four weeks between the 
 meetings and the summaries.
I appreciate your summary here. This was very insightful!
Jul 06 2023