www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Symmetry Investments and the D Language Foundation are Hiring

reply Mike Parker <aldacron gmail.com> writes:
Looking for a full-time or part-time gig? Not only is Symmetry 
Investments hiring D programmers, they are also generously 
funding two positions for ecosystem work under the D Language 
Foundation. And they've put up a bounty for a new DUB feature. 
Read all about it here:

https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/
Aug 30
next sibling parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
 permanent position
This is huge! Thank you Symmetry! — Bastiaan.
Aug 30
prev sibling next sibling parent reply Arjan <arjan ask.me.to> writes:
On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
 Looking for a full-time or part-time gig? Not only is Symmetry 
 Investments hiring D programmers, they are also generously 
 funding two positions for ecosystem work under the D Language 
 Foundation. And they've put up a bounty for a new DUB feature. 
 Read all about it here:

 https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/
Fantastic! Thanks Symmetry!
Aug 30
parent Tove <tove fransson.se> writes:
On Sunday, 30 August 2020 at 20:12:41 UTC, Arjan wrote:
 On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
 Looking for a full-time or part-time gig? Not only is Symmetry 
 Investments hiring D programmers, they are also generously 
 funding two positions for ecosystem work under the D Language 
 Foundation. And they've put up a bounty for a new DUB feature. 
 Read all about it here:

 https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/
Fantastic! Thanks Symmetry!
Epic, this is a gamechanger!
Aug 31
prev sibling next sibling parent bachmeier <no spam.net> writes:
On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
 Looking for a full-time or part-time gig? Not only is Symmetry 
 Investments hiring D programmers, they are also generously 
 funding two positions for ecosystem work under the D Language 
 Foundation. And they've put up a bounty for a new DUB feature. 
 Read all about it here:

 https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/
Thanks to Symmetry and the Foundation for this. This is a big deal. I hope there will be announcements posted and promoted after the hires are made. From a marketing perspective, it is initiatives like this that give others the confidence to invest in the language.
Aug 31
prev sibling next sibling parent reply Arun <aruncxy gmail.com> writes:
On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
 Looking for a full-time or part-time gig? Not only is Symmetry 
 Investments hiring D programmers, they are also generously 
 funding two positions for ecosystem work under the D Language 
 Foundation. And they've put up a bounty for a new DUB feature. 
 Read all about it here:

 https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/

 SHA-1 hashing throughout
Why not SHA-256?
Aug 31
parent reply starcanopy <starcanopy protonmail.com> writes:
On Monday, 31 August 2020 at 19:55:32 UTC, Arun wrote:
 On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
 Looking for a full-time or part-time gig? Not only is Symmetry 
 Investments hiring D programmers, they are also generously 
 funding two positions for ecosystem work under the D Language 
 Foundation. And they've put up a bounty for a new DUB feature. 
 Read all about it here:

 https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/

 SHA-1 hashing throughout
Why not SHA-256?
I think the majority opinion is that sha1 is fine for non-security purposes like monitoring files for changes to obviate superfluous work in a build system; for which DUB seemingly would use the sha1 checksums. I have (had) a script that would backup files, and I'd use sha1 to detect changes instead of sha256 because the latter's storage and computational requirements were not insignificant compared to the former. This is just my perspective, however. I could be talking nonsense.
Aug 31
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 31 August 2020 at 21:24:53 UTC, starcanopy wrote:
 I think the majority opinion is that sha1 is fine for 
 non-security purposes like monitoring files for changes to 
 obviate superfluous work in a build system;
Not a big fan of using hashes, since collisions do arise when you scale things up, but there are fast noncryptographic hashes. Some are like 10-500x faster than SHA1: https://github.com/rurban/smhasher
Sep 01
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 1 September 2020 at 11:53:25 UTC, Ola Fosheim Grøstad 
wrote:
 you scale things up, but there are fast noncryptographic 
 hashes. Some are like 10-500x faster than SHA1:
More like 10-50x...
Sep 01
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
 Looking for a full-time or part-time gig? Not only is Symmetry 
 Investments hiring D programmers, they are also generously 
 funding two positions for ecosystem work under the D Language 
 Foundation. And they've put up a bounty for a new DUB feature. 
 Read all about it here:

 https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/
As an alternative to use SHA-1 hashing. There's the option to have a daemon running the background listing on filesystem events. BTW, is timestamps vs SHA-1 hashing really the most pressing issue with Dub? -- /Jacob Carlborg
Sep 01
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg 
wrote:
 On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
 Looking for a full-time or part-time gig? Not only is Symmetry 
 Investments hiring D programmers, they are also generously 
 funding two positions for ecosystem work under the D Language 
 Foundation. And they've put up a bounty for a new DUB feature. 
 Read all about it here:

 https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/
As an alternative to use SHA-1 hashing. There's the option to have a daemon running the background listing on filesystem events. BTW, is timestamps vs SHA-1 hashing really the most pressing issue with Dub? -- /Jacob Carlborg
We think that not recompiling certain modules which have not changed will improve our build times. And the task proposed is actually something that can go in without too much struggle. Whereas deeper issues in dub likely take much longer.
Sep 01
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/1/20 5:38 AM, Stefan Koch wrote:
 On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg wrote:
 BTW, is timestamps vs SHA-1 hashing really the most pressing issue 
 with Dub?
We think that not recompiling certain modules which have not changed will improve our build times. And the task proposed is actually something that can go in without too much struggle. Whereas deeper issues in dub likely take much longer.
I have to agree with Jacob -- what common situation is changing the timestamps of your files but not the data? -Steve
Sep 01
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 1 September 2020 at 13:28:07 UTC, Steven 
Schveighoffer wrote:
 On 9/1/20 5:38 AM, Stefan Koch wrote:
 On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg 
 wrote:
 BTW, is timestamps vs SHA-1 hashing really the most pressing 
 issue with Dub?
We think that not recompiling certain modules which have not changed will improve our build times. And the task proposed is actually something that can go in without too much struggle. Whereas deeper issues in dub likely take much longer.
I have to agree with Jacob -- what common situation is changing the timestamps of your files but not the data? -Steve
git update. on CI.
Sep 01
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 1 September 2020 at 13:30:33 UTC, Stefan Koch wrote:
 On Tuesday, 1 September 2020 at 13:28:07 UTC, Steven 
 Schveighoffer wrote:
 On 9/1/20 5:38 AM, Stefan Koch wrote:
 On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg 
 wrote:
 BTW, is timestamps vs SHA-1 hashing really the most pressing 
 issue with Dub?
We think that not recompiling certain modules which have not changed will improve our build times. And the task proposed is actually something that can go in without too much struggle. Whereas deeper issues in dub likely take much longer.
I have to agree with Jacob -- what common situation is changing the timestamps of your files but not the data? -Steve
git update. on CI.
at least I think that was the problem. I have not looked into this personally.
Sep 01
prev sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Tuesday, 1 September 2020 at 13:28:07 UTC, Steven 
Schveighoffer wrote:
 On 9/1/20 5:38 AM, Stefan Koch wrote:
 On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg 
 wrote:
 BTW, is timestamps vs SHA-1 hashing really the most pressing 
 issue with Dub?
We think that not recompiling certain modules which have not changed will improve our build times. And the task proposed is actually something that can go in without too much struggle. Whereas deeper issues in dub likely take much longer.
I have to agree with Jacob -- what common situation is changing the timestamps of your files but not the data?
git checkout branch git checkout -
Sep 01
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/1/20 2:38 PM, Patrick Schluter wrote:
 On Tuesday, 1 September 2020 at 13:28:07 UTC, Steven Schveighoffer wrote:
 On 9/1/20 5:38 AM, Stefan Koch wrote:
 On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg wrote:
 BTW, is timestamps vs SHA-1 hashing really the most pressing issue 
 with Dub?
We think that not recompiling certain modules which have not changed will improve our build times. And the task proposed is actually something that can go in without too much struggle. Whereas deeper issues in dub likely take much longer.
I have to agree with Jacob -- what common situation is changing the timestamps of your files but not the data?
git checkout branch git checkout -
Is that a part of normal development process? Typically when I want incremental building, I'm editing a file, then rebuilding. I mean, you check out a different branch, but you don't want to rebuild everything? I would. And with D, where there are so many templates, almost everything is going to need rebuilding anyway. This update to dub might replace a build-time problem with a build inconsistency problem (hopefully linker error, but possibly code generation differences). -Steve
Sep 02
parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Wednesday, 2 September 2020 at 12:50:35 UTC, Steven 
Schveighoffer wrote:
 On 9/1/20 2:38 PM, Patrick Schluter wrote:
 On Tuesday, 1 September 2020 at 13:28:07 UTC, Steven 
 Schveighoffer wrote:
 On 9/1/20 5:38 AM, Stefan Koch wrote:
 [...]
I have to agree with Jacob -- what common situation is changing the timestamps of your files but not the data?
git checkout branch git checkout -
Is that a part of normal development process? Typically when I want incremental building, I'm editing a file, then rebuilding. I mean, you check out a different branch, but you don't want to rebuild everything? I would. And with D, where there are so many templates, almost everything is going to need rebuilding anyway. This update to dub might replace a build-time problem with a build inconsistency problem (hopefully linker error, but possibly code generation differences).
Yes, it happens from time to time and on makefile based it involves recompilations of unchanged files but granted it could be just me.
Sep 02
prev sibling parent reply Mathias LANG <geod24 gmail.com> writes:
On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg 
wrote:
 On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
 Looking for a full-time or part-time gig? Not only is Symmetry 
 Investments hiring D programmers, they are also generously 
 funding two positions for ecosystem work under the D Language 
 Foundation. And they've put up a bounty for a new DUB feature. 
 Read all about it here:

 https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/
As an alternative to use SHA-1 hashing. There's the option to have a daemon running the background listing on filesystem events. BTW, is timestamps vs SHA-1 hashing really the most pressing issue with Dub? -- /Jacob Carlborg
Agreed. A server approach would probably scale much better, if the intent is to speed up the developer's CTR cycle. But in any case, thanks to Symmetry for doing this! This is huge.
Sep 01
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 1 September 2020 at 12:59:00 UTC, Mathias LANG wrote:
 On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg 
 wrote:
 On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
 Looking for a full-time or part-time gig? Not only is 
 Symmetry Investments hiring D programmers, they are also 
 generously funding two positions for ecosystem work under the 
 D Language Foundation. And they've put up a bounty for a new 
 DUB feature. Read all about it here:

 https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/
As an alternative to use SHA-1 hashing. There's the option to have a daemon running the background listing on filesystem events. BTW, is timestamps vs SHA-1 hashing really the most pressing issue with Dub? -- /Jacob Carlborg
Agreed. A server approach would probably scale much better, if the intent is to speed up the developer's CTR cycle. But in any case, thanks to Symmetry for doing this! This is huge.
We need both a fs-watcher daemon support and "offline" incremental build support, based on SHA-256 (note that git is moving away from SHA1 to SHA256 [1]). I'd say SHA-256 is cheap enough these days [2] that I don't see a reason not to use it even for "online" fs-watcher daemon compilation. [1]: https://git-scm.com/docs/hash-function-transition/ [2]: https://bench.cr.yp.to/impl-hash/sha256.html
Sep 01
parent reply drug <drug2004 bk.ru> writes:
On 9/1/20 7:34 PM, Petar Kirov [ZombineDev] wrote:
 On Tuesday, 1 September 2020 at 12:59:00 UTC, Mathias LANG wrote:
 On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg wrote:
 On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
 Looking for a full-time or part-time gig? Not only is Symmetry 
 Investments hiring D programmers, they are also generously funding 
 two positions for ecosystem work under the D Language Foundation. 
 And they've put up a bounty for a new DUB feature. Read all about it 
 here:

 https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-fo
ndation-are-hiring/ 
As an alternative to use SHA-1 hashing. There's the option to have a daemon running the background listing on filesystem events. BTW, is timestamps vs SHA-1 hashing really the most pressing issue with Dub? -- /Jacob Carlborg
Agreed. A server approach would probably scale much better, if the intent is to speed up the developer's CTR cycle. But in any case, thanks to Symmetry for doing this! This is huge.
We need both a fs-watcher daemon support and "offline" incremental build support, based on SHA-256 (note that git is moving away from SHA1 to SHA256 [1]). I'd say SHA-256 is cheap enough these days [2] that I don't see a reason not to use it even for "online" fs-watcher daemon compilation. [1]: https://git-scm.com/docs/hash-function-transition/ [2]: https://bench.cr.yp.to/impl-hash/sha256.html
We can easily use the following option: ``` dub build --hash=sha1 dub build --hash=sha256 ``` and let the user to make the final choice
Sep 01
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 1 September 2020 at 16:45:55 UTC, drug wrote:
 On 9/1/20 7:34 PM, Petar Kirov [ZombineDev] wrote:
 On Tuesday, 1 September 2020 at 12:59:00 UTC, Mathias LANG 
 wrote:
 On Tuesday, 1 September 2020 at 09:09:36 UTC, Jacob Carlborg 
 wrote:
 [...]
Agreed. A server approach would probably scale much better, if the intent is to speed up the developer's CTR cycle. But in any case, thanks to Symmetry for doing this! This is huge.
We need both a fs-watcher daemon support and "offline" incremental build support, based on SHA-256 (note that git is moving away from SHA1 to SHA256 [1]). I'd say SHA-256 is cheap enough these days [2] that I don't see a reason not to use it even for "online" fs-watcher daemon compilation. [1]: https://git-scm.com/docs/hash-function-transition/ [2]: https://bench.cr.yp.to/impl-hash/sha256.html
We can easily use the following option: ``` dub build --hash=sha1 dub build --hash=sha256 ``` and let the user to make the final choice
I understand your idea to make this configurable, but this will introduce more complexity than necessary because then $HOME/.dub/packages will have build artifacts based on both SHA1 and SHA256, and so dub will need to support mixed mode file integrity checking.
Sep 02
parent reply drug <drug2004 bk.ru> writes:
On 9/2/20 12:53 PM, Petar Kirov [ZombineDev] wrote:
 
 I understand your idea to make this configurable, but this will 
 introduce more complexity than necessary because then 
 $HOME/.dub/packages will have build artifacts based on both SHA1 and 
 SHA256, and so dub will need to support mixed mode file integrity checking.
Yesterday I thought about it and now I believe that if you change sha1 to sha256 or vice versa then dub just rebuild all because sha1 and sha256 has different sizes so cached hashes will be different to calculated ones and collisions are impossible. No need to support mixed mode. The drawback is total rebuild on algorithm changing but I suppose it will be seldom operation. Another way is to having several hashes simultaneously and select one of them depending on option value. Implementation of this will be simple.
Sep 02
parent reply drug <drug2004 bk.ru> writes:
On 9/2/20 1:18 PM, drug wrote:
 it will be seldom operation. Another way is to having several hashes 
 simultaneously and select one of them depending on option value. 
I meant having several hashes in dub cache. Then if the user changes the hash algorithm dub checks calculated hashes against cached ones and avoids total rebuilding.
Sep 02
parent drug <drug2004 bk.ru> writes:
On 9/2/20 1:21 PM, drug wrote:
 On 9/2/20 1:18 PM, drug wrote:
 it will be seldom operation. Another way is to having several hashes 
 simultaneously and select one of them depending on option value. 
I meant having several hashes in dub cache. Then if the user changes the hash algorithm dub checks calculated hashes against cached ones and avoids total rebuilding.
Hmm, probably I was too confident. I've implemented support for both sha1 and sha256. It takes about 30 lines of additional code but what is worse the code uses classes now and dynamic memory allocation. I would avoid it. But in other it works as I expected: ``` dub build --hash=sha1 dub build --hash=sha256 ``` I do not test it in big projects but building dub does not show any difference in performance
Sep 02
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
 Looking for a full-time or part-time gig? Not only is Symmetry 
 Investments hiring D programmers, they are also generously 
 funding two positions for ecosystem work under the D Language 
 Foundation. And they've put up a bounty for a new DUB feature. 
 Read all about it here:

 https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/
This is a great initiative! It's also a great opportunity for up-and-coming D contributors. I know that if I had this chance back many years ago when I was unemployed I would have jumped at the opportunity. So if you're in a similar position don't hesitate to apply!
Sep 02
prev sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Sunday, 30 August 2020 at 14:13:36 UTC, Mike Parker wrote:
 Looking for a full-time or part-time gig? Not only is Symmetry 
 Investments hiring D programmers, they are also generously 
 funding two positions for ecosystem work under the D Language 
 Foundation. And they've put up a bounty for a new DUB feature. 
 Read all about it here:

 https://dlang.org/blog/2020/08/30/symmetry-investments-and-the-d-language-foundation-are-hiring/
One other thing that hashing enables is caching builds. So if you build, then change a file, then build again, and then revert the file again it would be great if the next build just restored the cached build from the very first build. The less time spent waiting on builds, the better.
Sep 05