www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Required DMD changes for Mir and few thoughts about D future

reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
 I think we need to make it a point to support Mir in dmd. -- 
 Andrei


Required features for Level 3:
1. https://issues.dlang.org/show_bug.cgi?id=16489
2. https://issues.dlang.org/show_bug.cgi?id=16488
3. AVX & AVX2 floating point vector arithmetic
4. Generic unaligned load/store like (like LDC loadUnaligned and 
storeUnaligned)
5. Generic routine to pack and unpack real and imaginary parts. 
For usage example, see 
https://github.com/libmir/mir/blob/master/source/mir/glas/internal/copy.d#L699

Level 1 and Level 2 requires additional features like automatic 
loop unrolling with associative math.

In the same time, I don't think that DMD support is important for 
D and Mir. The following features are _much_ more important:

1. Lightweight `nothrow  nogc` threads, implemented using 
`struct`s
2. Lightweight `nothrow  nogc` mutexes and barriers, implemented 
using `struct`s
3. Low level and detailed CPUID with proper cache sizes (already 
implemented for x86 here: https://github.com/libmir/cpuid)
4. Ability to use some phobos modules like std.traits, std.meta, 
std.complex.Complex (without ^^), std.experimental.ndslice, 
std.allocator without current (old) DRuntime, "module info" and 
other harmful for "better C" feature.
5. LDC compiler support for ARM, MIPS, MIPS64, Alpha
6. LDC SDK, which can work out of the box (including windows). It 
is required for D to be popular for scientists

This features are good not only for Mir, but for the D future. I 
think that D can not be more popular then Go and Rust. This is 
reality and I don't think that it is bad. Why? Because D can be a 
backend language fo Go and Rust. It can replace C/C++/Fortran in 
following fields:

1. Low level high performance computation libraries like BLAS, 
Eigen, Intel MKL, MAGMA, scaLAPACK, and many others
2. Close to metal drivers
3. OS kernels
4. Aerospace and related software
5. The Khronos Group software (!)

We have not new concurrents in this fields, but C/C++/Fortran. 
Our problem is that we trying to be more hight level language 
than the IT industry needs. Lets forget about Go and Rust 
"concurrents" and remember about the initial D goal, which is 
more actual then ever: replace C/C++ in the IT industry.

Best regards,
Ilya
Sep 26 2016
next sibling parent reply bachmeier <no spam.net> writes:
On Monday, 26 September 2016 at 18:43:38 UTC, Ilya Yaroshenko 
wrote:

 I think we need to make it a point to support Mir in dmd. -- 
 Andrei




[...]


 We have not new concurrents in this fields, but C/C++/Fortran. 
 Our problem is that we trying to be more hight level language 
 than the IT industry needs. Lets forget about Go and Rust 
 "concurrents" and remember about the initial D goal, which is 
 more actual then ever: replace C/C++ in the IT industry.

 Best regards,
 Ilya


To me, it seems pointless to support Mir in DMD. The 
"functionality" that it brings is speed. Nobody needing 
absolutely the fastest code is not going to have any interest in 
DMD. From what I understand of Mir (but correct me if I'm wrong) 
it is easy enough to replicate its functionality by linking to 
existing linear algebra libraries. Others can construct a 
compatible interface if that's something they want.

I think Ilya and everyone else working on Mir should push forward 
with the LDC-based project. Anything else is a distraction.
Sep 26 2016
next sibling parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Monday, 26 September 2016 at 19:49:27 UTC, bachmeier wrote:

 On Monday, 26 September 2016 at 18:43:38 UTC, Ilya Yaroshenko 
 wrote:

 [...]




 [...]


 [...]


 To me, it seems pointless to support Mir in DMD. The 
 "functionality" that it brings is speed. Nobody needing 
 absolutely the fastest code is not going to have any interest 
 in DMD. From what I understand of Mir (but correct me if I'm 
 wrong) it is easy enough to replicate its functionality by 
 linking to existing linear algebra libraries. Others can 
 construct a compatible interface if that's something they want.

 I think Ilya and everyone else working on Mir should push 
 forward with the LDC-based project. Anything else is a 
 distraction.


Yes, the same true for Mir too. A precompiled library based on 
top of Mir GLAS can be used with DMD.
Sep 26 2016
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Monday, 26 September 2016 at 20:11:19 UTC, Ilya Yaroshenko 
wrote:

 Yes, the same true for Mir too. A precompiled library based on 
 top of Mir GLAS can be used with DMD.


Ah, then no worries.
Sep 26 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/26/16 10:11 PM, Ilya Yaroshenko wrote:

 A precompiled library based on top of Mir GLAS can be used with DMD.


That would work out as long as interaction is seamless. Please advise. 
Overall: I think Ilya's work can make a real difference for D, and we 
can't afford it to not work with the reference implementation. -- Andrei
Sep 26 2016
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 26 September 2016 at 22:34:59 UTC, Andrei Alexandrescu 
wrote:

 On 9/26/16 10:11 PM, Ilya Yaroshenko wrote:

 A precompiled library based on top of Mir GLAS can be used 
 with DMD.


 That would work out as long as interaction is seamless. Please 
 advise. Overall: I think Ilya's work can make a real difference 
 for D, and we can't afford it to not work with the reference 
 implementation. -- Andrei


Mir will be (optionally) integrated with dcompute (once it is 
ready)
and dcompute CANNOT be compiled at all by dmd, although I suppose
you could use dmd for rapid development cycle, but you would have 
to
make dmd only figure out the .mangleof of the kernel (as the 
.mangleof
is required for the library-side API bashing magic) and not try 
to do any
semantic analysis or compilation, as it would fail horribly.
Sep 26 2016
parent reply Laeeth Isharc <laeethnospam nospam.laeeth.com> writes:
On Tuesday, 27 September 2016 at 01:55:17 UTC, Nicholas Wilson 
wrote:

 On Monday, 26 September 2016 at 22:34:59 UTC, Andrei 
 Alexandrescu wrote:

 On 9/26/16 10:11 PM, Ilya Yaroshenko wrote:

 A precompiled library based on top of Mir GLAS can be used 
 with DMD.


 That would work out as long as interaction is seamless. Please 
 advise. Overall: I think Ilya's work can make a real 
 difference for D, and we can't afford it to not work with the 
 reference implementation. -- Andrei


 Mir will be (optionally) integrated with dcompute (once it is 
 ready)
 and dcompute CANNOT be compiled at all by dmd, although I 
 suppose
 you could use dmd for rapid development cycle, but you would 
 have to
 make dmd only figure out the .mangleof of the kernel (as the 
 .mangleof
 is required for the library-side API bashing magic) and not try 
 to do any
 semantic analysis or compilation, as it would fail horribly.


Can dcompute even be compiled by stock ldc?  If so, you should 
change the documents as code.dlang.org suggests otherwise.

As I understand it dcompute is a GPU library.   Not everyone will 
want to or need GPU for doing work with matrices,  so even 
if/when dcompute can be compiled by stock ldc, I wonder if it 
makes sense to prevent Mir being compiled without the dependency. 
   You say it us optional,  but then the rest of your message when 
read quickly almost suggests it's difficult not to include it.

The emphasis is that mir is faster than openblas,  which is 
great,  and natural as a library author to be proud of.   I guess 
to displace other options in other languages you will also need 
to be faster than any other sensible choices,  including Blaze 
(and I don't know if this is yet the case).   It's a tough sell 
persuading the Julia and Rust guys to use your library instead,  
because people aren't completely rational about languages,  and 
it's almost an admission of defeat for them to adopt your 
solution permanently (more likely in my view us they try to use 
your tricks to improve their own performance) - it would need to 
be a lot better,  is my guess.

Whatever the outcome there is, the emphasis in superior 
performance isn't the only one that users will have.   If you 
want to do matrix work in D, then having a sensible native 
solution is much easier - speed is nice,  but mostly about ease 
of use and reduction in complexity and context switching.   Most 
people are not at the frontier of performance needs,  so this 
point applies to many potential users, not a minority.
Sep 26 2016
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
 Can dcompute even be compiled by stock ldc?  If so, you should 
 change the documents as code.dlang.org suggests otherwise.


PR is open, CI is green, but needs some more work before it will 
be accepted.


 As I understand it dcompute is a GPU library.   Not everyone 
 will want to or need GPU for doing work with matrices,  so even 
 if/when dcompute can be compiled by stock ldc, I wonder if it 
 makes sense to prevent Mir being compiled without the 
 dependency.


DCompute will target OpenCL and CUDA so it can run on the cpu as 
well.
OpenCL requires a "non-standard" llvm.

   You say it us optional,  but then the rest of your message 
 when read quickly almost suggests it's difficult not to include 
 it.


As in it will be a setting in mir to support dcompute.
It will be difficult to do the 'compile kernels with ldc and the 
rest with dmd'
trick that Ilya suggested for using mir with dmd.

 The emphasis is that mir is faster than openblas,  which is 
 great,  and natural as a library author to be proud of.   I 
 guess to displace other options in other languages you will 
 also need to be faster than any other sensible choices,  
 including Blaze (and I don't know if this is yet the case).   
 It's a tough sell persuading the Julia and Rust guys to use 
 your library instead,  because people aren't completely 
 rational about languages,  and it's almost an admission of 
 defeat for them to adopt your solution permanently (more likely 
 in my view us they try to use your tricks to improve their own 
 performance) - it would need to be a lot better,  is my guess.

 Whatever the outcome there is, the emphasis in superior 
 performance isn't the only one that users will have.   If you 
 want to do matrix work in D, then having a sensible native 
 solution is much easier - speed is nice,  but mostly about ease 
 of use and reduction in complexity and context switching.


I can't speak for mir, but I intend dcompute to be as user 
friendly as CUDA.


 Most people are not at the frontier of performance needs,  so 
 this point applies to many potential users, not a minority.
Sep 26 2016
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On Monday, 26 September 2016 at 22:34:59 UTC, Andrei Alexandrescu 
wrote:

 That would work out as long as interaction is seamless. Please 
 advise. Overall: I think Ilya's work can make a real difference 
 for D, and we can't afford it to not work with the reference 
 implementation. -- Andrei


There is no point in running number crunching with a crappy 
optimizer.
The time to make it somehow still work, albeit slow, is much 
better spent on completing the science ecosystem.
Sep 29 2016
next sibling parent Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Thursday, 29 September 2016 at 09:22:56 UTC, Martin Nowak 
wrote:

 On Monday, 26 September 2016 at 22:34:59 UTC, Andrei 
 Alexandrescu wrote:

 That would work out as long as interaction is seamless. Please 
 advise. Overall: I think Ilya's work can make a real 
 difference for D, and we can't afford it to not work with the 
 reference implementation. -- Andrei


 There is no point in running number crunching with a crappy 
 optimizer.


But there _is_ a point in _developing_ number crunching software 
with said compiler, and switch to a better one in production.
Sep 29 2016
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On Thursday, 29 September 2016 at 09:22:56 UTC, Martin Nowak 
wrote:

 On Monday, 26 September 2016 at 22:34:59 UTC, Andrei 
 Alexandrescu wrote:

 That would work out as long as interaction is seamless. Please 
 advise. Overall: I think Ilya's work can make a real 
 difference for D, and we can't afford it to not work with the 
 reference implementation. -- Andrei




Yes agreed, we don't want this to become the reference 
implementation.
But just as the OP mentioned I think the cost for a workable dmd 
support is fairly high, in terms of missing dmd features and 
porting mir to dmd.

Integrating this with a pre-compiled ldc library is a fantastic 
idea OTOH.
If we can make this work, it will be much less effort and yield 
the fastest implementation. Also would speed up the development 
cycle a bit b/c the kernels don't need to be recompiled/optimized.
Oct 08 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/8/16 1:22 PM, Martin Nowak wrote:

 Integrating this with a pre-compiled ldc library is a fantastic idea OTOH.
 If we can make this work, it will be much less effort and yield the
 fastest implementation. Also would speed up the development cycle a bit
 b/c the kernels don't need to be recompiled/optimized.


You mean dmd/ldc/etc interop at binary level? Yes, that would be pretty 
rad indeed! -- Andrei
Oct 08 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/8/16 2:49 PM, Andrei Alexandrescu wrote:

 On 10/8/16 1:22 PM, Martin Nowak wrote:

 Integrating this with a pre-compiled ldc library is a fantastic idea
 OTOH.
 If we can make this work, it will be much less effort and yield the
 fastest implementation. Also would speed up the development cycle a bit
 b/c the kernels don't need to be recompiled/optimized.


 You mean dmd/ldc/etc interop at binary level? Yes, that would be pretty
 rad indeed! -- Andrei


(after thinking a bit more) ... but Mir seems to rely in good part on 
templates, which makes pre-compiled libraries less effective. -- Andrei
Oct 08 2016
next sibling parent reply Martin Nowak <code dawg.eu> writes:
On Saturday, 8 October 2016 at 18:53:32 UTC, Andrei Alexandrescu 
wrote:

 You mean dmd/ldc/etc interop at binary level? Yes, that would 
 be pretty




Should already work, but of courses isn't well tested.


 (after thinking a bit more) ... but Mir seems to rely in good 
 part on templates, which makes pre-compiled libraries less 
 effective. -- Andrei


Exactly, this is what I was wondering. Maybe it uses a finite set 
of precompilable kernels?
Oct 09 2016
parent Martin Nowak <code dawg.eu> writes:
On Monday, 10 October 2016 at 05:20:56 UTC, Martin Nowak wrote:

 (after thinking a bit more) ... but Mir seems to rely in good 
 part on templates, which makes pre-compiled libraries less 
 effective. -- Andrei


 Exactly, this is what I was wondering. Maybe it uses a finite 
 set of precompilable kernels?


Well at least for gemm pointers to kernels are used, but they are 
still templated atm.
https://github.com/libmir/mir/blob/f7a904161df7af0a8443a0237a958460432f980c/source/mir/glas/internal/gemm.d#L97
Oct 09 2016
prev sibling next sibling parent reply Martin Nowak <code dawg.eu> writes:
On Saturday, 8 October 2016 at 18:53:32 UTC, Andrei Alexandrescu 
wrote:

 (after thinking a bit more) ... but Mir seems to rely in good 
 part on templates, which makes pre-compiled libraries less 
 effective. -- Andrei


On Saturday, 8 October 2016 at 18:53:32 UTC, Andrei Alexandrescu 
wrote:
Ilya's answer
http://forum.dlang.org/post/rexuwvohqceaglcbrico forum.dlang.org

Sounds like a feasible approach for phobos inclusion w/ prolly 
very little usability restrictions on the generic API wrapping 
those.
Oct 09 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/10/16 2:05 AM, Martin Nowak wrote:

 On Saturday, 8 October 2016 at 18:53:32 UTC, Andrei Alexandrescu wrote:

 (after thinking a bit more) ... but Mir seems to rely in good part on
 templates, which makes pre-compiled libraries less effective. -- Andrei


 On Saturday, 8 October 2016 at 18:53:32 UTC, Andrei Alexandrescu wrote:
 Ilya's answer
 http://forum.dlang.org/post/rexuwvohqceaglcbrico forum.dlang.org

 Sounds like a feasible approach for phobos inclusion w/ prolly very
 little usability restrictions on the generic API wrapping those.


Yes, after talking to him this seems definitely a worthwhile pursuit. -- 
Andrei
Oct 10 2016
prev sibling parent Matthias Klumpp <matthias tenstral.net> writes:
On Saturday, 8 October 2016 at 18:53:32 UTC, Andrei Alexandrescu 
wrote:

 On 10/8/16 2:49 PM, Andrei Alexandrescu wrote:

 On 10/8/16 1:22 PM, Martin Nowak wrote:

 Integrating this with a pre-compiled ldc library is a 
 fantastic idea
 OTOH.
 If we can make this work, it will be much less effort and 
 yield the
 fastest implementation. Also would speed up the development 
 cycle a bit
 b/c the kernels don't need to be recompiled/optimized.


 You mean dmd/ldc/etc interop at binary level? Yes, that would 
 be pretty
 rad indeed! -- Andrei


 (after thinking a bit more) ... but Mir seems to rely in good 
 part on templates, which makes pre-compiled libraries less 
 effective. -- Andrei


Independent from Mir, a stable ABI for D which all compilers 
follow would be a tremendous win, especially from the perspective 
of shipping D stuff in Linux distributions.
So maybe this is worth attempting?
Oct 10 2016
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Monday, 26 September 2016 at 20:11:19 UTC, Ilya Yaroshenko 
wrote:

 Yes, the same true for Mir too. A precompiled library based on 
 top of Mir GLAS can be used with DMD.


Are you sure about this?  I thought there were ABI 
incompatibilities between D compilers.

On Monday, 26 September 2016 at 22:34:59 UTC, Andrei Alexandrescu 
wrote:

 That would work out as long as interaction is seamless. Please 
 advise. Overall: I think Ilya's work can make a real difference 
 for D, and we can't afford it to not work with the reference 
 implementation. -- Andrei


Why not?  I think people will understand that ldc is meant for 
higher performance, which you want from such code anyway.
Sep 26 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/27/16 2:52 AM, Joakim wrote:

 Why not?  I think people will understand that ldc is meant for higher
 performance, which you want from such code anyway.


I'm not going to argue this much further. Essentially Mir is touted as a 
highly generic and portable library. Having it only work on one language 
implementation works against that statement, the credibility of Mir, and 
the credibility of D as an universal platform for creating fast code. 
One way or another we need to get Mir to be portable across all D 
implementations and hopefully convince Ilya to move it into D's standard 
library.

Put another way: we have a beautiful use case of what primitives we need 
to offer to allow definition of a generic library that beats the pants 
off hand-optimized assembler. We should be all over implementing said 
primitives. There's clear proof and there's a clean short list of things 
we need to do. If we're asking ourselves "what kind of primitives we 
should add to D to make it awesomely fast?" this is the best experience 
report we could ever hope for.

Instead of debating minor things herein, and instead of adding "little 
projects that could" to an ever-growing wishlist, we should rally behind 
a real strong project and take it from provincial notability to global 
success. I've said in the past I'm not that good, but I am good enough 
to recognize great work. Ilya's work is that.


Andrei
Sep 26 2016
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 27 September 2016 at 01:17:16 UTC, Andrei 
Alexandrescu wrote:

 I'm not going to argue this much further. Essentially Mir is 
 touted as a highly generic and portable library. Having it only 
 work on one language implementation works against that 
 statement, the credibility of Mir, and the credibility of D as 
 an universal platform for creating fast code. One way or 
 another we need to get Mir to be portable across all D 
 implementations and hopefully convince Ilya to move it into D's 
 standard library.

 [...]


In order to create code as efficient as ldc or gdc we need to 
rewrite dmds backend.

Are you suggesting that ?

Or are you arguing for importing a huge amount of pre-generated 
asm into phobos,
which would be much more likely to succeed.
Sep 26 2016
prev sibling next sibling parent pham <home home.com> writes:
On Tuesday, 27 September 2016 at 01:17:16 UTC, Andrei 
Alexandrescu wrote:

 On 9/27/16 2:52 AM, Joakim wrote:

 Why not?  I think people will understand that ldc is meant for 
 higher
 performance, which you want from such code anyway.




......


 provincial notability to global success. I've said in the past 
 I'm not that good, but I am good enough to recognize great 
 work. Ilya's work is that.


 Andrei



Better support inline. i am not an expert but dmd can not handle 
inline if there are few if.. else
Sep 26 2016
prev sibling next sibling parent reply Temtaime <temtaime gmail.com> writes:
On Tuesday, 27 September 2016 at 01:17:16 UTC, Andrei 
Alexandrescu wrote:

 On 9/27/16 2:52 AM, Joakim wrote:

 Why not?  I think people will understand that ldc is meant for 
 higher
 performance, which you want from such code anyway.


 I'm not going to argue this much further. Essentially Mir is 
 touted as a highly generic and portable library. Having it only 
 work on one language implementation works against that 
 statement, the credibility of Mir, and the credibility of D as 
 an universal platform for creating fast code.


Just let dmd die.
Sep 27 2016
parent Temtaime <temtaime gmail.com> writes:
On Tuesday, 27 September 2016 at 08:40:38 UTC, Temtaime wrote:

 On Tuesday, 27 September 2016 at 01:17:16 UTC, Andrei 
 Alexandrescu wrote:

 On 9/27/16 2:52 AM, Joakim wrote:

 Why not?  I think people will understand that ldc is meant 
 for higher
 performance, which you want from such code anyway.


 I'm not going to argue this much further. Essentially Mir is 
 touted as a highly generic and portable library. Having it 
 only work on one language implementation works against that 
 statement, the credibility of Mir, and the credibility of D as 
 an universal platform for creating fast code.


 Just let dmd die.


This so-called "reference implementation" makes people suffer 
because of code portability and another differences between the 
compilers (e.g. compiler building, compiler's command line 
arguments, compiler's extensions).
Sep 27 2016
prev sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Tuesday, 27 September 2016 at 01:17:16 UTC, Andrei 
Alexandrescu wrote:

 I'm not going to argue this much further. Essentially Mir is 
 touted as a highly generic and portable library. Having it only 
 work on one language implementation works against that 
 statement, the credibility of Mir, and the credibility of D as 
 an universal platform for creating fast code.


Isn't it just a matter of adding "version(LDC)" around the more 
optimized blocks?
Having it work in DMD, however slower, is good enough.
Sep 27 2016
next sibling parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 27 September 2016 at 10:44:28 UTC, Guillaume Piolat 
wrote:

 On Tuesday, 27 September 2016 at 01:17:16 UTC, Andrei 
 Alexandrescu wrote:

 I'm not going to argue this much further. Essentially Mir is 
 touted as a highly generic and portable library. Having it 
 only work on one language implementation works against that 
 statement, the credibility of Mir, and the credibility of D as 
 an universal platform for creating fast code.


 Isn't it just a matter of adding "version(LDC)" around the more 
 optimized blocks?
 Having it work in DMD, however slower, is good enough.


50 times slower for modern CPUs.
Sep 27 2016
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/27/2016 3:48 AM, Ilya Yaroshenko wrote:

 50 times slower for modern CPUs.


I understand, and that's a drastic speed difference. But removing support for 
dmd can have the effect of balkanizing the D community. That's happened in the 
past, and it was terrible for all of us.

I have looked at:

   https://github.com/libmir/mir/pull/347/files

I propose as a first step we can work towards eliminating the conditional 
compilation based on which compiler is running. Then we can look for low
hanging 
fruit on the codegen to reduce the performance gap.
Sep 27 2016
prev sibling next sibling parent Guillaume Piolat <first.last gmail.com> writes:
On Tuesday, 27 September 2016 at 10:48:40 UTC, Ilya Yaroshenko 
wrote:

 On Tuesday, 27 September 2016 at 10:44:28 UTC, Guillaume Piolat 
 wrote:

 On Tuesday, 27 September 2016 at 01:17:16 UTC, Andrei 
 Alexandrescu wrote:

 I'm not going to argue this much further. Essentially Mir is 
 touted as a highly generic and portable library. Having it 
 only work on one language implementation works against that 
 statement, the credibility of Mir, and the credibility of D 
 as an universal platform for creating fast code.


 Isn't it just a matter of adding "version(LDC)" around the 
 more optimized blocks?
 Having it work in DMD, however slower, is good enough.


 50 times slower for modern CPUs.


version (DMD)
{
     pragma(message, "It seems you are building with DMD instead 
of LDC. Be aware that some functions will go up to 50x slower");
}

=> no misrepresentation of Mir performance
=> the user testing Mir keeps testing instead of having it not 
working
Sep 27 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/27/16 12:48 PM, Ilya Yaroshenko wrote:

 On Tuesday, 27 September 2016 at 10:44:28 UTC, Guillaume Piolat wrote:

 On Tuesday, 27 September 2016 at 01:17:16 UTC, Andrei Alexandrescu wrote:

 I'm not going to argue this much further. Essentially Mir is touted
 as a highly generic and portable library. Having it only work on one
 language implementation works against that statement, the credibility
 of Mir, and the credibility of D as an universal platform for
 creating fast code.


 Isn't it just a matter of adding "version(LDC)" around the more
 optimized blocks?
 Having it work in DMD, however slower, is good enough.


 50 times slower for modern CPUs.


Just to reiterate my point: I'd say a 30% penalty would be still within 
batting range of the best out there and would be my lower bound for 
declaring dmd/Mir a success. -- Andrei
Sep 27 2016
prev sibling parent reply Johan Engelen <j j.nl> writes:
On Tuesday, 27 September 2016 at 10:44:28 UTC, Guillaume Piolat 
wrote:

 On Tuesday, 27 September 2016 at 01:17:16 UTC, Andrei 
 Alexandrescu wrote:

 I'm not going to argue this much further. Essentially Mir is 
 touted as a highly generic and portable library. Having it 
 only work on one language implementation works against that 
 statement, the credibility of Mir, and the credibility of D as 
 an universal platform for creating fast code.


 Isn't it just a matter of adding "version(LDC)" around the more 
 optimized blocks?
 Having it work in DMD, however slower, is good enough.


(copying from the previous thread:)

I thought so too but if the algorithm is 50x slower, it probably 
means you can't develop that algorithm any more (I wouldn't). I 
think the common use-case for Mir is a calculation that takes 
seconds, so 50x turns a test into a run of several minutes, 
defeating the compilation speed advantage of DMD. The way I see 
it, faster development with Mir+DMD is not possible.

It is easy to want something, but someone else has to do it and 
live with it too. It's up to the Mir devs (**volunteers!**) to 
choose which compilers they support. As you can see from the PR 
that removed DMD support, the extra burden is substantial.
https://github.com/libmir/mir/pull/347

An extra subjective comment from recent experience: I think LDC 
has been very responsive to Mir's needs, thinking _with_ Mir 
development instead of fighting it and debating things to death. 
Imagine you are developing Mir, want to get something done, and 
then read the discussion starting here
https://forum.dlang.org/post/brieiuuuslpzfeioxuql forum.dlang.org
The LDC PR with the requested functionality was submitted less 
than two weeks after
(pull was stalled because we don't control our own frontend).
Sep 27 2016
next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Tuesday, 27 September 2016 at 11:33:54 UTC, Johan Engelen 
wrote:

 An extra subjective comment from recent experience: I think LDC 
 has been very responsive to Mir's needs, thinking _with_ Mir 
 development instead of fighting it and debating things to 
 death. Imagine you are developing Mir, want to get something 
 done, and then read the discussion starting here
 https://forum.dlang.org/post/brieiuuuslpzfeioxuql forum.dlang.org
 The LDC PR with the requested functionality was submitted less 
 than two weeks after
 (pull was stalled because we don't control our own frontend).


I've read the thread. I think Walter was right and correctly 
advised you the whole thread. One thing Mir users will do is 
copying executables from machine to machines. If it has runtime 
dispatch, it will work everywhere.
Sep 27 2016
parent Johan Engelen <j j.nl> writes:
On Tuesday, 27 September 2016 at 13:36:50 UTC, Guillaume Piolat 
wrote:

 
 If it has runtime dispatch, it will work everywhere.


I'm sorry but, like others, clearly you have not understood the 
issue.
Sep 27 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/27/16 1:33 PM, Johan Engelen wrote:

 I thought so too but if the algorithm is 50x slower, it probably means
 you can't develop that algorithm any more (I wouldn't). I think the
 common use-case for Mir is a calculation that takes seconds, so 50x
 turns a test into a run of several minutes, defeating the compilation
 speed advantage of DMD. The way I see it, faster development with
 Mir+DMD is not possible.


Yes, 50x is not going to pass. In my experience, generally (not for 
numerics code) dmd generated code is somewhat in between 5% faster and 
40% slower. I'd say if we get it within the same range for Ilya's 
library we can call it a successful port.


 It is easy to want something, but someone else has to do it and live
 with it too. It's up to the Mir devs (**volunteers!**) to choose which
 compilers they support. As you can see from the PR that removed DMD
 support, the extra burden is substantial.
 https://github.com/libmir/mir/pull/347


We need to do what it takes to make dmd work with Mir properly.


 An extra subjective comment from recent experience: I think LDC has been
 very responsive to Mir's needs, thinking _with_ Mir development instead
 of fighting it and debating things to death. Imagine you are developing
 Mir, want to get something done, and then read the discussion starting here
 https://forum.dlang.org/post/brieiuuuslpzfeioxuql forum.dlang.org
 The LDC PR with the requested functionality was submitted less than two
 weeks after
 (pull was stalled because we don't control our own frontend).


That's far from the worst we've had, but I agree that generally the 
endless debates are often gridlocking our progress.


Andrei
Sep 27 2016
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-09-27 02:52, Joakim wrote:

 On Monday, 26 September 2016 at 20:11:19 UTC, Ilya Yaroshenko wrote:

 Yes, the same true for Mir too. A precompiled library based on top of
 Mir GLAS can be used with DMD.


 Are you sure about this?  I thought there were ABI incompatibilities
 between D compilers.


They're not ABI compatible.

-- 
/Jacob Carlborg
Sep 27 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 27 September 2016 at 07:01:37 UTC, Jacob Carlborg 
wrote:

 On 2016-09-27 02:52, Joakim wrote:

 On Monday, 26 September 2016 at 20:11:19 UTC, Ilya Yaroshenko 
 wrote:

 Yes, the same true for Mir too. A precompiled library based 
 on top of
 Mir GLAS can be used with DMD.


 Are you sure about this?  I thought there were ABI 
 incompatibilities
 between D compilers.


 They're not ABI compatible.


They are (except Win32?), but we need to exclude DRuntime 
dependency. Mir does not use Druntime and will not use it anyway 
because DRuntime is not `nothrow  nogc`.
Sep 27 2016
next sibling parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 27 September 2016 at 08:53:36 UTC, Ilya Yaroshenko 
wrote:

 On Tuesday, 27 September 2016 at 07:01:37 UTC, Jacob Carlborg 
 wrote:

 On 2016-09-27 02:52, Joakim wrote:

 On Monday, 26 September 2016 at 20:11:19 UTC, Ilya Yaroshenko 
 wrote:

 Yes, the same true for Mir too. A precompiled library based 
 on top of
 Mir GLAS can be used with DMD.


 Are you sure about this?  I thought there were ABI 
 incompatibilities
 between D compilers.


 They're not ABI compatible.


 They are (except Win32?), but we need to exclude DRuntime 
 dependency. Mir does not use Druntime and will not use it 
 anyway because DRuntime is not `nothrow  nogc`.


I mean extern(C) code
Sep 27 2016
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2016-09-27 10:53, Ilya Yaroshenko wrote:


 They are (except Win32?), but we need to exclude DRuntime dependency.
 Mir does not use Druntime and will not use it anyway because DRuntime is
 not `nothrow  nogc`.


I tried this using ldc 1.1.0 beta 2 (Frontend, druntime and Phobos are 
at version 2.071.1) and DMD 2.071.1:

$ cat main.d
module main;

import std.stdio;

void main()
{
     writeln("foo");
}

$ ./ldc2 -c main.d && dmd main.o
Undefined symbols for architecture x86_64:
   "__D3std3utf6toUTF8FNaNbNiNfNkJG4awZAa", referenced from:
       __D3std5stdio4File17LockingTextWriter10__T3putTwZ3putMFNbNiNfwZv 
in main.o
   "__D3std5stdio12__ModuleInfoZ", referenced from:
       __D4main12__ModuleInfoZ in main.o
   "__D3std5stdio13trustedStdoutFNdNeZS3std5stdio4File", referenced from:
       __D3std5stdio16__T7writelnTAyaZ7writelnFNfAyaZv in main.o
   "__D3std5stdio14fputc_unlockedFNbNiiPS4core4stdc5stdio7__sFILEZi", 
referenced from:
 
__D3std5stdio4File17LockingTextWriter10__T3putTwZ3putMFNfwZ12trustedFPUTCFNbNiNeiPS4core4
tdc5stdio7__sFILEZi 
in main.o
 
__D3std5stdio4File17LockingTextWriter10__T3putTaZ3putMFNfaZ12trustedFPUTCFNbNiNeiPS4core4
tdc5stdio7__sFILEZi 
in main.o
   "__D3std5stdio15fputwc_unlockedFNbNiwPS4core4stdc5stdio7__sFILEZi", 
referenced from:
 
__D3std5stdio4File17LockingTextWriter10__T3putTwZ3putMFNfwZ13trustedFPUTWCFNbNiNewPS4core4
tdc5stdio7__sFILEZi 
in main.o
 
__D3std5stdio4File17LockingTextWriter10__T3putTaZ3putMFNfaZ13trustedFPUTWCFNbNiNewPS4core4
tdc5stdio7__sFILEZi 
in main.o
   "__D3std5stdio4File17LockingTextWriter6__dtorMFNeZv", referenced from:
       __D3std5stdio16__T7writelnTAyaZ7writelnFNfAyaZv in main.o
 
"__D3std5stdio4File17lockingTextWriterMFNfZS3std5stdio4File1
LockingTextWriter", 
referenced from:
       __D3std5stdio16__T7writelnTAyaZ7writelnFNfAyaZv in main.o
   "__D3std5stdio4File6__dtorMFNfZv", referenced from:
       __D3std5stdio16__T7writelnTAyaZ7writelnFNfAyaZv in main.o
 
"__D3std9exception14ErrnoException6__ctorMFNeAyaAyamZC3std9excepti
n14ErrnoException", 
referenced from:
       __D3std5stdio4File17LockingTextWriter12__T3putTAyaZ3putMFNfAyaZv 
in main.o
 
__D3std9exception187__T12errnoEnforceTiVAyaa76_2f55736572732f6a61636f622f446f776e6c6f6164732f6c6463322d312e312e302d62657461322d6f73782d7838365f36342f62696e2f2e2e2f696d706f72742f7374642f737464696f2e64Vmi2640Z12er
noEnforceFNfiLAyaZi 
in main.o
   "__D3std9exception14ErrnoException6__initZ", referenced from:
       __D3std5stdio4File17LockingTextWriter12__T3putTAyaZ3putMFNfAyaZv 
in main.o
 
__D3std9exception187__T12errnoEnforceTiVAyaa76_2f55736572732f6a61636f622f446f776e6c6f6164732f6c6463322d312e312e302d62657461322d6f73782d7838365f36342f62696e2f2e2e2f696d706f72742f7374642f737464696f2e64Vmi2640Z12er
noEnforceFNfiLAyaZi 
in main.o
   "__D3std9exception14ErrnoException6__vtblZ", referenced from:
       __D3std5stdio4File17LockingTextWriter12__T3putTAyaZ3putMFNfAyaZv 
in main.o
 
__D3std9exception187__T12errnoEnforceTiVAyaa76_2f55736572732f6a61636f622f446f776e6c6f6164732f6c6463322d312e312e302d62657461322d6f73782d7838365f36342f62696e2f2e2e2f696d706f72742f7374642f737464696f2e64Vmi2640Z12er
noEnforceFNfiLAyaZi 
in main.o
   "__D3std9exception14ErrnoException7__ClassZ", referenced from:
       __D3std5stdio4File17LockingTextWriter12__T3putTAyaZ3putMFNfAyaZv 
in main.o
 
__D3std9exception187__T12errnoEnforceTiVAyaa76_2f55736572732f6a61636f622f446f776e6c6f6164732f6c6463322d312e312e302d62657461322d6f73782d7838365f36342f62696e2f2e2e2f696d706f72742f7374642f737464696f2e64Vmi2640Z12er
noEnforceFNfiLAyaZi 
in main.o
   "__Dmodule_ref", referenced from:
       __D4main16__moduleinfoCtorZ in main.o
   "__d_eh_personality", referenced from:
       Dwarf Exception Unwind Info (__eh_frame) in main.o
   "__d_eh_resume_unwind", referenced from:
       __D3std5stdio16__T7writelnTAyaZ7writelnFNfAyaZv in main.o
   "__d_throw_exception", referenced from:
       __D3std5stdio4File17LockingTextWriter12__T3putTAyaZ3putMFNfAyaZv 
in main.o
 
__D3std9exception187__T12errnoEnforceTiVAyaa76_2f55736572732f6a61636f622f446f776e6c6f6164732f6c6463322d312e312e302d62657461322d6f73782d7838365f36342f62696e2f2e2e2f696d706f72742f7374642f737464696f2e64Vmi2640Z12er
noEnforceFNfiLAyaZi 
in main.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see 
invocation)
--- errorlevel 1

-- 
/Jacob Carlborg
Sep 27 2016
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On Monday, 26 September 2016 at 20:11:19 UTC, Ilya Yaroshenko 
wrote:

 Yes, the same true for Mir too. A precompiled library based on 
 top of Mir GLAS can be used with DMD.


Is this feasible, i.e. is there a finite amount of kernels that 
we can precompile and use?
I thought the kernels were fully template generated, but your 
idea suggests sth. different.
Oct 08 2016
parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Saturday, 8 October 2016 at 17:28:14 UTC, Martin Nowak wrote:

 On Monday, 26 September 2016 at 20:11:19 UTC, Ilya Yaroshenko 
 wrote:

 Yes, the same true for Mir too. A precompiled library based on 
 top of Mir GLAS can be used with DMD.


 Is this feasible, i.e. is there a finite amount of kernels that 
 we can precompile and use?
 I thought the kernels were fully template generated, but your 
 idea suggests sth. different.


Mir is generic library and it will have few dub configurations. 
For example a configuration which provides precompiled extern(C) 
BLAS API for common types. This is useful for D as for other 
languages like C or Julia. In addition, it allows to have few 
precompiled versions, optimized for different CPUs: x87, SSE2, 
AVX intel, AVX amd, AVX2 intel, AVX2 amd, and etc. The proper 
configuration may be chosen RT or CT.
Oct 08 2016
prev sibling next sibling parent Laeeth Isharc <laeethnospam nospam.laeeth.com> writes:
On Monday, 26 September 2016 at 19:49:27 UTC, bachmeier wrote:

 On Monday, 26 September 2016 at 18:43:38 UTC, Ilya Yaroshenko 
 wrote:

 I think we need to make it a point to support Mir in dmd. -- 
 Andrei




 [...]


 We have not new concurrents in this fields, but C/C++/Fortran. 
 Our problem is that we trying to be more hight level language 
 than the IT industry needs. Lets forget about Go and Rust 
 "concurrents" and remember about the initial D goal, which is 
 more actual then ever: replace C/C++ in the IT industry.

 Best regards,
 Ilya


 To me, it seems pointless to support Mir in DMD. The 
 "functionality" that it brings is speed. Nobody needing 
 absolutely the fastest code is not going to have any interest 
 in DMD. From what I understand of Mir (but correct me if I'm 
 wrong) it is easy enough to replicate its functionality by 
 linking to existing linear algebra libraries. Others can 
 construct a compatible interface if that's something they want.

 I think Ilya and everyone else working on Mir should push 
 forward with the LDC-based project. Anything else is a 
 distraction.


The functionality that it brings is also ease of use and 
simplicity if what you are doing is not only matrix stuff and 
will have other code in D.  It's an impressive effort,  and 
congratulations to Ilya and others.

DMD is useful for fast edit / compile cycle.   Maybe there are 
some advantages for gdc too for some people in some 
circumstances.  It's behind currently,  but in the respects that 
matter here I don't suppose it will always be so. Sometimes LDC 
has bugs that makes it difficult to compile one's code,  and new 
features will always show up on DMD first.

It's a pity to fragment the ecosystem - to potentially have a 
library that works on one of three main compilers, but that's 
sufficiently good that nobody else will write a competing library 
that works on all compilers.

It's not for your or me to tell Ilya and colleagues what to do,  
and it certainly won't do any good to do so.   One could access 
Mir in DMD from a D wrapper on top of C bindings to Mir,  but is 
anyone likely to write these and keep them up to date? I am not 
so sure.

Everyone is entitled to their own view about the proper role of 
D,  and I know Ilya has strong views on this topic,  and that's 
normal because people think in different ways and is in a 
different situation.

So I think Andrei's goal makes much sense strategically,  though 
I have no idea how it might be achieved.


Laeeth
Sep 26 2016
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-09-26 21:49, bachmeier wrote:


 To me, it seems pointless to support Mir in DMD. The "functionality"
 that it brings is speed. Nobody needing absolutely the fastest code is
 not going to have any interest in DMD. From what I understand of Mir
 (but correct me if I'm wrong) it is easy enough to replicate its
 functionality by linking to existing linear algebra libraries. Others
 can construct a compatible interface if that's something they want.

 I think Ilya and everyone else working on Mir should push forward with
 the LDC-based project. Anything else is a distraction.


He mentions several front end issues, those apply to LDC as well.

-- 
/Jacob Carlborg
Sep 27 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 27 September 2016 at 07:01:08 UTC, Jacob Carlborg 
wrote:

 On 2016-09-26 21:49, bachmeier wrote:


 To me, it seems pointless to support Mir in DMD. The 
 "functionality"
 that it brings is speed. Nobody needing absolutely the fastest 
 code is
 not going to have any interest in DMD. From what I understand 
 of Mir
 (but correct me if I'm wrong) it is easy enough to replicate 
 its
 functionality by linking to existing linear algebra libraries. 
 Others
 can construct a compatible interface if that's something they 
 want.

 I think Ilya and everyone else working on Mir should push 
 forward with
 the LDC-based project. Anything else is a distraction.


 He mentions several front end issues, those apply to LDC as 
 well.


No, LDC for example does not have the issue 
https://issues.dlang.org/show_bug.cgi?id=16488 and others too.
Sep 27 2016
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-09-27 10:50, Ilya Yaroshenko wrote:


 No, LDC for example does not have the issue
 https://issues.dlang.org/show_bug.cgi?id=16488 and others too.


Does LDC has: "Lightweight `nothrow  nogc` threads, implemented using 
`struct`s" ?

-- 
/Jacob Carlborg
Sep 27 2016
parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 27 September 2016 at 09:20:28 UTC, Jacob Carlborg 
wrote:

 On 2016-09-27 10:50, Ilya Yaroshenko wrote:


 No, LDC for example does not have the issue
 https://issues.dlang.org/show_bug.cgi?id=16488 and others too.


 Does LDC has: "Lightweight `nothrow  nogc` threads, implemented 
 using `struct`s" ?


They are coming soon. LLVM has OpenCL and CUDA backend. OpenCL 
can be used both for CPU and GPU targets. So, yes, Mir will have 
[1, 2] lightweight `nothrow  nogc` threads with LDC.

1. https://github.com/libmir/dcompute
2. https://github.com/ldc-developers/ldc/pull/1786
Sep 27 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/27/16 10:50 AM, Ilya Yaroshenko wrote:

 On Tuesday, 27 September 2016 at 07:01:08 UTC, Jacob Carlborg wrote:

 On 2016-09-26 21:49, bachmeier wrote:


 To me, it seems pointless to support Mir in DMD. The "functionality"
 that it brings is speed. Nobody needing absolutely the fastest code is
 not going to have any interest in DMD. From what I understand of Mir
 (but correct me if I'm wrong) it is easy enough to replicate its
 functionality by linking to existing linear algebra libraries. Others
 can construct a compatible interface if that's something they want.

 I think Ilya and everyone else working on Mir should push forward with
 the LDC-based project. Anything else is a distraction.


 He mentions several front end issues, those apply to LDC as well.


 No, LDC for example does not have the issue
 https://issues.dlang.org/show_bug.cgi?id=16488 and others too.


Can we make sure all issues that Mir has with dmd and gdc be present in 
bugzilla and tagged with "Mir"? Thanks! -- Andrei
Sep 27 2016
next sibling parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 27 September 2016 at 10:20:09 UTC, Andrei 
Alexandrescu wrote:

 On 9/27/16 10:50 AM, Ilya Yaroshenko wrote:

 On Tuesday, 27 September 2016 at 07:01:08 UTC, Jacob Carlborg 
 wrote:

 On 2016-09-26 21:49, bachmeier wrote:


 [...]


 He mentions several front end issues, those apply to LDC as 
 well.


 No, LDC for example does not have the issue
 https://issues.dlang.org/show_bug.cgi?id=16488 and others too.


 Can we make sure all issues that Mir has with dmd and gdc be 
 present in bugzilla and tagged with "Mir"? Thanks! -- Andrei


Will fill them tomorrow --Ilya
Sep 27 2016
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/27/2016 3:36 AM, Ilya Yaroshenko wrote:

 Will fill them tomorrow --Ilya


Thank you. Looking forward to it.
Sep 27 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/27/16 12:36 PM, Ilya Yaroshenko wrote:

 On Tuesday, 27 September 2016 at 10:20:09 UTC, Andrei Alexandrescu wrote:

 On 9/27/16 10:50 AM, Ilya Yaroshenko wrote:

 On Tuesday, 27 September 2016 at 07:01:08 UTC, Jacob Carlborg wrote:

 On 2016-09-26 21:49, bachmeier wrote:


 [...]


 He mentions several front end issues, those apply to LDC as well.


 No, LDC for example does not have the issue
 https://issues.dlang.org/show_bug.cgi?id=16488 and others too.


 Can we make sure all issues that Mir has with dmd and gdc be present
 in bugzilla and tagged with "Mir"? Thanks! -- Andrei


 Will fill them tomorrow --Ilya


Awes, thx. -- Andrei
Sep 27 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 27 September 2016 at 15:31:41 UTC, Andrei 
Alexandrescu wrote:

 On 9/27/16 12:36 PM, Ilya Yaroshenko wrote:

 On Tuesday, 27 September 2016 at 10:20:09 UTC, Andrei 
 Alexandrescu wrote:

 On 9/27/16 10:50 AM, Ilya Yaroshenko wrote:

 On Tuesday, 27 September 2016 at 07:01:08 UTC, Jacob 
 Carlborg wrote:

 On 2016-09-26 21:49, bachmeier wrote:


 [...]


 He mentions several front end issues, those apply to LDC as 
 well.


 No, LDC for example does not have the issue
 https://issues.dlang.org/show_bug.cgi?id=16488 and others 
 too.


 Can we make sure all issues that Mir has with dmd and gdc be 
 present
 in bugzilla and tagged with "Mir"? Thanks! -- Andrei


 Will fill them tomorrow --Ilya


 Awes, thx. -- Andrei


DMD performance Issues list 
https://github.com/libmir/mir/wiki/Compiler-and-druntime-bugs#dmd-performance-issues
Sep 28 2016
parent Robert Rat <rrrrrrrrrrrrr youtubetube.com> writes:
On Wednesday, 28 September 2016 at 09:04:23 UTC, Ilya Yaroshenko 
wrote:

 On Tuesday, 27 September 2016 at 15:31:41 UTC, Andrei 
 Alexandrescu wrote:

 On 9/27/16 12:36 PM, Ilya Yaroshenko wrote:

 On Tuesday, 27 September 2016 at 10:20:09 UTC, Andrei 
 Alexandrescu wrote:

 On 9/27/16 10:50 AM, Ilya Yaroshenko wrote:

 On Tuesday, 27 September 2016 at 07:01:08 UTC, Jacob 
 Carlborg wrote:

 On 2016-09-26 21:49, bachmeier wrote:


 [...]


 He mentions several front end issues, those apply to LDC 
 as well.


 No, LDC for example does not have the issue
 https://issues.dlang.org/show_bug.cgi?id=16488 and others 
 too.


 Can we make sure all issues that Mir has with dmd and gdc be 
 present
 in bugzilla and tagged with "Mir"? Thanks! -- Andrei


 Will fill them tomorrow --Ilya


 Awes, thx. -- Andrei


 DMD performance Issues list 
 https://github.com/libmir/mir/wiki/Compiler-and-druntime-bugs#dmd-performance-issues


Maybe this one one too:

https://issues.dlang.org/show_bug.cgi?id=16497

The backend can allocate on the pointer stack for a move. It's 
not just about DMD being not the fatest compiler, it's about 
something stupid done by the code gen.
Sep 29 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/27/2016 3:20 AM, Andrei Alexandrescu wrote:

 Can we make sure all issues that Mir has with dmd and gdc be present in
bugzilla
 and tagged with "Mir"? Thanks! -- Andrei


The ones I've seen so far have all been SIMD issues, and I've been tagging them 
that way.
Sep 27 2016
prev sibling next sibling parent Nikolay <sibnick gmail.com> writes:
On Monday, 26 September 2016 at 18:43:38 UTC, Ilya Yaroshenko 
wrote:

 5. LDC compiler support for ARM, MIPS, MIPS64, Alpha


Alpha CPU nowadays? I supposed it is died forever, except small 
amount of old hardware.
Sep 26 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/26/2016 11:43 AM, Ilya Yaroshenko wrote:

 I think we need to make it a point to support Mir in dmd. -- Andrei


 Required features for Level 3:
 1. https://issues.dlang.org/show_bug.cgi?id=16489
 2. https://issues.dlang.org/show_bug.cgi?id=16488
 3. AVX & AVX2 floating point vector arithmetic
 4. Generic unaligned load/store like (like LDC loadUnaligned and
storeUnaligned)
 5. Generic routine to pack and unpack real and imaginary parts. For usage
 example, see
 https://github.com/libmir/mir/blob/master/source/mir/glas/internal/copy.d#L699


Could you provide more detail on these, please?

For example, on the last link there is no documentation on what those templates 
do or what their parameters are. For example,

   template _mix0(V)
   {
     import ldc.simd;
     enum _pred(size_t a) = (a & 1) == 0 ? a / 2 : a / 2 + V.length;
     alias _mix0 = shufflevector!(V, staticMap!(_pred, Iota!(V.length)));
   }

??

I'd like to see each of the points at least put into an Enhancement Request on 
bugzilla, where we can evaluate them in an organized fashion. (Like what you've 
done for (1) and (2).)
Sep 26 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 27 September 2016 at 03:49:18 UTC, Walter Bright 
wrote:

 On 9/26/2016 11:43 AM, Ilya Yaroshenko wrote:

 I think we need to make it a point to support Mir in dmd. -- 
 Andrei


 Required features for Level 3:
 1. https://issues.dlang.org/show_bug.cgi?id=16489
 2. https://issues.dlang.org/show_bug.cgi?id=16488
 3. AVX & AVX2 floating point vector arithmetic
 4. Generic unaligned load/store like (like LDC loadUnaligned 
 and storeUnaligned)
 5. Generic routine to pack and unpack real and imaginary 
 parts. For usage
 example, see
 https://github.com/libmir/mir/blob/master/source/mir/glas/internal/copy.d#L699


 Could you provide more detail on these, please?

 For example, on the last link there is no documentation on what 
 those templates do or what their parameters are. For example,

   template _mix0(V)
   {
     import ldc.simd;
     enum _pred(size_t a) = (a & 1) == 0 ? a / 2 : a / 2 + 
 V.length;
     alias _mix0 = shufflevector!(V, staticMap!(_pred, 
 Iota!(V.length)));
   }

 ??

 I'd like to see each of the points at least put into an 
 Enhancement Request on bugzilla, where we can evaluate them in 
 an organized fashion. (Like what you've done for (1) and (2).)


Bug report for (5) https://issues.dlang.org/show_bug.cgi?id=16550 
:

----
Vec1
re0 im0 re1 im1 re2 im2 re3 im3 // __vector(float[8])

Vec2
re4 im4 re5 im5 re6 im6 re7 im7 // __vector(float[8])
----
< unpack ------ pack >
----
VecReal
re0 re1 re2 re3 re4 re5 re6 re7 // __vector(float[8])

VecIm
im0 im1 im2 im3 im4 im5 im6 im7 // __vector(float[8])
----
Sep 27 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/27/2016 2:21 AM, Ilya Yaroshenko wrote:

 Bug report for (5) https://issues.dlang.org/show_bug.cgi?id=16550 :


Thank you. Please see my comment on it. Also, please tag all SIMD Bugzilla 
issues with the SIMD keyword (I already did it for 16550).
Sep 27 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 27 September 2016 at 10:36:09 UTC, Walter Bright 
wrote:

 On 9/27/2016 2:21 AM, Ilya Yaroshenko wrote:

 Bug report for (5) 
 https://issues.dlang.org/show_bug.cgi?id=16550 :


 Thank you. Please see my comment on it. Also, please tag all 
 SIMD Bugzilla issues with the SIMD keyword (I already did it 
 for 16550).


Done. Full DMD performance Issues related to Mir list can be 
found here 
https://github.com/libmir/mir/wiki/Compiler-and-druntime-bugs#dmd-performance-issues

Thank you,
Ilya
Sep 28 2016
parent reply Jacob Carlborg <doob me.com> writes:
On 2016-09-28 11:06, Ilya Yaroshenko wrote:


 Done. Full DMD performance Issues related to Mir list can be found here
 https://github.com/libmir/mir/wiki/Compiler-and-druntime-bugs#dmd-performance-issues


It found be better to use the tag field in bugzilla instead of putting 
"[Mir]" in the title.

-- 
/Jacob Carlborg
Sep 28 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Wednesday, 28 September 2016 at 09:41:02 UTC, Jacob Carlborg 
wrote:

 On 2016-09-28 11:06, Ilya Yaroshenko wrote:


 Done. Full DMD performance Issues related to Mir list can be 
 found here
 https://github.com/libmir/mir/wiki/Compiler-and-druntime-bugs#dmd-performance-issues


 It found be better to use the tag field in bugzilla instead of 
 putting "[Mir]" in the title.


There are both tags and "[Mir]"
Sep 28 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2016 2:48 AM, Ilya Yaroshenko wrote:

 On Wednesday, 28 September 2016 at 09:41:02 UTC, Jacob Carlborg wrote:

 On 2016-09-28 11:06, Ilya Yaroshenko wrote:


 Done. Full DMD performance Issues related to Mir list can be found here
 https://github.com/libmir/mir/wiki/Compiler-and-druntime-bugs#dmd-performance-issues


 It found be better to use the tag field in bugzilla instead of putting "[Mir]"
 in the title.


 There are both tags and "[Mir]"


I added the 'performance' keyword to each of the issues.
Oct 08 2016
prev sibling next sibling parent Guillaume Piolat <first.last gmail.com> writes:
On Monday, 26 September 2016 at 18:43:38 UTC, Ilya Yaroshenko 
wrote:


 1. Lightweight `nothrow  nogc` threads, implemented using 
 `struct`s
 2. Lightweight `nothrow  nogc` mutexes and barriers, 
 implemented using `struct`s


FWIW I have some  nogc mutex and semaphore here:
https://github.com/AuburnSounds/dplug/blob/master/core/dplug/core/unchecked_sync.d

Ways to make the runtime more optional than it is would be great.
Sep 27 2016
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On Monday, 26 September 2016 at 18:43:38 UTC, Ilya Yaroshenko 
wrote:

 4. Generic unaligned load/store like (like LDC loadUnaligned 
 and storeUnaligned)


See 
https://github.com/MartinNowak/druntime/blob/23373260e65af5edea989b61d6660832fedbec15/src/core/internal/arrayop.d#L78.
Oct 08 2016
next sibling parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Saturday, 8 October 2016 at 17:26:17 UTC, Martin Nowak wrote:

 On Monday, 26 September 2016 at 18:43:38 UTC, Ilya Yaroshenko 
 wrote:

 4. Generic unaligned load/store like (like LDC loadUnaligned 
 and storeUnaligned)


 See 
 https://github.com/MartinNowak/druntime/blob/23373260e65af5edea989b61d6660832fedbec15/src/core/internal/arrayop.d#L78.


Could you please give an example how it works for user?
I mean aligned vs unaligned.

Does this is always inlined intrinsic (i mean this function has 
not any its machine code in the object file / library e.g. always 
inlined into the function body even in debug compilaiton)?
Oct 08 2016
parent Martin Nowak <code dawg.eu> writes:
On Saturday, 8 October 2016 at 18:10:14 UTC, Ilya Yaroshenko 
wrote:

 https://github.com/MartinNowak/druntime/blob/23373260e65af5edea989b61d6660832fedbec15/src/core/internal/arrayop.d#L78.

 Could you please give an example how it works for user?
 I mean aligned vs unaligned.


???

You could pack them like so.
float4 vec = loadUnaligned!float4(ptrToFloats);
float4 vec = loadAligned!float4(ptrToFloats);

The wrappers are only necessary because DMD/GDC/ldc have 
different SIMD implementations. Would be great if someone wrote a 
common basis library, unfortunately Manu's std.simd got stuck in 
progress.


 Does this is always inlined intrinsic (i mean this function has 
 not any its machine code in the object file / library e.g. 
 always inlined into the function body even in debug 
 compilaiton)?


D doesn't have macros and can't force inline, to inline w/o 
inliner you could use mixin templates, but relying on the inliner 
to do it's job would be cleaner.
Oct 09 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/8/2016 10:26 AM, Martin Nowak wrote:

 See
 https://github.com/MartinNowak/druntime/blob/23373260e65af5edea989b61d6660832fedbec15/src/core/internal/arrayop.d#L78.


Further information should be posted here:

https://issues.dlang.org/show_bug.cgi?id=16558
Oct 08 2016