www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Encouraging preliminary results implementing memcpy in D

reply Mike Franklin <slavo5150 yahoo.com> writes:
I had a little fun today kicking the crap out of C's memcpy with 
a D implementation.

https://github.com/JinShil/memcpyD

Request for help: I don't have a Linux system running on real 
hardware at this time, nor do I have a wide range of platforms 
and machines to test with.  If you'd like to help me with this 
potentially foolish endeavor, please run the program on your 
hardware and send me the results.

Feedback, advise, and pull requests to improve the implementation 
are most welcome.

Mike
Jun 12
next sibling parent Antonio Corbi <acrb ggmail.com> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.
Hi Mike, These are my results running your program under archlinux x86_64 with the zen-kernel 4.17.1, the hardware is powered by an ancient "Intel(R) Core(TM)2 CPU 6600 2.40GHz" with 4GB of ram: size memcpyC memcpyD 1 67607 56810 2 68105 57638 4 66760 58949 8 66943 61262 16 71937 43821 32 70955 48392 64 111473 54226 128 144784 77165 256 183504 113597 512 289039 180930 1024 450526 1314835 2048 782029 1890236 4096 1627622 3165319 8192 2751701 5614202 16384 6361074 11484517 32768 30931212 42805529 65536 61878379 86000892 size memcpyC memcpyD 1 66796 44745 1 66773 44343 1 66780 44157 2 66769 44370 2 66792 44529 4 66776 44298 4 66775 44412 8 66766 44409 8 70945 44359 4 66804 44367 8 71007 44432 16 75210 50656
Jun 13
prev sibling next sibling parent reply drug <drug2004 bk.ru> writes:
Ubuntu 18.04 Linux 4.15.0-23-generic
AMD® Fx(tm)-8350 eight-core processor × 8

size memcpyC memcpyD
1 51089 36921
2 45896 35733
4 46079 36200
8 48443 37509
16 48669 24925
32 52917 27787
64 55631 44928
128 84282 47795
256 107350 66009
512 159310 126795
1024 247683 452560
2048 440687 673211
4096 1129135 1304085
8192 4740910 4095254
16384 8389579 8874273
32768 16630336 17370310
65536 33032013 42904705

size memcpyC memcpyD
1 52354 28365
1 48407 28445
1 50264 30273
2 51312 27708
2 46138 28973
4 52753 28535
4 52150 27418
8 52220 27276
8 49625 27804
4 49356 33510
8 48529 27668
16 52662 135357

second run

size memcpyC memcpyD
1 47248 36964
2 45624 35627
4 45535 35596
8 47920 37012
16 47960 25107
32 52798 27394
64 55444 44282
128 76819 41055
256 105852 66429
512 157629 126243
1024 253841 448974
2048 438973 667101
4096 1144280 1337549
8192 3647558 4141162
16384 8301059 8722185
32768 16413116 17506957
65536 32958933 40381270

size memcpyC memcpyD
1 48513 26288
1 46080 26842
1 48526 26989
2 48634 26419
2 43522 27150
4 48229 25737
4 52841 28117
8 49632 25913
8 46325 25487
4 40267 32343
8 45990 25220
16 46509 124042
Jun 13
parent Mike Franklin <slavo5150 yahoo.com> writes:
On Wednesday, 13 June 2018 at 08:55:40 UTC, drug wrote:
 Ubuntu 18.04 Linux 4.15.0-23-generic
 AMD® Fx(tm)-8350 eight-core processor × 8

 size memcpyC memcpyD
 1 51089 36921
 2 45896 35733
 4 46079 36200
 8 48443 37509
 16 48669 24925
 32 52917 27787
 64 55631 44928
 128 84282 47795
 256 107350 66009
 512 159310 126795
 1024 247683 452560
 2048 440687 673211
 4096 1129135 1304085
 8192 4740910 4095254
 16384 8389579 8874273
 32768 16630336 17370310
 65536 33032013 42904705

 size memcpyC memcpyD
 1 52354 28365
 1 48407 28445
 1 50264 30273
 2 51312 27708
 2 46138 28973
 4 52753 28535
 4 52150 27418
 8 52220 27276
 8 49625 27804
 4 49356 33510
 8 48529 27668
 16 52662 135357
Interesting! I have an AMD 8370 running Windows 8, and I get more favorable results in Windows: size memcpyC memcpyD 1 45361 43626 2 55091 43791 4 70507 43714 8 50910 42854 16 63328 28831 32 72817 30790 64 76307 45823 128 97180 55368 256 164935 68362 512 230508 132100 1024 502189 490590 2048 892968 823070 4096 1896480 1456353 8192 4530645 4516681 16384 10886602 9921215 32768 21717080 19116839 65536 59787610 43549445 size memcpyC memcpyD 1 48770 30084 1 49169 30921 1 43370 30144 2 51404 27571 2 56002 29729 4 69588 29804 4 63743 29510 8 55492 29002 8 46752 31793 4 72673 28858 8 48989 27547 10 55527 121628 In your results, I see that for sizes 1024 and higher (that's when is dispatches to the REP MOVSB algorithm), the performance begins to degrade for Linux. I'm going to install Linux soon and see if I can fix that. Thanks for the data, Mike
Jun 13
prev sibling next sibling parent reply ToRuSer <ToRuSer ToRuSer.com> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.

 Mike
All Tor users now apparently have their posts subjected to 'moderation'. (i.e. someone, will, perhaps, at some point, get around to reviewing their posts, and then, perhaps, it might reach the forum, or not.) So... maybe..you'll get those posts...or maybe not...and when.. is anyones guess. So well done to the D community, for discriminating against all the Tor users out there. You've done yourself proud.
Jun 13
next sibling parent Basile B. <b2.b2.b2.b2.b2.temp.temp.temp gmx.gmx.gmx.com.com> writes:
On Wednesday, 13 June 2018 at 09:07:21 UTC, ToRuSer wrote:
 On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.

 Mike
All Tor users now apparently have their posts subjected to 'moderation'. (i.e. someone, will, perhaps, at some point, get around to reviewing their posts, and then, perhaps, it might reach the forum, or not.) So... maybe..you'll get those posts...or maybe not...and when.. is anyones guess. So well done to the D community, for discriminating against all the Tor users out there. You've done yourself proud.
The problem is likely more that someone has used Tor to troll here and then the enpoint used was blacklisted.
Jun 13
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 06/13/2018 02:07 AM, ToRuSer wrote:

 So well done to the D community, for discriminating against all the Tor
 users out there. You've done yourself proud.
I am part of the D community. I haven't discriminated against anyone. I don't know what a Tor user is. I've just searched: So Tor is an old idea of mine, implemented. :o) Ali
Jun 13
parent reply errExit <err error.com> writes:
On Wednesday, 13 June 2018 at 17:04:11 UTC, Ali Çehreli wrote:
 I am part of the D community. I haven't discriminated against 
 anyone. I don't know what a Tor user is.

 I've just searched: So Tor is an old idea of mine, implemented. 
 :o)

 Ali
Tor is our last line of defence against an Orson Wells future, where everyones actions are scrutinized by big brother, so that big brother can use that knowledge to put fear into, control and manipulate, those that don't conform. assert("bad tor user" != "all tor users are bad"); (actually there are more bad non-tor users) Unfortunately, it's becoming increasingly, the norm, to discriminate against tor users (no doubt those doing that discrimination are those that are happy to conform, of which there will be many, sadly). https://people.torproject.org/~lunar/20160331-CloudFlare_Fact_Sheet.pdf
Jun 13
next sibling parent reply Cym13 <cpicard openmailbox.org> writes:
On Thursday, 14 June 2018 at 02:32:51 UTC, errExit wrote:
 Tor is our last line of defence against an Orson Wells future, 
 where everyones actions are scrutinized by big brother, so that 
 big brother can use that knowledge to put fear into, control 
 and manipulate, those that don't conform.

 assert("bad tor user" != "all tor users are bad");

 (actually there are more bad non-tor users)

 Unfortunately, it's becoming increasingly, the norm, to 
 discriminate against tor users (no doubt those doing that 
 discrimination are those that are happy to conform, of which 
 there will be many, sadly).

 https://people.torproject.org/~lunar/20160331-CloudFlare_Fact_Sheet.pdf
Don't mistake spammer management with discrimination. I share your frustration that TOR isn't more usable than it is today with CloudFlare etc, but coming with political reasons holds no water if the reason why it was blacklisted wasn't political in the first place. It's not false, it just won't work. Hopefully once that particular user gets discouraged or we find a way to actually avoid user impersonification, then things will be able to come back to normal.
Jun 13
next sibling parent reply errExit <errExit err.com> writes:
On Thursday, 14 June 2018 at 03:59:47 UTC, Cym13 wrote:
 Don't mistake spammer management with discrimination. I share 
 your frustration that TOR isn't more usable than it is today 
 with CloudFlare etc, but coming with political reasons holds no 
 water if the reason why it was blacklisted wasn't political in 
 the first place. It's not false, it just won't work.

 Hopefully once that particular user gets discouraged or we find 
 a way to actually avoid user impersonification, then things 
 will be able to come back to normal.
The D Foundation now subjects all users having an ip originating from a tor exit node, to having their posts moderated (but by whom, when, how, criteria ?? etc). Literally millions of people could, and probably would, be using that exit node. So that is plain discrimination. It's not spammer management. Forcing people to identify themselves, is also not about spammer management either. The D Foundation IS now discimantory against those that want that believe that freedom and privacy is some to be protected. This becomes problematic for those of us who work in 'certain organisations', that insist on tracking it's employees online activities (even outside of the workplace). It's a shame the D Foundation has finally succumed to the big brother mentality - under the guise of protecting you from spam. https://blog.torproject.org/dont-let-facebook-or-any-tracker-follow-you-web
Jun 13
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 06/14/2018 02:34 AM, errExit wrote:
 
 The D Foundation now subjects all users having an ip originating from a 
 tor exit node, to having their posts moderated (but by whom, when, how, 
 criteria ?? etc).
 
 Literally millions of people could, and probably would, be using that 
 exit node.
 
 So that is plain discrimination. It's not spammer management.
 
 Forcing people to identify themselves, is also not about spammer 
 management either.
 
 The D Foundation IS now discimantory against those that want that 
 believe that freedom and privacy is some to be protected.
 
 This becomes problematic for those of us who work in 'certain 
 organisations', that insist on tracking it's employees online activities 
 (even outside of the workplace).
 
 It's a shame the D Foundation has finally succumed to the big brother 
 mentality - under the guise of protecting you from spam.
 
 https://blog.torproject.org/dont-let-facebook-or-any-tracker-follow-you-web
 
I'm with you on a lot of that, however, this part troubles me: "This becomes problematic for those of us who work in 'certain organisations', that insist on tracking it's employees online activities (even outside of the workplace)." If I worked in such an organization that tracked its employees activities *outside the workplace*, I'd LEAVE it ASAP, and I'd strongly suggest anyone else do the same. I mean what insane workplace is that, the 1920's mob? Apple? Honestly, if you believe strongly enough in Tor to use it, why in the world would you willfully aid and abed an organization that does that sort of thing? It doesn't make any sense at all. It's EXTREMELY self-contradictory and completely erodes your entire stance. If you're going to preach for personal freedom and privacy, at least have the basic integrity to LIVE the basic ideals you're preaching even when doing so ISN'T so trivial as installing a mere web browser.
Jun 14
next sibling parent errExit <errExit err.com> writes:
On Thursday, 14 June 2018 at 07:19:31 UTC, Nick Sabalausky 
(Abscissa) wrote:
 I'm with you on a lot of that, however, this part troubles me:

 "This becomes problematic for those of us who work in 'certain 
 organisations', that insist on tracking it's employees online 
 activities (even outside of the workplace)."

 If I worked in such an organization that tracked its employees 
 activities *outside the workplace*, I'd LEAVE it ASAP, and I'd 
 strongly suggest anyone else do the same. I mean what insane 
 workplace is that, the 1920's mob? Apple?

 Honestly, if you believe strongly enough in Tor to use it, why 
 in the world would you willfully aid and abed an organization 
 that does that sort of thing? It doesn't make any sense at all. 
 It's EXTREMELY self-contradictory and completely erodes your 
 entire stance. If you're going to preach for personal freedom 
 and privacy, at least have the basic integrity to LIVE the 
 basic ideals you're preaching even when doing so ISN'T so 
 trivial as installing a mere web browser.
Sadly, it's increasingly 'standard practice' in HR to do just that. Monitor what employees, and potential employees have done or made available on the internet. I don't support that approach, which is exactly why I use Tor. Now, their attempts are moot, and therefore I am in no way supporting those actions. And so, your comments about 'self-contradictory' are moot also. In addition, HR data is increasingly becoming a valuable target for attack, due to the 'profiles' they keep on people. So now the situation gets even worse. Cause not only does HR have this info, so will others... eventually. The only real world option, is to prevent them from gathering data on you in the first place. btw. Some people in my team (those that use D), might have contributed to this post, but now, likely will not.
Jun 14
prev sibling parent reply AnotherTorUser <me me.com> writes:
On Thursday, 14 June 2018 at 07:19:31 UTC, Nick Sabalausky 
(Abscissa) wrote:
 I'm with you on a lot of that, however, this part troubles me:

 "This becomes problematic for those of us who work in 'certain 
 organisations', that insist on tracking it's employees online 
 activities (even outside of the workplace)."

 If I worked in such an organization that tracked its employees 
 activities *outside the workplace*, I'd LEAVE it ASAP, and I'd 
 strongly suggest anyone else do the same. I mean what insane 
 workplace is that, the 1920's mob? Apple?

 Honestly, if you believe strongly enough in Tor to use it, why 
 in the world would you willfully aid and abed an organization 
 that does that sort of thing? It doesn't make any sense at all. 
 It's EXTREMELY self-contradictory and completely erodes your 
 entire stance. If you're going to preach for personal freedom 
 and privacy, at least have the basic integrity to LIVE the 
 basic ideals you're preaching even when doing so ISN'T so 
 trivial as installing a mere web browser.
sorry. but what world do you live in? If all such people stopped working for such companies, what do you think the economic impact would be? Tor prevents tracking, and therefore it is not contradictory to work for such companies - because they can't track you (or at least, it become much more difficult to do so).
Jun 14
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 06/14/2018 05:01 AM, AnotherTorUser wrote:
 
 sorry. but what world do you live in?
Note this part of what I said:
 If I worked in such an organization that tracked its employees
 activities *outside the workplace*
That part is KEY, but it sounds like you an errExit have chosen to ignore that part.
 If all such people stopped working for such companies, what do you think 
 the economic impact would be?
What do you think is the social impact if they don't? And don't even try to pretend the companies can't trivially solve the "economic" issues for themselves in an instant by knocking off the behaviour that causes loss of talent.
Jun 14
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, June 14, 2018 16:04:32 Nick Sabalausky  via Digitalmars-d-
announce wrote:
 On 06/14/2018 05:01 AM, AnotherTorUser wrote:
 If all such people stopped working for such companies, what do you think
 the economic impact would be?
What do you think is the social impact if they don't? And don't even try to pretend the companies can't trivially solve the "economic" issues for themselves in an instant by knocking off the behaviour that causes loss of talent.
But that would imply that they have a frontal lobe. :) In all seriousness, it is surprising how frequently companies seem to be incapable of making decisions that would fix a lot of their problems, and they seem to be incredibly prone to thinking about things in a shortsighted manner. I'm reminded of an article by Joel Spoelskey where he talks about how one of the key things that a source control software solution can do to make it more likely for folks to be willing to try it is to make it easy to get your source code and history back out again and into another source control system. However, companies typically freak out at the idea of making it easy to switch from their product to another product. They're quite willing to make it easy to switch _to_ their product so that they can start making money off of you, but the idea that making it low cost to leave could actually improve the odds of someone trying their product - and thus increase their profits - seems to be beyond them. Another case which is closer to the exact topic at hand is that many companies seem to forget how much it costs to hire someone when they consider what they should do to make it so that their employees are willing - or even eager - to stay. Spending more money on current employees (be that on salary or something else to make the workplace desirable) or avoiding practices that tick employees off so that they leave can often save money in the long run, but companies frequently ignore that fact. They're usually more interested in saving on the bottom line right now than making decisions that save money over time. So, while I completely agree that companies can technically make decisions that solve some of their problems with things like retaining talent, it seems like it's frequently the case that they're simply incapable of doing it in practice - though YMMV; some companies are better about it than others. - Jonathan M Davis
Jun 14
parent Joakim <dlang joakim.fea.st> writes:
On Thursday, 14 June 2018 at 20:59:06 UTC, Jonathan M Davis wrote:
 On Thursday, June 14, 2018 16:04:32 Nick Sabalausky  via 
 Digitalmars-d- announce wrote:
 On 06/14/2018 05:01 AM, AnotherTorUser wrote:
 If all such people stopped working for such companies, what 
 do you think the economic impact would be?
What do you think is the social impact if they don't? And don't even try to pretend the companies can't trivially solve the "economic" issues for themselves in an instant by knocking off the behaviour that causes loss of talent.
But that would imply that they have a frontal lobe. :) In all seriousness, it is surprising how frequently companies seem to be incapable of making decisions that would fix a lot of their problems, and they seem to be incredibly prone to thinking about things in a shortsighted manner. I'm reminded of an article by Joel Spoelskey where he talks about how one of the key things that a source control software solution can do to make it more likely for folks to be willing to try it is to make it easy to get your source code and history back out again and into another source control system. However, companies typically freak out at the idea of making it easy to switch from their product to another product. They're quite willing to make it easy to switch _to_ their product so that they can start making money off of you, but the idea that making it low cost to leave could actually improve the odds of someone trying their product - and thus increase their profits - seems to be beyond them. Another case which is closer to the exact topic at hand is that many companies seem to forget how much it costs to hire someone when they consider what they should do to make it so that their employees are willing - or even eager - to stay. Spending more money on current employees (be that on salary or something else to make the workplace desirable) or avoiding practices that tick employees off so that they leave can often save money in the long run, but companies frequently ignore that fact. They're usually more interested in saving on the bottom line right now than making decisions that save money over time. So, while I completely agree that companies can technically make decisions that solve some of their problems with things like retaining talent, it seems like it's frequently the case that they're simply incapable of doing it in practice - though YMMV; some companies are better about it than others.
This was an interesting read on that topic, which I've linked on this forum before, where an engineer points out that companies would be better off not chasing "rockstars" with hot keywords on their resumes but improving their training, processes, and culture so that even average programmers can be productive, including mentioning using source control and the Joel test that you just referenced: https://danluu.com/programmer-moneyball/ Of course, the reason companies mostly don't do it is they're prone to the same cognitive failings as anybody else: it's easier to chase a quick fix than doing the hard work of putting in a system like this.
Jun 14
prev sibling parent Arafel <er.krali gmail.com> writes:
Well, e-mail was never meant to be reliable or secure... BTW, there are 
already solutions to prevent impersonation: sign the messages with 
either PGP or S/MIME... the former is more decentralised, and the later 
usually comes together with stronger verifications, like personal 
identification of the sender (I mean, whoever issues the certificate for 
S/MIME should ideally check your name, etc.) :-)

Although admittedly that'd work only for the NNTP / mailing list, I 
don't know how that could be adapted to the web "forum" interface...

On 06/14/2018 05:59 AM, Cym13 wrote:
 
 Hopefully once that particular user gets discouraged or we find a way to 
 actually avoid user impersonification, then things will be able to come 
 back to normal.
Jun 15
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 14 June 2018 at 02:32:51 UTC, errExit wrote:
 On Wednesday, 13 June 2018 at 17:04:11 UTC, Ali Çehreli wrote:
 I am part of the D community. I haven't discriminated against 
 anyone. I don't know what a Tor user is.

 I've just searched: So Tor is an old idea of mine, 
 implemented. :o)

 Ali
Tor is our last line of defence against an Orson Wells future, where everyones actions are scrutinized by big brother, so that big brother can use that knowledge to put fear into, control and manipulate, those that don't conform. assert("bad tor user" != "all tor users are bad"); (actually there are more bad non-tor users) Unfortunately, it's becoming increasingly, the norm, to discriminate against tor users (no doubt those doing that discrimination are those that are happy to conform, of which there will be many, sadly). https://people.torproject.org/~lunar/20160331-CloudFlare_Fact_Sheet.pdf
Tor is merely one tool used to route around those building centralized systems on top of the internet. The real solution is that as more and more decentralized tech does well, like git or cryptocurrencies, to get rid of these obsolete centralized systems altogether.
Jun 14
parent YetAnotherTorUser <YetAnotherTorUser YetAnotherTorUser.com> writes:
On Thursday, 14 June 2018 at 07:45:15 UTC, Joakim wrote:
 Tor is merely one tool used to route around those building 
 centralized systems on top of the internet. The real solution 
 is that as more and more decentralized tech does well, like git 
 or cryptocurrencies, to get rid of these obsolete centralized 
 systems altogether.
The problem with the D Foundation's 'new approach' is, that there is simply no visibility as to what is being moderated, how long it takes, whom does it, and for what the filtering criteria is. We're just being told that it is too avoid 'spammers'?? If that is not a tool for abuse, then I don't know what is.
Jun 14
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/13/18 10:32 PM, errExit wrote:
 On Wednesday, 13 June 2018 at 17:04:11 UTC, Ali Çehreli wrote:
 I am part of the D community. I haven't discriminated against anyone. 
 I don't know what a Tor user is.

 I've just searched: So Tor is an old idea of mine, implemented. :o)

 Ali
Tor is our last line of defence against an Orson Wells future, where everyones actions are scrutinized by big brother, so that big brother can use that knowledge to put fear into, control and manipulate, those that don't conform.
Sorry KingJoffrey/psychoticRabbit/etc, you literally only started using Tor when your steady IP was banned. This argument kind of falls down when your past behavior is examined. It's really the impersonation that is the problem, not the anonymity. If you just would stick to one persona, and especially not impersonate people who actually post on this forum, you would not have to use Tor at all, and the forum moderators wouldn't have to worry about active moderation. -Steve
Jun 14
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 15/06/2018 1:22 AM, Steven Schveighoffer wrote:
 It's really the impersonation that is the problem, not the anonymity. If 
 you just would stick to one persona, and especially not impersonate 
 people who actually post on this forum, you would not have to use Tor at 
 all, and the forum moderators wouldn't have to worry about active 
 moderation.
Not quite. It was the attacking of the D community at large which convinced those with the power to actively ban, to ban "him". We have in essence decided that this person will never be willing to have positive experiences within our community and have out right decided that we do not want them here under any circumstance. A lot of work has gone into getting rid of "him". As far as I am concerned the original IP address that was used from Australia was in fact a proxy and had the single desire for this person, was to attack us. Once they are gone, we can deactivate the extra protections that have been put in place, because until this person came along, we were quite happy moderating ourselves. It lasted nearly 20 years that peace...
Jun 14
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/14/18 9:31 AM, rikki cattermole wrote:
 On 15/06/2018 1:22 AM, Steven Schveighoffer wrote:
 It's really the impersonation that is the problem, not the anonymity. 
 If you just would stick to one persona, and especially not impersonate 
 people who actually post on this forum, you would not have to use Tor 
 at all, and the forum moderators wouldn't have to worry about active 
 moderation.
Not quite. It was the attacking of the D community at large which convinced those with the power to actively ban, to ban "him".
Well, for me, it wasn't attacks. There have been quite a few people in this community who were rude, insulting, and IMO, that is not worth moderation. In fact some of them have even came around and become quite good D contributors. Those problems usually work themselves out because we have a great community which does not give trolls the attention they desire. But when you start impersonating people, especially ones that post to this forum, you have crossed the line and are literally putting words into other's mouths. We've seen this before, and generally they go away, but this guy does not want to. Now he's claiming "discrimination" ;)
 Once they are gone, we can deactivate the extra protections that have 
 been put in place, because until this person came along, we were quite 
 happy moderating ourselves. It lasted nearly 20 years that peace...
I agree, this will be a blip in our forum experience, and next month we'll barely remember him. -Steve
Jun 14
parent bachmeier <no spam.net> writes:
On Thursday, 14 June 2018 at 13:42:14 UTC, Steven Schveighoffer 
wrote:

 Well, for me, it wasn't attacks. There have been quite a few 
 people in this community who were rude, insulting, and IMO, 
 that is not worth moderation. In fact some of them have even 
 came around and become quite good D contributors. Those 
 problems usually work themselves out because we have a great 
 community which does not give trolls the attention they desire.
That seems to be the opinion of most. The problem I have is that this forum is the main way to communicate about the language, and thus it is how others form their opinion of the language.
Jun 14
prev sibling next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.
If I read your benchmark graphs right, they claimed that allocating 16 kilobytes takes over 10^^6 usecs, with both mallocs. Doesn't that mean over a second, 16 kilobytes? Can't be! Are you confusing usecs with nsecs?
Jun 13
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Wednesday, 13 June 2018 at 09:40:05 UTC, Dukc wrote:

 If I read your benchmark graphs right, they claimed that 
 allocating 16 kilobytes takes over 10^^6 usecs, with both 
 mallocs. Doesn't that mean over a second, 16 kilobytes? Can't 
 be! Are you confusing usecs with nsecs?
The benchmark doesn't allocate any data; it's just copying data. Each benchmark is run 10,000,000 times to smooth out some of the entropy in the results: https://github.com/JinShil/memcpyD/blob/2e0d3c33ea876a25a04358a3ae505b2eba f99cb/memcpyd.d#L78 The usecs in the graph is the time it takes to run the benchmark 10,000,000 times. Mike
Jun 13
parent reply Dukc <ajieskola gmail.com> writes:
On Wednesday, 13 June 2018 at 09:59:52 UTC, Mike Franklin wrote:
 The benchmark doesn't allocate any data; it's just copying data.

 Mike
Ah of course. I was thinking other stuff while writing.
Jun 13
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Wednesday, 13 June 2018 at 10:13:13 UTC, Dukc wrote:
 On Wednesday, 13 June 2018 at 09:59:52 UTC, Mike Franklin wrote:
 The benchmark doesn't allocate any data; it's just copying 
 data.

 Mike
Ah of course. I was thinking other stuff while writing.
Well, actually, I probably should divide that time by 10,000,000 to make a more accurate representation. Thanks for the feedback, Mike
Jun 13
parent Arredondo <arm.plus gmail.com> writes:
On Wednesday, 13 June 2018 at 10:17:10 UTC, Mike Franklin wrote:
 Well, actually, I probably should divide that time by 
 10,000,000 to make a more accurate representation.
For rigorous benchmarking, check out the first part of Andrei's Writing Fast Code: https://www.youtube.com/watch?v=vrfYLlR8X8k One takeaway is that taking the average of many runtimes is not the best use of your dataset.
Jun 13
prev sibling next sibling parent reply Fra Mecca <me francescomecca.eu> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.

 Mike
I get this on Linux 4.16.3-gentoo, AMD FX(tm)-6100 Six-Core Processor, 8GiB ram, using ldc2 -O3L: size memcpyC memcpyD 1 5 0 2 0 0 4 0 0 8 0 0 16 1519 0 32 1833 0 64 3816 0 128 7543 0 256 146500 0 512 194818 0 1024 329593 846142 2048 714945 1117132 4096 1596170 1803621 8192 5899818 6110026 16384 12338384 14141850 32768 24971315 26771484 65536 49806637 63260128 size memcpyC memcpyD 1 0 0 1 0 0 1 0 0 2 0 0 2 0 0 4 0 0 4 0 0 8 0 0 8 0 0 4 0 0 8 0 0 core.exception.AssertError memcpyd.d(9): Assertion failure ---------------- ??:? _d_assert [0xcaf056d5] ??:? [0xa015e7fe] ??:? [0xa0158cb0] ??:? void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll() [0xcaf29daf] ??:? _d_run_main [0xcaf29c7b] ??:? __libc_start_main [0xca160eeb] ??:? [0xa0158b29]
Jun 13
parent Mike Franklin <slavo5150 yahoo.com> writes:
On Wednesday, 13 June 2018 at 12:45:26 UTC, Fra Mecca wrote:
 I get this on Linux 4.16.3-gentoo, AMD FX(tm)-6100 Six-Core 
 Processor, 8GiB ram,

 using ldc2 -O3L:
 size memcpyC memcpyD
 1 5 0
 2 0 0
 4 0 0
 8 0 0
 16 1519 0
 32 1833 0
 64 3816 0
 128 7543 0
 256 146500 0
 512 194818 0
 1024 329593 846142
 2048 714945 1117132
 4096 1596170 1803621
 8192 5899818 6110026
 16384 12338384 14141850
 32768 24971315 26771484
 65536 49806637 63260128

 size memcpyC memcpyD
 1 0 0
 1 0 0
 1 0 0
 2 0 0
 2 0 0
 4 0 0
 4 0 0
 8 0 0
 8 0 0
 4 0 0
 8 0 0
 core.exception.AssertError memcpyd.d(9): Assertion failure
 ----------------
 ??:? _d_assert [0xcaf056d5]
 ??:? [0xa015e7fe]
 ??:? [0xa0158cb0]
 ??:? void rt.dmain2._d_run_main(int, char**, extern (C) int 
 function(char[][])*).runAll() [0xcaf29daf]
 ??:? _d_run_main [0xcaf29c7b]
 ??:? __libc_start_main [0xca160eeb]
 ??:? [0xa0158b29]
Yes, optimizations are throwing code away. I'm working on it. Mike
Jun 13
prev sibling next sibling parent reply Uknown <sireeshkodali1 gmail.com> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.

 Mike
I'm glad to see this kind of improvement! I noticed there wasn't any macOS testing though, so heres the results on my machine: size memcpyC memcpyD 1 38613 5185 2 42475 7259 4 54272 3450 8 29391 3457 16 32947 3626 32 33761 7253 64 44152 14510 128 51985 27711 256 65700 58044 512 95315 119977 1024 150726 387021 2048 265975 667858 4096 521565 1113945 8192 965322 2039064 16384 3923818 3997524 32768 15586237 15669232 65536 33195890 31640205 size memcpyC memcpyD 1 40057 5180 1 36664 5226 1 37625 5280 2 36584 5178 2 36956 5252 4 51086 3448 4 49952 3456 8 29497 3455 8 29710 3987 4 51539 3453 8 29715 3449 16 29747 23535 sys info : Intel Core i5 4258u (28w), 8GB DDR3 1600MHz RAM Looks very promising. One question though, why not use std.datetime.stopwatch.benchmark? I think it has some protection against optimizing compilers (I may be wrong though). Also, LDC has attributes to control optimizations on a per function basis.see : https://wiki.dlang.org/LDC-specific_language_changes My linux install got messed up, but I'll post benchmarks as soon as its up
Jun 13
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, June 13, 2018 14:34:28 Uknown via Digitalmars-d-announce 
wrote:
 Looks very promising. One question though, why not use
 std.datetime.stopwatch.benchmark? I think it has some protection
 against optimizing compilers (I may be wrong though). Also, LDC
 has attributes to control optimizations on a per function
 basis.see : https://wiki.dlang.org/LDC-specific_language_changes
Unfortunately, std.datetime.stopwatch.benchmark does not yet have such protections. It has been discussed, but there were issues with it that still need to be sorted out. In any case, what he has implemented is pretty much what's in Phobos except for the fact that he set up his to take arguments, whereas Phobos' solution just takes the function(s) to call, so anything that it does has to be self-contained. - Jonathan M Davis
Jun 14
parent Uknown <sireeshkodali1 gmail.com> writes:
On Thursday, 14 June 2018 at 07:54:20 UTC, Jonathan M Davis wrote:
 On Wednesday, June 13, 2018 14:34:28 Uknown via 
 Digitalmars-d-announce wrote:
 Looks very promising. One question though, why not use 
 std.datetime.stopwatch.benchmark? I think it has some 
 protection against optimizing compilers (I may be wrong 
 though). Also, LDC has attributes to control optimizations on 
 a per function basis.see : 
 https://wiki.dlang.org/LDC-specific_language_changes
Unfortunately, std.datetime.stopwatch.benchmark does not yet have such protections. It has been discussed, but there were issues with it that still need to be sorted out. In any case, what he has implemented is pretty much what's in Phobos except for the fact that he set up his to take arguments, whereas Phobos' solution just takes the function(s) to call, so anything that it does has to be self-contained. - Jonathan M Davis
huh. I saw the PR and assumed it was accepted. Anyway, for DMD putting `asm` blocks seems to still disable optimizations, and for LDC, the pragma is perfect. GDC is the only unknown.
Jun 14
prev sibling next sibling parent UnpaidTester <UnpaidTester UnpaidTester.com> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.

 Mike
specs ......................................... (note: tried DMD v2.078.3) > dmd -O (and got strange assertion errors). (used LDC 1.8.0 instead) > ldc2 -O2 Intel(R) Core(TM) i7 CPU 920 2.67GHz 24GB Mem (Speed: 1066 MT/s) Kubuntu 18.04 LTS - 4.15.0-23-generic ........................................ results ........................................ size memcpyC memcpyD 1 0 0 2 0 0 4 0 0 8 0 0 16 1247 0 32 1889 0 64 3055 0 128 5040 0 256 91407 0 512 158527 0 1024 253191 870780 2048 474243 1170349 4096 932151 1933045 8192 1894059 3284901 16384 4447945 6122015 32768 18572704 20772417 65536 37470068 40211203 size memcpyC memcpyD 1 0 0 1 0 0 1 0 0 2 0 0 2 0 0 4 0 0 4 0 0 8 0 0 8 0 0 4 0 0 8 0 0 16 0 0 ..........................
Jun 13
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/13/18 2:46 AM, Mike Franklin wrote:
 I had a little fun today kicking the crap out of C's memcpy with a D 
 implementation.
 
 https://github.com/JinShil/memcpyD
 
 Request for help: I don't have a Linux system running on real hardware 
 at this time, nor do I have a wide range of platforms and machines to 
 test with.  If you'd like to help me with this potentially foolish 
 endeavor, please run the program on your hardware and send me the results.
Just FYI, Linux results on a VM are still valid -- many people (including myself) only use Linux on a VM, so even though it's not a definitive win against glibc memcpy on Linux, the end result is, it's still faster for those users :) I won't add much, since I'm using a Mac, and those numbers have already been posted. But I wanted to say don't discount those findings off-hand. -Steve
Jun 13
parent Cym13 <cpicard openmailbox.org> writes:
On Wednesday, 13 June 2018 at 16:21:53 UTC, Steven Schveighoffer 
wrote:
 I won't add much, since I'm using a Mac, and those numbers have 
 already been posted.
Reproduction is an important part of the scientific process, please post away ;) Also: memcpyD commit f5fb0fda0dbacc960ced59d7171ff76a95cc2abf on Archlinux native x86_64 4.16.13-2 with Intel(R) Core(TM) i7-3520M CPU 2.90GHz and 7681MB of RAM, with dmd and ldc, no flag and optimized. DMD: size memcpyC memcpyD 1 50392 30745 2 50189 33515 4 47493 33456 8 50282 33428 16 36385 36061 32 36212 32142 64 50141 31137 128 52947 52785 256 86422 76346 512 131191 128344 1024 227875 291414 2048 444865 449210 4096 826391 823613 8192 1542429 1545051 16384 3264964 3228990 32768 11644816 11902418 65536 23658237 24026304 size memcpyC memcpyD 1 43209 37462 1 42412 38043 1 42079 38002 2 56503 39923 2 50544 38033 4 47760 37239 4 48910 36976 8 51110 38065 8 53761 36297 4 48201 35548 8 54820 38036 16 39360 35409 DMD -O: core.exception.AssertError memcpyd.d(9): Assertion failure ---------------- ??:? _d_assertp [0xc4e79cc5] ??:? pure nothrow nogc void memcpyd.verify!(real).verify(const(real), const(real)) [0xc4e77c2c] ??:? void memcpyd.test!(real).test() [0xc4e76d60] ??:? _Dmain [0xc4e63ecd] 1 42014 26518 2 46086 26486 4 45984 29272 8 48813 26484 16 34866 18126 32 32036 18107 64 46073 20892 128 45964 27993 256 82344 50250 512 133484 94766 1024 216402 270462 2048 436465 443122 4096 815875 801596 8192 1524872 1530453 16384 3721245 3584620 32768 11776185 11739396 65536 23702074 23589480 size memcpyC memcpyD 1 37755 15424 1 40486 15319 1 40505 15352 2 47097 15653 2 46160 15319 4 43180 18110 4 46041 15321 8 46014 15342 8 46066 15341 4 43277 18105 8 45962 15321 LDC: size memcpyC memcpyD 1 56378 48873 2 59461 49025 4 50481 45299 8 53786 49517 16 46103 39122 32 48100 46139 64 83864 67401 128 83849 90426 256 122471 138309 512 169668 228472 1024 260461 276878 2048 444937 472365 4096 860962 825468 8192 1578929 1567154 16384 3429235 3284611 32768 11941494 11868947 65536 24052536 24112980 size memcpyC memcpyD 1 47853 33403 1 47623 32924 1 48190 33100 2 59752 33146 2 59574 34371 4 53928 35042 4 54131 31991 8 57929 35864 8 57372 32174 4 54901 33253 8 62537 34535 16 52487 38358 LDC -O2: core.exception.AssertError memcpyd.d(9): Assertion failure ---------------- ??:? _d_assert [0x7fba47d79b35] ??:? void memcpyd.test!(real).test() [0x5638cb483d6e] ??:? _Dmain [0x5638cb47ea30] size memcpyC memcpyD 1 0 0 2 0 0 4 0 0 8 0 0 16 385 0 32 792 0 64 1542 0 128 2981 0 256 90108 0 512 126316 0 1024 217881 391419 2048 415182 620853 4096 1244805 1240074 8192 2428417 2414095 16384 4863280 4971193 32768 12968909 12966676 65536 26393408 26395410 size memcpyC memcpyD 1 0 0 1 0 0 1 0 0 2 0 0 2 0 0 4 0 0 4 0 0 8 0 0 8 0 0 4 0 0 8 0 0
Jun 13
prev sibling next sibling parent Arun Chandrasekaran <aruncxy gmail.com> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.

 Mike
On 8 core, 16 GB Intel Skull Candy box running Ubuntu 18.04 64 bit. https://gist.githubusercontent.com/carun/f7c2c200b1be20d0a9489296d6601332/raw/db01bb8bc909c6048288fccc500bd15e5ee491b2/memcpyd-output.log Hope this helps.
Jun 13
prev sibling next sibling parent Diego <diego example.com> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.

 Mike
Hello Mike, These are my results: Ubuntu 16.04.4 amd64 Linux 4.16.0-ck1+ Intel Xeon E5-2620 0 2.00GHz - 12 cores 32 GB RAM dmd -O -release memcpyd.d # dmd 2.080.0 size memcpyC memcpyD 1 125349 47658 2 155014 50492 4 173099 52669 8 228236 52676 16 107897 32621 32 128039 32604 64 163644 37658 128 223840 50420 256 338769 90300 512 584772 171038 1024 878093 995813 2048 1346958 1254141 4096 2439378 2101284 8192 5631202 3554307 16384 9873090 6496635 32768 22489302 21328288 65536 50522961 45748356 size memcpyC memcpyD 1 123241 27631 1 130758 28165 1 123247 32748 2 142964 27587 2 140914 28103 4 168084 32616 4 171166 27590 8 228274 27604 8 233249 27605 4 168624 27597 8 238049 29435 16 103956 52730 ldc2 -O3 -release memcpyd.d # ldc2 1.10.0-beta1 (I think these are strange results) size memcpyC memcpyD 1 0 0 2 0 0 4 0 0 8 0 0 16 559 0 32 1003 0 64 0 0 128 0 0 256 0 0 512 0 0 1024 460182 1519048 2048 739148 1973641 4096 1533047 3168472 8192 2913463 5560106 16384 6385370 10353178 32768 20889322 21487968 65536 44920382 48339716 size memcpyC memcpyD 1 0 0 1 0 0 1 0 0 2 0 0 2 0 0 4 0 0 4 0 0 8 0 0 8 0 0 4 0 0 8 0 0 16 0 0
Jun 14
prev sibling next sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.
Just a little remark. Alignment has also an extremely heavy impact on the performance of memcpy(). Does your test include it?
Jun 14
parent reply baz <b2.b2.b2.temp.temp.temp gmx.gmx.gmx.com.com> writes:
On Thursday, 14 June 2018 at 17:27:27 UTC, Patrick Schluter wrote:
 On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
 I had a little fun today kicking the crap out of C's memcpy 
 with a D implementation.

 https://github.com/JinShil/memcpyD

 Request for help: I don't have a Linux system running on real 
 hardware at this time, nor do I have a wide range of platforms 
 and machines to test with.  If you'd like to help me with this 
 potentially foolish endeavor, please run the program on your 
 hardware and send me the results.

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.
Just a little remark. Alignment has also an extremely heavy impact on the performance of memcpy(). Does your test include it?
I asked on IRC yesterday and actually tHose memcpy are not the memcpy we use to copy wide chunks, apparently it's rather for an internal druntime thing, i.e cpy type to type so likely always aligned.
Jun 14
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Thursday, 14 June 2018 at 20:35:23 UTC, baz wrote:

 I asked on IRC yesterday and actually tHose memcpy are not the 
 memcpy we use to copy wide chunks, apparently it's rather for 
 an internal druntime thing, i.e cpy type to type so likely 
 always aligned.
Correct! D already has features like `a[] = b[]` so there is no reason to call `memcpy` directly; that is a job for the druntime. `memcpyD` is intended to be an internal druntime utility. However, we should still be good D citizens when we write our low level druntime code, so the interface will be `memcpy(T)(T a, T b)` and `memcpy(T)(T[] a, T[] b)` instead of `memcpy(void* a, void* b, size_t size)`. This will ensure the code, at compile-time, adheres to D's guarantees such as ` safe`, `nothrow`, and `pure`. And, given that it won't require `TypeInfo` it will be usable in -betterC. Although rare, I believe it is still possible to have misaligned memory in D. My understanding is that misaligned copies usually involve copying smaller chunks of the memory until they are aligned, and then copying the aligned memory in larger chunks. I suspect that will work well with the D implementation. TL;DR, Unaligned memory will be handled after optimized aligned memory copies are implemented. Mike
Jun 14
parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Friday, 15 June 2018 at 00:15:35 UTC, Mike Franklin wrote:
 On Thursday, 14 June 2018 at 20:35:23 UTC, baz wrote:

 [...]
Correct! D already has features like `a[] = b[]` so there is no reason to call `memcpy` directly; that is a job for the druntime. `memcpyD` is intended to be an internal druntime utility. [...]
Ok, thanks for the explanation.
Jun 15
prev sibling parent reply David Nadlinger <code klickverbot.at> writes:
On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
 https://github.com/JinShil/memcpyD

 […]

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.
The memcpyD implementation is buggy; it assumes that all arguments are aligned to their size. This isn't necessarily true. For example, `ubyte[1024].alignof == 1`, and struct alignment can also be set explicitly using align(N). On x86, you can get away with this in a lot of cases even though it's undefined behaviour [1], but this is not necessarily the case for SSE/AVX instructions. In fact, that's probably a pretty good guess as to where those weird crashes you mentioned come from. On other architectures, unaligned accesses might be outright unsupported. For loading into vector registers, you can use core.simd.loadUnaligned instead (ldc.simd.loadUnaligned for LDC – LDC's druntime has not been updated yet after {load, store}Unaligned were added upstream as well). — David [1] This is applying C rules to D, where creation of unaligned pointers is undefined behaviour. The D specification doesn't mention
Jun 17
next sibling parent David Nadlinger <code klickverbot.at> writes:
On Sunday, 17 June 2018 at 17:00:00 UTC, David Nadlinger wrote:
 core.simd.loadUnaligned instead
Ah, well… https://issues.dlang.org/show_bug.cgi?id=19001 — David
Jun 17
prev sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Sunday, 17 June 2018 at 17:00:00 UTC, David Nadlinger wrote:
 On Wednesday, 13 June 2018 at 06:46:43 UTC, Mike Franklin wrote:
 https://github.com/JinShil/memcpyD

 […]

 Feedback, advise, and pull requests to improve the 
 implementation are most welcome.
The memcpyD implementation is buggy; it assumes that all arguments are aligned to their size. This isn't necessarily true. For example, `ubyte[1024].alignof == 1`, and struct alignment can also be set explicitly using align(N).
Yes, I'm already aware of that. My plan is to create optimized implementations for aligned data, and then handled unaligned data as compositions of the various aligned implementations. For example a 3 byte copy would be a short copy plus a byte copy. That may not be appropriate for all cases. I'll have to measure, and adapt.
 On x86, you can get away with this in a lot of cases even 
 though it's undefined behaviour [1], but this is not 
 necessarily the case for SSE/AVX instructions. In fact, that's 
 probably a pretty good guess as to where those weird crashes 
 you mentioned come from.
Thanks! I think you're right.
 For loading into vector registers, you can use 
 core.simd.loadUnaligned instead (ldc.simd.loadUnaligned for LDC 
 – LDC's druntime has not been updated yet after {load, 
 store}Unaligned were added upstream as well).
Unfortunately the code gen is quite a bit worse: Exibit A: https://run.dlang.io/is/jIuHRG *(cast(void16*)(&s2)) = *(cast(const void16*)(&s1)); _Dmain: push RBP mov RBP,RSP sub RSP,020h lea RAX,-020h[RBP] xor ECX,ECX mov [RAX],RCX mov 8[RAX],RCX lea RDX,-010h[RBP] mov [RDX],RCX mov 8[RDX],RCX movdqa XMM0,-020h[RBP] movdqa -010h[RBP],XMM0 xor EAX,EAX leave ret add [RAX],AL .text._Dmain ends Exhibit B: https://run.dlang.io/is/PLRfhW storeUnaligned(cast(void16*)(&s2), loadUnaligned(cast(const void16*)(&s1))); _Dmain: push RBP mov RBP,RSP sub RSP,050h lea RAX,-050h[RBP] xor ECX,ECX mov [RAX],RCX mov 8[RAX],RCX lea RDX,-040h[RBP] mov [RDX],RCX mov 8[RDX],RCX mov -030h[RBP],RDX mov -010h[RBP],RAX movdqu XMM0,[RAX] movdqa -020h[RBP],XMM0 movdqa XMM1,-020h[RBP] movdqu [RDX],XMM1 xor EAX,EAX leave ret add [RAX],AL .text._Dmain ends If the code gen was better, that would definitely be the way to go; to have unaligned and aligned share the same implementation. Maybe I can fix the DMD code gen, or implement a `copyUnaligned` intrinsic. Also, there doesn't seem to be any equivalent 32-byte implementations in `core.simd`. Is that just because noone's bother to implement them yet? And with AVX512, we should probably have 64-byte implementations as well. Mike
Jun 17
parent Mike Franklin <slavo5150 yahoo.com> writes:
On Monday, 18 June 2018 at 02:31:25 UTC, Mike Franklin wrote:

 Unfortunately the code gen is quite a bit worse:
Scratch that. If compiling with -O it seems to do the right thing. Mike
Jun 17