www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 2356] New: array literal as non static initializer generates horribly inefficient code.

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356

           Summary: array literal as non static initializer generates
                    horribly inefficient code.
           Product: D
           Version: unspecified
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Keywords: wrong-code
          Severity: major
          Priority: P2
         Component: DMD
        AssignedTo: bugzilla digitalmars.com
        ReportedBy: tomas famolsen.dk


this simple test:

void main()
{
    int[3] x = [1,2,3];
}

should just initialize x with the values 1,2,3

what happens in reality is not as nice as this output of obj2asm shows:

_Dmain:
                push    EBP
                mov     EBP,ESP
                sub     ESP,0Ch
                push    EBX
                lea     EAX,-0Ch[EBP]
                push    EAX
                push    3
                push    3
                push    2
                push    1
                push    3
                mov     ECX,offset FLAT:_D11TypeInfo_Ai6__initZ SYM32
                push    ECX
                call    near ptr _d_arrayliteralT PC32
                add     ESP,014h
                mov     EDX,EAX
                mov     EBX,3
                push    EDX
                push    EBX
                push    4
                call    near ptr _d_arraycopy PC32
                xor     EAX,EAX
                add     ESP,014h
                pop     EBX
                leave
                ret


I'm marking it wrong-code, even though the code does compile and link, it's
just wrong ... I'm using 1.034 but I'd guess it's an older bug, so I'm not
marking a specific version.


-- 
Sep 11 2008
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


tomas famolsen.dk changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|unspecified                 |1.034




------- Comment #1 from tomas famolsen.dk  2008-09-15 08:52 -------
I think this issue might have been introduced in DMD 1.034, as I've hit several
issues in LLVMDC, that used to work, due to some initializers suddenly being
ArrayLiterals instead of ArrayInitializers. Marking version 1.034.


-- 
Sep 15 2008
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


clugdbug yahoo.com.au changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|wrong-code                  |performance




------- Comment #2 from clugdbug yahoo.com.au  2009-04-08 08:01 -------
This is a performance issue, not wrong-code.


-- 
Apr 08 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


bearophile_hugs eml.cc changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bearophile_hugs eml.cc


--- Comment #3 from bearophile_hugs eml.cc 2010-07-19 23:18:32 PDT ---
*** Issue 4488 has been marked as a duplicate of this issue. ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 19 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


Kenji Hara <k.hara.pg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |patch


--- Comment #4 from Kenji Hara <k.hara.pg gmail.com> 2011-09-09 09:35:07 PDT ---
https://github.com/D-Programming-Language/dmd/pull/375

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 09 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356



--- Comment #5 from Kenji Hara <k.hara.pg gmail.com> 2011-09-09 09:36:49 PDT ---
After applying my patch, the sample code generates like follows:
(output of ddbg in 64-bit Windows 7)

c:\d\test.d:1 void main()
00402010: c80c0000                enter 0xc, 0x0
c:\d\test.d:3     int[3] x = [1,2,3];
00402014: c745f401000000          mov dword [ebp-0xc], 0x1
0040201b: c745f802000000          mov dword [ebp-0x8], 0x2
00402022: c745fc03000000          mov dword [ebp-0x4], 0x3
00402029: 31c0                    xor eax, eax
test.obj
0040202b: c9                      leave
0040202c: c3                      ret

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 09 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


dawg dawgfoto.de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dawg dawgfoto.de


--- Comment #6 from dawg dawgfoto.de 2011-10-07 19:09:44 PDT ---
This is also a huge issue when assigning .init to a static array.

ubyte[1024] v;
v = typeof(v).init;

This will generate a dynamically allocated [0, 0, 0 ... ] Arrayliteral for the
rhs expression before calling _d_arraycopy. Even worse the allocated
ArrayLiteral is initialized using 1024 comma expressions. Using -O the compiler
will subsequently almost starve from O(N^2) behavior during comsub
eliminations.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 07 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


yebblies <yebblies gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |rsinfu gmail.com


--- Comment #7 from yebblies <yebblies gmail.com> 2012-02-02 14:27:49 EST ---
*** Issue 4298 has been marked as a duplicate of this issue. ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 01 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


yebblies <yebblies gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sean invisibleduck.org


--- Comment #8 from yebblies <yebblies gmail.com> 2012-02-02 15:34:11 EST ---
*** Issue 4881 has been marked as a duplicate of this issue. ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 01 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


yebblies <yebblies gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jarrett.billingsley gmail.c
                   |                            |om


--- Comment #9 from yebblies <yebblies gmail.com> 2012-02-02 20:49:34 EST ---
*** Issue 2237 has been marked as a duplicate of this issue. ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 02 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


Kenji Hara <k.hara.pg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|patch                       |


--- Comment #10 from Kenji Hara <k.hara.pg gmail.com> 2012-05-05 00:42:06 PDT
---
Pull #375 was not sufficient, so removed 'patch' keyword.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
May 05 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


Jonathan M Davis <jmdavisProg gmx.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |malteskarupke web.de


--- Comment #11 from Jonathan M Davis <jmdavisProg gmx.com> 2012-10-14 11:36:34
PDT ---
*** Issue 8820 has been marked as a duplicate of this issue. ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 14 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


yebblies <yebblies gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |siegelords_abode yahoo.com


--- Comment #12 from yebblies <yebblies gmail.com> 2012-10-29 05:14:56 EST ---
*** Issue 8903 has been marked as a duplicate of this issue. ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 28 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


Denis Shelomovskij <verylonglogin.reg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |verylonglogin.reg gmail.com


--- Comment #13 from Denis Shelomovskij <verylonglogin.reg gmail.com>
2012-10-30 17:17:29 MSK ---
Workaround for those who like "a, b, c" initialization but need more
performance (not: it still calls `_d_arraycopy`):

---
T[n] makeStaticArray(T, size_t n)(T[n] data...)
// { return data; }
{ T[n] res; res = data; return res; } // Issue 8914 workaround

void setStaticArray(T, size_t n)(ref T[n] array, T[n] data...)
{ array = data; }

void main()
{
    auto x = makeStaticArray(1, 2, 3);
    static assert(is(typeof(x) == int[3]));
    assert(x == [1, 2, 3]);

    int[3] y;
    y.setStaticArray(1, 2, 3);
    assert(y == [1, 2, 3]);
}
---

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 30 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


Kenji Hara <k.hara.pg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |pull


--- Comment #14 from Kenji Hara <k.hara.pg gmail.com> 2013-04-10 10:31:22 PDT
---
New D2 fix:
https://github.com/D-Programming-Language/dmd/pull/1883

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 10 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356



--- Comment #15 from bearophile_hugs eml.cc 2013-04-10 19:20:51 PDT ---
(In reply to comment #14)
 New D2 fix:
 https://github.com/D-Programming-Language/dmd/pull/1883
From the pull request (dmd -O -inline -g test after): c:\d\test.d:18 int[3] y = [n, n, n]; 004020aa: 6a03 push 0x3 004020ac: 6a05 push 0x5 004020ae: 8d4c241c lea ecx, [esp+0x1c] 004020b2: 51 push ecx 004020b3: e880020000 call 0x402338 __memset32 Isn't calling memset for just 3 integers slower than inlining their assignments? I suggest to not call memset if the number of bytes to be copied is so small (I think LDC is already doing similar optimizations). Maybe a benchmark is also useful here. c:\d\test.d:20 S[3] z = [s2, s2, s2]; 004020b8: 8d542418 lea edx, [esp+0x18] 004020bc: 52 push edx 004020bd: 8d442430 lea eax, [esp+0x30] 004020c1: e86affffff call 0x402030 test.S.__cpctor c:\d\test.d:3 004020c6: 8d5c2418 lea ebx, [esp+0x18] 004020ca: 53 push ebx 004020cb: 8d442434 lea eax, [esp+0x34] 004020cf: e85cffffff call 0x402030 test.S.__cpctor c:\d\test.d:3 004020d4: 53 push ebx 004020d5: 8d442438 lea eax, [esp+0x38] 004020d9: e852ffffff call 0x402030 test.S.__cpctor c:\d\test.d:3 004020de: 83c40c add esp, 0xc 004020e1: 31c0 xor eax, eax If the s2 variable already contains the struct, then what's the purpose of those calls to 0x402030? In the "before" there are no calls to struct constructors: c:\d\test.d:20 S[3] z = [s2, s2, s2]; 00403913: 8d542474 lea edx, [esp+0x74] 00403917: b960014200 mov ecx, 0x420160 0040391c: 52 push edx 0040391d: 6a03 push 0x3 0040391f: 6a03 push 0x3 00403921: 51 push ecx 00403922: e8fd0a0000 call 0x404424 __d_arrayliteralTX 00403927: 83c408 add esp, 0x8 0040392a: 8d542470 lea edx, [esp+0x70] 0040392e: 52 push edx 0040392f: 89c6 mov esi, eax -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Apr 10 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356



--- Comment #16 from Kenji Hara <k.hara.pg gmail.com> 2013-04-10 19:34:08 PDT
---
(In reply to comment #15)
 (In reply to comment #14)
 New D2 fix:
 https://github.com/D-Programming-Language/dmd/pull/1883
From the pull request (dmd -O -inline -g test after): c:\d\test.d:18 int[3] y = [n, n, n]; 004020aa: 6a03 push 0x3 004020ac: 6a05 push 0x5 004020ae: 8d4c241c lea ecx, [esp+0x1c] 004020b2: 51 push ecx 004020b3: e880020000 call 0x402338 __memset32 Isn't calling memset for just 3 integers slower than inlining their assignments? I suggest to not call memset if the number of bytes to be copied is so small (I think LDC is already doing similar optimizations). Maybe a benchmark is also useful here.
It is lowered to: int[3] y = void; y[] = n; And currently dmd uses memset for `y[] = n;`. It is another optimization issue.
 c:\d\test.d:20     S[3] z = [s2, s2, s2];
 004020b8: 8d542418                lea edx, [esp+0x18]
 004020bc: 52                      push edx
 004020bd: 8d442430                lea eax, [esp+0x30]
 004020c1: e86affffff              call 0x402030 test.S.__cpctor c:\d\test.d:3
 004020c6: 8d5c2418                lea ebx, [esp+0x18]
 004020ca: 53                      push ebx
 004020cb: 8d442434                lea eax, [esp+0x34]
 004020cf: e85cffffff              call 0x402030 test.S.__cpctor c:\d\test.d:3
 004020d4: 53                      push ebx
 004020d5: 8d442438                lea eax, [esp+0x38]
 004020d9: e852ffffff              call 0x402030 test.S.__cpctor c:\d\test.d:3
 004020de: 83c40c                  add esp, 0xc
 004020e1: 31c0                    xor eax, eax
 
 If the s2 variable already contains the struct, then what's the purpose of
 those calls to 0x402030?


 In the "before" there are no calls to struct constructors:
 
 c:\d\test.d:20     S[3] z = [s2, s2, s2];
 00403913: 8d542474                lea edx, [esp+0x74]
 00403917: b960014200              mov ecx, 0x420160
 0040391c: 52                      push edx
 0040391d: 6a03                    push 0x3
 0040391f: 6a03                    push 0x3
 00403921: 51                      push ecx
 00403922: e8fd0a0000              call 0x404424 __d_arrayliteralTX
 00403927: 83c408                  add esp, 0x8
 0040392a: 8d542470                lea edx, [esp+0x70]
 0040392e: 52                      push edx
 0040392f: 89c6                    mov esi, eax
Before, cpctor(==postblit) calls are done in __d_arrayliteralTX, so they are hidden. Now they are directly called on the stack memory z[0..3]. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Apr 10 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356



--- Comment #17 from bearophile_hugs eml.cc 2013-04-10 19:44:37 PDT ---
(In reply to comment #16)

 It is lowered to:
   int[3] y = void;
   y[] = n;
 
 And currently dmd uses memset for `y[] = n;`. It is another optimization issue.
OK.
 Before, cpctor(==postblit) calls are done in __d_arrayliteralTX, so they are
 hidden. Now they are directly called on the stack memory z[0..3].
Sorry I have missed it was the postblit, thank you. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Apr 10 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356



--- Comment #18 from github-bugzilla puremagic.com 2013-04-11 01:36:48 PDT ---
Commits pushed to master at https://github.com/D-Programming-Language/dmd

https://github.com/D-Programming-Language/dmd/commit/8cd5f790a78e7514e46618d0325e92cbd6e00e48
fix Issue 2356 - array literal as non static initializer generates horribly
inefficient code.

https://github.com/D-Programming-Language/dmd/commit/d4b20baee7a1c9ee8a9271724feb5d1031e773d4
Merge pull request #1883 from 9rnsr/fix2356

Issue 2356 - array literal as non static initializer generates horribly
inefficient code.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 11 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356



--- Comment #19 from bearophile_hugs eml.cc 2013-04-11 05:26:29 PDT ---
The patch seems to work. With it I have removed five optimizations from my
code. Very good.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 11 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


Andrei Alexandrescu <andrei erdani.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |andrei erdani.com
         Resolution|                            |FIXED


--- Comment #20 from Andrei Alexandrescu <andrei erdani.com> 2013-04-11
06:12:47 PDT ---
Thanks, Kenji!

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 11 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356


Benjamin Thaut <code benjamin-thaut.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code benjamin-thaut.de


--- Comment #21 from Benjamin Thaut <code benjamin-thaut.de> 2013-06-06
00:04:57 PDT ---
Awesome, thanks for fixing this. This was my number 1 most annoying bug in D.
Because when not using a GC the old behavior always leaked memory.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jun 06 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356



--- Comment #22 from Denis Shelomovskij <verylonglogin.reg gmail.com>
2013-10-25 00:21:28 MSD ---
The fix is partially reverted in dmd pull 2682 [1] as it introduced Issue
11238.

Only construction is optimized now:
---
void main()
{
    int[2] m = [4, 6]; // still optimized
    m = [m[1], m[0]]; // swap, currently calls `_d_arrayliteralTX`
    assert(m == [6, 4]); // was [6, 6] with original fix (Issue 11238)
}
---

Opened Issue 11345 for assignment case.

Also some duplicates of current issue and duplicates of duplicates of current
issue are now really duplicates of Issue 11345.

[1] https://github.com/D-Programming-Language/dmd/pull/2682

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 24 2013
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2356



--- Comment #23 from Denis Shelomovskij <verylonglogin.reg gmail.com>
2013-10-25 00:45:30 MSD ---
(In reply to comment #22)
 Also some duplicates of current issue and duplicates of duplicates of current
 issue are now really duplicates of Issue 11345.
Looks like Issue 4881 was the only one. Changed its "DUPLICATE of". -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 24 2013