www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 10970] New: Segfault in a simple test compiled without -g.

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970

           Summary: Segfault in a simple test compiled without -g.
           Product: D
           Version: D2
          Platform: x86_64
        OS/Version: Windows
            Status: NEW
          Severity: critical
          Priority: P2
         Component: DMD
        AssignedTo: nobody puremagic.com
        ReportedBy: pycerl qq.com



http://dpaste.dzfl.pl/8cc57a33

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 05 2013
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970




After some test, the bug is likely to be that, an Array instantiated by
make!(Array!T) WITHOUT ANY ARGUMENT, cannot be used as a value of AA.

following code fail to run:

alias Array!int A;
A[int] m;
m[0] = make!A;

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 05 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970




Further test shows that, Array just cann't be used as an value of AA in
dmd2.063(on dpaste), while under git-head version of dmd, segfault only occured
when an array is created by make!Array without any argument.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 05 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970


Andrej Mitrovic <andrej.mitrovich gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |andrej.mitrovich gmail.com



19:34:24 PDT ---
I think this is a 64-bit only bug. Could you try compiling with -m32 to
confirm?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 05 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970




19:35:48 PDT ---


 I think this is a 64-bit only bug. Could you try compiling with -m32 to
 confirm?


Ok nevermind, I was testing your
http://d.puremagic.com/issues/show_bug.cgi?id=10970#c1 instead of the dpaste
one (try not to use dpaste for short examples).

Your test-case:

-----
import std.stdio;
import std.container;
import std.process;

class A
{
    this(string name)
    {
        m[name] = make!Arr;
    }

    alias Array!A Arr;
    Arr[string] m;
}

int main()
{
    A a = new A("test");
    return 0;
}
-----

Crashes on win32 as well.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 05 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970








 I think this is a 64-bit only bug. Could you try compiling with -m32 to
 confirm?


 
 Ok nevermind, I was testing your
 http://d.puremagic.com/issues/show_bug.cgi?id=10970#c1 instead of the dpaste
 one (try not to use dpaste for short examples).
 
 Your test-case:
 
 -----
 import std.stdio;
 import std.container;
 import std.process;
 
 class A
 {
     this(string name)
     {
         m[name] = make!Arr;
     }
 
     alias Array!A Arr;
     Arr[string] m;
 }
 
 int main()
 {
     A a = new A("test");
     return 0;
 }
 -----
 
 Crashes on win32 as well.


Hope it helps and the issue be fixed asap, thank you!

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 05 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970


monarchdodra gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |monarchdodra gmail.com





 Crashes on win32 as well.


If I recall correctly, I think the problem is "simply" that AA has *very* bad
support for types with elaborate CC, and destruction. Take this "simple" use
case:


import std.stdio;

int count;

struct S
{
    this(int){++count; writeln("construction");}
    this(this){assert(0);} //never called
    ~this(){--count; writeln("destruction");}
}

void main()
{
    S[int] aa;
    writeln("----");
    aa[5] = S(3);
    writeln("----");
    aa.remove(5);
    writeln("----");
    assert(aa.length == 0);
    assert(count >= 0); //Fails here
}

----
construction
destruction
destruction
----
----


In just this little snippet, there are *all* kinds of wrong:
1. insertion: when inserting a new item, CC ("this(this)") is never called for
some strange reason. Furthermore, destruction happens *twice* (again, for some
strange reason).
2. removal: Where's the destructor?

These use cases are probably reported already, you are experiencing the results
of these defects.

Long story short, at the end of the day, your array is "over-destroyed". Given
that AA is a *strongly* relies on reference counting to provide deterministic
behavior, things simply don't go well.

A lot of work is being done on fixing AAs (AFAIK), but don't expect it to be
fixed any time soon.

As a workaround, you could try using slices instead of AA's?

//----
import std.stdio;

class A
{
    this(string name)
    {
        m[name] = Arr.init;
    }

    alias A[] Arr;
    Arr[string] m;
}

int main()
{
    A a = new A("test");
    return 0;
}
//----

In particular, "A" is a class, so it can't have a destructor (in the sense that
a *reference* doesn't get destroyed). As such, there is minimal use to keeping
an Array over a slice.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 06 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970




07:18:43 PDT ---
Yeah it's very unfortunate the AA situation we have in D today, it's really
giving D a bad rep.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 06 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970








 Crashes on win32 as well.


 
 If I recall correctly, I think the problem is "simply" that AA has *very* bad
 support for types with elaborate CC, and destruction. Take this "simple" use
 case:
 

 import std.stdio;
 
 int count;
 
 struct S
 {
     this(int){++count; writeln("construction");}
     this(this){assert(0);} //never called
     ~this(){--count; writeln("destruction");}
 }
 
 void main()
 {
     S[int] aa;
     writeln("----");
     aa[5] = S(3);
     writeln("----");
     aa.remove(5);
     writeln("----");
     assert(aa.length == 0);
     assert(count >= 0); //Fails here
 }

 ----
 construction
 destruction
 destruction
 ----
 ----

 
 In just this little snippet, there are *all* kinds of wrong:
 1. insertion: when inserting a new item, CC ("this(this)") is never called for
 some strange reason. Furthermore, destruction happens *twice* (again, for some
 strange reason).
 2. removal: Where's the destructor?
 
 These use cases are probably reported already, you are experiencing the results
 of these defects.
 
 Long story short, at the end of the day, your array is "over-destroyed". Given
 that AA is a *strongly* relies on reference counting to provide deterministic
 behavior, things simply don't go well.
 
 A lot of work is being done on fixing AAs (AFAIK), but don't expect it to be
 fixed any time soon.
 
 As a workaround, you could try using slices instead of AA's?
 
 //----
 import std.stdio;
 
 class A
 {
     this(string name)
     {
         m[name] = Arr.init;
     }
 
     alias A[] Arr;
     Arr[string] m;
 }
 
 int main()
 {
     A a = new A("test");
     return 0;
 }
 //----
 
 In particular, "A" is a class, so it can't have a destructor (in the sense that
 a *reference* doesn't get destroyed). As such, there is minimal use to keeping
 an Array over a slice.


Thank you, I can work around it by changing the design, however, this really
need to be fixed in the near future!

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 06 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970






 Yeah it's very unfortunate the AA situation we have in D today, it's really
 giving D a bad rep.


Absolutely, after hanging around D community for days, I decide to start a
private project in D, however, the first class I implemented broke by this
issue!

FYI, I'm a game programmer and GC is an another trouble-maker for me, hope it
can be improved in the near futher too.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 06 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970





[...]

 In just this little snippet, there are *all* kinds of wrong:
 1. insertion: when inserting a new item, CC ("this(this)") is never called for
 some strange reason. Furthermore, destruction happens *twice* (again, for some
 strange reason).
 2. removal: Where's the destructor?


[...]

Ugh. I looked at the code, and it's just one big ugly mess.

For starters, _aaGetX, which implements "aa[key] = value", takes only a *size*
parameter to the value to be stored. If the Slot for that key doesn't already
exist, it creates one for it, and then proceeds to memset the value part of the
Slot to binary zero. So already, we have a problem: now there's a Slot for
which the value isn't properly initialized (e.g., if the value type has a
non-trivial ctor that sets things up).

Next, when you call remove(), it ultimately goes to _aaDelX, which calls
GC.free on the Slot, which, according to the docs, explicitly does NOT finalize
the block. So, the key and value originally in that slot, if they have dtors,
won't have their dtors called.

To top things off, I added writeln's before and after inserting the element
into the AA, and discovered that the ctor call and BOTH of the dtor calls
happen *during insertion into the AA*. So it looks like after _aaGetX created
an invalid instance of S (bypassing the ctor), the compiler is attempting to
perform an assignment to it from a temporary copy of S it created from the RHS
of the statement. Since the compiler believes (wrongly) that the AA Slot
already contains a value of type S, it calls S's dtor in order to delete the
old value, then copies the new value into it. Afterwards, it destructs the
temporary copy of S (so this accounts for the 1 ctor call and 2 dtor calls).
But here, the mess is topped off with yet another bug: the postblit isn't being
called after this assignment!!

Ugh! This code is an embarrassment!! Makes me want to fork dmd, rip out every
last bit of the existing AA implementation, and redo it from scratch. It'll
probably be easier than trying to sort out this rabbit warren of AA bugs.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 06 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970




17:03:34 PDT ---


 Ugh! This code is an embarrassment!! Makes me want to fork dmd, rip out every
 last bit of the existing AA implementation, and redo it from scratch. It'll
 probably be easier than trying to sort out this rabbit warren of AA bugs.


Yeah these bugs you listed are kind of "known" in the community, but nobody
really wants to touch the code for associative arrays. Maybe it's because
they're so tied to the compiler.

I can list about ~50 open bugs related to hashes:

http://d.puremagic.com/issues/buglist.cgi?query_format=advanced&order=Bug%20Number&short_desc=associative&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=VERIFIED&short_desc_type=allwordssubstr&component=coffimplib&component=DMD&component=druntime&component=dstress&component=general&component=glue%20layer&component=htod&component=installer&component=make&component=obj2asm&component=Optlink&component=Phobos&component=TestComponent&component=websites&product=D&product=DGCC%20aka%20GDC&product=DStress&product=puremagic.com&product=TestProduct

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 06 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970




Only 50? I have 78 on my list, and that's not including bugs that don't say AA
or hash in the subject line, like this one.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 06 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970




17:10:41 PDT ---


 Ugh! This code is an embarrassment!! Makes me want to fork dmd, rip out every
 last bit of the existing AA implementation, and redo it from scratch. It'll
 probably be easier than trying to sort out this rabbit warren of AA bugs.


If you do that, we will build a statue in your honor in Menlo Park.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 06 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970




17:11:24 PDT ---


 Only 50? I have 78 on my list, and that's not including bugs that don't say AA
 or hash in the subject line, like this one.


My bugzilla query string must be inadequate, what's yours?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 06 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970




http://d.puremagic.com/issues/buglist.cgi?query_format=advanced&short_desc=aa%20associative%20hash&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&short_desc_type=anywordssubstr&version=D1%20%26%20D2&version=D2&known_name=aa&query_based_on=aa

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 06 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970


Kenji Hara <k.hara.pg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |pull, wrong-code
           Platform|x86_64                      |All
         OS/Version|Windows                     |All



---
The root cause is a dup of bug 6178.

https://github.com/D-Programming-Language/dmd/pull/2539

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 21 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970




Commit pushed to master at https://github.com/D-Programming-Language/dmd

https://github.com/D-Programming-Language/dmd/commit/36a44cbcea9fbb18b221f77e00f2dca77f21bc88
fix Issue 10970 - Segfault in a simple test compiled without -g.

Fixes `CondExp::toElem` to avoid "Internal error: backend\cgcs.c 351"

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 22 2013
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10970


Kenji Hara <k.hara.pg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 22 2013