www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Using emplace leading to memory corruption

reply d coder <dlang.coder gmail.com> writes:
--001a11c2c6744fb4de04e77ff702
Content-Type: text/plain; charset=ISO-8859-1

Greetings

I am trying to use emplace and it is seemingly leading to memory
corruption. Before I file a bug on DMD Buzilla, I want to find out if I am
doing something wrong or if it is a known issue. I am using a linux box and
this testcase is freaky -- if I comment out either line 20 or line 24, the
segfault vanishes.

Thanks
- Puneet

$ rdmd --force -version=EMPLACE -Isrc -J.. test.d
4000
Segmentation fault (core dumped)
$ rdmd --force -Isrc -J.. test.d
4000

struct bdd {}                                              // 01
class Frop {                                               // 02
  int[] _var;                                              // 03
  int[] var() {                                            // 04
    return _var;                                           // 05
  }                                                        // 06
  this() {                                                 // 07
    this._var.length = 1;                                  // 08
  }                                                        // 09
}                                                          // 10
class Foo {                                                // 11
  long[] nodes;                                            // 12
  Frop[] frops;                                            // 13
  long[] table;                                            // 14
  this() {                                                 // 15
    nodes.length = 120000;                                 // 16
    frops.length = 1;                                      // 17
    frops[0] = new Frop();                                 // 18
    initTable();                                           // 19
    zoo(frops[0].var);                                     // 20
  }                                                        // 21
  void initTable() {                                       // 22
    import std.stdio;                                      // 23
    writeln(4000);                                         // 24
    table.length = 40000;                                  // 25
  }                                                        // 26
  void zoo(int[] varset) {}                                // 27
}                                                          // 28
class Bar {                                                // 29
  Foo _foo;                                                // 30
  this() {                                                 // 31
    version(EMPLACE) {                                     // 32
      import std.conv, core.stdc.stdlib;                   // 33
      enum size_t size = __traits(classInstanceSize, Foo); // 34
      assert(size is 32);                                  // 35
      void* tmp = core.stdc.stdlib.malloc(size);           // 36
      if (!tmp)                                            // 37
        throw new Exception("Memory allocation failed");   // 38
      void[] mem = tmp[0..size];                           // 39
      _foo = emplace!(Foo)(mem);                           // 40
    }                                                      // 41
    else {                                                 // 42
      _foo = new Foo();                                    // 43
    }                                                      // 44
  }                                                        // 45
}                                                          // 46
void main() {                                              // 47
  auto bar = new Bar;                                      // 48
}                                                          // 49

--001a11c2c6744fb4de04e77ff702
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Greetings</div><div><br></div><div>I am trying to use=
 emplace and it is seemingly leading to memory corruption. Before I file a =
bug on DMD Buzilla, I want to find out if I am doing something wrong or if =
it is a known issue. I am using a linux box and this testcase is freaky -- =
if I comment out either line 20 or line 24, the segfault vanishes.</div>

<div><br></div><div>Thanks</div><div>- Puneet</div><div><br></div><div>$ rd=
md --force -version=3DEMPLACE -Isrc -J.. test.d</div><div>4000</div><div>Se=
gmentation fault (core dumped)</div><div>$ rdmd --force -Isrc -J.. test.d</=
div>

<div>4000</div><div><br></div><div><div><font face=3D"courier new, monospac=
e">struct bdd {} =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0// 01</font></div><div><font face=3D"cou=
rier new, monospace">class Frop { =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 // 02</font></div>

<div><font face=3D"courier new, monospace">=A0 int[] _var; =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
// 03</font></div><div><font face=3D"courier new, monospace">=A0 int[] var(=
) { =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0// 04</font></div>

<div><font face=3D"courier new, monospace">=A0 =A0 return _var; =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 //=
 05</font></div><div><font face=3D"courier new, monospace">=A0 } =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0// 06</font></div>

<div><font face=3D"courier new, monospace">=A0 this() { =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 // 07</font></div><div><font face=3D"courier new, monospace">=A0 =A0 th=
is._var.length =3D 1; =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0// 08</font></div>

<div><font face=3D"courier new, monospace">=A0 } =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0// 09</font></div><div><font face=3D"courier new, monospace">} =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0// 10</font></div>

<div><font face=3D"courier new, monospace">class Foo { =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
// 11</font></div><div><font face=3D"courier new, monospace">=A0 long[] nod=
es; =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0// 12</font></div>

<div><font face=3D"courier new, monospace">=A0 Frop[] frops; =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
// 13</font></div><div><font face=3D"courier new, monospace">=A0 long[] tab=
le; =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0// 14</font></div>

<div><font face=3D"courier new, monospace">=A0 this() { =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 // 15</font></div><div><font face=3D"courier new, monospace">=A0 =A0 no=
des.length =3D 120000; =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 // 16</font></div>

<div><font face=3D"courier new, monospace">=A0 =A0 frops.length =3D 1; =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0// 1=
7</font></div><div><font face=3D"courier new, monospace">=A0 =A0 frops[0] =
=3D new Frop(); =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 // 18</font></div>

<div><font face=3D"courier new, monospace">=A0 =A0 initTable(); =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 //=
 19</font></div><div><font face=3D"courier new, monospace">=A0 =A0 zoo(frop=
s[0].var); =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 // 20</font></div>

<div><font face=3D"courier new, monospace">=A0 } =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0// 21</font></div><div><font face=3D"courier new, monospace">=
=A0 void initTable() { =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 // 22</font></div>

<div><font face=3D"courier new, monospace">=A0 =A0 import std.stdio; =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0// 2=
3</font></div><div><font face=3D"courier new, monospace">=A0 =A0 writeln(40=
00); =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 // 24</font></div>

<div><font face=3D"courier new, monospace">=A0 =A0 table.length =3D 40000; =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0// 25</f=
ont></div><div><font face=3D"courier new, monospace">=A0 } =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0// 26</font></div>

<div><font face=3D"courier new, monospace">=A0 void zoo(int[] varset) {} =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0// 27</font>=
</div><div><font face=3D"courier new, monospace">} =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0// 28</font></div>

<div><font face=3D"courier new, monospace">class Bar { =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
// 29</font></div><div><font face=3D"courier new, monospace">=A0 Foo _foo; =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0// 30</font></div>

<div><font face=3D"courier new, monospace">=A0 this() { =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 // 31</font></div><div><font face=3D"courier new, monospace">=A0 =A0 ve=
rsion(EMPLACE) { =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 // 32</font></div>

<div><font face=3D"courier new, monospace">=A0 =A0 =A0 import std.conv, cor=
e.stdc.stdlib; =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 // 33</font></div><div><=
font face=3D"courier new, monospace">=A0 =A0 =A0 enum size_t size =3D __tra=
its(classInstanceSize, Foo); // 34</font></div>

<div><font face=3D"courier new, monospace">=A0 =A0 =A0 assert(size is 32); =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0// 35</f=
ont></div><div><font face=3D"courier new, monospace">=A0 =A0 =A0 void* tmp =
=3D core.stdc.stdlib.malloc(size); =A0 =A0 =A0 =A0 =A0 // 36</font></div>

<div><font face=3D"courier new, monospace">=A0 =A0 =A0 if (!tmp) =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0// 37</font></div><div><font face=3D"courier new, monospace">=A0 =A0 =
=A0 =A0 throw new Exception(&quot;Memory allocation failed&quot;); =A0 // 3=
8</font></div>

<div><font face=3D"courier new, monospace">=A0 =A0 =A0 void[] mem =3D tmp[0=
..size]; =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 // 39</font></=
div><div><font face=3D"courier new, monospace">=A0 =A0 =A0 _foo =3D emplace=
!(Foo)(mem); =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 // 40</fon=
t></div>

<div><font face=3D"courier new, monospace">=A0 =A0 } =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0// 41</font></div><div><font face=3D"courier new, monospace">=
=A0 =A0 else { =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 // 42</font></div>

<div><font face=3D"courier new, monospace">=A0 =A0 =A0 _foo =3D new Foo(); =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0// 4=
3</font></div><div><font face=3D"courier new, monospace">=A0 =A0 } =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0// 44</font></div>

<div><font face=3D"courier new, monospace">=A0 } =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0// 45</font></div><div><font face=3D"courier new, monospace">} =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0// 46</font></div>

<div><font face=3D"courier new, monospace">void main() { =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
// 47</font></div><div><font face=3D"courier new, monospace">=A0 auto bar =
=3D new Bar; =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0// 48</font></div>

<div><font face=3D"courier new, monospace">} =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0// 49</font></div></div><div><br></div></div>

--001a11c2c6744fb4de04e77ff702--
Sep 28 2013
next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Sunday, 29 September 2013 at 07:03:30 UTC, d coder wrote:
 Greetings

 I am trying to use emplace and it is seemingly leading to memory
 corruption. Before I file a bug on DMD Buzilla, I want to find 
 out if I am
 doing something wrong or if it is a known issue. I am using a 
 linux box and
 this testcase is freaky -- if I comment out either line 20 or 
 line 24, the
 segfault vanishes.

 Thanks
 - Puneet

I didn't see anything obviously wrong with your code. Remember though that when emplacing a class over some memory, you must account for any extra alignment (though that doesn't seem to be a problem here). Any chance you could simplify this and file it?
Sep 29 2013
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 9/29/13, monarch_dodra <monarchdodra gmail.com> wrote:
 Remember though that when emplacing a class over some memory, you
 must account for any extra alignment (though that doesn't seem to
 be a problem here).

emplace does do a check for misalignment this internally with the testEmplaceChunk helper function. It would throw otherwise.
Sep 29 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Sunday, 29 September 2013 at 09:46:04 UTC, Andrej Mitrovic 
wrote:
 On 9/29/13, monarch_dodra <monarchdodra gmail.com> wrote:
 Remember though that when emplacing a class over some memory, 
 you
 must account for any extra alignment (though that doesn't seem 
 to
 be a problem here).

emplace does do a check for misalignment this internally with the testEmplaceChunk helper function. It would throw otherwise.

Right, emplace *checks*, which means caller must take it into account. This doesn't explain the issue, but it should be kept in mind when making the allocation, and building the slice. OP clearly just assumed his slice would be aligned.
Sep 29 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Sunday, 29 September 2013 at 08:21:34 UTC, monarch_dodra wrote:
 I didn't see anything obviously wrong with your code.

Did some more toying around. I did notice an issue in emplace, but nothing that would explain what you are observing. By replacing your block with: //---- version(EMPLACE) { //Allocation import core.stdc.stdlib; enum size_t size = __traits(classInstanceSize, Foo); auto tmp = cast(byte*)core.stdc.stdlib.malloc(size); if (!tmp) throw new Exception("Memory allocation failed"); assert (cast(size_t)tmp % 16 == 0); //Result auto result = cast(Foo)tmp; //Construction auto mem = tmp[0..size]; auto init = typeid(Foo).init; assert(init.ptr); mem[] = init[]; result.__ctor(); //Finished building, assign _foo = result; } //---- I'm observing the same behavior (Killed by signal 11). I *think* the code is bug free, so I'm leaning towards a code gen bug. Another thing I noticed is that if I call a class function on a null instance, I'm not getting a NullObjectError. I thought I was supposed to...?
Sep 29 2013
prev sibling next sibling parent reply "lomereiter" <lomereiter gmail.com> writes:
You didn't tell GC that the allocated memory contains pointers to 
GC'd data.
Therefore, it thinks that the class members can be freed, which 
is not the case.

Adding lines
 import core.memory; GC.addRange(tmp, size);

you will free the allocated memory.
       void[] mem = tmp[0..size];                           // 39
       _foo = emplace!(Foo)(mem);                           // 40

Sep 29 2013
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/29/13 5:29 AM, lomereiter wrote:
 You didn't tell GC that the allocated memory contains pointers to GC'd
 data.
 Therefore, it thinks that the class members can be freed, which is not
 the case.

 Adding lines
 import core.memory; GC.addRange(tmp, size);

will free the allocated memory.
       void[] mem = tmp[0..size];                           // 39
       _foo = emplace!(Foo)(mem);                           // 40


I tried that and it still crashes. I don't think that's the problem. Puneet, the code looks legit, please file a bug report. Andrei
Sep 29 2013
prev sibling parent d coder <dlang.coder gmail.com> writes:
--001a1133158e56d1cd04e785f048
Content-Type: text/plain; charset=ISO-8859-1

 I tried that and it still crashes. I don't think that's the problem.
 Puneet, the code looks legit, please file a bug report.

Thanks for confirming. http://d.puremagic.com/issues/show_bug.cgi?id=11139 Regards - Puneet --001a1133158e56d1cd04e785f048 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><br><div class=3D"gmail_extra"><div class=3D"gmail_quote">= <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p= adding-left:1ex"> I tried that and it still crashes. I don&#39;t think that&#39;s the problem= . Puneet, the code looks legit, please file a bug report.</blockquote><div>= <br></div><div>Thanks for confirming.</div><div><br></div><div>=A0<a href= =3D"http://d.puremagic.com/issues/show_bug.cgi?id=3D11139">http://d.puremag= ic.com/issues/show_bug.cgi?id=3D11139</a></div> <div><br></div><div>Regards</div><div>- Puneet</div></div></div></div> --001a1133158e56d1cd04e785f048--
Sep 29 2013