www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How do I force something onto the heap? (need for libev)

reply Tyler Jameson Little <beatgammit gmail.com> writes:
I've been playing with libev in D lately, and I've run into a problem. I've
been able to hack around it, but it'd like to find a better, more general
solution. Here's a link to the code:

https://github.com/beatgammit/fun-with-d/blob/master/libev/tcp_server.d

The code is a basic TCP server that responds to connections in a
non-blocking fashion. It's not perfect, and the current problem I'm trying
to solve is how to get my Socket instance (from accept) to the handler.
 Since everything is asynchronous, and the return value of accept() will
get lost (garbage collected, I think). When I try to get the address of it,
the compiler complains that it's not an lvalue.

As you can see, I've hacked around it by grabbing the handle and recreating
the Socket instance in the handler (line 14). The problem, however, is that
this can only assume a single type in AddressFamily. Honestly, I will
probably only need INET, INET6, or UNIX (which can be set in a global), but
this sounds a bit hacky to me.

I was able to hack around the problem for libev structs, because creating a
new instance returns a pointer, which can be assigned somewhere else to be
garbage collected later. This doesn't seem to be the case for classes,
however.

Initially, I solved this before by having a global Socket[] and adding
sockets to it as I received them, but this was an even worse hack to get
around the GC (see previous version in version control if interested). This
did, however, allow me to get a reference from the array to assign to the
data value (a void*), where it could be retrieved in the callback.

Are there any other options that I've missed that would make this cleaner
and more general?

Also, I'd be interested if someone notices some badness in my code that
could lead to nasty side-effects. I'm trying to make this example pretty
robust in terms of cleaning up after myself and doing things correctly.

Thanks so much!!
Mar 05 2012
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On 3/6/2012 1:34 PM, Tyler Jameson Little wrote:
 I've been playing with libev in D lately, and I've run into a problem.
 I've been able to hack around it, but it'd like to find a better, more
 general solution. Here's a link to the code:

 https://github.com/beatgammit/fun-with-d/blob/master/libev/tcp_server.d

 The code is a basic TCP server that responds to connections in a
 non-blocking fashion. It's not perfect, and the current problem I'm
 trying to solve is how to get my Socket instance (from accept) to the
 handler.  Since everything is asynchronous, and the return value of
 accept() will get lost (garbage collected, I think). When I try to get
 the address of it, the compiler complains that it's not an lvalue.
Socket instance returned by accept won't be garbage collected (or lost) as long as you have a reference to it active somewhere in your program. It doesn't matter which thread. Just take the return value of accept and pass it to your handler as is. As long as your handler holds on to the reference, you're fine. No need to try and get the address, or hack around it.
Mar 05 2012
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On 3/6/2012 1:55 PM, Mike Parker wrote:
 On 3/6/2012 1:34 PM, Tyler Jameson Little wrote:
 I've been playing with libev in D lately, and I've run into a problem.
 I've been able to hack around it, but it'd like to find a better, more
 general solution. Here's a link to the code:

 https://github.com/beatgammit/fun-with-d/blob/master/libev/tcp_server.d

 The code is a basic TCP server that responds to connections in a
 non-blocking fashion. It's not perfect, and the current problem I'm
 trying to solve is how to get my Socket instance (from accept) to the
 handler. Since everything is asynchronous, and the return value of
 accept() will get lost (garbage collected, I think). When I try to get
 the address of it, the compiler complains that it's not an lvalue.
Socket instance returned by accept won't be garbage collected (or lost) as long as you have a reference to it active somewhere in your program. It doesn't matter which thread. Just take the return value of accept and pass it to your handler as is. As long as your handler holds on to the reference, you're fine. No need to try and get the address, or hack around it.
Ah, sorry. Never mind. I misunderstood the problem. I suggest you keep an associative array of Sockets, using req.handle as a key. Then, your code becomes this: // The map Socket[socket_t] sockets; // in connection_cb auto req = server.accept(); sockets[req.handle] = req; // then in socket_watcher_cb auto req = sockets[w.fd];
Mar 05 2012
parent Mike Parker <aldacron gmail.com> writes:
On 3/6/2012 2:01 PM, Mike Parker wrote:
 On 3/6/2012 1:55 PM, Mike Parker wrote:
 On 3/6/2012 1:34 PM, Tyler Jameson Little wrote:
 I've been playing with libev in D lately, and I've run into a problem.
 I've been able to hack around it, but it'd like to find a better, more
 general solution. Here's a link to the code:

 https://github.com/beatgammit/fun-with-d/blob/master/libev/tcp_server.d

 The code is a basic TCP server that responds to connections in a
 non-blocking fashion. It's not perfect, and the current problem I'm
 trying to solve is how to get my Socket instance (from accept) to the
 handler. Since everything is asynchronous, and the return value of
 accept() will get lost (garbage collected, I think). When I try to get
 the address of it, the compiler complains that it's not an lvalue.
Socket instance returned by accept won't be garbage collected (or lost) as long as you have a reference to it active somewhere in your program. It doesn't matter which thread. Just take the return value of accept and pass it to your handler as is. As long as your handler holds on to the reference, you're fine. No need to try and get the address, or hack around it.
Ah, sorry. Never mind. I misunderstood the problem. I suggest you keep an associative array of Sockets, using req.handle as a key. Then, your code becomes this: // The map Socket[socket_t] sockets; // in connection_cb auto req = server.accept(); sockets[req.handle] = req; // then in socket_watcher_cb auto req = sockets[w.fd];
Alternatively: struct Wrapper { Socket s; this(Socket s) { this.s = s; } } // in connection_cb auto req = server.accept(); auto wrapper = new Wrapper(req); Then assign the wrapper instance to the data pointer you mentioned.
Mar 05 2012
prev sibling parent reply "Tyler Jameson Little" <beatgammit gmail.com> writes:
On Tuesday, 6 March 2012 at 04:54:44 UTC, Mike Parker wrote:
 On 3/6/2012 1:34 PM, Tyler Jameson Little wrote:
 I've been playing with libev in D lately, and I've run into a 
 problem.
 I've been able to hack around it, but it'd like to find a 
 better, more
 general solution. Here's a link to the code:

 https://github.com/beatgammit/fun-with-d/blob/master/libev/tcp_server.d

 The code is a basic TCP server that responds to connections in 
 a
 non-blocking fashion. It's not perfect, and the current 
 problem I'm
 trying to solve is how to get my Socket instance (from accept) 
 to the
 handler.  Since everything is asynchronous, and the return 
 value of
 accept() will get lost (garbage collected, I think). When I 
 try to get
 the address of it, the compiler complains that it's not an 
 lvalue.
Socket instance returned by accept won't be garbage collected (or lost) as long as you have a reference to it active somewhere in your program. It doesn't matter which thread. Just take the return value of accept and pass it to your handler as is. As long as your handler holds on to the reference, you're fine. No need to try and get the address, or hack around it.
Maybe I'm missing something then. I tried to do this: Add the following after line 34 in my code: watcher.data = &req; Change line 14 to: auto req = cast(Socket*)w.data; That all compiled ok, but I got a seg fault, which I assume is because it's been garbage collected. w.data should be the pointer that I assigned to it, or a Socket*. w.data is defined as a void*, and the struct comes from C (I think), so it wouldn't be read by the garbage collector in its sweep. Maybe I'm wrong, but that's what I think is going on. Is there something else going on here that I'm not seeing? Thanks so much for your help!
Mar 05 2012
parent reply "Tyler Jameson Little" <beatgammit gmail.com> writes:
Oh, thanks! I missed your reply.

That sounds reasonable, and a lot better than my super hacky 
Socket[].

Thanks!
Mar 05 2012
parent reply Mike Parker <aldacron gmail.com> writes:
On 3/6/2012 2:10 PM, Tyler Jameson Little wrote:
 Oh, thanks! I missed your reply.

 That sounds reasonable, and a lot better than my super hacky Socket[].

 Thanks!
I've never used libev and am only vaguely familiar with it. But if the callbacks are called from outside the main thread, you'll likely need to mark your socket map as shared or __gshared to guarantee you aren't working with thread-local data.
Mar 05 2012
parent "Tyler Jameson Little" <beatgammit gmail.com> writes:
On Tuesday, 6 March 2012 at 05:17:20 UTC, Mike Parker wrote:
 On 3/6/2012 2:10 PM, Tyler Jameson Little wrote:
 Oh, thanks! I missed your reply.

 That sounds reasonable, and a lot better than my super hacky 
 Socket[].

 Thanks!
I've never used libev and am only vaguely familiar with it. But if the callbacks are called from outside the main thread, you'll likely need to mark your socket map as shared or __gshared to guarantee you aren't working with thread-local data.
I think it runs in the same thread as it was called in, but that's a good insight. With your help, I got it working (just pushed to the same file if you're interested). Thanks for the insight about shared and __gshared, I just read about those today. I think I may do that anyway, because I'll end up working with multiple threads, and having access to it would be nice.
Mar 05 2012
prev sibling parent deadalnix <deadalnix gmail.com> writes:
Le 06/03/2012 05:34, Tyler Jameson Little a écrit :
 I've been playing with libev in D lately, and I've run into a problem.
 I've been able to hack around it, but it'd like to find a better, more
 general solution. Here's a link to the code:

 https://github.com/beatgammit/fun-with-d/blob/master/libev/tcp_server.d

 The code is a basic TCP server that responds to connections in a
 non-blocking fashion. It's not perfect, and the current problem I'm
 trying to solve is how to get my Socket instance (from accept) to the
 handler.  Since everything is asynchronous, and the return value of
 accept() will get lost (garbage collected, I think). When I try to get
 the address of it, the compiler complains that it's not an lvalue.

 As you can see, I've hacked around it by grabbing the handle and
 recreating the Socket instance in the handler (line 14). The problem,
 however, is that this can only assume a single type in AddressFamily.
 Honestly, I will probably only need INET, INET6, or UNIX (which can be
 set in a global), but this sounds a bit hacky to me.

 I was able to hack around the problem for libev structs, because
 creating a new instance returns a pointer, which can be assigned
 somewhere else to be garbage collected later. This doesn't seem to be
 the case for classes, however.

 Initially, I solved this before by having a global Socket[] and adding
 sockets to it as I received them, but this was an even worse hack to get
 around the GC (see previous version in version control if interested).
 This did, however, allow me to get a reference from the array to assign
 to the data value (a void*), where it could be retrieved in the callback.

 Are there any other options that I've missed that would make this
 cleaner and more general?

 Also, I'd be interested if someone notices some badness in my code that
 could lead to nasty side-effects. I'm trying to make this example pretty
 robust in terms of cleaning up after myself and doing things correctly.

 Thanks so much!!
You can new stuff(); Alternatively, if you have of stuff passed by value, you can : new stuff(stuffByValue); // Don't always work. *([stuffByValue].ptr); // Crazy but works :D
Mar 06 2012