www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - std.socket classes

reply Jonathan Marler <johnnymarler gmail.com> writes:
Does anyone know why Socket and Address in phobos were created as 
classes instead of structs?

My guess is that making Socket a class prevents socket handle 
leaks because you can clean up the handle in the destructor when 
the memory gets freed if no one closes it.  Is this the reason it 
is a class and are there any other reasons?

As for Address, I can't think of a reason why this one is a 
class.  It doesn't have to free any underlying OS resources, it's 
just a chunk of memory that can be passed to and from the socket 
API.  Using sockaddr in C/C++ is more flexible because it allows 
the application to decide where the memory lives (which will 
almost always be on the stack).  It feels like whoever made 
Address a class probably wasn't familiar with sockaddr.  Is that 
the case or are there reasons why it was made a class?

If I was implementing sockaddr in D, I would have chosen to use 
addressFamily as a sort of "makeshift Vptr", which is really how 
it is used in C (even though C doesn't support classes). Using 
this technique, I believe you could expose pretty much the same 
API without the overhead of wrapping it inside a D object.  Does 
anyone know if this solution was considered?
Apr 09
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
Don't think too hard, times have changed since std.socket was written.
It certainly isn't designed for high performance hence e.g. libasync.
Apr 09
parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Sunday, 9 April 2017 at 14:49:14 UTC, rikki cattermole wrote:
 Don't think too hard, times have changed since std.socket was 
 written.
 It certainly isn't designed for high performance hence e.g. 
 libasync.
What an odd response... You don't think I should ask questions about why decisions were made? If I took that approach how would I learn? And if you discourage other people from asking questions by telling them they are "thinking too hard" what kind of effect does that have on the community? As for "high performance", my questions have less to do with performance than they do with an API that makes sense and doesn't feel "kludgy".
Apr 09
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 09/04/2017 3:56 PM, Jonathan Marler wrote:
 On Sunday, 9 April 2017 at 14:49:14 UTC, rikki cattermole wrote:
 Don't think too hard, times have changed since std.socket was written.
 It certainly isn't designed for high performance hence e.g. libasync.
What an odd response... You don't think I should ask questions about why decisions were made? If I took that approach how would I learn? And if you discourage other people from asking questions by telling them they are "thinking too hard" what kind of effect does that have on the community? As for "high performance", my questions have less to do with performance than they do with an API that makes sense and doesn't feel "kludgy".
Oh sorry, I had a brain derp and thought at the end there you had that you thought about it and it didn't make sense. Hence the slightly weirder response. What I meant is that, for common use cases it works well enough and it does use reasonably sound API even if a bit cludgy. When asking about classes, one of the big things is the vtable. They are slow (compared to final classes and structs). This is the main reason people want to switch over to structs instead of classes. However if you take a look at the more performance aware libraries like libasync you will see classes used extensively still. Here is my suggestion, its a little harder to toy with ideas without real code to show for it. All our more existing API's are mostly class based for high performance sockets, timers ext. So, would you like to have a go and explore this area since we are playing it a bit too safe for your liking?
Apr 09
parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Sunday, 9 April 2017 at 15:04:29 UTC, rikki cattermole wrote:
 On 09/04/2017 3:56 PM, Jonathan Marler wrote:
 On Sunday, 9 April 2017 at 14:49:14 UTC, rikki cattermole 
 wrote:
 Don't think too hard, times have changed since std.socket was 
 written.
 It certainly isn't designed for high performance hence e.g. 
 libasync.
What an odd response... You don't think I should ask questions about why decisions were made? If I took that approach how would I learn? And if you discourage other people from asking questions by telling them they are "thinking too hard" what kind of effect does that have on the community? As for "high performance", my questions have less to do with performance than they do with an API that makes sense and doesn't feel "kludgy".
Oh sorry, I had a brain derp and thought at the end there you had that you thought about it and it didn't make sense. Hence the slightly weirder response.
Ah ok. That response was surprising to me coming from you (based on what I've read from you in the past) but I see it was a misunderstanding.
 What I meant is that, for common use cases it works well enough 
 and it does use reasonably sound API even if a bit cludgy.

 When asking about classes, one of the big things is the vtable. 
 They are slow (compared to final classes and structs). This is 
 the main reason people want to switch over to structs instead 
 of classes. However if you take a look at the more performance 
 aware libraries like libasync you will see classes used 
 extensively still.

 Here is my suggestion, its a little harder to toy with ideas 
 without real code to show for it. All our more existing API's 
 are mostly class based for high performance sockets, timers 
 ext. So, would you like to have a go and explore this area 
 since we are playing it a bit too safe for your liking?
What I've found myself having to do is use the lower level platform specific APIs that use socket_t and sockaddr, but then I get platform dependencies and can't access alot of the library because it requires the higher level Socket and Address objects. I would be willing to explore this area, but before I do work in an area I research what's already been done. Hence why I'm asking the questions about why it was done this way in the first place. For all I know there are very good reasons it was done this way that I just don't know about.
Apr 09
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 09/04/2017 4:19 PM, Jonathan Marler wrote:
 On Sunday, 9 April 2017 at 15:04:29 UTC, rikki cattermole wrote:
 On 09/04/2017 3:56 PM, Jonathan Marler wrote:
 On Sunday, 9 April 2017 at 14:49:14 UTC, rikki cattermole wrote:
 Don't think too hard, times have changed since std.socket was written.
 It certainly isn't designed for high performance hence e.g. libasync.
What an odd response... You don't think I should ask questions about why decisions were made? If I took that approach how would I learn? And if you discourage other people from asking questions by telling them they are "thinking too hard" what kind of effect does that have on the community? As for "high performance", my questions have less to do with performance than they do with an API that makes sense and doesn't feel "kludgy".
Oh sorry, I had a brain derp and thought at the end there you had that you thought about it and it didn't make sense. Hence the slightly weirder response.
Ah ok. That response was surprising to me coming from you (based on what I've read from you in the past) but I see it was a misunderstanding.
 What I meant is that, for common use cases it works well enough and it
 does use reasonably sound API even if a bit cludgy.

 When asking about classes, one of the big things is the vtable. They
 are slow (compared to final classes and structs). This is the main
 reason people want to switch over to structs instead of classes.
 However if you take a look at the more performance aware libraries
 like libasync you will see classes used extensively still.

 Here is my suggestion, its a little harder to toy with ideas without
 real code to show for it. All our more existing API's are mostly class
 based for high performance sockets, timers ext. So, would you like to
 have a go and explore this area since we are playing it a bit too safe
 for your liking?
What I've found myself having to do is use the lower level platform specific APIs that use socket_t and sockaddr, but then I get platform dependencies and can't access alot of the library because it requires the higher level Socket and Address objects. I would be willing to explore this area, but before I do work in an area I research what's already been done. Hence why I'm asking the questions about why it was done this way in the first place. For all I know there are very good reasons it was done this way that I just don't know about.
I personally disagree with the designs used in libasync[0] and vibe-core(vibe.d's replacement/new core implementation). Very class heavy and not enough emphasis on the event loop. But as you can see with my lib SPEW[2] even I don't get away from classes (in fact I go all in). Keep in mind only a handful of people really try to toy with this area. So you may very well come up with a completely different strategy that'll blow our minds without too much work. Existing implementations all have there trade offs, you simply may choose a completely different set and that is exciting. [0] https://github.com/etcimon/libasync [1] https://github.com/vibe-d/vibe-core [2] https://github.com/Devisualization/spew
Apr 09
prev sibling next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 9 April 2017 at 14:47:39 UTC, Jonathan Marler wrote:
 Does anyone know why Socket and Address in phobos were created 
 as classes instead of structs?
It is probably just the historical evolution, but I find it pretty handy: I did a subclass of Socket to do SSL, then as is nice with classes, I can pass that to other code using the Socket interface and have it work. So I'd be sad if they changed it now too much, the class is legitimately useful here and actually not very expensive`.
Apr 09
parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Monday, 10 April 2017 at 04:32:20 UTC, Adam D. Ruppe wrote:
 On Sunday, 9 April 2017 at 14:47:39 UTC, Jonathan Marler wrote:
 Does anyone know why Socket and Address in phobos were created 
 as classes instead of structs?
It is probably just the historical evolution, but I find it pretty handy: I did a subclass of Socket to do SSL, then as is nice with classes, I can pass that to other code using the Socket interface and have it work. So I'd be sad if they changed it now too much, the class is legitimately useful here and actually not very expensive`.
An interesting benefit. However, I don't think this is the ideal way to support such a use case. I think it would have been better if there was a shared stream/socket-like interface that you could override to use raw sockets or SSL. I'll explain why. My first thought is that since the interface you are using (the Socket class) wasn't really designed to be overridden, you probably had to do some interesting hacks to make it work. For example, when you accept a new socket, you probably had to delete the Socket object you got and create a new SSLSocket object passing the handle from one to the other, and make sure that the original Socket object didn't close it. I'm guessing that to prevent this close you probably set the socket handle on the accepted Socket object to null/invalid? Or maybe you just called the raw accept function and created a new object. But if the library is the one calling accept, then you would obviously have to override the accept function and do something that I would call "hacky". My other thought is that by separating both the virtual interface and the raw socket functions, you have provided both a low-level and high-level API that each application can choose to use. The tradeoff being "control" vs "extensibility". The high-level being more extensible (can be overriden to support things like SSL), and the low-level being less abstracted and therefore provides more control or access to the underlying implementation. This low-level access is more useful for code that needs to use socket-specific features. I will say that one disadvantage with this approach is that by separating both the virtual interface and the direct socket interface, you open up the door for library writers to make the mistake of using the wrong level of the API. If a library used the lower-level API and you wanted to override it with say, an SSL implementation, then you are out of luck unless you update the library to use the higher level interface. Of course this is more of a "practical real world" disadvantage that "in theory" can be prevented with good libraries. ----------------------- DISCLAIMER ----------------------- I would like to say something to anyone who wants to contribute to this thread. These comments are meant to discuss the pros/cons of the std.socket design and DO NOT serve as justification for changing phobos. Such a change would require much more discussion. The problem I've seen is that people will immediately halt a conversation by jumping to the end and arguing that such ideas will never be implemented because the benefit to risk ratio is too low. Benefit to risk ratio is very good and necessary discussion to have, however it's not good to stop a conversation early by jumping to this stage before people have even had a chance to discuss the merits of design and ideas on their own. Any discussion on the ideas/design with your thoughts/feedback/experience are welcome. If you want to discuss whether ideas/changes should be carried out, I would hold off those comments since it derails good discussion. Thanks.
Apr 10
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 10 April 2017 at 16:18:20 UTC, Jonathan Marler wrote:
 An interesting benefit. However, I don't think this is the 
 ideal way to support such a use case.
If I was doing it myself, I'd probably do an interface / final class split too (which also opens up a wee bit of additional easy optimization), but the Socket class isn't *bad* for this.
 My first thought is that since the interface you are using (the 
 Socket class) wasn't really designed to be overridden, you 
 probably had to do some interesting hacks to make it work.
No, the code is very straight-forward, regular class method overrides with appropriate forwards to the base class implementation where needed.
  For example, when you accept a new socket, you probably had to 
 delete the Socket object you got and create a new SSLSocket 
 object passing the handle from one to the other, and make sure
I didn't implement the server, but if I did, it would be another simple case of override SslSocket accept() { return new SslSocket(....); } or better yet, `override SslSocket accepting() { ...}`, since there IS a method specifically designed for this: http://dpldocs.info/experimental-docs/std.socket.Socket.accepting.html That'd work fine too.
Apr 10
parent Jonathan Marler <johnnymarler gmail.com> writes:
On Monday, 10 April 2017 at 18:57:13 UTC, Adam D. Ruppe wrote:
 On Monday, 10 April 2017 at 16:18:20 UTC, Jonathan Marler wrote:
 An interesting benefit. However, I don't think this is the 
 ideal way to support such a use case.
If I was doing it myself, I'd probably do an interface / final class split too (which also opens up a wee bit of additional easy optimization), but the Socket class isn't *bad* for this.
Yeah I agree, not perfect but not *bad*.
 My first thought is that since the interface you are using 
 (the Socket class) wasn't really designed to be overridden, 
 you probably had to do some interesting hacks to make it work.
No, the code is very straight-forward, regular class method overrides with appropriate forwards to the base class implementation where needed.
  For example, when you accept a new socket, you probably had 
 to delete the Socket object you got and create a new SSLSocket 
 object passing the handle from one to the other, and make sure
I didn't implement the server, but if I did, it would be another simple case of override SslSocket accept() { return new SslSocket(....); } or better yet, `override SslSocket accepting() { ...}`, since there IS a method specifically designed for this: http://dpldocs.info/experimental-docs/std.socket.Socket.accepting.html That'd work fine too.
Ah, it seems someone already ran into this accept problem, hence why the new "accepting" function was added. Funny timing that this was added near the time I asked the question. Since my last post I've been looking through std.socket and I see quite a bit of inefficiency especially when it comes to GC memory. I think moving forward the best solution for me is to use my version of std.socket. It would also be great if Walter's new ref-counted exceptions proposal gets implemented soon because then I could make it nogc. Anyway, thanks for the feedback.
Apr 10
prev sibling parent David Nadlinger <code klickverbot.at> writes:
On Sunday, 9 April 2017 at 14:47:39 UTC, Jonathan Marler wrote:
 My guess is that making Socket a class prevents socket handle 
 leaks because you can clean up the handle in the destructor 
 when the memory gets freed if no one closes it.  Is this the 
 reason it is a class and are there any other reasons?
Just to definitively answer the question: The design has been done way before even D 1.0 – actually, 0.86, I just checked the Git history – and has stayed more or less unchanged since. Several people, including me, have patched up various issues since, but the clunky design stayed unchanged. Including it in D2 proper was of course a mistake – it certainly wouldn't even come close to making it into Phobos today. — David
Apr 10