www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.socket

reply Steve Teale <steve.teale britseyeview.com> writes:
Who is the resident expert? Sean maybe?

How do I get the receive buffer size up into the megabytes range for use 
between the D client and localhost. setOption seems to only get me about 
262k.

Steve
Nov 02 2011
next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Steve Teale wrote:
 Who is the resident expert? Sean maybe?
 
 How do I get the receive buffer size up into the megabytes range for use 
 between the D client and localhost. setOption seems to only get me about 
 262k.

How do you do it?
From the documentation I would expect that

auto sizeInBytes = 10 * 2^^20; mySocket.setOption(SocketOptionLevel.SOCKET, SocketOption.RCVBUF, sizeInBytes); should set it to 10 MiB; I'm unsure about the socket option level. There is also SocketOptionLevel.IP. Jens
Nov 02 2011
prev sibling next sibling parent Steve Teale <steve.teale britseyeview.com> writes:
On Wed, 02 Nov 2011 17:17:07 +0100, Jens Mueller wrote:
 should set it to 10 MiB;
 I'm unsure about the socket option level. There is also
 SocketOptionLevel.IP.
 

buffer size, but not by anything close to what you ask for. I've tried both SOCKET and IP. Steve
Nov 02 2011
prev sibling next sibling parent "Regan Heath" <regan netmail.co.nz> writes:
On Wed, 02 Nov 2011 16:35:25 -0000, Steve Teale  
<steve.teale britseyeview.com> wrote:

 On Wed, 02 Nov 2011 17:17:07 +0100, Jens Mueller wrote:
 should set it to 10 MiB;
 I'm unsure about the socket option level. There is also
 SocketOptionLevel.IP.

buffer size, but not by anything close to what you ask for. I've tried both SOCKET and IP.

What operating system are you on? This may be of interest (even if you're not on windows, as it references some RFCs etc which may have some bearing on how things work on your OS): http://msdn.microsoft.com/en-us/library/ms819736.aspx Also, this thread mentions fpathconf() which I've not heard of before (probably because it's more of a UNIX thing?): http://stackoverflow.com/questions/565804/max-value-for-so-rcvbuf-in-windows .. why do you want/need such a large TCPIP recv buffer? Regan -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 02 2011
prev sibling next sibling parent Steve Teale <steve.teale britseyeview.com> writes:
On Wed, 02 Nov 2011 16:45:30 +0000, Regan Heath wrote:

 Yes Jens, that's what I thought, and that approach does indeed change
 the buffer size, but not by anything close to what you ask for. I've
 tried both SOCKET and IP.

What operating system are you on? This may be of interest (even if you're not on windows, as it references some RFCs etc which may have some bearing on how things work on your OS): http://msdn.microsoft.com/en-us/library/ms819736.aspx Also, this thread mentions fpathconf() which I've not heard of before (probably because it's more of a UNIX thing?): http://stackoverflow.com/questions/565804/max-value-for-so-rcvbuf-in-

 
 .. why do you want/need such a large TCPIP recv buffer?

I'm using Ubuntu i386. I just thought that since MySQL talks blithely about packet sizes of 16mb I would investigate how my protocol interface performs with large buffers and large BLOBS. I believe Ubuntu imposes the buffer size limitation and you have to mess with kernel configuration files to change that. But I don't want to go there and wondered if I was missing some simple point. Steve
Nov 02 2011
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Steve Teale wrote:
 On Wed, 02 Nov 2011 17:17:07 +0100, Jens Mueller wrote:
 should set it to 10 MiB;
 I'm unsure about the socket option level. There is also
 SocketOptionLevel.IP.
 

buffer size, but not by anything close to what you ask for. I've tried both SOCKET and IP.

I'm looking into std.socket.setOption at the moment. There is (&value)[0 .. 1] where value is an int32_t. Then setsockopt is called using this void[] array. I wonder what this code (&value)[0 .. 1] does and I don't how the argument to setsockopt needs to be (need to consult the manual). Maybe it's an endianess issue. In sum, besides the above code setOption is a direct wrapper around setsockopt. Jens
Nov 02 2011
prev sibling next sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Nov 2, 2011, at 9:59 AM, Steve Teale wrote:

 On Wed, 02 Nov 2011 16:45:30 +0000, Regan Heath wrote:
=20
 .. why do you want/need such a large TCPIP recv buffer?

I'm using Ubuntu i386. I just thought that since MySQL talks blithely=20=

 about packet sizes of 16mb I would investigate how my protocol =

 performs with large buffers and large BLOBS.

I suspect that MySQL's "packet size" has no relation to TCP packet size. = Their max_packet_size is probably actually a max_request_size? Even if = you had a huge TCP buffer, the MTU used by the network (typically 1500 = bytes) would break a MySQL "packet" into multiple TCP packets.=
Nov 02 2011
prev sibling next sibling parent Steve Teale <steve.teale britseyeview.com> writes:
On Wed, 02 Nov 2011 16:17:12 -0700, Sean Kelly wrote:

 On Nov 2, 2011, at 9:59 AM, Steve Teale wrote:
 
 On Wed, 02 Nov 2011 16:45:30 +0000, Regan Heath wrote:
 
 .. why do you want/need such a large TCPIP recv buffer?


I suspect that MySQL's "packet size" has no relation to TCP packet size. Their max_packet_size is probably actually a max_request_size? Even if you had a huge TCP buffer, the MTU used by the network (typically 1500 bytes) would break a MySQL "packet" into multiple TCP packets.

Yes Sean, I'm sure from what I've seem that that is the case. They're logical packets, and I shall stop worrying about it.
Nov 02 2011
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Steve Teale wrote:
 On Wed, 02 Nov 2011 16:17:12 -0700, Sean Kelly wrote:
 
 On Nov 2, 2011, at 9:59 AM, Steve Teale wrote:
 
 On Wed, 02 Nov 2011 16:45:30 +0000, Regan Heath wrote:
 
 .. why do you want/need such a large TCPIP recv buffer?


I suspect that MySQL's "packet size" has no relation to TCP packet size. Their max_packet_size is probably actually a max_request_size? Even if you had a huge TCP buffer, the MTU used by the network (typically 1500 bytes) would break a MySQL "packet" into multiple TCP packets.

Yes Sean, I'm sure from what I've seem that that is the case. They're logical packets, and I shall stop worrying about it.

Shouldn't we anyway write a bug report for std.socket? Yesterday I tried doing it by hand using setsockopt but even that didn't not work. Jens
Nov 03 2011
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Steve Teale wrote:
 Who is the resident expert? Sean maybe?
 
 How do I get the receive buffer size up into the megabytes range for use 
 between the D client and localhost. setOption seems to only get me about 
 262k.
 
 Steve

I found what's causing this behavior (see man 7 socket (search for SO_RCVBUF)). The maximum value is specified in /proc/sys/net/core/rmem_max. On my system it's 131071. Jens
Nov 03 2011
prev sibling parent Steve Teale <steve.teale britseyeview.com> writes:
On Thu, 03 Nov 2011 10:24:19 +0100, Jens Mueller wrote:

 I found what's causing this behavior (see man 7 socket (search for
 SO_RCVBUF)).
 The maximum value is specified in /proc/sys/net/core/rmem_max. On my
 system it's 131071.
 
 Jens

Yup, I got there too, so I've dropped the idea. Thanks for helping. Steve
Nov 03 2011