www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [GSoC Proposal draft] High-Level Networking

reply Max Klyga <max.klyga gmail.com> writes:
Google Summer of Code 2011 Proposal Draft

Abstract

The D programming language standard library (Phobos) lacks modules for 
high-level
networking using application layer protocols. This project aims to 
provide design
and implementation of networking modules for the D programming language.

Project details

Networking abilities are essential for modern programming languages.
Currently Phobos contains modules for low-level networking 
(std.socket), but lacks
modules that provide implementation of application level protocols 
(FTP, HTTP, SMPT, etc.)
The goal of this proposal is to design and implement high-level interface for
interaction through application level protocols.

I plan to implement TELNET-client, TELNET-server, HTTP-client and FTP-client
interfaces. This modules will enable D users to easily build 
applications interacting
with web services, applications controlled via TELNET, etc.

I will familiarize myself with existing networking libraries in other 
programming
languages to gather information about commonly used techniques and 
interfaces that
got widely adopted. This information will help me with initial design.
To ensure ease of use, I will discuss design of modules with D 
community, seeking
for idiomatic D interfaces.

I will be using libcurl as a foundation for implementing this modules. 
This library
is portable, supports a wide range of protocols[1]. Using libcurl will 
provide a quick
start.

Benefits for D

- Greatly simplify creation of network-enabled applications.

About me

I'm Max Klyga. I am an undergraduate software engineering student at 
Belarusian State
University of Informatics and Radioelectronics (Minsk, Belarus).
I'm mainly interested in system programming and after using C++ for 
some time I found
D and fell in love with it. I also have great interest in data-mining 
and programming
language design. I have good C++, C# and Python skills and experience 
using Qt-framework.
I've been successfully using D in my class projects and enjoyed that 
experience a lot.
Lately I've been developing ODBC wrapper for D in my spare time[2].

References

    1. http://en.wikipedia.org/wiki/CURL#libcurl
    2. https://bitbucket.org/nekuromento/d-odbc
Mar 29 2011
next sibling parent reply Jonas Drewsen <jdrewsen nospam.com> writes:
On 30/03/11 03.04, Max Klyga wrote:
 Google Summer of Code 2011 Proposal Draft

 Abstract

 The D programming language standard library (Phobos) lacks modules for
 high-level
 networking using application layer protocols. This project aims to
 provide design
 and implementation of networking modules for the D programming language.

 Project details

 Networking abilities are essential for modern programming languages.
 Currently Phobos contains modules for low-level networking (std.socket),
 but lacks
 modules that provide implementation of application level protocols (FTP,
 HTTP, SMPT, etc.)
 The goal of this proposal is to design and implement high-level
 interface for
 interaction through application level protocols.

 I plan to implement TELNET-client, TELNET-server, HTTP-client and
 FTP-client
 interfaces. This modules will enable D users to easily build
 applications interacting
 with web services, applications controlled via TELNET, etc.

 I will familiarize myself with existing networking libraries in other
 programming
 languages to gather information about commonly used techniques and
 interfaces that
 got widely adopted. This information will help me with initial design.
 To ensure ease of use, I will discuss design of modules with D
 community, seeking
 for idiomatic D interfaces.

 I will be using libcurl as a foundation for implementing this modules.
 This library
 is portable, supports a wide range of protocols[1]. Using libcurl will
 provide a quick
 start.

 Benefits for D

 - Greatly simplify creation of network-enabled applications.

 About me

 I'm Max Klyga. I am an undergraduate software engineering student at
 Belarusian State
 University of Informatics and Radioelectronics (Minsk, Belarus).
 I'm mainly interested in system programming and after using C++ for some
 time I found
 D and fell in love with it. I also have great interest in data-mining
 and programming
 language design. I have good C++, C# and Python skills and experience
 using Qt-framework.
 I've been successfully using D in my class projects and enjoyed that
 experience a lot.
 Lately I've been developing ODBC wrapper for D in my spare time[2].

 References

 1. http://en.wikipedia.org/wiki/CURL#libcurl
 2. https://bitbucket.org/nekuromento/d-odbc

Great to see that you're interested in improving the Phobos library! There already is some work being done on this. I've created bindings for libcurl that was accepted into Phobos a couple of days ago (see etc.c.curl module). This is the foundation of some network client classes that I'm doing. I intent to submit this to Phobos when I've implemented the last of the suggestions that I received from the community earlier. My initial target is FTP and HTTP support. See: https://github.com/jcd/phobos/blob/curl/etc/curl.d This does have some overlap with your proposal but that doesn't have to be a bad thing. If this proposal gets accepted you have the libcurl bindings already as a kickstart. Some other options are to extend the client classes that I'm doing. Or to implement the network server classes. /Jonas
Mar 30 2011
parent reply BlazingWhitester <max.klyga gmail.com> writes:
On 2011-03-30 17:48:59 +0300, Jonas Drewsen said:

 On 30/03/11 03.04, Max Klyga wrote:
 snip

Great to see that you're interested in improving the Phobos library! There already is some work being done on this. I've created bindings for libcurl that was accepted into Phobos a couple of days ago (see etc.c.curl module). This is the foundation of some network client classes that I'm doing. I intent to submit this to Phobos when I've implemented the last of the suggestions that I received from the community earlier.

Could you give links to that suggestions? Is your work on network-clients avaliable online?
 My initial target is FTP and HTTP
 support.
 
 See:
 https://github.com/jcd/phobos/blob/curl/etc/curl.d
 
 This does have some overlap with your proposal but that doesn't have to 
 be a bad thing.
 
 If this proposal gets accepted you have the libcurl bindings already as 
 a kickstart.
 
 Some other options are to extend the client classes that I'm doing. Or 
 to implement the network server classes.

RFC compilant HTTP-server is very hard to implement, so one solution might be to implement different protocol clients (SMTP, ...) so our work will not overlap.
 
 /Jonas

Mar 30 2011
parent reply Jonas Drewsen <jdrewsen nospam.com> writes:
On 30/03/11 19.19, BlazingWhitester wrote:
 On 2011-03-30 17:48:59 +0300, Jonas Drewsen said:

 On 30/03/11 03.04, Max Klyga wrote:
 snip

Great to see that you're interested in improving the Phobos library! There already is some work being done on this. I've created bindings for libcurl that was accepted into Phobos a couple of days ago (see etc.c.curl module). This is the foundation of some network client classes that I'm doing. I intent to submit this to Phobos when I've implemented the last of the suggestions that I received from the community earlier.

Could you give links to that suggestions?

Look for threads in this newsgroup that I started and that have curl or libcurl in the title.
 Is your work on network-clients avaliable online?

Yes. I haven't pushed my working copy for some time but you can see an older version in the link I provided (see below).
 My initial target is FTP and HTTP
 support.

 See:
 https://github.com/jcd/phobos/blob/curl/etc/curl.d

 This does have some overlap with your proposal but that doesn't have
 to be a bad thing.

 If this proposal gets accepted you have the libcurl bindings already
 as a kickstart.

 Some other options are to extend the client classes that I'm doing. Or
 to implement the network server classes.

RFC compilant HTTP-server is very hard to implement, so one solution might be to implement different protocol clients (SMTP, ...) so our work will not overlap.

Actually I believe that a RFC compliant HTTP-server implementation with only mandatory features is not that hard to do. That would provide a good starting point for future improvements. But my initial thoughts on server classes was not HTTP specific though. It was more like a general server network framework. This framework could in turn be used to create a HTTP server if somebody wanted that.
 /Jonas


Mar 30 2011
parent reply Max Klyga <max.klyga gmail.com> writes:
I read your "Curl RFC" thread, great job. Somehow I missed it when it 
was originaly posted.

As you stated earlier, you plan to continue working on your curl 
wrapper, so to boost my proposal usefullness I think it needs to 
address unsolved problems.

Could you please be more specific about your plans on networking 
modules? This information might help me make a stronger proposal.

Thank you.

Things that phobos currently lacks, but would benefit from:
 - Tcp Server - a class implementing TCP-based server, handling 
incoming connections. (QTcpServer[1] ot Net::TcpServer[2] might act as 
a good reference)
 - Application level protocol servers.

References
 1. http://doc.qt.nokia.com/latest/qtcpserver.html#details
 2. http://www.ruby-doc.org/stdlib/libdoc/socket/rdoc/classes/TCPServer.html
Mar 30 2011
parent Jonas Drewsen <jdrewsen nospam.com> writes:
On 30/03/11 23.58, Max Klyga wrote:
 I read your "Curl RFC" thread, great job. Somehow I missed it when it
 was originaly posted.

 As you stated earlier, you plan to continue working on your curl
 wrapper, so to boost my proposal usefullness I think it needs to address
 unsolved problems.

 Could you please be more specific about your plans on networking
 modules? This information might help me make a stronger proposal.

I written about some of this in the other thread you've started "Asynchronicity in D".
 Thank you.

 Things that phobos currently lacks, but would benefit from:
 - Tcp Server - a class implementing TCP-based server, handling incoming
 connections. (QTcpServer[1] ot Net::TcpServer[2] might act as a good
 reference)

library.
 - Application level protocol servers.

in general would be very nice. /Jonas
 References
 1. http://doc.qt.nokia.com/latest/qtcpserver.html#details
 2. http://www.ruby-doc.org/stdlib/libdoc/socket/rdoc/classes/TCPServer.html

Mar 31 2011
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On 03/30/2011 03:04 AM, Max Klyga wrote:
 Google Summer of Code 2011 Proposal Draft

 Abstract

 The D programming language standard library (Phobos) lacks modules for
high-level
 networking using application layer protocols. This project aims to provide
design
 and implementation of networking modules for the D programming language.

 Project details

 Networking abilities are essential for modern programming languages.
 Currently Phobos contains modules for low-level networking (std.socket), but
lacks
 modules that provide implementation of application level protocols (FTP, HTTP,
 SMPT, etc.)
 The goal of this proposal is to design and implement high-level interface for
 interaction through application level protocols.

 I plan to implement TELNET-client, TELNET-server, HTTP-client and FTP-client
 interfaces. This modules will enable D users to easily build applications
 interacting
 with web services, applications controlled via TELNET, etc.

 I will familiarize myself with existing networking libraries in other
programming
 languages to gather information about commonly used techniques and interfaces
that
 got widely adopted. This information will help me with initial design.
 To ensure ease of use, I will discuss design of modules with D community,
seeking
 for idiomatic D interfaces.

 I will be using libcurl as a foundation for implementing this modules. This
 library
 is portable, supports a wide range of protocols[1]. Using libcurl will provide
 a quick
 start.

 Benefits for D

 - Greatly simplify creation of network-enabled applications.

Good news, I hope this project will be accepted. Anyone to mentor it? Denis -- _________________ vita es estrany spir.wikidot.com
Mar 30 2011
prev sibling next sibling parent reply Max Klyga <max.klyga gmail.com> writes:
Since my proposal theme shifted a little towards asyncronicity, I've 
been eagerly consuming information that might help me with this.

I would be glad to know if I am moving in to the right direction. 
Comments are welcome.

So far, here is the status report:
I looked at approaches to asyncronicity in Ruby's NeverBlock, C#, Qt, 
Boost.ASIO

NeverBlock uses Fibers for managing asyncronous operations. One fiber 
acts as a scheduler, polling async operations for complition and 
resuming other fibers as needed.

C# uses several approaches to asyn operations. There are three common 
patterns now:
 - Split async functon in two (BeginFoo, EndFoo). Begin... function 
returns IAsyncResult object and shedules operation in process-default 
thread pool. End... function takes IAsyncResult object that we got 
earlier and, if operation was complited, returns computation results or 
blocks until the operation is completed. All exceptions thrown in 
asyncronous operation are stored and rethrown when user calls End... 
function.
  Any function can be called asyncronously using built in wrapper.
 - Some classes provide the following interface for time consuming operations:
    class Foo {
        bar(...); // blocking operation
        asyncBar(...); // asyncronous operation

        cancelAsync();

        barCompleted;

        barResult;
    }
 - Task-based asyncronicity, similar to std.parallelism. Async 
operations (BeginFoo, EndFoo) can be wrapped as a task and scheduled to 
be executed on user controlled task pools.

Qt uses it's signal/slot mechanism for asyncronous operations. It's 
somewhat similar to C# events, or arrays of D delegates.

Boost.ASIO is based on callbacks/delegates. It's implementation is 
based on proactor design pattern and doesn't use Fibers or Threads.


I've also spent some time digging up info on event-based servers.
This is my to-read list on this weekend:
	[1] Ous96 John Ousterhout. Why threads are a bad idea (for most 
purposes). In USENIX Technical Conference (Invited Talk), Austin, TX, 
January 1996.
	[2] Rob von Behren, Jeremy Condit, and Eric Brewer. 2003. Why events 
are a bad idea (for high-concurrency servers). In Proceedings of the 
9th conference on Hot Topics in Operating Systems - Volume 9 
(HOTOS'03), Vol. 9. USENIX Association, Berkeley, CA, USA, 4-4.
	[3] Atul Adya , Jon Howell , Marvin Theimer , William J. Bolosky , 
John R. Douceur, Cooperative Task Management Without Manual Stack 
Management, Proceedings of the General Track: 2002 USENIX Annual 
Technical Conference, p.289-302, June 10-15, 2002 (related 
presentation: http://www.cs.nyu.edu/rgrimm/teaching/fa03-web/091603.pdf)
	[4] http://www.kegel.com/c10k.html
	[5] http://tomasp.net/blog/csharp-fsharp-async-intro.aspx (I should 
look at how F# handles asyncronicity)
Apr 01 2011
parent Jose Armando Garcia <jsancio gmail.com> writes:
Mind you that Async IO is different from Non-blocking IO (which is
what most literature talks about). I am not an expert with Async IO
but it has serious limitations in Windows like the kernel thread that
starts the IO must complete it. Not very familiar with the state of
AIO in Linux.

Good luck.

On Fri, Apr 1, 2011 at 10:51 PM, Jose Armando Garcia <jsancio gmail.com> wr=
ote:
 That all sounds good. I would add Flash to your investigation:
 http://www.cs.princeton.edu/~vivek/flash/. There is a must read
 research paper for any distributed system person at:
 http://www.cs.princeton.edu/~vivek/flash_usenix_99/

 On Fri, Apr 1, 2011 at 8:17 PM, Max Klyga <max.klyga gmail.com> wrote:
 Since my proposal theme shifted a little towards asyncronicity, I've bee=


 eagerly consuming information that might help me with this.

 I would be glad to know if I am moving in to the right direction. Commen=


 are welcome.

 So far, here is the status report:
 I looked at approaches to asyncronicity in Ruby's NeverBlock, C#, Qt,
 Boost.ASIO

 NeverBlock uses Fibers for managing asyncronous operations. One fiber ac=


 as a scheduler, polling async operations for complition and resuming oth=


 fibers as needed.

 C# uses several approaches to asyn operations. There are three common
 patterns now:
 - Split async functon in two (BeginFoo, EndFoo). Begin... function retur=


 IAsyncResult object and shedules operation in process-default thread poo=


 End... function takes IAsyncResult object that we got earlier and, if
 operation was complited, returns computation results or blocks until the
 operation is completed. All exceptions thrown in asyncronous operation a=


 stored and rethrown when user calls End... function.
 =A0Any function can be called asyncronously using built in wrapper.
 - Some classes provide the following interface for time consuming
 operations:
 =A0 class Foo {
 =A0 =A0 =A0 bar(...); // blocking operation
 =A0 =A0 =A0 asyncBar(...); // asyncronous operation

 =A0 =A0 =A0 cancelAsync();

 =A0 =A0 =A0 barCompleted;

 =A0 =A0 =A0 barResult;
 =A0 }
 - Task-based asyncronicity, similar to std.parallelism. Async operations
 (BeginFoo, EndFoo) can be wrapped as a task and scheduled to be executed=


 user controlled task pools.

 Qt uses it's signal/slot mechanism for asyncronous operations. It's some=


 similar to C# events, or arrays of D delegates.

 Boost.ASIO is based on callbacks/delegates. It's implementation is based=


 proactor design pattern and doesn't use Fibers or Threads.


 I've also spent some time digging up info on event-based servers.
 This is my to-read list on this weekend:
 =A0 =A0 =A0 =A0[1] Ous96 John Ousterhout. Why threads are a bad idea (fo=


 purposes). In USENIX Technical Conference (Invited Talk), Austin, TX,
 January 1996.
 =A0 =A0 =A0 =A0[2] Rob von Behren, Jeremy Condit, and Eric Brewer. 2003.=


 are a bad idea (for high-concurrency servers). In Proceedings of the 9th
 conference on Hot Topics in Operating Systems - Volume 9 (HOTOS'03), Vol=


 USENIX Association, Berkeley, CA, USA, 4-4.
 =A0 =A0 =A0 =A0[3] Atul Adya , Jon Howell , Marvin Theimer , William J. =


 John R. Douceur, Cooperative Task Management Without Manual Stack
 Management, Proceedings of the General Track: 2002 USENIX Annual Technic=


 Conference, p.289-302, June 10-15, 2002 (related presentation:
 http://www.cs.nyu.edu/rgrimm/teaching/fa03-web/091603.pdf)
 =A0 =A0 =A0 =A0[4] http://www.kegel.com/c10k.html
 =A0 =A0 =A0 =A0[5] http://tomasp.net/blog/csharp-fsharp-async-intro.aspx=


 look at how F# handles asyncronicity)


Apr 01 2011
prev sibling next sibling parent Jose Armando Garcia <jsancio gmail.com> writes:
That all sounds good. I would add Flash to your investigation:
http://www.cs.princeton.edu/~vivek/flash/. There is a must read
research paper for any distributed system person at:
http://www.cs.princeton.edu/~vivek/flash_usenix_99/

On Fri, Apr 1, 2011 at 8:17 PM, Max Klyga <max.klyga gmail.com> wrote:
 Since my proposal theme shifted a little towards asyncronicity, I've been
 eagerly consuming information that might help me with this.

 I would be glad to know if I am moving in to the right direction. Comment=

 are welcome.

 So far, here is the status report:
 I looked at approaches to asyncronicity in Ruby's NeverBlock, C#, Qt,
 Boost.ASIO

 NeverBlock uses Fibers for managing asyncronous operations. One fiber act=

 as a scheduler, polling async operations for complition and resuming othe=

 fibers as needed.

 C# uses several approaches to asyn operations. There are three common
 patterns now:
 - Split async functon in two (BeginFoo, EndFoo). Begin... function return=

 IAsyncResult object and shedules operation in process-default thread pool=

 End... function takes IAsyncResult object that we got earlier and, if
 operation was complited, returns computation results or blocks until the
 operation is completed. All exceptions thrown in asyncronous operation ar=

 stored and rethrown when user calls End... function.
 =A0Any function can be called asyncronously using built in wrapper.
 - Some classes provide the following interface for time consuming
 operations:
 =A0 class Foo {
 =A0 =A0 =A0 bar(...); // blocking operation
 =A0 =A0 =A0 asyncBar(...); // asyncronous operation

 =A0 =A0 =A0 cancelAsync();

 =A0 =A0 =A0 barCompleted;

 =A0 =A0 =A0 barResult;
 =A0 }
 - Task-based asyncronicity, similar to std.parallelism. Async operations
 (BeginFoo, EndFoo) can be wrapped as a task and scheduled to be executed =

 user controlled task pools.

 Qt uses it's signal/slot mechanism for asyncronous operations. It's somew=

 similar to C# events, or arrays of D delegates.

 Boost.ASIO is based on callbacks/delegates. It's implementation is based =

 proactor design pattern and doesn't use Fibers or Threads.


 I've also spent some time digging up info on event-based servers.
 This is my to-read list on this weekend:
 =A0 =A0 =A0 =A0[1] Ous96 John Ousterhout. Why threads are a bad idea (for=

 purposes). In USENIX Technical Conference (Invited Talk), Austin, TX,
 January 1996.
 =A0 =A0 =A0 =A0[2] Rob von Behren, Jeremy Condit, and Eric Brewer. 2003. =

 are a bad idea (for high-concurrency servers). In Proceedings of the 9th
 conference on Hot Topics in Operating Systems - Volume 9 (HOTOS'03), Vol.=

 USENIX Association, Berkeley, CA, USA, 4-4.
 =A0 =A0 =A0 =A0[3] Atul Adya , Jon Howell , Marvin Theimer , William J. B=

 John R. Douceur, Cooperative Task Management Without Manual Stack
 Management, Proceedings of the General Track: 2002 USENIX Annual Technica=

 Conference, p.289-302, June 10-15, 2002 (related presentation:
 http://www.cs.nyu.edu/rgrimm/teaching/fa03-web/091603.pdf)
 =A0 =A0 =A0 =A0[4] http://www.kegel.com/c10k.html
 =A0 =A0 =A0 =A0[5] http://tomasp.net/blog/csharp-fsharp-async-intro.aspx =

 look at how F# handles asyncronicity)

Apr 01 2011
prev sibling next sibling parent reply Max Klyga <max.klyga gmail.com> writes:
Thanks to everyone who provided feedback!

I'll post my proposal update tomorrow.
Apr 04 2011
next sibling parent Jonas Drewsen <jdrewsen nospam.com> writes:
On 04/04/11 22.06, Max Klyga wrote:
 Thanks to everyone who provided feedback!

 I'll post my proposal update tomorrow.

If possible it would be nice if the common types of async methods could be supported since each have their own forces: co-routines (fibers,scheduler), Futures (e.g. IAsyncResult) and events (select/epoll/kqueue). BTW - Personally I wouldn't like a dependency on an existing C library (e.g. libev) if this work is to go into the 'std' package of Phobos. But this is just my opinion and not anything officially stated. Best of luck Jonas
Apr 05 2011
prev sibling parent Max Klyga <max.klyga gmail.com> writes:
On 2011-04-06 02:30:44 +0300, Torarin said:

 2011/4/4 Max Klyga <max.klyga gmail.com>:
 Thanks to everyone who provided feedback!
 
 I'll post my proposal update tomorrow.
 
 

I'm looking forward to it too! I've done a fair amount of research on these topics and given networking in D a lot of thought. I'd prefer the standard library's socket interface to be more low level than it is today, and to have a socket type that provides non-blocking IO transparently through fibers. This enables both generic protocol implementations that can handle several types of concurrency, and efficient parsers using buffered input ranges. I also think the library should offer it's own event loop instead of linking to libev or libevent, which have their weaknesses in such a context. For instance, you can't simply hand over garbage collected objects to them, which is unfortunate given the usefulness of closures for handling events. I also think a lower-level API than Boost ASIO is wise, because one may need to use novel and platform-specific optimizations like linux splice, a must for efficient file servers. As I mentioned, I have started implementing some of this. Would you be interested in pairing up with me on some parts of the project? Best regards, Torarin

I'll be glad to cooperate if this isn't explicitly forbidden by GSoC rules (which is highly implausible). I too think that current socket interface could be improved.
Apr 06 2011
prev sibling next sibling parent Torarin <torarind gmail.com> writes:
2011/4/4 Max Klyga <max.klyga gmail.com>:
 Thanks to everyone who provided feedback!

 I'll post my proposal update tomorrow.

I'm looking forward to it too! I've done a fair amount of research on these topics and given networking in D a lot of thought. I'd prefer the standard library's socket interface to be more low level than it is today, and to have a socket type that provides non-blocking IO transparently through fibers. This enables both generic protocol implementations that can handle several types of concurrency, and efficient parsers using buffered input ranges. I also think the library should offer it's own event loop instead of linking to libev or libevent, which have their weaknesses in such a context. For instance, you can't simply hand over garbage collected objects to them, which is unfortunate given the usefulness of closures for handling events. I also think a lower-level API than Boost ASIO is wise, because one may need to use novel and platform-specific optimizations like linux splice, a must for efficient file servers. As I mentioned, I have started implementing some of this. Would you be interested in pairing up with me on some parts of the project? Best regards, Torarin
Apr 05 2011
prev sibling parent reply Max Klyga <max.klyga gmail.com> writes:
OK, I've absorbed enough informtion to transform my thoughts in the 
second proposal draft.
I'll be posting it direcltly to GSoC website and correct it, if necessary.

Proposal is situated here: 
http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/neku42/1

Comments 

are welcome.
Apr 06 2011
next sibling parent reply Jonas Drewsen <jdrewsen nospam.com> writes:
On 07/04/11 00.32, Max Klyga wrote:
 OK, I've absorbed enough informtion to transform my thoughts in the
 second proposal draft.
 I'll be posting it direcltly to GSoC website and correct it, if necessary.

 Proposal is situated here:
 http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/neku42/1

 Comments
 are welcome.

Seems good to me. Some comments: You mention that a reactor will probably be used for implementing the proactor. On windows it really doesn't make sense to do this it think. Instead overlapped IO should be used since it maps very well with the proactor pattern. I also suggest that you specify which OSes you will make support for. If that is only e.g. linux then building on reactors is the right way to go as you've planned. /Jonas
Apr 07 2011
parent Max Klyga <max.klyga gmail.com> writes:
On 2011-04-07 16:17:43 +0300, Torarin said:

 2011/4/7 Jonas Drewsen <jdrewsen nospam.com>:
 On 07/04/11 00.32, Max Klyga wrote:
 snip

Seems good to me. Some comments: You mention that a reactor will probably be used for implementing the proactor. On windows it really doesn't make sense to do this it think. Instead overlapped IO should be used since it maps very well with the proactor pattern. I also suggest that you specify which OSes you will make support for. If that is only e.g. linux then building on reactors is the right way to go as you've planned. /Jonas

Indeed it looks good! You can use the reactor pattern on Windows as well, but several sources say that overlapped IO is more efficient, and it probably is cleaner. But it's still nice to have a reactor that wraps select and friends, and then specialize the individual proactors as needed. Was that what you were thinking, Max? Torarin

I plan to start with a Mac implementation, because that's what I use most of the time, but I have a Windows box too and can install Linux on virtual machine, if that will be needed. Yes, I had an idea to implement reactor and proactor the way, user may swap implementations, e.g. portable reactor based on select and platform specific one using efficient implementation for that platform. This approach is used in Twisted python web-server (It's licence states that it needs attribution only on reuse of descent amounts of code, so it might be a good idea to peek for some platform specific details). Windows overlapped I/O allows to implement proactor on top of it without any additional layers, but reactor can be implemented on of overlapped I/O too.
Apr 07 2011
prev sibling parent Torarin <torarind gmail.com> writes:
2011/4/7 Jonas Drewsen <jdrewsen nospam.com>:
 On 07/04/11 00.32, Max Klyga wrote:
 OK, I've absorbed enough informtion to transform my thoughts in the
 second proposal draft.
 I'll be posting it direcltly to GSoC website and correct it, if necessary.

 Proposal is situated here:

 http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/neku42/1

 Comments
 are welcome.

Seems good to me. Some comments: You mention that a reactor will probably be used for implementing the proactor. On windows it really doesn't make sense to do this it think. Instead overlapped IO should be used since it maps very well with the proactor pattern. I also suggest that you specify which OSes you will make support for. If that is only e.g. linux then building on reactors is the right way to go as you've planned. /Jonas

Indeed it looks good! You can use the reactor pattern on Windows as well, but several sources say that overlapped IO is more efficient, and it probably is cleaner. But it's still nice to have a reactor that wraps select and friends, and then specialize the individual proactors as needed. Was that what you were thinking, Max? Torarin
Apr 07 2011