www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [GSoC Proposal draft] High-Level Networking

reply Max Klyga <max.klyga gmail.com> writes:
Google Summer of Code 2011 Proposal Draft

Abstract

The D programming language standard library (Phobos) lacks modules for 
high-level
networking using application layer protocols. This project aims to 
provide design
and implementation of networking modules for the D programming language.

Project details

Networking abilities are essential for modern programming languages.
Currently Phobos contains modules for low-level networking 
(std.socket), but lacks
modules that provide implementation of application level protocols 
(FTP, HTTP, SMPT, etc.)
The goal of this proposal is to design and implement high-level interface for
interaction through application level protocols.

I plan to implement TELNET-client, TELNET-server, HTTP-client and FTP-client
interfaces. This modules will enable D users to easily build 
applications interacting
with web services, applications controlled via TELNET, etc.

I will familiarize myself with existing networking libraries in other 
programming
languages to gather information about commonly used techniques and 
interfaces that
got widely adopted. This information will help me with initial design.
To ensure ease of use, I will discuss design of modules with D 
community, seeking
for idiomatic D interfaces.

I will be using libcurl as a foundation for implementing this modules. 
This library
is portable, supports a wide range of protocols[1]. Using libcurl will 
provide a quick
start.

Benefits for D

- Greatly simplify creation of network-enabled applications.

About me

I'm Max Klyga. I am an undergraduate software engineering student at 
Belarusian State
University of Informatics and Radioelectronics (Minsk, Belarus).
I'm mainly interested in system programming and after using C++ for 
some time I found
D and fell in love with it. I also have great interest in data-mining 
and programming

using Qt-framework.
I've been successfully using D in my class projects and enjoyed that 
experience a lot.
Lately I've been developing ODBC wrapper for D in my spare time[2].

References

    1. http://en.wikipedia.org/wiki/CURL#libcurl
    2. https://bitbucket.org/nekuromento/d-odbc
Mar 29 2011
next sibling parent reply Jonas Drewsen <jdrewsen nospam.com> writes:
On 30/03/11 03.04, Max Klyga wrote:

 Google Summer of Code 2011 Proposal Draft

 Abstract

 The D programming language standard library (Phobos) lacks modules for
 high-level
 networking using application layer protocols. This project aims to
 provide design
 and implementation of networking modules for the D programming language.

 Project details

 Networking abilities are essential for modern programming languages.
 Currently Phobos contains modules for low-level networking (std.socket),
 but lacks
 modules that provide implementation of application level protocols (FTP,
 HTTP, SMPT, etc.)
 The goal of this proposal is to design and implement high-level
 interface for
 interaction through application level protocols.

 I plan to implement TELNET-client, TELNET-server, HTTP-client and
 FTP-client
 interfaces. This modules will enable D users to easily build
 applications interacting
 with web services, applications controlled via TELNET, etc.

 I will familiarize myself with existing networking libraries in other
 programming
 languages to gather information about commonly used techniques and
 interfaces that
 got widely adopted. This information will help me with initial design.
 To ensure ease of use, I will discuss design of modules with D
 community, seeking
 for idiomatic D interfaces.

 I will be using libcurl as a foundation for implementing this modules.
 This library
 is portable, supports a wide range of protocols[1]. Using libcurl will
 provide a quick
 start.

 Benefits for D

 - Greatly simplify creation of network-enabled applications.

 About me

 I'm Max Klyga. I am an undergraduate software engineering student at
 Belarusian State
 University of Informatics and Radioelectronics (Minsk, Belarus).
 I'm mainly interested in system programming and after using C++ for some
 time I found
 D and fell in love with it. I also have great interest in data-mining
 and programming

 using Qt-framework.
 I've been successfully using D in my class projects and enjoyed that
 experience a lot.
 Lately I've been developing ODBC wrapper for D in my spare time[2].

 References

 1. http://en.wikipedia.org/wiki/CURL#libcurl
 2. https://bitbucket.org/nekuromento/d-odbc


Great to see that you're interested in improving the Phobos library!

There already is some work being done on this.

I've created bindings for libcurl that was accepted into Phobos a couple 
of days ago (see etc.c.curl module). This is the foundation of some 
network client classes that I'm doing. I intent to submit this to Phobos 
when I've implemented the last of the suggestions that I received from 
the community earlier. My initial target is FTP and HTTP
support.

See:
https://github.com/jcd/phobos/blob/curl/etc/curl.d

This does have some overlap with your proposal but that doesn't have to 
be a bad thing.

If this proposal gets accepted you have the libcurl bindings already as 
a kickstart.

Some other options are to extend the client classes that I'm doing. Or 
to implement the network server classes.

/Jonas
Mar 30 2011
parent reply BlazingWhitester <max.klyga gmail.com> writes:
On 2011-03-30 17:48:59 +0300, Jonas Drewsen said:


 On 30/03/11 03.04, Max Klyga wrote:

 snip


 
 Great to see that you're interested in improving the Phobos library!
 
 There already is some work being done on this.
 
 I've created bindings for libcurl that was accepted into Phobos a 
 couple of days ago (see etc.c.curl module). This is the foundation of 
 some network client classes that I'm doing. I intent to submit this to 
 Phobos when I've implemented the last of the suggestions that I 
 received from the community earlier.


Could you give links to that suggestions?
Is your work on network-clients avaliable online?

 My initial target is FTP and HTTP
 support.
 
 See:
 https://github.com/jcd/phobos/blob/curl/etc/curl.d
 
 This does have some overlap with your proposal but that doesn't have to 
 be a bad thing.
 
 If this proposal gets accepted you have the libcurl bindings already as 
 a kickstart.
 
 Some other options are to extend the client classes that I'm doing. Or 
 to implement the network server classes.


RFC compilant HTTP-server is very hard to implement, so one solution 
might be to implement different protocol clients (SMTP, ...) so our 
work will not overlap.

 
 /Jonas
Mar 30 2011
parent reply Jonas Drewsen <jdrewsen nospam.com> writes:
On 30/03/11 19.19, BlazingWhitester wrote:

 On 2011-03-30 17:48:59 +0300, Jonas Drewsen said:


 On 30/03/11 03.04, Max Klyga wrote:

 snip


 Great to see that you're interested in improving the Phobos library!

 There already is some work being done on this.

 I've created bindings for libcurl that was accepted into Phobos a
 couple of days ago (see etc.c.curl module). This is the foundation of
 some network client classes that I'm doing. I intent to submit this to
 Phobos when I've implemented the last of the suggestions that I
 received from the community earlier.


 Could you give links to that suggestions?


Look for threads in this newsgroup that I started and that have curl or 
libcurl in the title.


 Is your work on network-clients avaliable online?


Yes. I haven't pushed my working copy for some time but you can see an 
older version in the link I provided (see below).


 My initial target is FTP and HTTP
 support.

 See:
 https://github.com/jcd/phobos/blob/curl/etc/curl.d

 This does have some overlap with your proposal but that doesn't have
 to be a bad thing.

 If this proposal gets accepted you have the libcurl bindings already
 as a kickstart.

 Some other options are to extend the client classes that I'm doing. Or
 to implement the network server classes.


 RFC compilant HTTP-server is very hard to implement, so one solution
 might be to implement different protocol clients (SMTP, ...) so our work
 will not overlap.


Actually I believe that a RFC compliant HTTP-server implementation with 
only mandatory features is not that hard to do. That would provide a 
good starting point for future improvements.

But my initial thoughts on server classes was not HTTP specific though. 
It was more like a general server network framework. This framework 
could in turn be used to create a HTTP server if somebody wanted that.


 /Jonas
Mar 30 2011
parent reply Max Klyga <max.klyga gmail.com> writes:
I read your "Curl RFC" thread, great job. Somehow I missed it when it 
was originaly posted.

As you stated earlier, you plan to continue working on your curl 
wrapper, so to boost my proposal usefullness I think it needs to 
address unsolved problems.

Could you please be more specific about your plans on networking 
modules? This information might help me make a stronger proposal.

Thank you.

Things that phobos currently lacks, but would benefit from:
 - Tcp Server - a class implementing TCP-based server, handling 
incoming connections. (QTcpServer[1] ot Net::TcpServer[2] might act as 
a good reference)
 - Application level protocol servers.

References
 1. http://doc.qt.nokia.com/latest/qtcpserver.html#details
 2. http://www.ruby-doc.org/stdlib/libdoc/socket/rdoc/classes/TCPServer.html
Mar 30 2011
parent Jonas Drewsen <jdrewsen nospam.com> writes:
On 30/03/11 23.58, Max Klyga wrote:

 I read your "Curl RFC" thread, great job. Somehow I missed it when it
 was originaly posted.

 As you stated earlier, you plan to continue working on your curl
 wrapper, so to boost my proposal usefullness I think it needs to address
 unsolved problems.

 Could you please be more specific about your plans on networking
 modules? This information might help me make a stronger proposal.


I written about some of this in the other thread you've started 
"Asynchronicity in D".


 Thank you.

 Things that phobos currently lacks, but would benefit from:
 - Tcp Server - a class implementing TCP-based server, handling incoming
 connections. (QTcpServer[1] ot Net::TcpServer[2] might act as a good
 reference)



library.


 - Application level protocol servers.


Especially HTTP but I guess you knew that. A framework for such servers 
in general would be very nice.

/Jonas


 References
 1. http://doc.qt.nokia.com/latest/qtcpserver.html#details
 2. http://www.ruby-doc.org/stdlib/libdoc/socket/rdoc/classes/TCPServer.html
Mar 31 2011
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On 03/30/2011 03:04 AM, Max Klyga wrote:

 Google Summer of Code 2011 Proposal Draft

 Abstract

 The D programming language standard library (Phobos) lacks modules for
high-level
 networking using application layer protocols. This project aims to provide
design
 and implementation of networking modules for the D programming language.

 Project details

 Networking abilities are essential for modern programming languages.
 Currently Phobos contains modules for low-level networking (std.socket), but
lacks
 modules that provide implementation of application level protocols (FTP, HTTP,
 SMPT, etc.)
 The goal of this proposal is to design and implement high-level interface for
 interaction through application level protocols.

 I plan to implement TELNET-client, TELNET-server, HTTP-client and FTP-client
 interfaces. This modules will enable D users to easily build applications
 interacting
 with web services, applications controlled via TELNET, etc.

 I will familiarize myself with existing networking libraries in other
programming
 languages to gather information about commonly used techniques and interfaces
that
 got widely adopted. This information will help me with initial design.
 To ensure ease of use, I will discuss design of modules with D community,
seeking
 for idiomatic D interfaces.

 I will be using libcurl as a foundation for implementing this modules. This
 library
 is portable, supports a wide range of protocols[1]. Using libcurl will provide
 a quick
 start.

 Benefits for D

 - Greatly simplify creation of network-enabled applications.


Good news, I hope this project will be accepted. Anyone to mentor it?

Denis
-- 
_________________
vita es estrany
spir.wikidot.com
Mar 30 2011
prev sibling next sibling parent reply Max Klyga <max.klyga gmail.com> writes:
Since my proposal theme shifted a little towards asyncronicity, I've 
been eagerly consuming information that might help me with this.

I would be glad to know if I am moving in to the right direction. 
Comments are welcome.

So far, here is the status report:

Boost.ASIO

NeverBlock uses Fibers for managing asyncronous operations. One fiber 
acts as a scheduler, polling async operations for complition and 
resuming other fibers as needed.


patterns now:
 - Split async functon in two (BeginFoo, EndFoo). Begin... function 
returns IAsyncResult object and shedules operation in process-default 
thread pool. End... function takes IAsyncResult object that we got 
earlier and, if operation was complited, returns computation results or 
blocks until the operation is completed. All exceptions thrown in 
asyncronous operation are stored and rethrown when user calls End... 
function.
  Any function can be called asyncronously using built in wrapper.
 - Some classes provide the following interface for time consuming operations:
    class Foo {
        bar(...); // blocking operation
        asyncBar(...); // asyncronous operation

        cancelAsync();

        barCompleted;

        barResult;
    }
 - Task-based asyncronicity, similar to std.parallelism. Async 
operations (BeginFoo, EndFoo) can be wrapped as a task and scheduled to 
be executed on user controlled task pools.

Qt uses it's signal/slot mechanism for asyncronous operations. It's 


Boost.ASIO is based on callbacks/delegates. It's implementation is 
based on proactor design pattern and doesn't use Fibers or Threads.


I've also spent some time digging up info on event-based servers.
This is my to-read list on this weekend:
	[1] Ous96 John Ousterhout. Why threads are a bad idea (for most 
purposes). In USENIX Technical Conference (Invited Talk), Austin, TX, 
January 1996.
	[2] Rob von Behren, Jeremy Condit, and Eric Brewer. 2003. Why events 
are a bad idea (for high-concurrency servers). In Proceedings of the 
9th conference on Hot Topics in Operating Systems - Volume 9 
(HOTOS'03), Vol. 9. USENIX Association, Berkeley, CA, USA, 4-4.
	[3] Atul Adya , Jon Howell , Marvin Theimer , William J. Bolosky , 
John R. Douceur, Cooperative Task Management Without Manual Stack 
Management, Proceedings of the General Track: 2002 USENIX Annual 
Technical Conference, p.289-302, June 10-15, 2002 (related 
presentation: http://www.cs.nyu.edu/rgrimm/teaching/fa03-web/091603.pdf)
	[4] http://www.kegel.com/c10k.html
	[5] http://tomasp.net/blog/csharp-fsharp-async-intro.aspx (I should
Apr 01 2011
next sibling parent Jose Armando Garcia <jsancio gmail.com> writes:
Mind you that Async IO is different from Non-blocking IO (which is
what most literature talks about). I am not an expert with Async IO
but it has serious limitations in Windows like the kernel thread that
starts the IO must complete it. Not very familiar with the state of
AIO in Linux.

Good luck.

On Fri, Apr 1, 2011 at 10:51 PM, Jose Armando Garcia <jsancio gmail.com> wr=
ote:

 That all sounds good. I would add Flash to your investigation:
 http://www.cs.princeton.edu/~vivek/flash/. There is a must read
 research paper for any distributed system person at:
 http://www.cs.princeton.edu/~vivek/flash_usenix_99/

 On Fri, Apr 1, 2011 at 8:17 PM, Max Klyga <max.klyga gmail.com> wrote:

 Since my proposal theme shifted a little towards asyncronicity, I've bee=




n

 eagerly consuming information that might help me with this.

 I would be glad to know if I am moving in to the right direction. Commen=




ts

 are welcome.

 So far, here is the status report:

 Boost.ASIO

 NeverBlock uses Fibers for managing asyncronous operations. One fiber ac=




ts

 as a scheduler, polling async operations for complition and resuming oth=




er

 fibers as needed.


 patterns now:
 - Split async functon in two (BeginFoo, EndFoo). Begin... function retur=




ns

 IAsyncResult object and shedules operation in process-default thread poo=




l.

 End... function takes IAsyncResult object that we got earlier and, if
 operation was complited, returns computation results or blocks until the
 operation is completed. All exceptions thrown in asyncronous operation a=




re

 stored and rethrown when user calls End... function.
 =A0Any function can be called asyncronously using built in wrapper.
 - Some classes provide the following interface for time consuming
 operations:
 =A0 class Foo {
 =A0 =A0 =A0 bar(...); // blocking operation
 =A0 =A0 =A0 asyncBar(...); // asyncronous operation

 =A0 =A0 =A0 cancelAsync();

 =A0 =A0 =A0 barCompleted;

 =A0 =A0 =A0 barResult;
 =A0 }
 - Task-based asyncronicity, similar to std.parallelism. Async operations
 (BeginFoo, EndFoo) can be wrapped as a task and scheduled to be executed=




 on

 user controlled task pools.

 Qt uses it's signal/slot mechanism for asyncronous operations. It's some=




what



 Boost.ASIO is based on callbacks/delegates. It's implementation is based=




 on

 proactor design pattern and doesn't use Fibers or Threads.


 I've also spent some time digging up info on event-based servers.
 This is my to-read list on this weekend:
 =A0 =A0 =A0 =A0[1] Ous96 John Ousterhout. Why threads are a bad idea (fo=




r most

 purposes). In USENIX Technical Conference (Invited Talk), Austin, TX,
 January 1996.
 =A0 =A0 =A0 =A0[2] Rob von Behren, Jeremy Condit, and Eric Brewer. 2003.=




 Why events

 are a bad idea (for high-concurrency servers). In Proceedings of the 9th
 conference on Hot Topics in Operating Systems - Volume 9 (HOTOS'03), Vol=




. 9.

 USENIX Association, Berkeley, CA, USA, 4-4.
 =A0 =A0 =A0 =A0[3] Atul Adya , Jon Howell , Marvin Theimer , William J. =




Bolosky ,

 John R. Douceur, Cooperative Task Management Without Manual Stack
 Management, Proceedings of the General Track: 2002 USENIX Annual Technic=




al

 Conference, p.289-302, June 10-15, 2002 (related presentation:
 http://www.cs.nyu.edu/rgrimm/teaching/fa03-web/091603.pdf)
 =A0 =A0 =A0 =A0[4] http://www.kegel.com/c10k.html
 =A0 =A0 =A0 =A0[5] http://tomasp.net/blog/csharp-fsharp-async-intro.aspx=




 (I should
Apr 01 2011
prev sibling parent Jose Armando Garcia <jsancio gmail.com> writes:
That all sounds good. I would add Flash to your investigation:
http://www.cs.princeton.edu/~vivek/flash/. There is a must read
research paper for any distributed system person at:
http://www.cs.princeton.edu/~vivek/flash_usenix_99/

On Fri, Apr 1, 2011 at 8:17 PM, Max Klyga <max.klyga gmail.com> wrote:

 Since my proposal theme shifted a little towards asyncronicity, I've been
 eagerly consuming information that might help me with this.

 I would be glad to know if I am moving in to the right direction. Comment=


s

 are welcome.

 So far, here is the status report:

 Boost.ASIO

 NeverBlock uses Fibers for managing asyncronous operations. One fiber act=


s

 as a scheduler, polling async operations for complition and resuming othe=


r

 fibers as needed.


 patterns now:
 - Split async functon in two (BeginFoo, EndFoo). Begin... function return=


s

 IAsyncResult object and shedules operation in process-default thread pool=


.

 End... function takes IAsyncResult object that we got earlier and, if
 operation was complited, returns computation results or blocks until the
 operation is completed. All exceptions thrown in asyncronous operation ar=


e

 stored and rethrown when user calls End... function.
 =A0Any function can be called asyncronously using built in wrapper.
 - Some classes provide the following interface for time consuming
 operations:
 =A0 class Foo {
 =A0 =A0 =A0 bar(...); // blocking operation
 =A0 =A0 =A0 asyncBar(...); // asyncronous operation

 =A0 =A0 =A0 cancelAsync();

 =A0 =A0 =A0 barCompleted;

 =A0 =A0 =A0 barResult;
 =A0 }
 - Task-based asyncronicity, similar to std.parallelism. Async operations
 (BeginFoo, EndFoo) can be wrapped as a task and scheduled to be executed =


on

 user controlled task pools.

 Qt uses it's signal/slot mechanism for asyncronous operations. It's somew=


hat



 Boost.ASIO is based on callbacks/delegates. It's implementation is based =


on

 proactor design pattern and doesn't use Fibers or Threads.


 I've also spent some time digging up info on event-based servers.
 This is my to-read list on this weekend:
 =A0 =A0 =A0 =A0[1] Ous96 John Ousterhout. Why threads are a bad idea (for=


 most

 purposes). In USENIX Technical Conference (Invited Talk), Austin, TX,
 January 1996.
 =A0 =A0 =A0 =A0[2] Rob von Behren, Jeremy Condit, and Eric Brewer. 2003. =


Why events

 are a bad idea (for high-concurrency servers). In Proceedings of the 9th
 conference on Hot Topics in Operating Systems - Volume 9 (HOTOS'03), Vol.=


 9.

 USENIX Association, Berkeley, CA, USA, 4-4.
 =A0 =A0 =A0 =A0[3] Atul Adya , Jon Howell , Marvin Theimer , William J. B=


olosky ,

 John R. Douceur, Cooperative Task Management Without Manual Stack
 Management, Proceedings of the General Track: 2002 USENIX Annual Technica=


l

 Conference, p.289-302, June 10-15, 2002 (related presentation:
 http://www.cs.nyu.edu/rgrimm/teaching/fa03-web/091603.pdf)
 =A0 =A0 =A0 =A0[4] http://www.kegel.com/c10k.html
 =A0 =A0 =A0 =A0[5] http://tomasp.net/blog/csharp-fsharp-async-intro.aspx =


(I should
Apr 01 2011
prev sibling next sibling parent reply Max Klyga <max.klyga gmail.com> writes:
Thanks to everyone who provided feedback!

I'll post my proposal update tomorrow.
Apr 04 2011
next sibling parent Jonas Drewsen <jdrewsen nospam.com> writes:
On 04/04/11 22.06, Max Klyga wrote:

 Thanks to everyone who provided feedback!

 I'll post my proposal update tomorrow.


Looking forward to it!

If possible it would be nice if the common types of async methods could 
be supported since each have their own forces: co-routines 
(fibers,scheduler), Futures (e.g. IAsyncResult) and events 
(select/epoll/kqueue). BTW - Personally I wouldn't like a dependency on 
an existing C library (e.g. libev) if this work is to go into the 'std' 
package of Phobos. But this is just my opinion and not anything 
officially stated.

Best of luck
Jonas
Apr 05 2011
prev sibling parent reply Torarin <torarind gmail.com> writes:
2011/4/4 Max Klyga <max.klyga gmail.com>:

 Thanks to everyone who provided feedback!

 I'll post my proposal update tomorrow.


I'm looking forward to it too! I've done a fair amount of research on
these topics and given networking in D a lot of thought. I'd prefer
the standard library's socket interface to be more low level than it
is today, and to have a socket type that provides non-blocking IO
transparently through fibers. This enables both generic protocol
implementations that can handle several types of concurrency, and
efficient parsers using buffered input ranges.

I also think the library should offer it's own event loop instead of
linking to libev or libevent, which have their weaknesses in such a
context. For instance, you can't simply hand over garbage collected
objects to them, which is unfortunate given the usefulness of closures
for handling events. I also think a lower-level API than Boost ASIO is
wise, because one may need to use novel and platform-specific
optimizations like linux splice, a must for efficient file servers.

As I mentioned, I have started implementing some of this. Would you be
interested in pairing up with me on some parts of the project?

Best regards, Torarin
Apr 05 2011
parent Max Klyga <max.klyga gmail.com> writes:
On 2011-04-06 02:30:44 +0300, Torarin said:


 2011/4/4 Max Klyga <max.klyga gmail.com>:

 Thanks to everyone who provided feedback!
 
 I'll post my proposal update tomorrow.
 
 


 
 I'm looking forward to it too! I've done a fair amount of research on
 these topics and given networking in D a lot of thought. I'd prefer
 the standard library's socket interface to be more low level than it
 is today, and to have a socket type that provides non-blocking IO
 transparently through fibers. This enables both generic protocol
 implementations that can handle several types of concurrency, and
 efficient parsers using buffered input ranges.
 
 I also think the library should offer it's own event loop instead of
 linking to libev or libevent, which have their weaknesses in such a
 context. For instance, you can't simply hand over garbage collected
 objects to them, which is unfortunate given the usefulness of closures
 for handling events. I also think a lower-level API than Boost ASIO is
 wise, because one may need to use novel and platform-specific
 optimizations like linux splice, a must for efficient file servers.
 
 As I mentioned, I have started implementing some of this. Would you be
 interested in pairing up with me on some parts of the project?
 
 Best regards, Torarin


I'll be glad to cooperate if this isn't explicitly forbidden by GSoC 
rules (which is highly implausible).
I too think that current socket interface could be improved.
Apr 06 2011
prev sibling parent reply Max Klyga <max.klyga gmail.com> writes:
OK, I've absorbed enough informtion to transform my thoughts in the 
second proposal draft.
I'll be posting it direcltly to GSoC website and correct it, if necessary.

Proposal is situated here: 
http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/neku42/1

Comments 

are welcome.
Apr 06 2011
parent reply Jonas Drewsen <jdrewsen nospam.com> writes:
On 07/04/11 00.32, Max Klyga wrote:

 OK, I've absorbed enough informtion to transform my thoughts in the
 second proposal draft.
 I'll be posting it direcltly to GSoC website and correct it, if necessary.

 Proposal is situated here:
 http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/neku42/1

 Comments
 are welcome.


Seems good to me. Some comments:

You mention that a reactor will probably be used for implementing the 
proactor. On windows it really doesn't make sense to do this it think. 
Instead overlapped IO should be used since it maps very well with the 
proactor pattern. I also suggest that you specify which OSes you will 
make support for. If that is only e.g. linux then building on reactors 
is the right way to go as you've planned.

/Jonas
Apr 07 2011
parent reply Torarin <torarind gmail.com> writes:
2011/4/7 Jonas Drewsen <jdrewsen nospam.com>:

 On 07/04/11 00.32, Max Klyga wrote:

 OK, I've absorbed enough informtion to transform my thoughts in the
 second proposal draft.
 I'll be posting it direcltly to GSoC website and correct it, if necessary.

 Proposal is situated here:

 http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/neku42/1

 Comments
 are welcome.


 Seems good to me. Some comments:

 You mention that a reactor will probably be used for implementing the
 proactor. On windows it really doesn't make sense to do this it think.
 Instead overlapped IO should be used since it maps very well with the
 proactor pattern. I also suggest that you specify which OSes you will make
 support for. If that is only e.g. linux then building on reactors is the
 right way to go as you've planned.

 /Jonas


Indeed it looks good! You can use the reactor pattern on Windows as
well, but several sources say that overlapped IO is more efficient,
and it probably is cleaner. But it's still nice to have a reactor that
wraps select and friends, and then specialize the individual proactors
as needed. Was that what you were thinking, Max?

Torarin
Apr 07 2011
parent Max Klyga <max.klyga gmail.com> writes:
On 2011-04-07 16:17:43 +0300, Torarin said:


 2011/4/7 Jonas Drewsen <jdrewsen nospam.com>:

 On 07/04/11 00.32, Max Klyga wrote:

 snip


 
 Seems good to me. Some comments:
 
 You mention that a reactor will probably be used for implementing the
 proactor. On windows it really doesn't make sense to do this it think.
 Instead overlapped IO should be used since it maps very well with the
 proactor pattern. I also suggest that you specify which OSes you will make
 support for. If that is only e.g. linux then building on reactors is the
 right way to go as you've planned.
 
 /Jonas
 


 
 Indeed it looks good! You can use the reactor pattern on Windows as
 well, but several sources say that overlapped IO is more efficient,
 and it probably is cleaner. But it's still nice to have a reactor that
 wraps select and friends, and then specialize the individual proactors
 as needed. Was that what you were thinking, Max?
 
 Torarin



I plan to start with a Mac implementation, because that's what I use 
most of the time, but I have a Windows box too and can install Linux on 
virtual machine, if that will be needed.

Yes, I had an idea to implement reactor and proactor the way, user may 
swap implementations, e.g. portable reactor based on select and 
platform specific one using efficient implementation for that platform.

This approach is used in Twisted python web-server (It's licence states 
that it needs attribution only on reuse of descent amounts of code, so 
it might be a good idea to peek for some platform specific details).

Windows overlapped I/O allows to implement proactor on top of it 
without any additional layers, but reactor can be implemented on of 
overlapped I/O too.
Apr 07 2011