www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Working with Buffered streams

reply Lorenzo Villani <arbiter beyond-linux.org> writes:
Ok, this is a very noob-ish question :)

I'm writing a simple HTTP file downloader (you can find it attached to this
post) but I have some troubles getting a proper file after transfer. (eg: local
downloaded file is corrupted). Can someone help me? (Please note that i'm new
to this language :D )
Apr 06 2007
next sibling parent reply Brad Anderson <brad dsource.org> writes:
Lorenzo Villani wrote:

 Ok, this is a very noob-ish question :)
 
 I'm writing a simple HTTP file downloader (you can find it attached to this
post) but I have some troubles getting a proper file after transfer. (eg: local
downloaded file is corrupted). Can someone help me? (Please note that i'm new
to this language :D )
 


[snip code]

http://www.dsource.org/projects/tango/browser/trunk/example/networking/httpget.d

or

http://www.dsource.org/projects/tango/browser/trunk/example/networking/homepage.d

might help...

BA
Apr 06 2007
parent Lorenzo Villani <arbiter beyond-linux.org> writes:
Brad Anderson Wrote:


 Lorenzo Villani wrote:

 Ok, this is a very noob-ish question :)
 
 I'm writing a simple HTTP file downloader (you can find it attached to this
post) but I have some troubles getting a proper file after transfer. (eg: local
downloaded file is corrupted). Can someone help me? (Please note that i'm new
to this language :D )
 


 [snip code]
 
 http://www.dsource.org/projects/tango/browser/trunk/example/networking/httpget.d
 
 or
 
 http://www.dsource.org/projects/tango/browser/trunk/example/networking/homepage.d
 
 might help...
 
 BA


The fact is that i want to implement this class myself without being depedent
on tango :)
Apr 06 2007
prev sibling parent reply Bradley Smith <digitalmars-com baysmith.com> writes:
There is a bug in the Stream class. If you mix, readLine() and 
read(ubyte[]) calls to a stream, readLine() may not appear to perform as 
documented. As documented, readLine() stop at "some combination of 
carriage return [CR] and line feed [LF]". However, this is not 
necessarily true. If the stream is non-seekable, readLine() will stop at 
CR and set a prevCr flag indicating that a LF should be consumed on the 
next read. However, this only takes place in the getc() method and 
methods using getc(). Some methods, like read(ubyte[]), read the stream 
directly and the prevCr flag is not processed.

At first, I thought this might just be odd behavior rather than a bug, 
but I decided that it is a bug. In order to understand how to use the 
stream effectively, one must understand the implementation details. One 
must know that the behavior for seekable streams is different that 
non-seekable streams, and adjust the client code accordingly. For this 
reason, it should be considered a bug.

The workaround this bug is to check the first character of a 
non-seekable stream after reading with readLine(). For example:

       if(!ss.seekable) {
         ss.read(b);
         if (b[0] != '\n') {
           s.write(b);
         }
       }
       while(ss.read(b) != 0){
         s.write(b);
       }

Thanks,
   Bradley

Lorenzo Villani wrote:

 Ok, this is a very noob-ish question :)
 
 I'm writing a simple HTTP file downloader (you can find it attached to this
post) but I have some troubles getting a proper file after transfer. (eg: local
downloaded file is corrupted). Can someone help me? (Please note that i'm new
to this language :D )
 
 
 ------------------------------------------------------------------------
 
 import std.file;
 import std.string;
 import std.stdio;
 import std.stream;
 import std.socketstream;
 import std.socket;
 
 class HttpDownload {	
 	public:
 		this() {}
 		
 		void downloadFile(char[] domain, char[] dir, char[] file, int port = 80) {
 			Socket sock = new TcpSocket(new InternetAddress(domain, port));
 			SocketStream ss = new SocketStream(sock);
 			
 			// we prepare the request
 			if ( port != 80 )
 				domain = domain ~ ":" ~ cast(char)port;
 			
 			char[] request = "GET " ~ dir ~ "/" ~ file ~ " HTTP/1.1\r\n"
 				~ "Connection: Keep-Alive\r\n"
 				~ "User-Agent: Mozilla 5.0\r\n"
 				~ "Host: " ~ domain ~ "\r\n"
 				~ "\r\n";
 			
 			// we send the request
 			ss.writeString(request);
 			
 			// this for removes the HTTP header and looks for a positive response
 			// from the webserver
 			for(;;) {
 				char[] response = ss.readLine();
 				char[] RESPONSE = "HTTP/1.1 ";
 				
 				if(!response.length) {
 					break;
 				} else {
 					if (response.length > RESPONSE.length && !icmp(RESPONSE, response[0 ..
RESPONSE.length])) {
 						char[] code;
 						code = response[RESPONSE.length .. response.length];
 						int i = ifind(code, "200 OK");
 						if ( i != -1 ) {
 							writefln("---> Received a 200 OK");
 						} else {
 							throw new Exception("Webserver has replied with an error");
 							return;
 						}
 					}
 				}
 			}
 			
 			// write file
 			BufferedFile s = new BufferedFile();
 			s.create(getcwd() ~ "/" ~ file);
 			ubyte[1] b;
 			while(ss.read(b) != 0){
 				s.write(b);
 			}
 			ss.close();
 			s.close();
 		}
 }
 
 int main() {
 	HttpDownload download = new HttpDownload();
 	download.downloadFile("www.capponcino.it", "/tremulous/dl/pk3",
"highrise-b4.pk3", 80);
 	return 0;
 }
Apr 06 2007
parent Lorenzo Villani <arbiter beyond-linux.org> writes:
Bradley Smith Wrote:

 [cut]


Thanks a lot, it works! :D
Apr 07 2007