www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Curl and redirects

reply "Chris" <christian_stengel yahoo.de> writes:
Hi,

I am trying to log into a web application using curl. The web app
sends a 302 redirect after posting the data (successful log in -
I can see the successful login in the web apps log files).

Maybe I am doing something wrong - but I have just copied the
code of _basicHTTP form curl.d to my own method to authenticate.

The exception I'll receive on perform is:

std.net.curl.CurlException std/net/curl.d(3348): Send failed
since rewinding of the data stream failed on handle 7FCABA819400

My Code:


       void post(string url, const(void)[] sendData){
           HTTP client = HTTP();
           client.method = HTTP.Method.post;

client.addRequestHeader("Content-Type","application/x-www-form-urlencoded");

           scope (exit) {
               client.onReceiveHeader = null;
               client.onReceiveStatusLine = null;
               client.onReceive = null;
           }
           client.url = url;
           HTTP.StatusLine statusLine;
           ubyte[] content;
           string[string] headers;
           client.onReceive = (ubyte[] data)
           {
               content ~= data;
               return data.length;
           };

           if (sendData !is null &&
               (client.method == HTTP.Method.post || client.method
== HTTP.Method.put))
           {
               client.contentLength = sendData.length;
               client.onSend = delegate size_t(void[] buf)
               {
                   size_t minLen = std.algorithm.min(buf.length,
sendData.length);
                   if (minLen == 0) return 0;
                   buf[0..minLen] = sendData[0..minLen];
                   sendData = sendData[minLen..$];
                   return minLen;
               };
           }

           client.onReceiveHeader = (in char[] key,
                                     in char[] value)
           {
               if (auto v = key in headers)
               {
                   *v ~= ", ";
                   *v ~= value;
               }
               else
                   headers[key] = value.idup;
           };
           client.onReceiveStatusLine = (HTTP.StatusLine l) {
statusLine = l; };
           client.perform();

           // Default charset defined in HTTP RFC
           auto charset = "ISO-8859-1";
           if (auto v = "content-type" in headers)
           {
               auto m = match(cast(char[]) (*v),
regex("charset=([^;,]*)"));
               if (!m.empty && m.captures.length > 1)
               {
                   charset = m.captures[1].idup;
               }
           }
       }

Does anybody have a clue, what I am doing wrong?

Thanks,

Christian
Dec 17 2012
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-12-17 12:24, Chris wrote:
 Hi,

 I am trying to log into a web application using curl. The web app
 sends a 302 redirect after posting the data (successful log in -
 I can see the successful login in the web apps log files).

 Maybe I am doing something wrong - but I have just copied the
 code of _basicHTTP form curl.d to my own method to authenticate.

 The exception I'll receive on perform is:

 std.net.curl.CurlException std/net/curl.d(3348): Send failed
 since rewinding of the data stream failed on handle 7FCABA819400

 My Code:

 Does anybody have a clue, what I am doing wrong?

A guess: when using the command line tool one needs to explicit enable redirects, is there something similar in the API? -- /Jacob Carlborg
Dec 17 2012
prev sibling next sibling parent "Chris" <christian_stengel yahoo.de> writes:
Hi Jacob,

thanks for your quick response,

On Monday, 17 December 2012 at 14:12:26 UTC, Jacob Carlborg wrote:

 A guess: when using the command line tool one needs to explicit 
 enable redirects, is there something similar in the API?

I haven't found one - also redirects do work with get requests without setting any parameter. I tried to set the maxRedirects to 1000 without any success. Thanks, Christian
Dec 17 2012
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 17 December 2012 at 11:24:41 UTC, Chris wrote:
 std.net.curl.CurlException std/net/curl.d(3348): Send failed
 since rewinding of the data stream failed on handle 7FCABA819400

I believe I've fixed this problem in this pull request: https://github.com/D-Programming-Language/phobos/pull/860 Try using Phobos from git.
Dec 17 2012
prev sibling next sibling parent "Chris" <christian_stengel yahoo.de> writes:
Dear Vladimir,

thank you very much for your answer - and sorry for the delay - 
but I had problems compiling dmd myself on osx.

I checked out dmd, runtime, phobos and tools and compiled the 
stuff. Unfortunately I am still getting the exception:

-- snip --
std.net.curl.CurlException std/net/curl.d(3449): Send failed 
since rewinding of the data stream failed on handle 7FAA09819400
----------------
5   accept                              0x0000000101865cf6 void 
std.net.curl.Curl._check(int) + 130
6   accept                              0x0000000101865fb4 int 
std.net.curl.Curl.perform(bool) + 56
7   accept                              0x00000001018646bc int 
std.net.curl.HTTP._perform(bool) + 600
8   accept                              0x000000010186445e void 
std.net.curl.HTTP.perform() + 18
9   accept                              0x0000000101803c40 void 
http_client.HTTPClient.post1(immutable(char)[], const(void)[]) + 
424
10  accept                              0x00000001018031bc void 
http_client.HTTPClient.post() + 464
11  accept                              0x00000001017bf1bb _Dmain 
+ 239
12  accept                              0x0000000101819ded extern 
(C) int rt.dmain2._d_run_main(int, char**, extern (C) int 
function(char[][])*).void runMain() + 33
13  accept                              0x000000010181994d extern 
(C) int rt.dmain2._d_run_main(int, char**, extern (C) int 
function(char[][])*).void tryExec(scope void delegate()) + 45
14  accept                              0x0000000101819e34 extern 
(C) int rt.dmain2._d_run_main(int, char**, extern (C) int 
function(char[][])*).void runAll() + 56
15  accept                              0x000000010181994d extern 
(C) int rt.dmain2._d_run_main(int, char**, extern (C) int 
function(char[][])*).void tryExec(scope void delegate()) + 45
16  accept                              0x0000000101819905 
_d_run_main + 465
17  accept                              0x000000010181972c main + 
20
18  libdyld.dylib                       0x00007fff93cec7e1 start 
+ 0
19  ???                                 0x0000000000000001 0x0 + 1

-- snip --

Maybe it's an issue with MacOSX?

My settings:
- DMD64 D Compiler v2.061
- DFLAGS=-I/usr/local/include/d2 -L-L/usr/local/lib.

Thanks,

Chris



On Monday, 17 December 2012 at 18:57:25 UTC, Vladimir Panteleev 
wrote:
 On Monday, 17 December 2012 at 11:24:41 UTC, Chris wrote:
 std.net.curl.CurlException std/net/curl.d(3348): Send failed
 since rewinding of the data stream failed on handle 
 7FCABA819400

I believe I've fixed this problem in this pull request: https://github.com/D-Programming-Language/phobos/pull/860 Try using Phobos from git.

Dec 19 2012
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Wednesday, 19 December 2012 at 08:46:06 UTC, Chris wrote:
 Dear Vladimir,

 thank you very much for your answer - and sorry for the delay - 
 but I had problems compiling dmd myself on osx.

Sorry about that. In my haste to answer, I've misread your question. I see now that you're using the lower-level objects directly instead of the post function. My patch fixes the behavior for the post function. If you wish to continue using the HTTP type directly, you'll basically need to apply the same patch as in my pull request to your code, and add a client.handle.onSeek handler which implements seeking over the data you're uploading. Alternatively, have you tried constructing a custom HTTP object and passing that as the last parameter to post?
Dec 19 2012
prev sibling parent "Chris" <christian_stengel yahoo.de> writes:
Dear Vladimir,

On Thursday, 20 December 2012 at 02:51:58 UTC, Vladimir Panteleev 
wrote:
 Sorry about that. In my haste to answer, I've misread your 
 question. I see now that you're using the lower-level objects 
 directly instead of the post function. My patch fixes the 
 behavior for the post function.

that's my fault - should have looked in there too. After adding those fixes it works :) Thank you very much for your help, Christian
Dec 21 2012