www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Curl on Windows

reply "Brad Anderson" <eco gnuk.net> writes:
The newly accepted curl library sadly didn't get included in the 
2.058 release as it wasn't added to the Windows makefile.

I created a pull request to resolve this [1] but it had to be 
reverted because it broke the auto tester.

The tester broke because of the missing curl.lib library.  I've 
suggested curl.lib be included with phobos but Brad says it can't 
due to licensing problems.  I couldn't find a discussion on the 
newsgroup about what problems there are.

Curl is licensed under MIT [2]. The optional SSL support for curl 
uses OpenSSL which is dual licensed under both Apache License 1.0 
and SSLeay License, a 4-clause BSD License derivative [3].

I'm no expert on licensing issues but the only issues I can see 
are, first, the copyright notice of the Curl and OpenSSL licenses 
must be displayed whether it is distributed in binary or source 
code form (easily added as a text file and, for good measure, in 
a licensing page in the installer).  And second, the advertising 
clauses in OpenSSL:

      * 3. All advertising materials mentioning features or use of 
this
      *    software must display the following acknowledgment:
      *    "This product includes software developed by the 
OpenSSL Project
      *    for use in the OpenSSL Toolkit. 
(http://www.openssl.org/)"


and

      * 3. All advertising materials mentioning features or use of 
this software
      *    must display the following acknowledgement:
      *    "This product includes cryptographic software written by
      *     Eric Young (eay cryptsoft.com)"
      *    The word 'cryptographic' can be left out if the rouines 
from the library
      *    being used are not cryptographic related :-).
      * 4. If you include any Windows specific code (or a 
derivative thereof) from
      *    the apps directory (application code) you must include 
an acknowledgement:
      *    "This product includes software written by Tim Hudson 
(tjh cryptsoft.com)"

Both of these sound fairly reasonable to meet.  Lots of other 
software does (Python from Windows, Google Chrome, etc.)

Is there anything I'm missing here?

If not, we can talk about how to approach including it in the 
release.

[1] https://github.com/D-Programming-Language/phobos/pull/458
[2] http://curl.haxx.se/docs/copyright.html
[3] http://www.openssl.org/source/license.html
Feb 26 2012
next sibling parent Sean Kelly <sean invisibleduck.org> writes:
I think specific SSL algorithms in the OpenSSL package may be covered by dif=
ferent licenses.=20

On Feb 26, 2012, at 5:15 PM, "Brad Anderson" <eco gnuk.net> wrote:

 The newly accepted curl library sadly didn't get included in the 2.058 rel=

=20
 I created a pull request to resolve this [1] but it had to be reverted bec=

=20
 The tester broke because of the missing curl.lib library.  I've suggested c=

lems. I couldn't find a discussion on the newsgroup about what problems the= re are.
=20
 Curl is licensed under MIT [2]. The optional SSL support for curl uses Ope=

, a 4-clause BSD License derivative [3].
=20
 I'm no expert on licensing issues but the only issues I can see are, first=

ether it is distributed in binary or source code form (easily added as a tex= t file and, for good measure, in a licensing page in the installer). And se= cond, the advertising clauses in OpenSSL:
=20
     * 3. All advertising materials mentioning features or use of this
     *    software must display the following acknowledgment:
     *    "This product includes software developed by the OpenSSL Project
     *    for use in the OpenSSL Toolkit. (http://www.openssl.org/)"
=20
=20
 and
=20
     * 3. All advertising materials mentioning features or use of this soft=

     *    must display the following acknowledgement:
     *    "This product includes cryptographic software written by
     *     Eric Young (eay cryptsoft.com)"
     *    The word 'cryptographic' can be left out if the rouines from the l=

     *    being used are not cryptographic related :-).
     * 4. If you include any Windows specific code (or a derivative thereof=

     *    the apps directory (application code) you must include an acknowl=

     *    "This product includes software written by Tim Hudson (tjh crypts=

=20
 Both of these sound fairly reasonable to meet.  Lots of other software doe=

=20
 Is there anything I'm missing here?
=20
 If not, we can talk about how to approach including it in the release.
=20
 [1] https://github.com/D-Programming-Language/phobos/pull/458
 [2] http://curl.haxx.se/docs/copyright.html
 [3] http://www.openssl.org/source/license.html

Feb 26 2012
prev sibling next sibling parent Brad Roberts <braddr puremagic.com> writes:
On 2/26/2012 5:15 PM, Brad Anderson wrote:
 The newly accepted curl library sadly didn't get included in the 2.058 release
as it wasn't added to the Windows makefile.
 
 I created a pull request to resolve this [1] but it had to be reverted because
it broke the auto tester.
 
 The tester broke because of the missing curl.lib library.  I've suggested
curl.lib be included with phobos but Brad says
 it can't due to licensing problems.  I couldn't find a discussion on the
newsgroup about what problems there are.

Careful, that's not quite what I said: ============ libcurl should not be included *in* phobos as that would have licensing implications. It should probably be included in the release tarballs as both source and compiled libraries (also probably only on windows, non-windows can suggest how to install via the standard package managers). A reasonable alternative is keeping it completely separate and just listing it on the download page on the website. ============ The problem is the attribution clause(s) of most licenses, including MIT. While you might not consider it a problem to give attribution, the core libraries of D must not require it. Google for past discussions on the topic, it's well trodden territory at this point.
Feb 26 2012
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/26/12 7:15 PM, Brad Anderson wrote:
 The newly accepted curl library sadly didn't get included in the 2.058
 release as it wasn't added to the Windows makefile.

One possibility would be to contact the author and ask if he'd be willing to relicense libcurl for purpose of inclusion in phobos. He seems nice and cooperative, so Brad Anderson, could you please take that up? Andrei
Feb 26 2012
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/26/2012 6:39 PM, Brad Anderson wrote:
 If phobos can include zlib, I see no reason it couldn't include libcurl.

Yes, we do include zlib, but I don't think that is a precedent we should follow. Including such third party libraries is potentially a large support burden and source of conflict as those libraries get updated. It's already been a problem with zlib. It's beyond the scope of what D should be to take this stuff on. Instead, we should provide download links so the user can install it themselves as required. As for the auto tester breaking, a better approach might be to install curl on the test system, or simply not run those tests.
Feb 28 2012
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Monday, 27 February 2012 at 02:24:49 UTC, Andrei Alexandrescu 
wrote:
 On 2/26/12 7:15 PM, Brad Anderson wrote:
 The newly accepted curl library sadly didn't get included in 
 the 2.058
 release as it wasn't added to the Windows makefile.

One possibility would be to contact the author and ask if he'd be willing to relicense libcurl for purpose of inclusion in phobos. He seems nice and cooperative, so Brad Anderson, could you please take that up? Andrei

http://curl.haxx.se/docs/faq.html#Can_you_please_change_the_curl_l I could try but based on that FAQ answer it doesn't sound like they are interested in offering that to anyone. It's already offered under a very permissive license so I don't believe including libcurl itself is an issue at all. If phobos can include zlib, I see no reason it couldn't include libcurl. It's OpenSSL that is more complicated. If the OpenSSL situation can't be worked out at the very least we can include libcurl without SSL support. Regards, Brad Anderson
Feb 26 2012
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Monday, 27 February 2012 at 02:24:49 UTC, Andrei Alexandrescu 
wrote:
 On 2/26/12 7:15 PM, Brad Anderson wrote:
 The newly accepted curl library sadly didn't get included in 
 the 2.058
 release as it wasn't added to the Windows makefile.

One possibility would be to contact the author and ask if he'd be willing to relicense libcurl for purpose of inclusion in phobos. He seems nice and cooperative, so Brad Anderson, could you please take that up? Andrei

Looks like Walter already contacted him: http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=112832 Regards, Brad Anderson
Feb 26 2012
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Monday, 27 February 2012 at 02:16:01 UTC, Brad Roberts wrote:
 On 2/26/2012 5:15 PM, Brad Anderson wrote:
 The newly accepted curl library sadly didn't get included in 
 the 2.058 release as it wasn't added to the Windows makefile.
 
 I created a pull request to resolve this [1] but it had to be 
 reverted because it broke the auto tester.
 
 The tester broke because of the missing curl.lib library.  
 I've suggested curl.lib be included with phobos but Brad says
 it can't due to licensing problems.  I couldn't find a 
 discussion on the newsgroup about what problems there are.

Careful, that's not quite what I said:

I'm sorry for paraphrasing incorrectly.
 ============
 libcurl should not be included *in* phobos as that would have 
 licensing implications. It should probably be included in
 the release tarballs as both source and compiled libraries 
 (also probably only on windows, non-windows can suggest how
 to install via the standard package managers). A reasonable 
 alternative is keeping it completely separate and just
 listing it on the download page on the website.
 ============

 The problem is the attribution clause(s) of most licenses, 
 including MIT.  While you might not consider it a problem to
 give attribution, the core libraries of D must not require it.  
 Google for past discussions on the topic, it's well
 trodden territory at this point.

I've done some more searching. It appears Walter contacted libcurl's author and asked about it and the author said the attribution clause does not apply to binary distributions of libcurl. I think I'll try to make a new pull request using your first option (adding the curl source and adding it to the phobos build process on Windows in the same manner that zlib already is). OpenSSL is still an issue, of course. I could make downloading it an optional part of the installer so people have to opt-in to the restrictions OpenSSL usage would impose on them. I'm not sure if that would actually help with anything though. Regards, Brad Anderson
Feb 26 2012
prev sibling next sibling parent "Jesse Phillips" <jessekphillips+D gmail.com> writes:
On Monday, 27 February 2012 at 02:39:39 UTC, Brad Anderson wrote:

 If phobos can include zlib, I see no reason it couldn't include 
 libcurl.

Maybe the discussion here will give some light https://github.com/D-Programming-Language/phobos/pull/46
Feb 27 2012
prev sibling next sibling parent "igeek" <evermind live.ru> writes:
Could anybody explain why you want to include curl in Phobos? 
Maybe Phobos have some native function that allow to do some 
thing that curl are doing?
Feb 27 2012
prev sibling next sibling parent Brad Anderson <eco gnuk.net> writes:
--e89a8f23458faa31bb04ba0095e5
Content-Type: text/plain; charset=ISO-8859-1

On Mon, Feb 27, 2012 at 9:07 PM, Jesse Phillips
<jessekphillips+D gmail.com>wrote:

 On Monday, 27 February 2012 at 02:39:39 UTC, Brad Anderson wrote:

  If phobos can include zlib, I see no reason it couldn't include libcurl.

Maybe the discussion here will give some light https://github.com/D-**Programming-Language/phobos/**pull/46<https://github.com/D-Programming-Language/phobos/pull/46>

Seeing how that was integrated with help, certainly. Thanks. Curl is a lot more involved than I anticipated. Nearly 100 source files and the most convoluted build system I've ever seen. Does anyone know who created the libcurl built with dmc that was offered up in the curl wrapper review? If they had a makefile or that'd give me a nice headstart on this. Regards, Brad Anderson --e89a8f23458faa31bb04ba0095e5 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Mon, Feb 27, 2012 at 9:07 PM, Jesse Phillips <span dir=3D"ltr">&lt;<a hr= ef=3D"mailto:jessekphillips%2BD gmail.com">jessekphillips+D gmail.com</a>&g= t;</span> wrote:<br><div class=3D"gmail_quote"><blockquote class=3D"gmail_q= uote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1e= x"> <div class=3D"im">On Monday, 27 February 2012 at 02:39:39 UTC, Brad Anderso= n wrote:<br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> If phobos can include zlib, I see no reason it couldn&#39;t include libcurl= .<br> </blockquote> <br></div> Maybe the discussion here will give some light<br> <br> <a href=3D"https://github.com/D-Programming-Language/phobos/pull/46" target= =3D"_blank">https://github.com/D-<u></u>Programming-Language/phobos/<u></u>= pull/46</a><br> </blockquote></div><br><div>Seeing how that was integrated with help, certa= inly. Thanks. Curl is a lot more involved than I anticipated. =A0Nearly 100= source files and the most convoluted build system I&#39;ve ever seen. Does= anyone know who created the libcurl built with dmc that was offered up in = the curl wrapper review? =A0If they had a makefile or that&#39;d give me a = nice headstart on this.</div> <div><br></div><div>Regards,</div><div>Brad Anderson</div> --e89a8f23458faa31bb04ba0095e5--
Feb 27 2012
prev sibling next sibling parent Brad Anderson <eco gnuk.net> writes:
--f46d04083b051fc56c04ba098419
Content-Type: text/plain; charset=ISO-8859-1

On Mon, Feb 27, 2012 at 11:45 PM, igeek <evermind live.ru> wrote:

 Could anybody explain why you want to include curl in Phobos? Maybe Phobos
 have some native function that allow to do some thing that curl are doing?

It's already in Phobos as part of std.net.curl. It's just not very usable on Windows yet. Curl has a phobos friendly license and implementing it in D is a fairly large undertaking (just look at the size of curl). Nothing is stopping anyone from doing a pure D networking client module, of course. Using curl was just a very quick way of getting this functionality into Phobos. Regards, Brad Anderson --f46d04083b051fc56c04ba098419 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Mon, Feb 27, 2012 at 11:45 PM, igeek <span dir=3D"ltr">&lt;<a href=3D"ma= ilto:evermind live.ru">evermind live.ru</a>&gt;</span> wrote:<br><div class= =3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8= ex;border-left:1px #ccc solid;padding-left:1ex"> Could anybody explain why you want to include curl in Phobos? Maybe Phobos = have some native function that allow to do some thing that curl are doing?<= br> </blockquote></div><br><div>It&#39;s already in Phobos as part of std.net.c= url. =A0It&#39;s just not very usable on Windows yet.</div><div><br></div><= div>Curl has a phobos friendly license and implementing it in D is a fairly= large undertaking (just look at the size of curl). =A0Nothing is stopping = anyone from doing a pure D networking client module, of course. =A0Using cu= rl was just a very quick way of getting this functionality into Phobos.</di= v> <div><br></div><div>Regards,</div><div>Brad Anderson</div> --f46d04083b051fc56c04ba098419--
Feb 28 2012
prev sibling next sibling parent Brad Anderson <eco gnuk.net> writes:
--0015175cd36e8bc54c04ba0d7b75
Content-Type: text/plain; charset=ISO-8859-1

On Tue, Feb 28, 2012 at 2:36 PM, Walter Bright
<newshound2 digitalmars.com>wrote:

 On 2/26/2012 6:39 PM, Brad Anderson wrote:

 If phobos can include zlib, I see no reason it couldn't include libcurl.

Yes, we do include zlib, but I don't think that is a precedent we should follow. Including such third party libraries is potentially a large support burden and source of conflict as those libraries get updated. It's already been a problem with zlib. It's beyond the scope of what D should be to take this stuff on. Instead, we should provide download links so the user can install it themselves as required. As for the auto tester breaking, a better approach might be to install curl on the test system, or simply not run those tests.

Ok, take that approach for the revised pull request. Regards, Brad Anderson --0015175cd36e8bc54c04ba0d7b75 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Tue, Feb 28, 2012 at 2:36 PM, Walter Bright <span dir=3D"ltr">&lt;<a hre= f=3D"mailto:newshound2 digitalmars.com">newshound2 digitalmars.com</a>&gt;<= /span> wrote:<br><div class=3D"gmail_quote"><blockquote class=3D"gmail_quot= e" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div class=3D"im">On 2/26/2012 6:39 PM, Brad Anderson wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> If phobos can include zlib, I see no reason it couldn&#39;t include libcurl= .<br> </blockquote> <br></div> Yes, we do include zlib, but I don&#39;t think that is a precedent we shoul= d follow. Including such third party libraries is potentially a large suppo= rt burden and source of conflict as those libraries get updated. It&#39;s a= lready been a problem with zlib. It&#39;s beyond the scope of what D should= be to take this stuff on.<br> <br> Instead, we should provide download links so the user can install it themse= lves as required.<br> <br> As for the auto tester breaking, a better approach might be to install curl= on the test system, or simply not run those tests.<br> </blockquote></div><br><div>Ok, take that approach for the revised pull req= uest.</div><div><br></div><div>Regards,</div><div>Brad Anderson</div> --0015175cd36e8bc54c04ba0d7b75--
Feb 28 2012
prev sibling next sibling parent "Kapps" <opantm2+spam gmail.com> writes:
On Tuesday, 28 February 2012 at 21:36:28 UTC, Walter Bright wrote:
 On 2/26/2012 6:39 PM, Brad Anderson wrote:
 If phobos can include zlib, I see no reason it couldn't 
 include libcurl.

Yes, we do include zlib, but I don't think that is a precedent we should follow. Including such third party libraries is potentially a large support burden and source of conflict as those libraries get updated. It's already been a problem with zlib. It's beyond the scope of what D should be to take this stuff on. Instead, we should provide download links so the user can install it themselves as required. As for the auto tester breaking, a better approach might be to install curl on the test system, or simply not run those tests.

It seems somewhat strange to me to have a hidden dependency that, despite being included in the standard library, can not be used with a default install. What about an option in the default installer to download these third party libraries (and on Windows coffimplib them)?
Feb 28 2012
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Wednesday, 29 February 2012 at 00:53:55 UTC, Kapps wrote:
 On Tuesday, 28 February 2012 at 21:36:28 UTC, Walter Bright 
 wrote:
 On 2/26/2012 6:39 PM, Brad Anderson wrote:
 If phobos can include zlib, I see no reason it couldn't 
 include libcurl.

Yes, we do include zlib, but I don't think that is a precedent we should follow. Including such third party libraries is potentially a large support burden and source of conflict as those libraries get updated. It's already been a problem with zlib. It's beyond the scope of what D should be to take this stuff on. Instead, we should provide download links so the user can install it themselves as required. As for the auto tester breaking, a better approach might be to install curl on the test system, or simply not run those tests.

It seems somewhat strange to me to have a hidden dependency that, despite being included in the standard library, can not be used with a default install. What about an option in the default installer to download these third party libraries (and on Windows coffimplib them)?

https://github.com/D-Programming-Language/installer/pull/7 Regards, Brad Anderson
Feb 28 2012
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Tuesday, 28 February 2012 at 21:36:28 UTC, Walter Bright wrote:
 On 2/26/2012 6:39 PM, Brad Anderson wrote:
 If phobos can include zlib, I see no reason it couldn't 
 include libcurl.

Yes, we do include zlib, but I don't think that is a precedent we should follow. Including such third party libraries is potentially a large support burden and source of conflict as those libraries get updated. It's already been a problem with zlib. It's beyond the scope of what D should be to take this stuff on. Instead, we should provide download links so the user can install it themselves as required. As for the auto tester breaking, a better approach might be to install curl on the test system, or simply not run those tests.

https://github.com/D-Programming-Language/phobos/pull/466 https://github.com/D-Programming-Language/d-programming-language.org/pull/92 https://github.com/D-Programming-Language/installer/pull/7 Regards, Brad Anderson
Feb 28 2012
prev sibling parent "Marco Leise" <Marco.Leise gmx.de> writes:
Am 27.02.2012, 03:15 Uhr, schrieb Brad Roberts <braddr puremagic.com>:

 [=E2=80=A6] non-windows can suggest how to install via the standard pa=

On Gentoo USE flags handle such cases. They enable additional dependenci= es on Curl, Python, zlib or whatever may be used with the installed pack= age. Often these flags directly relate to calling "./configure --enable/= disable-flag" in the extracted source of the program and pulling a requi= red library into the dependencies list. Now I know that DMD uses a plain Makefile, but it would be nice if Phobo= s compiled without curl installed. If not, that's no problem either and = curl would just become a static dependency for DMD. That's already the c= ase for LibreOffice, GoogleEarth, cmake and gnupg, to name a few program= s I have installed. Git and Gimp have curl as an optional dependency OTO= H.
Mar 01 2012