www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - bigint <-> python long

reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
Hey.

Investigating the possibility of providing this conversion in pyd.

Python provides an api for accessing the underlying bytes.

std.bigint seemingly doesn't. Am I missing anything?
Sep 05 2012
next sibling parent reply "Paul D. Anderson" <paul.d.removethis.anderson comcast.andthis.net> writes:
On Wednesday, 5 September 2012 at 18:13:40 UTC, Ellery Newcomer 
wrote:
 Hey.

 Investigating the possibility of providing this conversion in 
 pyd.

 Python provides an api for accessing the underlying bytes.

 std.bigint seemingly doesn't. Am I missing anything?

No, I don't believe so. AFAIK there is no public access to the underlying array, but I think it is a good idea. I suspect the reason for not disclosing the details is to disallow anyone putting the data into an invalid state. But read-only access would be safe.
Sep 05 2012
parent Don Clugston <dac nospam.com> writes:
On 05/09/12 21:23, Paul D. Anderson wrote:
 On Wednesday, 5 September 2012 at 18:13:40 UTC, Ellery Newcomer wrote:
 Hey.

 Investigating the possibility of providing this conversion in pyd.

 Python provides an api for accessing the underlying bytes.

 std.bigint seemingly doesn't. Am I missing anything?

No, I don't believe so. AFAIK there is no public access to the underlying array, but I think it is a good idea. I suspect the reason for not disclosing the details is to disallow anyone putting the data into an invalid state. But read-only access would be safe.

No, it's just not disclosed because I didn't know the best way to do it. I didn't want to put something in unless I was sure it was correct. (And a key part of that, is what is required to implement BigFloat).
Sep 06 2012
prev sibling next sibling parent "Paul D. Anderson" <paul.d.removethis.anderson comcast.andthis.net> writes:
On Wednesday, 5 September 2012 at 19:23:11 UTC, Paul D. Anderson 
wrote:

 No, I don't believe so. AFAIK there is no public access to the 
 underlying array, but I think it is a good idea.

I meant to say I think that access to the array is a good idea, not the lack of access. Words are hard!
Sep 05 2012
prev sibling next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Ellery Newcomer:

 Investigating the possibility of providing this conversion in 
 pyd.

Are you updating Pyd? :-) Bye, bearophile
Sep 05 2012
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/05/2012 05:02 PM, bearophile wrote:
 Ellery Newcomer:

 Investigating the possibility of providing this conversion in pyd.

Are you updating Pyd? :-) Bye, bearophile

Yep. Have any suggestions for supported conversion out of the box? From the standard library, I already have Complex, Tuple, and now BigInt. Was thinking maybe the datetime stuff.. Also, which of the following looks more appealing to you? wrap_class!(Foo, Def!(Foo.bar, PyName!"__len__"))(); wrap_class!(Foo, Len!(Foo.bar))(); they wrap Foo and give it an overriden __len__; the latter is how it is done at the moment (with similar templates for the other operators), but boost::python does something akin to the former. PyName is in the former because Def takes too many optional string arguments.
Sep 05 2012
next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/05/2012 07:10 PM, bearophile wrote:
 Some D lazy ranges <==> Python lazy iterators/generators

I'll look into this one.
 array.array <==> D arrays

just checked, looks like we have it: PyStmts(q"{from array import array; a = array('i', [44,33,22,11]);}", "testing"); assert(PyEval!(int[])("a", "testing") == [44,33,22,11]); I think if the python object is iterable, it can be converted to array. Matrices might pose a problem. But user can define custom conversions if need be.
 NumPy arrays <==> D arrays

I have never used NumPy. Are its arrays special?
 Also, which of the following looks more appealing to you?

I don't know.

ok, I'll be lazy then.
Sep 05 2012
next sibling parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/06/2012 04:11 AM, bearophile wrote:
 Ellery Newcomer:

 array.array <==> D arrays

just checked, looks like we have it: PyStmts(q"{from array import array; a = array('i', [44,33,22,11]);}", "testing"); assert(PyEval!(int[])("a", "testing") == [44,33,22,11]); I think if the python object is iterable, it can be converted to array.

array.array are special, they aren't Python lists. array.array contains uniform data, so conversion to D arrays is a memcpy (or it's nearly nothing if you don't copy the data). Bye, bearophile

I see. The docs for array.array suggest that it implements the buffer interface, but it doesn't seem to implement new or old style buffers, at least according to PyObject_CheckBuffer and PyBuffer_Check. I think I'll add support for new style buffers anyways. a memoryview would be good, too. Guess I'll hack together a special case for array using buffer_info.
Sep 06 2012
prev sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/06/2012 12:07 AM, Russel Winder wrote:
 I am guessing this is interfacing to CPython, remember there is also
 PyPy and ActiveState, they have different ways of doing things.  Well
 the ActiveState C API will be very close to the CPython C API, but PyPy
 (which is the best Python 2.7 just now) doesn't have a C API since it is
 written in RPython.

Yep, CPython for now. If activestate supports the CPython C API, then supporting it will probably just be a matter of linkage. Googling, it looks like PyPy is/was trying to grow some CPython C API compatibility, so hopefully we can squeeze it in at some point.
 Oh yes.

 NumPy is basically a subsystem that Python applications make calls into.
 Although the data structure can be accessed and amended, algorithms on
 the data structures should never be written in Python, they should
 always be function calls into the NumPy framework.

I had a look at numpy, and at least ndarray supports the new style buffer interface (which is pretty freaking complicated), so convertion is totally doable. I'll fold in support as soon as I can get dmd to stop stabbing me in the face. just used your scons fork to build the pyd embedded unittests. works pretty nice
Sep 07 2012
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/08/2012 03:09 AM, Russel Winder wrote:
 On Fri, 2012-09-07 at 15:21 -0700, Ellery Newcomer wrote:
 On 09/06/2012 12:07 AM, Russel Winder wrote:

 just used your scons fork to build the pyd embedded unittests. works
 pretty nice

Splendid :-)

Okay, here: https://bitbucket.org/ariovistus/deimos-elfutils/overview I have some code with a working makefile and a nonworking SConstruct file. I believe the issue is the header files have pragma(lib, X) in them, and a single call to dmd links the appropriate lib in, but scons' link step loses that information. Do you have any intention of supporting pragma(lib) in scons?
Sep 08 2012
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/09/2012 07:27 AM, bearophile wrote:
 Johannes Pfau:

 Putting linker flags into source files is just not a good idea:

Thank you for the explanation. Then maybe we should deprecate

 Bye,
 bearophile

Similar to pragmas in C and C++, D pragmas allow language extensions: "Pragmas are a way to pass special information to the compiler and to add vendor specific extensions to D." http://dlang.org/pragma.html Ali
Sep 09 2012
parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/09/2012 08:48 AM, Russel Winder wrote:
 On Sun, 2012-09-09 at 08:05 -0700, Ali Çehreli wrote:

     http://dlang.org/pragma.html

This states that: pragma(lib, "blah.lib") is a part of the D language and not just a DMD extension.

I hadn't read that far. :*)
 Personally I would say that putting dependencies such as this in the
 source code is anathema, it is the role of the build framework to deal
 with this.

Agreed. Ali
Sep 09 2012
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-09-06 04:10, bearophile wrote:

 There are several important cases, like:

 Some D lazy ranges <==> Python lazy iterators/generators

 array.array <==> D arrays

 NumPy arrays <==> D arrays

Associative arrays? -- /Jacob Carlborg
Sep 05 2012
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/05/2012 11:19 PM, Jacob Carlborg wrote:
 On 2012-09-06 04:10, bearophile wrote:

 There are several important cases, like:

 Some D lazy ranges <==> Python lazy iterators/generators

 array.array <==> D arrays

 NumPy arrays <==> D arrays

Associative arrays?

check. https://bitbucket.org/ariovistus/pyd/wiki/TypeConversion
Sep 06 2012
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/06/2012 09:48 AM, Ellery Newcomer wrote:
 On 09/05/2012 11:19 PM, Jacob Carlborg wr
 Associative arrays?

check.

eh, that was check as in yes, not check as in look it up yourself. didn't seem ambiguous at the time.
Sep 07 2012
prev sibling next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/05/2012 07:10 PM, bearophile wrote:
 NumPy arrays <==> D arrays

I've been thinking about this one a bit more, and I am not sure it belongs in pyd. First, the conversion is not symmetric. One can convert a numpy.ndarray to a d array like so: PyObject* ndarray; double[][] matrix = d_type!(double[][])(ndarray); however, going back PyObject* res = _py(matrix); It is not at all clear that the user wants res to be a numpy.ndarray. The problem is partially that d arrays would be overloaded to a few too many things (list, str, array, any iterable, any buffer). That last one is a doozy. d_type never actually touches ndarray's type, so _py can hardly know what to use to convert matrix. (what if ndarray is actually a foo.BizBar matrix?) I could just specialize _py for numpy.ndarrays, defaulting to lists of lists (which is what we do already), but I kinda want a specialized type for numpy.ndarrays. Also, all these conversions imply data copying; is this reasonable for numpy arrays? It is easy enough to get a void* and shape information out of the ndarray, but building a decent matrix type out of them is not trivial. Is there a good matrix library for D that would be suitable for this? Oh yeah, also: rectangular matrices. For static arrays, the conversion is 1 memcpy. For dynamic arrays: lots of memcpys. I suppose I could abuse slicing much.
Sep 10 2012
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/10/2012 12:11 PM, bearophile wrote:
 I understand. The point of Pyd is to interface D and Python, while NumPy
 is something external. So if you find difficulties just keep it out.
 Adding it later is possible.

Thing is, pyd will convert a ndarray to d array already, it just won't do it as quickly as it could if it made use of the underlying c array, and _py(d_type!(double[][])(ndarray)) will result in a list of lists. So it's really a question of should I add more oddness to an already odd situation. <OT> Bugger, I'm going to have to go through pyd and replace all usages of str with unicode. </OT>
Sep 10 2012
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/10/2012 10:50 PM, Russel Winder wrote:
 Python 2 and Python 3 are totally different in this regard. I don't have
 a obvious proposal to make to avoid having PyD for Python 2 and a
 different PyD for Python 3, but the six package might have some hints as
 it is intended to support creating Python codebases guaranteed to run
 under Python 2 and Python 3.

Pyd doesn't really have a python codebase, I was talking mostly about PyString_AsString -> PyUnicode_Whatever, since even for Python 2, unicode is much more appropriate for anything interfacing with D. For getting pyd to support python 3, its mostly a matter of choosing the right C API functions, and anyways I have version identifiers I can rely on if there is divergence. Wait, CeleriD is python. I might need that six package after all. Thanks for the tip.
Sep 11 2012
prev sibling parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/05/2012 07:10 PM, bearophile wrote:
 Ellery Newcomer:

 Yep.

Oh, good.
 Have any suggestions for supported conversion out of the box?

There are several important cases, like: Some D lazy ranges <==> Python lazy iterators/generators array.array <==> D arrays NumPy arrays <==> D arrays

Welp. I started on NumPy arrays <== D arrays, and it turned out to be pretty easy. It's in its own function; maybe I'll put it in pyd.extras or something. But now I have just about all of the above cases working. Bearophile: would you be interested in contributing some code showcasing what we can do with numpy? Just, say, a D function operating on D arrays that does something that maybe numpy doesn't have built in.
Sep 11 2012
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Ellery Newcomer:

 Yep.

Oh, good.
 Have any suggestions for supported conversion out of the box?

There are several important cases, like: Some D lazy ranges <==> Python lazy iterators/generators array.array <==> D arrays NumPy arrays <==> D arrays
 Also, which of the following looks more appealing to you?

I don't know. Bye, bearophile
Sep 05 2012
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Sorry, I missed earlier bits of this thread=E2=80=A6

On Wed, 2012-09-05 at 19:37 -0700, Ellery Newcomer wrote:
 On 09/05/2012 07:10 PM, bearophile wrote:
 Some D lazy ranges <=3D=3D> Python lazy iterators/generators

I'll look into this one. =20
 array.array <=3D=3D> D arrays

just checked, looks like we have it: =20 PyStmts(q"{from array import array; a =3D array('i', [44,33,22,11]);}",=

 "testing");
 assert(PyEval!(int[])("a", "testing") =3D=3D [44,33,22,11]);
=20
 I think if the python object is iterable, it can be converted to array.

I am guessing this is interfacing to CPython, remember there is also PyPy and ActiveState, they have different ways of doing things. Well the ActiveState C API will be very close to the CPython C API, but PyPy (which is the best Python 2.7 just now) doesn't have a C API since it is written in RPython.=20
 Matrices might pose a problem. But user can define custom conversions if=

 need be.
=20
 NumPy arrays <=3D=3D> D arrays

I have never used NumPy. Are its arrays special?

Oh yes. NumPy is basically a subsystem that Python applications make calls into. Although the data structure can be accessed and amended, algorithms on the data structures should never be written in Python, they should always be function calls into the NumPy framework.
 Also, which of the following looks more appealing to you?

I don't know.

ok, I'll be lazy then. =20

--=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Sep 06 2012
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Ellery Newcomer:

 array.array <==> D arrays

just checked, looks like we have it: PyStmts(q"{from array import array; a = array('i', [44,33,22,11]);}", "testing"); assert(PyEval!(int[])("a", "testing") == [44,33,22,11]); I think if the python object is iterable, it can be converted to array.

array.array are special, they aren't Python lists. array.array contains uniform data, so conversion to D arrays is a memcpy (or it's nearly nothing if you don't copy the data). Bye, bearophile
Sep 06 2012
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Ellery Newcomer:

 I had a look at numpy, and at least ndarray supports the new 
 style buffer interface (which is pretty freaking complicated), 
 so convertion is totally doable. I'll fold in support as soon 
 as I can get dmd to stop stabbing me in the face.

If you support NumPy well and efficiently through Pyd, I think some people will start using the D language just for this :-) NumPy is good but it usually forces you to program vectorially, like Matlab. Sometimes this is not nice, so people write some routines in Fortran, C, etc. With Pyd some scientific programmers maybe will want to use D instead. Bye, bearophile
Sep 07 2012
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sat, 2012-09-08 at 00:58 +0200, bearophile wrote:
[=E2=80=A6]
 If you support NumPy well and efficiently through Pyd, I think=20
 some people will start using the D language just for this :-)
=20
 NumPy is good but it usually forces you to program vectorially,=20
 like Matlab. Sometimes this is not nice, so people write some=20
 routines in Fortran, C, etc. With Pyd some scientific programmers=20
 maybe will want to use D instead.

The idea of a Python/NumPy/D combination, especially if connected to OpenCL does have great appeal. NumPy and D for computation and Python for data visualization and coordination would be an improvement over the currently canonical Python/NumPy/C/C++/Fortran. But there is a large inertia to the system that has built up in the bioinformatics and HEP communities. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Sep 08 2012
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, 2012-09-07 at 15:21 -0700, Ellery Newcomer wrote:
 On 09/06/2012 12:07 AM, Russel Winder wrote:

 just used your scons fork to build the pyd embedded unittests. works=20
 pretty nice

Splendid :-) If you see any problems or glitches, let me know and/or post a bug report. Although I said use the BitBucket issue system, that was a hang over from the days of a totally separate package for the D system. Now that D support is being evolved as a fork, it has been pointed out to me I should have pointed people to the SCons issue tracker http://scons.tigris.org/project_issues.html for posting reports about D tooling issues as they are in the SCons core. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Sep 08 2012
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sat, 2012-09-08 at 07:20 -0700, Ellery Newcomer wrote:
[=E2=80=A6]
 Okay, here: https://bitbucket.org/ariovistus/deimos-elfutils/overview
=20
 I have some code with a working makefile and a nonworking SConstruct file=

=20
 I believe the issue is the header files have pragma(lib, X) in them, and=

 a single call to dmd links the appropriate lib in, but scons' link step=

 loses that information.
=20
 Do you have any intention of supporting pragma(lib) in scons?

If that is valid Dv2 and SCons doesn't deal with it then the SCons D tools are broken and need fixing. Is there a tiny project replicating this that I can turn into a unit/system test. The red so caused will necessitate action :-) --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Sep 08 2012
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Sat, 08 Sep 2012 16:25:49 +0100
schrieb Russel Winder <russel winder.org.uk>:

 On Sat, 2012-09-08 at 07:20 -0700, Ellery Newcomer wrote:
 [=E2=80=A6]
 Okay, here:
 https://bitbucket.org/ariovistus/deimos-elfutils/overview
=20
 I have some code with a working makefile and a nonworking
 SConstruct file.
=20
 I believe the issue is the header files have pragma(lib, X) in
 them, and a single call to dmd links the appropriate lib in, but
 scons' link step loses that information.
=20
 Do you have any intention of supporting pragma(lib) in scons?

If that is valid Dv2 and SCons doesn't deal with it then the SCons D tools are broken and need fixing. =20 Is there a tiny project replicating this that I can turn into a unit/system test. The red so caused will necessitate action :-) =20

Please note that pragma(lib) is an evil feature. For example it will never work in gdc.
Sep 09 2012
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, 2012-09-09 at 10:15 +0200, Johannes Pfau wrote:
[=E2=80=A6]
 Please note that pragma(lib) is an evil feature. For example it will
 never work in gdc.

So this is a DMD-only (*) feature and not a feature of the D programming language per se? (*) and hence LDC. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Sep 09 2012
prev sibling next sibling parent " bearophile" <bearophileHUGS lycos.com> writes:
Johannes Pfau:

 pragma(lib) is an evil feature. For example it will never work 
 in gdc.

Do you know why it's impossible to implement it in GDC? And if it's impossible to implement in GDC, then maybe it's better to ask Walter to deprecate its usage in general. Bye, bearophile
Sep 09 2012
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Sun, 09 Sep 2012 13:53:23 +0200
schrieb " bearophile" <bearophileHUGS lycos.com>:

 
 Do you know why it's impossible to implement it in GDC?
 
 And if it's impossible to implement in GDC, then maybe it's
 better to ask Walter to deprecate its usage in general.
 
 Bye,
 bearophile

IIRC this is the reason: GCC programs are always split into 2 executables, for gdc it's 'gdc' and 'cc1d' (for c it's gcc and cc1, etc). This allows the gcc executable to compile d sources, you can actually call gcc file.d and it'll invoke cc1d. cc1d parses the source files and compiles code, so cc1d is the tool which knows about pragma(lib). but cc1d is not responsible for linking, linking is done by gdc. And IIRC there's no easy, portable way to pass the information from cc1d to gdc. So putting linker flags in source file doesn't fit gcc very well. Note that it may not work with dmd either when linking incrementally. Putting linker flags into source files is just not a good idea: test1.d ---- pragma(lib, "curl"); ---- test2.d ---- import std.net.curl; import test1; void main() { Curl c; } ---- dmd test1.d test2.d //OK dmd test1.d -c dmd test2.d test1.o //FAIL
Sep 09 2012
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Johannes Pfau:

 Putting linker flags into source files is just not a good idea:

Thank you for the explanation. Then maybe we should deprecate pragma(lib). Bye, bearophile
Sep 09 2012
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Sunday, 9 September 2012 at 14:26:48 UTC, bearophile wrote:
 Thank you for the explanation. Then maybe we should deprecate 
 pragma(lib).

If it doesn't work, there's no difference between having it and not - you have to do it yourself anyway. So deprecation accomplishes nothing except breaking it for the people who *do* use it. At least the pragma could be printed out to the user (treat it like pragma(msg) where lib isn't supported?) or read by some other tool even if the compiler doesn't use it.
Sep 09 2012
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, 2012-09-09 at 08:05 -0700, Ali =C3=87ehreli wrote:
[=E2=80=A6]
 Similar to pragmas in C and C++, D pragmas allow language extensions:=20
 "Pragmas are a way to pass special information to the compiler and to=20
 add vendor specific extensions to D."
=20
    http://dlang.org/pragma.html

This states that: pragma(lib, "blah.lib") is a part of the D language and not just a DMD extension. This means that the code is platform specific (due to extensions embedded in code) as well as predicating that the toolchain will deal with explicit toolchain information in the source code. Personally I would say that putting dependencies such as this in the source code is anathema, it is the role of the build framework to deal with this. That way you can deal with the platform specific extensions and naming as well as toolchain issues. These dependencies are link time not source time ones. So I am with Johannes, pragma(lib,=E2=80=A6) is "evil" and should be remove= d. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Sep 09 2012
prev sibling next sibling parent Brad Roberts <braddr puremagic.com> writes:
On 9/9/2012 1:15 AM, Johannes Pfau wrote:
 Am Sat, 08 Sep 2012 16:25:49 +0100
 schrieb Russel Winder <russel winder.org.uk>:
 
 On Sat, 2012-09-08 at 07:20 -0700, Ellery Newcomer wrote:
 […]
 Okay, here:
 https://bitbucket.org/ariovistus/deimos-elfutils/overview

 I have some code with a working makefile and a nonworking
 SConstruct file.

 I believe the issue is the header files have pragma(lib, X) in
 them, and a single call to dmd links the appropriate lib in, but
 scons' link step loses that information.

 Do you have any intention of supporting pragma(lib) in scons?

If that is valid Dv2 and SCons doesn't deal with it then the SCons D tools are broken and need fixing. Is there a tiny project replicating this that I can turn into a unit/system test. The red so caused will necessitate action :-)

Please note that pragma(lib) is an evil feature. For example it will never work in gdc.

It's not impossible and never is rather defeatist. Using the frontend as is and grabing the json output, part of which includes the pragmas, would be easy. Then invoking gdc with the appropriate options to get the library linked in. rdmd is a good example of this sort of process.
Sep 09 2012
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Sun, 09 Sep 2012 12:55:19 -0700
schrieb Brad Roberts <braddr puremagic.com>:

 On 9/9/2012 1:15 AM, Johannes Pfau wrote:
 Am Sat, 08 Sep 2012 16:25:49 +0100
 schrieb Russel Winder <russel winder.org.uk>:
=20
 On Sat, 2012-09-08 at 07:20 -0700, Ellery Newcomer wrote:
 [=E2=80=A6]
 Okay, here:
 https://bitbucket.org/ariovistus/deimos-elfutils/overview

 I have some code with a working makefile and a nonworking
 SConstruct file.

 I believe the issue is the header files have pragma(lib, X) in
 them, and a single call to dmd links the appropriate lib in, but
 scons' link step loses that information.

 Do you have any intention of supporting pragma(lib) in scons?

If that is valid Dv2 and SCons doesn't deal with it then the SCons D tools are broken and need fixing. Is there a tiny project replicating this that I can turn into a unit/system test. The red so caused will necessitate action :-)

Please note that pragma(lib) is an evil feature. For example it will never work in gdc. =20

It's not impossible and never is rather defeatist. Using the frontend as is and grabing the json output, part of which includes the pragmas, would be easy. Then invoking gdc with the appropriate options to get the library linked in. rdmd is a good example of this sort of process. =20 =20

Is there a special flag to enable pragmas for the json output? It does not work with gdc right now, but it should be possible to make it work. Sorry, I should have said 'It'll _probably_ never be supported in gdc'. There are some possible solutions but: * It must be good enough to get approved when gdc is merged into gcc. (remember it must be portable and gpl and you can't use stdout/stdin...) * Someone would have to implement the solution. I guess Iain had his reasons not to implement it so somebody else would have to do that. Of course you can always try to make it work with external build tools. But a solution _in_ gdc seems not very likely. I don't want to badmouth the pragma(lib) feature, in some cases it's nice to have (mainly building simple script-like programs with few source files). But for bigger projects, pragma(lib) makes things difficult (incremental compilation; build tools usually check if a library is available before trying to link against it so they can put out a nice warning. pragma(lib) in dmd subverts this feature; can't specify linker path with pragma lib, can't specify static vs dynamic linking, ...). The C/C++ architecture splits compilation and linking. Trying to conflate those concepts as pragma(lib) does, might even be a good idea(other languages have done it for some time now). But as we have to deal with tools that were designed for C/C++ (linkers, gcc) we'll always hit some issues with pragma(lib).
Sep 10 2012
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Mon, 10 Sep 2012 14:48:30 +0200
schrieb Johannes Pfau <nospam example.com>:

 Sorry, I should have said 'It'll _probably_ never be supported in
 gdc'. There are some possible solutions but:
 
 * It must be good enough to get approved when gdc is merged into gcc.
   (remember it must be portable and gpl and you can't use
   stdout/stdin...)
 * Someone would have to implement the solution. I guess Iain had his
   reasons not to implement it so somebody else would have to do that.
 
 Of course you can always try to make it work with external build
 tools. But a solution _in_ gdc seems not very likely.

For reference: Here's the gdc bug report for pragma(lib): http://d.puremagic.com/issues/show_bug.cgi?id=1690 Filed 2007, closed 2012 as RESOLVED/WONTFIX.
Sep 10 2012
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Ellery Newcomer:

 I've been thinking about this one a bit more, and I am not sure 
 it belongs in pyd.

I understand. The point of Pyd is to interface D and Python, while NumPy is something external. So if you find difficulties just keep it out. Adding it later is possible. Bye, bearophile
Sep 10 2012
prev sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2012-09-10 at 15:54 -0700, Ellery Newcomer wrote:
[=E2=80=A6]
 <OT> Bugger, I'm going to have to go through pyd and replace all usages=

 of str with unicode. </OT>

Python 2 and Python 3 are totally different in this regard. I don't have a obvious proposal to make to avoid having PyD for Python 2 and a different PyD for Python 3, but the six package might have some hints as it is intended to support creating Python codebases guaranteed to run under Python 2 and Python 3. It is a pity the world doesn't just spontaneously switch to Python 3 so that Python 2 is just a "we used to use that" technology." --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Sep 10 2012