digitalmars.D.announce - autowrap v0.0.1 - Automatically wrap existing D code for use in Python

Atila Neves (35/35) Apr 18 2018 http://code.dlang.org/packages/autowrap

jmh530 (2/6) Apr 18 2018 Cool. I bet something similar would work with embedr as well.
karita (4/8) Apr 20 2018 Awesome. I'm also working on numpy.ndarray <-> mir.ndslice
Laeeth Isharc (2/9) Apr 23 2018 https://www.reddit.com/r/programming/comments/8eb12m/autowrap_v001_autom...
Nikos (15/15) May 10 2018 Interesting stuff.
Nikos (3/7) May 10 2018 which still tries to find python36. Why doesn't it look for 3.5?

Laeeth Isharc (3/10) May 10 2018 Hi. On my phone so can't copy paste. Edit your dub.sdl under
Atila Neves (3/10) May 11 2018 Copy + paste error, sorry. Fixed now.

Nikos (2/17) May 12 2018 That worked like a charm! Thanks man!
Nikos (9/22) May 13 2018 I'm trying to wrap drepl

Laeeth Isharc (4/27) May 13 2018 It won't wrap a templated type or function. You will have to
Laeeth Isharc (3/26) May 13 2018 Eg turn this into a function and try wrapping this instead:

Nikos (4/9) Jun 24 2018 Thanks for your help. I'm doing this in my spare time which is,
Nikos (1/1) Jul 29 2018 Ok, I made a stupid mistake. It works now. Thanks a lot!
Nikos (11/25) Jul 29 2018 and tested it in ipython successfully.

Nicholas Wilson (22/28) Jul 31 2018 Can you export an instance of `interpreter(dmdEngine())`?

jmh530 (2/5) Jul 31 2018 Excellent.
Nikos (7/38) Aug 05 2018 Thank you very much for your feedback. Unfortunately, none of the

Laeeth Isharc (3/53) Aug 05 2018 Working example of Python calling D Repl is here.
Nicholas Wilson (20/28) Aug 05 2018 Sure https://github.com/thewilsonator/jupyterd

Nicholas Wilson (4/5) Aug 09 2018 The echo interpreter now works, and the dmd one is there. I've

Atila Neves <atila.neves gmail.com> writes:

http://code.dlang.org/packages/autowrap

This came out of the need at work to take existing D code and 
make it available for both Excel and Python.

Both pyd and excel-d make the reasonable assumption that one is 
using them to write code specifically for those environments. 
That breaks when there's existing production D code one wants to 
wrap.

The idea is to not "dirty" the existing code with dependencies on 
either pyd or excel-d and instead wrap them from outside the 
existing dub packages, whilst still using pyd and excel-d behind 
the scenes.

The end result is that if you have two D modules called 
`my.module1` and `my.module2` and you want to create a Python 
extension called `mylibrary`, then one dub.sdl and one 
source/app.d such as this is enough:


     import autowrap.python;
     mixin(
         wrapAll(
             LibraryName("mylibrary"),
             Modules("my.module1", "my.module2", /* ... */),
         )
     );

Seriously, that's it.* Well, other than the fact that dub will 
produce libmylibrary.so on Linux and what you need is 
mylibrary.so, so you'll have to rename the file.

The functions that are to be enabled for wrapping must be marked 
`export`, both for Python and Excel. It is a form of tagging and 
the production code must be changed to accomodate it, but at 
least it introduces no extra dependencies.

For Python, any struct used in an `export` function's parameters 
or return type is automatically wrapped, as well as any structs 
inside it.


Atila


* Except maybe for those pesky bug things I can't seem to get rid 
of.

Apr 18 2018

jmh530 <john.michael.hall gmail.com> writes:

On Wednesday, 18 April 2018 at 15:28:07 UTC, Atila Neves wrote:
 http://code.dlang.org/packages/autowrap

 This came out of the need at work to take existing D code and 
 make it available for both Excel and Python.

 [snip]

Cool. I bet something similar would work with embedr as well.

Apr 18 2018

karita <shigekikarita gmail.com> writes:

On Wednesday, 18 April 2018 at 15:28:07 UTC, Atila Neves wrote:
 http://code.dlang.org/packages/autowrap

 This came out of the need at work to take existing D code and 
 make it available for both Excel and Python.

 [...]

Awesome. I'm also working on numpy.ndarray <-> mir.ndslice 
automatic wrapper

https://github.com/ShigekiKarita/mir-pybuffer

Apr 20 2018

Laeeth Isharc <laeeth laeeth.com> writes:

On Wednesday, 18 April 2018 at 15:28:07 UTC, Atila Neves wrote:
 http://code.dlang.org/packages/autowrap

 This came out of the need at work to take existing D code and 
 make it available for both Excel and Python.

 Both pyd and excel-d make the reasonable assumption that one is 
 using them to write code specifically for those environments. 
 That breaks when there's existing production D code one wants 
 to wrap.

https://www.reddit.com/r/programming/comments/8eb12m/autowrap_v001_automatically_wrap_existing_d_code/

Apr 23 2018

Nikos <nikos home.com> writes:

Interesting stuff.

In http://code.dlang.org/packages/autowrap it says:

"""
  Python versions

Since autowrap depends on PyD, the python version must be 
explicitly stated as a dub configuration and defaults to 3.6. To 
use another version, pass -c $CONFIG to dub where $CONFIG is one 
of:

     python27
     python34
     python35
     python36
"""

Could you please provide an example of how can I do this?

Thanks

May 10 2018

Nikos <nikos home.com> writes:

In my dub.sdl file I have

 configuration "python35" {
  subConfiguration "autowrap" "python35"
}

and I run

 dub build --config=python35

which still tries to find python36. Why doesn't it look for 3.5?

May 10 2018

Laeeth Isharc <laeethnospam nospam.laeeth.com> writes:

On Thursday, 10 May 2018 at 19:50:40 UTC, Nikos wrote:
 In my dub.sdl file I have

 configuration "python35" {
  subConfiguration "autowrap" "python35"
}

 and I run

 dub build --config=python35

 which still tries to find python36. Why doesn't it look for 3.5?

Hi.  On my phone so can't copy paste.  Edit your dub.sdl under 
the python35 subconfiguration and change python 36 to python35.

May 10 2018

Atila Neves <atila.neves gmail.com> writes:

On Thursday, 10 May 2018 at 19:50:40 UTC, Nikos wrote:
 In my dub.sdl file I have

 configuration "python35" {
  subConfiguration "autowrap" "python35"
}

 and I run

 dub build --config=python35

 which still tries to find python36. Why doesn't it look for 3.5?

Copy + paste error, sorry. Fixed now.

Atila

May 11 2018

Nikos <nikos home.com> writes:

On Friday, 11 May 2018 at 14:16:15 UTC, Atila Neves wrote:
 On Thursday, 10 May 2018 at 19:50:40 UTC, Nikos wrote:
 In my dub.sdl file I have

 configuration "python35" {
  subConfiguration "autowrap" "python35"
}

 and I run

 dub build --config=python35

 which still tries to find python36. Why doesn't it look for 
 3.5?

 Copy + paste error, sorry. Fixed now.

 Atila

That worked like a charm! Thanks man!

May 12 2018

Nikos <nikos home.com> writes:

I'm trying to wrap drepl 
(https://github.com/dlang-community/drepl)

My dub.sdl files is

import autowrap.python;
mixin(
    wrapAll(
        LibraryName("drepl"),
        Modules("drepl.interpreter"),
    )
);

I also flagged `export` the interpreter function

export Interpreter!Engine interpreter(Engine)(return scope 
Engine e) if (isEngine!Engine)
{
    // workaround Issue 18540
    return Interpreter!Engine(()  trusted { return move(e); }());
}

I build the library with python35, but when I import it from 
python idle, I cannot access the `interpreter` function at all.
I have the feeling I miss something essential here, but I don't 
know what it is.
Any ideas?

May 13 2018

Laeeth Isharc <laeeth laeeth.com> writes:

On Sunday, 13 May 2018 at 16:23:49 UTC, Nikos wrote:
 I'm trying to wrap drepl 
 (https://github.com/dlang-community/drepl)

 My dub.sdl files is

import autowrap.python;
mixin(
    wrapAll(
        LibraryName("drepl"),
        Modules("drepl.interpreter"),
    )
);

 I also flagged `export` the interpreter function

export Interpreter!Engine interpreter(Engine)(return scope 
Engine e) if (isEngine!Engine)
{
    // workaround Issue 18540
    return Interpreter!Engine(()  trusted { return move(e); 
 }());
}

 I build the library with python35, but when I import it from 
 python idle, I cannot access the `interpreter` function at all.
 I have the feeling I miss something essential here, but I don't 
 know what it is.
 Any ideas?

It won't wrap a templated type or function.  You will have to 
make a little shim that instantiates the templated function with 
a particular type.

May 13 2018

Laeeth Isharc <laeeth laeeth.com> writes:

On Sunday, 13 May 2018 at 16:23:49 UTC, Nikos wrote:
 I'm trying to wrap drepl 
 (https://github.com/dlang-community/drepl)

 My dub.sdl files is

import autowrap.python;
mixin(
    wrapAll(
        LibraryName("drepl"),
        Modules("drepl.interpreter"),
    )
);

 I also flagged `export` the interpreter function

export Interpreter!Engine interpreter(Engine)(return scope 
Engine e) if (isEngine!Engine)
{
    // workaround Issue 18540
    return Interpreter!Engine(()  trusted { return move(e); 
 }());
}

 I build the library with python35, but when I import it from 
 python idle, I cannot access the `interpreter` function at all.
 I have the feeling I miss something essential here, but I don't 
 know what it is.
 Any ideas?

Eg turn this into a function and try wrapping this instead:

     auto intp = interpreter(dmdEngine());

May 13 2018

Nikos <nikoskaragiannakis gmail.com> writes:

 Eg turn this into a function and try wrapping this instead:

     auto intp = interpreter(dmdEngine());

Thanks for your help. I'm doing this in my spare time which is, 
unfortunately, not much.
I did what you said

 export {
     auto intp = interpreter(echoEngine);
 }


but when I import drepl from python idle I still can access intp.

Jun 24 2018

Nikos <nikoskaragiannakis gmail.com> writes:

Ok, I made a stupid mistake. It works now. Thanks a lot!

Jul 29 2018

Nikos <nikoskaragiannakis gmail.com> writes:

 Eg turn this into a function and try wrapping this instead:

     auto intp = interpreter(dmdEngine());

Actually, I manage to export the `interpret` method

 export:
     auto intp(char[] txt) {
         return interpreter(dmdEngine()).interpret(txt);
     }

and tested it in ipython successfully.

But when I try to export the whole dmdEngine

 export:

    auto engine(char[] txt) {
            return interpreter(dmdEngine());
    }

it complains about copying Interpreter!(DMDEngine).Interpreter

 ../../../.dub/packages/pyd-master/pyd/infrastructure/pyd/ma
e_object.d(249,30): Error: struct
drepl.interpreter.Interpreter!(DMDEngine).Interpreter is not copyable because
it is annotated with  disable

I removed  disable, but then complained about accessing the 
members `_engine` and `_incomplete` in Interpreter 
(https://github.com/dlang-community/drepl/blob/master/src/drepl/interpreter.d#L147-L148)

 ../../../.dub/packages/pyd-master/pyd/infrastructure/pyd/struct_wra
.d-mixin-56(56,15): Deprecation: std.array.Appender!(char[]).Appender._data is
not visible from module
../../../.dub/packages/pyd-master/pyd/infrastructure/pyd/struct_wra
.d-mixin-56(56,15): Error: struct std.array.Appender!(char[]).Appender member
_data is not accessible

After I made those public, it complained about `Appender`

 ../../../.dub/packages/pyd-master/pyd/infrastructure/pyd/struct_wra
.d-mixin-56(56,15): Deprecation: std.array.Appender!(char[]).Appender._data is
not visible from module
 ../../../.dub/packages/pyd-master/pyd/infrastructure/pyd/struct_wra
.d-mixin-56(56,15): Error: struct std.array.Appender!(char[]).Appender member
_data is not accessible

Is there something I can do here or would it better to talk to 
the Drepl guys?

Thank you

Jul 29 2018

Nicholas Wilson <iamthewilsonator hotmail.com> writes:

On Sunday, 29 July 2018 at 18:14:31 UTC, Nikos wrote:
 But when I try to export the whole dmdEngine

 export:

    auto engine(char[] txt) {
            return interpreter(dmdEngine());
    }



Can you export an instance of `interpreter(dmdEngine())`?

e.g.

__gshared auto dmdi = interpreter(dmdEngine());

export ref dmd()
{
     return dmdi;
}

or if that doesn't work, proxy it

__gshared auto dmdi = interpreter(dmdEngine());

struct Dmd
{
     mixin Proxy!dmdi;
}
export auto dmd()
{
     Dmd d;
     return d;
}

That is pretty much required if you want to maintain state across.

Also I'm working on a D kernel for Jupyter notebook which should 
be done soon.

Jul 31 2018

jmh530 <john.michael.hall gmail.com> writes:

On Tuesday, 31 July 2018 at 09:09:11 UTC, Nicholas Wilson wrote:
 [snip]

 Also I'm working on a D kernel for Jupyter notebook which 
 should be done soon.

Excellent.

Jul 31 2018

Nikos <nikoskaragiannakis gmail.com> writes:

On Tuesday, 31 July 2018 at 09:09:11 UTC, Nicholas Wilson wrote:
 On Sunday, 29 July 2018 at 18:14:31 UTC, Nikos wrote:
 But when I try to export the whole dmdEngine

 export:

    auto engine(char[] txt) {
            return interpreter(dmdEngine());
    }



 Can you export an instance of `interpreter(dmdEngine())`?

 e.g.

 __gshared auto dmdi = interpreter(dmdEngine());

 export ref dmd()
 {
     return dmdi;
 }

 or if that doesn't work, proxy it

 __gshared auto dmdi = interpreter(dmdEngine());

 struct Dmd
 {
     mixin Proxy!dmdi;
 }
 export auto dmd()
 {
     Dmd d;
     return d;
 }

 That is pretty much required if you want to maintain state 
 across.

 Also I'm working on a D kernel for Jupyter notebook which 
 should be done soon.


Thank you very much for your feedback. Unfortunately, none of the 
above worked.
By the way, the reason I'm trying all this is to create a Jupyter 
notebook. I've already made a simple version of it some time ago 
(https://github.com/nikoskaragiannakis/d-jupyter-kernel). Since 
you are also working on a D kernel, maybe we could work together?

Aug 05 2018

Laeeth Isharc <laeeth laeeth.com> writes:

On Sunday, 5 August 2018 at 20:01:22 UTC, Nikos wrote:
 On Tuesday, 31 July 2018 at 09:09:11 UTC, Nicholas Wilson wrote:
 On Sunday, 29 July 2018 at 18:14:31 UTC, Nikos wrote:
 But when I try to export the whole dmdEngine

 export:

    auto engine(char[] txt) {
            return interpreter(dmdEngine());
    }



 Can you export an instance of `interpreter(dmdEngine())`?

 e.g.

 __gshared auto dmdi = interpreter(dmdEngine());

 export ref dmd()
 {
     return dmdi;
 }

 or if that doesn't work, proxy it

 __gshared auto dmdi = interpreter(dmdEngine());

 struct Dmd
 {
     mixin Proxy!dmdi;
 }
 export auto dmd()
 {
     Dmd d;
     return d;
 }

 That is pretty much required if you want to maintain state 
 across.

 Also I'm working on a D kernel for Jupyter notebook which 
 should be done soon.


 Thank you very much for your feedback. Unfortunately, none of 
 the above worked.
 By the way, the reason I'm trying all this is to create a 
 Jupyter notebook. I've already made a simple version of it some 
 time ago 
 (https://github.com/nikoskaragiannakis/d-jupyter-kernel). Since 
 you are also working on a D kernel, maybe we could work 
 together?

Working example of Python calling D Repl is here.

https://github.com/kaleidicassociates/pydrepl

Aug 05 2018

Nicholas Wilson <iamthewilsonator hotmail.com> writes:

On Sunday, 5 August 2018 at 20:01:22 UTC, Nikos wrote:
 Thank you very much for your feedback. Unfortunately, none of 
 the above worked.
 By the way, the reason I'm trying all this is to create a 
 Jupyter notebook. I've already made a simple version of it some 
 time ago 
 (https://github.com/nikoskaragiannakis/d-jupyter-kernel). Since 
 you are also working on a D kernel, maybe we could work 
 together?

Sure https://github.com/thewilsonator/jupyterd

There are 5 main objects:

Channel, an abstraction over raw ZMQ sockets.
Interpreter, abstracted for debugging and also other kernels (I 
work for Laeeth and they have an internal language they wan't to 
use from this.)
Kernel, does the higher level network logic and calls the 
interpreter.
Message and Wire message are high and low level representation of 
the packet layout.

I'm currently trying to figure out why the message signing is 
bjorked (I swear it did use to work!), the IOPub channel fails 
intermittently (the worst kind) and the Shell/Control fails all 
of the time.

Any effort pointing out where I done something stupid is much 
appreciated (and I do mean something stupid, the last major 
hurdle was trying to figure out why the packets were going 
nowhere: turns out I was missing the routing prefix).

Are you on slack?

Aug 05 2018

Nicholas Wilson <iamthewilsonator hotmail.com> writes:

On Monday, 6 August 2018 at 00:47:12 UTC, Nicholas Wilson wrote:
 Sure https://github.com/thewilsonator/jupyterd

The echo interpreter now works, and the dmd one is there. I've 
got no idea if it works because I'm on OSX (yay no shared library 
support) if someone wants to test it that would be great.

Aug 09 2018

D Programming

C/C++ Programming

Other

digitalmars.D.announce - autowrap v0.0.1 - Automatically wrap existing D code for use in Python