www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - C++ Binding Generator

reply Paul O'Neil <redballoon36 gmail.com> writes:
I've been thinking about how to generate bindings for C++ (esp. Qt!) for 
a little while now.  Unfortuneatly, I won't have time to implement 
anything until my semester is over, but with the recent talk of 
projects, maybe someone else will pick it up before then.

Qt bindings come up every so often; they are not going to be written 
manually.  The existing Qdd[1] was a port of Qt Jambi[2], which was the 
Java binding.  This makes some sense, since Java is also an 
object-oriented language that isn't C++.  Qt Jambi consists of two 
parts: a generator and generated code.  The generator reads the Qt 
headers as well as XML files that describe how to map the C++ into Java. 
  The generator creates both C++ and Java files that get compiled into a 
JNI library.  Given that D is conceptually closer to C++ than Java, I 
think that binding to D should be easier than Java.  Another issue here 
is that both Qt Jambi and QtD target Qt 4 and not version 5.

Also relevant is that the libclang bindings in Deimos[3] are 2 years and 
several versions old (3.1 -> 3.4 is out).  This is probably due to the 
lack of interest in libclang from D programmers (No surprise.  Who wants 
to write code in D to parse C(++)?).  It binds to libclang's C interface 
and may have been generated with DStep.

Here's my proposed path forward:
Write a binding generator in C++ that uses libclang to parse headers and 
auxillary XML / json / etc. files to guide the generation of D source. 
Clang provides data structures for traversing the C++ AST.  This tool 
could be used to bind Qt 5 and libclang so the conversion tool can 
eventually be migrated to D.  My hope is that using clang to avoid 
writing another C++ parser and D's similarity to C++ make this an easier 
(though still significant) undertaking than the Java bindings.

I suspect that there are several patterns in C++ and D that are 
implemented using slightly different constructions in each of them.  For 
instance, figuring out which C++ classes should map to D structs. 
Automatically identifying these or finding a description language for 
them would be a huge benefit to automation.

On the other hand, I don't yet see a way to map generic containers (such 
as QList<T>) into D yet.  But Qt Jambi found some way to solve this 
problem even when dealing with type erasure in Java.

Thoughts?

Thanks for reading my wall of text.
Paul O'Neil

[1]
QtD is not actively maintained.  It existed on
BitBucket: https://bitbucket.org/qtd/repo
before someone else forked it and moved to
GitHub: https://github.com/qtd-developers/qtd

[2]
Homepage (appears dead): http://qt-jambi.org/
Gitorious (shows recent activity): https://qt.gitorious.org/qt-jambi

[3]
Github: https://github.com/D-Programming-Deimos/libclang

[4]
Clang: http://clang.llvm.org
An example of using Clang to find CXXRecordDecl (classes, unions, 
structs, and the like):
http://clang.llvm.org/docs/RAVFrontendAction.html
Feb 14 2014
next sibling parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Friday, 14 February 2014 at 23:52:43 UTC, Paul O'Neil wrote:
 I've been thinking about how to generate bindings for C++ (esp. 
 Qt!) for a little while now.  Unfortuneatly, I won't have time 
 to implement anything until my semester is over, but with the 
 recent talk of projects, maybe someone else will pick it up 
 before then.

Why wouldn't you want to expand on DStep and add support for C++, it already uses clang?
Feb 14 2014
parent Jacob Carlborg <doob me.com> writes:
On 2014-02-15 04:04, Jesse Phillips wrote:

 Why wouldn't you want to expand on DStep and add support for C++, it
 already uses clang?

Yes, it's using libclang. -- /Jacob Carlborg
Feb 15 2014
prev sibling next sibling parent reply "michaelc37" <michaelc37 msn.com> writes:
 [1]
 QtD is not actively maintained.  It existed on
 BitBucket: https://bitbucket.org/qtd/repo
 before someone else forked it and moved to
 GitHub: https://github.com/qtd-developers/qtd

Probably worth mentioning. There are also some recent startup efforts to generate qt bindings based on smoke qt. https://github.com/w0rp/dqt https://github.com/GlobecSys/smoke.d However, its qt4.
Feb 14 2014
parent reply Xavier Bigand <flamaros.xavier gmail.com> writes:
Le 17/02/2014 01:36, w0rp a écrit :
 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 wrote:
 I'll try Qt5 later. The main reason I'm trying to get Qt4 working at the
 moment is that I have experience with Qt4, but not with Qt5, and from my
 understanding a lot of the advantages of Qt5 come from QML, and I don't
 quite understand yet how QML would tie together with a D API.

If someone is interested by Qt mainly for QML, maybe the better way is to help us on DQuick, cause it's really similar, but we try to take advantages of D to do something more friendly.
Feb 16 2014
next sibling parent Xavier Bigand <flamaros.xavier gmail.com> writes:
Le 17/02/2014 08:48, w0rp a écrit :
 On Monday, 17 February 2014 at 01:16:44 UTC, Xavier Bigand wrote:
 Le 17/02/2014 01:36, w0rp a écrit :
 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 wrote:
 I'll try Qt5 later. The main reason I'm trying to get Qt4 working at the
 moment is that I have experience with Qt4, but not with Qt5, and from my
 understanding a lot of the advantages of Qt5 come from QML, and I don't
 quite understand yet how QML would tie together with a D API.

If someone is interested by Qt mainly for QML, maybe the better way is to help us on DQuick, cause it's really similar, but we try to take advantages of D to do something more friendly.

Writing a native D GUI library instead of wrapping a C++ one is a good idea. My big problem with the wrapper around Qt I'm building is that it will never run as efficiently as a Qt application written in C++ would, so that might turn some people away. The main advantage of writing the wrapper is pretty much that it's just less work, as creating a cross platform toolkit with an API as nice as Qt's takes a lot of time and effort.

Yep, that why we are focus on the smallest part of Qt needed to create GUI. IMO all features of QtCore have to be directly in phobos (Network, serialization, signals,...), so only Widgets and QtQuick API stay here. The advantage of QtQuick his the modernity and it stay really small, it's mainly a script engine (property binding) coupled with a 2D modern renderer based on rasterization. We took lua as script language cause it can support property bindings without modifications. And the renderer just wait for contributions, the minimal functional target isn't really hard to get. But polish stay hard to do.
Feb 17 2014
prev sibling next sibling parent Xavier Bigand <flamaros.xavier gmail.com> writes:
Le 17/02/2014 09:18, eles a écrit :
 On Monday, 17 February 2014 at 07:48:51 UTC, w0rp wrote:
 On Monday, 17 February 2014 at 01:16:44 UTC, Xavier Bigand wrote:
 Le 17/02/2014 01:36, w0rp a écrit :
 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 wrote:



 Writing a native D GUI library instead of wrapping a C++ one is a good
 idea. My big problem with the wrapper around Qt I'm building is that
 it will never run as efficiently as a Qt application written in C++
 would, so that might turn some people away. The main advantage of
 writing the wrapper is pretty much that it's just less work, as
 creating a cross platform toolkit with an API as nice as Qt's takes a
 lot of time and effort.

I agree with this but, still, a wrapper around Qt 5 (not 4!) would be a huge thing, especially if one wants to write Android Qt-based applications.

I can tell you Qt 5 miss some features for Android or have some bugs. I wrote some java bindings to solve Qt issues. It's not a big deal. The main issue is the port of D and phobos on Android. With DQuick we target openGL 2.1 to be compatible with openGL ES. We have iOS and Android in mind too.
Feb 17 2014
prev sibling parent Xavier Bigand <flamaros.xavier gmail.com> writes:
Le 17/02/2014 09:54, Abdulhaq a écrit :
 On Monday, 17 February 2014 at 01:16:44 UTC, Xavier Bigand wrote:
 Le 17/02/2014 01:36, w0rp a écrit :
 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 wrote:
 I'll try Qt5 later. The main reason I'm trying to get Qt4 working at the
 moment is that I have experience with Qt4, but not with Qt5, and from my
 understanding a lot of the advantages of Qt5 come from QML, and I don't
 quite understand yet how QML would tie together with a D API.

If someone is interested by Qt mainly for QML, maybe the better way is to help us on DQuick, cause it's really similar, but we try to take advantages of D to do something more friendly.

From my personal point of view my main interest is in the programmatic Qt constructions (ie. Core & Widget module, not QML) as that's my itch that needs scratching. I'm not applying any effort ATM into the QML side of Qt. I think that a native DQuick implementation would be great and would probably provide a very compelling alternative to a wrapped QML (if that ever gets done by anybody) so I hope you get lets of help from anyone interested in that side of things.

I am completely agree with you, binding of Qt must be focused on Widgets and the core module. I am dreaming to be able to convince some QtQuick developers to contribute to DQuick, we already have some things working better. I am thinking to the property binding loop detection on which we are able to print the full call stack, where QML engine is only able to give the item not the property affected. And with the power of D it's not necessary to modify existent code, here with Qt wrappers are needed.
Feb 17 2014
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
On Sat, 2014-02-15 at 03:36 +0000, michaelc37 wrote:
 [1]
 QtD is not actively maintained.  It existed on
 BitBucket: https://bitbucket.org/qtd/repo
 before someone else forked it and moved to
 GitHub: https://github.com/qtd-developers/qtd


There is https://bitbucket.org/michaelc37/qtd-experimental as well. This was a bit active for a while but has gone quiet recently.
 Probably worth mentioning.
 There are also some recent startup efforts to generate qt 
 bindings based on smoke qt.
 
 https://github.com/w0rp/dqt
 https://github.com/GlobecSys/smoke.d
 
 However, its qt4.

If using Qt let's stick to Qt5 and not create brand new legacy APIs. In the Pythonverse, Riverbank have some mechanism for creating their Python bindings, but I bet the tool is proprietary. PySide used some binding generation tool that appears to have to be rewritten for Qt5 so they are staying with Qt4. This has stopped the PyQt → PySide rush for licencing reasons, people are just using PyQt5 and finding ways round the licencing problems. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Feb 14 2014
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-02-15 00:52, Paul O'Neil wrote:

 Also relevant is that the libclang bindings in Deimos[3] are 2 years and
 several versions old (3.1 -> 3.4 is out).  This is probably due to the
 lack of interest in libclang from D programmers (No surprise.  Who wants
 to write code in D to parse C(++)?).

I originally create those bindings to be used by DStep. But never bother updating them after that. DStep contains its own bindings. I haven't updated the bindings to anything later than 3.1 since I don't use any of these features yet.
 It binds to libclang's C interface  and may have been generated with DStep.

Actually, I just did some simple search-and-replace on the original C header. libclang is the easiest binding I've created :)
 Here's my proposed path forward:
 Write a binding generator in C++ that uses libclang to parse headers and
 auxillary XML / json / etc. files to guide the generation of D source.
 Clang provides data structures for traversing the C++ AST.  This tool
 could be used to bind Qt 5 and libclang so the conversion tool can
 eventually be migrated to D.  My hope is that using clang to avoid
 writing another C++ parser and D's similarity to C++ make this an easier
 (though still significant) undertaking than the Java bindings.

Why not contribute to DStep?
 I suspect that there are several patterns in C++ and D that are
 implemented using slightly different constructions in each of them.  For
 instance, figuring out which C++ classes should map to D structs.
 Automatically identifying these or finding a description language for
 them would be a huge benefit to automation.

 On the other hand, I don't yet see a way to map generic containers (such
 as QList<T>) into D yet.  But Qt Jambi found some way to solve this
 problem even when dealing with type erasure in Java.

You could hard code a couple of these containers. QList<T> could be converted to T[] or a linked list. I've done that with a couple of other types: like "BOOL" is translated to "bool" and "wchar_t" is translated to "wchar". -- /Jacob Carlborg
Feb 15 2014
parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Saturday, 15 February 2014 at 11:25:09 UTC, Jacob Carlborg 
wrote:
 and "wchar_t" is translated to "wchar".

That doesn't seem correct, it should be `wchar` on Windows only, and `dchar` on other systems, i.e. it should translate to core.stdc.stddef.wchar_t.
Feb 15 2014
parent Jacob Carlborg <doob me.com> writes:
On 2014-02-15 12:54, Jakob Ovrum wrote:

 That doesn't seem correct, it should be `wchar` on Windows only, and
 `dchar` on other systems, i.e. it should translate to
 core.stdc.stddef.wchar_t.

I was mistaken. Currently DStep recognizes the "wchar_t" typedef, then it checks the size of it, translating to either "wchar" or "dchar". This is correct but will not generate cross-platform bindings. core.stdc.stddef.wchar_t seems like a better choice. Thanks. -- /Jacob Carlborg
Feb 15 2014
prev sibling next sibling parent "TC" <chalucha gmail.com> writes:
Can SWIG[1] be used to help here? It even supports D somehow. As 
I understand, Python uses SIP[2] to create Qt and other bindings.

[1]http://www.swig.org/
[2]http://www.riverbankcomputing.co.uk/software/sip/intro
Feb 15 2014
prev sibling next sibling parent reply "Abdulhaq" <alynch4047 gmail.com> writes:
I'm working on a C++ binding generator that is coming on well,
called Smidgen (first there was SWIG, then sip, now Smidgen...).
I have a small demo working with both VTK and the beginnings of
Qt5 GUI. I have to pick it back up (after moving house with my
family) so would like help on various things such as building it
on Windows (currently Linux 64bit only but it's only the build
system that is Linux specific, and it's cmake so should be easy
to get going on Windows).

The code is tested and just-about one-click build. Currently it's
in a subversion repository but I plan to move to github. I'd love
someone to use Smidgen to try wrapping a small C++ library as I
think it would make the job quite easy, if anyone reads this
(since I have not formally announced the thing yet) then please
let me know and I'll get the code out there).

I have targeted the sip format binding specification (as used in
sip / pyqt) so that I then get a Qt binding on the cheap (but it
will therefore be GPL only). It is going well and as I said I
have wrapped a small part of Qt GUI using it (and the PyQt sip
files) which demonstrates working:

* Using the PyQt sip files is working and I am parsing nearly all
sip tags - extending to the rest of the PyQt sip files should not
be _too_ arduous

* Subclassing a wrapped C++ class, and overriding protected and
virtual functions (e.g. paint method and mouseMoveEvent)

* Nested classes (e.g. Qt::Connection)

* Multiple packages / modules

* Enums half-done

* Sip Transfer, TransferThis and TransferBack annotations for
method arguments (which govern object ownership)

* Multiple C++ inheritance => multiple pointers (still needs a
little bit of work but generally there)

* An automated route for type conversion e.g. QString <-> D
string - needs a bit more work for e.g. QList

My TODO List
============


* non-primitive Typedefs

* Primitive types C -> D conversion
     - *.conf file
      %CToDType long = long
      %CToDType unsigned char = ubyte  (half done)

* Sip If clauses for timelines
      %Timeline {Qt_5_0_0 Qt_5_0_1 Qt_5_0_2}  (half done)


* KeepReference for arguments + tests for Transfer etc. - Easy,
in class with KeepRef
    e.g. View.setModel(model /KeepReference/) it has an extra
attribute
        -   void* setModel_SMIKeepRef
        then in setModel() {
          View_setModel_SMIX23(model.wrappedObject);
          setModel_SMIKeepRef = model;
        }

        This will make D keep a reference to the model as long as
the View instance is alive.
        Each view will have its own reference so the total can go
above 1 for a given model.

* getCastPointerForInterface can be easily improved by not
switching on a name but instead
    each class has a separate variable for each base class pointer,
that is populated in
    the constructor

    - each class has one extra pointer per interface implemented
      - override virtual void*[] getExtraPointers() {
          void*[] extraPointers = super.getExtraPointers();
          extraPointers ~= wrappedObject_Calculator;
          return extraPointers;
      }

    - in constructor this() {
          wrappedObject_Calculator =
castRectAsCalculator(wrappedObject);
      }

    - in destructor ~this() {
          deregisterWrappedObject(wrappedObject);
          deregisterWrappedObejct(wrappedObject_Calculator);
      }

* instance_wrapper et al., need to also register base class
pointers

* getWrappedObject / getClassName - how to handle this in a
x-module fashion.

* Add support for wrapping members

* Add QTest support

* int arguments that take a default enum value & enum defaults

* char** -> Use this _idea_
      this(string[] args)
      {
          //        if (m_instance != null)
          //            throw new
RuntimeException("QCoreApplication can only be initialized once");

          argc = cast(int)args.length;
          argv = toStringzArray(args);
          this(&argc, argv);

          //        m_instance.aboutToQuit.connect(m_instance,
"disposeOfMyself()");
      }

* Threading? - wrappedObjects[] should be shared as with CPP
instance tracker. Use signal.d's
    WeakRef and InvisibleAddress

* qRegisterMetaType??

* Add %UsesConverter to package.sip, so that the correct includes
are in the package and
    we don't get problems trying to include e.g. widget.h in 
core.so

* Converted types that are returned are not passed through
getWrapper - might cause
    duplicate D objects around same C++ object e.g. QList<QWidget>?
Is this a problem?

* In CPP wrapper, the conversion from argument can have a memory
leak (e.g. new string created
   and not deleted)  BETTER - pass new type in as a reference if
possible

* Enums don't have C++ value if it was specified - need a CPP
program that writes
    a text file e.g. Color::Red 124
                     Color::Green 234
    and then write enum wrappers which include the values using the
text lookup (at build time
    of wrapper NOT at runtime of final program)

* Operator overloading - do nicely in D

* Free function operator overloads in CPP

* Handle array arguments

* Multiple inheritance - if a method returns an instance of an
interface and it has
    been created by CPP then the *Impl is returned - it is not
currently typechecked.
    We should instead check if it's already on a registered object.

* Interfaces - allow virtual calls on the interface

* Lifetime management - VTK?

* VTK/ Qt differences - have factory methods for Method, Klass,
Package etc., and
    different wrapping flavours can have differing factories. Needs
API stability.

* Virtual functions, overloading etc. - will the right call be
made if from an inherited class I make a call
    on an inherited D class method for a virtual method - will it
call the C++ base equivalent,
    thereby not calling the correct virtual call on the derived
class? See GOTW 5,
    calling virtual funcs on a base pointer will not call the
derived class method.

* Answer to above Q - yes, so need to implement wrappers for all
inherited virtual functions
    even if not mentioned in SIP file

* Signals and slots

* Enums declared inside classes

* In CPP wrapper class remove factory function/private
constructor - it is not reqd.

* Nested classes and multiple inheritance combination probably
not working (e.g. Klass.getWrappedClassName)
Feb 15 2014
parent Jacob Carlborg <doob me.com> writes:
On 2014-02-17 10:45, Andrej Mitrovic wrote:

 Clang itself has a C API, but from what I've read online it doesn't
 export enough C++ typeinfo either, so I'd have to use the Clang C++
 interface instead (maybe libtooling or something of that sort).

Why not contribute to Clang by adding the missing functionality. Then contributing to DStep ;) -- /Jacob Carlborg
Feb 17 2014
prev sibling next sibling parent "michaelc37" <michaelc37 msn.com> writes:
 The code is tested and just-about one-click build. Currently 
 it's
 in a subversion repository but I plan to move to github. I'd 
 love
 someone to use Smidgen to try wrapping a small C++ library as I
 think it would make the job quite easy, if anyone reads this
 (since I have not formally announced the thing yet) then please
 let me know and I'll get the code out there).

Feb 15 2014
prev sibling next sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
 yes github please.. i'd like to check this out.

OK I'm in the process of migrating the code to github. I was in such a hurry yesterday evening I didn't list the features properly: FEATURES ======== * All D * Understandable, maintainable code * Wraps protected and virtual methods, allows virtual methods to be overridden in D * Mixin classes in target C++ library supported * Allows custom type conversions between C++ and D types * C++ enums mapped to D enums and are type checked in D * Wraps nested C++ classes * Tested * Based on the sip format. This is well proven and allows simplified maintenance of wrappers for multiple versions of the target library. (All larger target libraries will need some ongoing maintenance of the wrapper regardless of the wrapping technology). Once all the code is in github I'll just check that a totally clean build is working, and then give you a heads up. FYI the repository is at https://github.com/alynch4047/smidgen/tree/master/morsel
Feb 16 2014
prev sibling next sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
 https://github.com/alynch4047/smidgen/tree/master/morsel

that should of course be https://github.com/alynch4047/smidgen
Feb 16 2014
prev sibling next sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Sunday, 16 February 2014 at 02:37:50 UTC, michaelc37 wrote:
 The code is tested and just-about one-click build. Currently 
 it's
 in a subversion repository but I plan to move to github. I'd 
 love
 someone to use Smidgen to try wrapping a small C++ library as I
 think it would make the job quite easy, if anyone reads this
 (since I have not formally announced the thing yet) then please
 let me know and I'll get the code out there).


OK I believe that smidgen is now fully hosted on github. Using smidgen as it stands I would hope that you could wrap a simple C++ library. However, no-one else has tried it yet so there are bound to be a few teething problems, please feel free to email me (anyone trying it out or just taking a look) at alynch4047 at gmail dot com if you have any questions / problems. Alternatively perhaps we could use one of the forums here?? I have not yet uploaded the initial Qt5 wrapping (or VTK), I'll try to do that over the next day or two.
Feb 16 2014
prev sibling next sibling parent "ed" <growlercab gmail.com> writes:
On Sunday, 16 February 2014 at 20:05:27 UTC, Abdulhaq wrote:
 On Sunday, 16 February 2014 at 02:37:50 UTC, michaelc37 wrote:
 The code is tested and just-about one-click build. Currently 
 it's
 in a subversion repository but I plan to move to github. I'd 
 love
 someone to use Smidgen to try wrapping a small C++ library as 
 I
 think it would make the job quite easy, if anyone reads this
 (since I have not formally announced the thing yet) then 
 please
 let me know and I'll get the code out there).


OK I believe that smidgen is now fully hosted on github. Using smidgen as it stands I would hope that you could wrap a simple C++ library. However, no-one else has tried it yet so there are bound to be a few teething problems, please feel free to email me (anyone trying it out or just taking a look) at alynch4047 at gmail dot com if you have any questions / problems. Alternatively perhaps we could use one of the forums here?? I have not yet uploaded the initial Qt5 wrapping (or VTK), I'll try to do that over the next day or two.

D wrappings for VTK would be great! I have a SWIG version of D-VTK but it is a little flaky and soooo slow to generate. I've just started work developing D wrapper for VTK based on their Java wrapper but it's very early stages (started yesterday in fact). If you've got a VTK wrapping that generates D code I'd love to check it out. Cheers, ed
Feb 16 2014
prev sibling next sibling parent "w0rp" <devw0rp gmail.com> writes:
On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 wrote:
 [1]
 QtD is not actively maintained.  It existed on
 BitBucket: https://bitbucket.org/qtd/repo
 before someone else forked it and moved to
 GitHub: https://github.com/qtd-developers/qtd

Probably worth mentioning. There are also some recent startup efforts to generate qt bindings based on smoke qt. https://github.com/w0rp/dqt https://github.com/GlobecSys/smoke.d However, its qt4.

I'm working on this slowly. I'll post more when I actually have something worth looking at. My library takes the SMOKE data generated by smokeqt and then does at least 90% of the work in D code to manipulate it, for generating D source files and loading C++ function pointers at runtime so you can wrap C++ classes in D classes. If you have the SMOKE data already in a shared library already I recommend this, as you can get much better results this way. There is some additional C++ code for two reasons. First, because smokeqt doesn't generate bindings for QString, which is weird and annoying, so I have to provide functions with C calling conventions written in C++ for creating QStrings. The second reason is that the functions for creating and deleting the QtGUI and QtCore SMOKE structs just don't seem to work when you call them directly from D, but do work when I wrap them again in functions with C calling conventions. Right now I have a generator which spits out a full list of methods and classes, also enums with the right values at the moment, and I need to do more work to get it to map from C++ type to D type, generate required import lines for tying things together, and then generate the snippets of code in methods to tie to the code I wrote before for loading the function pointers on startup. After that it should work, but not well enough to be that useful. I'll try Qt5 later. The main reason I'm trying to get Qt4 working at the moment is that I have experience with Qt4, but not with Qt5, and from my understanding a lot of the advantages of Qt5 come from QML, and I don't quite understand yet how QML would tie together with a D API. If I manage to get my bindings to work, I could see how you could take that and move on to creating a generalised C++ binding generator, but what you get would never be that great. It would be much better to have some kind of support in the language or runtime for doing the remaining things D can't do at the moment, like matching C++ function calling conventions, calling virtual methods, I don't know how you'd get multiple inheritance or stack-based classes to work. I get why all of this stuff just doesn't exist already, because it's really hard to implement and requires a lot of careful thought about it.
Feb 16 2014
prev sibling next sibling parent "w0rp" <devw0rp gmail.com> writes:
On Monday, 17 February 2014 at 01:16:44 UTC, Xavier Bigand wrote:
 Le 17/02/2014 01:36, w0rp a écrit :
 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 
 wrote:
 I'll try Qt5 later. The main reason I'm trying to get Qt4 
 working at the
 moment is that I have experience with Qt4, but not with Qt5, 
 and from my
 understanding a lot of the advantages of Qt5 come from QML, 
 and I don't
 quite understand yet how QML would tie together with a D API.

If someone is interested by Qt mainly for QML, maybe the better way is to help us on DQuick, cause it's really similar, but we try to take advantages of D to do something more friendly.

Writing a native D GUI library instead of wrapping a C++ one is a good idea. My big problem with the wrapper around Qt I'm building is that it will never run as efficiently as a Qt application written in C++ would, so that might turn some people away. The main advantage of writing the wrapper is pretty much that it's just less work, as creating a cross platform toolkit with an API as nice as Qt's takes a lot of time and effort.
Feb 16 2014
prev sibling next sibling parent "eles" <eles eles.com> writes:
On Monday, 17 February 2014 at 07:48:51 UTC, w0rp wrote:
 On Monday, 17 February 2014 at 01:16:44 UTC, Xavier Bigand 
 wrote:
 Le 17/02/2014 01:36, w0rp a écrit :
 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 
 wrote:



 Writing a native D GUI library instead of wrapping a C++ one is 
 a good idea. My big problem with the wrapper around Qt I'm 
 building is that it will never run as efficiently as a Qt 
 application written in C++ would, so that might turn some 
 people away. The main advantage of writing the wrapper is 
 pretty much that it's just less work, as creating a cross 
 platform toolkit with an API as nice as Qt's takes a lot of 
 time and effort.

I agree with this but, still, a wrapper around Qt 5 (not 4!) would be a huge thing, especially if one wants to write Android Qt-based applications.
Feb 17 2014
prev sibling next sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
 I have a SWIG version of D-VTK but it is a little flaky and 
 soooo slow to generate. I've just started work developing D 
 wrapper for VTK based on their Java wrapper but it's very early 
 stages (started yesterday in fact).

 If you've got a VTK wrapping that generates D code I'd love to 
 check it out.

The smidgen binding generator is (I believe) sufficiently feature rich to wrap all of VTK, but I've only wrapped a half dozen classes as proof of concept for now (I've been focussing on Qt5). My plan on the VTK side has been to use the automated binding generator in VTK to produce the *.sip (similar to *.h) files that smidgen needs. VTK is quite a large library so I would expect the wrapper to be a bit slow to generate, however it shouldn't be flaky when running. I would suspect that you've got a glitch in object ownership somewhere? My plan has been that the automatic VTK binding generator for e.g. python knows about object ownership, and so I can tap in to that to generate the correct smidgen annotations that are needed for that aspect of the binding. I've also found that you can get problems when wrapped methods return instances typed as base classes that need casting to more specific types. I'm wondering if we can pool our efforts somehow? Because I need to finish the last parts of smidgen (e.g. enums with specific values and subtleties with C++ multiple inheritance)and the Qt5 wrapping side of it I couldn't divert much from my current trajectory, but if you get a chance then have a look at what smidgen is doing and see if you think you'd get any benefit using it (and you'd have my efforts working to extend smidgen for you if needed). As soon as I've uploaded the current smidgen-vtk I'll notify here. Another way we might be able to cooperate is getting a full understanding of the current built-in VTK binding generator technology. Is your SWIG wrapping online somewhere? - I'd like to take a look.
Feb 17 2014
prev sibling next sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Monday, 17 February 2014 at 01:16:44 UTC, Xavier Bigand wrote:
 Le 17/02/2014 01:36, w0rp a écrit :
 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 
 wrote:
 I'll try Qt5 later. The main reason I'm trying to get Qt4 
 working at the
 moment is that I have experience with Qt4, but not with Qt5, 
 and from my
 understanding a lot of the advantages of Qt5 come from QML, 
 and I don't
 quite understand yet how QML would tie together with a D API.

If someone is interested by Qt mainly for QML, maybe the better way is to help us on DQuick, cause it's really similar, but we try to take advantages of D to do something more friendly.

From my personal point of view my main interest is in the programmatic Qt constructions (ie. Core & Widget module, not QML) as that's my itch that needs scratching. I'm not applying any effort ATM into the QML side of Qt. I think that a native DQuick implementation would be great and would probably provide a very compelling alternative to a wrapped QML (if that ever gets done by anybody) so I hope you get lets of help from anyone interested in that side of things.
Feb 17 2014
prev sibling next sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Monday, 17 February 2014 at 00:36:58 UTC, w0rp wrote:
 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 wrote:
 [1]
 QtD is not actively maintained.  It existed on
 BitBucket: https://bitbucket.org/qtd/repo
 before someone else forked it and moved to
 GitHub: https://github.com/qtd-developers/qtd

Probably worth mentioning. There are also some recent startup efforts to generate qt bindings based on smoke qt. https://github.com/w0rp/dqt https://github.com/GlobecSys/smoke.d However, its qt4.

I'm working on this slowly. I'll post more when I actually have something worth looking at. My library takes the SMOKE data generated by smokeqt and then does at least 90% of the work in D code to manipulate it, for generating D source files and loading C++ function pointers at runtime so you can wrap C++ classes in D classes.

I'm sure we're facing similar problems from time to time, we should probably have a chat about that. Here's to hoping we both get a successful Qt wrapping and suddenly there is a choice of Qt wrapper! Here's a great article about the problems with C++ bindings and Qt in particular: http://setanta.wordpress.com/binding-c/
Feb 17 2014
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/16/14, Abdulhaq <alynch4047 gmail.com> wrote:
 that should of course be

 https://github.com/alynch4047/smidgen

Nice. I've had a codegen project before which is currently stalled: https://github.com/AndrejMitrovic/dgen Some docs I wrote about peculiarities of wrapping C++: https://github.com/AndrejMitrovic/dgen/wiki Unfortunately the tool has a long history, it went through several rewrites. From basing the typeinfo extracted from doxygen (initially the tool was only supposed to be used to generate wxWidgets bindings), to later basing it on GCCXML, to later forking GCCXML and adding my own features (https://github.com/AndrejMitrovic/gccxml), to finally realizing I won't be able to get all C++ typeinfo from GCCXML because it's based on an old GCC parser. It was an uphill battle. Ultimately I would have to re-purpose the tool to use Clang. I did have some initial success. I've managed to autowrap the TagLib C++ library, some small XML libraries, but ultimately the tool needs proper typeinfo that GCCXML doesn't provide. Clang itself has a C API, but from what I've read online it doesn't export enough C++ typeinfo either, so I'd have to use the Clang C++ interface instead (maybe libtooling or something of that sort). I might get back to working on that tool one day, I don't know yet. I've got a big queue of things to work on. Your project looks nice though.
Feb 17 2014
prev sibling next sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Monday, 17 February 2014 at 09:46:15 UTC, Andrej Mitrovic 
wrote:
 Nice. I've had a codegen project before which is currently 
 stalled:
 https://github.com/AndrejMitrovic/dgen

 Some docs I wrote about peculiarities of wrapping C++:
 https://github.com/AndrejMitrovic/dgen/wiki

Interesting reading, it seems we've run into the same problems (not surprising I suppose) and even come up with the same solutions (virtual functions, return-by-value).
 Unfortunately the tool has a long history, it went through 
 several
 rewrites. From basing the typeinfo extracted from doxygen 
 (initially
 the tool was only supposed to be used to generate wxWidgets 
 bindings),
 to later basing it on GCCXML, to later forking GCCXML and 
 adding my
 own features (https://github.com/AndrejMitrovic/gccxml), to 
 finally
 realizing I won't be able to get all C++ typeinfo from GCCXML 
 because
 it's based on an old GCC parser. It was an uphill battle.

 Ultimately I would have to re-purpose the tool to use Clang.

 I did have some initial success. I've managed to autowrap the 
 TagLib
 C++ library, some small XML libraries, but ultimately the tool 
 needs
 proper typeinfo that GCCXML doesn't provide.

 Clang itself has a C API, but from what I've read online it 
 doesn't
 export enough C++ typeinfo either, so I'd have to use the Clang 
 C++
 interface instead (maybe libtooling or something of that sort).

I feel your pain! Because my main target library was Qt I was able to bypass the whole typeinfo extraction issue by basing the binding generator on sip files and then primarily using the PyQt sip files - with the penalty that the PyQt wrapping will therefore be GPL. However, seeing how hard it can be to extract the typeinfo automatically (and the object ownership information still will need to be done manually) I'm very glad that I fell into doing it that way. A feature evolution for me would be to try and bolt on an automated typeinfo extractor front end. Riverbank already have one for generating sip files but it's practically undocumented and hard to use.
 I might get back to working on that tool one day, I don't know 
 yet.
 I've got a big queue of things to work on.

Same here - so much to do and so little time! If I'm not careful I end up with analysis paralysis and just browse HN and these forums instead of catching up on my reading list or coding something up. Speaking of which....
Feb 17 2014
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/17/14, Abdulhaq <alynch4047 gmail.com> wrote:
 Interesting reading, it seems we've run into the same problems
 (not surprising I suppose) and even come up with the same
 solutions (virtual functions, return-by-value).

And then there are things like trying to allow access to protected methods (meaning you have to re-declare them public for the C wrappers to access them), which isn't easy because simply re-declaring them only works in /some/ contests (specifically when overloads come into play it's a mess). And then having to handle namespaces.. etc etc. It's very easy to burn out. :) I can't even recall how I planned to support multiple inheritance. I think I was planning on using interfaces, but to keep all classes neatly castable up the hirearchy at the D side it would probably mean having to make /all/ classes inherit from interfaces. And then you have to worry about memory management. Fun stuff..
Feb 17 2014
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/17/14, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:
 On 2/17/14, Abdulhaq <alynch4047 gmail.com> wrote:
 Interesting reading, it seems we've run into the same problems
 (not surprising I suppose) and even come up with the same
 solutions (virtual functions, return-by-value).

And then there are things like trying to allow access to protected methods

It looks like I already described this in the wiki, I'm repeating myself here. :)
Feb 17 2014
prev sibling next sibling parent "Joao Matos" <joao tritao.eu> writes:
On Monday, 17 February 2014 at 09:46:15 UTC, Andrej Mitrovic 
wrote:
 On 2/16/14, Abdulhaq <alynch4047 gmail.com> wrote:
 that should of course be

 https://github.com/alynch4047/smidgen

Nice. I've had a codegen project before which is currently stalled: https://github.com/AndrejMitrovic/dgen Some docs I wrote about peculiarities of wrapping C++: https://github.com/AndrejMitrovic/dgen/wiki

Nice tools, I've also been working on a C++ binding tool myself (mostly C#/.NET though). https://github.com/mono/CppSharp It is based on the Clang C++ parser. All the hard work of parsing and C++ AST support is done, so in theory a D backend could be added, if the days had more hours I'd work on one myself :) The previous maintainer for Qyoto (Qt C# binding) and SMOKE (the binding generator I've seen referenced before in this thread) has been working on a new Qt C#-based binding effort, called QtSharp, that uses C++# (CppSharp). https://github.com/ddobrev/QtSharp
Feb 17 2014
prev sibling next sibling parent "ed" <growlercab gmail.com> writes:
On Monday, 17 February 2014 at 08:49:09 UTC, Abdulhaq wrote:
 I have a SWIG version of D-VTK but it is a little flaky and 
 soooo slow to generate. I've just started work developing D 
 wrapper for VTK based on their Java wrapper but it's very 
 early stages (started yesterday in fact).

 If you've got a VTK wrapping that generates D code I'd love to 
 check it out.

The smidgen binding generator is (I believe) sufficiently feature rich to wrap all of VTK, but I've only wrapped a half dozen classes as proof of concept for now (I've been focussing on Qt5). My plan on the VTK side has been to use the automated binding generator in VTK to produce the *.sip (similar to *.h) files that smidgen needs. VTK is quite a large library so I would expect the wrapper to be a bit slow to generate, however it shouldn't be flaky when running. I would suspect that you've got a glitch in object ownership somewhere? My plan has been that the automatic VTK binding generator for e.g. python knows about object ownership, and so I can tap in to that to generate the correct smidgen annotations that are needed for that aspect of the binding. I've also found that you can get problems when wrapped methods return instances typed as base classes that need casting to more specific types. I'm wondering if we can pool our efforts somehow? Because I need to finish the last parts of smidgen (e.g. enums with specific values and subtleties with C++ multiple inheritance)and the Qt5 wrapping side of it I couldn't divert much from my current trajectory, but if you get a chance then have a look at what smidgen is doing and see if you think you'd get any benefit using it (and you'd have my efforts working to extend smidgen for you if needed). As soon as I've uploaded the current smidgen-vtk I'll notify here. Another way we might be able to cooperate is getting a full understanding of the current built-in VTK binding generator technology. Is your SWIG wrapping online somewhere? - I'd like to take a look.

https://github.com/lyrebirdsw/vtkdbind The SWIG bindings were left in a very bad state so they won't work at the moment. It should generate the D code and possibly compile but I don't think it will link I sort of gave up on it and decided to try the VTK wrapper generator. The last bindings I generated are in the repo under swigd/bindings/d Oh and some of the scripts will have a copyright in them, ignore it. It is just the automatic header our editors at work stamp in. It is all opensource, consider it BSD, I just haven't bothered putting any license in the files themselves. As for the VTK wrapper bindings they are convoluted but I'm making some headway. It is spread across C++/CMake files to complete the full process. The C++ creates a parser which extracts the required details, CMake then feeds the output of this into the wrapper generator. I have it generating half D / half Java at the moment. If I get it all D and compiling I'll upload it. Cheers, ed
Feb 17 2014
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/18/14, Joao Matos <joao tritao.eu> wrote:
 Nice tools, I've also been working on a C++ binding tool myself
 (mostly C#/.NET though).

 https://github.com/mono/CppSharp

Does this use Clang's C++ API or libclang which is a C API? If the former, I didn't know you could call into C++ code from C#, at least not very easily. Was it hard to do? I guess I'm looking for any experiences you can share.
Feb 17 2014
prev sibling next sibling parent "ed" <growlercab gmail.com> writes:
On Tuesday, 18 February 2014 at 06:44:19 UTC, ed wrote:
[snip]

 Is your SWIG wrapping online somewhere? - I'd like to take a 
 look.

https://github.com/lyrebirdsw/vtkdbind The SWIG bindings were left in a very bad state so they won't work at the moment. It should generate the D code and possibly compile but I don't think it will link I sort of gave up on it and decided to try the VTK wrapper generator. The last bindings I generated are in the repo under swigd/bindings/d Oh and some of the scripts will have a copyright in them, ignore it. It is just the automatic header our editors at work stamp in. It is all opensource, consider it BSD, I just haven't bothered putting any license in the files themselves. As for the VTK wrapper bindings they are convoluted but I'm making some headway. It is spread across C++/CMake files to complete the full process. The C++ creates a parser which extracts the required details, CMake then feeds the output of this into the wrapper generator. I have it generating half D / half Java at the moment. If I get it all D and compiling I'll upload it. Cheers, ed

Oh let me say this is real hack code I wrote for experimentation only :D
Feb 18 2014
prev sibling next sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Monday, 17 February 2014 at 22:58:57 UTC, Andrej Mitrovic 
wrote:
 And then there are things like trying to allow access to 
 protected
 methods (meaning you have to re-declare them public for the C 
 wrappers
 to access them), which isn't easy because simply re-declaring 
 them
 only works in /some/ contests (specifically when overloads come 
 into
 play it's a mess). And then having to handle namespaces.. etc 
 etc.
 It's very easy to burn out. :)

There are many problems to solve, it take a lot of thought and time. I've ploughed all my spare time into it for a couple of months, then I moved house. Now I've got to get back in to it again :-).
 I can't even recall how I planned to support multiple 
 inheritance. I
 think I was planning on using interfaces, but to keep all 
 classes
 neatly castable up the hirearchy at the D side it would 
 probably mean
 having to make /all/ classes inherit from interfaces.

I'm only trying to handle mixin-style multiple inheritance, and doing it with interfaces for the mixins. I've got to finish up dealing with the fact that the C++ pointer can change for the same object with multiple inheritance, but it's just about done.
 And then you have to worry about memory management. Fun stuff..

It's so depressing when it all seems golden and then the application using the wrapper crashes! Anyway I believe I've got that aspect sorted out now. I wish D had proper support for weak references. I'm glad to have this conversation it's given me renewed energy to finish it off...
Feb 18 2014
prev sibling next sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Tuesday, 18 February 2014 at 08:32:07 UTC, ed wrote:
 On Tuesday, 18 February 2014 at 06:44:19 UTC, ed wrote:
 [snip]

 Is your SWIG wrapping online somewhere? - I'd like to take a 
 look.

https://github.com/lyrebirdsw/vtkdbind The SWIG bindings were left in a very bad state so they won't work at the moment. It should generate the D code and possibly compile but I don't think it will link I sort of gave up on it and decided to try the VTK wrapper generator. The last bindings I generated are in the repo under swigd/bindings/d Oh and some of the scripts will have a copyright in them, ignore it. It is just the automatic header our editors at work stamp in. It is all opensource, consider it BSD, I just haven't bothered putting any license in the files themselves. As for the VTK wrapper bindings they are convoluted but I'm making some headway. It is spread across C++/CMake files to complete the full process. The C++ creates a parser which extracts the required details, CMake then feeds the output of this into the wrapper generator. I have it generating half D / half Java at the moment. If I get it all D and compiling I'll upload it. Cheers, ed

Oh let me say this is real hack code I wrote for experimentation only :D

Thanks for taking the effort to upload it! If you do get the VTK wrapper approach going then please do upload that. I'd really like to know how the wrapper generator deals with object ownership transferal, or is it using some kind of reference counting?
Feb 18 2014
prev sibling next sibling parent "ed" <growlercab gmail.com> writes:
On Tuesday, 18 February 2014 at 09:15:11 UTC, Abdulhaq wrote:
 On Tuesday, 18 February 2014 at 08:32:07 UTC, ed wrote:
 On Tuesday, 18 February 2014 at 06:44:19 UTC, ed wrote:
 [snip]

 Is your SWIG wrapping online somewhere? - I'd like to take a 
 look.

https://github.com/lyrebirdsw/vtkdbind The SWIG bindings were left in a very bad state so they won't work at the moment. It should generate the D code and possibly compile but I don't think it will link I sort of gave up on it and decided to try the VTK wrapper generator. The last bindings I generated are in the repo under swigd/bindings/d Oh and some of the scripts will have a copyright in them, ignore it. It is just the automatic header our editors at work stamp in. It is all opensource, consider it BSD, I just haven't bothered putting any license in the files themselves. As for the VTK wrapper bindings they are convoluted but I'm making some headway. It is spread across C++/CMake files to complete the full process. The C++ creates a parser which extracts the required details, CMake then feeds the output of this into the wrapper generator. I have it generating half D / half Java at the moment. If I get it all D and compiling I'll upload it. Cheers, ed

Oh let me say this is real hack code I wrote for experimentation only :D

Thanks for taking the effort to upload it! If you do get the VTK wrapper approach going then please do upload that. I'd really like to know how the wrapper generator deals with object ownership transferal, or is it using some kind of reference counting?

Well I only know the Java wrapper as that is the one I'm trying to adapt. It uses reference counting and a custom garbage collector in the Java layer (vtkJavaGarbageCollector). I'm looking at adapting this over to D. So far all goes well, Java -->> D is straight forward. It will get interesting though when things statr compiling :) I'll post here if I get any further... Cheers, ed vtkJava
Feb 18 2014
prev sibling next sibling parent "ed" <growlercab gmail.com> writes:
On Tuesday, 18 February 2014 at 09:28:07 UTC, ed wrote:
[snip]
 Java -->> D is straight forward. It will get interesting though 
 when things statr compiling :)

s/compiling/linking/
Feb 18 2014
prev sibling next sibling parent "Dejan Lekic" <dejan.lekic gmail.com> writes:
C++ binding generator is definitely something we will use. 
However, I would rather have a good, robust tool that can be used 
to generate bindings/wrappers to C libraries. That is of equal 
(or even higher I think) importance as C++ binding generator.
Feb 18 2014
prev sibling next sibling parent "Moritz Maxeiner" <moritz ucworks.org> writes:
On Sunday, 16 February 2014 at 14:10:02 UTC, Abdulhaq wrote:
 yes github please.. i'd like to check this out.

OK I'm in the process of migrating the code to github. I was in such a hurry yesterday evening I didn't list the features properly: FEATURES ======== * All D * Understandable, maintainable code * Wraps protected and virtual methods, allows virtual methods to be overridden in D * Mixin classes in target C++ library supported * Allows custom type conversions between C++ and D types * C++ enums mapped to D enums and are type checked in D * Wraps nested C++ classes * Tested * Based on the sip format. This is well proven and allows simplified maintenance of wrappers for multiple versions of the target library. (All larger target libraries will need some ongoing maintenance of the wrapper regardless of the wrapping technology). Once all the code is in github I'll just check that a totally clean build is working, and then give you a heads up. FYI the repository is at https://github.com/alynch4047/smidgen/tree/master/morsel

This looks very nice! Two questions, though: 1) Is it possible to use an already existing C API with morsel? The reason I'm asking is, because LLVM classes contain many methods where an automatic conversion is likely to cause memory leaks to appear. This is the main reason why I decided to use the C-API for my llvm-d project, instead of attempting to use automatic tools like SWIG (that and the fact that LLVM did not export C++ symbols properly for a Windows DLL, so you needed to use the C API anyway). 2) What is the main difference between using morsel and SWIG, other than the language used to specify the interface?
Feb 18 2014
prev sibling next sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Tuesday, 18 February 2014 at 11:05:20 UTC, Moritz Maxeiner 
wrote:
 This looks very nice!
 Two questions, though:

 1) Is it possible to use an already existing C API with morsel? 
 The reason I'm asking is, because LLVM classes contain many 
 methods where an automatic conversion is likely to cause memory 
 leaks to appear. This is the main reason why I decided to use 
 the C-API for my llvm-d project, instead of attempting to use 
 automatic tools like SWIG (that and the fact that LLVM did not 
 export C++ symbols properly for a Windows DLL, so you needed to 
 use the C API anyway).

At the moment smidgen (morsel is the example C++ library I use for testing) only wraps C++ classes, on the TODO is of course to wrap C++ top level functions etc. It makes a lot of sense to add support for C functions too, but to be totally honest I have no personal need for that ATM so it would come after everything else. I would class smidgen as at beta stage at the moment in terms of the functionality it currently supports. It needs to be (ab)used by some other developers testing it out on some small APIs, and then once I feel it has settled down others could contribute code to support C APIs. Regarding memory leaks, you're right, to wrap a C++ library there has to be manual effort on the part of the developer to tell the wrapper where object ownership is transferred etc. This cannot be automated. Smidgen is based on the sip format and it's pretty easy to annotate the API to indicate where ownership is transferred.
 2) What is the main difference between using morsel and SWIG, 
 other than the language used to specify the interface?

I don't know much about SWIG but the biggest difference would have to be SWIG's great maturity and extensive feature set. Smidgen has its own plusses too though: * It's pure D and the code is tested and maintainable - i.e. if it doesn't work for someone they could probably extend/change it so that it did * It's a very thin wrapper around the client library, so I suspect it's a bit faster at runtime than the more generic SWIG * Did I mention it's in D :-) I should stress that this is beta code and needs a bit of a hammering on smaller libraries before anyone spends a lot of effort with it. However, a smaller library would not take long to wrap using smidgen, so little time would be wasted if it didn't work out.
Feb 18 2014
prev sibling next sibling parent "Joao Matos" <joao tritao.eu> writes:
On Tuesday, 18 February 2014 at 07:49:20 UTC, Andrej Mitrovic 
wrote:
 On 2/18/14, Joao Matos <joao tritao.eu> wrote:
 Nice tools, I've also been working on a C++ binding tool myself
 (mostly C#/.NET though).

 https://github.com/mono/CppSharp

Does this use Clang's C++ API or libclang which is a C API? If the former, I didn't know you could call into C++ code from C#, at least not very easily. Was it hard to do? I guess I'm looking for any experiences you can share.

I actually started with the libclang C API but found out that it was too limited, so I switched to the C++ one. Well, .NET can actually call directly into C++ via C++/CLI (it's limited to Windows though) so that's what the first parser relied on. We now have a second parser that uses pure C++ and uses CppSharp to generate C# and bootstrap itself so that it can work on all platforms. And yes, it's been pretty hard to do, binding C++ _is_ hard, though D should have a big advantage here, it's semantics are closer to C++ than many other languages.
Feb 18 2014
prev sibling next sibling parent "Moritz Maxeiner" <moritz ucworks.org> writes:
On Tuesday, 18 February 2014 at 11:52:24 UTC, Abdulhaq wrote:
 At the moment smidgen (morsel is the example C++ library I use 
 for testing) only wraps C++ classes, on the TODO is of course 
 to wrap C++ top level functions etc. It makes a lot of sense to 
 add support for C functions too, but to be totally honest I 
 have no personal need for that ATM so it would come after 
 everything else. I would class smidgen as at beta stage at the 
 moment in terms of the functionality it currently supports. It 
 needs to be (ab)used by some other developers testing it out on 
 some small APIs, and then once I feel it has settled down 
 others could contribute code to support C APIs.

 Regarding memory leaks, you're right, to wrap a C++ library 
 there has to be manual effort on the part of the developer to 
 tell the wrapper where object ownership is transferred etc. 
 This cannot be automated. Smidgen is based on the sip format 
 and it's pretty easy to annotate the API to indicate where 
 ownership is transferred.

 2) What is the main difference between using morsel and SWIG, 
 other than the language used to specify the interface?

I don't know much about SWIG but the biggest difference would have to be SWIG's great maturity and extensive feature set. Smidgen has its own plusses too though: * It's pure D and the code is tested and maintainable - i.e. if it doesn't work for someone they could probably extend/change it so that it did * It's a very thin wrapper around the client library, so I suspect it's a bit faster at runtime than the more generic SWIG * Did I mention it's in D :-)

Thanks for the comprehensive write-up, I'll have a look at smidgen the next time I'm fiddling with llvm-d, then. It seems promising, but I'm not yet entirely convinced that it'd fit well with LLVM: About 70% of their C++ API not in the backend is essentially helpers classes to deal with things either already present or not needed in languages like D. The problem is that the methods you actually want to access very often require arguments with the type being one of these helper classes. And there's no benefit of having these classes in D other than to use instances of them as arguments to the rest of the LLVM API, since their functionality is already present in D or not needed. Anyway, I'll try it out when I have time and see if it beats writing the wrapper manually.
Feb 18 2014
prev sibling next sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Saturday, 15 February 2014 at 19:57:40 UTC, Abdulhaq wrote:
 I'm working on a C++ binding generator that is coming on well,
 called Smidgen (first there was SWIG, then sip, now Smidgen...).
 I have a small demo working with both VTK and the beginnings of
 Qt5 GUI. I have to pick it back up (after moving house with my
 family) so would like help on various things such as building it
 on Windows (currently Linux 64bit only but it's only the build
 system that is Linux specific, and it's cmake so should be easy
 to get going on Windows).

I've tidied a few things up and the Qt5 example is on github at https://github.com/alynch4047/sqt It's only wrapping about 20 classes or so, from QtCore and QtWidget. I could easily include many more but it would slow down development of new binding features so I've resisted the temptation. The main thing missing iro Qt is signals and slots - I know how I will do it, I just a need a couple of days free time.
Feb 22 2014
prev sibling next sibling parent "w0rp" <devw0rp gmail.com> writes:
https://github.com/w0rp/dqt

I have been working on my generator over the past few weeks, and 
extensively this weekend. It took many hours of work, but I just 
got generation of D source files from SMOKE data to work. Here is 
what it does and roughly how it works.

1. Earlier on, SMOKE is by a third party to generate a dynamic 
library. This library contains data held in a "Smoke" class 
defined in C++, which functions provided for creating, deleting, 
and getting a reference to the pointer for the class with what 
are *supposed* to be functions with C linkage. (I had to wrap 
them anyway with my own functions due to issues.)
2. Because the class members in Smoke are all C primitives, I can 
match the layout with a struct in D and load everything from the 
struct in D code. At the time of generation, I copy everything I 
need out of the struct into a saner representation in D, which I 
have called "SmokeContainer."
3. SmokeContainer is supposed to tell you everything you need to 
know to make generation work. So I use that data and pass it to 
my class which handles generation of D source files, 
"SmokeGenerator." It does all kinds of work to translate C++ 
classes from the SMOKE data to D code, with some ability for 
customisation like ignoring certain types or classes, replacing 
classes with different ones in D or just wrappers for generating 
them, etc.
4. The SmokeGenerator spits out the D source files and requires 
having a "SmokeLoader" defined in prefix D file, which loads the 
same SMOKE data again at runtime for finding the class method 
pointers needed to call C++ functions and so on.
5. You run a D program which uses the generated D source files 
and now you have C++ code being run from D.

While my project is actually called DQt above and the motivation 
for it is to get Qt working in D, I have written it all in such a 
way as to make the Qt specific part of the code and the general 
case SMOKE binding stuff loosely coupled. The only Qt specific 
stuff now lies in some C++ code needed for certain circumstances, 
(like wrapping QString) a few pre-baked D source files used as 
finishing touches for the generation specific to Qt, and the 
current test programs which generate code and run Hello World 
respectively. I shouldn't have too much trouble moving most of 
this stuff into a "dsmoke" repository instead.

If you want to try out running Hello World from the generated 
code yourself, and you're willing to work around my sloppy 
project structure, do this.

1. Build smokegen and smokeqt. I wrote two guides for Linux and 
Windows on this.
    1. Linux: 
https://github.com/w0rp/dqt/blob/master/doc/smoke_linux_build_guide.rst
    2. Windows: 
https://github.com/w0rp/dqt/blob/master/doc/smoke_win32_build_guide.rst
2. Use CMake to build my code. I think 'cmake . && make' will do 
the job on Linux, if you're in my project directory.
3. Put 'libsmoke_cwrapper' in your library path. I make it easy 
on myself by copying it out of the lib/ directory into src/ and 
by running 'export LD_LIBRARY_PATH=.' Not the best way, but it's 
a quick way to get it going.
4. 'rdmd -g -Llibsmoke_cwrapper.so -d -debug -w new_test.d' Will 
run the generator and spit some stuff into src/dqt for now.
5. 'rdmd -Llibsmoke_cwrapper.so -d smoke_test.d' will now run the 
Hello World program, supposing everything else went well.

I am going to continue working on this. There are some things to 
iron out and a lot of interesting work ahead. Hopefully I can get 
a somewhat usable Qt in D library out of this.
Feb 23 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e0122991aa5b15404f3221537
Content-Type: text/plain; charset=UTF-8

On 17 February 2014 06:05, Abdulhaq <alynch4047 gmail.com> wrote:

 On Sunday, 16 February 2014 at 02:37:50 UTC, michaelc37 wrote:

 The code is tested and just-about one-click build. Currently it's
 in a subversion repository but I plan to move to github. I'd love
 someone to use Smidgen to try wrapping a small C++ library as I
 think it would make the job quite easy, if anyone reads this
 (since I have not formally announced the thing yet) then please
 let me know and I'll get the code out there).

  yes github please.. i'd like to check this out.


OK I believe that smidgen is now fully hosted on github. Using smidgen as it stands I would hope that you could wrap a simple C++ library. However, no-one else has tried it yet so there are bound to be a few teething problems, please feel free to email me (anyone trying it out or just taking a look) at alynch4047 at gmail dot com if you have any questions / problems. Alternatively perhaps we could use one of the forums here?? I have not yet uploaded the initial Qt5 wrapping (or VTK), I'll try to do that over the next day or two.

This is awesome. I have a comprehensive game engine, and I've been looking for a solution to auto-generate my D bindings, but so far, I haven't found anything capable of doing a proper job. I'll definitely be checking this out. I maintain bindings by hand currently, and I'm often fighting with keeping them in sync. I have a couple of interesting requirements, I'll be curious to see how it works out in practise. --089e0122991aa5b15404f3221537 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= 7 February 2014 06:05, Abdulhaq <span dir=3D"ltr">&lt;<a href=3D"mailto:aly= nch4047 gmail.com" target=3D"_blank">alynch4047 gmail.com</a>&gt;</span> wr= ote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border= -left:1px #ccc solid;padding-left:1ex"> <div class=3D"HOEnZb"><div class=3D"h5">On Sunday, 16 February 2014 at 02:3= 7:50 UTC, michaelc37 wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><blockquote class=3D"gmail_quote" style=3D"m= argin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br> The code is tested and just-about one-click build. Currently it&#39;s<br> in a subversion repository but I plan to move to github. I&#39;d love<br> someone to use Smidgen to try wrapping a small C++ library as I<br> think it would make the job quite easy, if anyone reads this<br> (since I have not formally announced the thing yet) then please<br> let me know and I&#39;ll get the code out there).<br> <br> </blockquote> yes github please.. i&#39;d like to check this out.<br> </blockquote> <br></div></div> OK I believe that smidgen is now fully hosted on github.<br> <br> Using smidgen as it stands I would hope that you could wrap a simple C++ li= brary. However, no-one else has tried it yet so there are bound to be a few= teething problems, please feel free to email me (anyone trying it out or j= ust taking a look) at alynch4047 at gmail dot com if you have any questions= / problems. Alternatively perhaps we could use one of the forums here??<br=

<br> I have not yet uploaded the initial Qt5 wrapping (or VTK), I&#39;ll try to = do that over the next day or two.<br></blockquote><div><br></div><div>This = is awesome. I have a comprehensive game engine, and I&#39;ve been looking f= or a solution to auto-generate my D bindings, but so far, I haven&#39;t fou= nd anything capable of doing a proper job.</div> <div>I&#39;ll definitely be checking this out. I maintain bindings by hand = currently, and I&#39;m often fighting with keeping them in sync.</div><div>= <br></div><div>I have a couple of interesting requirements, I&#39;ll be cur= ious to see how it works out in practise.</div> </div></div></div> --089e0122991aa5b15404f3221537--
Feb 23 2014
prev sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Monday, 24 February 2014 at 07:40:42 UTC, Manu wrote:
 On 17 February 2014 06:05, Abdulhaq <alynch4047 gmail.com> 
 wrote:

 On Sunday, 16 February 2014 at 02:37:50 UTC, michaelc37 wrote:

 The code is tested and just-about one-click build. Currently 
 it's
 in a subversion repository but I plan to move to github. I'd 
 love
 someone to use Smidgen to try wrapping a small C++ library 
 as I
 think it would make the job quite easy, if anyone reads this
 (since I have not formally announced the thing yet) then 
 please
 let me know and I'll get the code out there).

  yes github please.. i'd like to check this out.


OK I believe that smidgen is now fully hosted on github. Using smidgen as it stands I would hope that you could wrap a simple C++ library. However, no-one else has tried it yet so there are bound to be a few teething problems, please feel free to email me (anyone trying it out or just taking a look) at alynch4047 at gmail dot com if you have any questions / problems. Alternatively perhaps we could use one of the forums here?? I have not yet uploaded the initial Qt5 wrapping (or VTK), I'll try to do that over the next day or two.

This is awesome. I have a comprehensive game engine, and I've been looking for a solution to auto-generate my D bindings, but so far, I haven't found anything capable of doing a proper job. I'll definitely be checking this out. I maintain bindings by hand currently, and I'm often fighting with keeping them in sync. I have a couple of interesting requirements, I'll be curious to see how it works out in practise.

It would be great if you could give it a run, it's beta software so please email me if you run into any problems.
Feb 24 2014