www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Custom calling conventions

reply Manu <turkeyman gmail.com> writes:
--002354470c703a672104b9775b27
Content-Type: text/plain; charset=UTF-8

So I was thinking about this extern(language) thing, the obvious ones are
supported, but it would be really nice to be able to implement custom
conventions for other languages/scripting languages.

For instance, I'm thinking about Android, I have JNI binding code
everywhere, it's really ugly.
I'd love to be able to declare:
  extern(Java) int someJavaFunc(int x, float y)

And then use my function like any regular function, with the 'extern(Java)'
bit handling the JNI business behind the scenes.
I also regularly interact with javascript, lua, C#/mono, and these could
all be implemented the same way.

I'm imaging some mechanism to declare a calling convention (which would be
resolved within the extern(...) statement), and define it with a template,
something like:

callconv Java
{
  R call(T...)
  {
    // process tuple of args, make the call, return something?
  }

  R thisCall(Class, T...)
  {
    // also need a way to implementing methods... this might be enough.
  }
}

Some fancy code in there could conceivably call into any foreign language,
and this would be great!
Now when I: import java.jni;
I have the jni interface, but I also have access to extern(Java), and
that's awesome! :)

The main benefit over using a template, for
instance: jniCall!"functionName"(args...), would be the function name is
not a string, or require custom code construct (facilitating later
re-factoring or delegation to script without changing masses of existing
code, something I have done often), and if it was seen by the language as a
regular function call, you can mark it with all the usual stuff,
const/pure/etc, and apply the expected set of optimisations to the call.

I'm sure this has been discussed before... so go on, tear it apart :)

--002354470c703a672104b9775b27
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

So I was thinking about this extern(language) thing, the obvious ones are s=
upported, but it would be really nice to be able to implement custom conven=
tions for other languages/scripting languages.<div><br></div><div>For insta=
nce, I&#39;m thinking about Android, I have JNI binding code everywhere, it=
&#39;s really ugly.</div>
<div>I&#39;d love to be able to declare:</div><div>=C2=A0 extern(Java) int =
someJavaFunc(int x, float y)</div><div><br></div><div>And then use my funct=
ion like any regular function, with the &#39;extern(Java)&#39; bit handling=
 the JNI business behind the scenes.</div>
<div>I also regularly interact with javascript, lua, C#/mono, and these cou=
ld all be implemented the same way.</div><div><br></div><div>I&#39;m imagin=
g some mechanism to declare a calling convention (which would be resolved w=
ithin the extern(...) statement), and define it with a template, something =
like:</div>
<div><div><br></div><div>callconv Java</div><div>{</div><div>=C2=A0 R call(=
T...)</div><div>=C2=A0 {</div><div>=C2=A0 =C2=A0 // process tuple of args, =
make the call, return something?</div><div>=C2=A0 }</div><div><br></div><di=
v>=C2=A0 R thisCall(Class, T...)</div>
<div>=C2=A0 {</div><div>=C2=A0 =C2=A0 // also need a way to implementing me=
thods... this might be enough.</div><div>=C2=A0 }</div><div>}</div><br clas=
s=3D"Apple-interchange-newline"></div><div>Some fancy code in there could c=
onceivably call into any foreign language, and this would be great!</div>
<div>Now when I:=C2=A0import java.jni;</div><div>I have the jni interface, =
but I also have access to extern(Java), and that&#39;s awesome! :)</div><di=
v><br></div><div>The main benefit over using a template, for instance:=C2=
=A0jniCall!&quot;functionName&quot;(args...), would be the function name is=
 not a string, or require custom code construct (facilitating later re-fact=
oring or delegation to script without changing masses of existing code, som=
ething I have done often), and if it was seen by the language as a regular =
function call, you can mark it with all the usual stuff, const/pure/etc, an=
d apply the expected set of optimisations to the call.</div>
<div><br></div><div>I&#39;m sure this has been discussed before... so go on=
, tear it apart :)</div>

--002354470c703a672104b9775b27--
Feb 21 2012
next sibling parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
Why can't you do that with existing language features?

alias JniExternFunc!(void function(int)) someJNIFuncYouWantToCallFromD;
mixin JniExportFunc!(&dFuncYouWantToCallUsingJNI);

Where the templates generate the wrapper code/calling convention arg shuffle 
for each function. 
Feb 21 2012
parent Paulo Pinto <pjmlp progtools.org> writes:
I think this fails in the same trap as extern "language" in C++.

Besides C and C++, I think the only variation I saw so far for the 
language part has been fortran, and cannot recall anylonger in what
compiler it was.

In C++'s case, since it is implementation defined, most compiler vendors
did not bother to support more than what the standard specifies (C and C++).

In you proposal, how to keep the feature portable across D implementations?

If you make it part of D language specification, which languages would 
be the chosen ones to be made available in any D compiler?

How to garantee correct interoperability (ABI) with other languages? 
After deciding the set of chosen languages, which compilers/runtimes 
would be the lucky ones?

--
Paulo

Am 21.02.2012 13:29, schrieb Manu:
 On 21 February 2012 14:13, Daniel Murphy <yebblies nospamgmail.com
 <mailto:yebblies nospamgmail.com>> wrote:

     Why can't you do that with existing language features?

     alias JniExternFunc!(void function(int)) someJNIFuncYouWantToCallFromD;
     mixin JniExportFunc!(&dFuncYouWantToCallUsingJNI);

     Where the templates generate the wrapper code/calling convention arg
     shuffle
     for each function.


 You could, but that's really ugly. I did make the point in my OP that it
 can be hacked right now, but I also listed the benefits of making it a
 proper feature:
   1. It would be very neat and tidy. Clarity and readability is
 important for code longevity.
   2. Ease of refactoring/delegation to scripts without refactoring all
 the calling code throughout your app (this is real, I have done this on
 many occasions)
   3. Declaring functions in the regular way, and annotating them with
 various properties (const/pure/etc) allows all the usual type safety and
 optimisation potentials available to the language.
       As far as the compiler is concerned, it is a real and normal
 function call. All surrounding code gen/optimisations are still
 applicable, the language will simply substitute the marshalling function
 at the final step of the call.
   4-ish. It actually seems relatively simple to implement... and it's
 not a breaking change. I can't really see any down sides.

Feb 21 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--002354470c70f6330a04b9788fdc
Content-Type: text/plain; charset=UTF-8

On 21 February 2012 14:13, Daniel Murphy <yebblies nospamgmail.com> wrote:

 Why can't you do that with existing language features?

 alias JniExternFunc!(void function(int)) someJNIFuncYouWantToCallFromD;
 mixin JniExportFunc!(&dFuncYouWantToCallUsingJNI);

 Where the templates generate the wrapper code/calling convention arg
 shuffle
 for each function.

You could, but that's really ugly. I did make the point in my OP that it can be hacked right now, but I also listed the benefits of making it a proper feature: 1. It would be very neat and tidy. Clarity and readability is important for code longevity. 2. Ease of refactoring/delegation to scripts without refactoring all the calling code throughout your app (this is real, I have done this on many occasions) 3. Declaring functions in the regular way, and annotating them with various properties (const/pure/etc) allows all the usual type safety and optimisation potentials available to the language. As far as the compiler is concerned, it is a real and normal function call. All surrounding code gen/optimisations are still applicable, the language will simply substitute the marshalling function at the final step of the call. 4-ish. It actually seems relatively simple to implement... and it's not a breaking change. I can't really see any down sides. --002354470c70f6330a04b9788fdc Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div class=3D"gmail_quote">On 21 February 2012 14:13, Daniel Murphy <span d= ir=3D"ltr">&lt;<a href=3D"mailto:yebblies nospamgmail.com">yebblies nospamg= mail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style= =3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Why can&#39;t you do that with existing language features?<br> <br> alias JniExternFunc!(void function(int)) someJNIFuncYouWantToCallFromD;<br> mixin JniExportFunc!(&amp;dFuncYouWantToCallUsingJNI);<br> <br> Where the templates generate the wrapper code/calling convention arg shuffl= e<br> for each function.<br></blockquote><div><br></div><div>You could, but that&= #39;s really ugly. I did make the point in my OP that it can be hacked righ= t now, but I also listed the benefits of making it a proper feature:</div> <div>=C2=A01. It would be very neat and tidy. Clarity and readability is im= portant for code longevity.</div><div>=C2=A02. Ease of refactoring/delegati= on to scripts without refactoring all the calling code throughout your app = (this is real,=C2=A0I have done this on many occasions)</div> <div>=C2=A03. Declaring functions in the regular way, and annotating them w= ith various properties (const/pure/etc) allows all the usual type safety an= d optimisation potentials available to the language.</div><div>=C2=A0 =C2= =A0 =C2=A0As far as the compiler is concerned, it is a real and normal func= tion call. All surrounding code gen/optimisations are still applicable, the= language will simply substitute the marshalling function at the final step= of the call.</div> <div>=C2=A04-ish. It actually seems relatively simple to implement... and i= t&#39;s not a breaking change. I can&#39;t really see any down sides.</div>= </div> --002354470c70f6330a04b9788fdc--
Feb 21 2012
prev sibling next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-02-21 11:03:09 +0000, Manu <turkeyman gmail.com> said:

 
 So I was thinking about this extern(language) thing, the obvious ones are
 supported, but it would be really nice to be able to implement custom
 conventions for other languages/scripting languages.
 
 For instance, I'm thinking about Android, I have JNI binding code
 everywhere, it's really ugly.
 I'd love to be able to declare:
   extern(Java) int someJavaFunc(int x, float y)
 
 And then use my function like any regular function, with the 'extern(Java)'
 bit handling the JNI business behind the scenes.
 I also regularly interact with javascript, lua, C#/mono, and these could
 all be implemented the same way.
 
 I'm imaging some mechanism to declare a calling convention (which would be
 resolved within the extern(...) statement), and define it with a template,
 something like:
 
 callconv Java
 {
   R call(T...)
   {
     // process tuple of args, make the call, return something?
   }
 
   R thisCall(Class, T...)
   {
     // also need a way to implementing methods... this might be enough.
   }
 }
 
 Some fancy code in there could conceivably call into any foreign language,
 and this would be great!
 Now when I: import java.jni;
 I have the jni interface, but I also have access to extern(Java), and
 that's awesome! :)

In all currently existing cases, extern(Lang) offers not only a way to declare and call foreign-language functions, but also a way to define functions to be called from the foreign language. Basically, you have two-way compatibility. How would that work with Java? - - - Also, I see you're thinking about methods, which means you're thinking about declaring Java classes. How is that supposed to work exactly? I have some experience bridging Objective-C and D. I once built a complete wrapper system for Objective-C objects, each object was wrapped by a D one. It worked very well, but it generated so much bloat that it became unusable as soon as I started defining enough classes for it to be useful. See the D/Objective-C bridge: <http://michelf.com/projects/d-objc-bridge/>. Maybe you'll have more luck, but for me wrappers just couldn't work. So I decided making the compiler understand the Objective-C runtime and object model was a much better solution. It was fun to do and wasn't that hard actually. Here's the result: <http://michelf.com/projects/d-objc/>. Unfortunately I don't have much time to work on it anymore. More background: <http://michelf.com/weblog/2010/dobjc-dead-end-start-anew/> -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Feb 21 2012
next sibling parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <xtzgzorex gmail.com> writes:
On 21-02-2012 18:03, Manu wrote:
 On 21 February 2012 16:59, Michel Fortin <michel.fortin michelf.com
 <mailto:michel.fortin michelf.com>> wrote:

     On 2012-02-21 11:03:09 +0000, Manu <turkeyman gmail.com
     <mailto:turkeyman gmail.com>> said:


         So I was thinking about this extern(language) thing, the obvious
         ones are
         supported, but it would be really nice to be able to implement
         custom
         conventions for other languages/scripting languages.

         For instance, I'm thinking about Android, I have JNI binding code
         everywhere, it's really ugly.
         I'd love to be able to declare:
           extern(Java) int someJavaFunc(int x, float y)

         And then use my function like any regular function, with the
         'extern(Java)'
         bit handling the JNI business behind the scenes.
         I also regularly interact with javascript, lua, C#/mono, and
         these could
         all be implemented the same way.

         I'm imaging some mechanism to declare a calling convention
         (which would be
         resolved within the extern(...) statement), and define it with a
         template,
         something like:

         callconv Java
         {
           R call(T...)
           {
             // process tuple of args, make the call, return something?
           }

           R thisCall(Class, T...)
           {
             // also need a way to implementing methods... this might be
         enough.
           }
         }

         Some fancy code in there could conceivably call into any foreign
         language,
         and this would be great!
         Now when I: import java.jni;
         I have the jni interface, but I also have access to
         extern(Java), and
         that's awesome! :)


     In all currently existing cases, extern(Lang) offers not only a way
     to declare and call foreign-language functions, but also a way to
     define functions to be called from the foreign language. Basically,
     you have two-way compatibility. How would that work with Java?


 Interesting point.
 Java (or any VM based language) can't hard link to the binary, so it
 never could try and call back in through an extern-ed function directly,

Sure it could. Why wouldn't it be able to? This is how extern + DllImport works in C#.
 it must be explicitly supplied the pointer in the first place. In that
 case, the API that supplies the callback to Java can produce the
 appropriate incoming call wrapper.

 I can imagine a way to doing a hard-extern under my proposal if you were
 hard linking against an unsupported language language... you might need
 to provide a name mangler in the definition block I describe to match
 the foreign language, and an incoming call template would also need to
 be defined as a naked function with the responsibility of managing the
 args/stack appropriately, and calling the externed function as a regular
 D-call (this would probably only be some light register/stack
 management, I don't think it would bloat a lot). This would allow D code
 to also call the externed function, since the physical definition is
 still a D-call, you just call it directly internally.

 Anyway, while I can imagine a solution, I don't think it would be needed
 immediately. It'll just open a further can of worms like foreign
 language struct layout as people try to use it in complex scenarios ;)

 I don't think this reduces the usefulness of my proposal at all though,
 ie. being able to apply it to outgoing calls for any foreign language.

     - - -

     Also, I see you're thinking about methods, which means you're
     thinking about declaring Java classes. How is that supposed to work
     exactly?


 Well I was thinking about that with respect to calling functions on
 classes that I have imported from a foreign language, not declared
 locally for export to a foreign language.
 I clearly didn't think that through though, I'll try and clarify my
 thoughts to that end... I get the feeling that idea was a dead-end though.

     I have some experience bridging Objective-C and D. I once built a
     complete wrapper system for Objective-C objects, each object was
     wrapped by a D one. It worked very well, but it generated so much
     bloat that it became unusable as soon as I started defining enough
     classes for it to be useful. See the D/Objective-C bridge:
     <http://michelf.com/projects/__d-objc-bridge/
     <http://michelf.com/projects/d-objc-bridge/>>.


 What was the primary cause of the bloat? I can't imagine my proposal
 causing any more bloat than the explicit jni call (or equivalent) woudl
 have otherwise.

     Maybe you'll have more luck, but for me wrappers just couldn't
     work.. So I decided making the compiler understand the Objective-C
     runtime and object model was a much better solution. It was fun to
     do and wasn't that hard actually. Here's the result:
     <http://michelf.com/projects/__d-objc/
     <http://michelf.com/projects/d-objc/>>. Unfortunately I don't have
     much time to work on it anymore.


 That sounds appropriate for Obj-C, but I don't think it applies to VM
 based languages however you slice it, since there's not actually a
 direct binary linkage. There's no way to know about the
 Java/.NET/Lua/etc object model internally, the VM owns that entirely, so
 you're forced to marshal through API's. That's okay though, you have to
 do it anyway. I'm proposing a way to unify the concept of calling
 functions without obscuring the actual call with junk, and still
 allowing the D language to annotate and optimise to its fullest extent.

     More background:
     <http://michelf.com/weblog/__2010/dobjc-dead-end-start-__anew/
     <http://michelf.com/weblog/2010/dobjc-dead-end-start-anew/>>


     --
     Michel Fortin
     michel.fortin michelf.com <mailto:michel.fortin michelf.com>
     http://michelf.com/

-- - Alex
Feb 21 2012
parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <xtzgzorex gmail.com> writes:
On 21-02-2012 20:43, Manu wrote:
 On 21 February 2012 19:10, Alex Rønne Petersen <xtzgzorex gmail.com
 <mailto:xtzgzorex gmail.com>> wrote:

     On 21-02-2012 18:03, Manu wrote:

         On 21 February 2012 16:59, Michel Fortin
         <michel.fortin michelf.com <mailto:michel.fortin michelf.com>
         <mailto:michel.fortin michelf.__com
         <mailto:michel.fortin michelf.com>>> wrote:

             On 2012-02-21 11:03:09 +0000, Manu <turkeyman gmail.com
         <mailto:turkeyman gmail.com>
         <mailto:turkeyman gmail.com <mailto:turkeyman gmail.com>>> said:

         Interesting point.
         Java (or any VM based language) can't hard link to the binary, so it
         never could try and call back in through an extern-ed function
         directly,


     Sure it could. Why wouldn't it be able to? This is how extern +
     DllImport works in C#.


 But that only works because C# has DLL support, which knows about the C
 calling convention, so you just need to export an extern(C) in that
 case, like any regular DLL.

Surely Java 'native' methods require the native functions to conform to some calling convention? -- - Alex
Feb 21 2012
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-02-21 18:03, Manu wrote:
 On 21 February 2012 16:59, Michel Fortin <michel.fortin michelf.com
     I have some experience bridging Objective-C and D. I once built a
     complete wrapper system for Objective-C objects, each object was
     wrapped by a D one. It worked very well, but it generated so much
     bloat that it became unusable as soon as I started defining enough
     classes for it to be useful. See the D/Objective-C bridge:
     <http://michelf.com/projects/__d-objc-bridge/
     <http://michelf.com/projects/d-objc-bridge/>>.


 What was the primary cause of the bloat? I can't imagine my proposal
 causing any more bloat than the explicit jni call (or equivalent) woudl
 have otherwise.

Template bloat. Every call bridging D/Objective-C is made throw a series of templates. This is for making it possible (less verbose) to create bindings. It might be possible to decrease the template bloat by having a tool that automatically generates the bindings and outputs what the templates do inline. -- /Jacob Carlborg
Feb 21 2012
next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2012-02-21 19:45:37 +0000, Manu <turkeyman gmail.com> said:

 On 21 February 2012 20:12, Jacob Carlborg <doob me.com> wrote:
 
 On 2012-02-21 18:03, Manu wrote:
 
 On 21 February 2012 16:59, Michel Fortin <michel.fortin michelf.com
 What was the primary cause of the bloat? I can't imagine my proposal
 causing any more bloat than the explicit jni call (or equivalent) woudl
 have otherwise.

Template bloat. Every call bridging D/Objective-C is made throw a series of templates. This is for making it possible (less verbose) to create bindings. It might be possible to decrease the template bloat by having a tool that automatically generates the bindings and outputs what the templates do inline.

Why aren't the templates inline themselves? Although if the templates do a lot of work, wouldn't that INCREASE the code volume? I can't really imagine how Obj-C linkage could bloat so much, what was involved? What did you have to do in addition to what a regular Obj-C function call would have done?

I answered in my other post. In short: virtual functions you can override to override Objective-C methods. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Feb 21 2012
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-02-21 20:45, Manu wrote:
 On 21 February 2012 20:12, Jacob Carlborg <doob me.com
 <mailto:doob me.com>> wrote:

     On 2012-02-21 18:03, Manu wrote:

         On 21 February 2012 16:59, Michel Fortin
         <michel.fortin michelf.com <mailto:michel.fortin michelf.com>
             I have some experience bridging Objective-C and D. I once
         built a
             complete wrapper system for Objective-C objects, each object was
             wrapped by a D one. It worked very well, but it generated so
         much
             bloat that it became unusable as soon as I started defining
         enough
             classes for it to be useful. See the D/Objective-C bridge:
         <http://michelf.com/projects/____d-objc-bridge/
         <http://michelf.com/projects/__d-objc-bridge/>
         <http://michelf.com/projects/__d-objc-bridge/
         <http://michelf.com/projects/d-objc-bridge/>>>.


         What was the primary cause of the bloat? I can't imagine my proposal
         causing any more bloat than the explicit jni call (or
         equivalent) woudl
         have otherwise.


     Template bloat. Every call bridging D/Objective-C is made throw a
     series of templates. This is for making it possible (less verbose)
     to create bindings.

     It might be possible to decrease the template bloat by having a tool
     that automatically generates the bindings and outputs what the
     templates do inline.


 Why aren't the templates inline themselves? Although if the templates do
 a lot of work, wouldn't that INCREASE the code volume?
 I can't really imagine how Obj-C linkage could bloat so much, what was
 involved? What did you have to do in addition to what a regular Obj-C
 function call would have done?

Michel Fortin explained this better in two of his answers. You can also read the documentation of my implementation, very similar to Michel's: http://www.dsource.org/projects/dstep/wiki/ObjcBridge/BridgeInternals But to give a quick example: class Foo : NSObject { Foo foo () { return invokeObjcSelf!(Foo, "foo"); } Foo bar () { return invokeObjcSelf!(Foo, "bar"); } } "invokeObjcSelf" is a template function that calls an Objective-C method. Basically each time "invokeObjcSelf" is called a new instantiation of the template is created and that is put in the symbol table. "invokeObjcSelf" then calls several more template functions making the template bloat increase exponentially. -- /Jacob Carlborg
Feb 21 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-02-21 22:01, Manu wrote:
 On 21 February 2012 22:35, Jacob Carlborg <doob me.com
 <mailto:doob me.com>> wrote:

     On 2012-02-21 20:45, Manu wrote:

         On 21 February 2012 20:12, Jacob Carlborg <doob me.com
         <mailto:doob me.com>
         <mailto:doob me.com <mailto:doob me.com>>> wrote:

             On 2012-02-21 18:03, Manu wrote:

                 On 21 February 2012 16:59, Michel Fortin
         <michel.fortin michelf.com <mailto:michel.fortin michelf.com>
         <mailto:michel.fortin michelf.__com
         <mailto:michel.fortin michelf.com>>

                     I have some experience bridging Objective-C and D. I
         once
                 built a
                     complete wrapper system for Objective-C objects,
         each object was
                     wrapped by a D one. It worked very well, but it
         generated so
                 much
                     bloat that it became unusable as soon as I started
         defining
                 enough
                     classes for it to be useful. See the D/Objective-C
         bridge:
         <http://michelf.com/projects/______d-objc-bridge/
         <http://michelf.com/projects/____d-objc-bridge/>
         <http://michelf.com/projects/____d-objc-bridge/
         <http://michelf.com/projects/__d-objc-bridge/>>

         <http://michelf.com/projects/____d-objc-bridge/
         <http://michelf.com/projects/__d-objc-bridge/>
         <http://michelf.com/projects/__d-objc-bridge/
         <http://michelf.com/projects/d-objc-bridge/>>>>.


                 What was the primary cause of the bloat? I can't imagine
         my proposal
                 causing any more bloat than the explicit jni call (or
                 equivalent) woudl
                 have otherwise.


             Template bloat. Every call bridging D/Objective-C is made
         throw a
             series of templates. This is for making it possible (less
         verbose)
             to create bindings.

             It might be possible to decrease the template bloat by
         having a tool
             that automatically generates the bindings and outputs what the
             templates do inline.


         Why aren't the templates inline themselves? Although if the
         templates do
         a lot of work, wouldn't that INCREASE the code volume?
         I can't really imagine how Obj-C linkage could bloat so much,
         what was
         involved? What did you have to do in addition to what a regular
         Obj-C
         function call would have done?


     Michel Fortin explained this better in two of his answers. You can
     also read the documentation of my implementation, very similar to
     Michel's:

     http://www.dsource.org/__projects/dstep/wiki/__ObjcBridge/BridgeInternals
     <http://www.dsource.org/projects/dstep/wiki/ObjcBridge/BridgeInternals>

     But to give a quick example:

     class Foo : NSObject
     {
         Foo foo ()
         {
             return invokeObjcSelf!(Foo, "foo");
         }

         Foo bar ()
         {
             return invokeObjcSelf!(Foo, "bar");
         }
     }

     "invokeObjcSelf" is a template function that calls an Objective-C
     method. Basically each time "invokeObjcSelf" is called a new
     instantiation of the template is created and that is put in the
     symbol table. "invokeObjcSelf" then calls several more template
     functions making the template bloat increase exponentially.


 But they should all be inlined, and the symbol table should be stripped,
 which shouldn't leave anything in the end other than the inlined
 function calling code, and in my examples, this will be basically the
 exact same code that you'd have to write anyway to call through some
 vm's API...

Well, that's not what happen with templates. -- /Jacob Carlborg
Feb 21 2012
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-02-21 23:13:31 +0000, Manu <turkeyman gmail.com> said:

 
 On 21 February 2012 23:06, Jacob Carlborg <doob me.com> wrote:
 
 On 2012-02-21 22:01, Manu wrote:
 
 On 21 February 2012 22:35, Jacob Carlborg <doob me.com
 But to give a quick example:
 
 class Foo : NSObject
 {
 Foo foo ()
 {
 return invokeObjcSelf!(Foo, "foo");
 }
 
 Foo bar ()
 {
 return invokeObjcSelf!(Foo, "bar");
 }
 }
 
 "invokeObjcSelf" is a template function that calls an Objective-C
 method. Basically each time "invokeObjcSelf" is called a new
 instantiation of the template is created and that is put in the
 symbol table. "invokeObjcSelf" then calls several more template
 functions making the template bloat increase exponentially.

But they should all be inlined, and the symbol table should be stripped, which shouldn't leave anything in the end other than the inlined function calling code, and in my examples, this will be basically the exact same code that you'd have to write anyway to call through some vm's API...

Well, that's not what happen with templates.

... really? why? You've just made me very very scared for my simd module :/ I've had a serious concern about D's lack of a force-inline for quite a while...

I'm not sure what Jacob is referring to. But here's what I can tell: Foo's methods can't be stripped because they're referenced by the virtual table. invokeObjcSelf could be inlined in theory, but it probably isn't because it is not that small considering it actually does exception bridging and argument/return value conversions as needed. Also, for each method there's also a trampoline function generated (by a mixin which is not in his example) which is referenced by the corresponding Objective-C reverse wrapper class (which is added dynamically on demand), so the Objective-C side can call the D method if you subclass the D wrapper and override one of its methods. The problem is that if you write bindings for a significant part of Cocoa, you end up with thousands of those functions, and none can be stripped because they're all referenced somehow because they're needed for dynamic dispatch. Sure, you could inline what is inside the virtual function (invokeObjcSelf), but beside saving a few function prologues I don't expect it'll have a very noticeable effect. Even if you saved 25% of the symbols it won't bring things to a reasonable size or a reasonable compilation time. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Feb 21 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-02-22 05:38, Michel Fortin wrote:
 On 2012-02-21 23:13:31 +0000, Manu <turkeyman gmail.com> said:

 On 21 February 2012 23:06, Jacob Carlborg <doob me.com> wrote:

 On 2012-02-21 22:01, Manu wrote:

 On 21 February 2012 22:35, Jacob Carlborg <doob me.com
 But to give a quick example:

 class Foo : NSObject
 {
 Foo foo ()
 {
 return invokeObjcSelf!(Foo, "foo");
 }

 Foo bar ()
 {
 return invokeObjcSelf!(Foo, "bar");
 }
 }

 "invokeObjcSelf" is a template function that calls an Objective-C
 method. Basically each time "invokeObjcSelf" is called a new
 instantiation of the template is created and that is put in the
 symbol table. "invokeObjcSelf" then calls several more template
 functions making the template bloat increase exponentially.

But they should all be inlined, and the symbol table should be stripped, which shouldn't leave anything in the end other than the inlined function calling code, and in my examples, this will be basically the exact same code that you'd have to write anyway to call through some vm's API...

Well, that's not what happen with templates.

... really? why? You've just made me very very scared for my simd module :/ I've had a serious concern about D's lack of a force-inline for quite a while...

I'm not sure what Jacob is referring to.

I referring to that every instantiation of a template function is put in the symbol table. If this is an implementation issue with DMD not being able to properly optimize this I don't know. I thought that this was one of the major reason for the executable size when using the Objective-C/D bridge. For example: void foo () {} void main () {} $ dmd -inline -O -release main.d $ nm main | grep foo 00000001000012b4 T _D4main3fooFZv Outputs one symbol as expected --------------------------------------- void foo (T) () {} void main () {} $ dmd -inline -O -release main.d $ nm main | grep foo Doesn't output anything since the template functions is not instantiated, it won't even be fully compiled (only lexed and perhaps parsed). -------------------------------------------------------------- void foo (T) () {} void main () { foo!(int); foo!(char); } $ dmd -inline -O -release main.d $ nm main | grep foo 00000001000012b8 T _D4main10__T3fooTaZ3fooFZv 00000001000012b0 T _D4main10__T3fooTiZ3fooFZv Outputs two symbols as expected, one for each instantiation. -- /Jacob Carlborg
Feb 21 2012
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-02-22 07:41:21 +0000, Jacob Carlborg <doob me.com> said:

 void foo (T) () {}
 void main ()
 {
      foo!(int);
      foo!(char);
 }
 
 $ dmd -inline -O -release main.d
 $ nm main | grep foo
 
 00000001000012b8 T _D4main10__T3fooTaZ3fooFZv
 00000001000012b0 T _D4main10__T3fooTiZ3fooFZv
 
 Outputs two symbols as expected, one for each instantiation.

That's expected indeed. This doesn't mean the inliner will not inline the templates. In fact, if you correct the example and look at the assembler output you'll see it will (in your example there is nothing to inline since you're just instantiating the template without calling it). What doesn't happen is stripping the unreferenced symbols from the executable. On OS X, try adding "-L-dead_strip" to DMD's argument to instruct the linker to do so. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Feb 22 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-02-22 13:53, Michel Fortin wrote:
 On 2012-02-22 07:41:21 +0000, Jacob Carlborg <doob me.com> said:

 void foo (T) () {}
 void main ()
 {
 foo!(int);
 foo!(char);
 }

 $ dmd -inline -O -release main.d
 $ nm main | grep foo

 00000001000012b8 T _D4main10__T3fooTaZ3fooFZv
 00000001000012b0 T _D4main10__T3fooTiZ3fooFZv

 Outputs two symbols as expected, one for each instantiation.

That's expected indeed. This doesn't mean the inliner will not inline the templates. In fact, if you correct the example and look at the assembler output you'll see it will (in your example there is nothing to inline since you're just instantiating the template without calling it). What doesn't happen is stripping the unreferenced symbols from the executable. On OS X, try adding "-L-dead_strip" to DMD's argument to instruct the linker to do so.

1. The example is correct, you can call a method without parentheses 2. Adding -L-dead_strip does not seem to strip the "foo" symbols 3. Adding -L-dead_strip causes a segmentation fault with this example (DMD 2.057) import std.stdio; void foo (T) () { writeln("asd"); } void main () { foo!(int); foo!(char); } 4. I thought the reason for the big executable using the Objective-C/D bridge was due to all these template symbols. -- /Jacob Carlborg
Feb 22 2012
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-02-22 17:32:14 +0000, Jacob Carlborg <doob me.com> said:

 On 2012-02-22 13:53, Michel Fortin wrote:
 On 2012-02-22 07:41:21 +0000, Jacob Carlborg <doob me.com> said:
 
 void foo (T) () {}
 void main ()
 {
 foo!(int);
 foo!(char);
 }
 
 $ dmd -inline -O -release main.d
 $ nm main | grep foo
 
 00000001000012b8 T _D4main10__T3fooTaZ3fooFZv
 00000001000012b0 T _D4main10__T3fooTiZ3fooFZv
 
 Outputs two symbols as expected, one for each instantiation.

That's expected indeed. This doesn't mean the inliner will not inline the templates. In fact, if you correct the example and look at the assembler output you'll see it will (in your example there is nothing to inline since you're just instantiating the template without calling it). What doesn't happen is stripping the unreferenced symbols from the executable. On OS X, try adding "-L-dead_strip" to DMD's argument to instruct the linker to do so.

1. The example is correct, you can call a method without parentheses

Ah, seems I forgot this.
 2. Adding -L-dead_strip does not seem to strip the "foo" symbols
 3. Adding -L-dead_strip causes a segmentation fault with this example 
 (DMD 2.057)
 
 import std.stdio;
 
 void foo (T) () { writeln("asd"); }
 void main ()
 {
       foo!(int);
       foo!(char);
 }

I'm still on v2.055. And I can confirm -L-dead_strip worked for me, but I did a change similar to yours so the template was calling printf (that makes it easier to see if it was inlined when looking at the assembly). Everything worked as expected with printf. Calling writeln, which is a template itself, which could be inlined into foo, could cause foo to not be inlined, although it doesn't explain the segfault. I'm starting to suspect the empty version of foo doesn't get inlined for some reason.
 4. I thought the reason for the big executable using the Objective-C/D 
 bridge was due to all these template symbols.

I might have been, but I'd be surprised I didn't try striping the executable to fix the problem. In the end, mostly all the instantiated code has to be there for the virtual tables. There's almost one instanciation per method, and that is a lot of bloat. Whether the template instanciation is inlined or not just changes under which symbol the code resides, the code doesn't magically disappear, it still has to be there somewhere. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Feb 22 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-02-22 19:39, Michel Fortin wrote:
 On 2012-02-22 17:32:14 +0000, Jacob Carlborg <doob me.com> said:

 On 2012-02-22 13:53, Michel Fortin wrote:
 On 2012-02-22 07:41:21 +0000, Jacob Carlborg <doob me.com> said:

 void foo (T) () {}
 void main ()
 {
 foo!(int);
 foo!(char);
 }

 $ dmd -inline -O -release main.d
 $ nm main | grep foo

 00000001000012b8 T _D4main10__T3fooTaZ3fooFZv
 00000001000012b0 T _D4main10__T3fooTiZ3fooFZv

 Outputs two symbols as expected, one for each instantiation.

That's expected indeed. This doesn't mean the inliner will not inline the templates. In fact, if you correct the example and look at the assembler output you'll see it will (in your example there is nothing to inline since you're just instantiating the template without calling it). What doesn't happen is stripping the unreferenced symbols from the executable. On OS X, try adding "-L-dead_strip" to DMD's argument to instruct the linker to do so.

1. The example is correct, you can call a method without parentheses

Ah, seems I forgot this.
 2. Adding -L-dead_strip does not seem to strip the "foo" symbols
 3. Adding -L-dead_strip causes a segmentation fault with this example
 (DMD 2.057)

 import std.stdio;

 void foo (T) () { writeln("asd"); }
 void main ()
 {
 foo!(int);
 foo!(char);
 }

I'm still on v2.055. And I can confirm -L-dead_strip worked for me, but I did a change similar to yours so the template was calling printf (that makes it easier to see if it was inlined when looking at the assembly). Everything worked as expected with printf. Calling writeln, which is a template itself, which could be inlined into foo, could cause foo to not be inlined, although it doesn't explain the segfault. I'm starting to suspect the empty version of foo doesn't get inlined for some reason.
 4. I thought the reason for the big executable using the Objective-C/D
 bridge was due to all these template symbols.

I might have been, but I'd be surprised I didn't try striping the executable to fix the problem. In the end, mostly all the instantiated code has to be there for the virtual tables. There's almost one instanciation per method, and that is a lot of bloat. Whether the template instanciation is inlined or not just changes under which symbol the code resides, the code doesn't magically disappear, it still has to be there somewhere.

Yes, exactly. -- /Jacob Carlborg
Feb 22 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-02-22 18:40, H. S. Teoh wrote:
 On Wed, Feb 22, 2012 at 06:32:14PM +0100, Jacob Carlborg wrote:
 [...]
 1. The example is correct, you can call a method without parentheses

This is not directly related, but isn't calling methods without parentheses deprecated, unless it's marked property? I thought we will soon enforce property.

As I understand it, that will most likely be where we're going. But we're not there yet. -- /Jacob Carlborg
Feb 22 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-02-22 00:13, Manu wrote:
 On 21 February 2012 23:06, Jacob Carlborg <doob me.com
     Well, that's not what happen with templates.


 ... really? why?
 You've just made me very very scared for my simd module :/

I've answered to this in an answer to Michel's answer.
 I've had a serious concern about D's lack of a force-inline for quite a
 while...

-- /Jacob Carlborg
Feb 21 2012
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Feb 22, 2012 at 06:32:14PM +0100, Jacob Carlborg wrote:
[...]
 1. The example is correct, you can call a method without parentheses

This is not directly related, but isn't calling methods without parentheses deprecated, unless it's marked property? I thought we will soon enforce property. T -- Talk is cheap. Whining is actually free. -- Lars Wirzenius
Feb 22 2012
prev sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2012-02-21 17:03:56 +0000, Manu <turkeyman gmail.com> said:

 What was the primary cause of the bloat? I can't imagine my proposal
 causing any more bloat than the explicit jni call (or equivalent) woudl
 have otherwise.

I had one D class for every corresponding Objective-C class, each method wrapped in both direction, each wrapper doing automatic mapping of D and Objective-C wrapper objects and translating exceptions. I could actually override a method in a derived class of the D wrapper and it'd effectively override the method on the Objective-C side. But with all those D classes having virtual tables referring to all these wrapper functions, even if you use only 1% of all the bindings nothing can be stripped of the final executable. Quoting from a blog post of mine: """ But the real show stopper is the size of the generated code for the bindings. The system I created needs to generate a lot of code for each and every one of the functions and classes you define bindings for. This makes the final executable awfully big (and might also explain in part the slowness while compiling). The small test application included with the bridge takes about 60 Mb of code, thats for something that’d be only a few kilobytes when written in Objective-C. This wasn’t the case at first. This bloat appeared as I added bindings for more and more classes in Cocoa. It is a real problem because no one in his right mind will want to ship a 60 Mb application that contains so much binding code that stay unused most of the time. A solution could be that everyone would have to write bindings that only contains the classes and methods he needs, but this is of rather limited interest. """
 Maybe you'll have more luck, but for me wrappers just couldn't work. So I
 decided making the compiler understand the Objective-C runtime and object
 model was a much better solution. It was fun to do and wasn't that hard
 actually. Here's the result: 
 <http://michelf.com/projects/**d-objc/<http://michelf.com/projects/d-objc/>>.


 
 I don't have much time to work on it anymore.

That sounds appropriate for Obj-C, but I don't think it applies to VM based languages however you slice it, since there's not actually a direct binary linkage. There's no way to know about the Java/.NET/Lua/etc object model internally, the VM owns that entirely, so you're forced to marshal through API's.

That's not actually very different from what I'm doing for Objective-C. Call to Objective-C methods are all done through low-level API functions in the Objective-C runtime. Class definitions are static data, but could be done dynamically through API calls in module constructors. The most interesting bit however is that class *declarations* require nothing to be compiled, the compiler will just emit the right code when calling a method, using objc_msgSend(). This has a major impact on compilation time if you have thousands of such functions.
 That's okay though, you have to do it anyway. I'm proposing a way to
 unify the concept of calling functions without obscuring the actual call
 with junk, and still allowing the D language to annotate and optimise to
 its fullest extent.

The goal is certainly worthwhile. I think your proposal has merit if you goal is just to call functions and not define function or classes, although I'm skeptical overloading extern() is the right way to do that (perhaps a pragma?). But I'd suggest you define your goals more throughly before deciding on an implementation. Some questions you should try to have an answer for regarding OO: if you're interacting with classes, how should it work in an ideal world? Then, do you need to define new classes that'd be available in the foreign language? Do you need to override functions on existing classes? How do you want code using foreign objects to look? Then ask yourself what would be an acceptable compromise, and start implementing it. For my part, I came to the conclusion that you couldn't really use Objective-C APIs in a meaningful way without the ability to create subclasses and override functions… which led me to the current design where you can use Objective-C classes as if they were D classes. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Feb 21 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--00248c711795803b9404b97c65da
Content-Type: text/plain; charset=UTF-8

On 21 February 2012 16:59, Michel Fortin <michel.fortin michelf.com> wrote:

 On 2012-02-21 11:03:09 +0000, Manu <turkeyman gmail.com> said:


 So I was thinking about this extern(language) thing, the obvious ones are
 supported, but it would be really nice to be able to implement custom
 conventions for other languages/scripting languages.

 For instance, I'm thinking about Android, I have JNI binding code
 everywhere, it's really ugly.
 I'd love to be able to declare:
  extern(Java) int someJavaFunc(int x, float y)

 And then use my function like any regular function, with the
 'extern(Java)'
 bit handling the JNI business behind the scenes.
 I also regularly interact with javascript, lua, C#/mono, and these could
 all be implemented the same way.

 I'm imaging some mechanism to declare a calling convention (which would be
 resolved within the extern(...) statement), and define it with a template,
 something like:

 callconv Java
 {
  R call(T...)
  {
    // process tuple of args, make the call, return something?
  }

  R thisCall(Class, T...)
  {
    // also need a way to implementing methods... this might be enough.
  }
 }

 Some fancy code in there could conceivably call into any foreign language,
 and this would be great!
 Now when I: import java.jni;
 I have the jni interface, but I also have access to extern(Java), and
 that's awesome! :)

In all currently existing cases, extern(Lang) offers not only a way to declare and call foreign-language functions, but also a way to define functions to be called from the foreign language. Basically, you have two-way compatibility. How would that work with Java?

Interesting point. Java (or any VM based language) can't hard link to the binary, so it never could try and call back in through an extern-ed function directly, it must be explicitly supplied the pointer in the first place. In that case, the API that supplies the callback to Java can produce the appropriate incoming call wrapper. I can imagine a way to doing a hard-extern under my proposal if you were hard linking against an unsupported language language... you might need to provide a name mangler in the definition block I describe to match the foreign language, and an incoming call template would also need to be defined as a naked function with the responsibility of managing the args/stack appropriately, and calling the externed function as a regular D-call (this would probably only be some light register/stack management, I don't think it would bloat a lot). This would allow D code to also call the externed function, since the physical definition is still a D-call, you just call it directly internally. Anyway, while I can imagine a solution, I don't think it would be needed immediately. It'll just open a further can of worms like foreign language struct layout as people try to use it in complex scenarios ;) I don't think this reduces the usefulness of my proposal at all though, ie. being able to apply it to outgoing calls for any foreign language. - - -
 Also, I see you're thinking about methods, which means you're thinking
 about declaring Java classes. How is that supposed to work exactly?

Well I was thinking about that with respect to calling functions on classes that I have imported from a foreign language, not declared locally for export to a foreign language. I clearly didn't think that through though, I'll try and clarify my thoughts to that end... I get the feeling that idea was a dead-end though.
 I have some experience bridging Objective-C and D. I once built a complete
 wrapper system for Objective-C objects, each object was wrapped by a D one.
 It worked very well, but it generated so much bloat that it became unusable
 as soon as I started defining enough classes for it to be useful. See the
 D/Objective-C bridge:
<http://michelf.com/projects/**d-objc-bridge/<http://michelf.com/projects/d-objc-bridge/>
.


What was the primary cause of the bloat? I can't imagine my proposal causing any more bloat than the explicit jni call (or equivalent) woudl have otherwise.
 Maybe you'll have more luck, but for me wrappers just couldn't work. So I
 decided making the compiler understand the Objective-C runtime and object
 model was a much better solution. It was fun to do and wasn't that hard
 actually. Here's the result:
<http://michelf.com/projects/**d-objc/<http://michelf.com/projects/d-objc/>>.
 Unfortunately I don't have much time to work on it anymore.

That sounds appropriate for Obj-C, but I don't think it applies to VM based languages however you slice it, since there's not actually a direct binary linkage. There's no way to know about the Java/.NET/Lua/etc object model internally, the VM owns that entirely, so you're forced to marshal through API's. That's okay though, you have to do it anyway. I'm proposing a way to unify the concept of calling functions without obscuring the actual call with junk, and still allowing the D language to annotate and optimise to its fullest extent. More background:
 <http://michelf.com/weblog/**2010/dobjc-dead-end-start-**anew/<http://michelf.com/weblog/2010/dobjc-dead-end-start-anew/>

-- Michel Fortin michel.fortin michelf.com http://michelf.com/

--00248c711795803b9404b97c65da Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div class=3D"gmail_quote">On 21 February 2012 16:59, Michel Fortin <span d= ir=3D"ltr">&lt;<a href=3D"mailto:michel.fortin michelf.com">michel.fortin m= ichelf.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style= =3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div class=3D"HOEnZb"><div class=3D"h5">On 2012-02-21 11:03:09 +0000, Manu = &lt;<a href=3D"mailto:turkeyman gmail.com" target=3D"_blank">turkeyman gmai= l.com</a>&gt; said:<br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> <br> So I was thinking about this extern(language) thing, the obvious ones are<b= r> supported, but it would be really nice to be able to implement custom<br> conventions for other languages/scripting languages.<br> <br> For instance, I&#39;m thinking about Android, I have JNI binding code<br> everywhere, it&#39;s really ugly.<br> I&#39;d love to be able to declare:<br> =C2=A0extern(Java) int someJavaFunc(int x, float y)<br> <br> And then use my function like any regular function, with the &#39;extern(Ja= va)&#39;<br> bit handling the JNI business behind the scenes.<br> I also regularly interact with javascript, lua, C#/mono, and these could<br=

<br> I&#39;m imaging some mechanism to declare a calling convention (which would= be<br> resolved within the extern(...) statement), and define it with a template,<= br> something like:<br> <br> callconv Java<br> {<br> =C2=A0R call(T...)<br> =C2=A0{<br> =C2=A0 =C2=A0// process tuple of args, make the call, return something?<br=

<br> =C2=A0R thisCall(Class, T...)<br> =C2=A0{<br> =C2=A0 =C2=A0// also need a way to implementing methods... this might be e= nough.<br> =C2=A0}<br> }<br> <br> Some fancy code in there could conceivably call into any foreign language,<= br> and this would be great!<br> Now when I: import java.jni;<br> I have the jni interface, but I also have access to extern(Java), and<br> that&#39;s awesome! :)<br> </blockquote> <br></div></div> In all currently existing cases, extern(Lang) offers not only a way to decl= are and call foreign-language functions, but also a way to define functions= to be called from the foreign language. Basically, you have two-way compat= ibility. How would that work with Java?<br> </blockquote><div><br></div><div>Interesting point.</div><div>Java (or any = VM based language) can&#39;t hard link to the binary, so it never could try= and call back in through an extern-ed function directly, it must be explic= itly supplied the pointer in the first place. In that case, the API that su= pplies the callback to Java can produce the appropriate incoming call wrapp= er.<br> </div><div><br></div><div>I can imagine a way to doing a hard-extern under = my proposal if you were hard linking against an unsupported language langua= ge... you might need to provide a name mangler in the definition block I de= scribe to match the foreign language, and an incoming call template would a= lso need to be defined as a naked function with the responsibility of manag= ing the args/stack appropriately, and calling the externed function as a re= gular D-call (this would probably only be some light register/stack managem= ent, I don&#39;t think it would bloat a lot). This would allow D code to al= so call the externed function, since the physical definition is still a D-c= all, you just call it directly internally.</div> <div><br></div><div>Anyway, while I can imagine a solution, I don&#39;t thi= nk it would be needed immediately. It&#39;ll just open a further can of wor= ms like foreign language struct layout as people try to use it in complex s= cenarios ;)</div> <div><br></div><div>I don&#39;t think this reduces the usefulness of my pro= posal at all though, ie. being able to apply it to outgoing calls for any f= oreign language.</div><div><br></div><blockquote class=3D"gmail_quote" styl= e=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> - - -<br> <br> Also, I see you&#39;re thinking about methods, which means you&#39;re think= ing about declaring Java classes. How is that supposed to work exactly?<br>= </blockquote><div><br></div><div>Well I was thinking about that with respec= t to calling functions on classes that I have imported from a foreign langu= age, not declared locally for export to a foreign language.</div> <div>I clearly didn&#39;t think that through though, I&#39;ll try and clari= fy my thoughts to that end... I get the feeling that idea was a dead-end th= ough.</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"marg= in:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> I have some experience bridging Objective-C and D. I once built a complete = wrapper system for Objective-C objects, each object was wrapped by a D one.= It worked very well, but it generated so much bloat that it became unusabl= e as soon as I started defining enough classes for it to be useful. See the= D/Objective-C bridge: &lt;<a href=3D"http://michelf.com/projects/d-objc-br= idge/" target=3D"_blank">http://michelf.com/projects/<u></u>d-objc-bridge/<= /a>&gt;.<br> </blockquote><div><br></div><div>What was the primary cause of the bloat? I= can&#39;t imagine my proposal causing any more bloat than the explicit jni= call (or equivalent) woudl have otherwise.</div><div>=C2=A0</div><blockquo= te class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc so= lid;padding-left:1ex"> Maybe you&#39;ll have more luck, but for me wrappers just couldn&#39;t work= . So I decided making the compiler understand the Objective-C runtime and o= bject model was a much better solution. It was fun to do and wasn&#39;t tha= t hard actually. Here&#39;s the result: &lt;<a href=3D"http://michelf.com/p= rojects/d-objc/" target=3D"_blank">http://michelf.com/projects/<u></u>d-obj= c/</a>&gt;. Unfortunately I don&#39;t have much time to work on it anymore.= <br> </blockquote><div><br></div><div>That sounds appropriate for Obj-C, but I d= on&#39;t think it applies to VM based languages however you slice it, since= there&#39;s not actually a direct binary linkage. There&#39;s no way to kn= ow about the Java/.NET/Lua/etc object model internally, the VM owns that en= tirely, so you&#39;re forced to=C2=A0marshal=C2=A0through API&#39;s. That&#= 39;s okay though, you have to do it anyway. I&#39;m proposing a way to unif= y the concept of calling functions without obscuring the actual call with j= unk, and still allowing the D language to annotate and optimise to its full= est extent.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex">More background:<br> &lt;<a href=3D"http://michelf.com/weblog/2010/dobjc-dead-end-start-anew/" t= arget=3D"_blank">http://michelf.com/weblog/<u></u>2010/dobjc-dead-end-start= -<u></u>anew/</a>&gt;<span class=3D"HOEnZb"><font color=3D"#888888"><br> <br> <br> -- <br> Michel Fortin<br> <a href=3D"mailto:michel.fortin michelf.com" target=3D"_blank">michel.forti= n michelf.com</a><br> <a href=3D"http://michelf.com/" target=3D"_blank">http://michelf.com/</a><b= r> <br> </font></span></blockquote></div><br> --00248c711795803b9404b97c65da--
Feb 21 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--0023544719ac8a9cb104b97ea137
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 21 February 2012 19:10, Alex R=C3=B8nne Petersen <xtzgzorex gmail.com> w=
rote:

 On 21-02-2012 18:03, Manu wrote:

 On 21 February 2012 16:59, Michel Fortin <michel.fortin michelf.com
 <mailto:michel.fortin michelf.**com <michel.fortin michelf.com>>> wrote:

    On 2012-02-21 11:03:09 +0000, Manu <turkeyman gmail.com
    <mailto:turkeyman gmail.com>> said:

 Interesting point.
 Java (or any VM based language) can't hard link to the binary, so it
 never could try and call back in through an extern-ed function directly,

Sure it could. Why wouldn't it be able to? This is how extern + DllImport works in C#.

But that only works because C# has DLL support, which knows about the C calling convention, so you just need to export an extern(C) in that case, like any regular DLL. --0023544719ac8a9cb104b97ea137 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div class=3D"gmail_quote">On 21 February 2012 19:10, Alex R=C3=B8nne Peter= sen <span dir=3D"ltr">&lt;<a href=3D"mailto:xtzgzorex gmail.com">xtzgzorex = gmail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style= =3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div class=3D"im">On 21-02-2012 18:03, Manu wrote:<br> </div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"><div class=3D"im"> On 21 February 2012 16:59, Michel Fortin &lt;<a href=3D"mailto:michel.forti= n michelf.com" target=3D"_blank">michel.fortin michelf.com</a><br></div><di= v class=3D"im"> &lt;mailto:<a href=3D"mailto:michel.fortin michelf.com" target=3D"_blank">m= ichel.fortin michelf.<u></u>com</a>&gt;&gt; wrote:<br> <br> =C2=A0 =C2=A0On 2012-02-21 11:03:09 +0000, Manu &lt;<a href=3D"mailto:turk= eyman gmail.com" target=3D"_blank">turkeyman gmail.com</a><br></div><div><d= iv class=3D"h5"> =C2=A0 =C2=A0&lt;mailto:<a href=3D"mailto:turkeyman gmail.com" target=3D"_= blank">turkeyman gmail.com</a>&gt;&gt; said:<br> <br></div></div><div class=3D"im"> Interesting point.<br> Java (or any VM based language) can&#39;t hard link to the binary, so it<br=


<br> Sure it could. Why wouldn&#39;t it be able to? This is how extern + DllImpo= rt works in C#.<br></blockquote><div><br></div><div>But that only works bec= ause C# has DLL support, which knows about the C calling convention, so you= just need to export an extern(C) in that case, like any regular DLL.</div> </div> --0023544719ac8a9cb104b97ea137--
Feb 21 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--00248c711795aee80104b97ea74c
Content-Type: text/plain; charset=UTF-8

On 21 February 2012 20:12, Jacob Carlborg <doob me.com> wrote:

 On 2012-02-21 18:03, Manu wrote:

 On 21 February 2012 16:59, Michel Fortin <michel.fortin michelf.com
    I have some experience bridging Objective-C and D. I once built a
    complete wrapper system for Objective-C objects, each object was
    wrapped by a D one. It worked very well, but it generated so much
    bloat that it became unusable as soon as I started defining enough
    classes for it to be useful. See the D/Objective-C bridge:
    <http://michelf.com/projects/_**_d-objc-bridge/<http://michelf.com/projects/__d-objc-bridge/>
    <http://michelf.com/projects/**d-objc-bridge/<http://michelf.com/projects/d-objc-bridge/>
.


What was the primary cause of the bloat? I can't imagine my proposal causing any more bloat than the explicit jni call (or equivalent) woudl have otherwise.

Template bloat. Every call bridging D/Objective-C is made throw a series of templates. This is for making it possible (less verbose) to create bindings. It might be possible to decrease the template bloat by having a tool that automatically generates the bindings and outputs what the templates do inline.

Why aren't the templates inline themselves? Although if the templates do a lot of work, wouldn't that INCREASE the code volume? I can't really imagine how Obj-C linkage could bloat so much, what was involved? What did you have to do in addition to what a regular Obj-C function call would have done? --00248c711795aee80104b97ea74c Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div class=3D"gmail_quote">On 21 February 2012 20:12, Jacob Carlborg <span = dir=3D"ltr">&lt;<a href=3D"mailto:doob me.com">doob me.com</a>&gt;</span> w= rote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;borde= r-left:1px #ccc solid;padding-left:1ex"> <div class=3D"im">On 2012-02-21 18:03, Manu wrote:<br> </div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"><div class=3D"im"> On 21 February 2012 16:59, Michel Fortin &lt;<a href=3D"mailto:michel.forti= n michelf.com" target=3D"_blank">michel.fortin michelf.com</a><br></div><di= v class=3D"im"> =C2=A0 =C2=A0I have some experience bridging Objective-C and D. I once bui= lt a<br> =C2=A0 =C2=A0complete wrapper system for Objective-C objects, each object = was<br> =C2=A0 =C2=A0wrapped by a D one. It worked very well, but it generated so = much<br> =C2=A0 =C2=A0bloat that it became unusable as soon as I started defining e= nough<br> =C2=A0 =C2=A0classes for it to be useful. See the D/Objective-C bridge:<br=
</div><div class=3D"im">

target=3D"_blank">http://michelf.com/projects/_<u></u>_d-objc-bridge/</a><b= r> =C2=A0 =C2=A0&lt;<a href=3D"http://michelf.com/projects/d-objc-bridge/" ta= rget=3D"_blank">http://michelf.com/projects/<u></u>d-objc-bridge/</a>&gt;&g= t;.<br> <br> <br> What was the primary cause of the bloat? I can&#39;t imagine my proposal<br=

have otherwise.<br> </div></blockquote> <br> Template bloat. Every call bridging D/Objective-C is made throw a series of= templates. This is for making it possible (less verbose) to create binding= s.<br> <br> It might be possible to decrease the template bloat by having a tool that a= utomatically generates the bindings and outputs what the templates do inlin= e.</blockquote><div><br></div><div>Why aren&#39;t the templates inline them= selves? Although if the templates do a lot of work, wouldn&#39;t that INCRE= ASE the code volume?</div> <div>I can&#39;t really imagine how Obj-C linkage could bloat so much, what= was involved? What did you have to do in addition to what a regular Obj-C = function call would have done?</div></div> --00248c711795aee80104b97ea74c--
Feb 21 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--002354530350a223d004b97ed841
Content-Type: text/plain; charset=UTF-8

On 21 February 2012 21:03, Paulo Pinto <pjmlp progtools.org> wrote:

 I think this fails in the same trap as extern "language" in C++.

 Besides C and C++, I think the only variation I saw so far for the
 language part has been fortran, and cannot recall anylonger in what
 compiler it was.

 In C++'s case, since it is implementation defined, most compiler vendors
 did not bother to support more than what the standard specifies (C and
 C++).

 In you proposal, how to keep the feature portable across D implementations?

Well since the languages I'm using as examples are VM based, the interaction is through a standardised API, which is most likely part of the same module that defines the custom extern type, both would be presented in the same package. If you make it part of D language specification, which languages would be
 the chosen ones to be made available in any D compiler?

I'm not making any language part of the D specification, I'm suggesting a system to integrate custom calling code, which can be introduced in libraries. No intrinsic support in D for any particular language. How to garantee correct interoperability (ABI) with other languages? After
 deciding the set of chosen languages, which compilers/runtimes would be the
 lucky ones?

VM based languages have a strictly defined API, this is guaranteed, no problem. Extern to hard-linking languages requires the extra bits I detailed in a prior post (name mangler, call receiver), and probably some version() mess to support different compilers just like C/C++ does all over the place. Many C compilers can't link against each others binaries, but with a mature lib, you'd be able to support the calling convention of popular compilers for your language of interest I would think. t's probably quite do-able, but I'm not proposing to take the system that far initially. Custom calling conventions to integrate with VM's seems more useful, and much simpler to start with. --002354530350a223d004b97ed841 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div class=3D"gmail_quote">On 21 February 2012 21:03, Paulo Pinto <span dir= =3D"ltr">&lt;<a href=3D"mailto:pjmlp progtools.org">pjmlp progtools.org</a>= &gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0= 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> I think this fails in the same trap as extern &quot;language&quot; in C++.<= br> <br> Besides C and C++, I think the only variation I saw so far for the language= part has been fortran, and cannot recall anylonger in what<br> compiler it was.<br> <br> In C++&#39;s case, since it is implementation defined, most compiler vendor= s<br> did not bother to support more than what the standard specifies (C and C++)= .<br> <br> In you proposal, how to keep the feature portable across D implementations?= <br></blockquote><div><br></div><div>Well since the languages I&#39;m using= as examples are VM based, the interaction is through a standardised API, w= hich is most likely part of the same module that defines the custom extern = type, both would be presented in the same package.</div> <div><br></div><div><br></div><blockquote class=3D"gmail_quote" style=3D"ma= rgin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">If you make it= part of D language specification, which languages would be the chosen ones= to be made available in any D compiler?<br> </blockquote><div><br></div><div>I&#39;m not making any language part of th= e D specification, I&#39;m suggesting a system to integrate custom calling = code, which can be introduced in libraries. No intrinsic support in D for a= ny particular language.</div> <div><br></div><div><br></div><blockquote class=3D"gmail_quote" style=3D"ma= rgin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">How to garante= e correct interoperability (ABI) with other languages? After deciding the s= et of chosen languages, which compilers/runtimes would be the lucky ones?<b= r> </blockquote><div><br></div><div>VM based languages have a strictly defined= API, this is guaranteed, no problem.</div><div>Extern to hard-linking lang= uages requires the extra bits I detailed in a prior post (name mangler, cal= l receiver), and probably some version() mess to support different compiler= s just like C/C++ does all over the place. Many C compilers can&#39;t link = against each others binaries, but with a mature lib, you&#39;d be able to s= upport the calling convention of popular compilers for your language of int= erest I would think.</div> <div>t&#39;s probably quite do-able, but I&#39;m not proposing to take the = system that far initially. Custom calling conventions to integrate with VM&= #39;s seems more useful, and much simpler to start with.</div></div> --002354530350a223d004b97ed841--
Feb 21 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--f46d04138a9d1d067f04b97fb649
Content-Type: text/plain; charset=UTF-8

On 21 February 2012 22:35, Jacob Carlborg <doob me.com> wrote:

 On 2012-02-21 20:45, Manu wrote:

 On 21 February 2012 20:12, Jacob Carlborg <doob me.com
 <mailto:doob me.com>> wrote:

    On 2012-02-21 18:03, Manu wrote:

        On 21 February 2012 16:59, Michel Fortin
        <michel.fortin michelf.com
<mailto:michel.fortin michelf.**com<michel.fortin michelf.com>

I have some experience bridging Objective-C and D. I once built a complete wrapper system for Objective-C objects, each object was wrapped by a D one. It worked very well, but it generated so much bloat that it became unusable as soon as I started defining enough classes for it to be useful. See the D/Objective-C bridge: <http://michelf.com/projects/_**___d-objc-bridge/<http://michelf.com/projects/____d-objc-bridge/> <http://michelf.com/projects/_**_d-objc-bridge/<http://michelf.com/projects/__d-objc-bridge/>

<http://michelf.com/projects/_**_d-objc-bridge/<http://michelf.com/projects/__d-objc-bridge/> <http://michelf.com/projects/**d-objc-bridge/<http://michelf.com/projects/d-objc-bridge/>
.



What was the primary cause of the bloat? I can't imagine my proposal causing any more bloat than the explicit jni call (or equivalent) woudl have otherwise. Template bloat. Every call bridging D/Objective-C is made throw a series of templates. This is for making it possible (less verbose) to create bindings. It might be possible to decrease the template bloat by having a tool that automatically generates the bindings and outputs what the templates do inline. Why aren't the templates inline themselves? Although if the templates do a lot of work, wouldn't that INCREASE the code volume? I can't really imagine how Obj-C linkage could bloat so much, what was involved? What did you have to do in addition to what a regular Obj-C function call would have done?

Michel Fortin explained this better in two of his answers. You can also read the documentation of my implementation, very similar to Michel's: http://www.dsource.org/**projects/dstep/wiki/**ObjcBridge/BridgeInternals<http://www.dsource.org/projects/dstep/wiki/ObjcBridge/BridgeInternals> But to give a quick example: class Foo : NSObject { Foo foo () { return invokeObjcSelf!(Foo, "foo"); } Foo bar () { return invokeObjcSelf!(Foo, "bar"); } } "invokeObjcSelf" is a template function that calls an Objective-C method. Basically each time "invokeObjcSelf" is called a new instantiation of the template is created and that is put in the symbol table. "invokeObjcSelf" then calls several more template functions making the template bloat increase exponentially.

But they should all be inlined, and the symbol table should be stripped, which shouldn't leave anything in the end other than the inlined function calling code, and in my examples, this will be basically the exact same code that you'd have to write anyway to call through some vm's API... --f46d04138a9d1d067f04b97fb649 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div class=3D"gmail_quote">On 21 February 2012 22:35, Jacob Carlborg <span = dir=3D"ltr">&lt;<a href=3D"mailto:doob me.com">doob me.com</a>&gt;</span> w= rote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;borde= r-left:1px #ccc solid;padding-left:1ex"> <div class=3D"im">On 2012-02-21 20:45, Manu wrote:<br> </div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"><div class=3D"im"> On 21 February 2012 20:12, Jacob Carlborg &lt;<a href=3D"mailto:doob me.com= " target=3D"_blank">doob me.com</a><br></div><div class=3D"im"> &lt;mailto:<a href=3D"mailto:doob me.com" target=3D"_blank">doob me.com</a>= &gt;&gt; wrote:<br> <br> =C2=A0 =C2=A0On 2012-02-21 18:03, Manu wrote:<br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0On 21 February 2012 16:59, Michel Fortin<br></d= iv> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"mailto:michel.fortin michelf.com= " target=3D"_blank">michel.fortin michelf.com</a> &lt;mailto:<a href=3D"mai= lto:michel.fortin michelf.com" target=3D"_blank">michel.fortin michelf.<u><= /u>com</a>&gt;<div class=3D"im"> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0I have some experience bridging O= bjective-C and D. I once<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0built a<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0complete wrapper system for Objec= tive-C objects, each object was<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0wrapped by a D one. It worked ver= y well, but it generated so<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0much<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0bloat that it became unusable as = soon as I started defining<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0enough<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0classes for it to be useful. See = the D/Objective-C bridge:<br></div> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"http://michelf.com/projects/____= d-objc-bridge/" target=3D"_blank">http://michelf.com/projects/_<u></u>___d-= objc-bridge/</a><br> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"http://michelf.com/projects/__d-= objc-bridge/" target=3D"_blank">http://michelf.com/projects/_<u></u>_d-objc= -bridge/</a>&gt;<div class=3D"im"><br> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"http://michelf.com/projects/__d-= objc-bridge/" target=3D"_blank">http://michelf.com/projects/_<u></u>_d-objc= -bridge/</a><br> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"http://michelf.com/projects/d-ob= jc-bridge/" target=3D"_blank">http://michelf.com/projects/<u></u>d-objc-bri= dge/</a>&gt;&gt;&gt;.<br> <br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0What was the primary cause of the bloat? I can&= #39;t imagine my proposal<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0causing any more bloat than the explicit jni ca= ll (or<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0equivalent) woudl<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0have otherwise.<br> <br> <br> =C2=A0 =C2=A0Template bloat. Every call bridging D/Objective-C is made thr= ow a<br> =C2=A0 =C2=A0series of templates. This is for making it possible (less ver= bose)<br> =C2=A0 =C2=A0to create bindings.<br> <br> =C2=A0 =C2=A0It might be possible to decrease the template bloat by having= a tool<br> =C2=A0 =C2=A0that automatically generates the bindings and outputs what th= e<br> =C2=A0 =C2=A0templates do inline.<br> <br> <br></div><div class=3D"im"> Why aren&#39;t the templates inline themselves? Although if the templates d= o<br> a lot of work, wouldn&#39;t that INCREASE the code volume?<br> I can&#39;t really imagine how Obj-C linkage could bloat so much, what was<= br> involved? What did you have to do in addition to what a regular Obj-C<br> function call would have done?<br> </div></blockquote> <br> Michel Fortin explained this better in two of his answers. You can also rea= d the documentation of my implementation, very similar to Michel&#39;s:<br> <br> <a href=3D"http://www.dsource.org/projects/dstep/wiki/ObjcBridge/BridgeInte= rnals" target=3D"_blank">http://www.dsource.org/<u></u>projects/dstep/wiki/= <u></u>ObjcBridge/BridgeInternals</a><br> <br> But to give a quick example:<br> <br> class Foo : NSObject<br> { =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 <br> =C2=A0 =C2=A0Foo foo ()<br> =C2=A0 =C2=A0{<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0return invokeObjcSelf!(Foo, &quot;foo&quot;);<b= r> =C2=A0 =C2=A0}<br> <br> =C2=A0 =C2=A0Foo bar ()<br> =C2=A0 =C2=A0{<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0return invokeObjcSelf!(Foo, &quot;bar&quot;);<b= r> =C2=A0 =C2=A0}<br> }<br> <br> &quot;invokeObjcSelf&quot; is a template function that calls an Objective-C= method. Basically each time &quot;invokeObjcSelf&quot; is called a new ins= tantiation of the template is created and that is put in the symbol table. = &quot;invokeObjcSelf&quot; then calls several more template functions makin= g the template bloat increase exponentially.</blockquote> <div><br></div><div>But they should all be inlined, and the symbol table sh= ould be stripped, which shouldn&#39;t leave anything in the end other than = the inlined function calling code, and in my examples, this will be basical= ly the exact same code that you&#39;d have to write anyway to call through = some vm&#39;s API...</div> </div> --f46d04138a9d1d067f04b97fb649--
Feb 21 2012
prev sibling parent Manu <turkeyman gmail.com> writes:
--0023544719ac3c04bb04b9818f28
Content-Type: text/plain; charset=UTF-8

On 21 February 2012 23:06, Jacob Carlborg <doob me.com> wrote:

 On 2012-02-21 22:01, Manu wrote:

 On 21 February 2012 22:35, Jacob Carlborg <doob me.com

 <mailto:doob me.com>> wrote:

    On 2012-02-21 20:45, Manu wrote:

        On 21 February 2012 20:12, Jacob Carlborg <doob me.com
        <mailto:doob me.com>
        <mailto:doob me.com <mailto:doob me.com>>> wrote:

            On 2012-02-21 18:03, Manu wrote:

                On 21 February 2012 16:59, Michel Fortin
        <michel.fortin michelf.com
<mailto:michel.fortin michelf.**com<michel.fortin michelf.com>

<mailto:michel.fortin michelf.**com <michel.fortin michelf.com>>> I have some experience bridging Objective-C and D. I once built a complete wrapper system for Objective-C objects, each object was wrapped by a D one. It worked very well, but it generated so much bloat that it became unusable as soon as I started defining enough classes for it to be useful. See the D/Objective-C bridge: <http://michelf.com/projects/_**_____d-objc-bridge/<http://michelf.com/projects/______d-objc-bridge/> <http://michelf.com/projects/_**___d-objc-bridge/<http://michelf.com/projects/____d-objc-bridge/>

<http://michelf.com/projects/_**___d-objc-bridge/<http://michelf.com/projects/____d-objc-bridge/> <http://michelf.com/projects/_**_d-objc-bridge/<http://michelf.com/projects/__d-objc-bridge/>


<http://michelf.com/projects/_**___d-objc-bridge/<http://michelf.com/projects/____d-objc-bridge/> <http://michelf.com/projects/_**_d-objc-bridge/<http://michelf.com/projects/__d-objc-bridge/>

<http://michelf.com/projects/**d-objc-bridge/<http://michelf.com/projects/d-objc-bridge/>
.




What was the primary cause of the bloat? I can't imagine my proposal causing any more bloat than the explicit jni call (or equivalent) woudl have otherwise. Template bloat. Every call bridging D/Objective-C is made throw a series of templates. This is for making it possible (less verbose) to create bindings. It might be possible to decrease the template bloat by having a tool that automatically generates the bindings and outputs what the templates do inline. Why aren't the templates inline themselves? Although if the templates do a lot of work, wouldn't that INCREASE the code volume? I can't really imagine how Obj-C linkage could bloat so much, what was involved? What did you have to do in addition to what a regular Obj-C function call would have done? Michel Fortin explained this better in two of his answers. You can also read the documentation of my implementation, very similar to Michel's: http://www.dsource.org/__**projects/dstep/wiki/__** ObjcBridge/BridgeInternals<http://www.dsource.org/__projects/dstep/wiki/__ObjcBridge/BridgeInternals> <http://www.dsource.org/**projects/dstep/wiki/** ObjcBridge/BridgeInternals<http://www.dsource.org/projects/dstep/wiki/ObjcBridge/BridgeInternals>

But to give a quick example: class Foo : NSObject { Foo foo () { return invokeObjcSelf!(Foo, "foo"); } Foo bar () { return invokeObjcSelf!(Foo, "bar"); } } "invokeObjcSelf" is a template function that calls an Objective-C method. Basically each time "invokeObjcSelf" is called a new instantiation of the template is created and that is put in the symbol table. "invokeObjcSelf" then calls several more template functions making the template bloat increase exponentially. But they should all be inlined, and the symbol table should be stripped, which shouldn't leave anything in the end other than the inlined function calling code, and in my examples, this will be basically the exact same code that you'd have to write anyway to call through some vm's API...

Well, that's not what happen with templates.

... really? why? You've just made me very very scared for my simd module :/ I've had a serious concern about D's lack of a force-inline for quite a while... --0023544719ac3c04bb04b9818f28 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div class=3D"gmail_quote">On 21 February 2012 23:06, Jacob Carlborg <span = dir=3D"ltr">&lt;<a href=3D"mailto:doob me.com">doob me.com</a>&gt;</span> w= rote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;borde= r-left:1px #ccc solid;padding-left:1ex"> On 2012-02-21 22:01, Manu wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On 21 February 2012 22:35, Jacob Carlborg &lt;<a href=3D"mailto:doob me.com= " target=3D"_blank">doob me.com</a><div class=3D"im"><br> &lt;mailto:<a href=3D"mailto:doob me.com" target=3D"_blank">doob me.com</a>= &gt;&gt; wrote:<br> <br> =C2=A0 =C2=A0On 2012-02-21 20:45, Manu wrote:<br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0On 21 February 2012 20:12, Jacob Carlborg &lt;<= a href=3D"mailto:doob me.com" target=3D"_blank">doob me.com</a><br> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;mailto:<a href=3D"mailto:doob me.com" targe= t=3D"_blank">doob me.com</a>&gt;<br></div><div class=3D"im"> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;mailto:<a href=3D"mailto:doob me.com" targe= t=3D"_blank">doob me.com</a> &lt;mailto:<a href=3D"mailto:doob me.com" targ= et=3D"_blank">doob me.com</a>&gt;&gt;&gt; wrote:<br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0On 2012-02-21 18:03, Manu wrote:<= br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0On 21 February 2012= 16:59, Michel Fortin<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"mailto:michel.fortin michelf.com= " target=3D"_blank">michel.fortin michelf.com</a> &lt;mailto:<a href=3D"mai= lto:michel.fortin michelf.com" target=3D"_blank">michel.fortin michelf.<u><= /u>com</a>&gt;<br></div> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;mailto:<a href=3D"mailto:michel.fortin mich= elf." target=3D"_blank">michel.fortin michelf.</a><u></u>__com<div class=3D= "im"><br> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;mailto:<a href=3D"mailto:michel.fortin mich= elf.com" target=3D"_blank">michel.fortin michelf.<u></u>com</a>&gt;&gt;<br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0I hav= e some experience bridging Objective-C and D. I<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0once<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0built a<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0compl= ete wrapper system for Objective-C objects,<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0each object was<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0wrapp= ed by a D one. It worked very well, but it<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0generated so<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0much<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0bloat= that it became unusable as soon as I started<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0defining<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0enough<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0class= es for it to be useful. See the D/Objective-C<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0bridge:<br></div> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"http://michelf.com/projects/____= __d-objc-bridge/" target=3D"_blank">http://michelf.com/projects/_<u></u>___= __d-objc-bridge/</a><br> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"http://michelf.com/projects/____= d-objc-bridge/" target=3D"_blank">http://michelf.com/projects/_<u></u>___d-= objc-bridge/</a>&gt;<div><div class=3D"h5"><br> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"http://michelf.com/projects/____= d-objc-bridge/" target=3D"_blank">http://michelf.com/projects/_<u></u>___d-= objc-bridge/</a><br> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"http://michelf.com/projects/__d-= objc-bridge/" target=3D"_blank">http://michelf.com/projects/_<u></u>_d-objc= -bridge/</a>&gt;&gt;<br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"http://michelf.com/projects/____= d-objc-bridge/" target=3D"_blank">http://michelf.com/projects/_<u></u>___d-= objc-bridge/</a><br> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"http://michelf.com/projects/__d-= objc-bridge/" target=3D"_blank">http://michelf.com/projects/_<u></u>_d-objc= -bridge/</a>&gt;<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"http://michelf.com/projects/__d-= objc-bridge/" target=3D"_blank">http://michelf.com/projects/_<u></u>_d-objc= -bridge/</a><br> =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"http://michelf.com/projects/d-ob= jc-bridge/" target=3D"_blank">http://michelf.com/projects/<u></u>d-objc-bri= dge/</a>&gt;&gt;&gt;&gt;.<br> <br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0What was the primar= y cause of the bloat? I can&#39;t imagine<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0my proposal<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0causing any more bl= oat than the explicit jni call (or<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0equivalent) woudl<b= r> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0have otherwise.<br> <br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Template bloat. Every call bridgi= ng D/Objective-C is made<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0throw a<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0series of templates. This is for = making it possible (less<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0verbose)<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0to create bindings.<br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0It might be possible to decrease = the template bloat by<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0having a tool<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0that automatically generates the = bindings and outputs what the<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0templates do inline.<br> <br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0Why aren&#39;t the templates inline themselves?= Although if the<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0templates do<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0a lot of work, wouldn&#39;t that INCREASE the c= ode volume?<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0I can&#39;t really imagine how Obj-C linkage co= uld bloat so much,<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0what was<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0involved? What did you have to do in addition t= o what a regular<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0Obj-C<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0function call would have done?<br> <br> <br> =C2=A0 =C2=A0Michel Fortin explained this better in two of his answers. Yo= u can<br> =C2=A0 =C2=A0also read the documentation of my implementation, very simila= r to<br> =C2=A0 =C2=A0Michel&#39;s:<br> <br></div></div> =C2=A0 =C2=A0<a href=3D"http://www.dsource.org/__projects/dstep/wiki/__Obj= cBridge/BridgeInternals" target=3D"_blank">http://www.dsource.org/__<u></u>= projects/dstep/wiki/__<u></u>ObjcBridge/BridgeInternals</a><div class=3D"im= "><br> =C2=A0 =C2=A0&lt;<a href=3D"http://www.dsource.org/projects/dstep/wiki/Obj= cBridge/BridgeInternals" target=3D"_blank">http://www.dsource.org/<u></u>pr= ojects/dstep/wiki/<u></u>ObjcBridge/BridgeInternals</a>&gt;<br> <br> =C2=A0 =C2=A0But to give a quick example:<br> <br> =C2=A0 =C2=A0class Foo : NSObject<br> =C2=A0 =C2=A0{<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0Foo foo ()<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0{<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return invokeObjcSelf!(Foo, &quot= ;foo&quot;);<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0}<br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0Foo bar ()<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0{<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return invokeObjcSelf!(Foo, &quot= ;bar&quot;);<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0}<br> =C2=A0 =C2=A0}<br> <br> =C2=A0 =C2=A0&quot;invokeObjcSelf&quot; is a template function that calls = an Objective-C<br> =C2=A0 =C2=A0method. Basically each time &quot;invokeObjcSelf&quot; is cal= led a new<br> =C2=A0 =C2=A0instantiation of the template is created and that is put in t= he<br> =C2=A0 =C2=A0symbol table. &quot;invokeObjcSelf&quot; then calls several m= ore template<br> =C2=A0 =C2=A0functions making the template bloat increase exponentially.<b= r> <br> <br></div><div class=3D"im"> But they should all be inlined, and the symbol table should be stripped,<br=

function calling code, and in my examples, this will be basically the<br> exact same code that you&#39;d have to write anyway to call through some<br=

</div></blockquote> <br> Well, that&#39;s not what happen with templates.</blockquote><div><br></div=
<div>... really? why?</div><div>You&#39;ve just made me very very scared f=

n about D&#39;s lack of a force-inline for quite a while...</div> </div> --0023544719ac3c04bb04b9818f28--
Feb 21 2012