www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Another "D is cool" post

reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
So, recently in one of my pet projects I have a bit of code that takes a
string, fills in a code template, invokes the D compiler to create a
shared object, then loads the object with dlopen() and calls dlsym() to
get the entry point into the compiled code as a function pointer.

I've written the equivalent code in C before -- and while it worked, it
required a lot of care to ensure things don't blow up, or if they do,
that errors are correctly handled and properly cleaned up. This time
round, though, I found several aspects of D very nice for writing this
sort of code:

1) String-processing: the input string needs some processing before
being put into the code template, so here I get to use nice built-in
facilities like std.array.split. Whereas in the C version, I'd have to
use things like strchr or strstr, carefully make copies of the results
or, if the input string was writable, insert null terminators (which
makes for a poorer API from a design POV), then take care to free
buffers afterwards.  Normal course of the day for writing C code, it's
true, but rather fidgety, error-prone, and, after having written string
manipulation C code for the n'th time, it gets tiring really fast. The
resulting code in D is much simpler (only 4-5 lines; the C equivalent
would probably take about twice that, what with null checking, copying
buffers, etc.), more readable, and more maintainable.

2) Instantiating the code template and invoking the compiler: in D, it
just takes a single call to std.file.write() to write the code template
to a temporary file. Very handy. In C, but I'd have to worry about
dealing with FILE*, handling errors, not forgetting to fclose(), etc..
Again, normal course of the day for writing C code, but just more
fidgety and error-prone.  Invoking the compiler in C involves trickery:
I'd have to explicitly fork-and-exec, which means lots of manual
housekeeping to make sure fork succeeds, keep track of whether I'm in
the parent or child, setting up the arguments to exec*(), manually
handle return codes, etc.. In D, thanks to the awesome API of
std.process (kudos to Lars Kyllingstad, et al, for the excellent revamp
of the original klunky std.process), all I need to do is:

	auto result = execute([
		compiler,
		option1,
		option2,
		srcFile,
		"-of" ~ soFile
	]);
	if (result.status != 0) { /* handle compiler errors */ }

No need to manually fiddle with fork(), dup(), close(), execv(), and all
the paraphrenalia associated with them.

3) Even better yet, std.process.execute captures compiler output
automatically. In C, if I want to do that, I'd have to deal with
manually redirecting stdout/stderr, set up a pipe for copying the output
to a buffer, manage the buffer, etc.. Again, nothing out of the ordinary
for a C coder but it's just a lot of fussy housekeeping that reduces
productivity.  In D, std.process handles it all nicely for me -- plus it
works on Windows too, should I ever desire to compile the code on
Windows.  In C, I'd have to rewrite the entire code to use Windows API
calls instead of Posix calls -- a major undertaking.  (Of course,
replacing dlopen & co with Windows calls is a different story -- perhaps
Phobos should have a new module for handling this!)

4) As it turns out, I *did* need to capture the compiler's output
eventually.  Here's where some of D's features made things highly
productive: the compiled code template declares a public symbol that the
main program needs to know, so that it know what symbol to look up with
dlsym() when loading the resulting shared object.  I could easily
hardcode the mangling of this symbol, since I control what goes in the
code template.  However, it's ugly, and not future-proof: if D's
mangling scheme were to change in the future, the code would break.
Plus, it's just ugly to have to manually write out a mangled symbol when
the compiler can already do it for you.

The tricky part, though, is that .mangleof only works on an identifier
defined in the *current* program; the compiler can't do it for a symbol
in a string that's to be passed at runtime to another invocation of the
compiler.  And AFAIK, there's currently no way to ask the compiler "what
would be the mangling of mymodule.symbol?" if 'mymodule' and 'symbol'
only exist in the shared object, not in the main program.

So the solution is to insert a `pragma(msg, symbol.mangleof)` in the
code template, and have the main program parse the output of the
compiler when it compiles the shared object, so that it learns the
mangled identifier directly.  If this were C++ code, I'd be stuck up the
creek without a paddle: hardcoding the mangled symbol would be
unavoidable. But thanks to pragma(msg) and .mangleof in D, this was a
cinch.  Not to mention that std.process.execute already captures output
for me automatically, so all I need to do is to search for the symbol in
the captured output.

(Granted, in C there'd be no need for this dance, since I could just use
the (unmangled) symbol directly. But still, it's nice that D provides
the facilities for dealing with identifiers easily even in the face of
mangling schemes.)

5) Exceptions in Phobos are not a bad thing: they help greatly
streamline the code that interacts with the OS: std.process.execute,
std.file.write, etc.. I can just write the steps sequentially without
the verbosity of C's `if ((err = syscall(...)) != 0) goto CLEANUP;`
dance; exceptions take care of handling error conditions for me without
uglifying the code.


All in all, I was able to write all of this code in just few hours'
worth of coding and unittesting, whereas the C equivalent took me
several days.  Hooray for D!


T

-- 
Customer support: the art of getting your clients to pay for your own
incompetence.
May 29 2017
next sibling parent reply David Gileadi <gileadisNOSPM gmail.com> writes:
On 5/29/17 12:07 PM, H. S. Teoh via Digitalmars-d wrote:

 [snip an excellent post]


I think a longish post like this would make an excellent shortish post 
for the D blog.

In any case, great writeup!
May 29 2017
parent reply Mike Parker <aldacron gmail.com> writes:
On Monday, 29 May 2017 at 19:51:26 UTC, David Gileadi wrote:

 On 5/29/17 12:07 PM, H. S. Teoh via Digitalmars-d wrote:

 [snip an excellent post]


 I think a longish post like this would make an excellent 
 shortish post for the D blog.


Yes, please.
May 29 2017
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, May 30, 2017 at 01:44:58AM +0000, Mike Parker via Digitalmars-d wrote:

 On Monday, 29 May 2017 at 19:51:26 UTC, David Gileadi wrote:

 On 5/29/17 12:07 PM, H. S. Teoh via Digitalmars-d wrote:

 [snip an excellent post]


 I think a longish post like this would make an excellent shortish
 post for the D blog.


 
 Yes, please.


OK, this is the 3rd or 4th time somebody asked about this.  What exactly
is involved in making a post on the D blog?  Hopefully it would not
require too much more effort, because I usually wouldn't have much time
to spend on top of the time I already spent writing the original post.
If it's just a matter of copy-n-pasting the text somewhere, then perhaps
somebody could do that for me?


T

-- 
Those who've learned LaTeX swear by it. Those who are learning LaTeX swear at
it. -- Pete Bleackley
May 29 2017
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Tuesday, 30 May 2017 at 05:14:21 UTC, H. S. Teoh wrote:


 OK, this is the 3rd or 4th time somebody asked about this.  
 What exactly is involved in making a post on the D blog?  
 Hopefully it would not require too much more effort, because I 
 usually wouldn't have much time to spend on top of the time I 
 already spent writing the original post. If it's just a matter 
 of copy-n-pasting the text somewhere, then perhaps somebody 
 could do that for me?


For a guest post, you write it up and submit it to me (plain 
text, Markdown, HTML, gist, whatever, aldacron gmail.com). I go 
through a few editing passes and submit my suggestions to you for 
approval. Then I take care of the rest. After that, you'll get 
email notifications when someone comments on the post, and it's a 
good idea to watch the reddit thread, but that's it.
May 29 2017
prev sibling next sibling parent qznc <qznc web.de> writes:
On Tuesday, 30 May 2017 at 05:14:21 UTC, H. S. Teoh wrote:

 On Tue, May 30, 2017 at 01:44:58AM +0000, Mike Parker via 
 Digitalmars-d wrote:

 On Monday, 29 May 2017 at 19:51:26 UTC, David Gileadi wrote:

 On 5/29/17 12:07 PM, H. S. Teoh via Digitalmars-d wrote:

 [snip an excellent post]


 I think a longish post like this would make an excellent 
 shortish post for the D blog.


 
 Yes, please.


 OK, this is the 3rd or 4th time somebody asked about this.  
 What exactly is involved in making a post on the D blog?  
 Hopefully it would not require too much more effort, because I 
 usually wouldn't have much time to spend on top of the time I 
 already spent writing the original post. If it's just a matter 
 of copy-n-pasting the text somewhere, then perhaps somebody 
 could do that for me?


For a more general audience, I would suggest to expand the first 
paragraph and tell more about your pet project.

Some parts should be extended, so non-D-programmers can 
understand it. For example, the mention of `pragma` is probably 
confusing for C/C++ people.

Inserting  a few links would be nice (e.g. for std.array.split).

If you really want to put in work, insert lots of code examples 
with C and D versions.

Finally, a question: Did you count lines of code C vs D? Or 
better yet, number of tokens?

Anyways, I was tempted to send the forum link around, so it is 
already a nice blog post as it is.
May 30 2017
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2017-05-30 07:14, H. S. Teoh via Digitalmars-d wrote:


 OK, this is the 3rd or 4th time somebody asked about this.  What exactly
 is involved in making a post on the D blog?  Hopefully it would not
 require too much more effort, because I usually wouldn't have much time
 to spend on top of the time I already spent writing the original post.
 If it's just a matter of copy-n-pasting the text somewhere, then perhaps
 somebody could do that for me?


When I wrote my blog post [1] I wrote it in without much thought, I just 
describe how DVM works, I didn't even have an ending. I wrote it in 
Markdown and emailed it to Mike Parker. He gave me some input on the 
text which I fixed. He then added an ending, added the bio and converted 
the Markdown to whatever the blog site is using.

[1] http://dlang.org/blog/2016/08/17/inside-d-version-manager

-- 
/Jacob Carlborg
May 30 2017
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/29/2017 6:44 PM, Mike Parker wrote:

 On Monday, 29 May 2017 at 19:51:26 UTC, David Gileadi wrote:

 On 5/29/17 12:07 PM, H. S. Teoh via Digitalmars-d wrote:

 [snip an excellent post]


 I think a longish post like this would make an excellent shortish post for the 
 D blog.


 
 Yes, please.
 


I agree. Do it!
May 30 2017
prev sibling next sibling parent aberba <karabutaworld gmail.com> writes:
On Monday, 29 May 2017 at 19:07:03 UTC, H. S. Teoh wrote:

 So, recently in one of my pet projects I have a bit of code 
 that takes a string, fills in a code template, invokes the D 
 compiler to create a shared object, then loads the object with 
 dlopen() and calls dlsym() to get the entry point into the 
 compiled code as a function pointer.

 [...]


Besides the syntax (clean code),  productivity is one main reason 
I choose D as my primary language.
May 29 2017
prev sibling parent reply Guillaume Boucher <guillaume.boucher.d gmail.com> writes:
On Monday, 29 May 2017 at 19:07:03 UTC, H. S. Teoh wrote:

 So, recently in one of my pet projects I have a bit of code 
 that takes a string, fills in a code template, invokes the D 
 compiler to create a shared object, then loads the object with 
 dlopen() and calls dlsym() to get the entry point into the 
 compiled code as a function pointer.


Seems like the perfect job for a scripting language like Python.

I'm not sure why you decided to compare everything to C.  Even C 
programmers will agree with you that in D you can do things in a 
shorter way -- just slower/bloated/more magic/with less 
control/not portable/<that C programmers favourite excuse>.

In all the examples you bring, there's nothing special about D.  
You can do anything in C++/Go/Rust/Swift/Python/<any other non-C 
language> with comparable complexity.


 The tricky part, though, is that .mangleof only works on an 
 identifier defined in the *current* program; the compiler can't 
 do it for a symbol in a string that's to be passed at runtime 
 to another invocation of the compiler.  And AFAIK, there's 
 currently no way to ask the compiler "what would be the 
 mangling of mymodule.symbol?" if 'mymodule' and 'symbol' only 
 exist in the shared object, not in the main program.


Not a problem in any other language (C++ has a well-defined ABI, 
dynamic languages don't need that).

I enjoy D, but some of those fanboy posts are just totally 
worthless.
May 29 2017
parent bachmeier <no spam.net> writes:
On Monday, 29 May 2017 at 19:54:22 UTC, Guillaume Boucher wrote:


 Seems like the perfect job for a scripting language like Python.

 I'm not sure why you decided to compare everything to C.  Even 
 C programmers will agree with you that in D you can do things 
 in a shorter way -- just slower/bloated/more magic/with less 
 control/not portable/<that C programmers favourite excuse>.


Just to be sure, you realize D is not a scripting language, don't 
you? This is a rather confused argument. First you say a 
scripting language would be perfect for this case (without any 
explanation of how it would be better than D) and then you list a 
bunch of objections that would be a valid response to someone 
doing this in a scripting language but not D.
May 29 2017