www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Writing some built-in functions for Bash, possible?

reply Ky-Anh Huynh <saigon example.net> writes:
Hi,

I'm using Bash heavily in my systems. Things become slow and slow 
when I have tons of scripts :) And sometimes it's not easy to 
manipulate data.

You may have heard of recutils [1] which has a C extension to be 
loaded by Bash. Is it possible to write similar things in D, for 
Bash? I am not good at C; it's great if I explore this field:)

Some examples in C are in [2].

My experience: Dlang has `pipe` support however the syntax is not 
as clean as Bash :) Most of the times I see short (<1k loc) Bash 
scripts are easy to maintain than Ruby (and now D things) scripts.

Thanks for your reading.

[1]: https://news.ycombinator.com/item?id=15302035
[2]: 
http://git.savannah.gnu.org/cgit/bash.git/tree/examples/loadables/cat.c
Oct 17 2017
next sibling parent Ky-Anh Huynh <saigon example.net> writes:
On Wednesday, 18 October 2017 at 03:48:01 UTC, Ky-Anh Huynh wrote:

 Some examples in C are in [2].

 My experience: Dlang has `pipe` support however the syntax is 
 not as clean as Bash :) Most of the times I see short (<1k loc) 
 Bash scripts are easy to maintain than Ruby (and now D things) 
 scripts.


And yeah writing in Bash has a lot of side effects ^.^
Oct 17 2017
prev sibling next sibling parent reply evilrat <evilrat666 gmail.com> writes:
On Wednesday, 18 October 2017 at 03:48:01 UTC, Ky-Anh Huynh wrote:

 Hi,

 I'm using Bash heavily in my systems. Things become slow and 
 slow when I have tons of scripts :) And sometimes it's not easy 
 to manipulate data.

 You may have heard of recutils [1] which has a C extension to 
 be loaded by Bash. Is it possible to write similar things in D, 
 for Bash? I am not good at C; it's great if I explore this 
 field:)

 Some examples in C are in [2].

 My experience: Dlang has `pipe` support however the syntax is 
 not as clean as Bash :) Most of the times I see short (<1k loc) 
 Bash scripts are easy to maintain than Ruby (and now D things) 
 scripts.

 [2]: 
 http://git.savannah.gnu.org/cgit/bash.git/tree/examples/loadables/cat.c


(Not a linux pro or even bash user here, don't know anything 
about bash API or internals, not even a C user, proceed on your 
own risk)

Need to investigate how bash actually handles loading

 From the look of it there is configuration struct of type 
'builtin' that has basically and entry point function that serves 
as main('cat_builtin'), documentation function ('cat_doc') and 
other required stuff.

But yes, in theory nothing crazy and seems doable.


Now according to the readme in examples folder
"""
Loadable builtins are loaded into a running shell with

	enable -f filename builtin-name
"""


This give a hint to look at what this 'enable' implementation 
does, my guess it can do a simple dlopen(filename) and then 
dlsym(builtin-name) that most likely expected to be one of those 
struct of type 'builtin' or that *_builtin() function, and the 
rest is implemenetation details.

And... yes (can search for dlsym) - 
http://git.savannah.gnu.org/cgit/bash.git/tree/builtins/enable.def#n365


Now with that knowledge it should be possible to our test plugin.

Some (pseudo) code that could serve as starting point...
-------------------------------------------------

// first we need 'builtin' struct in D

// (from 
http://git.savannah.gnu.org/cgit/bash.git/tree/builtins.h)
struct builtin {
   char* name;			/* The name that the user types. */
   sh_builtin_func_t function;	/* The address of the invoked 
function. */
   int flags;			/* One of the #defines above. */
   const(char)* const* long_doc;	/* NULL terminated array of 
strings. */
   const(char)* short_doc;	/* Short version of documentation. */
   char* handle;			/* for future use */
}

// add some declarations

alias sh_builtin_func_t = extern(C) int sh_builtin_func_t 
(WORD_LIST *);
enum BUILTIN_ENABLED = 0x01; // builtins.h
// TODO: find WORD_LIST declaration (sorry)

extern(C) static builtin plugtest_struct = {
	"testcommand", // function will be acessible by this name?
	test_builtin,
	BUILTIN_ENABLED,
	test_doc.ptr, // will need to convert string[] to char**
         // the
	"testcommand [-] [file ...]",
	0
};

string[] test_doc = [
	"Out test function."
];


// 
http://git.savannah.gnu.org/cgit/bash.git/tree/examples/loadables/cat.c#n91
// seems like default int return, if you got a segfault then it 
is something else
extern(C) static int test_builtin(WORD_LIST* list)
{
         import core.runtime; // don't remember the exact names, 
sorry, but this one is required
	import std.stdio : writeln;

         Runtime.initialize(); // you would probably need to track 
this one because command can be called multiple times during 
plugin lifetime

         writeln("it works!"); // if you see this in terminal you 
are lucky, otherwise find out what is 'write' and use it instead
	return 0;
}
---------------------------------------------------------------

This isn't the actual code but should give you a hint, the rest 
is up to you.
Oct 18 2017
next sibling parent evilrat <evilrat666 gmail.com> writes:
On Wednesday, 18 October 2017 at 08:15:53 UTC, evilrat wrote:

 ...
 extern(C) static int test_builtin(WORD_LIST* list)
 ...


This of course should be nothrow also, because if it throws 
something really bad may(will) happen
Oct 18 2017
prev sibling parent Ky-Anh Huynh <saigon example.net> writes:
On Wednesday, 18 October 2017 at 08:15:53 UTC, evilrat wrote:

 [...]

 This isn't the actual code but should give you a hint, the rest 
 is up to you.


Woh Thanks a ton. I can have some working code after a few hours 
:D

https://github.com/icy/dusybox/blob/master/lib/dusybox/bash_builtin_hello/package.d

(A screenshot: 
https://github.com/icy/dusybox#a-bash-builtin-command)

I got problem with type conversion. I had to use inline 
declaration for `long_doc`:

```
extern(C) static builtin dz_hello_struct =
{
   name: cast (char*) "dz_hello",
   func: &dz_hello_builtin,
   flags: BUILTIN_ENABLED,
   long_doc: [
     "Hello, it's from Dlang.",
     "",
     "A Hello builtin command written in Dlang."
   ],
   short_doc: cast (char*) "dz_hello",
   handle: null
};
```

otherwise the compiler reports that some variable is not be read 
at compile time, or kind of `cannot use non-constant CTFE pointer 
in an initializer`.

There are many things I need to study from your post. So far it's 
good :)

Thanks again
Oct 21 2017
prev sibling parent reply Andrea Fontana <nospam example.com> writes:
On Wednesday, 18 October 2017 at 03:48:01 UTC, Ky-Anh Huynh wrote:

 Hi,

 I'm using Bash heavily in my systems. Things become slow and 
 slow when I have tons of scripts :) And sometimes it's not easy 
 to manipulate data.

 You may have heard of recutils [1] which has a C extension to 
 be loaded by Bash. Is it possible to write similar things in D, 
 for Bash? I am not good at C; it's great if I explore this 
 field:)

 Some examples in C are in [2].

 My experience: Dlang has `pipe` support however the syntax is 
 not as clean as Bash :) Most of the times I see short (<1k loc) 
 Bash scripts are easy to maintain than Ruby (and now D things) 
 scripts.

 Thanks for your reading.

 [1]: https://news.ycombinator.com/item?id=15302035
 [2]: 
 http://git.savannah.gnu.org/cgit/bash.git/tree/examples/loadables/cat.c


You can write your script in D using

as shebang line.

Or, using dstep, you can convert C headers to D imports, so you 
can compile your own extension in D.

Andrea
Oct 18 2017
parent evilrat <evilrat666 gmail.com> writes:
On Wednesday, 18 October 2017 at 08:22:09 UTC, Andrea Fontana 
wrote:

 On Wednesday, 18 October 2017 at 03:48:01 UTC, Ky-Anh Huynh 
 wrote:

 You can write your script in D using

 as shebang line.

 Or, using dstep, you can convert C headers to D imports, so you 
 can compile your own extension in D.

 Andrea


oh... yes, also with dub

https://code.dlang.org/advanced_usage
Oct 18 2017