www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Get Dll functions at compile time

reply Johnson Jones <JJ Dynomite.com> writes:
I like to create code that automates much of the manual labor 
that we, as programmers, are generally forced to do. D generally 
makes much of this work automatable. For example, I have created 
the following code which makes loading dlls similar to libs:



/* Import DLL functions in to type T. The following example shows 
methodology
struct DLLImports
{
	 ("DLLImport") public static extern(Windows)
	{
		 ("libgdk-3-0.dll")
		{
			void* function(GdkWindow *window) gdk_win32_window_get_handle;
		}
	}
}

// Fixes static functions and function pointers to point to their 
specified DLL's
DllImport!DLLImports;
*/
void DLLImport(alias T)()
{
	version(Windows)
	{
		import core.sys.windows.windows, std.conv, std.meta, std.traits;
		HINSTANCE[string] DLLs;
	
		foreach(fname; __traits(allMembers, T))
		{	
			mixin("enum isf = isFunction!(T."~fname~");");
			mixin("enum isfp = isFunctionPointer!(T."~fname~");");
			mixin("enum attrs = __traits(getAttributes, T."~fname~");");		

			static if ((isf || isfp) && attrs.length == 2 && attrs[0] == 
"DLLImport")
			{
				auto dllName = attrs[1];
				if (dllName !in DLLs)
					DLLs[dllName] = LoadLibrary(to!wstring(dllName~"\0").ptr);

				auto dll = DLLs[dllName];
				if (dll == null)
					assert(0, "Cannot load DLL `"~dllName~"'");

				auto func = GetProcAddress(dll, fname);

				if (isf)
					mixin("auto p = cast(void**)&"~T.stringof~"."~fname~"; *p = 
cast(typeof(p))func;");
				else		
					mixin(""~T.stringof~"."~fname~" = 
cast(typeof("~T.stringof~"."~fname~"))func;");	
			}
		
		}
	}



But this got me thinking that we don't even need to have to 
specify the function in D, hell, they already exist in the lib 
and we are just duplicating work.

What if, at compile time, D could get all the functions and their 
type information and build a class for them for us? We could then 
just write something like

struct DLLImports
{
	 ("DLLImport") string libgdk = "libgdk-3-0.dll";
}


and have some ctfe meta functions extract all the function from 
libgdk and insert them in to the struct.

There are two problems with this, one easy and one 
hard/impossible(which would be easy if people were intelligent 
enough to have foresight):


1. Get the dll function by name from the dll at compile time. 
This would probably require manually reading the dll file and 
scanning for the function.

2. Get the type information to build a declaration. This is 
probably impossible since dll's do not contain the type 
information about their parameters and return type(or do they?). 
If they did, it would be easy. I would suggest that all dll's 
generated by D include this information somewhere and an easy way 
to extract it for future programmers so such things could be 
implemented.

Alternatively, maybe a master database could be queried for such 
information by using the function names and dll name? I don't 
know if D has network capabilities at compile time though.

Alternatively, completely scrap the lethargic way things are done 
in the name of backwards compatibility and $$$ and do things 
right(learn from the past, stop repeating same mistakes, etc). 
Sure it's a lot of work, but in the end is far less than one 
thinks considering the increased productivity... but I guess the 
we gotta keep buying the kids christmas presents.
Aug 08
next sibling parent Johnson Jones <JJ Dynomite.com> writes:
On Wednesday, 9 August 2017 at 02:11:13 UTC, Johnson Jones wrote:
 I like to create code that automates much of the manual labor 
 that we, as programmers, are generally forced to do. D 
 generally makes much of this work automatable. For example, I 
 have created the following code which makes loading dlls 
 similar to libs:



 /* Import DLL functions in to type T. The following example 
 shows methodology
 struct DLLImports
 {
 	 ("DLLImport") public static extern(Windows)
 	{
 		 ("libgdk-3-0.dll")
 		{
 			void* function(GdkWindow *window) 
 gdk_win32_window_get_handle;
 		}
 	}
 }

 // Fixes static functions and function pointers to point to 
 their specified DLL's
 DllImport!DLLImports;
 */
 void DLLImport(alias T)()
 {
 	version(Windows)
 	{
 		import core.sys.windows.windows, std.conv, std.meta, 
 std.traits;
 		HINSTANCE[string] DLLs;
 	
 		foreach(fname; __traits(allMembers, T))
 		{	
 			mixin("enum isf = isFunction!(T."~fname~");");
 			mixin("enum isfp = isFunctionPointer!(T."~fname~");");
 			mixin("enum attrs = __traits(getAttributes, 
 T."~fname~");");		

 			static if ((isf || isfp) && attrs.length == 2 && attrs[0] == 
 "DLLImport")
 			{
 				auto dllName = attrs[1];
 				if (dllName !in DLLs)
 					DLLs[dllName] = LoadLibrary(to!wstring(dllName~"\0").ptr);

 				auto dll = DLLs[dllName];
 				if (dll == null)
 					assert(0, "Cannot load DLL `"~dllName~"'");

 				auto func = GetProcAddress(dll, fname);

 				if (isf)
 					mixin("auto p = cast(void**)&"~T.stringof~"."~fname~"; *p 
 = cast(typeof(p))func;");
 				else		
 					mixin(""~T.stringof~"."~fname~" = 
 cast(typeof("~T.stringof~"."~fname~"))func;");	
 			}
 		
 		}
 	}



 But this got me thinking that we don't even need to have to 
 specify the function in D, hell, they already exist in the lib 
 and we are just duplicating work.

 What if, at compile time, D could get all the functions and 
 their type information and build a class for them for us? We 
 could then just write something like

 struct DLLImports
 {
 	 ("DLLImport") string libgdk = "libgdk-3-0.dll";
 }


 and have some ctfe meta functions extract all the function from 
 libgdk and insert them in to the struct.

 There are two problems with this, one easy and one 
 hard/impossible(which would be easy if people were intelligent 
 enough to have foresight):


 1. Get the dll function by name from the dll at compile time. 
 This would probably require manually reading the dll file and 
 scanning for the function.

 2. Get the type information to build a declaration. This is 
 probably impossible since dll's do not contain the type 
 information about their parameters and return type(or do 
 they?). If they did, it would be easy. I would suggest that all 
 dll's generated by D include this information somewhere and an 
 easy way to extract it for future programmers so such things 
 could be implemented.

 Alternatively, maybe a master database could be queried for 
 such information by using the function names and dll name? I 
 don't know if D has network capabilities at compile time though.

 Alternatively, completely scrap the lethargic way things are 
 done in the name of backwards compatibility and $$$ and do 
 things right(learn from the past, stop repeating same mistakes, 
 etc). Sure it's a lot of work, but in the end is far less than 
 one thinks considering the increased productivity... but I 
 guess the we gotta keep buying the kids christmas presents.
And while we are at it, here is a set of meta functions that make using glade files easier: // Loads the glade interface element id's and types directly from the glade interface file so we do not have to declare them manually // Use: ("Glade") { mixin(DeclareGladeInterfaceElements!gladeFile); } where ever variables will be defined and InstantiateGladeInterfaceElements( to initalize them public string DeclareGladeInterfaceElements(string filename)() { auto token1 = "<object class=\""; auto token2 = " id=\""; string res = ""; import std.file; auto data = import(filename); for(int i = 0; i < data.length; i++) { // Matched class/object type, get name actual type then get id if it has one. if (i+token1.length < data.length && data[i..i+token1.length] == token1) { i += token1.length; auto pos = i; while(i < data.length && data[i] != '"') i++; auto type = cast(string)data[pos..i]; // Match id if it has one. while(i+token2.length < data.length && data[i] != '>' && data[i] != '<' && data[i] != '"' && data[i..i+token2.length] != token2) i++; i++; if (data[i] == '>') { continue; } i += token2.length; pos = i; if (data[i] == '>') { continue; } while(i+1 < data.length && data[i] != '"') i++; res ~= "public gtk."~type[3..$] ~ " " ~ data[pos..i] ~ ";\n"; } if (i > data.length - 7130) { continue; } } return res; } public void InstantiateGladeInterfaceElements(T)(Builder builder, T t) { import std.meta; foreach(m; __traits(allMembers, T)) { mixin("enum w = __traits(getAttributes, T."~m~");"); static if (w.length > 0 && w[0] == "Glade") { mixin("t."~m~" = (cast(typeof(T."~m~"))builder.getObject(`"~m~"`));"); mixin("if (t."~m~" is null) { error(\"Error: `"~m~"` not found, User Interface Corrupted, Cannot continue!\"); }"); } } } They automatically extract objects that have ID's with them, add/declare them in D and instantiate them. This means one doesn't have to duplicate code. (DeclareGladeInterfaceElements could probably be optimized using proper D code but I just hacked something other) Mainly posted for posterity and it might be useful to a few others, but is in line with using D's meta capabilities to generate D code at compile time from external sources so less keystrokes are required.
Aug 08
prev sibling next sibling parent Johnson Jones <JJ Dynomite.com> writes:
Was buggy due to refactoring.


----------------------------


module DLLImport;

/* Import DLL functions in to type T. The following example shows 
methodology
struct DLL_gdk
{
	 ("DLLImport") public static extern(Windows)
	{
		 ("libgdk-3-0.dll")
		{
			void* function(GdkWindow *window) gdk_win32_window_get_handle;
		}
	}
}

// Fixes static functions and function pointers to point to their 
specified DLL's
ImportDLLs!DLL_gdk;
*/
void ImportDLLs(T)()
{
	version(Windows)
	{
		import core.sys.windows.windows, std.conv, std.meta, std.traits;
		HINSTANCE[string] DLLs;
	
		foreach(fname; __traits(allMembers, T))
		{	
			mixin("enum isf = isFunction!(T."~fname~");");
			mixin("enum isfp = isFunctionPointer!(T."~fname~");");
			mixin("enum attrs = __traits(getAttributes, T."~fname~");");		
			static if ((isf || isfp) && attrs.length == 2 && attrs[0] == 
"DLLImport")
			{
				auto dllName = attrs[1];
				if (dllName !in DLLs)
					DLLs[dllName] = LoadLibrary(to!wstring(dllName~"\0").ptr);

				auto dll = DLLs[dllName];
				if (dll == null)
					assert(0, "Cannot load DLL `"~dllName~"'");

				auto func = GetProcAddress(dll, fname);

				mixin("import "~moduleName!(T)~";");
				static if (isf)
					mixin("auto p = cast(void**)&"~T.stringof~"."~fname~"; *p = 
cast(typeof(p))func;");
				else static if (isfp)
					mixin(""~T.stringof~"."~fname~" = 
cast(typeof("~T.stringof~"."~fname~"))func;");	
				else
					static assert("DLLImport Error");
			}
		
		}
	}
}
Aug 09
prev sibling parent Moritz Maxeiner <moritz ucworks.org> writes:
On Wednesday, 9 August 2017 at 02:11:13 UTC, Johnson Jones wrote:
 I like to create code that automates much of the manual labor 
 that we, as programmers, are generally forced to do. D 
 generally makes much of this work automatable. For example, I 
 have created the following code which makes loading dlls 
 similar to libs:

 [...]
Yes, this is essentially what people are doing right now, see e.g. [1] where all of the dynamic loading components are generated from the declarations used for linking.
 But this got me thinking that we don't even need to have to 
 specify the function in D, hell, they already exist in the lib 
 and we are just duplicating work.

 What if, at compile time, D could get all the functions and 
 their type information and build a class for them for us? We 
 could then just write something like

 struct DLLImports
 {
 	 ("DLLImport") string libgdk = "libgdk-3-0.dll";
 }


 and have some ctfe meta functions extract all the function from 
 libgdk and insert them in to the struct.

 There are two problems with this, one easy and one 
 hard/impossible(which would be easy if people were intelligent 
 enough to have foresight):
There's a third one: This is not what object code is for, you'd have to write code for every object code format, because they're inherently platform specific.
 1. Get the dll function by name from the dll at compile time. 
 This would probably require manually reading the dll file and 
 scanning for the function.
And you'd have to demangle all the symbol names for anything that's not C mangled. Such as D. Or C++.
 2. Get the type information to build a declaration. This is 
 probably impossible since dll's do not contain the type 
 information about their parameters and return type(or do 
 they?). If they did, it would be easy. I would suggest that all 
 dll's generated by D include this information somewhere and an 
 easy way to extract it for future programmers so such things 
 could be implemented.
Again, this is not what object code is for: If you want to bind to anything that was written in D, you're going to have the declarations, anyway (either in .d or .di files), so you wouldn't need the information in the object code. If it's not written in D, it's not going to be in the object code, anyway.
 Alternatively, maybe a master database could be queried for 
 such information by using the function names and dll name? I 
 don't know if D has network capabilities at compile time though.
D doesn't have arbitrary I/O at compile time. [1] https://github.com/Calrama/llvm-d/blob/master/source/llvm/functions/load.d#L124
Aug 09