digitalmars.D.learn - Is it possible to dynamically load a safe function from a shared
- wjoe (36/36) Mar 13 2020 I've got a plug-in which is a shared library. Like this
- Mike Parker (4/8) Mar 13 2020 It probably has nothing to do with @safe, but is because of the
- wjoe (3/13) Mar 13 2020 Than works, thanks :)
- Mike Parker (3/20) Mar 13 2020 Yes, but when you take the address of *any* kind of pointer, you
- wjoe (5/26) Mar 13 2020 Didn't know that. I'm curious as to why this is the case.
- H. S. Teoh (19/25) Mar 13 2020 [...]
- wjoe (8/35) Mar 13 2020 I wasn't aware that pragma(mangle, ..) can practically name any
- Steven Schveighoffer (11/15) Mar 13 2020 But that's the same as the linker anyway. It's possible to stub the
- H. S. Teoh (11/19) Mar 13 2020 [...]
- Steven Schveighoffer (7/24) Mar 15 2020 Of course it has to be trusted. The underlying function uses void *.
- wjoe (18/27) Mar 14 2020 Thanks for your reply.
- Steven Schveighoffer (22/51) Mar 15 2020 My original point is that forging is possible without using a dynamic
- H. S. Teoh (32/35) Mar 13 2020 [...]
I've got a plug-in which is a shared library. Like this module plugin; safe int VersionOfAPI() { return 1; } this is builds to plugin.so in main.d I'm loading the plugin and bind the those functions like so: module app; safe: alias apiverfn = int function(); apiverfn apiVersion; void main() { LoadPlugin("plugin.so"); } void LoadPlugin(string path) { void* plugin = loadLibrary(path); trusted void* bindSymbol(void** pfn, const(char)* symbolName) { *pfn = dlsym(plugin, symbolName); } bindSymbol(&apiVersion, "VersionOfAPI"); } The compiler now refuses to call bindSymbol: Error: function bindSymbol(void**, const(char)*) is not callable using argument types (int function() safe*, string) cannot pass &apiVersion of type int function() safe* to parameter void** pfn It makes sense that the compiler refuses to assign a system function* to a safe one but dlsym is a system function which returns a system fn* and I know that the function is safe. Is it possible to convince the compiler to look the other way while binding safe functions from the plugin ?
Mar 13 2020
On Friday, 13 March 2020 at 15:16:06 UTC, wjoe wrote:bindSymbol(&apiVersion, "VersionOfAPI"); }Is it possible to convince the compiler to look the other way while binding safe functions from the plugin ?It probably has nothing to do with safe, but is because of the void**. bindSymbol(cast(void**)&apiVersion, "VersionOfAPI");
Mar 13 2020
On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker wrote:On Friday, 13 March 2020 at 15:16:06 UTC, wjoe wrote:Than works, thanks :) But isn't apiVersion a function pointer ?bindSymbol(&apiVersion, "VersionOfAPI"); }Is it possible to convince the compiler to look the other way while binding safe functions from the plugin ?It probably has nothing to do with safe, but is because of the void**. bindSymbol(cast(void**)&apiVersion, "VersionOfAPI");
Mar 13 2020
On Friday, 13 March 2020 at 16:11:53 UTC, wjoe wrote:On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker wrote:Yes, but when you take the address of *any* kind of pointer, you can't assign it to void** without a cast.On Friday, 13 March 2020 at 15:16:06 UTC, wjoe wrote:Than works, thanks :) But isn't apiVersion a function pointer ?bindSymbol(&apiVersion, "VersionOfAPI"); }Is it possible to convince the compiler to look the other way while binding safe functions from the plugin ?It probably has nothing to do with safe, but is because of the void**. bindSymbol(cast(void**)&apiVersion, "VersionOfAPI");
Mar 13 2020
On Friday, 13 March 2020 at 17:05:32 UTC, Mike Parker wrote:On Friday, 13 March 2020 at 16:11:53 UTC, wjoe wrote:Didn't know that. I'm curious as to why this is the case. This also means that LoadPlugin() can't be safe - or at least the call to bindSymbol. Thanks for the reply.On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker wrote:Yes, but when you take the address of *any* kind of pointer, you can't assign it to void** without a cast.On Friday, 13 March 2020 at 15:16:06 UTC, wjoe wrote:Than works, thanks :) But isn't apiVersion a function pointer ?bindSymbol(&apiVersion, "VersionOfAPI"); }Is it possible to convince the compiler to look the other way while binding safe functions from the plugin ?It probably has nothing to do with safe, but is because of the void**. bindSymbol(cast(void**)&apiVersion, "VersionOfAPI");
Mar 13 2020
On Fri, Mar 13, 2020 at 06:11:01PM +0000, wjoe via Digitalmars-d-learn wrote:On Friday, 13 March 2020 at 17:05:32 UTC, Mike Parker wrote:[...]On Friday, 13 March 2020 at 16:11:53 UTC, wjoe wrote:On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker wrote:[...]bindSymbol(cast(void**)&apiVersion, "VersionOfAPI");This also means that LoadPlugin() can't be safe - or at least the call to bindSymbol.[...] Of course it cannot be safe, because it depends on whether the symbol defined in the library you loaded is actually safe. You cannot know that for sure (for example, maybe it exports a symbol that happens to coincide with the mangling of a safe function, but isn't in fact safe). Similarly, at least on Posix, shared libraries only export symbol names, the actual type is not part of the shared library API other than what is encoded in the mangled symbol. So you don't know for sure that you're actually casting to the correct type, for example; if you make a mistake, you might get UB and memory corruption. So essentially, you're trusting that the symbol you just looked up is actually pointing to what you think it's pointing to. Therefore, it makes sense that such calls have to be trusted. T -- If it breaks, you get to keep both pieces. -- Software disclaimer notice
Mar 13 2020
On Friday, 13 March 2020 at 18:30:51 UTC, H. S. Teoh wrote:On Fri, Mar 13, 2020 at 06:11:01PM +0000, wjoe via Digitalmars-d-learn wrote:I wasn't aware that pragma(mangle, ..) can practically name any function anything. So from what I understand, because, at least on Posix, since there's only a symbol name there's nothing I can do in my loader to verify that a function is or does what it claim to be/do. This is kind of disappointing but well worth the lessons learned. Thanks for your reply.On Friday, 13 March 2020 at 17:05:32 UTC, Mike Parker wrote:[...]On Friday, 13 March 2020 at 16:11:53 UTC, wjoe wrote:On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker wrote:[...]bindSymbol(cast(void**)&apiVersion, "VersionOfAPI");This also means that LoadPlugin() can't be safe - or at least the call to bindSymbol.[...] Of course it cannot be safe, because it depends on whether the symbol defined in the library you loaded is actually safe. You cannot know that for sure (for example, maybe it exports a symbol that happens to coincide with the mangling of a safe function, but isn't in fact safe). Similarly, at least on Posix, shared libraries only export symbol names, the actual type is not part of the shared library API other than what is encoded in the mangled symbol. So you don't know for sure that you're actually casting to the correct type, for example; if you make a mistake, you might get UB and memory corruption. So essentially, you're trusting that the symbol you just looked up is actually pointing to what you think it's pointing to. Therefore, it makes sense that such calls have to be trusted. T
Mar 13 2020
On 3/13/20 4:22 PM, wjoe wrote:I wasn't aware that pragma(mangle, ..) can practically name any function anything. So from what I understand, because, at least on Posix, since there's only a symbol name there's nothing I can do in my loader to verify that a function is or does what it claim to be/do.But that's the same as the linker anyway. It's possible to stub the function as safe, but implement it as system and just pragma(mangle) the thing. Therefore the compiler is no safer. I would expect that something could be written to turn a signature string into a mangling and also provide the correct type upon return. Something like: auto f = getFunction!( safe void function(int))("package.module.symbol"); and have it properly mangle the expected function name and pull it from the dynamic library. -Steve
Mar 13 2020
On Fri, Mar 13, 2020 at 04:31:16PM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote: [...]I would expect that something could be written to turn a signature string into a mangling and also provide the correct type upon return. Something like: auto f = getFunction!( safe void function(int))("package.module.symbol"); and have it properly mangle the expected function name and pull it from the dynamic library.[...] This would still have to be trusted, of course, since there's no telling what's actually inside the object file. But this sort of facility totally should be in Phobos, or at least in some dub package somewhere. It will make working with dynamically loaded libraries in D so much more convenient. T -- If you think you are too small to make a difference, try sleeping in a closed room with a mosquito. -- Jan van Steenbergen
Mar 13 2020
On 3/13/20 5:06 PM, H. S. Teoh wrote:On Fri, Mar 13, 2020 at 04:31:16PM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote: [...]Of course it has to be trusted. The underlying function uses void *. But not because you can't trust the mangling.I would expect that something could be written to turn a signature string into a mangling and also provide the correct type upon return. Something like: auto f = getFunction!( safe void function(int))("package.module.symbol"); and have it properly mangle the expected function name and pull it from the dynamic library.[...] This would still have to be trusted, of course, since there's no telling what's actually inside the object file.But this sort of facility totally should be in Phobos, or at least in some dub package somewhere. It will make working with dynamically loaded libraries in D so much more convenient.I'm not really motivated to make such a package, as I don't do any software with runtime loading of libraries. But I bet it would be either accepted into Phobos and/or wouldn't be that hard to add for a library. -Steve
Mar 15 2020
On Friday, 13 March 2020 at 20:31:16 UTC, Steven Schveighoffer wrote:On 3/13/20 4:22 PM, wjoe wrote: I would expect that something could be written to turn a signature string into a mangling and also provide the correct type upon return. Something like: auto f = getFunction!( safe void function(int))("package.module.symbol"); and have it properly mangle the expected function name and pull it from the dynamic library. -SteveThanks for your reply. core.demangle: mangle; comes to mind. And in fact, because the exports weren't extern(C), that's how I imported the symbols; prior to reading H. S. Teoh's reply. That's when I realized that I've got a problem. Even though I know that the function being exported are all compiled safe, that doesn't mean I can count on the fact. Since a plugin is a separate file it can be swapped out with a version that exports all the same (forged) symbols names but with a system implementation. Forging these symbol names is as easy as something like mangle!(int function(int))("a.b")); (=_D1a1bPFiZi) and copy/pasting that into pragma(mangle, "_D1a1bPFiZi") and voila, the loader can nothing but trust that this is true. Forging is maybe too hard a word but my vocabulary lacks a better one.
Mar 14 2020
On 3/14/20 6:06 AM, wjoe wrote:On Friday, 13 March 2020 at 20:31:16 UTC, Steven Schveighoffer wrote:My original point is that forging is possible without using a dynamic library. Anyone who has pragma(mangle)'d the symbol can "forge" anything they want, it's not that hard. Even without using pragma(mangle, it's possible): mod1.d: __gshared someGlobal; extern(C) void realImpl() // no mangling { someGlobal = 5; // yay side effects! *(cast(ubyte*)0xdeadbeef) = 5; // yay no safety! throw new Exception("yay, I can throw!"); } mod2.d: extern(C) void realImpl() safe pure nothrow; void foo() safe pure nothrow // mangled properly { realImpl(); } So why worry about the dynamic library case? I would say if you obey the mangling rules, the compiler should trust the mangling. -SteveOn 3/13/20 4:22 PM, wjoe wrote: I would expect that something could be written to turn a signature string into a mangling and also provide the correct type upon return. Something like: auto f = getFunction!( safe void function(int))("package.module.symbol"); and have it properly mangle the expected function name and pull it from the dynamic library.Thanks for your reply. core.demangle: mangle; comes to mind. And in fact, because the exports weren't extern(C), that's how I imported the symbols; prior to reading H. S. Teoh's reply. That's when I realized that I've got a problem. Even though I know that the function being exported are all compiled safe, that doesn't mean I can count on the fact. Since a plugin is a separate file it can be swapped out with a version that exports all the same (forged) symbols names but with a system implementation. Forging these symbol names is as easy as something like mangle!(int function(int))("a.b")); (=_D1a1bPFiZi) and copy/pasting that into pragma(mangle, "_D1a1bPFiZi") and voila, the loader can nothing but trust that this is true. Forging is maybe too hard a word but my vocabulary lacks a better one.
Mar 15 2020
On Fri, Mar 13, 2020 at 08:22:53PM +0000, wjoe via Digitalmars-d-learn wrote: [...]So from what I understand, because, at least on Posix, since there's only a symbol name there's nothing I can do in my loader to verify that a function is or does what it claim to be/do.[...] As far as I know, it's the same thing on Windows PE format. Basically, once your code is compiled down to the object file level, there really isn't anything at the level of higher programming language concepts anymore, like type signatures and stuff. As far as object files are concerned, it's just a bunch of opaque binary data with string labels tacked on them, plus some extra information like relocation data and other such OS-level concepts. What's in that opaque binary data isn't really the purview of the object file format; things like types and function signatures are an interpretation laid upon the data by higher-level application code. The OS doesn't know what it is, and doesn't care (nor should it, that's not its job). So given some arbitrary object file, there's really no real guarantee as to what the contents are inside. Anybody can craft an object file that exports symbol names that look like the symbols generated by some higher-level programming language, but the actual binary data the names point to may do something completely different. In fact, this is exactly why pragma(mangle) is so useful: Adam Ruppe's jni.d, for example, makes extensive use of this in order to make the D code inside the object file appear like Java JNI symbols to the JVM. D's C++ interop is also based on the same concept: export symbols that look like C++ mangled symbols, but inside is actually D code, not C++ code. Far from being "disappointing", I think this stuff is very powerful, and lots of fun if you do it right. But it does come with the caveat that you're essentially meddling around under the hood, so it's your responsibility not to do something that will cause the engine to blow up. Hence the trusted tag. T -- Never step over a puddle, always step around it. Chances are that whatever made it is still dripping.
Mar 13 2020