www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Centralizable configured compile-time class instantiation

reply shd <alienballance gmail.com> writes:
Hello,
Original goal i'd like to achieve is to completely abstract user of my
`library` from concrete code implementations which would be set in one
centralized place (such as config file). While it's common to do that in C+=
+ at
runtime, i'd like to let compiler optimize more by making this in compile-
time.
So, the first solution i came up is to make a 'cfg' or 'impl' template, whi=
ch
accepts interface as an argument. It would be something like configurable
std.container : make. Inside this template, i could read and parse simple
configuration file to find proper implementation, and return it for user.
Unfortunately, i'm not aware of any method of instantiating any class in
template like this.
That's why i started to implementing other, simpler method. I assumed, that
CMake which I'm using, will control include paths correctly and swap
implementations on every build. What a dissapointment i met, when after
refactoring my code in this way, user can see only interface because this i=
s
the first matching file module. Well, it's quite my fault because i believe=
 it
was mentioned in TDPL, that module =3D 1 file.

So, to sum up:
My project is divided in two sections:
* interfaces
* implementations
How to make in my D library possibility of configurable instantiation (look=
up
at compile time) by having only interface at client side?

Thanks,
Mariusz Gliwi=F1ski
Jul 10 2011
parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
shd Wrote:

 Hello,
 Original goal i'd like to achieve is to completely abstract user of my
 `library` from concrete code implementations which would be set in one
 centralized place (such as config file). While it's common to do that in C++ at
 runtime, i'd like to let compiler optimize more by making this in compile-
 time.
 So, the first solution i came up is to make a 'cfg' or 'impl' template, which
 accepts interface as an argument. It would be something like configurable
 std.container : make. Inside this template, i could read and parse simple
 configuration file to find proper implementation, and return it for user.
 Unfortunately, i'm not aware of any method of instantiating any class in
 template like this.
 That's why i started to implementing other, simpler method. I assumed, that
 CMake which I'm using, will control include paths correctly and swap
 implementations on every build. What a dissapointment i met, when after
 refactoring my code in this way, user can see only interface because this is
 the first matching file module. Well, it's quite my fault because i believe it
 was mentioned in TDPL, that module = 1 file.
 
 So, to sum up:
 My project is divided in two sections:
 * interfaces
 * implementations
 How to make in my D library possibility of configurable instantiation (lookup
 at compile time) by having only interface at client side?
 
 Thanks,
 Mariusz Gliwiñski
I'm sorry to say that I don't exactly know what you are trying to achieve or problems you are having. I'm going to trying and provide details on what I have gotten out of your message. You can do compile-time configuration, this however is not a replacement to run-time configuration files. What you are trying to do, you probably don't want to do. It sounds like you should be providing a library that can be expanded on rather than a configurable implementation. Maybe this is what you are trying to do, and it might help to think of it in this form instead. D provides "Anti-Highjacking" measures to prevent a module from calling into code it did not know existed upon creation. This does prevent some interesting (and useful) patterns from being used, but generally it just results in unexpected, hard to track bugs.
Jul 11 2011
parent reply shd <alienballance gmail.com> writes:
Firstly, thanks for response.
2011/7/11 Jesse Phillips <jessekphillips+D gmail.com>:
 I'm sorry to say that I don't exactly know what you are trying to achieve=
or problems you are having. I'm going to trying =A0and provide details on = what I have gotten out of your message. Hopefully i'll clarify it more.
 You can do compile-time configuration, this however is not a replacement =
to run-time configuration files. I'm aware, and that's not my goal - let me clarify difference between my run-time and compile-time configs. Run-time configuration for my project might be delegated to optional `util.cfg` package which is able to read let's say XML file that will store things like resolution or something. While i'm trying to make my potentially very big project possible to make in reasonable time, i'm pretty focused on code-reusing and making use of third-party libraries. Because external code quality might change, my preferences might change, and APIs are not what i'd love to work with i'd like to try pay price of composing D object interfaces i like, and implement them with wrappers that are connecting with external code. So, back on to my previous example of runtime `util.cfg`, it might use filesystem/http reader and xml/cvs formatter from package `io` (this is gonna be run-time configuration) BUT besides of that i might like to change xml phobos wrapper (actually this example is not the best) with rapidxml or whatever and *this* is gonna be compile-time config. So, config has to be configured and it is going to be configured statically. Ps. Don't get me wrong, i'm not in love with wrapping standard library, but there are other classes which i'm going to use which have no interface in phobos. This was just an example.
 What you are trying to do, you probably don't want to do. It sounds like =
you should be providing a library that can be expanded on rather than a con= figurable implementation. Maybe this is what you are trying to do, and it m= ight help to think of it in this form instead. Well, my end-goal is to provide complete application. Because i embrace code reusability, i love specialized libraries and when i'll need to implement something specialized, then share it - i'd love to do that. I'm afraid it's more like framework than library, but i don't like frameworks because they force you to use everything. What i'd like to achieve is to let someone use only parts of my code for plugging it into his application, lessen dependencies as much as i can, while remain type-safe. Actually, project structure looks like that: * project/ * bin/ * work/ * docs/ * iface/ * ae * ge * pe (...) * impl/ * glfw * ge.window * io.hid.keyboard * io.hid.mouse (you can add other window toolkits, or just separate OpenGL context creation code and user-input handling from different libraries) * h3d * ge.renderer (...) (you can add any other renderers or parts of game engines if they let you to do that) * project_name * <package.interfaces implemented by me> * pe * bullet * ode * newton (code is opensource and publicly accessible but excuse me i don't quote anything because it's currently ugly and unprofessional so i'm not proud of it yet, although i figured it out it's best way to do my way, it just needs time and work to be proud) So, i could easily change libraries, someone might use parts of my code by implementing bounding interfaces and as side effect i could easily benchmark how different implementations act in my project (like with C++ PAL). So, if i'll handle task of abstracting all this out, and get reasonable performance that's the other topic, but i believe it's worth a try. All i need is to have a tool that lets me choose implementation at compile time without interface file modification. Any clues?
 D provides "Anti-Highjacking" measures to prevent a module from calling i=
nto code it did not know existed upon creation. This does prevent some inte= resting (and useful) patterns from being used, but generally it just result= s in unexpected, hard to track bugs. Any anti-anti-hijacking ideas? :)
Jul 12 2011
next sibling parent reply Kagamin <spam here.lot> writes:
shd Wrote:

 Actually, project structure looks like that:
 * project/
   * bin/
   * work/
   * docs/
   * iface/
     * ae
     * ge
     * pe
     (...)
   * impl/
     * glfw
       * ge.window
       * io.hid.keyboard
       * io.hid.mouse
       (you can add other window toolkits, or just separate OpenGL
 context creation code and user-input handling from different
 libraries)
       * h3d
         * ge.renderer
         (...)
       (you can add any other renderers or parts of game engines if
 they let you to do that)
     * project_name
       * <package.interfaces implemented by me>
   * pe
     * bullet
     * ode
     * newton
 (code is opensource and publicly accessible but excuse me i don't
 quote anything because it's currently ugly and unprofessional so i'm
 not proud of it yet, although i figured it out it's best way to do my
 way, it just needs time and work to be proud)
 
 So, i could easily change libraries, someone might use parts of my
 code by implementing bounding interfaces and as side effect i could
 easily benchmark how different implementations act in my project (like
 with C++ PAL).
 
 So, if i'll handle task of abstracting all this out, and get
 reasonable performance that's the other topic, but i believe it's
 worth a try. All i need is to have a tool that lets me choose
 implementation at compile time without interface file modification.
 Any clues?
You want an IOC container like Unity? And you want the interface-to-class mapping to be configurable externally rather than version out these classes directly in source? like version(UseGL) { static import wrappers.gl; alias wrappers.gl.WindowImpl Window; } else version(UseWhat) { static import wrappers.what; alias wrappers.what.WindowImpl Window; } container.register!(IWindow,Window)(); client: IWindow myWindow=container.resolve!(IWindow)();
Jul 12 2011
parent reply shd <alienballance gmail.com> writes:
2011/7/12 Kagamin <spam here.lot>:
 You want an IOC container like Unity?
exactly
 And you want the interface-to-class mapping to be configurable externally=
rather than version out these classes directly in source?
 like
 version(UseGL)
 {
 =A0static import wrappers.gl;
 =A0alias wrappers.gl.WindowImpl Window;
 }
 else version(UseWhat)
 {
 =A0static import wrappers.what;
 =A0alias wrappers.what.WindowImpl Window;
 }
 container.register!(IWindow,Window)();

 client:

 IWindow myWindow=3Dcontainer.resolve!(IWindow)();
that's right :)
Jul 12 2011
parent reply Kagamin <spam here.lot> writes:
shd Wrote:

 2011/7/12 Kagamin <spam here.lot>:
 You want an IOC container like Unity?
exactly
 And you want the interface-to-class mapping to be configurable externally
rather than version out these classes directly in source?
 like
 version(UseGL)
 {
 �static import wrappers.gl;
 �alias wrappers.gl.WindowImpl Window;
 }
 else version(UseWhat)
 {
 �static import wrappers.what;
 �alias wrappers.what.WindowImpl Window;
 }
 container.register!(IWindow,Window)();

 client:

 IWindow myWindow=container.resolve!(IWindow)();
that's right :)
Hmm... If you have wrappers/gl/guiImpl/... and wrappers/what/guiImpl/... and conditionally add them to includes for example with -I wrappers/gl you should be able to import guiImpl; and use guiImpl.Window. If it's -I wrappers/what, guiImpl.Window is still guiImpl.Window.
Jul 13 2011
parent reply shd <alienballance gmail.com> writes:
2011/7/13 Kagamin <spam here.lot>:
 Hmm... If you have
 wrappers/gl/guiImpl/...
 and
 wrappers/what/guiImpl/...
 and conditionally add them to includes
 for example
 with -I wrappers/gl you should be able to import guiImpl; and use
guiImpl.Window. If it's -I wrappers/what, guiImpl.Window is still
guiImpl.Window.
Yeah, that might work. I tend to see everywhere very strange problems (for other people). Like I think of including interface file from user-app, and then give him only this interface compliant instance. And if he wants to use some implementation-depended functionalities he should clearly instantiate IObjectSpecialized or import specialized.path.to.package; By doing it in way you are talking about (which is kind of less perfectionist version of method mentioned by me in first post, although *it works*), user is able to import path.to.package; then instantiate object by standard instantiation, and accidentally use implementation-dependant method. Then, he commits and someone else wants to compile end-app with different set of implementation libraries. Program won't compile because someone used method which isn't specified in portable interface. In the same time, program is assumed to work so, because it's not clear that person who wrote this call is acting unportable (so we're relying on documentation rather than no-can-do method). So it's a fail of my 'framework', which claims to let use x, y and z libraries interchangeably by only changing cmake definition, while the build breaks. Author of framework didn't gave programmer, clear feedback that he is acting unportable. I hope i didn't bored you too much with my imagination because there is something more i don't like. Directory hierarchy :) It's not D-related though. Soo, the only solutions we came up is doing it out of language (by -I impl/library/path), or messing with import/aliases in interfaces which isn't nice. Although first solution isn't as good as i hoped for, it works... which is a good start, so i'll probably do it this way :)
Jul 13 2011
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 13.07.2011 15:39, shd wrote:
 I hope i didn't bored you too much with my imagination because there
 is something more i don't like. Directory hierarchy :) It's not
 D-related though.
Understanding a risk of going completely OT: you are not alone on this one :) Time ago I thought of something more like tag based file system, where filetypes and e.g. date of modification are just that - one of myriad of tags, besides any additional that are user/system defined. Though I have never reached any more or less complete design, the idea still thrills me. -- Dmitry Olshansky
Jul 13 2011
parent shd <alienballance gmail.com> writes:
2011/7/13 Dmitry Olshansky <dmitry.olsh gmail.com>
 Time ago I thought of something more like tag based file system, where
 filetypes and e.g. date of modification are just that =A0- one of myriad =
of
 tags, besides any additional that are user/system defined.
 Though I have never reached any more or less complete design, the idea st=
ill
 thrills me.
Been thinking about it too, so i suppose it's quite common idea :) I didn't even started designing it for real (to code) because i don't think i had/have now necessary qualifications, time and... come on - it had to be part of my uber secure and (quite...) easy to use operating system with semi-formally tested microkernel... but doh :) Yeah, one of this extremely unlikely to achieve ideas. But this is good when other people have similar thoughts. Maybe someday, someone will make it.
Jul 13 2011
prev sibling next sibling parent Jesse Phillips <jessekphillips+D gmail.com> writes:
 
 Any anti-anti-hijacking ideas? :)
Hopefully I remember to reply more in-depth when I get home. But this was a question on SO that sounds kind of like your request. I tried explaining why anti-hijacking was created.
Jul 12 2011
prev sibling parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
shd Wrote:

 Firstly, thanks for response.
After reading Kagamin's response, I'm not sure I understand you still. You want to have code reuse, you want to have run-time and compile-time configuration, but you don't want to make use of the technologies/techniques/patterns which facilitate these. You say no to libraries, you say no to frameworks, you say no to versioning and desire the benefits of these without what they entail. If you come up with an alternative, then please do bring it forward. I don't see how anyone will be able to get you what you want.
Jul 12 2011
parent reply shd <alienballance gmail.com> writes:
2011/7/13 Jesse Phillips <jessekphillips+D gmail.com>:
 After reading Kagamin's response, I'm not sure I understand you still. You
want to have code reuse, you want to have run-time and compile-time
configuration, but you don't want to make use of the
technologies/techniques/patterns which facilitate these.
I'm not sure are you referring 'technologies/techniques/patterns' to code reusability or configuration. If the first one, read one. If second, i don't know which techniques/patterns you are talking about. version() doesn't match for me here i think.
 You say no to libraries
That's not true! I'm not sure how did you came up to this conclusions, but i love libraries for applications same as command-line programs (like ffmpeg) for GUI apps.
, you say no to frameworks,
That's true i don't really like frameworks, but only because i felt like they're trying to be irreplaceable. If you're using one part, it's hard to make other parts they implemented in your own way (so they're too tightly integrated). Maybe i just tried wrong ones, i don't know. Libraries on the other hand, let you compose your program from solutions i like most, and that's why i'm trying to do this clear division between interface and implementation. So i can use different specialized libraries, and later make my own, so if someone likes my entity system he doesn't have to use my poor renderer. So hopefully i won't end-up with some big chunk of code as ogre3d.
 you say no to versioning and desire the benefits of these without what they
entail. If you come up with an alternative, then please do bring it forward. I
don't see how anyone will be able to get you what you want.
I like version control, and that's why i'm using one (git), even if i don't really need to operate with someone because of current project phase. Versions we're talking about are rather like compilation profiles, so you can use different sets of libraries to try different implementations without code modification. So, in one moment there would be code for many alternative implementations, and versions would affect only resulting binary which isn't commited at all.
Jul 13 2011
parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
shd Wrote:

 2011/7/13 Jesse Phillips <jessekphillips+D gmail.com>:
 You say no to libraries
That's not true! I'm not sure how did you came up to this conclusions, but i love libraries for applications same as command-line programs (like ffmpeg) for GUI apps.
It comes from, "and APIs are not what i'd love to work with[...]" A good library provides a good API. You build reusable code by creating a library that defines interfaces and routines that can be used by many applications. You make a library replaceable by providing defined interfaces and routines that another library can mimic. You generally don't provide a project that allows swapping out an implementation detail from a compiler switch, such as which Associative Array library to use. If you are doing graphical display you might want to support rendering to Cairo or QT or Windows Forms, but this means you will have to write the code that works with them. Or you'll have to define the API for which your library expects of a graphics rendering library and the person wanting to use one will need to write a wrapper for one of these libraries to conform to your API. This means they will have to write some code and will be using their own build setup and not one you have provided. I hope I'm being a little helpful, or at least you can see where I'm getting confused. Anyway it seems you got something workable (I was hoping someone else would pick this up if I asked the initial questions and it worked).
Jul 13 2011
parent shd <alienballance gmail.com> writes:
2011/7/13 Jesse Phillips <jessekphillips+D gmail.com>:
 shd Wrote:
 2011/7/13 Jesse Phillips <jessekphillips+D gmail.com>:
 You say no to libraries
That's not true! I'm not sure how did you came up to this conclusions, but i love libraries for applications same as command-line programs (like ffmpeg) for GUI apps.
It comes from, "and APIs are not what i'd love to work with[...]"
Yeah, but it doesn't mean i don't like library concept. The other thing is, i'm not saying libraries i'm using got bad APIs. They're not D, so they have to be C which means APIs aren't OO and i'd like to use in my code OO things which are good integrating with D language features. Furthermore, API is the only one of aspects for choosing library for yourself.
 A good library provides a good API. You build reusable code by creating a=
library that defines interfaces and routines that can be used by many appl= ications.
 You make a library replaceable by providing defined interfaces and routin=
es that another library can mimic.
 You generally don't provide a project that allows swapping out an impleme=
ntation detail from a compiler switch, such as which Associative Array libr= ary to use. If you are doing graphical display you might want to support re= ndering to Cairo or QT or Windows Forms, but this means you will have to wr= ite the code that works with them. Or you'll have to define the API for whi= ch your library expects of a graphics rendering library and the person want= ing to use one will need to write a wrapper for one of these libraries to c= onform to your API. This means they will have to write some code and will b= e using their own build setup and not one you have provided. I see your point, but i'm young enough to try things on my way first. If it won't work, i'll refactor/drop whatever.The only things i don't want to implement, are if someone already tried it, described that it failed (and WHY), If you see something against swappable implementations besides of performance and 'you can abstract it out', then i'll gladly hear. It's a personal project where i'm making first steps in this area, so i'd like to take advantage of my freshness. I'm not doing it because i have to feed myself and my family, nor someone (besides of me) isn't attached to success of it. If only thing i get from it is knowledge (and fun), good for me. So, if the only thing against is, 'generally people are doing other way' - it's ok for me to try my way. Anyway, it's not true that people aren't swapping implementations during compilation. Most famous example i believe would be linux kernel and it's config. Can't you tweak scheduler algorithm, or 439284 other options? Yes, you can. I think it's cool.
 I hope I'm being a little helpful, or at least you can see where I'm gett=
ing confused. Anyway it seems you got something workable (I was hoping some= one else would pick this up if I asked the initial questions and it worked)= . Yep, you helped a lot. Thank you.
Jul 13 2011