Implementation Strategies for Extensible Application (plugin API)

Quote:

Original post by Oluseyi I normally ask these kinds of questions on #gamedev so I have near-immediate turn around, but my current internet access solution doesn't appear to free port 6667, and as I've taken a Facebook hiatus for now, I can't put the question to the regulars that way. (Yes, they're all my FB friends - Promit, Washu, Superpig, jpetrie, benryves and many more. You guys rock!) [smile] (Hush, Washu, you'll get your articles/reviews when I get my freedom from West Africa! LOL)

Delays, delays.. and yet more delays. You know, that Hello Kitty Oluseyi blog might go live ANY DAY NOW.

Quote:

I have an application which currently has a primitive plugin API. We're looking to make it considerably more robust, which has me thinking about how to best structure the application-plugin data bridge, particularly for rich types with associated behaviors. In a previous incarnation of the application, a dynamic link library containing the object code for all "core" types had to be linked against each plugin, guaranteeing the availability of all implementations. This... smelled to me, for lack of a more detailed technical objection, and I refactored that out. However, that introduced a number of constraints on the nature of the application-plugin bridge: now plugins could only be defined in terms of pure abstract types, necessitating some reworking of various types within the application core to express the correct interfaces.

I vaguely recall this application and this refactoring.

Quote:

Is this wise? Is this idiomatic? Does this potentially introduce more problems than it solves? What is the most robust strategy for implementing a plugin architecture with the ability to communicate reasonably rich (POD) types? Our application is built on Qt, and all plugins must link against Qt anyway so we can use a number of Qt types with value semantics (and/or pointers to them) for data exchange; is it that much better than linking against our own DLL/SO and directly using our rich types? What do the SDKs for plugins to, say, Photoshop or Office do, if any of you know? I'm starting to think that removing the "core types" DLL was a mistake, which I have no problem admitting if so. Any insights, pointers and advice are all very welcome. Thanks.

Maybe, maybe not. There is nothing that stops you from sharing complex POD types, provided they are binary compatible across both the application and plugin. Typically that's best done through the use of shared headers with the same packing flags explicitly set (#pragma pack for instance).

That being said, you will need to ensure that ownership is very explicitly stated and handled across the entire platform. See DirectX for just how nasty that can get, but it's the basic idea.

The Microsoft application plugin frameworks for office and visual studio are COM based, which pretty much tells you everything you need to know. POD types (including complex ones) can be exchanged, but concrete classes are not exchanged except through interfaces. That doesn't mean you have to do it that way, but it is the simplest and most compiler and platform agnostic method of doing it.

Quote:

Original post by Washu Quote: Original post by Oluseyi (Hush, Washu, you'll get your articles/reviews when I get my freedom from West Africa! LOL) Delays, delays.. and yet more delays. You know, that Hello Kitty Oluseyi blog might go live ANY DAY NOW.

Heh. Well, one upside of my quitting social networking has been a significant productivity boost. Don't want to make any promises, but maybe I'll start to squeeze them out between mood boards for the cartoon show I'm animation supervisor for.

Quote:

Maybe, maybe not. There is nothing that stops you from sharing complex POD types, provided they are binary compatible across both the application and plugin. Typically that's best done through the use of shared headers with the same packing flags explicitly set (#pragma pack for instance).

Yep, that's my strategy so far.

Quote:

That being said, you will need to ensure that ownership is very explicitly stated and handled across the entire platform. See DirectX for just how nasty that can get, but it's the basic idea.

So far I've only been working on/permitting export plugins, so the application retains ownership and the plugin can only request duplicates of static data via identifiers the host provides. However the long-term plan is to allow essentially arbitrary plugins that can create data in the host as well, so that's definitely something to keep in mind.

Quote:

The Microsoft application plugin frameworks for office and visual studio are COM based, which pretty much tells you everything you need to know. POD types (including complex ones) can be exchanged, but concrete classes are not exchanged except through interfaces. That doesn't mean you have to do it that way, but it is the simplest and most compiler and platform agnostic method of doing it.

Hmm, yeah COM really does say a lot! I guess the strategy of data exchange via interfaces is sound; I'll just have to keep debugging and disentangling (this app was written with a novice's disregard for encapsulation). This is one of the few times when it almost seems like a complete rewrite might be warranted, but then my natural fear of rewriting over refactoring kicks in. I'm doing a limited rewrite in many areas, in which I build a new "outer shell" for a component and then port the "guts" from the old version, refactoring it into place.

Thanks.

In my application (which happens to be Qt based too), I use plugins to implement everything but the plugin system, which is in the host program. The host program provides a means of loading, unloading and configuring plugins, a messaging system and a way to query the system for existing plugins.

For my own convenience, loading, unloading and configuring of plugin-specific settings is done through a configuration scripting system. Plugins can send it commands directly or it can load them from config files (each plugin has an associated init script, for example).

The messaging system is how plugins communicate. The plugin API provides a way to lookup message types by string names and dispatch them. Plugins can also register their own message types by providing a string name and a "MessageAllocator" (which has Message* create () and void destroy (Message*) methods. They can create and destroy objects subclassed from message any way they want, but I provide a templated memory pool implementation which I use for any messages I define). Finally, the API allows you to bind methods to message types. These methods are then called whenever that message is sent to the messaging system. Once sent, messages are immutable, so they can safely be shared between plugins (in parallel - though I have some controls over which thread they are processed in because, eg, some Qt functions must be called from the main thread). Plugins obviously need access to the headers of messages they want to use, but otherwise the messaging system is anonymous and asynchronous. A few message types are defined by the host: a configuration script message (with a string field, containing the script - this allows plugins to run configuration scripts programatically) and notification and error reporting messages. Any application-specific messages are defined in plugins.

Finally, the API can query the system for already loaded plugins and plugins can return their own plugin-specific API. That means that if a plugin knows about another plugin, it can use its API directly, instead of using the messaging system. This typically has better performance and control, but also means the plugins have tighter coupling.

The plugins share some headers with the host - the API interface, the plugin interface, an empty Message interface, the MessageAllocator interface, the memory pool template class and some smart pointers. Typically the host takes ownership of any objects passed to it: once a plugin calls host->sendMEssage(message), the message object is owned by the host. The same goes for message allocators, plugins and any other data objects created by or passed to the host. Any other data is typically passed around in smart pointers. This simplifies plugins because they no longer need to worry about managing memory. Fire and forget etc.

So far this has been working pretty well for me.

issch,

Fantastic! That is almost exactly what I'm looking to implement in my system. You shared some interesting insights I hadn't considered in advance, like Qt functions that have to be called from the main thread. Thanks a ton for that; I'll probably spend a while really digesting all the aspects you've mentioned and see how to synthesize them into my application framework.

Cheers! [smile]

Quote:

Original post by issch In my application (which happens to be Qt based too), I use plugins to implement everything but the plugin system, which is in the host program. The host program provides a means of loading, unloading and configuring plugins, a messaging system and a way to query the system for existing plugins.

Congratulations - you just re-invented the microkernel! [smile]

The bottleneck of microkernels has classicality been the messaging facilities, because every call to another module requires at least a pair of context switches (switch to kernel, check validity/permission of message, switch to other module). In your case you don't have to deal with the overhead of context switches, but keep in mind the overhead of message dispatch, especially if you need to keep some degree of security between plugins.

So before I go reimplementing a microkernel architecture, does anyone with Qt experience have opinions on using Qt's signals and slots mechanism as a message passing method for plugins? I've done some preliminary tests which are very encouraging, and I'll be doing some further tests to see how elaborate the objects I'm passing as signal/slot parameters can be, but if this doesn't have any substantial caveats then I'd rather not spend the time writing a functional equivalent of my own.

Quote:

Original post by Oluseyi So before I go reimplementing a microkernel architecture, does anyone with Qt experience have opinions on using Qt's signals and slots mechanism as a message passing method for plugins? I've done some preliminary tests which are very encouraging, and I'll be doing some further tests to see how elaborate the objects I'm passing as signal/slot parameters can be, but if this doesn't have any substantial caveats then I'd rather not spend the time writing a functional equivalent of my own.

Well, if built correctly, you can always swap it out with boost's signals/slots implementation as well.

Quote:

Original post by Washu Well, if built correctly, you can always swap it out with boost's signals/slots implementation as well.

That's an interesting point. The Qt signals and slots documentation says you can use their implementation along with a third party signals/slots system, of which Boost is the most well-know (to me, at least).

It may or may not be a significant factor later. For now I'm going to continue with my Qt signals/slots experiments in plugins rather than writing my own asynchronous message passing architecture to accomplish the same thing. Fingers crossed...

This microkernel approach is the same I'm taking on a recent project of mine. If you dig up my posts from wayyyyy-too long ago, I talked on it some.

My current design is more component-oriented than object-oriented:
* Every major subsystem is a Component: the library, the playback controller, the UI, etc.
* Components can discover other components *if* they implement a well-known interface. At initialization time, each component is given a context that contains a reference to the ComponentManager and the EventBus. Direct component-to-component communication is only used when a method call makes more logical sense than using events...
* Which leads us to the push mechanism: the EventBus. This is similar to signals and slots but you do not know who is generating the event (IIRC, the signal lives somewhere. If this is not the case, disregard it.) The EventBus maintains a map of std::type_info objects to handlers. It also verifies that you are sending Events from the UI thread. This is inspired heavily by enterprise-level messaging middleware.
* A rich set of Events are created: PlaybackStartedEvent, PlaybackStoppedEvent, TracksAddedEvent, etc. These are PODs with a couple of relevant fields. (By now, you can tell what type of application I'm writing, just from the events.)

I can't say enough good things about the architecture. The coupling is low, you can instrument the EventBus to quickly debug things. Well-known events and interfaces are kept in a central location.

The author of Imperfect C++ has a chapter entitled "Objects Across Borders," where he attempts to use DLLs from one compiler with another, and documents the difficulties he encounters. This sounds similar to what you want to do. Memory management can be difficult. I have done something like this on a previous project, but it can be tedious. I went the common DLL route as well, but I dislike it.