🔒 Locked

Plugins in Python

Started by Knarkles May 24, 2007 at 4:43 AM 3 replies 2.7k views

Original Post

May 24, 2007 at 4:43 AM

I'm rewriting part of my engine (here's the dev journal) to support pluggable components for the window manager, renderer, audio etc... I haven't yet arrived at a satisfying implementation for it though, and that's why I'm asking for your opinions and suggestions. Note that the engine's not really structured exactly like this, I'm simplifying a bit to make the point clear. At first, components were structured like this:

engine
  components
    audio
      dummy
      openal
    renderer
      dummy
      opengl
        mesh

When a component was loaded, its module or package was copied to the root level. In effect, something like: engine.renderer = components.renderer.opengl The problem with this approach was that you couldn't import anything from inside packages. Ie. from renderer.mesh import TriangleMesh wouldn't work; instead, you'd have to write TriangleMesh = renderer.mesh.TriangleMesh. I wasn't happy with this, and researched alternatives. Then I found/remembered the __path__ attribute of packages, which you can modify to alter the paths considered part of the package. So, I restructured the engine by moving the component packages to the top level. Components were now loaded by modifying the __path__ attribute of the component category package and copying component-level attributes to the component category package. In other words, like this: engine.renderer.__path__ += "engine/renderer/opengl" engine.renderer.__dict__.update(engine.renderer.opengl.__dict__) I was initially happy with the solution, but discovered a different problem with the new approach. Copying the attributes means that modules inside the component (for example, opengl.mesh) don't see the updated component-level attributes anymore, they see the attributes inside the opengl package. For example, calling renderer.init(), which updates renderer.max_texture_units; results in renderer.opengl.max_texture_units to remain at the old value, and this is the value which the component modules see. This is where I'm stuck currently. One option would be to simply not allow component-level attributes, but I don't think that's a good solution. Do you have any new ideas for implementing plugin loading, or suggestions for fixing the problem with component-level attributes? If any part of the explanation is unclear, please ask for more information. [smile] Thanks.

Vorpy

May 24, 2007 at 12:40 PM

I think you could use:

engine.renderer.__dict__ = engine.renderer.opengl.__dict__

Possibly saving the original __dict__ somewhere just in case. I have no idea if this could be considered a good idea or not, but I think it would do what you want. Edit: This is definitly a bad idea, the python documentation says not to assign a new value to a module's __dict__.

edit: A better idea might be to simply copy all of the names that are expected to be provided by the plug-in into the root level. You can write a function that takes a target dict, a source dict, and a list of names to be copied from the source to the target, and then use this function to implement plug-in loading.

[Edited by - Vorpy on May 24, 2007 6:40:19 PM]

Knarkles

May 24, 2007 at 5:35 PM

Ok, I solved it, kinda.

Since the problem was only with attributes which changed after the "separation" (copying attributes from __dict__), I replaced such attributes with functions that return them. This works because unlike variables, functions remain in the original scope they were defined in, so they also read the variables from their original component. No, not 100% Pythonic, but not really ugly either IMO.

For example, if I assign renderer.get_max_lights = renderer.opengl.get_max_lights, the function still reads the variable _max_lights from the opengl module, even if called through the renderer module.

Still, if anyone has better suggestions, I'd be happy to hear from you. [smile]

Vorpy

May 24, 2007 at 6:17 PM

Wrap the getter (and setter, if the variables are mutable) functions in properties so you can access them using the same syntax you would use for variables. Actually I guess properties only work for classes, not modules...which leads to another idea: make the plug-ins objects instead of modules.

Knarkles

May 24, 2007 at 6:27 PM

Quote:
Original post by Vorpy
Wrap the getter (and setter, if the variables are mutable) functions in properties so you can access them using the same syntax you would use for variables. Actually I guess properties only work for classes, not modules...which leads to another idea: make the plug-ins objects instead of modules.

A good idea, but... in that case my original solution would have worked well. The reason I went with my current implementation is because I want to allow plugins to be packages (for example, the renderer module currently consists of 12 modules, counting __init__.py, and 1100 lines of code, with more to come).

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.