File Workbench

FWB uses a binary object model (DynObj) to enable run-time addition of new functionality (plugins). The DynObj project home page is: www.sf.net/projects/dynobj.

DynObj related Wiki pages:

• General interfaces. The page describes general (not application specific) interfaces used by FWB.
• FWB core interfaces. Here the more FWB specific interfaces are documented.

DynObj Background

There are no standardized ways of extending a compiled C/C++ program. Once a developer has compiled an application and made it available to a user, extending or modifying that binary application has been tricky. The person modifying it usually has to do a full rebuild, accessing the full source code base of the application and often having to use the same set of development tools as the original developer(s).

The DynObj object framework fills that run-time extension space. It provides a (largely) compiler and platform neutral way of describing and identifying plugin interfaces, and also for plugins to navigate other plugins/loaded interfaces and use them.

Developing a DynObj plugin one typically only needs a few header/interface files from the source code base, and one can use a different version / brand of the compiler. The plugin can even be developed in a different language than C++.

The DynObj framework is in several ways similar to other plugin / binary object frameworks, such as COM (Windows specific) or XPCOM (used in Mozilla based software). The DynObj framework is more light weight and almost transparent to use from a C++ application. It is also a well defined component in itself, without big bindings to specific platforms or other big software packages.

Interfaces, Implementations and Objects

If we think of a machine such as a tape recorder, then the interface equals its outer displays, controls, buttons and knobs. It’s how a user typically interacts with the tape recorder. The implementation is the interior of the tape recorder. Typically when using it, we have little interest in it. But, without the interiors, the tape recorder is just an empty shell. Both are needed.

The combination of interface and implementation form the useful object.

Plugins refer to that whole object, but, of course, in practice we only use their interfaces. The implementations lie hidden inside the compiled, loaded run-time link libraries (*dll/ *.so, …). We just trust they work.

Since DynObj can use plain C++ header files to describe interfaces, it follows that to the program, plugin objects and ordinary (non-plugin) objects are handled just in the same way. So plugins can be used very transparently.

DynObj Basics

Any run-time object that has virtual functions (implying it has a vtable pointer) can be regarded as a plugin object. In practice, the vtable pointer is the first member of almost all binary objects. So, what is needed is a binding between vtable:s and type descriptions. This is achieved through registering an object the first time one of a given type is created. That can happen either automatically or manually. So, we can use pretty much any existing class / object hierarchy as DynObj:s, often without modifying their source codes.

There is a further distinction between free and DynI rooted plugin objects:

• Free interfaces – The objects don’t have a shared base class, but they do have a vtable as their first member.
• DynI interfaces – The objects are based on the DynI class and have some shared, very basic methods for asking about its type, querying for other supported types and handling errors.

The DynObj class library provides a small class hierarchy based in DynI that is useful when developing new plugin object types. An object can implement several interfaces and free and DynI based interfaces can be mixed freely.

Naming Conventions

Usually a letter I is appended to the end of a DynObj type name, to hint that it’s an interface:

But interfaces are often intended to be used in slightly different ways. If we have and interface:

it is reasonable to expect it to be extended (Triangle, Ellipse, …). These forms would implement the basic functions of ShapeI.

For some other interfaces, we don’t really expect them to be extended very much. They are intended for a pretty final class. In the DynObj library, we have a string interface class DynStr. Technically, it’s an interface, but most of the time we’re happy with the default implementation, so we settle for the name DynStr and omit the I here.

DynObj Type ID:s and Names

Each DynObj interface is identified with a (usually) unique 32 bit integer type ID. This is small, fast and easy to handle.

This 32 bit range is further divided a public and private range. Type ID:s in the public range are shared and refer to the exact same type when used in different applications (they are like public IP addresses). ID:s in the private range are not public and can refer to different types from app to app (they correspond to dynamic, re-used IP addresses).

Many binary object models use identifiers that are 128 or even 256 bits wide. This is useful to store additional type information in the ID itself, but from a strict type identification perspective, so many bits are not needed. Given that each type usually has several KB of code associated with it, a very large program will usually not use more than some 1000+ plugin types (corresponding to 10 needed type bits, far below the available 32 bits).

The type ID corresponds also to a type name string. There is no guarantee however that the type string is unique though.

Once an interface has been reported as public and stable, it cannot be changed in any significant way. It is frozen. However, in a process leading up to this, the interface is ‘semi-public’ but can still be modified (a candidate interface, it’s in beta).

The public stable interfaces can be building blocks in new applications. And, new interfaces can be derived from (inherit) existing stable interfaces, providing a way for evolving and adding to existing public types.

Limitations

Given that DynObj plugin objects and the host program can be used in such a seamless way, even if they’re built with a quite different compilers, what are the limitations?

What happens when loading a DynObj plugin library is that the the ordinary static linking between the main app and the plugin is skipped. The only way for them to interact is through the interface (and any other mutually known interfaces). There is no static linking between the two.

Static linking is quite a complex and compiler specific thing. On the other hand, linking to tables of virtual functions is as simple as choosing an integer index into the table. And most compilers tend to do that in the same way.

As a consequence, no shared global name space is available. This is where all global functions (non-virtual members and non-members) and data is stored.

But, apart from that, we can have inline member functions, C++ template member functions (they are inline by default) and anything else that does not generate static linking.

Also, the ordinary dynamic_cast() operator will not work across compiler boundaries. The cross compiler portable do_cast() operator should be used instead (and it does the very same thing, working across plugin boundaries).

Experience has also shown that some compilers put functions into vtables in different ways when several functions have the same name (overloading). So, that’s better avoided in a DynObj interface.