Skip to main content
gamedev.net
🔒 Locked

[C++] Variable Argument CallBack/Delegate Template

Started by Amnesiac5 May 8, 2009 at 5:13 PM 13 replies 4.2k views
Original Post
Amnesiac5
Amnesiac5
Apologies upfront, this is a bit involved. A bit of background for context: OpenGL lacks a standard 3d model format so the user must implement their own. The obj file format is text based therefore easy to examine and edit manually. Existing obj file readers read information into their own data structures, imposing them on the user. I am trying to implement a reader in C++ that allows the user to read data into their own data structures. I'd like it to be quick too, but ultimately flexibility is more important. Each line of an obj file is a comment, a list of strings or a list of numbers. Parsing the file is straightforward; writing the parsed data into the user's structures using their functions is the tricky part. By using function pointers I hope to use the user's own functions to write the data to their structures. Functions may be member or non-member, with differing return types and argument lists Below are examples of the functions, data types and data structures that may be used. 
struct Pt {
  double x, y;
};
struct Mesh {
  std::vector<Pt> pts;
  
  int AddPtI(Pt pt) { pts.push_back(pt); return pts.size(); };
  int AddPtI(double x, double y) { Pt pt; pt.x = x, pt.y = y; pts.push_back(pt); return pts.size(); };
  int AddPtI(std::vector<double> pts_in) { Pt pt; pt.x = pts_in[0], pt.y = pts_in[1]; pts.push_back(pt); return pts.size(); };

  void AddPtV(Pt pt) { pts.push_back(pt); };
  void AddPtV(double x, double y) { Pt pt; pt.x = x, pt.y = y; pts.push_back(pt); };
  void AddPtV(std::vector<double> pts_in) { Pt pt; pt.x = pts_in[0], pt.y = pts_in[1]; pts.push_back(pt); };
};

std::vector<Pt> pts;

int AddPtI(Pt pt) { pts.push_back(pt); return pts.size(); };
int AddPtI(double x, double y) { Pt pt; pt.x = x, pt.y = y; pts.push_back(pt); return pts.size(); };
int AddPtI(std::vector<double> pts_in) { Pt pt; pt.x = pts_in[0], pt.y = pts_in[1]; pts.push_back(pt); return pts.size(); };

void AddPtV(Pt pt) { pts.push_back(pt); };
void AddPtV(double x, double y) { Pt pt; pt.x = x, pt.y = y; pts.push_back(pt); };
void AddPtV(std::vector<double> pts_in) { Pt pt; pt.x = pts_in[0], pt.y = pts_in[1]; pts.push_back(pt); };

The following is a stab at the syntax the user would use to bind functions and load from a file.

ObjReader::BindFunc<Mesh, void, Pt>(&Mesh::AddPtV);
ObjReader::BindFunc< Mesh, void, std::vector<double> > (&Mesh::AddPtV);
ObjReader::Load(meshInstance, " file.obj ");
ObjReader::Load("file.obj");
I'm making (slow) progress using the following three articles as my principle reference: * style Case Study #2: Generic Callbacks (Herb Sutter) [http://www.gotw.ca/gotw/083.htm] covers the basics. * Delegate in Standard C++ (Ben Chun Pong Chan) [http://www.codeguru.com/cpp/cpp/cpp_mfc/article.php/c4119] adds non-member function support. * C++ Callback Solution (Arash Partow) [http://www.partow.net/programming/templatecallback/index.html] adds multiple argument support. (Template parameters are fully named which makes it easier to understand what's what.) It looks as though this is the Command pattern. So far, my implementation looks something like this:
class ObjReader {
public:
  static CallbackBase* cb;

  template<typename T, typename R, typename A>
  static void BindFunc(R (T::*f)(A)) { 
    static Callback3<T,R,A> cb1(f);
    cb = &cb1
    // 2/ at this point we know the function type etc
    // so now would be ideal time to set up the associated line parse function... but how
  };
  static void Load(void* meshInstance , std::string filename) {
    std::vector<double> pt;
    pt.push_back(10);
    pt.push_back(11);
    cb(meshInstance, pt); // 1/ doesn't work 'cos doesn't know underlying type of cb 
  }
}
So far, I've encountered the following problems: 1/ The ObjReader keeps track of the function pointers with a pointer to a base class static CallbackBase* cb, so when it comes time to invoke the function there is no way to identify its type. There seem to be two ways around this; casting to the appropriate type when needed (but to which type), or do away with the container and have n pointers, one for each supported function. 2/ When the ObjReader parses a line how can it identify the native or user defined type and process it accordingly? The following is not legal:
// for std::vector<number> - a container of any type
template<typename T>
static void ParsePoint () {
  T f1, f2;
  m_fileStream >> f1 >> f2;
  vector<T> pt;
  pt.push_back(f1);
  pt.push_back(f2);

  AddPoint(pt);
};

// for 2-tuple – any type that has x and y
template<typename T>
static void ParsePoint () {
  T t;
  m_fileStream >> t.x >> t.y;

  AddPoint(pt);
};
The functions could have different names, but still, how would I select the correct function and set up a pointer? Is it possible (or necessary to store the argument type(s) when the function is bound? This looks like it may be the Bridge, Adapter, Letter/Envelope or Handle/Body Pattern. I'm hoping there is some way to tackle the issues I've encountered, so any pointers would be most welcome (no pun intended). I'm trying to do this in "vanilla" C++, so BOOST, LOKI, etc. aren't options. TIA A5
Constipation is the thief of time.Diaorrhea waits for no man.
Antheus
Antheus
Quote:
This looks like it may be the Bridge, Adapter, Letter/Envelope or Handle/Body Pattern.


Holy pattern overload... I never even heard about Letter/Envelope...


The way I solved this problem was in simplest possible manner:
struct ObjCallback {  virtual void addVertex(float x, float y, float z, float w) = 0;  virtual void addFace(int indices[], int n);  ...  // and so on, for each command};
This is all that user implements, then passes it to parser. ObjCallback holds necessary state for user to process data as it's being read.

Then, depending on needs, I can implement arbitrary callback. For example, if I want to allocate structures up-front, I implement a counting callback, which doesn't handle parameters, but merely counts number of each vertices, faces, etc...

In C++, it's possible to implement the above using no virtual functions, simply by templating by Callback.

Quote:
Existing obj file readers read information into their own data structures, imposing them on the user.


Callback approach imposes nothing whatsoever, doesn't require function pointer hacks, can be allocation-free (if memory is your concern) and is completely trivial to implement using LALR or similar recursive descent parser.

Note that I also don't need to define extra data types, since formats of commands are very specific and well-defined.

Quote:
I'm trying to do this in "vanilla" C++, so BOOST, LOKI, etc. aren't options.

If you choose callback, then everything, except for callback object itself, can be plain C. IMHO, complicating with function pointers for true C solution isn't really worth it, but is perfectly possible.

If you're merely trying to learn function pointers, then ignore the above.
CmpDev
CmpDev
Quote:
I am trying to implement a reader in C++ that allows the user to read data into their own data structures. I'd like it to be quick too, but ultimately flexibility is more important.

I would have to disagree with you here speed is very much so more important.
If this is a learning exercise then carry on by all means. On the other hand if you want people to use this library, I would argue it would be better to create a pipeline tool to bake the data to in a format which suits the users requirements.
Amnesiac5
Amnesiac5
Thanks for the feedback.
Quote:
Original post by Antheus
Holy pattern overload... I never even heard about Letter/Envelope...
Indeed. My initial inquiries were aimed at callbacks and delegates before finding the Command Pattern. Sadly, there is far less information available about the latter than the former. It seems that patterns aren't that widely used or recognised, which is great shame. So, even identifying patterns isn't always that useful. [sad]
Quote:
The way I solved this problem was in simplest possible manner:
I'm always a fan of simple! [smile]
Quote:
struct ObjCallback {  virtual void addVertex(float x, float y, float z, float w) = 0;  virtual void addFace(int indices[], int n);  ...  // and so on, for each command};
This is all that user implements, then passes it to parser. ObjCallback holds necessary state for user to process data as it's being read.

>>> The sound of screeching brakes!!! <<<

Bit of a change of direction for me, right there. Give me minute to get my head around it -- although it has given me an idea for a totally different tack.

I think I see what you're saying, but how would the parser know which functions to call from ObJCallback? Are they predefined and the user has to implement them?
Quote:
Then, depending on needs, I can implement arbitrary callback. For example, if I want to allocate structures up-front, I implement a counting callback, which doesn't handle parameters, but merely counts number of each vertices, faces, etc...

In C++, it's possible to implement the above using no virtual functions, simply by templating by Callback.
Sorry, I don't follow.
Quote:
Callback approach imposes nothing whatsoever, doesn't require function pointer hacks, can be allocation-free (if memory is your concern) and is completely trivial to implement using LALR or similar recursive descent parser.
New terms for me, warranting further research... sigh, if only Google were my friend... I'm using Visual Studio, can it cope or do I need an additional parser/pre-processor?
Quote:
If you're merely trying to learn function pointers, then ignore the above.

Merely? Yes, I'm using the oppportunity to learn about function pointers... and patterns, but there is some practical use I wish to derive. I just thought (hoped, wished) that this would provide a more elegant solution for the user (me). If I don't resolve these issues soon I'll probably have to settle on a half-way house solution and move on.

Thanks again,

A5
Constipation is the thief of time.Diaorrhea waits for no man.
sebbit
sebbit
Quote:
Original post by Amnesiac5
I think I see what you're saying, but how would the parser know which functions to call from ObJCallback? Are they predefined and the user has to implement them?


ObjCallback in that instance serves as the interface for a collection of callback methods. So, instead of passing a different callback for each element of the object file you could jast pass it one concrete instance of an object that implements ObjCallback. Then your parser could just call those methods for every element of the parsed file (addVertex for each vertex and so on for each normal, face ...).

Nice and simple and safes you alot of fumbling around with callbacks (which can be quite hard to achive for instance methods in a robust way in c++ if you don't want to use one of the existing libraries).
Antheus
Antheus
Quote:
I think I see what you're saying, but how would the parser know which functions to call from ObJCallback? Are they predefined and the user has to implement them?


Since you mentioned patterns, start with the whole development methodology. How will user use your class? The "story", or "use-case".

Parser p("myfile.obj");p.parse(myCallback);
A file, unless you support includes or shell commands, is parsed top-to-bottom as a whole. The result is a set of structures.

Callback instance will therefore contain the entire state of what is represented in file.

Parser goes through file, line by line, extracting commands. Some commands may span multiple lines in same way C/C++ macros do ('\'). First word of a command always identifies the command.

To determine the format of each command, use the complete reference. You'll notice that they are quite trivial to parse, and have fixed format.

For example, vertex ('v') has the following form:
v x y z w

If w is not defined in file (if there are only 3 values), w defaults to 1.0. If not all x,y,z are defined, it's an invalid vertex, and you may abort.

Therefore you have your callback:
void addVertex(float x, float y, float z, float w);

Now you can implement a simple callback:
struct MyCallback : public ObjCallback {  virtual void addVertex(float x, float y, float z, float w) {    vertices.push_back(Vertex4D(x,y,z,w));  }  std::vector<Vertex4D> vertices;};
To expand on original example:
MyCallback myCallback;Parser p("myfile.obj");p.parse(&myCallback);std::cout << "File contains: " << myCallback.vertices.size() << " vertices.";


Note that Vertex4D and vertices are user defined structures. Library passes values via function parameters and doesn't impose any structure at all.

Now all that's left is to go through specification, and define parser for each command, and appropriately convenient callback.


Another example - pre-allocating memory:
struct CountCallback : ObjCallback {  CountCallback() : nVertices(0) {}  virtual void addVertex(float x, float y, float z, float w) {    nVertices++;  }    int nVertices;};struct PreAllocatedCallback : ObjCallback {  PreAllocatedCallback(int nVertices)     : vertices( new Vertex4D(nVertices) )    , n(0)  {}  virtual void addVertex(float x, float y, float z, float w) {    vertices[n].x = x;    vertices[n].y = y;    vertices[n].z = z;    vertices[n].x = w;    n++;  }  Vertex4D * vertices;  int n;};....  Parser p("myFile.obj");  // first, parse the file to count the vertices  CountCallback ccb;  p.parse(&ccb);  // parse again to read the vertex values  // now that we know how many there are  PreAllocatedCallback pcb(ccb.nVertices);  p.parse(&pcb);
Amnesiac5
Amnesiac5
Thanks for the feedback.
Quote:
Original post by CmpDev
I would have to disagree with you here speed is very much so more important.
If this is a learning exercise then carry on by all means.
For me in this particular context flexibility is more important. Naturally, if, after all this effort, it takes half-an-hour to load an 8 vertex cube then speed would become an issue.

My first attempt used non-member function pointers and only returned vectors of strings which necessitated using an intermediate builder. That was about half the speed of Nate Robbins glm.c. A not unreasonable speed, but it lead to the original motivation for this latest exercise to see if converting the stream input to numerical directly would speed things up, but one thing lead to another... [grin]
Quote:

On the other hand if you want people to use this library, I would argue it would be better to create a pipeline tool to bake the data to in a format which suits the users requirements.
Bake? How so?

A5
Constipation is the thief of time.Diaorrhea waits for no man.
Amnesiac5
Amnesiac5
Quote:
Original post by sebbit
ObjCallback in that instance serves as the interface for a collection of callback methods. So, instead of passing a different callback for each element of the object file you could jast pass it one concrete instance of an object that implements ObjCallback. Then your parser could just call those methods for every element of the parsed file (addVertex for each vertex and so on for each normal, face ...).

Nice and simple and safes you alot of fumbling around with callbacks (which can be quite hard to achive for instance methods in a robust way in c++ if you don't want to use one of the existing libraries).
Sorry, I'm not trying to be obtuse but I really can't see what you mean. An example might help.

I have pressing matters to attend to, I'll spend some time trying to grok this later.

A5
Constipation is the thief of time.Diaorrhea waits for no man.
Amnesiac5
Amnesiac5
Quote:
Original post by Antheus
Since you mentioned patterns, start with the whole development methodology. How will user use your class? The "story", or "use-case".

>>> snip <<<
This article Callbacks In C++ Using Template Functors by Rich Hickey (way back from 1994) has been pointed out to me on a newsgroup and seems to cover exactly the problems I'm having, tackling them with a translator. It certainly warrants further investigation.

The points you've raised intrigue me greatly (mainly because I'm really struggling to understand what you're talking about) and I'll pursue them further in a week or so after my exams. But for now I really should get on with some revision.

Many thanks for the comments and taking the time to answer my queries, all very much appreciated.

A5
Constipation is the thief of time.Diaorrhea waits for no man.
Antheus
Antheus
Quote:
Original post by theOcelot
If I understand what Antheus is saying, you can think of ObjCallback as a combination Strategy/Builder object. As long as we're thinking in design patterns...


It's a simple visitor.
for (line in Lines) {  Command c = parseLine();  visitor.visit(c);};


File is parsed line by line, with one of x predetermined commands being invoked on each of them.

Most of problems that involve iteration over a set of elements and taking one action are solved via visitor, and have traditionally been implemented using a user-provided callback.
theOcelot
theOcelot
Crap. I never really understood the visitor pattern, but that's because I thought it was more complicated than that code snippet.

So the ObjCallback is visiting the lines of the file. The parse method of the Parser is equivalent to the accept method of a typically visitable object.

With this loose definition, the function/functor I pass to a std algorithm is also a visitor, and the algorithm itself is the accept method. Am I getting this right?
Antheus
Antheus
Quote:
Original post by theOcelot

So the ObjCallback is visiting the lines of the file. The parse method of the Parser is equivalent to the accept method of a typically visitable object.


It's actually a foreach loop:
foreach (Item i : items) visit(i);
The only difference in Visitor pattern is that visitor might have multiple methods for different types.

I consider this to be language-specific workaround. Dispatch based on type of Item can be done in several ways, either inside iterator, or inside visitor. Visitors can also become very bloated and clumsy, Java implementations are very susceptible to this.

But original problem description is just that.
- I have a set of strings
- For each string in entire set, I want to perform an action
- I do not care how data is represented internally

Such decoupling can drastically improve code reuse, since visitors are completely independent from iterator, and iterator is bound to very specific interface. C++ streams library is an example of this. So are various Enumerations and Enumerable types, iterators and similar.

Quote:
With this loose definition, the function/functor I pass to a std algorithm is also a visitor, and the algorithm itself is the accept method. Am I getting this right?
Yes.

Iterator knows how to iterate, but can't handle.
I know how to handle individual elements, but can't iterate.
visit(iterator, callback);


This is where my beef with GoF school of patterns comes from. It emphasizes the canonical implementation of a Pattern over what it really represents. In C# and Java, the canonical implementations are often good enough, or one of few reasonable ways to implement them. But patterns apply to all languages and all paradigms, they just differ in implementation.

Prototype pattern - generalized (C++) copy constructor or other forms of copy construction (AssignFrom() in Delphi for example).
Composite - anything iterable, where individual elements can optionally be iterated over as well; any container.
Factory - any allocator, as well as new/malloc (new and malloc are just factory functions which return pointers).
and so on, there is quite some redundancy.

I consider any differences in those to be mere implementation details, but patterns remain the same. But as mentioned, I am biased against Patterns and prefer patterns instead.
theOcelot
theOcelot
Quote:
Original post by Antheus
Quote:
Original post by theOcelot

So the ObjCallback is visiting the lines of the file. The parse method of the Parser is equivalent to the accept method of a typically visitable object.


It's actually a foreach loop:
foreach (Item i : items) visit(i);

Wait, which is a foreach? The parse method? How exactly does that fit in with the pattern? I don't want to degeneralize it, I just want to see how it's analogous to the explicit, GoF-type visitor.
Quote:
The only difference in Visitor pattern is that visitor might have multiple methods for different types.


In C++-land that means different overloads, right? At least the canonical version. You could have different-named ones, just to make it explicit that different visitation methods are being called. Hmm...

Quote:
I consider this to be language-specific workaround. Dispatch based on type of Item can be done in several ways, either inside iterator, or inside visitor. Visitors can also become very bloated and clumsy, Java implementations are very susceptible to this.

But original problem description is just that.
- I have a set of strings
- For each string in entire set, I want to perform an action
- I do not care how data is represented internally


Which data? The string data, the results of the string processing, or all of it?

Quote:
Such decoupling can drastically improve code reuse, since visitors are completely independent from iterator, and iterator is bound to very specific interface. C++ streams library is an example of this. So are various Enumerations and Enumerable types, iterators and similar.

Quote:
With this loose definition, the function/functor I pass to a std algorithm is also a visitor, and the algorithm itself is the accept method. Am I getting this right?
Yes.

Iterator knows how to iterate, but can't handle.
I know how to handle individual elements, but can't iterate.
visit(iterator, callback);


You keep using visit as a free function. What exactly did that last line mean? I thought visit was the callback.

Quote:
This is where my beef with GoF school of patterns comes from. It emphasizes the canonical implementation of a Pattern over what it really represents. In C# and Java, the canonical implementations are often good enough, or one of few reasonable ways to implement them. But patterns apply to all languages and all paradigms, they just differ in implementation.

Prototype pattern - generalized (C++) copy constructor or other forms of copy construction (AssignFrom() in Delphi for example).
Composite - anything iterable, where individual elements can optionally be iterated over as well; any container.
Factory - any allocator, as well as new/malloc (new and malloc are just factory functions which return pointers).
and so on, there is quite some redundancy.

I consider any differences in those to be mere implementation details, but patterns remain the same. But as mentioned, I am biased against Patterns and prefer patterns instead.


I don't think I ever went to the extent of viewing malloc as a factory function, but I totally get what you're saying here. These patterns can be expanded (or simplified) far beyond what they are in the GoF book.

Thanks for helping me get this. The connections are starting to be made in my head.
Amnesiac5
Amnesiac5
Yay, exams over, so back to the grind...
Quote:
Original post by Antheus
Since you mentioned patterns, start with the whole development methodology. How will user use your class? The "story", or "use-case".
I thought I had. This thread is focusing on specific problems I'm having.
Quote:
Note that Vertex4D and vertices are user defined structures. Library passes values via function parameters and doesn't impose any structure at all.
If Vertex4D and vertices are going to be defined by the user some time in the future how can I access (or plan for) them now?

TIA

A5

Constipation is the thief of time.Diaorrhea waits for no man.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.