How to call native functions in C++-bytecode

The ABI for different platforms varies. Windows x64 and Win32 are different. The x64 version is simpler as there has been less time for cruft to accumulate.

Here is the x64 calling convention: https://docs.microsoft.com/en-us/cpp/build/x64-calling-convention?view=vs-2019​​

The 32-bit conventions support multiple calling styles, including cdecl, stdcall, fastcall, vectorcall, safecall, thiscall, syscall, and optlink conventions. Each one has variations on which registers are used, their ordering, their preservation, and setup/cleanup.

The overall process is basically:
Prepare your own system by saving registers you need, putting the arguments directly into the corresponding register, or the address of the argument into the proper register, or on the stack.

Juliean said:
How do I even start? I looked a bit at the assembly that is involved in c++-code calling functions; but how would I achieve this myself? Do I have to write inline-assembly or something even more complicated?

The easiest is cdecl format since it is done entirely on the stack, which can be easily done in an __asm block. Push the frame information (which helps with debugging and exception handling but may be turned off with compiler options), push parameters on the stack in the correct order, use the asm call instruction to the function address, then pop results and restore the call frame.

The other formats place variables in registers, potentially using general purpose registers, floating point registers, SIMD registers, and also using the stack as needed. The basic pattern is similar to those already mentioned. Save what you need to save including frame information, put your parameters into the appropriate locations, call the function address, do the appropriate cleanup for the calling convention to capture returned values and restore the frame pointer.

Juliean said:
How would I go about passing arguments? Again I looked a bit at the assembly, and it seems for functions that take primitive types (starting easy), the compiler seems to be pushing arguments onto more and more registers. But like, how would I know which registers to use; what am I doing when I run out of registers and how do I handle more complex cases like struct-by-value etc…? While I could find that out probably by trial of error, I was wondering if there was some documentation/tutorial/articles that handles this in-depth?

Yes, there are both tutorials and documentations. MSDN has rather comprehensive documentation, I linked to the x64 version, but for the 32-bit variety you need to look up each calling convention individually.

Juliean said:
I have a limited know of data-types that could be passed, though there are a few variants (ie. primitive types int/float/uint8_t by value or by ref; objecty by pointer or ref or ** for modifying what object is stored at a certain location; structs by value or ref or const&)

You will need to know exactly what type of data is being passed and expected on the other side. There are some automatic promotion rules which may need to apply, such as storing your uint8_t in a 64 bit RCX register. Structs/unions/classes and certain SIMD types are passed as pointers, either pointing directly to the object, or when it was otherwise optimized away placing the value on the stack and pointing to the stack location, then using that pointer.

Juliean said:
Given any number of parameters not know until I run my own compiler, how do I generate the code necessary to handle those parameters in c++?

That's going to depend on your code and the calling convention you're using.

Again, 32-bit cdecl would be the easiest as values are all placed on the stack. Unless you're getting fancy with return values you can loop with mov, push, mov, push, mov, push… until they're all in place. If you're using non-POD data types you may need a bonus value for a hidden parameter, but it's easy enough to implement.

More complex functions would involve passing a mix of data types like integers, floating point values, SIMD variables, and POD structures where different data types need to be placed in different registers. Even more complex is when there are more parameters than will fit in the number of registers, like the six parameter version you described when the four registers used would be rcx, rdx, r8, and r9, then placed on the stack in the shadow space.

Juliean said:
So, can somebody give me some pointers?

Make sure you're comfortable with at least the rudimentary aspects of assembly programming first.

Learning the basics is not a particularly difficult task, but does take some time and acclimation. The environment is quite constrained with a relatively small number of registers and commonly used operations. You won't need to learn all the obscure opcodes, just enough to understand what is going on. As far as assembly language goes, being able to manipulate data in registers, manipulate the stack, and call functions are straightforward tasks.

I cut my teeth on it self-taught in high school (early 1990s) and two college courses included assembly programming. These days it is rarely taught in school, although a few schools have kept assembly as an optional course. Bulletin boards (imagine dial-up websites if you don't know them) had references like Ralf Brown's famous Interrupt List, which started as a university professor's personal documentation and grew to an enormous online reference for assembly programmers, most of it still applies today. I had a printout of the thing when it was close to 400 pages, all printed on an old dot matrix printer at roughly 6 point font.

At the level of “your own bytecode” you should probably translate arbitrary function calls into generic elementary operations that can be combined to handle the buffers you use as the stack of your function calls:

• Copy a value of a certain type from or to a representation with a certain format, either at an arbitrary position in memory or in a certain CPU register
• Alter the SP register (possibly combined with writing to the stack, like in the PUSH x86 instruction)
• The function call itself, presumably just a CALL instruction or something similar

@frob Thanks a lot, thats making things a lot clearer already! Even so, I'm still a bit confused about the details of:

frob said:
Again, 32-bit cdecl would be the easiest as values are all placed on the stack. Unless you're getting fancy with return values you can loop with mov, push, mov, push, mov, push… until they're all in place. If you're using non-POD data types you may need a bonus value for a hidden parameter, but it's easy enough to implement.

So with loop, do you mean loop within the asm-blocks or looping eigther within my bytecode or inside my bytecode-instruction before the actual call happens? I'm just unsure because I'm assuming that when I start to push my arguments on the stack, I cannot have my own code modify the stack as well as I need my arguments to be there when I issue the call-asm. Or is it just a matter to make sure that whatever my code does for pushing the argument is off-stack when I put the argument there via asm?

frob said:
You will need to know exactly what type of data is being passed and expected on the other side. There are some automatic promotion rules which may need to apply, such as storing your uint8_t in a 64 bit RCX register. Structs/unions/classes and certain SIMD types are passed as pointers, either pointing directly to the object, or when it was otherwise optimized away placing the value on the stack and pointing to the stack location, then using that pointer.

At least for having that information, its not a problem. And again, everything is pretty much streamlined to a few potential cases (everything else would be a compile-error. I just have primitive (int/float/uint8_t), string, enum class (arbitrary unsigned datatype), object/class-types, structs and thats it. No unions or SIMD-types. Obviously a few different modes by which arguments can be passed, but also limited (int& can be passed to signify a ref-argumentt; but const int& would be considered an error as convention for primitives is by-value). Its pretty much tailored towards the front-end representation of my unreal/blueprint-esque visual scripting; which cannot handle arbitrary values but more like a superset of types.
Well ok, there's also system/user-defined “supplied” values which are not represented in the graph like the current timestep, scene etc… but those also at least have a predefined mode for how they are passed.

frob said:
Make sure you're comfortable with at least the rudimentary aspects of assembly programming first. Learning the basics is not a particularly difficult task, but does take some time and acclimation. The environment is quite constrained with a relatively small number of registers and commonly used operations. You won't need to learn all the obscure opcodes, just enough to understand what is going on. As far as assembly language goes, being able to manipulate data in registers, manipulate the stack, and call functions are straightforward tasks.

I think I do understand the basics. I was looking at unoptimized assembly to get a basic understand of what I can/should do with my own bytecode. Not sure if thats enough though. At least for creating the bytecode format and my own stack myself I think I'm good for now, so I might just keep with my old format of calling functions to get started (that should mean that I won't need to use assembly). I'm still going to do it eventually, but that way I might get faster results. Currently my functions would be called by a wrapper, that via template-magic would expand to something like this:

void nativeFunction(int a, int b, float c);

void call_nativeFunction(CallState& state)
{
	nativeFunction(getArg<int>(state), getArg<int>(state), getArg<float>(state));
}

I think if I use that format, and instead of doing complicated stuff of dynamically looking up arguments and instead just put them on my own stack before the call and have getArg simply pop it, that might be good enough just to get things running.

LorenzoGatti said:
At the level of “your own bytecode” you should probably translate arbitrary function calls into generic elementary operations that can be combined to handle the buffers you use as the stack of your function calls: Copy a value of a certain type from or to a representation with a certain format, either at an arbitrary position in memory or in a certain CPU register Alter the SP register (possibly combined with writing to the stack, like in the PUSH x86 instruction) The function call itself, presumably just a CALL instruction or something similar

So essentially you mean having my own instructions for thuse kind of things, right? Aside from having push/pops onto my own stack I would then have separate instructions that would take values of the stack and put them into the designated registers and so on?

Juliean said:
Users must be free to register eigther global functions. lambdas or member functions w/o being required to write wrappers (though template-metaprocessing as well as reflection can obviously be used to generate the necessary code)

I think you have two options here, one is as discussed, you need to implement the proper ABI to deal with parameters and return types.

The other option is to only support a small number of specific signatures.

CPython for example I believe does this exclusively:

PyObject* print_score(PyObject* self, PyObject* args)
{
    const char *name;
    int score;
    if (!PyArg_ParseTuple(args, "si", &score))
        return NULL;
    printf("%s: %d\n", name, score);
    Py_RETURN_NONE; // Py_INCREF(Py_None); return Py_None;
}

In this way the byte code and implementation doesn't need to worry about different types etc., and is likely very similar to calling script functions, as all the data types are the same, in a way a script function is just a special exec native function that takes a block of bytecode as the first param.

On the C++ side you could potentially do clever stuff to automate some of that, but it is separate from the interpreter API itself, and doesn't have any special byte code representation.

For example a variadic template function can check the size of args, then unpack each.

SyncViews said:
In this way the byte code and implementation doesn't need to worry about different types etc., and is likely very similar to calling script functions, as all the data types are the same, in a way a script function is just a special exec native function that takes a block of bytecode as the first param. On the C++ side you could potentially do clever stuff to automate some of that, but it is separate from the interpreter API itself, and doesn't have any special byte code representation. For example a variadic template function can check the size of args, then unpack each.

Thats already how my current setup works ? At the moment, I invoke each function via a pointer to a global handler like so:

using ScriptFunction = void (*)(CallState& state);

And internally, I ran some very complicated template-process that will essentially generate a function that looks like what I posted above:

void nativeFunction(int a, int b, float c);

void call_nativeFunction(CallState& state)
{
	nativeFunction(getArg<int>(state), getArg<int>(state), getArg<float>(state));
}

(The one thing that I really do not wanna have to do is write wrapper functions myself. I started out like this, but ever since I'm able to do that:

	void SceneGameSystem::OnRegister(sys::StringView strFilter)
	{
		event::registerBindFunction(&SceneGameSystem::ActivateScene, strFilter, L"ActivateSceneNoBlock(scene)");

		event::registerBindFunction(&SceneGameSystem::GetActiveScene, strFilter, L"scene GetActiveScene");
		event::registerBindFunction(&SceneGameSystem::IsSceneLayerActive, strFilter, L"active IsSceneLayerActive(scene, layer)");
	}

There is no way I'm going back to writing those wrappers myself)

The main problem with the execution-approach in my current system is that everything is based upon calling those function, including primitives like conditions, loops, etc… and as its graph-based the entire flow of excection to be evaluated dynamically as well (there is a compiler, but its not really that complete).

So I do think I could keep my template-based binding code (by porting it to read from my new stack) to get started. I was just thinking that for performance, I would want to do native calls. Or is there not even that much difference?

Juliean said:
The main problem with the execution-approach in my current system is that everything is based upon calling those function, including primitives like conditions, loops, etc… and as its graph-based the entire flow of excection to be evaluated dynamically as well (there is a compiler, but its not really that complete).

I think that is fairly common with interpreters, if I recall Python and Ruby work that way for operators even, and in Rubys case most of the common loops (I tend to use x.each do |y| etc. a lot more than the language level while etc.). The main optimisation I recall is CPython stores the pointers to certain common functions directly in the type-struct objects, rather than a generic method table, and I think Ruby has an inline cache slot at every call site to save the function pointer as long as no types are modified, and in practice a lot of call sites will always be the same object type., so it ends up like

y = x + 5

// CPython effectively does this
x = x->ob_type->tp_as_number->nb_add(x, const_5);

// MRI Ruby does something like
// somewhere at compile/init time
static ID ID_ADD = rb_intern("+");
// call site
static int cache_global_method_state = -1; // global_method_state increments every time a method is defined somewhere
static int cache_type_id; // not sure exactly how Ruby defines this, I couldn't find a public API quickly, RubyVM.stat.class_serial is the counter I think
static const rb_callable_method_entry_t  *cache_func; // The cached method
if (global_method_state != cache_global_method_state || cache_type_id != type_id(x))
{
    cache_global_method_state = global_method_state;
    cache_type_id = type_id(x);
    cache_func = rb_search_method_entry(x, ID_ADD);
}
// y = cache_func(x, const_5);
VALUE y = rb_vm_call_kw(execution_context, x, ID_ADD, 1, &const_5, cache_func, RB_NO_KEYWORDS);

Juliean said:
Or is there not even that much difference?

There is some difference, but an interpreted language is always fairly slow to start with. And consider with dynamic types it can't know at compile time if “x + y” or “x.foo(a, b)” what x is, if it is a native function or a script function, if x even has such a method, if (a, b) is the correct param count, what the param types are, etc.

If you just push the argument count then a series of “PyObject” or “VALUE” style generic stuff onto the stack so the , that is basically what say the Python or Ruby is doing, the only real difference I see is them using a PyObject *tuple or int argc, VALUE *argv and letting the callee figure out if it is a valid call, while a native C function generally expects the caller to set the registers and stack up right.

Seems more like the sort of thing a JIT optimiser might do (and I haven't really looked at what say JS, Java or .NET do when they encounter either some sort of built in native function, or an extension/dllimport type thing), and JIT's I believe will create something more like statically typed code, and throw it out if something changes one of the type assumptions.

SyncViews said:
There is some difference, but an interpreted language is always fairly slow to start with. And consider with dynamic types it can't know at compile time if “x + y” or “x.foo(a, b)” what x is, if it is a native function or a script function, if x even has such a method, if (a, b) is the correct param count, what the param types are, etc.

The really bad thing in my case was that its not a problem about not knowing enough about the types upfront. I do have a very statically typed language, functions are all known at point of compilation (with the support for virtual calls). There are “wildcards” which are like generic/template-functions that only can have one generic argument. So there is no inherent slowness to how my language is represented, its just a matter of choosing an efficient representation format; and thats what I'm going for ?

I mean, I did go through various iterations of “using the same representation thats used for displaying the node-graph for execution” to “having a basic compiler that will compact from representable format to something more dense and optimized”. Though I'm still nowhere near the performance I want to have. Especially in debug its really bad, slower by like the factor of 75-100x (I'm not kidding). I don't have any direct comparisons to how other, (interpreted) languages would fair with what I'm doing, but I'm assuming you are supposed to get more than mid-complex player-controller script before your frametime goes to 3ms in debug builds ?

SyncViews said:
Seems more like the sort of thing a JIT optimiser might do (and I haven't really looked at what say JS, Java or .NET do when they encounter either some sort of built in native function, or an extension/dllimport type thing), and JIT's I believe will create something more like statically typed code, and throw it out if something changes one of the type assumptions.

Well, I'll be looking at JIT-ing my own bytecode to get even further; but only after everything is done. This transformation to bytecode alone will easily take weeks/months; as by this point my language is pretty far evolved so I have to lots of catch-up to get everything working again :D

Juliean said:
Users must be free to register eigther global functions. lambdas or member functions w/o being required to write wrappers (though template-metaprocessing as well as reflection can obviously be used to generate the necessary code)

When you write ‘users’ who are really? It seems to me you are referring to programmers using C++ (that is, engine programmers).

It happens I've done something similar before. Is your need to make a C++ function available to script?

Krohm said:
When you write ‘users’ who are really? It seems to me you are referring to programmers using C++ (that is, engine programmers). It happens I've done something similar before. Is your need to make a C++ function available to script?

I mean Engine and Plugin-Developers using C++, but at the moment what I really mean is just “me” :D

And yes, I want to make c++-functions available to script; without having to write a custom wrapper. Been there, done that, didn't like it, being able to write 1 call to register the function is so much sweeter. Also since thats already existing, I don't want to change it even further; but what I now want is optimally being able to replace my template-meta"compiler" with a bytecode-generator that'll setup the necessary state and call the function-pointer directly.

I'm not sure what sort of scripting architecture, scripting features and uses of scripting you are aiming for.

• Open-ended, high level scripts (like writing a game in Python, using your game engine and arbitrary other libraries as extension modules) or specific purpose “small” scripts (like scripting particle systems by describing the behaviour of every particle and particle source type with something like BulletML)?
  Both cases seem to imply exposing to scripts your game engine with a carefully designed API (respectively a neat and comprehensive interface for everything or small “windows” of restricted primitives), with no need to add arbitrary functions particularly often.
• To what extent your scripts can be processed and compiled ahead of time? Generating C or C++ code that can be compiled and amalgamated into your game (possibly as a DLL to allow loading it more dynamically) would be both simple and very high performance: there need to be specific reasons (for example, loading scripts from mods and levels) to justify slower and more complex compromises.
• As compromises go, turning a visual scripting graph into bytecode and bytecode into machine code is potentially high performance (not too much if you cut corners) but at the masochistic end of the complexity and difficulty spectrum.
  What benefits of decoupling the front end and back end of your scripting system with a bytecode representation apply to your needs? Is there a cheaper and simpler way to obtain those benefits?
  For example, a large and sophisticated node graph seems a good fit for generating code in an established high level language, and leveraging infrastructure for that language: C or C++ (or D, Rust, Go…) for direct embedding as suggested above, Lua or Javascript or other languages with ready made high performance embedded interpreters (probably with efficient JIT compilation), GPU shaders (maybe in part).

LorenzoGatti said:
Open-ended, high level scripts (like writing a game in Python, using your game engine and arbitrary other libraries as extension modules) or specific purpose “small” scripts (like scripting particle systems by describing the behaviour of every particle and particle source type with something like BulletML)? Both cases seem to imply exposing to scripts your game engine with a carefully designed API (respectively a neat and comprehensive interface for everything or small “windows” of restricted primitives), with no need to add arbitrary functions particularly often.

In short, I'm pretty much going to blueprint-route, even with a bit more focus on using the visual scripts for gameplay. The c++-side of the engine/plugin-architectur is mainly used to represent everything “generic" in a way. I'm lacking the proper word for it, but think like this: Everything thats related to implemented say the rendering, collision-detection, entity/game-object respresentation is done in C++, while everything related to gameplay (Player/Enemy-behaviour, cutscenes, spells, …) is done in my visual scripting. So I do need a good bit of the engine exposed, and as I said I've already been through having to manually write wrappers and really felt how it impaired productivity (even with using reflection ie. to expose all component-properties w/o having to define them myself).

LorenzoGatti said:
To what extent your scripts can be processed and compiled ahead of time? Generating C or C++ code that can be compiled and amalgamated into your game (possibly as a DLL to allow loading it more dynamically) would be both simple and very high performance: there need to be specific reasons (for example, loading scripts from mods and levels) to justify slower and more complex compromises.

While it would be totally ok to precompile all scripts for builds; I want to have quick iteration times while working in the editor. Lets just say, that my own “compiler” so far took like 0.1s, and I want to keep it that way. So if I can't use a C++ backend for the editor, I'll have to have another backend and I don't want to deal with having multiple of those. Thus, I'm searching for a middle-ground, which I belive can be achieved with a bytecode-compile language (I already made some great progress since yesterday and got basic stuff running, so I think that'll work). There's a few other reasons why I disregarded direct low-level code gen; mainly that my language is designed to allow support coroutine-like stopping/resuming with concurrently running code naturally, so having to express this in C++ would make the code-generation itself rather complicated I belive.

LorenzoGatti said:
As compromises go, turning a visual scripting graph into bytecode and bytecode into machine code is potentially high performance (not too much if you cut corners) but at the masochistic end of the complexity and difficulty spectrum.

Uh, I don't mind that. The whole scope of my engine is on the larger end of the spectrum anyways, so it doesn't bother me having to do something thats also difficult ?
And you know, with what I got since yesterday I'm even more confident that its going to work out. With maybe 8 hours invested, I got a basic compiler+bytecode-interpreter, hooked up to my current graph+typesystem, with basic integer-arithmetic and local variables working. There's going to be some things that I'll have to wrap my head around, but its all going in a direction I very much like. And truthfully, so far the code is way less complicated than what I had before.

LorenzoGatti said:
What benefits of decoupling the front end and back end of your scripting system with a bytecode representation apply to your needs? Is there a cheaper and simpler way to obtain those benefits? For example, a large and sophisticated node graph seems a good fit for generating code in an established high level language, and leveraging infrastructure for that language: C or C++ (or D, Rust, Go…) for direct embedding as suggested above, Lua or Javascript or other languages with ready made high performance embedded interpreters (probably with efficient JIT compilation), GPU shaders (maybe in part).

I'm not gonna use different languages. Its really only a subjective thing, I'm not using any external libraries for the engine (except for the bare basics of DX/WinAPI). No reason thats grounded in productivity, just how I like to do things.
I could do C++-generation, but mainly for the reasons listed above I'm not going to do it.

(Don't get me wrong, I really appreciate the suggestions ? But for one reason or another going that way is what I wanna be doing ? )

Considering how complex your project is becoming, I would second @lorenzogatti suggestion. Consider you might be going beyond the goal of something useful as a personal project. Keep in perspective UnrealScript as far as I understood is no more a thing, it seems like the middle ground isn't good enough.

I took the chance to check how I did it and considering it's about 10 years late and predating the introduction of lambdas as far as I can remember I doubt sharing its details would be useful. Yet I will try to distillate it a bit in case it sparks you an idea.

My scripting was based on walking an AST. Every time you wrote { … } (in script) you would create an InstructionBlock. Each instance of this class by default was a list of VM instructions. This was the standard implementation. For calls to native C++ I just (remember it's before lambdas) defined a new class with its Execution call implemented as a marshalling code to the C++ function. I did this for several dozens of calls.

In modern C++ I guess the special implementations would be lambdas, then writing a marshalling function which could force the type match at compile time. It wouldn't even have to be parameter-pack driven, for 6-8 parameters it could be possible to just write the variations by hand. The result of this function call would be the table entry for your compiler/interpreter.

I'd like to see how this could be done today. What is the specific issue stopping you? Your posts seem to describe mostly the desiderata than the roadblocks you are encountering.

Krohm said:
Considering how complex your project is becoming, I would second @lorenzogatti suggestion. Consider you might be going beyond the goal of something useful as a personal project. Keep in perspective UnrealScript as far as I understood is no more a thing, it seems like the middle ground isn't good enough.

I mean, this is kind of hard to argue based on just what I can tell you w/o showing details (I mean, I could, I'm always considering to continue writing my blog but I'm too lazy/too far busy with developing)… but its actually that other way around. Since I started this project about 8 years ago, for most of the time it has been something way too complex to ever be useful. While I was still enjoying working with my own engine simply because its fun for me, I always knew that it was not really productive. But ever since I made some drastic changes a few years ago (which granted took a long time itself), its actually became quite managable and useful. With the occasional breaks when I “have to” do stuff like that now, but I'm really liking the way the project is going and don't think its growing out of scope or anything ?

EDIT: Or, in simpler words, adding this bytecode-"language" is not whats going to be making the project too complex, trust me :D

My scripting was based on walking an AST. Every time you wrote { … } (in script) you would create an InstructionBlock. Each instance of this class by default was a list of VM instructions. This was the standard implementation. For calls to native C++ I just (remember it's before lambdas) defined a new class with its Execution call implemented as a marshalling code to the C++ function. I did this for several dozens of calls.

In modern C++ I guess the special implementations would be lambdas, then writing a marshalling function which could force the type match at compile time. It wouldn't even have to be parameter-pack driven, for 6-8 parameters it could be possible to just write the variations by hand. The result of this function call would be the table entry for your compiler/interpreter.

Well, in modern C++ this is actually very very easy thanks to (variadic) templates and fold expressions. You can pretty much write a function which does this

template<typename Functor, typename ReturnT, typename... Args>
void callNative(Functor f, Stack&amp;amp;amp; stack)
{
	if constexpr (!std::is_void_v<ReturnT>)
		stack.Push(f(getArg<Args>(stack)...));
	else
		f(getArg<Args>(stack)...);
}

You just have to use a few more templates to get the functions return/arg-packs and also have to store the function-poiner (yeah lambdas do help there), but in essence thats what I have already been doing. (the real code is a lot more complicated as I allow introducing new types for the function-signature that must not even be recognized by the type system, ie. being able to use “Sprite&” which will act as an entity in the graph but automatically fetch the component so I don't have to do that in the function. What can I say, I love automation and hate writing repetetive code :D )

Krohm said:
I'd like to see how this could be done today. What is the specific issue stopping you? Your posts seem to describe mostly the desiderata than the roadblocks you are encountering.

There's nothing from stopping me from doing that, I'm already in the process ? My question originally aimed at getting rid of this extra-step and in
how I would manage to call the function directly without any wrapper. Towards that question after @frob s answer, I'm still not sure how I would manage to setup the stack/registers for the function call while executing my own code to perform the setup.
But I'm going to stick for the wrapper-approach for now, as it means way less work and really only means minimal modification of my function-binding approach. So at this point were are probably just having a friendly discussion/exchange of ideas about the topic and not a specific road-bump I'm facing; which is fine by me ?

Juliean said:
My question originally aimed at getting rid of this extra-step and in how I would manage to call the function directly without any wrapper.

You could maybe look at what Python ctypes module does to call a native DLL/so function, it's not a bytecode feature so last I checked is a lot slower than a C extension taking the PyObject* array/tuple and dealing with that itself, but maybe something to learn.

Presumably they have a way in there to setup the stack such that once done with all the arguments it is ready to just do a CALL. I'd guess some manual stack manipulation which would need consideration for C++ exceptions, destructors, etc.

Do you have a common base type like PyObject or VALUE?

Not sure any interpreter I have used does this though as its primary way to call native functions, looking up Lua and JS (napi) they do something similar to as well with lua_State and napi_callback_info, basically relying on the native compiled code to unpack some sort of array-like argument into what it wants.

SyncViews said:
You could maybe look at what Python ctypes module does to call a native DLL/so function, it's not a bytecode feature so last I checked is a lot slower than a C extension taking the PyObject* array/tuple and dealing with that itself, but maybe something to learn. Presumably they have a way in there to setup the stack such that once done with all the arguments it is ready to just do a CALL. I'd guess some manual stack manipulation which would need consideration for C++ exceptions, destructors, etc.

Thanks, I'll have a look at it.

SyncViews said:
Do you have a common base type like PyObject or VALUE?

Somewhat. I have a class called “Variable” which can hold any value known to the type-system, with the caveat that its expensive when using in conjunction with arrays/strings/objects as it has to allocate/refcount those, so I'd rather not use that during interpreting. I'm currently using it to dictate what byte-code gets generated though. I do have a lighter variant called “VariableView” (its what string_view is to string), but obviously this requires the value to be stored somewhere else so at that point I might as well use the value directly from stack.

My idea for handling this is to go for a cdecl-style approach where I push all function-arguments on the stack with a default-convention (by-value for most primitives, by ref for everything costly) and then have my wrapper pop them off in inverse order. This doesn't require me to even have/use a base/wrapper-class for the values.

SyncViews said:
Not sure any interpreter I have used does this though as its primary way to call native functions, looking up Lua and JS (napi) they do something similar to as well with lua_State and napi_callback_info, basically relying on the native compiled code to unpack some sort of array-like argument into what it wants.

Yeah, this ought to be be good enough for me as well. I'm expecting drastic speedups anyways, as simply using a stack gets rid of so many dynamic allocations, indirections, lookups etc… And when I have everything up and running again I can still try to do wrapper-less native calls later.

Juliean said:
You just have to use a few more templates to get the functions return/arg-packs and also have to store the function-poiner (yeah lambdas do help there), but in essence thats what I have already been doing. (the real code is a lot more complicated as I allow introducing new types for the function-signature that must not even be recognized by the type system, ie. being able to use “Sprite&” which will act as an entity in the graph but automatically fetch the component so I don't have to do that in the function. What can I say, I love automation and hate writing repetetive code :D )

You might find useful to think in terms of the “backing store” of your script-object.

Ok, some more thinking. I think your getArg is pretty much what I would have developed as Resolver if the project could get along.

An aside, how does the parameter pack look like in case you need to debug it?

Krohm said:
You might find useful to think in terms of the “backing store” of your script-object.

Could you elaborate a bit on what you mean with that? Not sure I can follow?

Krohm said:
An aside, how does the parameter pack look like in case you need to debug it?

Also not sure I fully understand the question. The template-parameters are just 1:1 mapping of the function signature, so a specialisation in debugging might look like that:

callNative<XXX, int, float, const Object*>(...); 

Is that what you meant?

Did you already consider using OpCodes instead of byte-code to run your scripting language or even to embedd C# CLR into your engine instead of writing your own maybe more limited “CLR” and also provide a well-known language at the same time to your users?

Once I looked through the LUA Runtime code at GitHub, it looked very easy to create your own processor using OpCodes and a well designed combination of switch and goto statements along some registers and a stack that mimic the behavior of a real CPU-Core. Sure it's speed might be still limited to some point in opposite to real machine code but it is fast up to some point and you could design your “compiler” to optimise calls on OpCode level. Another pro feature would be to allow users to write “Assembly” in the same way C# does

Shaarigan said:
Did you already consider using OpCodes instead of byte-code to run your scripting language or even to embedd C# CLR into your engine instead of writing your own maybe more limited “CLR” and also provide a well-known language at the same time to your users?

Hold on, now. Thats already what I'm doing (=writing my own OpCodes), so now I'm confused. I assumed that thats what bytecode is/does. I'm having my own set of OpCodes which are run inside a loop via switch in some sort of VM/interpreter. Did I use the wrong term for that? Everything I searched for bytecode suggested that this is what that is. So what I'm doing is something like this:

enum class OpCodes : uint8_t
{
	PushIntConst,
	AddInt
};

// inside the VM
const auto opCode = bytecodeStream.Read<OpCodes>();
switch (opCode)
{
case OpCodes::PushIntConst:
{
	const auto constant = bytecodeStream.Read<int>();
	stack.Push(constant);
	break;
}
case OpCodes::AddInt:
{
	const auto value1 = stack.Pop<int>();
	const auto value2 = stack.Pop<int>();
	
	stack.Push(value1 + value2);
	break;
}
}

Shaarigan said:
or even to embedd C# CLR into your engine instead of writing your own maybe more limited “CLR” and also provide a well-known language at the same time to your users?

I considered that, but again for a few reasons I'm not going to use external code in this project. Aside from a few technical points that I assume would be more difficult if I didn't have 100% control over execution, its mostly just for the fact that “I don't use external code in this project” :D For no particular reason other than that I like having full control and enjoy learning/writing all aspects myself.

Shaarigan said:
Sure it's speed might be still limited to some point in opposite to real machine code but it is fast up to some point and you could design your “compiler” to optimise calls on OpCode level.

Yeah, I'm already getting good optimization-opportunities even on the level of control-flow (discarding unused returns, scoping local variables). And again, I'm already being to get such a huge improvement in terms of performance over my old system that it doesn't really matter that its still slower than native code (at which point I could turn to JIT if I really needed to).

Shaarigan said:
Another pro feature would be to allow users to write “Assembly” in the same way C# does

Yes, I also consider this a good addition. The current backend offered a lot of control over flow of execution to the plugins/users, so I definately wanna keep that (only now on compilation-level rather than runtime).