In asCScriptEngine::RefCastObject, the void opCast(?&out) fallback calls ctx->Prepare(universalCastFunc) on the result of RequestContext() without checking it for null:
// One last chance if the object has a void opCast(?&out) behaviour
if( universalCastFunc )
{
// TODO: Add proper error handling
asIScriptContext *ctx = RequestContext();
ctx->Prepare(universalCastFunc);RequestContext() returns whatever the application's callback from SetContextCallbacks gives it, and that can be null (for example a pool that is exhausted). The library's other callers of RequestContext() check for null (as_module.cpp, as_scriptobject.cpp, as_scriptengine.cpp), but this one doesn't. Besides the application calling RefCastObject directly, the ref add-on's cast also goes through it.
Repro
AngelScript 2.39.0 WIP (master at c9e74722), MSVC 19.44 (VS 2022), x64, static library from projects/cmake with default options.
#include <angelscript.h>
#include <cstdio>
struct A {};
struct B {};
// void A::opCast(?&out). Sets the output handle to null; never reached in the failing case.
void A_opCast(asIScriptGeneric *gen)
{
*static_cast<void**>(gen->GetArgAddress(0)) = 0;
}
// A context pool that has nothing to give.
asIScriptContext *RequestContext(asIScriptEngine *engine, void *)
{
// Workaround: never return null, fall back to a new context instead
// return engine->CreateContext();
return 0;
}
void ReturnContext(asIScriptEngine *, asIScriptContext *ctx, void *) { ctx->Release(); }
int main()
{
asIScriptEngine *engine = asCreateScriptEngine();
engine->RegisterObjectType("A", 0, asOBJ_REF | asOBJ_NOCOUNT);
engine->RegisterObjectType("B", 0, asOBJ_REF | asOBJ_NOCOUNT);
engine->RegisterObjectMethod("A", "void opCast(?&out)", asFUNCTION(A_opCast), asCALL_GENERIC);
engine->SetContextCallbacks(RequestContext, ReturnContext, 0);
A a;
void *result = 0;
printf("calling RefCastObject\n");
fflush(stdout);
int r = engine->RefCastObject(&a, engine->GetTypeInfoByName("A"), engine->GetTypeInfoByName("B"), &result);
printf("RefCastObject returned %d, result = %p\n", r, result);
engine->ShutDownAndRelease();
return 0;
}Expected
RefCastObject returns a negative error code and leaves result null.
Actual
a log of "calling RefCastObject" and then an access violation (exit code 0xC0000005) at ctx->Prepare(universalCastFunc), in both Release and Debug.