Original Post
I posted this on lua mailing list but got no replies, so i am trying here. First a background of the application (can’t tell the full details due to NDA requirements). We needed an embedded visual studio debugger for lua since we are currently using it for our game. Since we register a lot of things in the lua_State therefore for our debugger to work properly, it was required that the application load the debugger dll and provide it with the VM instance to be debugged. A seperate COM dll in the form of a visual studio addin provides the user-interface. Both dlls run in seperate processes, one in visual studio (the addin) and the other in the host application process. The debugger dll communicates with the visual studio addin via an IPC mechanism. The debugger dll basically halts the application in the hook procedure and communicates with the addin sending values of requested global/local variables back to the addin (providing a watch window), amongst other functionality like step in, step out etc. Our game scripts are nested i.e. one scripts calls another (via a registered C function), which in turn calls another and so on (upto a certain level, ofcourse). The problem arrives when the user requests a global variable that does not exist and lua returns a nil value when I call lua_getglobal() in the debugger dll. After this several executions of these nested scripts causes lua to crash (i can send the call stack if required). I have repeatedly checked the integrity of the stack and am positive that nothing outrageous is being done to it. If the global variable to be queried does exist in the global table, then the application runs as desired. If there are no nested scripts (i.e. no lua_dofile s being called from within a script) then all sorts of scenarios work perfectly. I am frankly out of ideas and am convinced that this is a lua runtime bug! Is there a way to catch lua_getglobal(...) errors (I read from the archives that older versions had fallback procedures, but they seemed to have disappeared from lua 5.0). I welcome suggestions and comments. The deadline of the project is looming and I need your help. Any help is highly appreciated.