Original Post
Hello! I know the topic of JITs have been up on the table before, but I couldn't find any details concerning actually implementing one. This is mostly directed to you, Andreas, but as this is information that will interest others wanting to add a JIT/AOT compilation for their processor of choice, I choose to post this here (and in English!) rather than emailing you directly. I started playing around with AOT compiling for AS as a fun sparetime project to get to know the ARM architecture a bit better. But before I get too deep into the actual implementation (and would have to rewrite the whole shebang to fit with your ideas) I figured I'd ask you what your thoughts are when it comes to JIT/AOT compiling and how it all would hook into the AngelScript engine. I think writing a JIT/AOT compiler is a big project to take on unless you can split it out into chunks and take it one step at a time. As such, one of the features I'm interested in is to be able to mix native machine code with AS bytecode and switch between them. Thus I suggest a new AS bytecode instruction is implemented "JIT" (or something else, but I'll use that name from now on). I've not thought this through 100% so the actual implementation will likely change during the course of the work on this. But my current idea is that the "JIT" bytecode would:
- if JIT is enabled:
- save l_bc, l_sp, l_fp to the context
- Call "ExecuteJIT(this, machineCode)",
where "this" is the asCContext and "machineCode" is a pointer to
the array containing the native code to execute (where this code
buffer is actually stored is to be decided...)
- load l_bc_l_sp,l_fp from the context
- else:
- nop The JIT/AOT compiler does not replace the implemented bytecodes but rather just injects the JIT bytecode before the section that it does support natively (see further down for an actual example). This allows us to disable the native code at runtime and treat it as a nop. Otherwise, as exactly what happens to l_bc can be unknown at compile time, l_bc in the context must be updated by the machineCode to skip the AS bytecodes and their arguments that were implemented by the JIT/AOT. It could also potentially work when implementing jumps/calls/suspend in native code, although exactly how the return/resume would work I've not figured out yet. But figuring out how to really compile calls into native code is IMO a separate issue as there are plenty of bytecodes to implement that would not be affected no matter if script calls are actually done within bytecode or machine code, so I've not really spent any time thinking about this. In any case, having this ability to mix native and AS bytecode, would allow me to implement one AS bytecode at a time, would not break as soon as a new bytecode is added (although if it's a frequently used one it would obviously hurt perf), would still allow for co-routines and would allow you to disable the JIT at runtime if you want to step through the AS bytecode with a debugger. So with this new bytecode instruction in mind, lets turn our attention to which part of the code would emit it. As I'm only in the very early testing phase, I added a call into my AStoARM compiler between Optimize and ResolveJumpAddresses in asCByteCode's Finalize method. This is easy and straight forward, however the biggest problem I can think of offhand is that this will mess with crossplatform loading and saving of the bytecode (when this becomes available). Surely code saved on one platform and loaded on another would want to have the JIT/AOT step redone for maximum perf. The other alternative I can think of is to save/load a version that does not have any JIT instructions and do the native compilation step as a post process operation. However it's important to note that all PC relative instructions would have to be patched when the new instruction is inserted between for example a jump and its destination. As I don't want the JIT to break once a new PC relative instruction is added, I suggest a common interface for adding the JIT instruction that will take care of this patching so that JIT don't have to know how to do it itself. The problem I can think of with this solution is that it would have to shuffle memory around when inserting the JIT instruction, although it could keep a buffer around with the JIT instructions to insert and not actually do it until the compiler says it is done. This or something similiar is probably a must for writing an actual JIT rather than an AOT. Any other suggestions? For the second alternative, where would the native compiler hook in? Another interface needed is one for taking ownership of the buffer that is to contain the native machine code. I suggest that the JIT instruction's parameter is an index that identifies which buffer to use, although exactly whom the JIT/AOT compiler should talk to to allocate the buffer and get the index I don't know. The script function, the module, the engine? What do you suggest? I'm sure there was something I forgot, but this has been a long post already. What do you think? Any alternative solutions you've been thinking of? Disregarding the problems with having the compilation done before ResolveJumpAddresses, and native machinecode buffer management (ie I only have one buffer), as a test I have implemented two of AS's bytecode instructions MULi and ADDi. The AngelScript source code used in the test was: int TestBasic(int a, int b, int c)
{
return a + b * c;
} Which compiles into: Temps: 1
0 0 * PUSH 1
- 3,5 -
1 1 * SUSPEND
2 0 JIT 0 ; Notice the new instruction
3 1 * MULi v1, v-1, v-2
5 1 * ADDi v1, v0, v1
7 1 * CpyVtoR4 v1
8 1 * 0:
8 0 * RET 3
With the native code for the JIT-supported section (from MULi to (including) ADDi) being:
0xe92d4030 stmdb sp!, {r4, r5, lr} ; Prologue
; Save return pointer and scratched
; registers we are required to save on
; the (native) stack
0xe5902028 ldr r2, [r0, #0x28] ; Load AS's stack frame pointer from the asCContext
0xe5921004 ldr r1, [r2, #0x4] ; Load v-1
0xe5923008 ldr r3, [r2, #0x8] ; Load v-2
0xe0040391 mul r4, r1, r3 ; Perform the multiplication
0xe5925000 ldr r5, [r2] ; Load v0
0xe0854004 add r4, r5, r4 ; Perform the add
; Epilogue
; Here at this code we notice that we don't support the next AS bytecode
; As such we need to flush any changed data and return from the code
;
; If on the other hand the next opcode would also have been supported, we
; would have continued executing even further and would not write any data
; back until it is needed, either by running out of native registers to
; use, or when it is time to exit
0xe5024004 str r4, [r2, #-0x4] ; Save v1 to the stack as it changed
0xe5904020 ldr r4, [r0, #0x20] ; Load byteCode pointer from the context
0xe2844014 add r4, r4, #0x14 ; Add the number of AS bytecode data to
; skip (includes the JIT instruction)
0xe5804020 str r4, [r0, #0x20] ; Save back to the context
0xe8bd8030 ldmia sp!, {r4, r5, pc} ; Restore scratched registers we had to
; save (according to the ARM calling
; convention) and return (by setting
; pc = lr)
This piece of code has been verifed as working. Obviously this is not optimized assembly code for the ARM platform, as multiple sequenceive loads could be done with a single instruction, and the multiply and add too. But optimizing the generated native code is a completely different topic.