DirectX Hooking

Some games will be easier to hook than others. It ultimately depends on how they initiate the connection to the DirectX DLLs.

To get you started on the subject, there's a DX hooking sample, but it's written in C++. Still, it should be enough to get you started on understanding what things you need to be doing.

As for VB6, I'm not sure how possible that will be.

Are you talking about Direct3D or DirectDraw?

I've never had to hook a DirectX program, but have some more general experience. I can't think of a more direct manner than to inject a hook for IDirect3DDevice9::Present (or IDirectDrawSurface::Flip, but I'll assume the former), and grab the front-buffer.

Perhaps somebody could suggest a simpler method, but I'm sure that what I'm about to describe will work. It does feel a little like a nut-cracking sledgehammer though.

For a simple local hook like this, you should inject some code into the target process, locate the Present method and hook it with a wrapper of your own. For the sake of brevity I won't discuss it, but I suggest you put an equal amount of effort and care into unhooking gracefully.

There are many variations on the theme, but I've found DLL-injection with IAT-hooking to be the most flexible and effective. There is a reasonable amount of low-level dword-pokery involved, but it's well within the capabilities of VB6, provided you're prepared to copy-paste all those Win32 API declarations [rolleyes].

You'll need to write a separate DLL containing the code that needs to execute inside the target process. I suggest you do this in C++, as it's the best tool for the job. Feasibly, you could write the code straight into the main (VB6) application and remotely load the exe as a secondary module, but it seems a little perverse.

1. Find the target process. If it's a windowed application, I've found that a sequence of calls to FindWindow, GetWindowThreadProcessId, OpenProcess does the trick. Use Spy++ or something similar to find the window class's name, and open the process with PROCESS_ALL_ACCESS.

2. Inject your DLL. I use the second technique described in the article I linked to above, which is very clever. That is, VirtualAllocEx a buffer into the target process and WriteProcessMemory the path-name of the DLL to this buffer (which you can determine using LoadLibraryW and GetModuleFileNameW). Next, trick the target process into loading your DLL in by calling CreateRemoteThread with the address of LoadLibraryW (which will be the same as in your process) and a pointer to the injected DLL path-name as the parameters.

3. Execute the injected code. You could use the CreateRemoteThread method like before, to execute a function at any time, or you can just rely on DllMain to do everything for you.

The idea is to compile a proxy-Present function into your DLL. This is made a little bit more difficult by the fact that it is a class member-function (and hence a thiscall). You'll need to create a function that reroutes the execution to where it needs to go (so the back-buffer is actually displayed), but do all your own business at the same time. Something like this:



As it stands, if this function is called in place of IDirect3DDevice9::Present, it will perform exactly the same way. The point is that you are now free to do whatever you like with the parameters. But before this is of any use, you'll need to:

5. Install the hook. This should be done in the injected DllMain. The most reliable method is to patch directly over the beginning of the DLL function, but this is really messy. If you're into that sort of sadistic business, writing a 'JMP PresentHook' at the address of IDirect3DDevice9::Present will do. Otherwise, a less soul-corroding method is to use an IAT hook:

Provided you know that Present will be called from the exe, it suffices to reroute the executable's import address table (IAT). Call GetModuleHandle, passing NULL as the module name. This will return the base address of the target exe. A call to VirtualProtect will give you read-access to the PE header. Now you can use a little pointer arithmetic (or the IMAGE_NT_HEADERS32 struct) to find the location and size of the IAT. Treating this chunk of memory as an array of 32-bit unsigned longs, perform a linear search for the address of IDirect3DDevice9::Present (which you can get with GetProcAddress). Once you've found it, simply overwrite it with the address of your hook.

Now, whenever the exe attempts to call Present, it will unknowingly find its way through your hook.

6. Take the screenshot. This should be pretty straightforward. Before your hook passes control to the real Present method, it should call CreateOffscreenPlainSurface, followed by IDirect3DDevice9::GetFrontBufferData. D3DXSaveSurfaceToFile will help you get the screenshot to disk.

And that's it; you're done. The process felt a whole lot shorter in my head [razz].

Edit: Stumbled upon this old post and spotted an error in my __thiscall emulation.

[Edited by - TheAdmiral on November 24, 2007 1:37:16 PM]

Well, it sucks that you don't know C++. In pure managed code its impossible to do what you want.

- HOWEVER -

You COULD write a short hooking DLL in C++, or someone else could, and make VB6 do the rest of the application. Someone recently tried to make a DWM clone for Windows XP. Though its kind of... well... useless in principal (It doesn't work right in most cases, and FastAero which does is not released yet.) it does in fact do the hooking correctly (Just the developer had the wrong idea when he made it) and funnily enough, he happened to use VB6 as a GUI and interface to a DLL (And that DLL connects to the DLL that does the hooking.)

Just try to understand the code. If you can't try ignoring what you don't understand and ask questions when you fail. Remember, DirectX and OpenGL work in C not VB so its not possible to even use VB6 code to swipe data (safely, atleast.)

http://www.aeroxp.org/board/index.php?showtopic=5556

Since you need to be a member, i uploaded it for you.
http://www.megaupload.com/?d=WNMSM9ZL

Quote:

Original post by SSkillZ First and Second step I think I can do. Third step ill rely on DllMain Fouth Step is missing...

Yeah. The fourth step was to have a beer [rolleyes].

Quote:

And whats the diffrence between Third and Fifth step?

The third step loads your DLL into the target process. DllMain is called implicitly, and so step 5 is triggered. Conceptually, though, step 5 is a completely different operation. Injecting the DLL is done from your application, whereas installing the hook is done by the DLL in the target process's address-space.
Putting it another way, without (3), (5) can't happen. Without (5), your injected DLL never gets to do its thing.

Quote:

Those two functions can be called in DLLmain no?

You mean the functions responsible for taking the screenshot? Unless you can think of a clever way to acquire a pointer to the graphics device, no.

Quote:

The dll will have a DLLmain function which exectues on DLL load and takes the screenshot and maybe save it to a file.

This won't work for two reasons. First, you can't call GetFrontBufferData without a pointer to the device, and second, even if you could call it, you wouldn't necessarily get a screenshot, as the scene may not have been rendered. On this note, the hooking function should rather look like:

Quote:

how its done with OpenGL?

It's the same idea, except you don't hook IDirect3DDevice9::Present, but glFlush/glFinish.

Admiral

This generic DLL hook is something I've had to use more than a couple of times. With a little foresight, I would have written a reusable hooking class. I have an exam tomorrow, so I hope this (very briefly tested) botch-job serves as something of a substitute:



It should be sufficiently general for your needs. For example;

InstallHookByName("Kernel32.dll", "CreateFileA", "User32.dll", "BeginPaint");

would overwrite the beginning of Kernel32!CreateFileA with a 'goto User32.BeginPaint'. Of course, this would promptly cause a crash on attempted creation of the file, but you get the idea. If any of the modules or functions are not found, the function fails, returning 0.

To only tricky bit to follow is generation of the relative JMP instruction - you'd need to know a little bit about x86 to fully understand it. I haven't covered removal of the hook, but it should be similarly done by saving the five bytes being patched and putting them back in place. Ideally, you'd do this in the screenshot-taking function, for a one-shot hook.

I don't need to to tell you that this is (necessarily) very hacky and won't port to anything other than 32-bit Windows (XP or 2000).

Admiral

Quote:

Original post by SSkillZ Still this is really appreciated but I Don't know C++ so I can't continue from here, Please help.

Sorry, I just don't think VB6 has the tools to get the job done. Even if it did, I don't really think anyone here knows how to do it.

Quote:

Original post by sirob Quote: Original post by SSkillZ Still this is really appreciated but I Don't know C++ so I can't continue from here, Please help. Sorry, I just don't think VB6 has the tools to get the job done. Even if it did, I don't really think anyone here knows how to do it.

It does - I've had the displeasure of doing it before. VB6 is pretty capable of doing this low-level stuff - more so than VB.NET in my experience, but it's a real pain to get things done. In particular, any API functions being used need to be translated to VB's typing and explicitly declared. In this situation, that's a whole lot of declarations. Perhaps more cripplingly, there is no support for unsigned 32-bit integers, so dealing with DWORDS is a real pain, as you have to explicitly poke around with two's complement checks and bit-shifting.

Quote:

Still this is really appreciated but I Don't know C++ so I can't continue from here, Please help.

Huh? That's not exactly meeting us half-way. If you can program, you can at least understand C++. I'm not prepared to rewrite the code in VB6, I'm afraid, and I strongly discourage you from doing so. If you know VB, you should be able to pick up C++ with relative ease. If you have any specific problems, we'll be more than happy to help, but nobody is going to do the work for you.

Admiral

You could feasibly have some success with this rudimentary system:

Send some Windows messages to the game, to force focus, wait a little while (until it has painted), grab a desktop screenshot as described here (in VB6) and try to manually crop the image using GetClientRect. I'm not sure how reliable this would be. There may be a better way to query the OS for the window's DC, but I don't know of one.

Admiral

In the same way that many counter-strike hacks work, couldn't you create fake DirectX and OpenGL DLLs and place them in the program's folder so that the program loads your DLLs instead? Your DLL would forward all the API calls to the real DLLs, except for Present which might do some extra work like spit out the contents of the back buffer. Maybe you should look into how these hacks work.

Quote:

Original post by SSkillZ I don't like the Idea of a anticheat tool using "hacking" methods.

The method is unimportant --- it's how you use it. I'm just trying to help you get the job done.

Quote:

Original post by SSkillZ Thanks, but anyway the anticheat VAC which is built in the game checks if the files have been modified because so many cheats as you said used that method for hacking... Thats why I don't like hacking the game files or the game itself.

Well a hook is about as hacky as it gets. In particular, a DLL hook will show up as an instant red-flag to VAC. If you want a non-intrusive method that can't be picked up by any anti-hack system then you'll need to do everything externally.

As far as I know, no such system will check the integrity of the API DLLs, as there are way too many versions to keep track of, and updates are frequent. For this reason, creating a wrapper DLL may be your best option. The biggest concern I have with this is that you'd need the target program to load up after your program has installed the fake DLLs.

Admiral

Windows services won't offer any more useful control than a regular user-mode application. The only reason we like to use DLLs is that it allows us to execute code inside the target process's address-space.

You should have mentioned Aero's role in the problem sooner - it changes everything [rolleyes]. All the code I've written in this thread has been 32-bit, so don't expect any of it to port nicely to Vista.
After a little Googling, I found your thread on CodeGuru, and the Windows Vista And OpenGL link that was posted. I haven't had the privilege of working with Vista yet, so I'm not qualified to give any advice. From what I can tell, though, the display model is radically different (with Aero enabled) and acquiring a handle to an applications front-buffer will be painstaking and driver-dependent for the foreseeable future. This is bad news for you.

Like I say, I no very little about Vista's display architecture, but the evidence suggests that getting the OS to capture the video overlay is a no-go. If this is the case, then you have no choice but to go in and get it for yourself. This means, barring the most contrived and audacious approaches, intercepting a render-call as we've discussed. So you're all out of leeway: it's hacker-methods or nothing.

Considering that your goals are so transient and non-intrusive (unlike those of a game trainer or suchlike), you can quite feasibly get around anti-cheat systems. But you'll need to be very efficient and slightly ingenious. This means injecting a DLL for any length of time won't be effective. However, if your probe can get in, get access, get the screen-capture and get out; all before the render-loop finishes, then the anti-cheat will have no idea anything happened. I have mentally prototyped a way to do this using the Windows Debug API, but I must warn you that it's no prettier than the previous method I described, and while not impossible, VB6 will have a relatively tough time delivering the goods. Interested?

May I ask why you're using a ten-year-old language? VB.NET is fairly easy to learn, if you know VB6, and it has a much wider support-base. Moreover, it is 64-bit compliant, is freely available as the Express Edition and will severely ease any attempted transition to C# (which is where everyone is headed).

Admiral