hook api doesn't work sometimes, why?

As mentioned in the announcement thread, the sample won't work with any applications that use LoadLibrary to load d3d9.dll and GetProcAddress to retrieve Direct3DCreate9. Applications built with the sample framework, for example, fall into this category, as well as many games/commercial applications. In fact, almost any game/app that has Direct3D and OpenGL renderers will does this.

I've not coded a solution myself, but I reckon it'd be one of two things:
1. Hook LoadLibrary and/or GetProcAddress, and return a pointer to your own Direct3DCreate9 when you intercept such a request.

2. Write your own d3d9.dll and place it in the application's folder. Then, inside your DLL, load the *real* d3d9.dll and forward all the calls you don't handle to it.

Did you turn off DEP?

Quote:

Original post by LessBread Did you turn off DEP?

Would DEP prevent you from modifing entries in the Import Address Table (enabling you to redirect calls)? This is what APIHijack does, which is what that sample uses.

If it does take measures to prevent modifying the IAT, it must not work very well [wink] (seeing how I have never had problems doing that). I'm pretty sure it's because of the LoadLibrary() behavior - just hook those too and you should be fine.

Quote:

Original post by circlesoft Quote: Original post by LessBread Did you turn off DEP? Would DEP prevent you from modifing entries in the Import Address Table (enabling you to redirect calls)? This is what APIHijack does, which is what that sample uses. If it does take measures to prevent modifying the IAT, it must not work very well [wink] (seeing how I have never had problems doing that). I'm pretty sure it's because of the LoadLibrary() behavior - just hook those too and you should be fine.

Did you read the description of DEP from that link?

Quote:

The primary benefit of DEP is that it helps prevent code execution from data pages, such as the default heap pages, various stack pages, and memory pool pages. Typically, code is not executed from the default heap and the stack. Hardware-enforced DEP detects code that is running from these locations and raises an exception when execution occurs. If the exception is unhandled, the process will be stopped. Execution of code from protected memory in kernel mode causes a Stop error. DEP can help block a class of security intrusions. Specifically, DEP can help block a malicious program in which a virus or other type of attack has injected a process with additional code and then tries to run the injected code. On a system with DEP, execution of the injected code causes an exception. Software-enforced DEP can help block programs that take advantage of exception-handling mechanisms in Windows.

This might not prevent a program from modifying it's own IAT for redirection to other functions, but it might interfere with a program modifying the IAT of another program (such as google earth). It probably depends on the type of hook used, specifically, whether there is any code injection or thunk stub used.

Quote:

Original post by LessBread Did you read the description of DEP from that link?

Yes, of course. It wasn't clear to me though if it would stop what we are doing here. Hence, my question back to you [smile]

Quote:

This might not prevent a program from modifying it's own IAT for redirection to other functions, but it might interfere with a program modifying the IAT of another program (such as google earth). It probably depends on the type of hook used, specifically, whether there is any code injection or thunk stub used.

APIHijack alters the stubs, but does not inject any code. So either DEP does not try to prevent that, or it's buggy.

Quote:

Original post by circlesoft Quote: Original post by LessBread This might not prevent a program from modifying it's own IAT for redirection to other functions, but it might interfere with a program modifying the IAT of another program (such as google earth). It probably depends on the type of hook used, specifically, whether there is any code injection or thunk stub used. APIHijack alters the stubs, but does not inject any code. So either DEP does not try to prevent that, or it's buggy.

What I meant by a thunk stub was a short snippet of redirection code stored as data, the address of which is inserted into the iat to replace the original function address. Examples of this approach applied to wndprocs can be found here and here and here.

With a function hook, this approach is typically used to replace arguments or to inject additional arguments into the call.

A buggy Microsoft Security feature? Inconceivable! [grin]

It's a touch frustrating! I'm running into this same problem, as I'm trying to write a tool which has an informational little popup (which should still be there in full-screen situations like atop a game). I could use overlays, but that seems non-ideal when basically everything I want to draw atop in this manner is likely to be D3D anyway; hooking to stick a simple little overlay in the corner seems best!

Arguably, there still has to be a (relatively) simple way to hook and overlay such D3D applications, given that PIXfW can seemingly do so. On the other hand, I wouldn't put it past Microsoft to have undocumented ways into D3D specifically for PIX.

(Microsoft? Undocumented private APIs? Nooo, that would NEVER happen!) ;P

Quote:

Original post by LessBread What I meant by a thunk stub was a short snippet of redirection code stored as data, the address of which is inserted into the iat to replace the original function address. Examples of this approach applied to wndprocs can be found here and here and here.

Ok, thanks for clearing that up. Now that I think about it...I did try to do something similar to this in the past, where I globally hooked OutputDebugString() to capture all D3D9 debug messages. I never did get it to work, this could be a reason why.

thank you guys, i have another question, how to control the hook function in the launcher, for example, changing the position of the cube on top of the scene in the launcher.exe?

this is how i modified the code, but doesn't work:

i keeped a global pointor in the direct3d-hook.cpp to point to the IDirect3D9 instance created with the hooked Direct3DCreate9 function.
MyDirect3D9 *myD3D=NULL;

and at the end of the MyDirect3DCreate9 function, i wrote:

IDirect3D9* WINAPI MyDirect3DCreate9(UINT sdk_version){	// Let the world know we're working.	MessageBeep(MB_ICONINFORMATION);	OutputDebugString( "Direct3D-Hook: MyDirect3DCreate9 called.\n" );	Direct3DCreate9_t old_func = (Direct3DCreate9_t) D3DHook.Functions[D3DFN_Direct3DCreate9].OrigFn;	IDirect3D9* d3d = old_func(sdk_version);		return d3d? (myD3D=new MyDirect3D9(d3d)) : 0;}

but i don't know why, even if the MyDirect3D9 generated the device successfully, the myD2D pointer is still null.

this function is the one i used to control the cube, but always beep:

D3DHOOK_API void Execute(){	if(myD3D)	{		if(myD3D->myDevice)		{			myD3D->myDevice->ControlCube();		}	}	else	{		MessageBeep(MB_ICONINFORMATION);	}}

:(

IDirect3D9* d3d = old_func(sdk_version);

myD3D = d3d ? new MyDirect3D9(d3d) : 0;

Maybe try validating this result before returning it.