Too much techniques

I'm currently in the process of making a small game engine (still basic). I have DX10 and PhysX running, Input/Sound, a small model loader (static, no animation/bones) and an entity system for the game and renderer. I am still missing lot of things like a scenegraph system (all my objects have their position, but it's not hierarchical), some culling algorithm and mostly 3D gfx eye candy. I wanted to implement some lighting/shadow first. I already done it in DX9 with the included lights and shadow volume, but now as I see it there have been so much new techniques that I don't know what I should use. Just check the DirectX SDK/NVIDIA SDK/FX Composer samples, there are like 10 different light techniques and 10 others for shadowing. I run a few, check the result, check the explanation, but I can't tell really which one seems better than another, which one is possible for a realtime game and is not only for demo-making. Which one is the "lower" or "higher" version of another technique in case the computer can't do it (I use DX10 so I would assume computers running my game can use the best one anyway). If I browse through SIGGRAPH, Google and even this site, the number of possible technique increase exponentially. Doing some point/directional light in HLSL code look fair simple enough, but as other topics here stated, it gives problems when using multiple lights/objects. If a new way gives better results without too much problem I think I should be aware and use it instead. After reading some post here I decided to check out Deferred Shading, but the thread was in 2003 so it might have changed a lot since then. Nvidia sample of deferred shading don't look like current games (too much blurry/strange) so I think there must be something else. Same thing for shadow, ShadowVolume/ShadowMap/SoftShadows/Variance Shadow Mapping/Percentage Closer Soft Shadows/ParallaxOcclusionMapping/etc. Theses are simply what the sample gives me, and I'm sure there a many others. I also know what the basics are, but not which one would give the best result while staying realistically possible for a real time game. This probably apply to tons of others element needed in games, there are too much techniques out there and we don't know what to look for anymore. The best answer to this thread would be a list of new commercial games using approximately the same technology (Unreal3, Devil may Cry4, GRID, Crysis (if not too different), and list the name of the techniques they use for various aspect of the game.