'Mixing' Shaders and code to shader organization

1) You're talking about effects that can be combined together but not in a single shader. To integrate shadows you need to do a shadow pass using a different shader to create a shadow map. Then in your lighting shader you can simply do lighting calculations like usual and calculate the "shadowFactor" (the % of the pixel that is under a shadow") using the shadow map.
-To use deferred rendering you also need to do multiple passes (GBuffer-pass and Lighting-pass) so you will need at least 2 shaders.
-HDR lighting is a bit different because it is mostly done on post processing , you render objects using your basic per-pixel lighting shader to a Floating-Point texture, then in post-processing you calculate the scene luminance and apply tonemapping.

You can mix all these effects in a uber-shader on a single file using #ifdef and then create multiple shader permutations at compilation-time, but you still will have multiple vertex/pixel shaders and a really confusing shader file.

What I use in my projects is basically this:
-GBuffer shader permutations file (a single shader file that contains different techniques to generate a the GBuffer, because not all objects need paralax-mapping etc).
-Shadow map shader file.
-Lighting shader file
-Material pass shader permutations file (that also contains multiple techniques).
-Post process shaders.

2) Depends greatly on the engine / API that you're using. It can be very cheap if done right, but in some engines it has a very high CPU cost.

-HDR lighting is a bit different because it is mostly done on post processing , you render objects using your basic per-pixel lighting shader to a Floating-Point Shader, then in post-processing you calculate the scene luminance and apply tonemapping.

Thanks. I think you meant to say 'Floating-Point [s]Shader[/s] textures' there.

Thanks. I think you meant to say 'Floating-Point [s]Shader[/s] textures' there.

That's right ;)

You really don't even need to use floating-point textures, either. There are plenty of fixed-point HDR encoding schemes out there that work very well, such as NAO32/LogLuv, some crazy shared-exponent deal or just 'cheating' a bit like Valve does and doing tonemapping directly in the lighting shader. Obviously these approaches can complicate your rendering pipeline a bit, but they can certainly result in a performance win or enable fancier effects on hardware that doesn't support (or supports poorly) floating-point textures.