optimizing state changes in a scene graph

It seems that "SceneGraph" refers to structure that may be very different from engines to engines. It can go from a very high level definition (nodes define universe part and behaviors) to very low level (nodes define state change, VBO, rendering process, passes, and such).

The "usual" aim of the scenegraph is to ease modifying your universe (the classics are transform group, switchs, alternatives,...). Therefore, if you stick to this aim, it is rather difficult to perform state sorting directly in the scenegraph.

You may as well define the main aim of your scenegraph to be state sorting. You will end up with a scenegraph which is in fact a state tree. It will work. The problem of handling transformations is not a big deal ; you will just have to handle the transformation separately (with a transform tree,...).

Anyway, I would not recommend this design ; your are removing the high abstraction level in favor of the low one where you could keep both. I use the following design : the high level scenegraph compiles into an optimized render frame (sort of state tree) which is then rendered. This could be understood as having a high level scenegraph + a low level one (the one you seems to require). This allows you to keep your main application abstracted enough from the rendering process and to update easily the rendering process when new rendering method / hardware arise.