🔒 Locked

Is Effects 11 basically abandoned junk?

Started by bvanevery Mar 24, 2010 at 1:25 PM 56 replies 18k views

Original Post

Mar 24, 2010 at 1:25 PM

The February 2010 SDK has only 3 DX11 samples containing a .fx file: BasicHLSL11, DDSWithoutD3DX11, and SimpleSample11. All 3 use "technique," i.e. DX9 syntax, not the DX11 "technique11" syntax. Clearly, these samples were just moved forwards from earlier SDK releases, with no updating of the .fx files. None of the other DX11 samples use .fx files, they're all straight .hlsl. There is no DX11 equivalent to D3DX10CreateEffectFromFile. You can't compile everything in a .fx file at once. Instead, you have to compile each shader individually, providing the name of the shader entrypoint to be compiled. This is silly. It means you have to load the .fx file multiple times, just to keep the lazy API happy. Techniques often have compile semantics in them; this again becomes redundant. It makes nonsense out of compiling an "effect," not just a shader contained within an effect. What's the entrypoint for an entire "effect" ? I say the API has gotten lazy because MS is abandoning Effects. Effect pools went away, which shows a serious disregard for compile time during development. Eh, what do they care, too much work for them? So they release the source and leave developers to fend for themselves. Manually providing shader entrypoints is just a legacy support workaround.

DieterVW

Mar 24, 2010 at 1:53 PM

Actually, the manner in which the Effect file is built has changed and the old Compile***FromFile have all been merged into D3DX11CompileFromFile(). This function will compile a shader or an effect - it only depends on the profile being compiled. The resulting compiled blob can then be handed to the function D3DXCreateEffectFromMemory() to build the ID3DX11Effect object.

So, everything still works fine and you don't have to go out of your way and compile each shader individually as you suggest.

The Effect src was released so that people can see and learn from it. Large studios do not use the Effects system (they prefer to use libraries that they can modify) and there are always lots of questions from the community about shader management. So having the Effects system be source available should help people understand how everything works and possibly even allow them to make improvements, or perhaps only take and use the pieces they need.

MJP

Mar 24, 2010 at 1:57 PM

Quote:
Original post by bvanevery
The February 2010 SDK has only 3 DX11 samples containing a .fx file: BasicHLSL11, DDSWithoutD3DX11, and SimpleSample11. All 3 use "technique," i.e. DX9 syntax, not the DX11 "technique11" syntax. Clearly, these samples were just moved forwards from earlier SDK releases, with no updating of the .fx files. None of the other DX11 samples use .fx files, they're all straight .hlsl.

*shrug*

Samples for native DX are nowhere near as important as they used to be. It used to be that a lot of beginners and students used native DX, and needed a lot of samples to learn the basics of graphics programming. Today most of those people use XNA, and native DX is mostly the realm of professionals. Those people don't need a million samples...they'll usually just need something that gets them started with the API, and a few that show how to use some of the very advanced features.

Quote:
Original post by bvanevery
There is no DX11 equivalent to D3DX10CreateEffectFromFile. You can't compile everything in a .fx file at once. Instead, you have to compile each shader individually, providing the name of the shader entrypoint to be compiled. This is silly. It means you have to load the .fx file multiple times, just to keep the lazy API happy. Techniques often have compile semantics in them; this again becomes redundant. It makes nonsense out of compiling an "effect," not just a shader contained within an effect. What's the entrypoint for an entire "effect" ? I say the API has gotten lazy because MS is abandoning Effects. Effect pools went away, which shows a serious disregard for compile time during development. Eh, what do they care, too much work for them? So they release the source and leave developers to fend for themselves. Manually providing shader entrypoints is just a legacy support workaround.

Before you start up the anti-MS battle cry, you might want to actually take a look through the documentation.

I really don't know how you came to the conclusion that they "abandoned" effects. With D3D11 you've got the best possible situation: the implementation is available and complete if you want to use it, and you can customize it for your needs since you have the source. Again, these days DX is mostly the real of professionals who have their own effects/shader management system. It's a lot more useful for these people to have a source library, as opposed to a black box.

bvanevery

Mar 24, 2010 at 8:00 PM

Quote:
Original post by MJP
Before you start up the anti-MS battle cry, you might want to actually take a look through the documentation.

What do you think I did for a few weeks before getting disgusted with this? You find me the definition of an entry point for an "effect," not a shader. It ain't there dude. Pretty much every example relies on pFunctionName, an abdication of responsibility to some higher caller.

bvanevery

Mar 24, 2010 at 8:15 PM

Quote:
Original post by MJP
and native DX is mostly the realm of professionals. Those people don't need a million samples...they'll usually just need something that gets them started with the API, and a few that show how to use some of the very advanced features.

Have you paid attention to how much horse hockey is necessary just to get a triangle up on the screen nowadays? The cumbersomeness is amazing. People have been complaining about this since DX5, and if it ever got better at some point, now it is clearly much worse.

Mike.Popoloski

Mar 24, 2010 at 9:09 PM

Quote:
Original post by bvanevery
Quote:
Original post by MJP
and native DX is mostly the realm of professionals. Those people don't need a million samples...they'll usually just need something that gets them started with the API, and a few that show how to use some of the very advanced features.

Have you paid attention to how much horse hockey is necessary just to get a triangle up on the screen nowadays? The cumbersomeness is amazing. People have been complaining about this since DX5, and if it ever got better at some point, now it is clearly much worse.

No one ever has to draw just one triangle, so no one really cares how difficult it is to do.

MJP

Mar 24, 2010 at 11:57 PM

Quote:
Original post by bvanevery
Quote:
Original post by MJP
Before you start up the anti-MS battle cry, you might want to actually take a look through the documentation.

What do you think I did for a few weeks before getting disgusted with this? You find me the definition of an entry point for an "effect," not a shader. It ain't there dude. Pretty much every example relies on pFunctionName, an abdication of responsibility to some higher caller.

Did you try just passing NULL? I just tried compiling an effect like this, and it worked:

D3DX11CompileFromFileW(L"BasicHLSL10.fx", NULL, NULL, NULL, "fx_5_0", 0, 0, NULL, &effectBlob, &compileErrors, NULL);

Also with regards to effect pools...I haven't actually used Effects11 so I don't know much about how this works, but the docs seem to suggest that the effect pool functionality was replaced rather than removed.

MJP

Mar 25, 2010 at 12:00 AM

Quote:
Original post by bvanevery
Quote:
Original post by MJP
and native DX is mostly the realm of professionals. Those people don't need a million samples...they'll usually just need something that gets them started with the API, and a few that show how to use some of the very advanced features.

Have you paid attention to how much horse hockey is necessary just to get a triangle up on the screen nowadays? The cumbersomeness is amazing. People have been complaining about this since DX5, and if it ever got better at some point, now it is clearly much worse.

Anybody using D3D11 doesn't care about how easy it is to draw a single triangle. If that's what they cared about, they would be using the old, slow-as-hell OpenGL immediate mode functions.

People who use D3D11 care about having an API that maps as close as it possibly can to the underlying hardware, so that they can extract the best performance and make full use of the available features. I've never seen anyone argue that D3D10/D3D11 didn't deliver on that front. If you want a library that makes it easier and quicker to get things on the screen, then there are much better choices available (like XNA).

bvanevery

Mar 25, 2010 at 9:38 AM

Quote:
Original post by MJP

Did you try just passing NULL? I just tried compiling an effect like this, and it worked:
D3DX11CompileFromFileW(L"BasicHLSL10.fx", NULL, NULL, NULL, "fx_5_0", 0, 0, NULL, &effectBlob, &compileErrors, NULL);

I didn't, as it isn't documented. I thought about it; in fact, I thought about hacking my way through every single possibility until I actually found the answer, based on fxc compiler errors. Instead, I spent a lot of time Googling for answers, which is frustrating because nobody is talking about this ill-defined notion of an "effect" entrypoint. I found a lot of circumstantial evidence to make me believe that MS doesn't care much about Effects 11 and that I'm wasting my time trying to learn to use it. Based on what large studios seem to be doing, I don't think I was wrong.

When you say, "it worked," do you mean you actually got an app to compile everything in a .fx file, then make use of the results, successfully rendering pretty things on the screen? Because "it didn't barf when I fed it that" doesn't mean it worked. I've found several examples of fxc failing later, without a lot of warning or fanfare about what it didn't like. That's why I didn't just hack it out, because it looked like there was a good chance it would further waste my time.

Quote:

but the docs seem to suggest that the effect pool functionality was replaced rather than removed.

Not really, from the standpoint of building an app with lots of effects.

[Edited by - bvanevery on March 25, 2010 10:38:58 AM]

bvanevery

Mar 25, 2010 at 9:53 AM

Quote:
Original post by MJP
Anybody using D3D11 doesn't care about how easy it is to draw a single triangle. If that's what they cared about, they would be using the old, slow-as-hell OpenGL immediate mode functions.

I do care about how much of my time is wasted by shoddy documentation and design. It's enough to make me evaluate the Mac as a platform. I've done Windows, I've done Linux. I don't think much of the engineering on either of them. I know performance 3D graphics just fine, I've written software rasterization libraries and device drivers. I wonder if those functions are bad on OpenGL in general, or just bad on Windows, or... maybe you're exaggerating.

Quote:
If you want a library that makes it easier and quicker to get things on the screen, then there are much better choices available (like XNA).

Of course then you're stuck with it, which is what MS wants.

swiftcoder

Mar 25, 2010 at 10:07 AM

Quote:
Original post by bvanevery
Quote:
Original post by MJP
Anybody using D3D11 doesn't care about how easy it is to draw a single triangle. If that's what they cared about, they would be using the old, slow-as-hell OpenGL immediate mode functions.
I wonder if those functions are bad on OpenGL in general, or just bad on Windows, or... maybe you're exaggerating.

Immediate mode was removed from OpenGL long ago, though they are still provided for backwards compatibility, and newcomers often use them by mistake. They are slow as hell on any platform, because they *don't map efficiently to the hardware*.

Modern OpenGL, just like DX11, maps efficiently to the hardware, and as a result takes more effort to do silly things like render a single triangle.

Quote:
Quote:
If you want a library that makes it easier and quicker to get things on the screen, then there are much better choices available (like XNA).
Of course then you're stuck with it, which is what MS wants.

You are aware that Microsoft also provides simpler alternatives, such as XNA, or an OpenGL implementation?

feal87

Mar 25, 2010 at 10:27 AM

I feel the sadness when I see this kind of post.
are you really fine with that mastodonting API that OpenGL has become!? (With those useless and slow immediate mode that is deprecated anyway)
Direct X 10/11 is a clear refactor to allow to discard the old fixed pipeline crap and to allow a more clear API for both driver maker and companies.
FX 11 just like the everything of the d3dx library is just an addon and almost any company does not use it at all (just like someone said before me), making it shared source has been a great improvement to allow companies to inspect and maybe implement their own version.
But well let's not continue on this further as it's off topic.

Your question has been answered already by MJP so I'll not go further

Nik02

Mar 25, 2010 at 10:30 AM

Do note that rendering a million-triangle mesh is just as easy as rendering a mesh with only one triangle. And with modern hardware it is lightning fast.

Since the hardware is so flexible, you need to set many states to tell it what you actually want to do. Once you got the basics down, you can reuse most of the boilerplate code to do this. However, there's no "default settings" because that simply doesn't make sense with the abundance of usage patterns possible with modern GPUs.

As to why you need an API to begin with - the GPU is logically (and often physically) separate piece of equipment that is decoupled from the CPU in order to enable efficient manufacturing of computer systems. Since this means that the memory spaces are often physically separated, you need an abstraction layer that can give you "handles" to refer to resources between logical memory boundaries instead of naïvely copying stuff from CPU to GPU just before it is needed. Because of this and quite a few other reasons, software and hardware rasterizing are two entirely different things today.

bvanevery

Mar 25, 2010 at 11:10 AM

Quote:
Original post by feal87
I feel the sadness when I see this kind of post.
are you really fine with that mastodonting API that OpenGL has become!?

Probably not, and Khronos does not inspire my confidence. But when I waste my time on an eyesore like the Effects 11 documentation, where developer compile time concerns were also abandoned, I wonder how could it possibly be any worse? If I'm going to be continuously annoyed by my 3d development, at least I could get some platform freedom in the bargain. I have to add, a sinking feeling: with MS, it will never ever improve. That's how they roll.

Quote:
FX 11 just like the everything of the d3dx library is just an addon and almost any company does not use it at all (just like someone said before me),

Well that's just it, call me dumb for taking the docs seriously. I will now see if the OpenGL docs are equally painful or not.

feal87

Mar 25, 2010 at 12:24 PM

I seriously doubt that the documentation said something about D3DX being a core part of DirectX. Anyway precompiled shader are pretty fast. (You should make your own wrapper for managing shaders too if you're working for something serious) Concern about real-time compiling of shaders is pretty naïve, compilation is a pretty CPU intensive stuff as it need to optimize your code. (and here a pretty big effect compile in 140 ms)
Anyway OpenGL used with the modern functionality is just a downgraded (some feature are still missing) directx. Their way of working is EXACTLY the same, so don't expect much.

MJP

Mar 25, 2010 at 12:31 PM

Quote:
Original post by bvanevery
I didn't, as it isn't documented. I thought about it; in fact, I thought about hacking my way through every single possibility until I actually found the answer, based on fxc compiler errors. Instead, I spent a lot of time Googling for answers, which is frustrating because nobody is talking about this ill-defined notion of an "effect" entrypoint. I found a lot of circumstantial evidence to make me believe that MS doesn't care much about Effects 11 and that I'm wasting my time trying to learn to use it. Based on what large studios seem to be doing, I don't think I was wrong.

What professional game studios use has nothing to do with how much Microsoft "cares" about Effects11. Professional studios simply have very complex needs that have to be bet, mixed in with a healthy dose of NIH. I wouldn't worry about that too much if I were you, unless you're making decisions for a professional game studio looking to make big-budget games with big-budget engines.

Quote:
Original post by bvanevery
When you say, "it worked," do you mean you actually got an app to compile everything in a .fx file, then make use of the results, successfully rendering pretty things on the screen? Because "it didn't barf when I fed it that" doesn't mean it worked. I've found several examples of fxc failing later, without a lot of warning or fanfare about what it didn't like. That's why I didn't just hack it out, because it looked like there was a good chance it would further waste my time.

No I didn't use it an app, since I don't have an app set up where I could do so. I was asking if you had tried it, since I'm trying to help you work through this particular hole in the documentation.

Quote:

but the docs seem to suggest that the effect pool functionality was replaced rather than removed.

Not really, from the standpoint of building an app with lots of effects.[/quote]

I don't get it. Like I said I haven't tried it, but the documentation suggets that groups let you share a bunch of common parameters among a bunch of techniques in different effect files. That sounds like exactly what that dude wants, unless I'm missing something.

MJP

Mar 25, 2010 at 12:36 PM

Quote:
Original post by bvanevery
I do care about how much of my time is wasted by shoddy documentation and design. It's enough to make me evaluate the Mac as a platform. I've done Windows, I've done Linux. I don't think much of the engineering on either of them. I know performance 3D graphics just fine, I've written software rasterization libraries and device drivers. I wonder if those functions are bad on OpenGL in general, or just bad on Windows, or... maybe you're exaggerating.

Please I encourage you, go have a look at the OpenGL documentation and its mess of extensions. OpenGL doesn't even have an effects library that you can use.

And yes like others have said, those immediate mode functions were horribly slow because they in no way represent how the underlying hardware works. That's why they were removed.

Quote:
Original post by bvanevery
Of course then you're stuck with it, which is what MS wants.

Then use something non-MS if you want, who cares? The point is that if you're using a low-level API for interacting with the hardware, it's not going to give you high-level abstractions. It's just going to expose the hardware to you, and assume that you know what you're doing.

Erik Rufelt

Mar 25, 2010 at 1:15 PM

I can understand the frustration, as I too really hate when there's something that has to be guessed about. That said, I think you might get a much better response if you write a clearer question about your actual problem. If you don't know what to specify for the entry-point, provide source for the effect file you wish to use, and your loading code, and ask a specific question about what is unclear. I've read your post but it's hard to identify the actual question, and you've started a discussion about something entirely different.

As for your actual problem, provided that the question you want to ask is what to specify for entry point, it seems to me DieterVW's response answers that question. I haven't used Effects11, but from the documentation and from his response I gather that when you compile the effect file, every "entry point" (technique) in the file is compiled. Then you can choose any of the techniques in the effect to use when you render. So there can be many techniques contained in the same effect object.
If you read about Effects (Direct3D 11), making sure to read every page, especially all those under Rendering an Effect, together with the specifications for the interfaces, it should become clear.

bvanevery

Mar 25, 2010 at 1:17 PM

Quote:
Original post by MJP
Then use something non-MS if you want, who cares? The point is that if you're using a low-level API for interacting with the hardware, it's not going to give you high-level abstractions. It's just going to expose the hardware to you, and assume that you know what you're doing.

I'm not buying the proposition that if something is low-level, and used by "professionals," that it's ok to have slipshod documentation and design.

My prediction on passing NULL as the "effect" entrypoint is that it's not gonna work. The fxc command line expects an entrypoint, so I have my doubts that it'll work without one. I am not going to worry about this anymore right now though. I've seen how useless Effects 11 actually is, and it's got me annoyed about DX11 and MS in general, so I'm going to put a few days of work into seeing if the grass is greener on the other side of the fence.

bvanevery

Mar 25, 2010 at 1:25 PM

Quote:
Original post by Erik Rufelt
I can understand the frustration, as I too really hate when there's something that has to be guessed about. That said, I think you might get a much better response if you write a clearer question about your actual problem. If you don't know what to specify for the entry-point, provide source for the effect file you wish to use, and your loading code, and ask a specific question about what is unclear. I've read your post but it's hard to identify the actual question, and you've started a discussion about something entirely different.

That's 'cuz I'm not really asking. I'm telling you that Effects 11 is junk. Whether it's "abandoned" junk seems to be a semantic debate. Sure smells like abandonware to me, or I haven't been watching the MS Shuffle for 15 years.

Quote:
As for your actual problem, provided that the question you want to ask is what to specify for entry point, it seems to me DieterVW's response answers that question. I haven't used Effects11, but from the documentation and from his response I gather that when you compile the effect file, every "entry point" (technique) in the file is compiled.

No, entry points are for shaders, not for techniques. At least according to the docs, all the examples given, and all the actual sample code. If a technique is in any way regarded as being an entry point, it is undocumented. I've already RTFM for a few weeks. At present I've declined the opportunity to reverse engineer TFM.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.