Converting MS-DOS-based Games to Windows 95: A Report from the Trenches

(Editor's Note - this article requires additional formatting work and is incomplete)

<%Topic="Converting MS-DOS-based Games to Windows 95: A Report from the Trenches"%> [font="Verdana"][size="2"]
Converting MS-DOS-based Games to Windows 95: A Report from the Trenches
August 15, 1996

Zachary Simpson, Titanic Entertainment

Table of Contents

Overview

Events

Drawing

Sound

Miscellaneous

Appendix A. Event Interface

Appendix B. Screen Interface

Appendix C. Sound Interface


Overview
This is not a FAQ, it's a ZAQ--a list of questions I asked when I was trying to convert a game from MS-DOS" to Windows". What follows is a guide that is far from complete but may offer answers to a few of the basic theory and coding questions that don't seem to be found elsewhere.

The best source of information about Windows is the MSDN Library. If you're not an MSDN member, stop whatever you're doing and sign up now. If, like me, you're too cheap to pay the member fee, find a friend who subscribes and get an old copy of the quarterly CD: everything you might want--with the exception of DirectX 2--will be there.


Events

Is it just me, or is the Windows event system just really stupid?
No, its not you; the Windows event system is a mess, and its inefficiencies become painfully apparent when you're writing a game. Remember that the system was never really designed to do what you want. Whereas a game typically polls the input devices once per frame, a normal Windows program is entirely event driven. That is, a "good" Windows program doesn't do anything until the user performs an action. This, of course, doesn't map well to games: a game is typically doing something, such as rendering or physics, whether or not the user is providing input.

One snag with implementing a polled system under the Windows WndProc() system is that keyboard events are reported in two separate, partially overlapping, messages. Furthermore, the definitions for the key constants are not interchangeable between ASCII characters and key codes. For example, the code for F1 is 0x70, which is the same as the ASCII value of "p." Thus, a single switch statement cannot contain both F1 and "p." Appendix A, "Event Interface," demonstrates code that converts all of the values into a homogenous set. Note that this is source from three separate files with functionality deliberately separated to avoid unnecessary header file inclusion.


What do I do with all this "message pump" stuff?
Windows 3.1 was designed to be cooperatively multitasked (as opposed to preemptively multitasked). In a cooperative system, each application is responsible for relinquishing control back to the operating system before another application gets any CPU. For example, suppose an application goes into an infinite loop. Under Windows 3.1, because the application would never relinquish control back to the OS, the machine would lock up. Under Windows 95, however, the OS would task-switch away from the process containing the infinite loop and other applications would continue to execute. Applications under Windows 3.1 had to have a standard place to return control back to the OS. In fact, this was done behind the scenes in the nonobvious GetMessage() function, which is, as you would expect, called from the main loop. Actually, in many typical Windows programs, it is the main loop.

The While loop is referred to here as the message pump. It causes Windows to check the pending queue of events for events that are applicable to this application. Here's the weird part: this function returns only when there are messages pending; in other words, this is a blocking call. The return state of GetMessage() is not true if message pending but rather true if message is something other than WM_QUIT. So you can see that this function is completely tailored to a traditional Windows application. Clearly, a game would not want to block in the main loop waiting for the rest of the OS. In Windows 3.1, a game would write a main loop as follows to avoid the block:

In this case, PeekMessage() is used instead of GetMessage(). PeekMessage() returns what you might expect: true if and only if message pending. The PM_NOYIELD tells the OS not to block and the PM_REMOVE tells it to toss the message once you've extracted it.

At the risk of repeating myself, note that this is an infinite loop. An infinite loop under Windows 3.1 would cause all other applications to stop; preemptive multitasking under Windows 95 would allow other applications to execute. Despite this, I still use the nonblocking calls under Windows 95 just so that my application will at least put up a fight.

I program some games to pause when the user switches to another application. It doesn't seem fair to let the bad guys kill the user while he's reading mail. The code looks something like this:

The meaning of the WM_PAINT message is explained in more detail below.


What are TranslateMessage and DispatchMessage doing?
Two more lines in the simple message pump need explanation--TranslateMessage() and DispatchMessage().

The TranslateMessage() call looks for certain combinations of events and translates them into Windows commands. For example, if you have a menu bar with, say, f as a hot key, then pressing alt+f should activate the menu and send a menu command message (as opposed to a keyboard message). Generally speaking, you should keep the TranslateMesssage() call in the message pump and avoid any alt+ combinations in your application.

Finally, DispatchMessage() takes the message and calls the WndProc() you registered for your window. Note that this is not the only time WndProc() will be called; occasionally, the OS will call it directly.

If the message pump is not running, you will not receive all events. Occasionally, you may want to call the message pump from some piece of code other than the main loop. For example, sometimes I write a little piece of debugging code that stops and waits for a key so that I can examine some state. In this case, I have a function called waitForKey(), which has a message pump and checks for a key press:

One last piece of advice: profile your input code. Some of the calls I wrote in the previous sample are surprisingly slow. You may need to hack the hell out of this, but the code will at least give you a basic understanding of what's going on.


All I want is the damn time of day in milliseconds. Is this too much to ask?
Yes, it's tragic: no single call does everything you want. If you want the actual time of day, you have to use GetSystemTime(). Although this function returns a field which is milliseconds, it actually has a resolution of about 50ms. You can also use a multimedia function (as if only "multimedia" programs need an accurate clock) called timeGetTime() which does indeed return an accurate time in milliseconds, but its value is the time since the system booted instead of the time of day. If you need accurate time for the purposes of profiling, you can use QueryPerformanceCounter() which gives extremely accurate timings in a completely arbitrary units that must be converted to meaningful units with QueryPerformanceFrequency().


Drawing

What is the difference between "WinG," "CreateDIBSection," and DirectDraw?
There is a lot of confusion over the difference between WinG and CreateDIBSection(). The latter was actually created first for Win32 (Windows NT and and Windows 95). WinG was a reverse port of this 32-bit code back to 16-bit Windows 3.1 and was sometimes referred to as "CreateDIBSection for 3.1." OK, so what is a DIB section and why do you care?

Under the original Windows specification, there was really no good way to create a simple frame buffer without a lot of overhead. For example, in the old way of doing it you had to create a bitmap in the application address space and then copy this bitmap first into the device driver's address space, where it was then often color converted pixel by pixel as it was copied across the bus into the video card's memory. Needless to say, that sucked. CreateDIBSection() solved both of these problems. First of all, it creates a frame buffer in shared device-driver/application memory avoiding the extraneous copy. Second, you can create it in the bit-depth that you want (typically 8bpp indexed) and create an "identity palette" that instructs the system-memory to video-memory blt() to avoid any per-pixel color translation. Unfortunately, writing this code is still very cumbersome, but it does indeed work well enough that you can actually play a game in a window if you are only mildly masochistic.

DirectDraw was a latter addition that got rid of a lot of the complexity of creating a frame-buffer with CreateDIBSection() and furthermore gives the game programmer access to a lot of the fun hardware that exists on all modern video cards such as super high-speed video-memory to video-memory blts and even esoteric features such as hardware scale and rotate. While most of these features were supported deep inside of the device-driver layer of window's GDI (graphic device interface, the API used for drawing spreadsheets, word processors, and other "normal" Windows applications), they were not accessible either directly nor though DOS. Thus, understanding how to take advantage of this hardware can significantly improve the performance of your games over a DOS implementation. For example, using a fill blt and page flipping roughly doubled the frame rate of Wing Command III? on a 486 at 640x480 when we ported the DOS version to Windows.

The practical details of all of this are beyond the scope of this paper. The best place to start is Animation Techniques in Win32 by Nigel Thompson or WinG Programmer's Reference by Chris Hecker (both available on the MSDN Library). The Game SDK (DirectX API) has a good programmers' reference but is short on theoretical explanation.

Here are the basics. Think of your computer and video card as two separate, high-performance computers that communicate via a nineteenth-century telegraph line. Obviously, the problem is the very slow connection, and thus you want to minimize the traffic across this line, or data bus. Now, the best form of compression would be to say something like, "Hey video card, go fill the rect from 0,0 to 639, 479 with black." Then, the video card would obey this command at super high speed since it fills the frame buffer with the computer and memory on its side of the bus. This would certainly be a hell of a lot better than touching each pixel independently across the bus. In fact, video cards can do a lot of little tricks like this as long as all the information necessary is on the remote side of the bus. For example, you can say: "Hey, copy (i.e. blt) the data in this rect of video memory to this other place in video memory." Again, nothing will go across the bus except the command. Not only that, but in many cases, the computer on the other side (the video chip) will actually do this operation asynchronously--that is, the CPU doesn't have to wait for the operation to complete! Other features, some of which are implemented on all cards, others of which aren't, are: line drawing, pattern filling, scaling, rotating, bltting with a transparent color, alpha blending, and page flipping.

So what do you do with this? The answer, of course, depends on the game. Here are a couple of ideas:

Space Combat. Suppose you're writing a space action game; let's call it "Wing Demander III." In DOS, I would allocate a 640x480 frame buffer in system memory. Once per frame, I would:

1. Clear frame buffer with black.
2. Render the stars and ships into the frame buffer.
3. Copy the cockpit sprite and HUD displays on top of this.
4. Wait for vertical blank.
5. Copy the whole damn 640x480 frame buffer across the bus (painfully slow).

Now, the silly thing is that for the vast majority of frames, significantly over half of the completed frame is either black or cockpit. Why should we have to copy this across the bus? With DirectDraw, we allocate two surfaces which page flipped on command. Then, once per frame we:

1. Tell the video card to fill-blt the back surface with black (practically instantaneous and also usually asynchronous).
2. Rendered the stars and ships across the bus into the back buffer.
3. Used the hardware blter to copy the cockpit from another location in video memory (we store it there when we first load). Again, this is lightning fast and asynchronous.
4. Request page flip. The video card automatically flips at the next vertical blank without us having to wait for it.

As you can imagine, this is significantly faster. The only time that it isn't faster is in cases where there is significant overdraw (multiple polygons get drawn on top of each other).

Figure 1. The previous frame (gray) has been scrolled up and to the left, revealing the two "sliver rects" in white at the bottom and right.

Top-down or side scrollers. Now, let's suppose you want to write a top-down scroller, let's call it "Sim-Convenience Store." In games like this, the camera is in a fixed position and the user scrolls around rapidly (or not so rapidly, depending on how good the programmers are). Under DOS, the render loop might look like:

1. If the screen has scrolled, shift the back buffer accordingly with a series of memcpy()s. Dirty the "sliver rects" that are revealed after a scroll. See Figure 1.
2. Traverse each of the dirty rects (including the sliver rects); draw everything that could possibly intersect each rect, all the while clipping the bitmaps to this dirty rect.
3. Wait for vertical blank.
4. Copy the whole 640x480 frame buffer across the bus.

Again, using the hardware blter to expedite the shift would allow you to avoid both the series of memcpy()s as well as the 300K bus copy.

1. If the screen has scrolled, tell the back buffer DirectDraw Surface to blt from itself to itself with the rects offset. This replaces the series of memcpy()s from above. Dirty the sliver rects.
2. Traverse the dirty rects, and draw everything as before, but now across the bus. But, instead of drawing directly into the back buffer, utilize all remaining video memory as a cache.
3. Check to see if each shape is already in the cache, if not, toss out the least recently used shapes and copy the shape from system memory to the video memory cache.
4. For each shape, use DirectDraw to blt from the video memory cache to the video-memory back buffer.
5. Request page flip. The video card automatically flips at the next vertical blank without us having to wait for it.

In this case, we can accelerate the renderer even more than we were able to under "Wing Demander III" because we can utilize extra video memory as a cache and avoid a lot of extra bus copies as well as shape decodes. Also, the expensive buffer shift and full frame across the bus copy have been eliminated. There is no case when this will be slower than the DOS equivalent.

First-person Dungeon Crawls. For the "Dooms" of the world, there's not much that the video card can do to help, other than maybe cache the unchanging window frame or weapons overlays and allow page flipping. Since every pixel is typically changing, there's no win from using the hardware blters. You might consider caching the animations of sprite objects in video memory, and with some video cards you might even be able to use the hardware blter to scale them, but generally this isn't going to add up to much savings.


What is frame pitch?


Figure 2. Surface width vs. surface pitch

The blters on some video cards can only copy to and from memory in certain widths, which is often not the same as the width of the surface. In other words, there is often extra unused memory when you allocate a surface. See Figure 2. To compensate, DirectDraw has a rectangular memory manager which will automatically use the wasted space if you allocate a surface which can fit. For example, we could fit a 384 pixel wide surface in the wasted (gray) space in . For the most part, you don't need to worry about the pitch. However, one time when it is very important is when you are copying data directly into a surface. In this case, you must first lock the surface and then be sure to advance your destination pointers by the pitch of the surface, not the width. See copyFromBuffer in Appendix B, "Screen Interface."


Do I care about CreateDIBSection now that there's DirectDraw?
Y es, you care. It is very difficult to debug using DirectDraw. For example, if you hit a breakpoint, you're screwed: you can't get back to the debugger to see what's going on, especially if you're page flipping The only way to keep your sanity is to implement your game so that it can run in either a window or DirectDraw and then do all of your development in a window and switch only into full screen when you are not expecting to hit a breakpoint or crash.

Setting up the abstraction between a DirectDraw surface and a DIB section is a little intimidating. Appendix B, "Screen Interface," contains the listing for an interface which abstracts a DirectDraw surface and a DIB section. This is intended for demonstration purposes only.


How do I set up my window? What do I do with the WM_PAINT message?
I usually make a very simple window that has nothing more than a title bar, menu, and a 640x480 client rect. My main loop updates the world which causes rects to get dirtied, and then the renderer fills in each of the dirty rects. In response to a WM_PAINT message, I simply mark the entire frame dirty. This may seem a bit extreme, but really WM_PAINT messages don't occur very often when you are the topmost window. For example, they occur most often when you drag one window on top of your game and then pull it away again. You could try to get fancy and dirty only the part that is exposed, but its really not worth the hassle. Note that earlier I suggested pausing the game when the application wasn't the in the foreground. Now, you can see why I put the call to setRepaintAll() into the if (foreground) statement.

Here's some prototype code for making a simple window.


Has the person who implemented palettes been executed yet?
I'll refrain from identifying the individual at Microsoft who implemented palettes. The legal department understandably keeps this is big secret lest a horde of irate programmers come and lynch this person.

Yes, the palette system is a giant piece of busted crap. Here's but one beautiful sample: the logical palette uses a structure PALETTEENTRY which is defined as CHAR R, G, B, flags, whereas DIBs use RGBQUAD, which is CHAR B, G, R flags--and, they're even defined in the same header file!

In Appendix B, "Screen Interface," you can examine the setPalette() function. Don't, I repeat don't, try to understand this code!


Weird Cursor Jumpiness?
The cursor functionality in DirectDraw has problems. For example, on some machines, you will get lots of flicker as you blt of page flip. With other video cards you won't see this. This problem comes up with some drivers only and will, one hopes, get worked out in the next few versions of DirectDraw. The cursor also becomes jumpy when bltting or flipping. This is due to a GDI lock deep in DirectDraw. The Microsoft folks say this will be removed by DirectX 3. In the meantime, there's a good little hack that seems to help a lot. Call GetCursorPos(), SetCursorPos() at least once per game loop, more often if you need to. For some reason this seems to alleviate the problem. I'm not sure how this affects speed consequences, nor am I sure if it fixes the problem on all drivers.


Sound

Does WaveMix Suck?
Yes. It is very slow and high latency. Use DirectSound instead.


What is DirectSound, and do I still need a sound system like AIL?
DirectSound is a very simple low-latency mixer API to replace the pathetically slow WaveMix. Although it is a little cumbersome to set up, it is conceptually simple and it works very well with practically no overhead. Here's the basic idea:

Instantiate a DirectSound object. From the object, allocate sound buffers. There are two kinds of sound buffers--primary and secondary. You need only one primary buffer, which serves as the master digital source from which DMA copies to the sound card. Allocate secondary buffers for your sound effects and then tell each of them to Play(). This will cause the secondary buffer to mix into the primary buffer. For digital music that you might be spooling off of a CD, set the looping flag on the secondary buffer and then write into the buffer just behind the read head. Under DOS, you would typically hook the timer interrupt and use this to feed to buffer to avoid skipping. Under Windows, you should create another thread. Although the Windows interface is a little tricky, it is much easier to program: you can debug it and it is generally more robust when something goes wrong. For a good introduction to multithreading, see chapter one in Advanced Windows NT by Jeffrey Richter (available on the MSDN Library).

DirectSound is for digital sound only. If you want to use MIDI, use a higher level sound system such as John Miles's AIL. John was actually involved in the implementation of DirectSound, so, as you might expect, AIL plugs nicely into DirectSound and gives you a single API for many different platforms such as the Mac and Playstation. See Appendix C, "Sound Interface," for more information.


Miscellanous

WINDOWS.H seems to take forever to compile. Is something w rong here?
No, you're not imagining this. WINDOWS.H with all of its include files is something like 15,000 lines long. Is it any wonder that it takes a long time to compile? Here's a big hint: Avoid, at all cost, including WINDOWS.H. I have a single MAIN.CPP that includes it to set up my Windows and screen interfaces, into which I shove tons of code. That way I avoid compiling WINDOWS.H more than once. Make your own free functions that encapsulate Windows functionality and use your own typedefs even if you have to duplicate Windows structures with your own types to accomplish this. Of course, this makes porting easier too, so its a doubly good idea.



Figure 3. Optimal settings for precomplied headers

There's one other trick. MSVC++ 2.0 sets the precomplied headers default incorrectly (or at least suboptimally). Set the project settings as demonstrated in . Then be sure that any C file that is going to include WINDOWS.H includes it first. This causes the precomplied header system to only precompile WINDOWS.H but it ensures that it will always do it without duplication. Although you lose on not precompiling your own code (you could put it before as long as you did the same in all modules), it is still a huge win to ensure that WINDOWS.H will never be compiled again. (This problem may be solved in versions past DirectX 2; I haven't tried it.)


What are the crazy error messages returned by DirectX?
When DirectX returns an error, you'll typically be looking at it in the debugger, which will happily give you a huge negative number printed in decimal. Convert this to hex. Ignore the top word. Convert the bottom word back to decimal. (In other words, just inspect the value with &0xFFFF). Look for that number in the appropriate D*.H file such as DDRAW.H. Have fun.


Why does Zack have such a bad attitude?
Five years of writing DOS games has made me generally mad at the world. Thank you for caring.


Appendix A. Event Interface

Appendix B. Screen Interface
This code is extracted from a larger system and will not compile due to external references to BaseFile, VFX, etc. This is intended for demonstration purposes only.

The main point of this code is to create a free-function interface that provides one interface for five different modes. The first three--Windowed, 2 page flipping, and 2 page bltting--are probably the most common. The last two modes--4 page bltting and 4 page flipping--are special modes that really only make sense in the context of the system I removed this from, but they do demonstrate the use of more than two video pages.

The DSurface class is a wrapper around DirectDraw's surface and palettes classes. This is certainly not the most efficient way to implement any of this, on the other hand, it keeps it pretty clear.

The "Simple" modes mean that the secondary surface is blted to the primary surface as opposed to flipping. This mode will tend to cause tearing, but is easier to understand and sometimes works when flipping doesn't.

The four page modes implemented here are designed to allow me to have static bitmap that doesn't move while I'm scrolling the foreground around. This is a little more complicated. The logic goes like this:

1. Fill the dirty rects on the "map surface" (the foreground in my case) to 255 which will be the transparent color.
2. Shift (i.e. scroll) the map surface if necessary.
3. Lock and draw dirty rects to the map surface.
4. Set the blter to transparent mode (doesn't work on all video cards, unfortunately).
5. Blt the background bitmap to the hidden surface.
6. Transparent blt the map surface on to the hidden surface.
7. Blt or flip depending on the mode.

Anyway, this is a lot of code. I hope it helps.[/font]