Cannot get multisampling AA working in glut program

You should get yourself a extension laoder like GLEW or GLEE.
They take care of loading extensions such as MULTISAMPLE_ARB.

Multisampling works fine on your graphics card I think, but you messed up something on loading that extension. Try it with on of the extension loaders above.

I've having the same problem with GL, so I have some recommendations:

1. Test OpenGL version at runtime. GL_MULTISAMPLE_ARB is supported form GL version 2.0 (or maybe 1.5, I'm not sure; google should help if you want).

2. If you detect an old version of GL, try installing new drivers. I have a GeForce 6600 AGP8 and it has 2.0 GL hardware support.

3. Do NOT force antialiasing. It is quite possible that forcing turns the extension GL_MULTISAMPLE_ARB off to avoid programs to change antialiasing.

4. Use RivaTuner ;), not NVidia control app, because NVidia app is pretty messy and buggy. Also, you have lots of settings in RivaTuner of which most are explained

I'm not sure how not to force it. Do you mean in the initial setting of display mode?

How do you detect at runtime the OpenGL version?

I never installed OpenGL, just using what came with the PC. Looks like even L_LIGHT_MODEL_COLOR_CONTROL is not defined so I can't add specularity over textures...

Which OpenGL should I install?

I'm looking at http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=46
I included glext.h and wglext.h which has all the defines needed.
Now, since my app is a console app and the GLUT is set to fullscreen so I assume that since I have no window handle I just use null for GetDC.

HDC dc(GetDC(0));
if (!dc) return 0;
PFNWGLCHOOSEPIXELFORMATARBPROC wglChoosePixelFormatARB = (PFNWGLCHOOSEPIXELFORMATARBPROC)wglGetProcAddress("wglChoosePixelFormatARB");
int fmt;
UINT nf;
float attf[] = {0,0};
int atti[] = {WGL_SUPPORT_OPENGL_ARB, GL_TRUE, WGL_ACCELERATION_ARB, WGL_FULL_ACCELERATION_ARB, WGL_COLOR_BITS_ARB, 24, WGL_DEPTH_BITS_ARB, 8, WGL_DOUBLE_BUFFER_ARB, GL_TRUE, WGL_SAMPLE_BUFFERS_ARB, GL_TRUE, WGL_SAMPLES_ARB, 4, 0, 0};
if (!wglChoosePixelFormatARB(dc, atti, attf, 1, &fmt, &nf) || nf <= 1) return 0;
PIXELFORMATDESCRIPTOR pfd;
int pfi(ChoosePixelFormat(dc, &pfd));
if (!pfi) return 0;

These all work fine (after of course I've initialized thigns by entering fullscreen mode). However, then in the article above he destroys the window an creates a new one with SetPixlelFormat. I don't know if this is compatible with GLUT, since I think that already does a SetPixelFormat internally and I don't know how to reset it. If I call SetPixelFormat after the glutEnterGameMode, it fails. Having to move from GLUT to something else is a problem as I'm using GLUT to handle all windowind and input issues... I don't know what to do :(

It says "glutEnterGameMode can be called when already in game mode. This will destroy the previous game mode window (including any OpenGL rendering state) and create a new game mode window with a new OpenGL rendering context."
However, how to make it use the new pixelformat with the multisampling that I've retreived with the code above?

I need help here...

glut hasnt been updated in years (thus doesnt handle multisample)

sdl OTOH does with

SDL_GL_SetAttribute( SDL_GL_MULTISAMPLEBUFFERS, intended_multisamplebuffers );
SDL_GL_SetAttribute( SDL_GL_MULTISAMPLESAMPLES, intended_multisamplesamples );

Well, I turned off stereo in the NVidia driver, and now if force FSAA is on, it works. However, having to force FSAA by the driver is still a crappy solution. I'll try SDL, the downside there being I have a ton of changes to do.

I just hardcoded the extension method in FreeGLUT since source was available. I guess that's that; works now.

Same problem for me.
Here is the thing.
You can setup the graphic card to do multisample anti-aliasing or leave it on default "application controlled", you can nou use GL_MULTISAMPLE_ARB.
But:
First you need to setup a special window for GL_MULTISAMPLE_ARB support (http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=46 this is a multisample tutorial, if this doesn't work than most likely your graphic card does not support multisample AA).
Do not use glut.
Second... you might not have the extension even if you can setup multisample from graphic card control panel.
to see all the extensions supported by your graphic card do this:
In an simple opengl application write this in a init function:
FILE *m_File;
m_File = fopen("extensions.txt", "wt");
const GLubyte *extensions = NULL;
extensions = glGetString(GL_EXTENSIONS);
fprintf(m_File, "supported:%s :", extensions);
Try to look for GL_MULTISAMPLE_ARB in the log file.