Grid computing

Wikipedia should give you some insight on the subject (especially the links at the bottom of the articles).

As for your program: there are several frameworks about parallel computing (also, I think it depends a lot on what are you computing , some algorithms just can't be parallelized :D), what language are you using ?

C/C++

Regarding the writing of such programs, MPI (Message Passing Interface) is very popular in high performance computing, and there are several implementations around. C, C++ and Fortran are the most common languages, and there's plenty of support for these. The website contains links to various tutorials and general information. You should be able to find a lot more with Google.

It's extremely easy to use. Each process has its own memory, and the MPI standard defines functions for sending and receiving messages. Writing a simple parallel algorithm can take just a few lines more than it would in C -- but obviously some algorithms are difficult (or impossible) to parallelize.

The standard also lets you do some fancy things with process topologies, but this is kinda optional.

I'm afraid I can't go into much more detail. I spent some time over the summer working on a parallel debugger, which involved debugging plenty of MPI programs, but not, generally, writing them. For the theory, you'll have to look to other posters and the 'net. But in most cases, the API was simple to use and understand, and I found it very intuitive.

EDIT: Of course, what you use will depend, to some degree, on your requirements. If your processes are to be regularly sharing large amounts of information, then MPI may not be the ideal solution. If I recall correctly, it's aimed mostly at applications in which the threads communicate relatively infrequently (though the time for a message to be passed will naturally depend on the system you're using).

[Edited by - MumbleFuzz on February 26, 2007 6:47:40 PM]

I'll have a look at MPI. Thanks.