Linux Development Part 2

Thanks a lot for this second tutorial.

It is not as easy as I would want it to (it never is :D ) but it's a great help.

I'll have to read it again and "digest" all the little bits.

Can't wait for the next one.


Best regards.

Thanks, JPGargoyle!

It looks intimidating when you first read it, but once you've built one or two custom libraries, it becomes fairly routine and easy to do.

Excellent article! I have been wondering for a while how one deals with compatibility across the dizzying world of Linux distros.
I wonder though... do any of the companies using this approach offer tech support for general linux, or only specific distros?

Quote:

Original post by swiftcoder Excellent article! I have been wondering for a while how one deals with compatibility across the dizzying world of Linux distros. I wonder though... do any of the companies using this approach offer tech support for general linux, or only specific distros?

Almost every software package I've seen for linux only offers support for certain distros.

Excellent second part I'm eager to read the next one!

As I was expecting, there are some complications to make the binary portable across distros but the process you explain seems simple enough.

Quote:

Original post by swiftcoder Excellent article! I have been wondering for a while how one deals with compatibility across the dizzying world of Linux distros. I wonder though... do any of the companies using this approach offer tech support for general linux, or only specific distros?

Thanks, swiftcoder!

My Game Company will support any distro. In the past eight months since Dirk Dashing's release, we've issued several patches for bugs found by users of Linspire, Debian, and other distros we didn't originally test on - and in the case of Ubuntu, we addressed audio problems introduced by the recent 7.04 release.

Overall, we have not had many technical support issues on Linux. Dirk Dashing is running very well on a wide variety of Linux distributions (this isn't an assumption based on no support requests, but rather it is based on direct feedback from Linux users who have written us to voice their enthusiasm), so we're not concerned about limiting our support. As a matter of fact, we're reusing Dirk's core engine for our next game, so it is in our best interest to fix anything that comes up so that our next game release goes that much more smoothly.

Thanks for your response!

Quote:

Original post by Codexus Excellent second part I'm eager to read the next one! As I was expecting, there are some complications to make the binary portable across distros but the process you explain seems simple enough.

Thanks, Codexus! Yeah, the process obviously isn't as easy as Windows, but it isn't impossible either. It's actually pretty straightforward, once you understand what you have to do (which has been the difficulty because nobody has publicly documented it before).

Quote:

Original post by GolfHacker Thanks, Codexus! Yeah, the process obviously isn't as easy as Windows, but it isn't impossible either. It's actually pretty straightforward, once you understand what you have to do (which has been the difficulty because nobody has publicly documented it before).

(FWIW, I'm attempting to explain why such doc isn't available, not to dismiss nor criticize the article)

The technique presented in the article is limited, although I do agree it's pretty much the only way to go if you're into proprietary software. The binaries will be available for x86 systems, without CPU-specific optimizations. They (obviously) won't run on PowerPC, Sparc, mips, S/390, ARM, m68k, Alpha, PA-RISC, SH, ... Not unless you build one binary per architecture.

UN*X-like systems are usually found in large servers, datacenters, clusters, etc: a lot of documentation is geared towards developers of software meant for such facilities, where non-x86 architectures aren't seen as exotic.

These two reasons alone probably explain why such documentation isn't more widely spread. Another one could be that free software developers write documentation for other free software developers, not for their proprietary software counterparts: because source code distribution addresses the issues mentioned in this article, they're actually non-issues in the free software world.

Good points, let_bound. Thanks for clarifying!

You're right - the binaries obviously won't run on anything other than x86 systems, unless you happen to have other system architectures available running Linux that you can build additional binaries on. Most independent developers like me do not. Of course, you could always partner with someone to create those binaries...

However, I don't think it is a significant problem. Most Linux home users, which I believe are the ones who would be primarily interested in games, are running x86 systems. I haven't had any requests for a binary of Dirk Dashing for non-x86 architectures - this isn't necessarily conclusive, but it does indicate that the return-on-investment ratio for me to support Linux binaries of non-x86 architectures isn't worth my effort. That's probably true for other game developers too.

great to see more linux tutorials, and (hopefully) more linux developers!

i use linux for development since several years now (no longer for games, though). Whenever i write software, it's written on linux first, and then ported to windows (if necessary). linux is just great - so many tools, so many possibilities. i can only encourage everybody to give it a try, and please don't be frustrated if the learning curve is a bit higher.

some tools i recommend:

vim - best editor ever ;) OK - no need to start a flame war. vi/vim is difficult to learn, but after two months, you're a LOT faster than in visual studio. it also comes with a GUI version.

gdb - the debugger. no need to use fancy GUI debuggers - usually, you just need a few commands. "b test.cpp:23" sets a breakpoint in line 23, test.cpp. "n" steps to the next instruction, "s" steps into the instruction, "finish" returns from this function, "r" runs the program, "c" continues the execution, "bt" prints the stack trace. that's all you need.

make - Makefiles suck. There are alternatives - scons is worth a try. I use autoconf/automake - they suck, too, but less.

valgrind - finds EVERY buffer overrun, memory corruption, access to uninitialized variables, memory leak etc. ALWAYS run your program with valgrind after you've run your tests.

gprof - a profiler. Gives you reliable stats about performance; tells you which function is called how many times, how much time is spent in the function etc.

and of course all tools are for free and open source...

Quote:

Original post by cruppstahl great to see more linux tutorials, and (hopefully) more linux developers! i use linux for development since several years now (no longer for games, though). Whenever i write software, it's written on linux first, and then ported to windows (if necessary). linux is just great - so many tools, so many possibilities. i can only encourage everybody to give it a try, and please don't be frustrated if the learning curve is a bit higher. some tools i recommend: vim - best editor ever ;) OK - no need to start a flame war. vi/vim is difficult to learn, but after two months, you're a LOT faster than in visual studio. it also comes with a GUI version. gdb - the debugger. no need to use fancy GUI debuggers - usually, you just need a few commands. "b test.cpp:23" sets a breakpoint in line 23, test.cpp. "n" steps to the next instruction, "s" steps into the instruction, "finish" returns from this function, "r" runs the program, "c" continues the execution, "bt" prints the stack trace. that's all you need. make - Makefiles suck. There are alternatives - scons is worth a try. I use autoconf/automake - they suck, too, but less. valgrind - finds EVERY buffer overrun, memory corruption, access to uninitialized variables, memory leak etc. ALWAYS run your program with valgrind after you've run your tests. gprof - a profiler. Gives you reliable stats about performance; tells you which function is called how many times, how much time is spent in the function etc. and of course all tools are for free and open source...

Not to forget objdump, for both size and performance optimiziation :)
and gas, the gnu assembler. if you know Intel assembly it will be shocking to see that there's a world behind intel syntax :D

Just a quick note: The home directory of the currently logged in user is always notated with the tilde symbol. Thus /home/myusername/whatever/ becomes ~/whatever/ so you don't have to change your paths for every user on the system.

Hi there,

I have not yet read the whole article, but I sure will soon - just listen to the title of the article: "Linux Development", can it get any better?

I skimmed it though, and the title "... Part 2" implies that there exists a "Part 1" of a series. If there is one, there should definitely exist a link to Part 1 somewhere in the article, and I couldn't find one.

Of course I can search through old articles, but it would be nice for future readers if links to all old articles in a series existed in one given article.

Regards,

/Mike

Option #2 (support distro packaging systems) isn't nearly as bad as it sounds. In fact, with APT (the package system used by Debian, Ubuntu, et al.) the program Synaptic usually comes with a distribution, which makes finding packages easy. And APT also takes care of finding, downloading, and installing dependencies, so that the user doesn't have to.

There is also Autopackage, which aims to support a single package format usable on any and all distributions, although there are problems with the system at this time.

Actually, with the proper tools, even supporting multiple packaging systems still isn't all that hard. And as long as you have access to the primary distributions (which you do, as long as you can download them and burn CDs) building for each is still a snap given virtualization tech today.

Another thing I'd like to point out, although it's not exactly related to the article, is that programs using their own custom directory layouts or including their own libraries alongside the executable (as the article seems to be headed) should be installed in /opt/XXX (where XXX is the name of the application). It's still okay to put a symlink to the application executable(s) in /usr/bin or /usr/local/bin but unless your application's files are stored as per the Filesystem Hierarchy Standard they should be in their own subdirectory of /opt (where nobody cares if you disobey the sometimes-anal FHS as long as you don't disobey it elsewhere).

I have to agree with Coldacid, packaging isn't at all hard and nothing prevents you from using packages as your primary distribution method. Packages are just like zip-files but with a special file that tells the package manager how to install it. People can download the package, double-click and go: the package management system will take care of it all for them, sucking down what it needs from the internet. These days it's so foolproof it's not sane to ignore it..

My approach is to choose one or two of the most popular distributions and package for those (Ubuntu and Fedora (or SuSE)). Ubuntu has around 9 million active users now and if you get it working on Ubuntu it's usually just a small step from there to get it running on any Debian based system thereafter (assuming there's interest). If you get cries to make an installer for other distributions setup a pledge system or similar to guage interest. If that's all too much work then so be it, stick with the popular distributions.

Set these distros up in a virtualised environment, make sure your package installs and uninstalls and ship your game, linked and ready to go. Your game will also reap the performance benefits of dynamic linkage.

This of course is all so much easier if your project is open-source, as the community will take a load of work of your hands and help get it into the main software repositories of other distributions.*

However.. if you do take the idSoftware/Google/Adobe approach of distributing for Linux you need to be careful. I don't think you should install outside of $HOME at all as then the user will have to have administrator privileges to install it and uninstall the game (assuming you even write an uninstaller). This is just unneccessary if you're going to be linking locally anyway! It's best I think to install it into a separate directory in $HOME so the user can just drag it to the trash when they're finished (just as on a Mac system). A good middle-ground is to give them the option of either installing in $HOME or outside (like /usr/share/games).

Moreso (and contrary to the article) you can't (and shouldn't) statically-link against everything. You don't want to be shipping the kitchen sink with your application and you'll only hit headaches if you try.

When do you _stop_ linking? If the OpenAL lib you ship depends on the the low-level ALSA library, should you ship that too? Nope. If you take that route you'll be compiling libstdc++ itself and wondering where it all went wrong. At some point you'll have to trust that the Linux distribution will need to have a few basics and you shouldn't be afraid to put that in a README (You need >= OpenGL version-x, >= ALSA version-x etc). People will be fine with installing this stuff if (in the rare case) it isn't on their machines already. Games typically have a ton of art also, so your installer could become huge fast if you're also shipping buckets of libraries.

If you don't go the neat and tidy package direction, don't forget to export the LD_LIBRARY_PATH (very important env variable) to point your binary to the libs you ship. To these ends it's a good idea to know how to write a shell-script as this will probably be what your client will click on to run the project (see both Pompom Games and Introversion software for examples here)

*Open-source software delivered through a package manager is the way most Linux users find and install software - Linux users aren't used to going to websites to get software. The way software is distributed, used and improved favours open-source software heavily and this is something you can play to your advantage if you want maximum distribution contact with low-maintenance overhead.

Naturally you wouldn't sell the software itself - as an object -but rather a service. Economies built around software on Linux relate usually to services and subscriptions and as such could work very well for a MMO type project - give the engine away but sell the server time (using a key-exchange to validate who's paid and who hasn't perhaps).

Thanks for reading.

[Edited by - delire on July 24, 2007 7:23:00 PM]

Wow, lots of comments to respond to today. I'll try to keep this succinct.

LINK TO PART 1

I agree, there should be a link to part 1 there. I would edit it if I could, but I can't. I'll send an e-mail to the editor.

REGARDING PACKAGING:

Most of delire's and coldacid's packaging-related comments will be addressed in article #3 of the series, which deals with packaging, installers, where to install, etc. For now, let me say this:

I think the use of a distro's native package management system is a good option if you open your source, as delire said. You can leverage community resources to build all of the packages and let them support the packages. That's a good strategy if you go the open source route, and I endorse it wholeheartedly.

But for closed source, I don't recommend this approach. As an example, try downloading and installing NeroLINUX. On Kubuntu 7.04, there are missing dependencies that I had to install separately - nothing automatically found and downloaded them for me (maybe I didn't do it right, but I'm pretty tech-savvy and if I can do it wrong, certainly other users will too). At any rate, for a lifetime Windows user like me, these extra steps are an exercise in frustration. This is not the impression you want to leave on a potential customer. It should install cleanly and at least as easily as it is does Windows. That's what article #3 will address.

Obviously if you have experience with package managers and you really want to go that route, that's your choice and good luck to you. But I think Gerry JJ's approach is considerably easier, especially for developers new to Linux, and that's what I'm recommending in my articles.

MISCELLANEOUS LINKING COMMENTS:

delire commented that "you can't (and shouldn't) statically-link against everything." First, for closed source commercial games, this is a non-issue because the LGPL license used by most libraries will prevent it anyway. Second, if you read my article carefully, you will note that I said not to worry about low-level libraries like ALSA, Arts, etc. Most game developers will use a third-party library like SDL, which means you probably won't ever need to link directly against these low-level libraries, either statically or dynamically (in fact, one of the configure options I specified for SDL was specifically for dynamic linking of low-level libraries). In the article, I also said you shouldn't bundle any low-level libaries with your app.

For reference, my Dirk Dashing Linux game currently only bundles 4 custom libraries: SDL, SDL_mixer, FreeImage, and libstdc++. So I'm not talking about "buckets of libraries". That's one of the reasons I recommend building custom libraries: to minimize dependencies so you don't have to ship everything and the kitchen sink with your game.

I hope the next tutorial includes some things about CVS. I want to learn more [smile]

An update on the link issue: I heard back from the editor, and he's looking into it. Hopefully we will see a link to Part 1 there soon. Thanks for catching that, GMMigge!