How to Log and stay Modular

Now, all over the code I have included a file that declares a global log that is used by the engine. This global log is used throughout the engine and really messes up my goal of keeping things modular. Now, what is the best way to log but still keep things modular? Please provide more than just, "Use cout or printf".

A global log is by definition, well, global. So I'm not sure what you're really after. But perhaps rather than calling functions on the global log directly, you could implement a reference mechanism in your log class and do something like this:


Log EngineLog("Engine.log");
Log GlobalLog(App.Log);
...
EngineLog.WriteToLog("Failed to load...");
GlobalLog.WriteToLog("foo...");


or, if you want to include data in both global and local logs,


Log MyLog("Engine.log", App.Log); //localfile_name, global_log_ref
...
MyLog.WriteToLocal("Failed to load...");
MyLog.WriteToGlobal("foo...");
MyLog.WriteToBoth("...bar");


Not really sure I understand what you're really after, perhaps...

Why does a log need to be an object? Any reason you can't just use a free function?

I am guessing you have an existing logging system implemented as per your snippet that forces you to specify the logfile name every time you want to log something. And you are not comfortable with having to specify this logfile name everywhere in your code.

Well the simplest solution is to just create a global variable for the filename that is initialized/set in only one place (if your code is like mine, you would probably have one or more header files somewhere that declares some global enums, variables, references, etc). And of course you could do stuff like having a startup object read from a config file to populate all these global variables when your game starts up.

Another way is for you to write a wrapper class around the existing Log class. This wrapper will manage (and hide) all the details of what logfile to open, when to open it, when to write, etc. So wherever you need to log a message, just call the constructor with the message and just let autoscoping take care of destroying the object.

LogWrapper EngineLog("this is a log...");

I have a logger class that collects log messages together in memory until a certain size before opening the logfile and dumping everything in. Works for logs that do not deal with debugging crashes!

At the risk of sounding like a broken record....

I don't understand the fascination with doing logging with objects. I think I've run into maybe three situations in my entire programming career where I needed more heavyweight state than I could handle with just a free function/static function someplace; for those cases it made sense to write a log wrapper object. Everywhere else... a lone function is more than sufficient.

Even better, write a lone function like Log() which can internally be hooked with various logging backends. Poof - all your code now logs to HTML or XML or the network or whatever else, just by changing the backend hook. No need to touch potentially hundreds of thousands of lines of code that rely on the logger.

In my experience, something like Log(EnumChannel, const char * text) is almost always enough. Specify the log destination/severity/etc. using an enum instead of hardcoded file names; this makes it a lot simpler to move things around/reorganize your logs if you so desire. It also lets you easily filter things based on their channel, so e.g. you can have Release builds only output Channel_Error or whatnot.

Even in languages that don't support true free functions, something like a static Logging.Write() function is a suitable alternative. This covers pretty much every base there is outside of functional languages.

I don't understand this fascination either. It would drive me insane having to create a logging object everywhere I needed to add an entry, or passing a logger reference to all my functions, let alone having to manually enter the filename to write to. In my opinion the single most useful and quickest to use logging method is:

#define Log(msg) SomeFreeLoggingFunction(msg, __FILE__, __LINE__)

Now go and worry about writing the game instead.

What I've seen in the past is that Log is a strategy associated with each module. Your 'glue' creates some common log and when the modules are created the glue specifies the common log as the strategy. Easier in not-C++, overkill for most things. As others have said (more or less), worry about functionality before maintainability/reusability; worry about your game not your code.

Brent,

It is exactly stuff like this that led to the development of aspect oriented programming. Unfortunately you won't find researching that helpful because solutions tend to involve inventing new language features.

But I tackled this exact question myself a while back, and the whole issue of "globals are bad".

Obviously, it would be easy enough to make globally visible log functions, but globals are bad right? But then I realized that the entire C runtime library is nothing but a bunch of globally visible functions. So if using globals destroyed programs, then you would never be able to do anything with a C++ program. So maybe it's not "globals" that are bad, but just global variables. But wait. Don't they amount to the same thing? Couldn't I call a function on a global object, and couldn't I access an object through a global function? What's the difference?

So after pondering the meaning of life for a while, I realized that the reason global variables are bad is because they store state which could change the behavior of any other part of the code that looks at them. If you think about the effects of, say, "printf", it is intended to effect the state of the world outside the program, but it is not intended to store any state which effects the future behavior of the program. In other words, nothing I do with printf now should change the behavior of the program when call printf later.

To a large degree, logging is similar. You want to provide a stateless function. Code makes calls through it to send information out to a log, but not to set or change any state which effects future behavior of the program. So it seems clear that if making printf global doesn't destroy programs, then making Log() global should be ok too.

But wait. What if I want the logging system to have state? What if it allows the code to log to arbitrary "channels" to keep logs about different stuff separate, e.g. Log(ChannelNameString, LogInfoString)? And what if I want to provide functions through which channels can be sent to different destinations, e.g. SetLogChannel("AI", ConsoleOuputFlag | FileStorageFlag)? Now what? Now I do want the logging system to store state. So can it use global functions or not?

The solution I came up with is to decompose certain systems into "Global-like" parts and "Object-like" parts.

So the logging system has a global Log() function which any code can access by including a header, and which has well defined default behavior when no state has been stored. Then there is a logging system class which provides control over the logging system's internal state, through an object which has a destructor to clean it up.

So in the end, any code which wants to fiddle with the system state needs to get access to the object, while any code at all can use the system in ways that don't change state. If no state is ever set, there's nothing to clean up, and if state is ever set, it's cleaned up correctly through normal object handling.

As another example, I used this same technique with a memory manager. It kept memory pools for different sized allocations and could be configured, say, with a configuration file. It had a fallback to call malloc() if there was no appropriate memory pool set up for the allocation request. The result was that if there was no configuration it would just forward all allocations to the system memory manager. And if it was configured well for the usage of the application it would improve memory characteristics considerably, but none of the code which used it ever had to handle an allocator object or worry about its internal state.

My engine is so modular that each module is an entirely different Microsoft® Visual Studio® project, combined into one solution.

The solution you are seeking is quite simple.
One of the modules should act as a foundation for all of the others. In my case, my foundation is the “L. Spiro Standard” library and its job is only to expose features that could be shared across not only the game engine but any project. For example, a CTime class, a StrLen() function, some routines for UTF handling, etc.

If I had a need for a logging function, it would go there.


There is nothing non-modular about this. My model library has no idea what a “terrain library” is, yet both of them are built on top of a few common foundation layers, which themselves also don’t know what models or terrain is.

If you do not have such an organization, you need to make such an organization.


L. Spiro

" but globals are bad right?"

No, they're not. They're bad if you're an academic computer scientist. Academic computer scientists never actually have to ship code. Globals work, use what works, ship code.

"really messes up my goal of keeping things modular."

Don't have a goal of being modular. Have a goal of "shipping code". What value to you is "being modular"? None. It's an **ideal**. It's not a goal. Your goal is to ship code. That means you get to sacrifice ideals sometimes. This could be one of those times.

You're over-thinking this problem. Everyone always does. Everyone thinks there's a problem with "logging" because it's always either too verbose or not verbose enough and it's never quite right and everyone thinks the way to fix it is to somehow make the logging production more complication.

The problem isn't with the logging. It's with the tool on the other end. Make your program be either "verbose" or "succinct" based on a flag. That's all you need on that front.

Then just output strings but have smarter tools looking at the output.

The UNIX world manages to handle tons and tons of logging using "syslog" which is hardly sophisticated. "syslogd", the program on the other end, that has versions which'll do everything up to handling global networks with millions of servers...

Don't try and solve your logging analysis problems during your log generation.

I agree with the general sentiment that you are over-engineering, or that you are forcibly applying the principle of EVERYTHING MUST BE AN OBJECT which you may have picked up at college / uni. OOP is a methodology of programming, not an ideology.

When it comes to logging, you have a finite number of options.

1) Create a new logger and then throw it away every time you want to log something
2) have a single global logger. (no, I didn't say singleton, I said single global instance, the terms are not synonymous)
3) use a free function, or depending on the language, a static function.
4) pass a logger to every single constructor of every single object your game uses.

It should be obvious from these choices what the correct course of action is. Let the academics continue to think inside their little boxes up there in that ivory tower while you like, get some work done.

Okay thank you to all of you for your replies! I think I finally have a grasp on what I should do.

Thanks,
Brent

Some say, 73% of GTA4's 100 million dollar budget went into the logging system alone.

On 9/3/2011 at 7:02 AM, Katie said:

" but globals are bad right?"

No, they're not. They're bad if you're an academic computer scientist. Academic computer scientists never actually have to ship code. Globals work, use what works, ship code.

The only reason academics are even aware of the troubles of global variables is because professionals, who were trying to ship products, discovered that global variables were one of the culprits keeping them from doing so, by making their code-bases unstable and unpredictable.

The biggest problem with the idea that "glabals are bad" is that it's so basic and so easy to remember and file away, that sometimes when it comes to implementing something and that old concept pops up a red flag, we might forget to take a moment to ponder the issue again, think about what exactly the problem was, and how exactly it applies to our current situation, if at all.

It would have been bad to let the concept back a code-base into the corner of passing a log object to every single function call. But as it happens, that doesn't mean the concept is wrong, just a little too easy to under-think.

I agree with the general sentiment that you are over-engineering, or that you are forcibly applying the principle of EVERYTHING MUST BE AN OBJECT which you may have picked up at college / uni. OOP is a methodology of programming, not an ideology.

When it comes to logging, you have a finite number of options.

1) Create a new logger and then throw it away every time you want to log something
2) have a single global logger. (no, I didn't say singleton, I said single global instance, the terms are not synonymous)
3) use a free function, or depending on the language, a static function.
4) pass a logger to every single constructor of every single object your game uses.

It should be obvious from these choices what the correct course of action is. Let the academics continue to think inside their little boxes up there in that ivory tower while you like, get some work done.

Option 4 with IoC wiring app/domain services is best.

" but globals are bad right?"

No, they're not. They're bad if you're an academic computer scientist. Academic computer scientists never actually have to ship code. Globals work, use what works, ship code.

"really messes up my goal of keeping things modular."

Don't have a goal of being modular. Have a goal of "shipping code". What value to you is "being modular"? None. It's an **ideal**. It's not a goal. Your goal is to ship code. That means you get to sacrifice ideals sometimes. This could be one of those times.

You're over-thinking this problem. Everyone always does. Everyone thinks there's a problem with "logging" because it's always either too verbose or not verbose enough and it's never quite right and everyone thinks the way to fix it is to somehow make the logging production more complication.

The problem isn't with the logging. It's with the tool on the other end. Make your program be either "verbose" or "succinct" based on a flag. That's all you need on that front.

Then just output strings but have smarter tools looking at the output.

The UNIX world manages to handle tons and tons of logging using "syslog" which is hardly sophisticated. "syslogd", the program on the other end, that has versions which'll do everything up to handling global networks with millions of servers...

Don't try and solve your logging analysis problems during your log generation.

While of course shipping code is the ultimate goal, it shouldn't be the principle you use to override other design considerations either. Modularity isn't just a valueless ideal, it helps in both designing the infrastructure of the system and in maintaining the system when bugs start to creep in. If your design isn't modular, then when you want to fix issues with rendering, for example, you need to be aware of how your collision detection works, and what game logic is affected and a hundred other things that you might have linked there because it 'worked', at the time, and it meant you got to ship that much faster. The result of ignoring modularity to ship code is that your code base gets spaghettified, which accrues ever increasing costs for maintenance and reuse later on.

You can go overboard, of course, with the issue of logging for example, but you shouldn't rail against programming ideals just because in the trenches you can't always see the value to spending more time in the design.

The biggest problem with the idea that "glabals are bad" is that it's so basic and so easy to remember and file away, that sometimes when it comes to implementing something and that old concept pops up a red flag, we might forget to take a moment to ponder the issue again, think about what exactly the problem was, and how exactly it applies to our current situation, if at all.

You need to fill a balloon or a blimp. Do you use hydrogen or helium? Helium.
Air cooled or water cooled engine? Water cooled.
4 stroke or 2 stroke engine? 4 stroke.

But, but, but... Each of the alternatives has advantages and tech has advanced and we can make it work and we've learned a lot and .....

This falls under engineering. You can always find an exception, there are always people who actually do know better (not just think they do), there might be a better choice. But it's rare to be working on a project that can benefit from those with people who are even interested into dealing with such tasks or where such gambles will have positive outcome.

Use the proven approach using methodology that was learned on hard failures on actual real world circumstances. Across thousands of projects, some methods are marginally better. And while 2% advantage on one single concept won't matter, combining different "better" ways will: 0.48*0.48*0.48*...* and the chance of failure is drastically affected.

Simple example - why do people claim globals/singletons are bad, then insist on using IoC/DI framework which commonly starts with singletons, yet doesn't exhibit any of mentioned problems? Because it's not globals/singletons, but most people involved in such discussions haven't been able to understand the true issues caused. So just don't use them.

But nothing.

This isn't a case of finding the exception to the rule, at all. It's a case of bothering to remember what that rule actually was in the first place.

"Money is the root of all evil", right? No. "The love of money is the root of all kinds of evil." Misquotes that are missing important nuance render reasonable ideas ridiculous.

It's easy enough to remember "gloabls are bad", but is there really much value in misremembering a principle and applying it incorrectly? Global variables cause problems. And they cause problems for a reason. Maybe remembering the 'misquote' is better than forgetting the issue altogether, but better yet is to take a second to understand the issue, so we can think about how the reasons apply to our situation.

"Just don't use 'em" is exactly the sort of thing that will have you trying to pass a log object to every function call in your codebase.

Yeah, there will be a lot of people who want to question various practices, and who might do so poorly because they don't understand the underlying reasoning (myself not excluded). But that's why we have colleagues and technical forums.

When discussing global variables, few people ever mention the security risks involved in using them.


L. Spiro

How is a global any more of a security risk than any other data in resident memory?