Compiler Creation

There are tons of tutorials online on the subject, but IMO not a single one of them is good. If you look over some of the threads I've started using my profile, you'll get some suggestions on books and the like if that is what you're after.

Quote:

Original post by Venerable Vampire Life would be so much easier if we could just get the source code.

this->omnipotent = true;
ROFL - Nice line!

Learn Flex and Bison. They make compiler creation far quicker and the end result is faster too.

Quote:

Original post by iMalc Quote: Original post by Venerable Vampire Life would be so much easier if we could just get the source code. this->omnipotent = true; ROFL - Nice line! Learn Flex and Bison. They make compiler creation far quicker and the end result is faster too.

While flex and bison are easy, they aren't the best things to actually learn how a compiler works with. Plus, they can be very very difficult to maintain. After working on the VC++ compiler for a summer (great time at it), and comparing it to the C# parser, I hate shift/reduce conflicts even more. I would learn a recursive descent parser. It's more intuitive and shows the concepts better, in my opinion.

Quote:

Original post by iMalc [...]Learn Flex and Bison. They make compiler creation far quicker and the end result is faster too.

CpMan is correct, but he left out the fact that you still have to know how a compiler works pretty much to use them properly and efficiently.

I opted not to use compiler compilers because they all spit out really ugly code with a very poor style that doesn't mesh with anything I've been working on. It seems to me that they are meant for standalone compilers that perform simple translations more than scripting libraries or things like that.

Quote:

Original post by CpMan I would learn a recursive descent parser. It's more intuitive and shows the concepts better, in my opinion.

Agreed. I used "Compilers - Principles, Techniques, and Tools" (commonly called the Dragon Book) as a reference at uni when we studied compiler construction and found it to be a great book. Simple compiler construction is not that difficult especailly when broken into steps. First we defined our grammar, then made a tokeniser/scanner and once that was working a recursive descent parser.

Mike

A guide to lex and yacc

Lizard

Let's Build a Compiler, by Jack Crenshaw

I can't say its the best resource I've found, but its pretty good.

I've heard great things about the dragon book mentioned above, and the book "Game Scripting Mastery" by Alex Varanese has proven itself quite good in my own experience. Check them out if you're able.

Quote:

Original post by Extrarius I opted not to use compiler compilers because they all spit out really ugly code with a very poor style that doesn't mesh with anything I've been working on. It seems to me that they are meant for standalone compilers that perform simple translations more than scripting libraries or things like that.

Why would you want to read the generated code in the first place?
You can't touch it anyway since it'll be regenerated any time you make a change to the syntax script.

Oh, and many complex compilers use them (GCC is a good example).

Quote:

Original post by doynax [...]Why would you want to read the generated code in the first place? You can't touch it anyway since it'll be regenerated any time you make a change to the syntax script. Oh, and many complex compilers use them (GCC is a good example).

Because if it the compiler is going to be part of one of my projects, I'd have to integrate it cleanly into the rest. Just putting stuff in the code they spit out wouldn't work, because for ex I'd want the Compile() and Execute() functions to be part of a class that maintains state information that would be used when compiling/executing/etc. I'd also want them to be thread-safe (not sure if the code they generate is or not), and stuff like that.
The generated code would have to fit into and work with the rest of my code. In other words, they were not suitable to my needs, which is why I didn't use them.

Thanks everyone, I'm currently checking out the pdf and I'll look at the online article later. Combined with the book (if it ever gets here) I should be able to fiddle around. Thanks again!

Game Scripting Mastery teaches you how to build a lexer, a syntax parser and generate output, basically the main stuff a compiler will do. It doesn't use Yacc or Lex and IMHO is a good introduction to the murky subject of compiler writing.

I liked the pxdscript tutorial, but nothing beats a good book. I also suggest that you should learn flex and bison since they will make thing a lot easier.

Quote:

Original post by Extrarius Quote: Original post by doynax [...]Why would you want to read the generated code in the first place? You can't touch it anyway since it'll be regenerated any time you make a change to the syntax script. Oh, and many complex compilers use them (GCC is a good example). Because if it the compiler is going to be part of one of my projects, I'd have to integrate it cleanly into the rest. Just putting stuff in the code they spit out wouldn't work, because for ex I'd want the Compile() and Execute() functions to be part of a class that maintains state information that would be used when compiling/executing/etc. I'd also want them to be thread-safe (not sure if the code they generate is or not), and stuff like that. The generated code would have to fit into and work with the rest of my code. In other words, they were not suitable to my needs, which is why I didn't use them.

I believe GCC recently changed to a recusive descent parser, and VC++ uses a version Yacc. It's heavily optimized, so it's not slow, but over the years it's become a bit hard to read. The advantage of an automated parser is that it ensures that we have a hard time breaking someone's other grammar changes (shift reduce conflicts appear). This is nice as people move from project to project every once in a while. A recursive descent parser is faster and cleaner, however it requires more control in the development process (when you have a lot of people around), to keep it working properly