Let us talk about scripting languages...

Everything said in this post is meant as constructive criticism.

If you attempt to create a system like this as a VM-Compiled Bytecode scripting language, you are going to end up with a LOT of opcodes. On the other hand, if you try to interpret a script written with all of these things, it will be sooooo slow. I would recommend toning it down, and simple going for a C-like implementation.

In my own scripting language(psudo ASM/compiled bytecode), variables are restricted to simple two letter combinations (easy, fast, variable indexing), and the language interfaces with the game/engine through function pointers (int opcode in script, passes the current stack to a function defined in code). And, I must say, I have yet to find a use for it.

I would say that, for a useful scripting language, the only really special things you should include would be basic arrays, and structures. Variables could be restricted to a DWORD, and possibly strings. I see NO reason to include threading, unless you are attempting to create a new programming language. States and events would be great to see though =).

Z.

Here''s what *I* think.

1. My only vote is ability to call native C/C++ functions

2. RPG

3. Interpreted

4. Integers & Strings

5. Not really

6. Not really

That''s what I think would be the best kind of language for a scripting artcile.

Continuing the trend of positivity, I''ve never been much enamoured of creating new tools for old jobs that are more than adequately executed with existing ones. My point is that there are a host of scriopting languages available; why not devise a general methodology for embedding any of them into applications? It would be a sort of switchable/componentized backend that would allow end users to script the application in the language they find most useful/convenient and the application to support it.

Properly defined, the "scripting framework" would be extensible, simply requiring a parser/interpreter to produce output in a certain form and conform to a specific set of guidelines.

How ''bout it?

---

I wanna work for Microsoft!

1. ability to call native C/C++ methods

2. RPG

3. VM + compiler

4. Integers, floats, bytes

5. Nope

6. Would be nice

Oluseyi, I think he''s more interested in writing something educational regarding the creation of something new, regardless of whether or not there is already something capable of doing the job.

"Don''t be afraid to dream, for out of such fragile things come miracles."

quote:

---

Original post by Redleaf
Oluseyi, I think he''s more interested in writing something educational regarding the creation of something new, regardless of whether or not there is already something capable of doing the job.

---

That much is obvious. Is it necessary, though? Well, that remains to be seen.

---

I wanna work for Microsoft!

First of all let me say thank you to those of you who responded.

Well I agree to some point with Oluseyi. Using an already developed and tested scripting language is a very viable and cost effective solution. Unfortunately that is not my goal.

I''ve talked to Tim Sweeny (Unreal/UnrealScript creator/head programmer) via email a few times regarding compilers and his implementation of UnrealScript. He actually wrote the parser/lexer by hand which surprised me. He found it very enjoyable to write and design UnrealScript. We''ve also spoken of programming languages in general and their shortcomings.

In my opinion it boils down to: most programmers really enjoy control. Being a professional programmer I''ve often looked at someone elses code and said, "Ew." Even though that code might be valid, I didn''t write it and it may not be as robust or usable as I might like. I''m sure many others have experienced this before when taking over a project.

The best way to learn a system is to tear it apart; get down to the nitty-gritty details. Learn the program piece by piece and understand the multitude of relationships between the classes. Don''t confuse this with black-box code reuse. I''m all for reusing code. If someone writes a memory manager that works and I inherit it, I''m not very likely going to rewrite. As long as it works then so be it. On the other hand, compiler and language design is a very personal and rewarding experience. Sure we could all go out and learn Python and add it to our "game" but where is the fun or satisfaction in that.

Therefore it is my goal to not only show others how to write a compiler/language processor but I also seek to make it fun and enjoyable. This undertaking will be to my benefit as well. I''m not a compiler writer or language designer by trade. I''ve studied and learned about compiler design for awhile now and have built a few toy compilers. Now I want to work on something more concrete. Something more complete. Reusing an existing language is not my goal. Sometimes reinventing something helps you to understand why it was invented in the first place.

So that is why I want to do this article. BTW It appears that another fellow here, RedLeaf wants to work together, which is fine by me.

Keep the "votes" coming.

Best regards,


Dire Wolf
www.digitalfiends.com

Since it is for "fun", go for the jugular, and throw it all in there. I would love to see an effective way of calling C/C++ functions (other then the one I am currently using). Also, add functionality so that native C/C++ functions can call script FUNCTIONS, and not just an entire script. Again, I stress the VM/bytecode approach, as it speeds up execution times... a LOT.

I much agree, that writing a scripting language is a very enjoyable experience. There is nothing like the feeling "I did that".

Z.

quote:

---

Original post by Dire.Wolf
1. What general concepts would you like to see in a game scripting language? Things like strict-typing, classes, exceptions, ability to call native C/C++ methods etc.

---

Strict-typing is handy because it will probably make for faster code when working with ''primitive'' types. Classes would be good, particularly if the game was also written using classes. Native method calls are essential.

quote:

---

2. What kind of game would you like to see a scripting language incorporated into? RPG, RTS, TBS, FPS etc.

---

Er. All of the above?

quote:

---

3. Would you like to have a virtual machine and therefore a compiler that would turn your code into byte-code or should the language by interpreted?

---

Bytecode is easy to multi-thread in software. An interpreted language (operating upon, say, a decorated syntax tree) is more difficult to multi-thread, but may be slightly faster.

quote:

---

4. What variable types should the language support? Examples, integers, floats, strings etc.

---

ints, floats, strings. Perhaps chars, although char could just be a synonym for int. A fixed type may also be useful.

quote:

---

5. Would you like to see native support for threads and synchronization?

---

Threads, yes. Synchronizing, yes, but more difficult.

quote:

---

6. What about incorporating the notion of "events" and "state", very much like UnrealScripts implementation?

---

States are cool because they make things faster (you simply have a vtable for each state and switch vtables for your object when you change state). Events are cool, although UnrealScript doesn''t actually do ''events'', per-say - UnrealScript events are just methods that have been called events. You can see proper events in VisualBasic (and particularly version 4 and above, where any variable can receive event notifications from its value) and C# (and probably Delphi and other GUI-orientated languages, although I only know about VB and C#''s events).

All your bases belong to us

quote:

---

Original post by Oluseyi
Continuing the trend of positivity, I''ve never been much enamoured of creating new tools for old jobs that are more than adequately executed with existing ones.

---

Right on.

quote:

---

My point is that there are a host of scriopting languages available;

---

In my experience, most scripting languages are not designed for gaming, and those that are aren''t very good. UnrealScript is the obvious exception, which is superior to most ''real'' languages. However, I assume that UnrealScript is proprietry.

quote:

---

why not devise a general methodology for embedding any of them into applications?

---

I don''t know, why not? In fact, if you don''t like to reinvent the wheel, why not use a methodology that''s already been created? GUILE? COM?

quote:

---

It would be a sort of switchable/componentized backend that would allow end users to script the application in the language they find most useful/convenient and the application to support it.

---

Again, both GUILE and COM adhere to this requirement. COM has the advantage that there are more languages available for it (and you can even ''script'' a COM object in a compiled language).

quote:

---

Properly defined, the "scripting framework" would be extensible, simply requiring a parser/interpreter to produce output in a certain form and conform to a specific set of guidelines.

---

And with COM, you get a parser/interpreter with the system: either the processor itself, the Windows scripting host, or the .NET ''common language runtime'', if that''s what it''s called.


All your bases belong to us

Of course, using existing scripting systems is all very well, but not very good from a ''how to write your own scripting language article'' point-of-view.

All your bases belong to us

quote:

---

Original post by Mayrel
I don''t know, why not? In fact, if you don''t like to reinvent the wheel, why not use a methodology that''s already been created? GUILE? COM?

---

Well, while we''re on the topic why not write another frontend for GCC? I mean, they all compile to the same intermediate form...

COM is attractive, but I don''t know how "heavy" it will be since it has to support general programming rather than the specifics of game programming. By restricting the problem domain, we might be able to devise a leaner component.

quote:

---

Of course, using existing scripting systems is all very well, but not very good from a ''how to write your own scripting language article'' point-of-view.

---

True.

---

I wanna work for Microsoft!

Let me throw another feature onto the fire. Through the years I''ve always wanted a programming language to natively support distributed method calls. Sure C++ has support for distributed computing if you use COM or RPC but the chore of getting to that point seems to much of a hassle for game programming. Having to define interfaces in IDL is fun and all but it really does add another degree of complexity.

Bjarne Stroustrup, the creator of C++, believes that a new language (including a revised C++) should natively support distributed method calls. Some also firmly believe a language needs the ability to dynamically discover a class'' interface (ala the IDispatch interface of COM). How important are these concepts for game programming? Distributed method invocation is incredibly powerful and while it may introduce overhead, I think the benefits could potentially outweigh the drawbacks - especially if the solution and usage is elegant.
What about dynamic method discovery though? How useful would that be to mod-makers? I can see the uses for it in my head but at the moment can''t seem to get them through my fingers and onto the screen

I look forward to hearing a response.

---------------------

Now back to the topic of the article. I feel the best way to educate people about the how-tos of writing a scripting language is to start off making a simple subset of this "game language." We should narrow it down to accomplishing a specific task. Just getting a simple language up and running, including compilation, a virtual machine, and an example of its usage in a trivial game, would probably cover about 5 or more, long articles.

After those articles are complete it would be a good opportunity to start adding some of the more advanced features.

For a simple language we will probably need the following features:

1. variables with strict-typing
2. constants
3. functions (including support for returning a value)
4. user-defined types
5. control expression (if, while, for)
6. standard binary expressions (add, mul, sub, div)
7. logical operators (and, or, xor, not)

I''m sure I''ve missed something so please feel free to suggest extra features. Remember we need to keep it simple for the first few articles.

BTW While I''m a C++ programmer at heart, I really do like Java''s variable types/sizes and its native support for Unicode. Is this important? Discuss amongst yourselves.

Best regards and thank you for all the input,






Dire Wolf
www.digitalfiends.com

I don't think the ability to call native C functions is all that important in a scripting language. You shouldn't be trying to write code with script - the overhead of LoadOP, ExecOP is greater than the total time to execute most functions, so performance on that level can't be a concern. It's much more important that the VM is efficently implemented.

What will the scripting language be used for?
Character AI
Puzzles? (game logic I guess)

What constructs do I need to easily implement those uses?
State-machines, neural-nets, & seperate modules of execution.
Consequentially you need to pump-events. Could use pre-emeptive threading, don't have to. State-machines need to be able to send events to other state-machines, including ones in other scripts - perhaps even ones other machines (that takes of the RPC problem).

How do I tell the scripting language about my game?
By exposing a standard interface that enumerates type-information about the other objects in the game. (feature godlike, code-completion, easily implement)

How do I manipulate a game object from script?
By calling methods exposed in the enumeration step. A minimal amount of code should be written in scripts. Their focus should be on controlling game logic.

That's my list of important questions and my answers. All of the rest of the implementation details flow from the answers.
(Neural-State Script)

...

Assertion: RPC syncronous
Assertion: RPC high-latency
Assertion: Application-syncronous STA fPoS
Conclusion: RPC the suck and die!
  

Magmai Kai Holmlor
- Not For Rent

Edited by - Magmai Kai Holmlor on November 8, 2001 11:29:21 PM

Edited by - Magmai Kai Holmlor on November 8, 2001 11:36:16 PM

I''d like to see a discussion of how latent functions are implemented.

Ut

quote:

---

Original post by Magmai Kai Holmlor
I don''t think the ability to call native C functions is all that important in a scripting language.
You shouldn''t be trying to write code with script - the overhead of LoadOP, ExecOP is greater than the total time to execute most functions, so performance on that level can''t be a concern.

---

It''s the uncommon functions that need to be called natively: things like raycasting, path-finding or special-effects rendering. These functions would take far longer to execute in a scripting language than in native code.

quote:

---

It''s much more important that the VM is efficently implemented.

---

Indeed.

quote:

---

What will the scripting language be used for?
Character AI
Puzzles? (game logic I guess)

---

All of the above? If the language is well designed, it should be possible to implement all parts of the system in it. Indeed, IMO, as much of the program as is possible should be scripted.

quote:

---

What constructs do I need to easily implement those uses?
State-machines, neural-nets, & seperate modules of execution.
Consequentially you need to pump-events. Could use pre-emeptive threading, don''t have to. State-machines need to be able to send events to other state-machines, including ones in other scripts - perhaps even ones other machines (that takes of the RPC problem).

---

For AI, sure. However, are you going to program a neural-net by hand? No, you''re going to define the environment of the net and let it evolve by itself: you don''t need a script after you''ve set up the net, and the language doesn''t need to be a special neural-net language.

Similarly, state machines are inconvient to program in a normal language. Chances are you''d want a seperate tool to make them, then you might use a script to load it into the engine - using scripting language that isn''t specific to fsa.

quote:

---

How do I tell the scripting language about my game?
By exposing a standard interface that enumerates type-information about the other objects in the game. (feature godlike, code-completion, easily implement)

---

Wouldn''t that standard interface include functions, which would have to implemented natively?

quote:

---

How do I manipulate a game object from script?
By calling methods exposed in the enumeration step. A minimal amount of code should be written in scripts. Their focus should be on controlling game logic.

---

Again, I assume that these exposed methods would be native.

All your bases belong to us

quote:

---

Original post by Ut
I''d like to see a discussion of how latent functions are implemented.
Ut

---

This is a pretty technical discussion of latent functions, and this is a quick definition. It''s hard to see how latent, or indeed manifest, functions could be implemented in a scripting language.

All your bases belong to us

I think he was referring to functions that execute in parallel with other code - very similar to threads and context switching. For example, in UnrealScript you can call latent functions from state functions. These latent functions, like playing an animation, will execute and not return until their job is complete. The key is that other script code is still allowed to run and be called by the engine. It is a very interesting concept but can be very difficult to implement without introducing full threading capabilities into the VM.

I''m sure this is something that Redleaf and I may touch on in one of the more advanced articles (once we find a good way to implement it our self

One of the things we''ll need for the articles is a simple game that uses the scripting language. This will allow us to demonstrate how a scripting language and VM interface with a game.

I wonder if we could find someone willing to take on the challenge?

Dire Wolf
www.digitalfiends.com

1. Strict-typing? No. Since the scripting language should be aimed at non-programmers/designers, the language should be simple and typeless, even without the need for variable declaration. Please though let us have variable names longer than 2 characters. E.g. scripting languages like VBScript, VBA and PHP.
Classes? No. The non-programmer/designer should only be using objects not defining them. E.g. early versions of VBScript, VBA. Scripts per object and module level variables would however support the idea of custom properties on a per game object basis.
Exceptions? No, not a must have for day 1, if something goes wrong in the script then end and report error.
Ability to call native C/C++ methods? The scripting language shouldn''t have the ability to declare C/C++ functions in any DLL and then call them. However, it should support the ability for the application programmer to add functions to the scripting language that reside in the application, so the scripting language can callback.
Function Declarations? Yes, you should be able to declare, implement and call functions in the scripting language.

2. Kind of sample game? As simple as possible so we''re learning about scripting, not about the game. A scriptable Pong/Breakout/Robots?

3. Bytecode Compiled/Interpreted? Not sure. Whatever''s easier for day 1, interpreted I guess. It would be good to keep it lightweight, for possible future use as a Java applet scripting language.

4. Variable types? Typeless, i.e. one variable type (variant), and arrays, that don''t have to be pre-declared, aimed at non-programmer/designer like VBScript, VBA, PHP. It would be good if application defined objects could be called from the scripting language (with properties and methods), e.g. have a ''this'' object on events.

5. Threads. Not sure. It would be good if the application could call functions within the scripting code and give the function only a certain amount of time to execute, then stop. It doesn''t need to start the function again at the next point.

6. Events? Yes. The application should be able to call functions of a certain name in the scripting language at given times. These function names and their parameters would be application specific. The application is in control and only calls the script events when necessary, much like a web browser calls java-script events in the HTML page''s code.
State? Yes. At a minimum, support for global variables, and an Application_Start and Application_End event would allow variables to persist for the running of the game.
Scripts per object and module level variables would support state information on a per game object basis.

A few people might not agree with what I''m suggesting, but my reasoning is do the bare mininum for day 1, and the scripting language shouldn''t give me what I want as a programmer, it should only provide what I want the non-programmer/designer to have.