Skip to main content
GameDev.net gamedev.net
🔒 Locked

Lisp Pimpin'

Started by Nathan Baum Jul 13, 2005 at 1:07 AM 387 replies 71.4k views
Original Post
Nathan Baum
Nathan Baum
I figured it was about time I pimped mo' Lisp on yo asses.

(defpackage turbine (:use) (:nicknames tb))

(defconstant tb::*fail* 'tb::*fail*)

(defmethod tb::fail ()
  (block nil
     (handler-bind ((control-error #'(lambda (&rest ignore) (return))))
               (throw tb::*fail* tb::*fail*))))

(defmacro tb::fail-if (test value)
  (if test value (tb::fail)))

(shadow 'with-gensyms)
(defmacro with-gensyms (syms &body body)
  `(let ,(mapcar #'(lambda (sym) `(,sym (gensym))) syms)
      ,@body))

(defmacro defpredicate (name pred)
  `(defmethod ,name (&rest args)
     (if (apply #',pred args)
         (car (last args))
       (tb::fail))))

(defmacro tb::suspend (value &body forms)
  `(values ,value
     #'(lambda () ,@forms)))

(defun tb::make-generator (fn init-args)
  (let ((the-fn #'(lambda () (apply fn init-args))))
    #'(lambda () (if (null the-fn) (tb::fail)
                  (multiple-value-bind (result new-fn)
                                       (funcall the-fn)
                    (setf the-fn new-fn)
                    result)))))

(defmacro tb::defgen (name lambda-list &body body)
  `(setf (get (defun ,name ,lambda-list ,@body) 'tb::generator) t))

(setf (get 'tb::iota 'tb::generator) t)

(defun genfunp (symb)
  (get symb 'tb::generator))

(defun extract-generators (form)
  (let (names forms)
    (labels
      ((addgen (name form) (setf names (cons name names)
                                 forms (cons form forms)))
       (walker (form) (cond ((not (consp form)) form)
                            ((or (not (consp (car form)))
                                 (not (genfunp (caar form))))
                                (cons (walker (car form))
                                      (walker (cdr form))))
                            (t (with-gensyms (gen-name)
                                  (addgen gen-name
                                            `(tb::make-generator #',(caar form) ,(cdar form)))
                                  `(,gen-name ,@(walker (cdr form))))))))
      (let ((x (walker (list form))))
        (values (reverse names) (reverse forms) (car x))))))

(defmacro tb::if (test then else)
  (let ((block-name (gensym))
    (tag (gensym)))
    `(block ,block-name
        (catch ',tag
          (catch 'tb::*fail*
            (throw ',tag ,test))
            (return-from ,block-name ,else))
        ,then)))

(defun closure-reader (stream char)
  (declare (ignore char))
  `(lambda (&rest args)
      (destructuring-bind (&optional _1 _2 _3 _4 _5 _6 _7 _8 _9 &rest _>) args
        (let ((_ _1) (_* args))
          ,@(read-delimited-list #\] stream t)))))
(set-macro-character #\[ #'closure-reader nil)
(set-macro-character #\] #'(lambda (stream char) t) t)

(defmacro tb::every (subject &body body)
  (with-gensyms (loop-name ever-successful result)
    (multiple-value-bind (symbol-names init-forms subject) (extract-generators subject)
      (let ((init-args (mapcar [(gensym)] init-forms)))
      `(let ,(mapcar [`(,_1 (list ,@(third _2)))] init-args init-forms)
      ,(let ((init-forms (mapcar [(list (first _2) (second _2) _1)]
                                 init-args init-forms)))
      (let ((fnames (mapcar [(gensym)] init-forms)))
      `(let ,(mapcar [`(,_1 ,_2)] fnames init-forms)
             (let ,(mapcar [`(,_1 (funcall ,_2))] symbol-names fnames)
               (let* ((,ever-successful nil) 
                      (,result (loop named ,loop-name 
                        do (tb::if ,subject (tb::guard (setf ,ever-successful t) ,@body) nil)
                        do ,(with-gensyms (block-name)
                               `(block ,block-name
                                      ,@(mapcar [`(tb::if (setf ,_1 (funcall ,_2))
                                                          (return-from ,block-name)
                                                    (progn (setf ,_2 ,_3)
                                                            (setf ,_1 (funcall ,_2))))]
                                           symbol-names fnames init-forms)
                                      (return-from ,loop-name))))))
                  (if ,ever-successful ,result (tb::fail))))))))))))

(import '(t nil) 'tb)

(defmacro tb::guard (&body forms)
  `(catch 'tb::*fail* ,@forms))

(defmacro tb::or (&rest forms)
  (with-gensyms (block-name)
  `(block ,block-name
     ,@(mapcar #'(lambda (form) `(tb::guard (return-from ,block-name ,form)))
                         forms))))

(defmethod tb::identical? (x y)
  (tb::fail-if (eq x y) y))

(defmethod tb::equal? (x y)
  (tb::fail-if (equalp x y) y))

(import '(1- 1+ + * - / max min abs) 'tb)

(defpredicate tb::> >)
(defpredicate tb::< <)
(defpredicate tb::>= >=)
(defpredicate tb::<= <=)
(defpredicate tb::/= [(not (equalp _1 _2))])
(defpredicate tb::== equalp)

(defpredicate tb::even? [(evenp _)])
(defpredicate tb::integer? integerp)

(defmacro tb::not (form)
  `(tb::if ,form (tb::fail) nil))

(defmacro tb::and (&rest args)
  `(progn ,@args))

(import '(format setq) 'tb)
(import '(&optional &rest &body &key) 'tb)

(in-package tb)

(defgen iota (from to &optional (by 1))
  (if (>= from to) from
    (suspend from (iota (+ from by) to))))


That beautiful flower is the beginnings of the definition of a minilanguage similar in spirit to the Icon programming language. Predicates either succeed, producing their last argument as a value, or fail, producing no value and short-circuiting evaluation up to the nearest failure guard. Generators produce sequences of values, and the every form repeatedly evaluates an expression until all generators are exhausted, executing the body for those iterations where the test was successful. If no tests where successful, the every fails as a whole. In the completed language, one would be able to do things like (if (== (one-of foo bar baz) (gen 10 20 30)) (print "One of foo, bar or baz is equal to 10, 20 or 30")), or (print "Foo appears before bar in \"" string "\" in positions: ") (every (print (> (find string "foo") (find string "bar"))) " "). Right now, it allows for stupidly concise, although terribly inefficient (I'm no compiler writer), code like the following:

(every (setq i (iota 1 100))
  (format t "~A is prime~%" i (not (every (> i (integer? (/ i (iota 1 (1- i)))) 1)))))


Roughly, it says: For every i from 1 to 100, say it's prime if there are no numbers from 1 to i - 1 by which i is divisible. The particularly odd thing is not simply conciseness -- in the prime number case, the program could be equally concise in almost any language -- but the fact that the algorithm can be turned inside out, if that happens to suit the problem at hand: compare (every (print (> (find string "foo") (find string "bar"))) with an equivalent in your favourite programming language. The fact that I'm able to knock up something this weird in only several hours, which interacts almost seamlessly with standard Lisp (only almost, sadly) is telling of Lisp's deep-rooted flexibility. In summary: Lisp rox your sox. PS. If you feel like trying it out, I only know it works in CLISP. Plus, I've only been testing it for about 4 hours. Plan accordingly.
Rayno
Rayno
Jeez, you lisp guys should start a religion or something. [wink]

Seriously though, I was planning on giving lisp a try this summer. What would you say is the best tutorial/book to learn from?

(By the way, did you catch the Pascal's Triangle thread? Good stuff.)
Trapper Zoid
Trapper Zoid
Ah, the brackets, they blind me! [wink]

I love the variety in different languages. Someday I've got to write a 3D renderer in a functional language like OCaml. Or Prolog!
FReY
FReY
Lisp is a wonderful way to learn purely recursive programming.
do unto others... and then run like hell.
silverphyre673
silverphyre673
Quote:
Original post by FReY
Lisp is a wonderful way to learn purely recursive programming.


Lisp is also a good way to drive yourself insane and choke to death on parenthesis. It is fun...
my siteGenius is 1% inspiration and 99% perspiration
FReY
FReY
Quote:

Lisp is also a good way to drive yourself insane and choke to death on parenthesis.


Not if you have a cool IDE that matches your parentheses up for you :)
do unto others... and then run like hell.
Trapper Zoid
Trapper Zoid
Although my joke above might have sounded a bit 'flamey', I really do get a kick out of functional programming. Is anyone in the commercial game industry using Lisp (or any other functional language) as part of their titles (say, as a scripting language?).

Or what about Prolog? I've used Prolog to prototype a few things, and it's really easy to build a text parser out of, but is it used in (commercial) games at all?
Diodor
Diodor
Quote:
Original post by Trapper Zoid
Is anyone in the commercial game industry using Lisp (or any other functional language) as part of their titles (say, as a scripting language?).


That's the wrong question to ask I think. "What great games can be written in Lisp that cannot be written in C++?" is the right one.

You may search on Jak and Daxter for an example of using Lisp to create the compiler for a Lisp-like scripting language. They had no suitable Lisp on the console they targeted so they made their own - and even with this huge overhead they still got out ahead.
Icefox
Icefox
What's that saying? "Every non-trivial program includes a slow, buggy and poorly-written implementation of half of Common Lisp". With the corrolary: "Including Common Lisp". :-P

I need to mess with that stuff more; it's utterly wonderful, but only up to the point when I realize I don't have anything I'd actually want to make in it. Any game engines out there using Lisp?
-----http://alopex.liLet's Program: http://youtube.com/user/icefox192
CoffeeMug
CoffeeMug
Lisp is awesome. I've been writing a Lisp interpreter for quite some time now (adding some debugging facilities now) and it's been a great experience. I use my interpreter to create a website which allows me to play around and create a set of web sub languages. The first level of abstraction is of course xhtml so I can do something like this:
(xhtml-header    (body        (p (println "hello world!"))))

It only takes about an hour or two to create a sublanguage that represenets all important xhtml tags.

The next step is creating a more abstract language to represent some commonly used web components. Navigation menu is a candidate, perhaps a news component, etc. Additionally it's necessary to create an infrustructure that will map links to closures and/or continuations so clicking a link would be equivalent to a function call.

Another words, Lisp is great.
CoffeeMug
CoffeeMug
Quote:
Original post by Icefox
Any game engines out there using Lisp?

Who cares? Few people appreciate the unique set of features that make Lisp a wonderful rapid development language that it is. Use it and have the competitive advantage over thousands of people and companies that didn't think of using it.
Max_Payne
Max_Payne
Quote:
Original post by CoffeeMug
Quote:
Original post by Icefox
Any game engines out there using Lisp?

Who cares? Few people appreciate the unique set of features that make Lisp a wonderful rapid development language that it is. Use it and have the competitive advantage over thousands of people and companies that didn't think of using it.


I haven't used common lisp myself, only scheme, for a class about programming languages where we had to write a simple interpreter. I don't know if all lisp dialects are like that, but the scheme variation we used made it for very hacked code, and was quite impractical. It was quick to write the interpreter, but I wouldn't say the result was neat and tidy. Its not something I see anyone writing real-world software with.

The point being. If its not practical for actual development, then you have no advantage at all. That doesn't mean it can't be good for research, or for experimenting around or developing and testing algorithms... But to me, Lisp is not very intuitive. It doesn't feel natural either. And no, its not just because I'm "used" to procedural programming, but because, in nature, humans are used to plan and execute actions step-by-step. Lisp works well for performing actions on lists, but as soon as things get mildly complicated, then you have to think of a way to make it work with the language without extreme complication. It also seems to me that whenever it comes to interfacing with the outside world, there is something lacking from Lisp.

I don't mean to diss Lisp. I'm just saying that there are reasons it hasn't really caught-on in the industry and in actual development while other procedural programming and scripting languages are gaining wide popularity. But by all means... If you think you can make your own 3D engine or operating system in Lisp and make it perform well, go ahead, if you succeeded at that, you could gain great popularity ;)

Looking for a serious game project?
www.xgameproject.com
CoffeeMug
CoffeeMug
Quote:
Original post by Max_Payne
The point being. If its not practical for actual development, then you have no advantage at all.

But it *is* practical. I'm not sure what exactly you've worked with but Lisp is suitable for very large, complicated systems that would perhaps be too complicated to write in languages like Java or C++. If your code was hacked up perhaps it was because your interpreter wasn't very well designed, or you didn't understand Lisp properly, or your professor didn't know how to explain it well, or any combination of these and others reasons. Real production systems are written in Lisp! Did you know that .NET CLR garbage collection rutines were written in Lisp and then ran through Lisp->C compiler?
Quote:
Original post by Max_Payne
But to me, Lisp is not very intuitive. It doesn't feel natural either.

That's because you don't understand it. What do you mean by 'Lisp' anyway? If you don't like something about Lisp it has tremendous facilities to change itself. Don't like something about Lisp? Change it in Lisp itself.
Quote:
Original post by Max_Payne
And no, its not just because I'm "used" to procedural programming, but because, in nature, humans are used to plan and execute actions step-by-step.

Lisp is not a functional language! If you don't like using functional paradigm, don't use it. Write procedural code, write OO code, write functional code, combine various approaches, whatever. With Lisp the floor is yours.
Quote:
Original post by Max_Payne
It also seems to me that whenever it comes to interfacing with the outside world, there is something lacking from Lisp.

My interpreter is not any harder to extend than Python, for example. Could you be more specific?
Max_Payne
Max_Payne
Quote:
Original post by CoffeeMug
But it *is* practical. I'm not sure what exactly you've worked with but Lisp is suitable for very large, complicated systems that would perhaps be too complicated to write in languages like Java or C++. If your code was hacked up perhaps it was because your interpreter wasn't very well designed, or you didn't understand Lisp properly, or your professor didn't know how to explain it well, or any combination of these and others reasons. Real production systems are written in Lisp! Did you know that .NET CLR garbage collection rutines were written in Lisp and then ran through Lisp->C compiler?


I'm talking about *complete* software products here. Do you have examples of *complete* and nontrivial software products written in Lisp that are available for the x86 platform ? (no, examples of exotic applications nobody has ever heard about don't count).

Quote:
That's because you don't understand it. What do you mean by 'Lisp' anyway?


I mean any dialect of Lisp.

Quote:
Lisp is not a functional language! If you don't like using functional paradigm, don't use it. Write procedural code, write OO code, write functional code, combine various approaches, whatever. With Lisp the floor is yours.


And what would be the point of using Lisp to "emulate" what procedural languages have been doing very well for decades?

Quote:
My interpreter is not any harder to extend than Python, for example. Could you be more specific?


I'm talking about interfacing with the outside world. Writing a driver in Lisp, for example, or working with 3D graphics, or even doing sound processing. All that in an efficient manner. It seems the tools for doing that in Lisp are quite lacking.

Looking for a serious game project?
www.xgameproject.com
CoffeeMug
CoffeeMug
Quote:
Original post by Max_Payne
I'm talking about *complete* software products here.

Emacs. AutoCAD uses Lisp heavily. Just take a look at this.
Quote:
Original post by Max_Payne
no, examples of exotic applications nobody has ever heard about don't count

Why not? People that write 'exotic' applications like telco switches, air traffic control, etc. tend to be really smart. They use Lisp. People that tend to write usual business software and user applications tend to be average. They don't use Lisp. What does that mean, if anything at all?
Quote:
Original post by Max_Payne
And what would be the point of using Lisp to "emulate" what procedural languages have been doing very well for decades?

You wouldn't emulate it. You'd use it naturally because it *is* a part of Lisp. The point of using it is because traditional languages (I don't want to say procedural since Lisp is as procedural as any language could be) are not doing it well. I'm at work coding a client for a trading system right now. I start it up. Wait. Log in. Wait. Get to whatever part I am modifying. Play around there. If something doesn't work I close it. Change the code. Repeat. Hundreds of times. With Lisp I get to the point I am interested in. Play around. Change the code. Play around. Change the code. Play around. Nice, no? I can use advanced constructs like continatuions and closures and multimethods without jumping through hoops of various patterns. I can use macros to design domain specific code. I cannot do that in traditional languages. What kind of argument is that anwyay? Why create anything new? What's the point of using Python if traditional languages do it well enough?
Quote:
Original post by Max_Payne
Writing a driver in Lisp, for example, or working with 3D graphics, or even doing sound processing.

Lisp is not designed for writing drivers. For that you're better off using C. Other things you mentioned are at least as good as traditional languages. Oh and Lisp is very efficient. Far more efficient than Java or Python.
Max_Payne
Max_Payne
Quote:
Original post by CoffeeMug
Emacs. AutoCAD uses Lisp heavily. Just take a look at this.


From what I see, those products are "using lisp", they were not made completely in lisp, or even primarily in lisp.

Quote:
Why not? People that write 'exotic' applications like telco switches, air traffic control, etc. tend to be really smart. They use Lisp. People that tend to write usual business software and user applications tend to be average. They don't use Lisp. What does that mean, if anything at all?


Writing software for a telco switch strikes me as not very complex. Its something someone could easily do in C. Air traffic control is probably a better example of good applications. However, the reason I don't really care about exotic applications is because they don't really show that lisp is a qualified language when it comes to creating applications for the everyday world, that take an important part in everyone's life.

Lisp has been around for quite a while now. Longer than C++ has. Yet its not really catching on when it comes to professional software development. Don't tell me its because people are "resistant to progress" either, or anything of that kind. I constantly see people ranting about the latest and most efficient programming techniques and languages. Languages like Lisp and OCaml have had plenty of times to prove their usefulness, yet they are still only present in a handful of useful applications, and some extremely specialized solutions designed by I-don't-know-who with special constraints in mind.

Quote:
Change the code. Repeat. Hundreds of times. With Lisp I get to the point I am interested in. Play around. Change the code. Play around. Change the code. Play around. Nice, no? I can use advanced constructs like continatuions and closures and multimethods without jumping through hoops of various patterns. I can use macros to design domain specific code. I cannot do that in traditional languages. What kind of argument is that anwyay? Why create anything new? What's the point of using Python if traditional languages do it well enough?


I have been using python for some time. My personal experience is that it works O.K. Its main advantage is portability (to some degree). Thats about it. It is more portable than C++ for sure. But other than that, its also much slower, and not as powerful or nicely constructed in some areas. The main thing I like about it is that it has some easy to use libraries. Things are done in a simpler manner. However, this is not really a C++ problem: its a library design problem. Many C and C++ libraries make things much more complex than they really need to be. The reason people like C# is mostly for winforms, because, lets face it, the win32 API is simply backwards.

As for playing around. I can't modify code while its running. At least not with current compilers, because the code is compiled. But from my experience, its easier to debug C++ code than lisp code. Because of its typeless nature, lisp will allow errors to slip many levels deep, and when the problem arises, it can be very hard to track the source, but then, I might be biased, because the DrScheme debugger sucked. But again, my point is not to diss lisp here. My point is that debugging in C++ is not really an issue, nor is recompilation.

Quote:
Lisp is not designed for writing drivers. For that you're better off using C. Other things you mentioned are at least as good as traditional languages. Oh and Lisp is very efficient. Far more efficient than Java or Python.


Well, it should be possible to write drivers in lisp. With a flexible language like C++, you can do pretty much anything in a reasonable manner. From game engines to operating systems to server-side scripting for websites.

As for the efficiency... Efficiency in terms of development time, or running time, or something else? If you mean speed... Faster than python? Possible. Python is entirely interpreted. Faster than java? Well... Considering java is JIT compiled, and still quite fast (easily millions of function calls per second on a modern CPU), I'm not sure. Lisp is bound to require more overhead for its special features (functions as first-class objects, for example). Faster than a language like C or C++, I don't think so. C++ is designed to easily map to hardware features, which makes it hard to beat in terms of speed, no matter what kind of fancy runtime optimization that hardly applies to the real world you use.

If you mean efficiency in terms of development time, I would have to see. And someone would have to tell me what makes it faster, really. Because most of the time I spend working on software is spent in designing the program and algorithmic details, not actually sitting and coding. The "You can code faster in this!" argument is of very little value to me.

Looking for a serious game project?
www.xgameproject.com
CoffeeMug
CoffeeMug
Ok, I see my arguments aren't very perceptible. I'll let someone else take over. I'll just answer one point.
Quote:
Original post by Max_Payne
My point is that debugging in C++ is not really an issue, nor is recompilation.

Let's see. It takes about five seconds to start up the java application I'm working on. Another five seconds to log in. And about five seconds for compilation of my changes. A total of fifteen seconds. While I'm fixing incorrect behavior I make changes->recompile->restart. Every two minutes. All day long. That means I lose about 12% of my development time (it's a lot more than that in practice, but whatever). If I do this for four hours every day, that accounts for half an hour of lost productivity every day, two and a half hours every week, 130 hours every year. Considering that average software developer costs about $50 per hour (including benefits, etc.) that accounts for $6,500 lost every year *per developer*. For a large company that employes, say, a thousand developers in different areas that accounts for six and a half million dollars of wasted time every year! Enough, in fact, to hire at least 65 developers!
bytecoder
bytecoder
Quote:

From what I see, those products are "using lisp", they were not made completely in lisp, or even primarily in lisp.

Aren't you a picky one. I know for a fact that emacs is made of at least 80% lisp--the extension modules that come with it. Perhaps you should be a bit more open-minded instead of demanding examples which are already right in front of you.
Quote:

Lisp has been around for quite a while now. Longer than C++ has. Yet its not really catching on when it comes to professional software development. Don't tell me its because people are "resistant to progress" either, or anything of that kind. I constantly see people ranting about the latest and most efficient programming techniques and languages. Languages like Lisp and OCaml have had plenty of times to prove their usefulness, yet they are still only present in a handful of useful applications, and some extremely specialized solutions designed by I-don't-know-who with special constraints in mind.

You've got to be kidding. You seem to be basing that solely on the fact that languages rise to the top based on how good they are; well, I've got news for you--they don't. Generally, it's a mix of productivity (learning curve counts), perceived usefulness, and how much it got shoved down your throat. Mostly the latter.
Quote:


I have been using python for some time. My personal experience is that it works O.K. Its main advantage is portability (to some degree). Thats about it. It is more portable than C++ for sure. But other than that, its also much slower, and not as powerful or nicely constructed in some areas. The main thing I like about it is that it has some easy to use libraries. Things are done in a simpler manner. However, this is not really a C++ problem: its a library design problem. Many C and C++ libraries make things much more complex than they really need to be. The reason people like C# is mostly for winforms, because, lets face it, the win32 API is simply backwards.

"Not as powerful"? Show me list comprehension, heterogeneous data structures, lambda functions, and closures in C++ without some ugly hacks, and then we'll talk.
Quote:

As for playing around. I can't modify code while its running. At least not with current compilers, because the code is compiled. But from my experience, its easier to debug C++ code than lisp code. Because of its typeless nature, lisp will allow errors to slip many levels deep, and when the problem arises, it can be very hard to track the source, but then, I might be biased, because the DrScheme debugger sucked. But again, my point is not to diss lisp here. My point is that debugging in C++ is not really an issue, nor is recompilation.

I don't see what being typeless has to do with debugability of a language. That's completely dependant on the development environment. There's nothing stopping anyone from creating a type-inferrable IDE for lisp that shows you the types of variables as you write it, so don't blame it on the language when it's the editor's fault. If I wanted to, I could create an IDE that completely strips off types from C++ source code and reinserts them before compiling--does that make C++ impossible to debug?
Quote:

Well, it should be possible to write drivers in lisp. With a flexible language like C++, you can do pretty much anything in a reasonable manner. From game engines to operating systems to server-side scripting for websites.

From what I hear, lisp can be compiled, meaning it can do all of those things. The point of him saying that was that lisp isn't very suitable for driver writing, not that it isn't possible. C++ is only suitable for one of those things--game engines, and even that kind of sways towards C.
Quote:

As for the efficiency... Efficiency in terms of development time, or running time, or something else? If you mean speed... Faster than python? Possible. Python is entirely interpreted. Faster than java? Well... Considering java is JIT compiled, and still quite fast (easily millions of function calls per second on a modern CPU), I'm not sure. Lisp is bound to require more overhead for its special features (functions as first-class objects, for example). Faster than a language like C or C++, I don't think so. C++ is designed to easily map to hardware features, which makes it hard to beat in terms of speed, no matter what kind of fancy runtime optimization that hardly applies to the real world you use.

If O'Caml can run neck-to-neck with C++, I'm sure lisp can.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.