Hi all, I'm trying my hand at a general-purpose AI (yes, it will eventually wash my clothes and solve world hunger), but I need an "IQ function", something that can evaluate the intelligence of my AI. Obviously, it's hard to find a very good one, but all I need is a comparative function (it's a genetic algorithm). For example, I think that chess would be a good function. Unfortunately, it will take the AI forever just to understand the basic rules and representation of the board. I would need something that is appropriate both for idiots (this idiot is better than that idiot) and for masters. Mother nature had a pretty good function, if we eventually evolved out of it, but I don't have billions of atoms in my basement to simulate life. Any thoughts? Cédric [edited by - cedricl on August 25, 2002 12:16:18 PM]

Well, any "IQ function" or "performance evaluator" is necessarily
domain or even problem specific. So if you''re evaluating
chess ability of your AI, you use a "chess IQ function".
If you''re evaluating face recognition, you use a "face
recognition IQ function".

Since you''re dealing with genetic algorithm, the "fitness
function" is what you call "IQ function".

If you''re dealing with neural network, you could use the
SSE (Sum Square Error) to evaluate performance.

So not only are "IQ functions" domain-specific, but they
are also "solution-specific".


Kami no Itte ga ore ni zettai naru!

Yes, but what is Mother Nature''s fitness function? The ability to reproduce oneself. To achieve reproduction, you have to be alive. And to be alive, you have to eat food. To eat food, you need...

That''s how intelligence came to be (from the Darwinian point of view, at least). What I''m looking for is a function that will generate maximum complexity. If my function is ''win in a tic-tac-toe game'', I doubt that the end result will be interesting.

Cédric

quote: Original post by cedricl
Yes, but what is Mother Nature''s fitness function? The ability to reproduce oneself. To achieve reproduction, you have to be alive. And to be alive, you have to eat food. To eat food, you need...

That''s how intelligence came to be (from the Darwinian point of view, at least).

The problem here, is that a 7ft brute with violent tendencies would be considered more ''intelligent'' than a chess grand-master or a rocket scientist. Intelligence is only one tool in survival and while higher intelligence is likely to mean higher survival rates, higher survival rates will not have the same effect on intelligence due to all the other potential factors. After all, there are far more insects than humans, by several orders of magnitude.

I''ve no idea how you can possibly implement this in your code, but most IQ tests tend to revolve around numerical, lingual, and spatial manipulation. Maybe give it some mathematics, anagrams, or mazes to solve.

[ MSVC Fixes | STL | SDL | Game AI | Sockets | C++ Faq Lite | Boost | Asking Questions | Organising code files | My stuff ]

It strikes me that creating a simulated environment would be a good place to test out different AI''s and different ''biologies''. A world that has in it the requirement for agents to survive and breed. To survive they need food, warmth and shelter. To breed they need to be healthy (i.e., surviving well) and find a healthy, compatible mate. You could have different species as well. Maybe populate the world with brutes that compete for food, but can be eaten if hunted. Perhaps 1 brute could kill several of your AI critters, meaning coordinated behaviour was necessary to bring down a brute. But doing so would feed many critters for some period of time (and maybe provide warmth from the brute hide!).

Get the idea?

It doesn''t have to be a pretty graphical environment... just make it text based. You''d need to develop a planning language for your critters if there were to be anything more than stimulus-response (reactive) agents. You could also design a basic environment-rules engine. For example: A piece of wood and a rock could be worked into an axe. An axe could be wielded to either chop down trees (for more wood... for fires, shelters, more axes) or kill brutes (or other critters!). Etc., etc.

You could then test out different artificial agents in this domain and see which ones survive to breed successfully and maintain a stable or even growing population.

Cheers,

Timkin

That was my first idea, but then I thought that emulating nature was probably not the best idea. Sure, it eventually caused the emergence of intelligence, but Nature didn''t have access to computers. It didn''t have the choice; it had to use atoms to accomplish anything. Maybe is a simulated natural environment the best way to do it, but I think that a more creative way could be found.

One of the things that I really don''t like about genetic algorithms is that they have plenty of preconceived notions. DNA wasn''t an act of God, it appeared as a result of natural selection just like everything else. The same goes for mutation, crossover and mating. If they hadn''t been efficient evolution strategies, they would have been "discarded" by evolution. Why should we replicate these operators for computer programs? Shouldn''t they appear spontaneously as a result of evolution?

Anyway, philosophical ramblings aside, the best idea I have right now is, as Kylotan suggested, to throw it mathematical problems and see if it can figure them out. There isn''t enough diversity in mathematical problems, but they are quite easy to code.

Thanks for your answers,

Cédric

quote: Original post by cedricl
The same goes for mutation, crossover and mating. If they hadn''t been efficient evolution strategies, they would have been "discarded" by evolution. Why should we replicate these operators for computer programs? Shouldn''t they appear spontaneously as a result of evolution?

Mutation and crossover can''t really be a ''result'' of evolution because it is an integral part of evolution (at least on the biological level). How would you evolve this, without having some other sort of operator to alter the genes that influence mutation and crossover? Before evolution would be a different process... chemical ones, presumably.

And it''s not necessarily true that, even if evolution strategies did evolve, they would be discarded if they were suboptimal - due to the massive population size (ie. every single lifeform with DNA/RNA/whatever), almost any evolution strategy is going to work to some degree. And a ''better'' strategy might never succeed since it is already outnumbered by the ''worse'' strategy which it is incompatible with. (eg. Anyone else''s software vs. Microsoft''s. Only kidding, folks.) Biological evolution is too wide and varied an area to treat it in the same simplistic way that we can view genetic algorithms.

[ MSVC Fixes | STL | SDL | Game AI | Sockets | C++ Faq Lite | Boost | Asking Questions | Organising code files | My stuff ]

quote: Original post by cedricl
There isn''t enough diversity in mathematical problems, but they are quite easy to code.

You''re putting the chicken before the egg here cedricl. Once you choose an encoding scheme for the mathematical problem, you are already limiting the necessary abilities of the AI. You are essentially interpreting the problem for the AI and presenting it in one of many possible encodings. You''re biasing your results. A true test of any AI would be to merely present it with the problem domain, ask it a question and sit back and see what happens. Of course there are only a few known species that can handle such tasks, humans being one of them.

What is it exactly that you are trying to test? Different evolutionary strategies? Different encoding schemes for your AI? Different parameters for your GA algorithm? More information might help us provide more realistic answers.

Cheers,

Timkin

quote: Original post by Kylotan
Mutation and crossover can''t really be a ''result'' of evolution because it is an integral part of evolution (at least on the biological level). How would you evolve this, without having some other sort of operator to alter the genes that influence mutation and crossover? Before evolution would be a different process... chemical ones, presumably.

There wasn''t any crossover before there was DNA. I don''t know when it appeared, but surely, there were DNA-less organisms.

Even mutation, if it is the primary factor of evolution, it wasn''t chosen by nature. It was a result of random atoms doing random stuff.

quote: And it''s not necessarily true that, even if evolution strategies did evolve, they would be discarded if they were suboptimal - due to the massive population size (ie. every single lifeform with DNA/RNA/whatever), almost any evolution strategy is going to work to some degree. And a ''better'' strategy might never succeed since it is already outnumbered by the ''worse'' strategy which it is incompatible with. (eg. Anyone else''s software vs. Microsoft''s. Only kidding, folks.) Biological evolution is too wide and varied an area to treat it in the same simplistic way that we can view genetic algorithms.

You''re right, but it is said that evolution "jumps" occur when there is a sudden change in the environment. So if a small population of better individuals can develop, eventually, it could become the dominant specie because of one such jump.

What I hinted at in my previous post is what I''m tring to do. I hate having to manage species. If I am to simulate evolution, I shouldn''t have to take care of reproduction, and define operators. These concepts should appear by themselves as a result of evolution.

In nature, a cell uses the materials it finds around itself to reproduce itself. In a computer program, I couldn''t really give my Genetic Programs access to the whole memory, because there would be bloody massacres of perfectly good GPs by irresponsible ones. So I''ve decided to give them memory as a result of the fitness function. If they are good, I''ll give them memory. If they use that memory for reproduction, it is their choice, but I don''t handle reproduction myself.

That''s why I need a really good fitness function. I haven''t found it yet.

Cédric

quote: Original post by cedricl
There wasn''t any crossover before there was DNA. I don''t know when it appeared, but surely, there were DNA-less organisms.

nope, having genetic material (DNA/RNA on this planet) is one of the criteria for being alive. i won''t ramble, since you can find good info on this quite easily, but DNA is the blueprint for proteins, which control EVERYTHING that happens in an organism.