Skip to main content
GameDev.net gamedev.net
🔒 Locked

Genetic Artwork

Started by djz Nov 11, 2009 at 12:23 AM 11 replies 2.7k views
Original Post
djz
djz
Free Image Hosting at www.ImageShack.us Thought I'd share what a long-time forum lurker has produced. This is a highly modified version of Roger Alsing's Method of art produced with a genetic algorithm. Alsing's exact execution may not be totally considered "GA", which inspired me to work on a more GA type implementation. The screenshots below highlight a variety of stages in implementation, but at "current spec" here's the major differences between my implementation and Alsing's.
  • Different plugins for the "artist", the "pool", and the "fitness" - The fitness module assigns a score to a Bitmap object generated by an artist. The artist produces an image (probably, but not necessarily, created by genes) and the pool manages artists, and decides based on their score what to do with them. By implementing these as plugins, I was able to experiment, very easily with different ideas while maintaining some core functionality that was totally isolated from the component I was working on.
  • "Bastardized Annealing"
    Simulated Annealing is half implemented by a combination between the "experimental artist" module and the "multi pool" module. While not a strict interpretation of the algorithm, here is how it is approximated: Within the artist, individuals "genes" (a drawing expression, a shape and its description) have a temperature which gradually cools. The temperature, when hot, allows the gene to bounce around a lot, move to more places on the canvas, have greater variety in colour and so forth. Within the gene pool module, there is a function - e(x,b) = sqrt( (1/x) + 1/3(1/b) ) - which determines the probability that a less efficient "solution" (artist) will be chosen, where x is the the absolute difference between the score of the best winner of a given generation and the current "elitist selected" winner, and b is the score of the "elitist" winner.
  • Extended Primitives
    The drawing expressions in the AST that I developed allow for curves, polygons, and outlined shapes. Additionally, the rotation of the expression can be mutated, as can the scale and origin.
  • Gene Momentum
    Individual branches of the AST can remember how they previously mutated. In the future, if that gene is called to mutate by the from the mutation method (mutations are called by the clone and child creation methods, and can be invoked directly by the pool.) it will tend to, though not necessarily, mutate in the same direction it has previously mutated
  • "Gravy" Colour
    In order to improve how the artist assigns colours, I created a colour space which I called gravy-space (because I couldn't think of a better word) that had more colour-family oriented transitions, rather than being very logical like RGB. I considered using HSV but since this is a rather experimental project, I decided to experiment with colour spaces as well. The conversion from gravy colour to rgba is as follows (gravy colour has four components, ABC and D): gravy_r = a / (1-(d*0.999)) / b ^ (c*d) gravy_g = b / (1-(d*0.999)) / c ^ (a*d) gravy_b = c / (1-(d*0.999)) / a ^ (b*d) gravy_a = d The 0.999 is obviously a repeated nine. It is approximated for the sake of appeasing the computer.
I learned a lot about programming and languages as I wrote this. I have always coded in CPP but recently started playing with C#. I am enjoying myself quite a bit, and the whole experience of writing this project has taught me alot about languages, as I research why C# does things differently than CPP while remaining syntactically similar. I am slowly learning to incorporate lambda expressions into my coding style and the whole process has inspired me to read up on languages like python, lisp, erlang etc. just to understand how languages differ so much. So much nifty shit out there, but only so many hours in the day to learn. Additionally, multithreading has been a huge challenge to understand. I'm still not quite there yet, but I'm very satisfied that my program no longer crashes for no reason and I still have a positive relationship with the debugger ;). My ultimate goal (and there has been some work towards this, though it is not pictured. If there is interest, I will post more screenshots) is to have the system generate original pieces. My experimentation thus far has been to make the fitness component a neural network that evaluates the image input. This has been interesting, but not entirely successful at producing the desired results (faces), more due to my lack of knowledge on how to efficiently program a fully recurrant neural network than a fault of the GA component. I am considering opening this up via Google Code or some other avenue if there is interest. If not, I will just keep working on it by myself as it is one of the most enjoyable software projects I have ever worked on. Free Image Hosting at www.ImageShack.us Free Image Hosting at www.ImageShack.us Free Image Hosting at www.ImageShack.us Note - For the above image, a technique was employed whereby after a substantial amount of time where the GA was unable to find an improved solution with the maximum number of drawing expressions (which varies, with some noise) it would dump the output to a buffer, and then draw "on top" of its buffer, leading to some interesting layering effects. Both images above were produced with a very old version of the program. Free Image Hosting at www.ImageShack.us This is Albert Hofmann. A major inspiration for just about everything I do in a given day. Free Image Hosting at www.ImageShack.us An incomplete portrait of Carl Sagan. Free Image Hosting at www.ImageShack.us And a rather terrifying cat. Long story short: GDNet has always inspired me to code. When I work my crappy desk job and browse away for the odd minute or too, read a post by some of the crazy coders around here I am inspired. Sure, the genius programmer may be a myth, but I think there's a few lurking around here who are close. And by that token, I would like to say that my code is not perfect. My math is mostly self-taught. I am learning, about math, programming, life in a corporate world, how to get ahead in advertising.... how to deal with life when you're on psychedelic mushrooms posting on GDNet for the first time...... But seriously, if people are interested in this sort of thing, I'd love to open my code up. The plugin framework might not be good enough for a genius, but it's good enough for me, and probably good enough for you too if you want to mess around with the host program. (But you'll have to believe me when I say although I've written many plugins for my own host, only a couple are completely compliant) I'm just not sure of the avenue, and not sure of the audience. If an audience exists, maybe they can point me to the right avenue. Peace out.
djz
djz
Some more images:

Free Image Hosting at www.ImageShack.us

Old version, hosting the NN plugin.

Free Image Hosting at www.ImageShack.us

Lots of images :)

Free Image Hosting at www.ImageShack.us

This shows the changemap that the fitness algo creates... it processes the source image, and then calculates the changes in RGB values for each pixel. This creates a coefficient that the error for that pixel is multiplied by. This causes solutions which duplicate the high contrast areas of an image to be favored for reproduction over solutions which duplicate low contrast areas; although ultimately, the only way to get a score of "0" (fittest) is to duplicate every pixel. This is calculated with the sum of the euclidean distance between the produced RGB and the source RGB for each pixel, multiplied by the amount of contrast per pixel.

[Edited by - djz on November 11, 2009 4:17:58 PM]
Beyond_Repair
Beyond_Repair
As far as I can tell: Using AI methods to "teach" a program what to draw, hopefully leading to results of artistic quality.

Some of those pics do look cool.
djz
djz
Quote:
Original post by zer0wolf
Could you tell us in English (erm... layman's terms) WTF this is?


A vector representation of a drawing (not a specific drawing, just a drawing, nothing more than a scribble) is generated. It is scored by some fitness algorithm. (in this case, the summed euclidean distance between the RGB values of all pixels in the generated drawing versus the source image) Drawings that are close have a high chance of reproducing and making more drawings. These child drawings will mutate (and tend to mutate as they have previously) and can expand. The number of points in an expression can increase or decrease, the number of drawing elements in the drawing can increase, decrease. Colour values can change, curve tension, etc.

Eventually it converges on something that satisfies the fitness function. In other words, it draws a picture through trial and error by "looking" at a source image. It's not really AI so much as GA influenced image-based rendering. I am currently researching face detection so that it will not be limited to re-generating a source image, but could also generate its own drawing that, according to the NN performing fitness, qualifies as a face - or any other object you train it to detect in an image. That's the dream, anyway.



owl
owl
Quote:
Original post by djz
Free Image Hosting at www.ImageShack.us


That's a green apple right there!
[size="2"]I like the Walrus best.
djz
djz



These were made by using the NN to filter an image.
Kirl
Kirl
looks very interesting, I like the Carl Sagan picture as it is!

The eventual goal sounds very cool, but you'd have to put in a large amount of similar head shots. To train it on faces, meaby you can put it online and ask visitors for a webcam mugshot. This should result in a good bunch of similarly positioned faces. It would be awesome to have the learning process output as a movie.

I always wondered what would happen if random learning algorithms were put online for long periods of time and allowed to learn by visitors interacting with it.

I would be very interested to see what comes out of something like that.
djz
djz
Quote:
Original post by Kirl
looks very interesting, I like the Carl Sagan picture as it is!

The eventual goal sounds very cool, but you'd have to put in a large amount of similar head shots. To train it on faces, meaby you can put it online and ask visitors for a webcam mugshot. This should result in a good bunch of similarly positioned faces.

+I always wondered what would happen if random learning algorithms were put online for long periods of time and allowed to learn by visitors interacting with it. I would be very interested to see what comes out of something like that.


Something similar to what you describe has been done, although not using mugshots but instead allowing users to rate how much a given drawing looks like a face; making human perception the fitness evaluator.

- Also, there are some pretty good image databases of faces, a bunch located here that I've been messing around with.
Kirl
Kirl
Cool, mutating pictures is looking very impressive. Thanks for the link! :)

Do you plan to incorporate the help of the intarwebz as well? It seems somewhat more interesting then pulling heads from a database, plus you can train on specifics (male/female/child, or various objects) without much effort.

I think a large audience ensures a wide range of experiments and results.

djz
djz
Quote:
Original post by Oberon_Command
Try putting The Scream through it. I'd be interested to see what it comes out with.


While I would love to, the convergence rate is quite slow for high-resolution images and could take several days if not weeks. The algorithm tends to do best with simple subjects (The Scream has a lot of background and other objects, which are hard for it to converge on.) Alsing's method, (the binaries of which you can download on the first link I posted) does a lot better, performance wise, in this regard. I think this is due to the limitations his algo places: there are only polygons, no curves or additional parameters to mutate, and he uses a gene pool size of exactly one child. The slowest part of the system is the scoring mechanism. While my implementation scores an image just as fast as Alsing's, I am using a much larger gene pool size.

I've found that the hillclimbing method (I wrote a hillclimber plugin for my system as part of the initial testing) performs very poorly as soon as you introduce more "exciting" parameters, such as curve tension and what not. I found that once I started using more varied drawing expressions the hillclimber would tend to get "stuck" for very long periods of time.

I would love to optimize this to work with higher resolution pictures but can't really see a solution aside from buying faster hardware. One option I had considered was to use dirty-rectangles and only test changed areas of the image. Alsing might already be doing this, but I haven't reviewed the code closely enough as I was mostly interested in how he was doing the fitness testing. I've also thought about doing the comparison in the frequency domain instead of on each individual pixel, but this requires an FFT per candidate image, as well as the comparison, so it may actually slow things down. It might also produce some cool pictures.

There are some other areas where speed could be improved, but according to my profiler the major bottleneck is still just the fitness comparison. Doing one generation with a gene-pool of one-hundred on an 80x80 image would be the equivalent in Alsing's method of doing one 8000x8000 image.

Quote:
Original post by Kirl
Cool, mutating pictures is looking very impressive. Thanks for the link! :)

Do you plan to incorporate the help of the intarwebz as well? It seems somewhat more interesting then pulling heads from a database, plus you can train on specifics (male/female/child, or various objects) without much effort.

I think a large audience ensures a wide range of experiments and results.


Absolutely! Unfortunately I know next to nothing about web programming or how to structure such a thing. One issue with building an experiment around mass-user input, however, is that you need to make sure there are a bunch of people out there willing to click on stuff in your program. That said, I really do love the idea of hosting genetic apps that respond and learn based on user input from anybody who wants to "tell" it something.

As an interesting variation on the "mutating faces" idea, why not a "mutating grandma?" i.e. "How much does this look like your grandma on a scale of one to ten?" Eventually you would theoretically have a picture that looked like everybody's grandma. The mean grandma, so to speak.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.