I'm a programmer. There's this terrible term in the tech industry called “Programmer Art”. The underlying idea behind the term is that if you are a programmer, you can't be an artist. Or at least, if you tried, you wouldn't be a “good” artist. I have found this to be a toxic mindset which serves to only stunt your own growth and abilities. In the past, my attitude was “I'm a coder, I'm not good at art, I should focus on what I do best which is writing code and designing systems, and if I have to work on art, then there is a systemic project management problem and something is wrong. Therefore, I shouldn't waste my time getting good at art because it's a weakness of mine and I should just play to my strengths.”

In past projects, there was a line which I didn't cross. If I have to open up Maya, which has a very intimidating UI, then “I'm doing it wrong!". The attitude was that I should buy art assets or hire people to create them for me. I would occasionally skirt the line though: I might open up Wings3D to create some throw away prototype models, but I didn't know how to UV unwrap and texture them, so they'd just get vertex colors or assigned material slots. I'd take existing skeletons for characters and use UE4 animation blueprints to create some pretty intricate, dynamic animation systems, but god forbid I have to open up a 3D authoring tool to edit bones or mesh weights. This self imposed restriction I placed on myself was starting to limit what I could do and put limits on what I could build.

A part of it has been from a project management perspective: Making art look good takes days and weeks of pain staking effort. Buying premade art assets is a more prudent use of my valuable time. But, the fine print nobody reads says “premade art assets are enough to get you 80% there and you still have to do the other 20%”. And… in my experience, getting that last 10-20% takes 50% of the time no matter how good you are.

The other part has been a general reluctance to learn new things. Learning is hard. It is a struggle. If you're an expert in one thing, learning something new makes you a novice at that new thing and you have to repeat that same struggle to get good again. That learning process can be overwhelming, intimidating, frustrating and take a really long time to get to the quality bar you're aiming for. It's no surprise that I've been a bit reluctant to learn the 3D art production pipeline.

In the past few months, that has changed. I've been working at facebook as a contracted graphics programmer for nearly a year and a half (they still haven't fired me yet!). If I need an art asset to do something different, what are my options?
1) I could find a different art asset which does what I need. But if its very specialized, that's nearly impossible.
2) I could find someone with the necessary skill set to do it for me. But who are they, where do I find them, and how busy are they? What if I can't find anyone who has the time and skills?
3) I could negotiate a way to change the requirement so that the task doesn't need to be done as defined. But that's a shitty work around / cop out, and isn't always an option. Especially if its a hard requirement.
4) I could take some time to develop the skill set and do it myself. But doing everything myself isn't scalable or always the most valuable use of my time.

For the most part, I've done everything but #4. But when nothing else is an option, I am forced to learn and do art. I think that's a ‘good thing’ because it gives me a richer understanding of the art production pipeline. I looked at all sorts of 3D authoring tools and what my requirements were. Money isn't really a problem for facebook, so if a software license costs $5000 or something crazy, they'll spend the money to get whatever tool is required to get the job done. But, there's still a procurement process to deal with. I looked at our requirements and I made a list of all the candidate software tools out there. What could help me get the job done the fastest, with the least amount of pain, while also being robust enough to support the more advanced capabilities I might need later? And, what skill set and software platform would be the most valuable to know for future projects? After looking at everything available, I landed on Blender. Holy cow, has that tool come a long way over five years! I remember it being a buggy UI nightmare for starving indie artists, but now… its polished, feature rich and free! The pitch is compelling: “Before you go out and spend $4500 on Maya, try Blender for free! If you don't like it, you don't lose any money anyways so what's the harm?”

So, I downloaded and installed Blender, watched hours of Youtube tutorials, and got half proficient at using Blender. What's really amazing about modern times is that you can find video tutorials to teach yourself anything you want by listening to experienced experts from around the world. I even watched a video created by some guy in Pakistan on how to make a character drink liquid from a coffee cup. You could hear background farm noises from his village. I didn't understand a word he was saying, but I could still figure out what was going on. I learned something valuable from someone on the opposite side of the world! How neat is that? Knowledge and skills are no longer gated behind universities and libraries. This stuff is available at your fingertips for free. No more taking expensive classes and getting knowledge delivered piecemeal!

I also taught myself x64 assembly by following some youtube tutorials. I have a passable knowledge of the concepts and am pretty excited about the performance possibilities with SIMD instructions. I've gotten comfortable enough with reading disassembly code and looking directly at memory and register values that I feel I can now debug anything – at least on a local machine. Trying to debug non-deterministic bugs on an enterprise scale architecture with possible race conditions is a whole different beast (sorry for any nightmares I just invoked).

I'm not going to identify myself as a “Programmer” or "software engineer" anymore. Instead, I'm going to call myself a “Developer”. Programmers and SWE's box themselves into a tightly defined role and don't step outside of it. A developer will do whatever it takes to build something because their role is broad enough to encompass everything in production, from art to audio to code and QA. There's no such thing as “programmer art”, it's just art made by a developer.

Anyways, why talk about all of this? Why go through all the trouble to learn the whole 3D art pipeline while also talking about high performance assembly code?

I believe there's a graphics and art production revolution coming up in the next 5-10 years called “Digital Humans”. We're starting to get wiffs of its scent, but I think for the most part, the wider tech and entertainment industry have not caught onto this upcoming revolution and the impact its going to have. The general idea is that we can create hyper realistic 3D models of a human being to such a level of detail that you can't tell the difference between the original and the digital replica. We can model every eye lash, every freckle, every pore in your skin, even the peach fuzz on your face, the subsurface scattering of light on skin, light transmission through thin skin membranes, hyper realistic hair simulation, and all of this can be driven by animations using fine tuned rigs and mocap data streamed in live from a performer. You can capture a performance by some actor/performer, which then gets replayed on that hyper realistic digital human in virtual reality. Thanks, UE4/5!

This stuff is all still very early, but I'm seeing the puzzle pieces starting to take shape and how they all fit together. You view and interact with these digital humans in virtual reality. You feel like you're in a shared environment with that character. You get that holy grail of “social presence”. AI driven by machine learning neural networks could be used to render and play subtle details, behaviors and gestures. How does the skin wrinkle around the eyes when someone smiles? How does the gaze slightly shift to give it a life like behavior pattern? AI can also be trained on voice patterns and speech mannerisms to emulate the speaking style of anyone, such that it sounds like the original person is speaking. So, we end up having a digital human who looks like the original, speaks like the original, moves and behaves like the original, and… what else is there left for the original human which the digital human can't do? Give hugs and provide original thoughts? What makes us who we are when all of it can be reproducible in a synthetic representation of ourselves?

I foresee some future legal and philosophical wrestling matches arising out of this in the coming decade. Deep fakes will be indiscernible/undetectable. They're pretty good now, but just wait. If deep fakery becomes common and rampant, how will people know what is authentic and what isn't? Not that there's ever been much trust in politicians, but could you trust what you hear them say on live TV? Hearing it straight from the horses mouth won't have much authentic truth value in the future if you don't know if the horse is even real anymore. There's also the upcoming issue of maintaining likeness rights over your own identity. If I create a digital avatar of my own likeness and put it out into the public, anyone could take a copy of me and make me say anything they want, even if it doesn't align with my own values and opinions. Maybe my digital copy is used to claim that I have some horrible, contagious, incurable disease and only some pharmaceutical product can alleviate the symptoms. Or maybe it's used to endorse and support hate groups. Or maybe someone creates doctored video testimony of me plotting some nefarious crime and it gets used against me in court – how could I refute what looks to be a video confession by me? What if most Hollywood actors get their likeness converted into digital humans and all of a sudden, their likeness stars in 15 movies in a year, yet the actor never set foot onto the set and got paid? Acting changes from how you look into how you act and deliver a performance. I'm sure I'm just skimming the wavetops on the future issues, challenges and implications here, but its good to try to anticipate future problems now so that we can start working out safeguards and protections now while we're designing them.

I can see a bright future for digital humans in the next 5-10 years, and anyone who masters the production pipeline today will be running the show tomorrow and commanding the future. It's a part of why I'm diving into the art production pipeline as well as the high performance tech art side. I think my first goal is to eventually create a digital human of myself. Maybe my digital copy can stand on a stage and deliver this as a TED talk, only to reveal at the very end that he isn't real?