The Next Huge Leap in Computing Power?

Search for a Gamespot video of Tim Sweeney, he tries to answer some of these questions, one of the things he was saying was about stacking silicon vertically, stacking layer upon layer of chips, also a bit about quantum computing, I think it's pretty much certain that INTEL/AMD will find a way to keep increasing power until technologies like quantum computer are made commercially viable.

You can also sift through some of these, http://en.wikipedia.org/wiki/List_of_emerging_technologies , things like Memristors, Quantum dot screens, Spintronics, 3D integrated circuits, Transistors made from Carbon nanotubes (could run at 1Thz). But of course these could all take decades to commercialize.

The question then is where can 3D graphics go after full Path Tracing is possible in real-time. The answer would be, just more and more emulation of nature, to the extent where we have a 4D world, we have a real-time time aspect where nature would grow as it would naturally, for example you could watch grass actually growing, you could have extremely accurate weather patterns, extremely accurate AI and Physics instead of the many static objects and approximations we have. Turn everything into a full dynamic living world. But that would take an extraordinary amount of CPU power.

I think one of the things that will eventually happen is the GPU will disappear when CPUs are powerful enough to handle all computation, after all a GPU is just a partially, fixed pipeline, CPU.

But we are limited by our natural senses, we can view up to about 11million colours which we have obviously already passed, can't distinguish after about 70FPS, I think Tim Sweeney mentioned that 8k screens will be some kind of limit to our vision that we can't surpass(check the video).

Just pump more and more CPU power till we have a living, breathing world of CGI graphics standards. But there must come a limit after all we are limited by our own senses, unless we start to genetically engineer ourselves into super beings with extra senses.

The question then is where can 3D graphics go after full Path Tracing is possible in real-time.

I don't think this is where 3D graphics is going, and more importantly I don't think it would actually solve any of the important problems in real-time graphics. If we had full path/ray tracing right now, at very high speeds, it would not improve visuals at all. On top of that, the biggest drawbacks of that approach actually involve what happens with memory and memory bandwidth rather than computing power. Betting on memory bandwidth over computational power is not smart or savvy IMO.

The answer would be, just more and more emulation of nature, to the extent where we have a 4D world, we have a real-time time aspect where nature would grow as it would naturally, for example you could watch grass actually growing, you could have extremely accurate weather patterns, extremely accurate AI and Physics instead of the many static objects and approximations we have. Turn everything into a full dynamic living world. But that would take an extraordinary amount of CPU power.

Sounds cool on paper. Makes no sense in practice. As an exercise, go spend some time watching grass grow. Next and somewhat more interesting, watch a time lapse video of grass growing. Kinda cool to watch for a few minutes, but ultimately not an interesting development. More seriously, these types of problems aren't limited by compute power in the first place. We lack a far more fundamental understanding of the process, and those who think that enough CPU power and machine learning will "solve" the problem are going to be disappointed.

I think one of the things that will eventually happen is the GPU will disappear when CPUs are powerful enough to handle all computation, after all a GPU is just a partially, fixed pipeline, CPU.

I've been hearing that story as long as I've been listening to talk about graphics and hardware. Nearly fourteen years now. Yet the reality of the situation is that monolithic CPUs are less important than they ever were, whereas GPUs have revolutionized a wide variety of fields over the last ten years. Betting against the GPU is a foolish move.

I think one of the things that will eventually happen is the GPU will disappear when CPUs are powerful enough to handle all computation, after all a GPU is just a partially, fixed pipeline, CPU.

GPUs these days have extremely little amounts of fixed pipeline left in them. WIth compute shaders, they're now basically an extremely-SIMD, massively hyper-threaded, many-core, RISC co-processor, with hierarchical and/or NUMA memory and an asynchronous DMA controller to boot.

CPUs are never going to replace such a thing - unless it becomes a GPU, which won't happen, because then they won't run legacy code efficiently.

Most of our code is stuck on the CPU because we all learned a particular way of writing software, and not enough people have re-learned how to engineer their software for other kinds of hardware architectures yet.

We still assume that it's a good abstraction for any bit of our code to be able to have a pointer to any bit of our data, that RAM is one huge big array of data, and that the CPU can operate directly on that data.

In reality, CPU makers have done a tonne of magic behind the scenes to try and make that abstraction seem like it works -- they mirror RAM into small caches, otherwise our software would be 1000x slower, they run statistical prediction algorithms to try and guess which bits of RAM to mirror at what times, they reorder our code to try and hide memory access latencies, they speculatively start executing branches that might not actually be taken, and then insert invisible stalls and fences to ensure that after all this parallel guesswork behaves just like the hypothetical, serial abstract machine that your C code was written for.

Instead of wasting transistors on all that magic, the SPE design discarded it all and spent their transistors on adding more cores. The cores didn't do magic behind the scenes - instead requiring programmers to write code differently. Instead of a magic cache that sometimes makes RAM seem fast, when it works, they gave us an asynchronous DMA controller, which lets you perform a non-blocking memcpy between RAM/cache and then poll to see if it's completed. You have to explicitly tell the CPU in advance that you need to move some data from RAM to cache so that the CPU can operate on it, instead of having the CPU pretend that it can operate on RAM (when it can't) and doing guesswork to operate a cache for you. They removed the statistical branch prediction magic, and instead relied on the programmer annotating their if statements as to whether they were statistically more likely to be true or false. If you need to write some results to RAM, you don't need the CPU to write out each word one by one, waiting for them to be written to cache -- you can kick off an async memcpy and continue doing useful work while the parallel DMA hardware does the work in the background.

The result was a CPU that required a radically different approach to writing software (arguably better, arguably worse, just different), but offered a completely insane amount of performance due to the completely different design.

But we can't have such a thing because of inertia. We're stuck with CPUs continuing to emulate designs that we decided on in the 80's, because that's the hypothetical abstract machine that our programming languages are designed around.

GPUs on the other hand have inertia of their own. The popularity of computer graphics has ensured that every single PC now has a GPU inside it. Computer graphics people were happy to learn how to write their code differently in order to gain performance, giving the manufacturers unlimited freedom to experiment with different hardware designs. The result is a huge amount of innovation and actual advancement in processing technology, and in parallel software engineering knowledge.

Because they've been so successful, and aren't going anywhere soon (because CPU architectures are designed so differently that they can't compete for parallel workloads) everyone else now has a chance to learn how to write their programs in the ultra-wide-SIMD compute shader pattern, if they care to.

In the last generation of game engines, we saw systems that have traditionally been single-threaded (and there's a lot of people who were preaching "games are inherently single-threaded, multi-core won't help!") be replaced by multi-core and NUMA-compatible systems.
In the next generation of game engines, we're going to see many of these systems move off the CPU and over to the GPU compute hardware instead.

The fact is, that in terms of ops per joule, GPUs, and processors such as the SPE's are far, far ahead of CPUs by design. Note that the PS3's Cell CPU is, what, 8 years old now, but it still matches a modern Core i7 in terms of FLOPS due to a more efficient (but non-traditional-CPU-like) design...

I am very interested in advances in voxel rendering.

I'm not talking about the boring minecraft style rendering.

I've seen some incredibly demos recently and I think that's one path forward which has real promise.

The 'tech gurus missed one important thing: most computer users aren't tech gurus and don't play the latest and greatest PC games.

For them, computers got 'good enough' about ten years ago, honestly. Why upgrade when there's no need to? If you have a machine that will do eveything you need and want it to do, why spend more money for things that aren't going to help you any in what you do? GPU's could become ten times faster and hold sixteen times the RAM they do now. That won't make your word processor type any faster, or pop your email up quicker than it does now, or even display 99.99% of the webpages any better than they do now.

Why do you think people are holding on to XP (putting aside the issues with Windows 8)? It works. All their software runs. Life is Good. They aren't going to spend hundreds to a thousand dollars for a new computer (the only way they know how to 'upgrade' an OS) to do the same things they're doing today. As long as they can look at photos, listen to music, watch a movie, and do their email, facebook, and farmville, what more could they possibly want?

I agree with Mouser9169.

For the average PC user the technology got where it needed to be around the time of the Pentium 4 or Pentium D. Since then the speed increases don't really affect anybody who isnt a gamer or a power user. And even in games and 3d modeling / CAD there hasn't really been much for the past 5 - 8 years that has really required a massive increase in power.

If anything the main areas of research for computing tech is how can we get the stuff we have now smaller and using less battery power so that it can be used in a mobile or wearable device.

You can also sift through some of these, http://en.wikipedia.org/wiki/List_of_emerging_technologies , things like Memristors [...]

Interesting, these memristors. A quote from Wikipedia, under the "Applications" section:

They can potentially be fashioned into non-volatile solid-state memory, which would allow greater data density than hard drives with access times similar to DRAM, replacing both components.^%5B55%5D HP prototyped a crossbar latch memory that can fit 100 gigabits in a square centimeter,^%5B9%5D and proposed a scalable 3D design (consisting of up to 1000 layers or 1 petabit per cm³).^%5B56%5D In May 2008 HP reported that its device reaches currently about one-tenth the speed of DRAM.^%5B57%5D The devices' resistance would be read with alternating current so that the stored value would not be affected.^%5B58%5D In May 2012 it was reported that access time had been improved to 90 nanoseconds if not faster, approximately one hundred times faster than contemporaneous flash memory, while using one percent as much energy.^%5B59%5D

1 petabit per cm³, just think of all of the... erm.. things you could store!

I think one of the things that will eventually happen is the GPU will disappear when CPUs are powerful enough to handle all computation, after all a GPU is just a partially, fixed pipeline, CPU.

I don't think it will disappear but I do think they're heading for a convergence. GPUs are now essentially at the stage where they can do anything, and the new APIs are going to (help) unlock the lower-level power, but the biggest remaining bottlenecks are bandwidth and shuffling data between different types of memory (and even between different buffers in the same memory). That's the next thing that's going to need to fall.

I think one of the things that will eventually happen is the GPU will disappear when CPUs are powerful enough to handle all computation, after all a GPU is just a partially, fixed pipeline, CPU.

I don't think it will disappear but I do think they're heading for a convergence. GPUs are now essentially at the stage where they can do anything, and the new APIs are going to (help) unlock the lower-level power, but the biggest remaining bottlenecks are bandwidth and shuffling data between different types of memory (and even between different buffers in the same memory). That's the next thing that's going to need to fall.

I'd also like to point out that they are designed to solve completely different problems. A CPU is optimised for solving a variety of sequential problems, while the GPU is optimised to solve an incredible amount of identical parallelisable problems.

Saying the "GPU can do everything" may be true, but it doesn't mean it's efficient or correct. The GPU will never be used for I/O operations, nor will it ever be used to handle code execution of today's software, or anything of the likes. The GPU is inefficient if you try to make it execute threads that don't do the same thing, or require approximately the same time to finish.

I also don't see the CPU taking over the GPU's job, because again, the CPU's architecture, including the RAM located around it, just isn't designed for graphics processing.

Is there a standard emerging that indicates what parallel / concurrency programming will look like in the future?

From the Parallella discussions I got the impression that a lot of effort has to go into very tailored concepts / solutions.

Will programmers need to get good at designing for concurrency ... or are there concepts that might put the parallel execution under a hood of some kind ... like automatic shared-memory multiprocessor systems for example? Or are those inherently inefficient?

I'd also like to point out that they are designed to solve completely different problems. A CPU is optimised for solving a variety of sequential problems, while the GPU is optimised to solve an incredible amount of identical parallelisable problems.

This.

GPUs are fast at what they do because they can run 32 or 64 threads in lock step AND many groups of this to hide latency. A high end GPU today keeps 1000s of threads in flight at once to keep the beast fed which makes them very good at embarrassingly parallel tasks and latency hiding but.. erm.. that's about it.

With 'branchy' code or code below a certain threshold of parallelism CPUs become better suited to the problem as they can run a single branchy thread fast. Not to say we couldn't do better here, out-of-order execution and intelligent prefetches hideth a multitude of programmer sins after all but they are still better than launching a single wave front on a single CU on a GPU to use a single thread of that wave front to do some work.

Really, what we also need, is a third class of processor in the same line as a SPU; something which can chew data quickly but can branch reasonably well and doesn't require loads of threads to keep it busy. Certain workloads would suit this kind of processor nicely, like they did in the PS3 days, without all the overhead of the CPU 'guessing' and the GPU launching more threads than required.

But even without that third class 'the future', as it were, is already here with CPUs with iGPUs attached; really what is needed is for them to become useable as just ALU arrays without a display attached then software can begin making better use of them.

The final problem however is what it has been for some years now; memory.
Memory is too slow by clock cycle standards with L1 and L2 taking between 3 and early/late teens of cycles to fetch from and god forbid you miss the lowest level of cache and outfox the pre-fetcher as 100s of cycles then go missing while you stall for data.

The next leap, or at least improvement, really needs to come from the memory side of things because that is forming the biggest bottlenecks these days - we are drowning in ALU power, just can't get the stuff to work on in the right place at the right time.

The GPU will never be used for I/O operations

Depends on the kind of I/O you're talking about modern GPUs have a dedicated asynchronous DMA controller, which in a HUMA system, could be used by the CPU to implement an asynchronous memcpy operation. Some of them can even do higher level processing as they move data around -- such as read a blob of bytes from this mapped address, run JPEG decompression on those bytes and write the resulting pixels to this other mapped address. Streaming linearly-indexed pixel arrays into a destination that uses tiled-indexing is another complex IO op that the DMA controller might be able to do for free.

With 'branchy' code or code below a certain threshold of parallelism CPUs become better suited to the problem as they can run a single branchy thread fast.

Regarding branchy code - modern GPUs can almost branch for free now a lot of the time (the branch setup happens in parallel with your ALU, so usually you'd be bottlenecked by ALU and get the branch setup at no extra cost) -- with the obvious caveat that if half of a SIMD vector takes one path and the other half of the SIMD vector takes the other path, then you've got to execute both. This is the same as with SSE or AVX code on the CPU though (e.g. such as in ispc, which is a really cool language BTW), except the GPU probably handles these cases better and that your CPU is probably 4, 8 or 16 wide, while your GPU is probably 32 or 64 wide.

Really, what we also need, is a third class of processor in the same line as a SPU; something which can chew data quickly but can branch reasonably well and doesn't require loads of threads to keep it busy.

No consumer PC will have one... but does the Xeon Phi (Larrabee) fit the bill? It sounds like an x86 version of the SPEs, with way more cores, more memory per core, 16-wide AVX and 4 HW-threads per core.

Is there a standard emerging that indicates what parallel / concurrency programming will look like in the future?
From the Parallella discussions I got the impression that a lot of effort has to go into very tailored concepts / solutions.

Will programmers need to get good at designing for concurrency ... or are there concepts that might put the parallel execution under a hood of some kind ...

A skill-set that is relevant now and into the future is writing functional style code -- this doesn't mean you have to run off and learn Haskell (I sure haven't!), you could keep using C, or pretty much any language, as long as you can get into a situation where you always know exactly what ranges of data are being used as input, and what ranges of data are being used as outputs at any one time.
Pure functions are implicitly thread-safe (assuming their input and output buffers aren't also being written by another thread, of course).
Structuring your code as a big directed acyclic graph of [Inputs -> Process -> Output] nodes means that it will be portable to pretty much any kind of parallel CPU. If your game frame is made up of thousands (or tens of thousands) of these little jobs, then you can run your game on a single-core up to a 32-core (about the best a consumer can find atm) and it will keep scaling.
Pretty much every game engine I've used in the PS3/360 generation has at least started to transition to supporting this style of processing, as it will run well on old desktops, new desktops, the PS3's SPUs (assuming sizeof(inputs)+sizeof(outputs) is < ~128KB...), the 360's tri-core, and next-gen consoles.

This is also how pretty much all graphics and compute programming works on GPUs - with the extra requirement that your processes should map well to a SIMD processor... but to get the most out of a modern CPU your workloads would ideally all map well to SIMD as well - it's just not quite as important as with a GPU.

Quantum computing would of course be useful, in a generic, all-embracing sort of way, but I don't think we'll be seeing quantum coprocessors anytime soon. Most likely the tech will start out slow, expensive and flimsy, just like computers were in the 50's (quantum computing isn't even born yet, it's trying to but there are still colossal engineering challenges to overcome). If one day it becomes as accessible and user-friendly as, say, a modern GPU, then there would definitely be applications in many different areas of software development and computer science in general, and once the tech is mature some problems would naturally be very well-suited to quantum computing and would see a huge speedup. But of course quantum computing in and of itself is not a substitute for classical computing.

I'm thinking the future is probably going to tend towards a unified software/hardware interface for high-speed serial processors (CPU's), massively parallel compute units (GPU's), and any other hardware devices, that will be able to interoperate with one another without having to go through a centralized control unit like we have now. Or something. The point is these hardware devices are not equivalent, they solve relatively different problems, they work well as a hybrid solution but are no good individually, so it makes sense to capitalize on this and try to bring them closer rather than try and turn one into the other.

Depends on the kind of I/O you're talking about wink.png modern GPUs have a dedicated asynchronous DMA controller, which in a HUMA system, could be used by the CPU to implement an asynchronous memcpy operation. Some of them can even do higher level processing as they move data around -- such as read a blob of bytes from this mapped address, run JPEG decompression on those bytes and write the resulting pixels to this other mapped address. Streaming linearly-indexed pixel arrays into a destination that uses tiled-indexing is another complex IO op that the DMA controller might be able to do for free.

Your example with the JPEG decompression is interesting, but I see one flaw. The JPEG file on the storage medium is accessed sequentially (assuming you wish to decompress it while streaming it from disk), so there is no benefit in doing parallel decompression. That is of course assuming that access speed to the storage medium is far slower than the GPU, which is currently the case.

If the uncompressed JPEG file were already loaded in memory, then yes, a hUMA system would greatly benefit from GPU accelerated decompression. One might even go as far as to say decompressing it on the fly during every refresh of the screen is more efficient than keeping the decompressed form in memory, i.e. saving memory at the cost of some speed.

My original quote was directed more towards I/O operations such as access to storage mediums, keyboard/mouse/joypad input, etc.

My original quote was directed more towards I/O operations such as access to storage mediums, keyboard/mouse/joypad input, etc.

And yet they perform I/O quite well. I/O is not limited to those few devices. Many devices are primarily output: printers, sound cards, radios, haptic devices, and yes, graphics. You probably have a half dozen output-only I/O systems on your computer right now, probably with at least 3 in use.

The newest video cards supporting 4k video or DisplayPort protocols can handle about 6GBps output (or about 50Gbit, if you prefer). As far as I/O devices go, video cards are usually the fastest I/O devices on a computer.

Your example with the JPEG decompression is interesting, but I see one flaw. The JPEG file on the storage medium is accessed sequentially (assuming you wish to decompress it while streaming it from disk), so there is no benefit in doing parallel decompression. That is of course assuming that access speed to the storage medium is far slower than the GPU, which is currently the case.

It doesn't use the GPU's parallel compute units (multi-core/SIMD), it's done by the DMA unit.
It's parallel in the sense that the CPU can request a JPEG to be loaded into a pixel array, and the DMA unit does the work asynchronously. The CPU can go off and do other useful work in the meantime while the data is being streamed from disk into the pixel array (with the actual JPEG decompression logic incurring zero cost on either the CPU or the GPU's compute/graphics units).

phantom, on 09 Apr 2014 - 2:05 PM, said:

Really, what we also need, is a third class of processor in the same line as a SPU; something which can chew data quickly but can branch reasonably well and doesn't require loads of threads to keep it busy.

No consumer PC will have one... but does the Xeon Phi (Larrabee) fit the bill? It sounds like an x86 version of the SPEs, with way more cores, more memory per core, 16-wide AVX and 4 HW-threads per core.

Can't the Xena coprocessor on the Amiga X1000 also be used in this fashion although nowhere near as powerful.

I know Euclideon isn't a very popular company here, but I personally believe they are most likely the "Next Huge Leap". They have demonstrated their tech is real for Geospatial and more than 15 companies have signed on to use their technology, so it isn't fake. When it comes to the gaming world or more appropriately, "IF" it comes to the gaming world, I think that's the biggest leap we'll see.

The second thing I believe could be the "Next Huge Leap", would be stacked chips. The fact that mobile has created such a huge divide between what is needed for phones/tablets and what people need for PCs is being blurred by SoCs by the day. Tegra, A Series APUs, Atom...etc., are all being designed for wider application stuff now. If stacking chips becomes truly the route everyone takes, then every device sees improvement, without such a divide anymore.

The third thing I believe could be the "Next Huge Leap", would be the cloud. If stacking chips becomes the route to take, which I believe almost everyone agrees with, then the switch to more CPU bound processing would greatly be benefited from the cloud.

Microsoft built DX12 and apparently focused more on CPU utilization more so than GPU kind of implies that may be the route alot of people take in the future. If they believe the cloud would work hand in hand for everyday use, that could be a huge leap.

Personally, I think if Euclideon is real, which they've stated is run entirely on the CPU, combined with stacked chips for CPUs and throw in Cloud Computing for good measure to offload CPU tasks to, would be not just a huge leap, but the "Ultimate Leap".

I know Euclideon isn't a very popular company here, but I personally believe they are most likely the "Next Huge Leap".

Meh, they still haven't actually released anything. Their "unlimited detail" as described could be easily implemented using CLOD techniques developed decades ago. Preprocess the voxels into view-dependent bricks and load them up when needed. Their first demos were in 2003, back when preprocessed CLOD and geo-morphing was still the best solution as programmable 3D cards were just barely released. These days the preprocessed tree generates 3D textures with a fancy shader and accomplishes the same thing.

Maybe once they actually release a product we can see how awesome it is. Until then it remains vaporware ... 11 years after being announced.