The holy grail of computation

Not long ago I was in a conference for high-performance computing where the "big guys" were talking about the future of computer architectures.
By now it is basically clear that there is no way, in the near future, to improve CPU's clock because of power and heating constraints. Everybody is going to multiple CPU's in a chip and multicores became the word of mouth and also a word to fear.
Whoever builds processors fear that multicore will not be able to bring a better user experience at our homes. Ohhh gosh, sales will go down, no volume, prices up and it will be the end of computer industry the way we know it. We must save it.

Computer architects think that if programmers were not so dumb they could write their code thinking in parallel and thinking a bit more in the computer architecture. Programmers are spoiled and the architects needed to put all the improvements transparent and dumb-proof. As programmers are so badly prepared for the task, somebody (maybe even us superior computer architects) will need to develop a layer that will abstract the parallelization complexity, taking advantage of several cores, saving the whole computing business and everybody will be happy ever after. We just can't really find out how.

Programmers do not give a damm for what computer architects think. They are a thousand abstraction layers above, over their java/perl/ruby virtual machines, creating applications, web-pages and database queries through some persistence layer and using high-level libraries.
I don't care about what sort of pieces my car engine is composed as far as the car goes well and gives me no trouble. I was very happy to get some extra performance without changing any piece of my code and this is the way I would like to keep being once I have much "real" work to do in order to keep my low-paid job anyway.

In-between we have the guys who develop middleware like compilers and libraries. And to these guys there is no money or recognition. Nor architects neither programmers really give much value to these guys. When was the last time you were asking yourself about which compiler to buy?
A middleware would be the solution to everybody and is the holy grail of computation nowadays. Everyone would like to build a compiler/library/run-time environment or any sort of layer that would enable not many or no change at all in the programs but would enable our software to scale in performance the more cores I have in my computer.
And it is funny to hear everyday a big company announcing they found the holy grail. Then you read about it and get into the details. This middleware is then either so limited or so hard to adapt your application or it just works for a couple of matrix multiplication implementations (exactly the ones they use for de demo).
My opinion is that this middleware does not exist such as we are searching for. This time we will need to rethink the way we do things and I believe this will happen from every layer simultaneously. From the hardware up to the VMs.

We are in a quite exciting moment where big players need to bet for their survival, where the old formula does not work anymore.

The "dumb" programmer whose software sells more if they use the multi-core computational power are, "surprisingly", adapting their applications and it works. Maybe programmers are not so dumb, they just follow the money, as anybody else.

Architects need to solve their computation bottlenecks and deliver more with every new processor generation. New levels of cache, different coherence protocols, placing new buses or changing them to use photons, using non-uniform memory access, stacking memory over the processor chip, reusing components through threads, or specializing the different cores.

In the middleware we have the traditional parallel computing libraries trying to go mainstream but failing, transactional memory, new innovative ways of simplifying working with threads. And money, quite some money, for the ideas that could unify it all in a simple and elegant way. Everybody is playing their cards and companies are financing the big universities and their own labs.

And you, what are you doing about it? Are you adapting your applications to the new era or just watching the battle?