Yesterday we had the opportunity of interviewing Bill Dally, the new Chief Scientist at NVIDIA, who replaced David Kirk several months ago. A specialist in massively parallel processing, Bill Dally brings important knowledge to NVIDIA in this domain. He has contributed to readying numerous technologies necessary for the structuring of massively parallel servers and launched startups in this domain such as Stream Processors, Inc. Moreover, before joining NVIDIA, he was head of the Computer Science department at Stanford where massively parallel processing was a major area of research.
The arrival of Bill Dally at the head of R&D for NVIDIA is then a clear indication of the company’s objectives: evolving from a 3D specialist towards a specialist in massively parallel processing with 3D “simply” one of the uses for this area. This transition could allow NVIDIA to place itself at the heart of future architectures and no longer as a supplier of an optional extra. More than simply a desire to change things in their favour, this is in fact more of a recognition by NVIDIA of an inevitable development. Bill Dally has then been appointed to ensure that NVIDIA has the right technology over the next 5 to 10 years.
We didn’t learn anything specific on NVIDIA’s plans or forthcoming products in our interview. Bill Dally has only just joined the company and only has partial knowledge of the current products and strategies and was happy just to repeat the offical company line. So there was nothing really new to learn there.
We were however able to discuss other matters. Billy Dally is reckoning on a processing power of 20 teraflops for GPUs by 2015, which corresponds to a doubling of processing power every two years. He also said that GPUs should also evolve to become more effective in the execution of less multi-threaded tasks, that’s to say exploiting parallelism at instruction level on top of data/thread level to which current GPUs are limited, in contrast to Intel’s forthcoming Larrabee.
When questionned on Intel’s competitive advantage, seeing as the CPU number 1 owns its own factories and can therefore potentially put new fab technologies into practice more quickly, Bill Dally said that this was a detail and that what was important was to develop the most effective architecture. Still on this subject and in respect of being able to use GlobalFoundaries instead of TSMC, Bill Dally said that he didn’t think SOI technology was worth pursuing as it is too expensive given the fact that it only brings a small advantage. As far as anything else goes anything is possible of course but both NVIDIA and AMD want to keep TSMC sweet for the time being as TSMC is currently their only partner. They therefore don’t have much to say at the moment on the subject of GlobalFoundaries.
Another interesting point made by Bill Dally with regard to projects at NVIDIA concerned making C for CUDA available on other platforms. NVIDIA however does not envisage supporting Intel’s Ct language, Bill Dally affirming that C for CUDA is currently the reference in the domain. It would seem that NVIDIA’s strategy will be to open C for CUDA to other architectures so as to maintain this position. Controlling, at least in part, the software side of forthcoming architectures is of course likely to help NVIDIA to position itself better and this makes it more advantageous in the long term to open up C for CUDA rather than to limit it to its own products to protect its smaller current market.
Bill Dally seems to take a very pragmatic approach to this and other subjects. Questionned on possible developments to be brought to the acceleration of Folding@home and similar apps, for example, he answered that this wasn’t a priority as these apps do not generate extra revenue for NVIDIA.