N1005 IBM Research: bluegene

Data Centric Systems: A new paradigm for computing

<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGrSfilcCHKp59XyfPzV0mYH7RVaMZY0hFGhqgLB0ZlIULvG1GAoECe-1nbY2mPzvFqvuaYcg91AC_HXZcjGuS806yaJg2CoqWYpZtSrh5JF3bw0Ta6V_swM8TYrLQvee3qrTMoXPp5xI/s1600/tilak.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGrSfilcCHKp59XyfPzV0mYH7RVaMZY0hFGhqgLB0ZlIULvG1GAoECe-1nbY2mPzvFqvuaYcg91AC_HXZcjGuS806yaJg2CoqWYpZtSrh5JF3bw0Ta6V_swM8TYrLQvee3qrTMoXPp5xI/s1600/tilak.jpg" /></a></div>Editor’s note: This article is by <a href="https://www.linkedin.com/pub/tilak-agerwala/21/aa5/a63">Tilak Agerwala</a>, Vice President of Data Centric Systems at IBM Research The use of high performance computers (HPC) to model, simulate and analyze everything from astrophysics to zoology goes back to the dawn of the modern computer era. Since the 1950s, the models studied by HPC systems have increased both in scale and in detail with ever more sophisticated users calling for – and planning on – increased computational power. This increase is expressed in the form of Floating Point Operations per Second or FLOPS. Aggregate installed FLOPS, as measured by the Top500, have increased exponentially since tracking began in 1993, going from less than 60 gigaflops to nearly 300 petaflops, today. And the demand for increased FLOPS is not likely to abate for the foreseeable future. However, we are now seeing a new trend emerge that will dramatically change how HPC system design moves forward – the emergence of <a href="http://www.forbes.com/sites/ibm/2014/06/30/why-big-data-is-the-new-natural-resource/" target="_blank">data as the world’s newest, and arguably largest, natural resource</a>. Today’s businesses, academic institutions and government agencies continue to have a multitude of applications for HPC, from fraud detection in transaction-based systems like credit cards serving millions of users simultaneously, to computational simulations the human heart beating in real time cell by cell. But unlike 60 years ago, these models now support a trillion fold more data – leaping from kilobytes to petabytes, and growing. This onslaught of this structured and unstructured data on HPC requires a flexible computing architecture capable of addressing the growing needs for the workloads this data scale demands. In order to prepare for the future, IBM has adopted Data Centric Systems (DCS) design as our new paradigm for computing. <div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaowisfBHfhFZODxpjE0HH7kbrB_pmyaPOkvZYtArp3oDhymDof4dEIgo-2dAZFhrdjboZ3hAEMz8dDiTP7QNq_sLHBJKzJwGDpakdPy6b4apCP17qLakZciZBeDlGXET-oJq2lEopYhU/s1600/DataCentricSystems6.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaowisfBHfhFZODxpjE0HH7kbrB_pmyaPOkvZYtArp3oDhymDof4dEIgo-2dAZFhrdjboZ3hAEMz8dDiTP7QNq_sLHBJKzJwGDpakdPy6b4apCP17qLakZciZBeDlGXET-oJq2lEopYhU/s1600/DataCentricSystems6.jpg" /></a></div><div class="separator" style="clear: both; text-align: center;"></div>The rationale is simple. As the size of data grows, the cost of moving data around becomes prohibitive. We must employ distributed intelligence to bring computation to the data.  How Data Centric Systems is like a supply chain To illustrate this, consider the evolution of today’s sophisticated supply chains. Today, detailed engineering, design and development work is taken on by highly capable members of the supply chain. Production complexity is distributed by processing components outside of the final manufacturing plant. These are finally brought together just in time for final assembly. This distributed application of specialized skills reduces waste in transport, and also results in a higher level of quality, efficiency, and innovation throughout the manufacturing work flow.  Like today’s modern supply chains, our next-generation systems will minimize data in motion. As data volumes grow exponentially the cost (power and time) of moving the data in and out of a central processor is becoming prohibitive. So why not move data less by running workloads where the data lives? Doing this will require “computing” at all levels of the system’s hierarchy by introducing active system elements such as network, memory and elastic storage. The architecture will also need to be composable, based on modular and upgradeable hardware design, and scalable from sub-rack to hundreds of racks.  Finally, it must be on an open platform, like our <a href="http://openpowerfoundation.org/" target="_blank">OpenPOWER</a> which allows deep innovation by partners at all levels of the ecosystem. Data Centric Systems will be future “Systems of Insight” In 2012, <a href="http://researcher.watson.ibm.com/researcher/view_group.php?id=2992" target="_blank">Project Cardioid</a>, a partnership with Lawrence Livermore National Lab, used Sequoia, the largest Blue Gene/Q system we've ever delivered, with over 700,000 cores and sustained performance of 12 petaflops, to simulate an entire human heart at nearly the cellular level. This groundbreaking simulation modeled 370 million pieces of the human heart, 1,200 times faster than previous, less-accurate simulations. Going from months to near-real time simulation of thousands of beats is helping scientists understand disease, design and test drugs, as well as simulate other human organs. <table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAa6IPw7oWiucHQaHKuqrKek8KhPqijRPafdSnrfrzvkza3CwGHP9RgiHfybiVpYZnTT589Pnq3WQaO-Jsj2yuFVXsD2z-LPSGDQqusoAQ1rSkbKT7kfWj7W0lBlt06m_D33kj7NXiHEw/s1600/seq_cupps_bertsch_high-res.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="334" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAa6IPw7oWiucHQaHKuqrKek8KhPqijRPafdSnrfrzvkza3CwGHP9RgiHfybiVpYZnTT589Pnq3WQaO-Jsj2yuFVXsD2z-LPSGDQqusoAQ1rSkbKT7kfWj7W0lBlt06m_D33kj7NXiHEw/s1600/seq_cupps_bertsch_high-res.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">BlueGene/Q Sequoia at the LLNL</td></tr></tbody></table>Let’s fast-forward a few years from now to exascale computing and full DCS systems, and how they can be applied for everything from weather forecasting to finding oil and gas reserves. New sources are increasingly difficult to find and access as companies explore increasingly remote and challenging geologies, like the Arctic and the deep ocean. New sensor technologies provide a lens into these deeply buried hydrocarbon reserves, generating vast amounts of data.  While the data enables us to build much higher-resolution models, like with the heart simulation, the opportunity in this industry is in creating multiple simultaneous, and coupled simulations, to ultimately characterize the reservoir at the molecular level. With this level of insight oil and gas recoveries will be optimized over time. But getting to this level of fidelity will require 100x the compute power of today's fastest, largest systems. We can begin to tackle this type of problem with current systems but to get reach the goal will ultimately require Data Centric Systems. True Data Centric Systems will be realized when we can consolidate modeling, simulation, analytics, Big Data, and machine learning and cognitive computing as “Systems of Insight.” They will provide these capabilities faster with more flexibility and lower energy costs, effectively ushering in a new category of computers. For more details <a href="http://icpp.cs.umn.edu/agerwala.pdf" target="_blank">download</a> my keynote presentation at the 2014 International Conference on Parallel Processing.

Editor’s note: This article is by Tilak Agerwala , Vice President of Data Centric Systems at IBM Research The use of high performance comput...

Rethinking HPC Benchmarks with a Focus on Applications and Power Efficiency

12:46 AM N1000 0

<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody><tr><td style="text-align: center;"><a href="https://farm6.staticflickr.com/5550/14464078686_bc3197b04a_z.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="213" src="https://farm6.staticflickr.com/5550/14464078686_bc3197b04a_z.jpg" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">From left to right: HPC scientists Costas Bekas, IBM; Pavel Klavik,IBM and Charles University in Prague; Yves Ineichen, IBM and Cristiano Malossi, IBM</td></tr></tbody></table>Today with the release of the <a href="http://www.top500.org/blog/lists/2014/06/press-release/" target="_blank">latest TOP500 supercomputing list</a> we see the largest and most powerful high performance computers in the world. And while it is impressive, it’s quickly becoming an antiquated measuring stick for several reasons including how to measure workload and power efficiency. Since 1993 the TOP500 list has been published to portray an exponential increase in performance, due to Moore’s law, but in the last 10 years, the core technology has dramatically changed with chip manufacturers turning to a recipe of multi-core chips and parallelism to keep the effect of Moore’s Law alive. In doing so a new set of critical issues and questions have arisen about the ability to program such machines and how to deal with the very fast increase in power requirements. As the industry moves towards more <a href="http://www.bbc.co.uk/news/business-15151431" target="_blank">data-centric computing workloads</a> IBM scientists believe that the answer to both questions are closely connected and have recently published a paper in the <a href="http://arxiv.org/abs/1405.4644" target="_blank">Proceedings of The Royal Society A</a> on this topic. Some of the paper's authors answered a few questions about their motivation. Your paper mentions the shortcomings of the FLOPs performance metric. What are they? <a href="http://researcher.watson.ibm.com/researcher/view.php?person=zurich-bek" target="_blank">Costas Bekas</a> (<a href="https://twitter.com/CostasBekas" target="_blank">@CostasBekas</a>): The main shortcoming of the FLOPs metric is that it concentrates on doing as many computations as possible on a given piece of hardware in the unit of time. This is not about trying to solve the problem quickly, but essentially measuring how many FLOPs we can squeeze out of the machine. This metric stems from the days when computing was very expensive — so the thinking was you better use every last transistor. But today with the cloud and massive high performance computing (HPC) machines the equation has flipped and energy is a significant cost factor. Today, the real issues are the results. Do you get the results fast and how accurate are they? So the question clients need to ask is, do you care that you run a machine to its full "nominal" capacity or that you get the results back as fast as possible and at the lowest possible cost? The answer lies in both hardware improvements, such as water-cooled design and photonics, combined with advanced algorithms. What should a new benchmark look like for measuring high performance computers? <a href="http://researcher.ibm.com/researcher/view.php?person=zurich-ACM" target="_blank">Cristiano Malossi</a>: The biggest challenge will be to find a metric which is accepted by everyone. From what I hear at various conferences is that the industry is in agreement that the LINPACK benchmark isn’t sufficient anymore. The <a href="http://www.graph500.org/" target="_blank">GRAPH500</a> is a good start, but we need to take it a step further. I see the need to have more than one metric, in fact 13 or one for each kernel, which are then weighed, scored and averaged. This is the bottom-up approach popularized by <a href="https://en.wikipedia.org/wiki/Phillip_Colella" target="_blank">Phil Collela's (or the Berkeley) dwarfs</a>. Costas: The big drivers are the HPC applications. The nature of applications has changed and there are hundreds. In the past the focus was numerical linear algebra, but today applications span bioinformatics, finance, traditional engineering, and data analytics. A new benchmark needs to account for speed to result and energy efficiency to result. Besides providing a ranking tool for marketing purposes, do benchmarks also help the development of HPC? Costas: Absolutely, the rankings provide scientists and engineers with a guide on designing the future systems. For instance, here in Zurich we worked at a very early stage with the prototype of the <a href="http://www-03.ibm.com/ibm/history/ibm100/us/en/icons/bluegene/" target="_blank">IBM BlueGene/Q</a>. The goal was to incorporate in the design loop as soon as possible feedback from application developers and algorithm specialists. So a benchmark is more than just touting first place, it helps to close the circle between developers and users. We shouldn’t forget that benchmarks are also important for politics and financial officials. These machines require budget after all and when using tax payer funding you need to demonstrate results and coming in the top of a benchmark is one way to do this. But a better way to demonstrate the ROI of a system is to solve a problem quickly and efficiently — designing a new drug, simulating a city to make it run more efficiently or engineering a safer product. Ask the average citizen if they prefer a blue ribbon for first place or a city with less traffic and the answer is obvious. Your paper proposes a new benchmark based on application, time to result and energy efficiency. How can HPC users begin testing your benchmark on their systems? Costas: In this <a href="http://arxiv.org/abs/1405.4644" target="_blank">publication</a> and our <a href="http://rd.springer.com/article/10.1007/s00450-010-0119-z#pa" target="_blank">previous paper</a> are the starting points. Together with my colleague, <a href="http://researcher.watson.ibm.com/researcher/view.php?person=zurich-cur" target="_blank">IBM Fellow Alessandro Curioni</a> (<a class="ibm-twitter-link" href="https://twitter.com/Ale_Curioni" target="_blank">@Ale_Curioni</a>) we have been evangelizing the need to move away from using the LINPACK benchmark alone for several years. This paper provides a framework and tools for accurate, on-chip and on-line power measurements in the context of energy-aware performance. The tools described the easy implementation of user code for the detailed power profiling. Beyond just the benchmark also describe a <a href="https://www.google.com/patents/US20120005247?dq=alessandro+curioni&hl=en&sa=X&ei=6CGYU5W2Ncf2O8qogNAJ&ved=0CCIQ6AEwATgK" target="_blank">patented</a> framework that enables users to combine a bulk of relaxed accuracy, non-reliable, calculations together with a small fraction of accurate (reliable) calculations in order to achieve full final accuracy. The result is up to 2 orders of reduction in energy to solution. They key is to change the computing paradigm. The key takeaway is the following. Within 10 years we will have HPCs running billions of threads. For comparison, our world record work last year, which led to the <a href="http://www-03.ibm.com/press/us/en/pressrelease/42490.wss" target="_blank">Gordon Bell Prize</a> (together with Alessandro and colleagues from ETH Zurich, Technical University of Munich and the Lawrence Livermore National Lab) was using 6.4 million threads. So with a billion you have tremendous scale, therefore you need to develop algorithms which can recognize failure, resolve failure, but also live with it for a reliable result. Where does academia fit into this new benchmark? Clearly we need to teach this to the next generation of HPC users. Costas: Indeed, and we are seeing signs of progress. For example, Pavel Klavik, one of the authors of this paper was an intern at our lab from the Charles University in Prague. He joined us after winning our annual <a href="http://www.zurich.ibm.com/greatminds/" target="_blank">IBM Great Minds Competition</a> and he obviously understands where this trend is going. Cristiano: These concepts are starting to be integrated into part of the academic curricula. In fact, we are participating in an EU project with several universities called <a href="http://exa2green-project.eu/" target="_blank">Exa2Green</a> with the goal of creating energy-aware sustainable computing for future exascale systems. This will help trickle down these ideas to the classroom. <a href="http://researcher.watson.ibm.com/researcher/view.php?person=zurich-YIN" target="_blank">Yves Ineichen</a>: I think we are in a transition period as the topic begins to reach the lecture halls. Part of the challenge is that many universities don’t have access to these HPC systems at the level required to actually run these algorithms and testing, but with collaborations this will change. But first we need a consensus from the HPC industry.