N1005 IBM Research: SDS

<br /><div class="MsoNormal"></div><div class="MsoNormal"><table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgwU4RDCtEOxkH6uxGkWdTY_8oU9bNHuFNMIosRfsMRSgdad9Rv5sT4LNhUl8JhSBjJ0mg6iPbIkhpA2fDHDyo9hNHeSxYZhmGuSReFdUBWWcGafgDtNceXJVYuftwnXLWYPhLNDlIvw7Lx/s1600/michael5.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgwU4RDCtEOxkH6uxGkWdTY_8oU9bNHuFNMIosRfsMRSgdad9Rv5sT4LNhUl8JhSBjJ0mg6iPbIkhpA2fDHDyo9hNHeSxYZhmGuSReFdUBWWcGafgDtNceXJVYuftwnXLWYPhLNDlIvw7Lx/s1600/michael5.jpg" width="191" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Michael Factor</td></tr></tbody></table><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="font-size: small;"><i>Editor’s note: This blog posting was authored by Michael Factor, Distinguished Engineer and expert on Storage Systems at IBM Research - Haifa</i></span></span></div><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="font-size: small;"><br /></span></span><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">Traditional file systems store our information in a tree structure. Although this works fine for small collections of data – like those on our local hard drive – they are not designed for the massive volumes of unstructured content most businesses are collecting, storing, and accessing on the cloud. </span></span><br /><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br /></span></span><span style="font-size: small;"><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">A new method of storing information is called object storage. This approach stores information as objects. Each object contains the data (the bits and bytes of our documents, movies, images, and so forth), together with metadata that holds user- and system-defined tags. These smart data objects include rich information – or metadata – that describes the content of the data, how the object is related to other objects, how the data should be handled, replicated, or backed up, and more.</span></span> </span><br /><div class="MsoNormal"><span style="font-size: small;"><br /></span></div><div class="MsoNormal"><div align="right"><table align="right" border="1" cellpadding="0" cellspacing="0" class="MsoTableGrid" style="border-collapse: collapse; border: medium none; margin-left: 6.75pt; margin-right: 6.75pt;"> <tbody><tr style="height: 71.5pt;">  <td style="border: 1pt solid white; height: 71.5pt; padding: 0in 5.4pt; width: 144.1pt;" valign="top" width="192"><div class="MsoNormal" style="line-height: 150%; margin-bottom: 0in;"><span style="font-size: x-large;"><b><span style="color: #0070c0; font-family: "Times New Roman"; line-height: 150%;">“But what if we could turn a software-defined object store into a smart storage platform?”</span></b></span></div></td> </tr></tbody></table></div><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="line-height: 115%;">Although object storage can store objects, manage them, protect them, and so on </span></span></span><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="line-height: 115%;"><span style="font-size: small;"><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">– </span></span></span>it doesn’t by itself dramatically increase the rate at which we can extract value from objects. But what if we could turn a software-defined object store into a smart storage platform?</span></span></span></div><div class="MsoNormal"><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br /></span></span></div><div class="MsoNormal"><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><b><span style="line-height: 115%;">Storlets bring the computation to the data</span></b></span></span></div><div class="MsoNormal"><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br /></span></span></div><div class="MsoNormal"><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="line-height: 115%;">A new research prototype called "storlets" holds the promise of greatly increasing the value we get out of storage and the speed at which we can access what we need. A new software-defined mechanism, storlets allow object storage to move the computation to the data, instead of the system having to move the data to a server to carry out the computation. </span></span></span><br /><br /><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="line-height: 115%;">Imagine if every time you wanted to cook a meal, you had to bring all your ingredients to a central neighborhood depot where stoves, appliances, and cooking utensils were available to "process" the food. That’s similar to the current situation with data on the cloud. Storlets come to remedy this situation by moving the heavy lifting to where it’s needed </span></span></span><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="line-height: 115%;"><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="line-height: 115%;"><span style="font-size: small;"><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">– </span></span></span></span></span></span>similar to allowing you to cook everything in your kitchen, where all your raw materials are already located. </span></span></span><br /><br /><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="line-height: 115%;">The impact of storlets is substantial. Stored data can be processed locally, and no longer needs to be transferred over the network to a remote computer, processed, and then put back onto the storage server – all of which incurs both network transfer latencies and thus real dollar costs.</span></span></span><br /><br /><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="line-height: 115%;">Our vision is to reduce costs, increase flexibility and improve security</span></span></span><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="line-height: 115%;"><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="line-height: 115%;"><span style="font-size: small;"><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"> </span></span></span></span></span></span>by turning the object store into a platform, and allowing the functionality of the object store to be extended using software.  </span></span></span></div><div class="MsoNormal"><br /><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><b><span style="line-height: 115%;">Getting more value from data – much faster</span></b></span></span></div><div class="MsoNormal"><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><br /></span></span></div><div class="MsoNormal"><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="line-height: 115%;">Aside from saving bandwidth by eliminating unnecessary data transfers, storlets are an ideal way to introduce new services. Storlets can analyze each object and extract its metadata, including size, subject, resolution, format, and more. Since storlets are dynamically loaded code, the function is only limited by the developer’s imagination.</span></span></span><br /><br /><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="line-height: 115%;">For example, a media company could upload a movie to an object store and have it automatically generate a representative image. A physician doing rounds could have the object store send only the portion of a patient’s x-ray needed to her wireless device for immediate viewing. A lawyer could request that the object store provide a document from a previous court case in a specific format. Or pathologists could have images analyzed in the object store itself. All this sophisticated computation is moved into the storage infrastructure via the software of dynamically loaded storlets, making it faster, more flexible, and far less expensive.</span></span></span></div><div class="MsoNormal"><br /><span style="font-size: large;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;"><span style="line-height: 115%;">IBM researchers in Haifa, Israel have been working on prototyping storlets for several years in the context of European Union Research projects such as <a href="http://ensure-fp7-plone.fe.up.pt/site/">ENSURE</a>, <a href="http://www.visioncloud.eu/">VISION Cloud</a>, and <a href="http://www.forgetit-project.eu/">Forget IT</a>. In short, we’re looking forward to seeing more value from data in faster and more flexible ways.</span></span></span></div><div class="MsoNormal"></div>

Storlets: Turning Object Storage into a Smart Storage Platform

Michael Factor Editor’s note: This blog posting was authored by Michael Factor, Distinguished Engineer and expert on Storage Systems at IBM...