N1005 IBM Research: big data

From Knowledge Graphs to Cognitive Computing

<br /> <i style="mso-bidi-font-style: normal;"><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt; line-height: 107%;">Editor’s note: This article was written by <a href="http://researcher.ibm.com/researcher/view.php?person=ie-marco.sbodio">Marco Luca Sbodio</a>, <span style="font-family: "helvneue light for ibm" , sans-serif;">R</span>esearch <span style="font-family: "helvneue light for ibm" , sans-serif;">S</span>taff <span style="font-family: "helvneue light for ibm" , sans-serif;">M</span>ember at IBM Research - <span style="font-family: "helvneue light for ibm" , sans-serif;">Ireland</span>, and originally appeared <span style="font-family: "helvneue light for ibm" , sans-serif;">on</span> 01net’s website as <a href="http://www.01net.it/dati-di-pubblica-utilita-dai-knowledge-graph-al-cognitive-computing/">Dati di pubblica utilità: dai knowledge graph al cognitive computing</a>; re-printed here with permission.</span></i><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt; line-height: 107%;"> </span><br /><br /><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt; line-height: 107%;">We live in the era of Big Data, and we use data for everything: from predicting which goods a customer may buy next, to forecasting the weather, and analyzing traffic in cities, or the spreading of diseases in the population of a country. Indeed data is an extremely powerful resource, an enabler for a large portion of our current technology.<br /><br />However, we should not forget that though data is the foundation underpinning information processing, the real value is in the information that we are able to surface by processing the data, in the patterns that we are able to identify and recognize, and, ultimately, in the knowledge that we extract. So, we can say that from Big Data we usually distill Small Knowledge, which has Big Value.</span><br /><a name="more"></a><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt; line-height: 107%;"><br />The problem of representing knowledge in a machine processable format is a well known field of Artificial Intelligence, which has been extensively studied since the 1970s. An approach to knowledge representation based on graph (mathematical structure represented as sets of nodes, or vertices, which may be connected by edges) has been discussed for a long time, starting from the introduction of conceptual graphs [1] to the more recent Linked Data initiative [2] - a method to publish data and knowledge over the Web by explicitly representing their relationships, thus enabling computers to directly access and semantically query such distributed knowledge graphs (example below).</span><br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxQQOsxmStnbJaiub_ISSmc51MGve5x0_Sn-6gd-wgSurCynWOXQcyiqMkVcuzMNN8chtxuXT_-McHIV4Dp0yhW55JBm8RSQrAyb7LZ7cPVgW2fEo-nqyd_oV-hKhaR6fxH5Y9rsYNTm8/s1600/Graph_View_DBPedia_Ireland.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="276" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxQQOsxmStnbJaiub_ISSmc51MGve5x0_Sn-6gd-wgSurCynWOXQcyiqMkVcuzMNN8chtxuXT_-McHIV4Dp0yhW55JBm8RSQrAyb7LZ7cPVgW2fEo-nqyd_oV-hKhaR6fxH5Y9rsYNTm8/s400/Graph_View_DBPedia_Ireland.jpg" width="400" /></a></div><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt; line-height: 107%;"><br /> Knowledge graphs are a possible approach to building large knowledge bases, structured collections of facts about the world that computer systems can use to reason, and to interact with humans more naturally. These are some of the key characteristics of cognitive computing. The impact of this new computing approach are potentially very high, and, combined with other technologies current under development, it promises to open up whole new ways for humans to use computers.<br /><br />Among the various fields of application of cognitive computing, a very interesting one is the integrated care domain, an emerging worldwide trend aiming at delivering more effective and coordinated forms of care provision spanning, among others, the social domain and the health domain. Social care and health care are knowledge intensive domain, where data and information are abundant. Delivering insights into the strengths and vulnerabilities of individuals with respect to the communities they live in and their social environment represents a key challenge for practitioners in the integrated care domain.<br /><br />Link2Outcome [3] [4] is a prototype of a cognitive system that helps care workers to make more informed decision to improve outcome for patients. Among other goals, Link2Outcome aims at facilitating access to knowledge across domains and data sources, and to use such knowledge to summarize the state of an individuals though a dynamic set of vulnerability indexes. An interesting example of how Link2Outcome can help doctors is shown in [3]: the system may import (for example from an open data city portal) linked data describing pollution levels in a city, and correlate such information with the area where a patient lives in and with knowledge of how pollution may affect clinical conditions of the patient. A doctor may see how the vulnerability indexes of the patien change after the system has ingested the new data, and thus better understand how the environment may affect this patient.<br /><br />As an extension to Link2Outcome, the more recent work BlueLENS [5], tackle the problem of collecting and providing the right information, to the right people, at the right time across the care continuum. The health of an individual is increasingly studied from multiple perspective, and analyzing multi-sectoral determinants of health is becoming a mainstream approach in integrated care. To enable this approach, data fusion from multiple heterogeneous sources is of key importance, and cognitive systems are uniquely positioned to help practitioners in surfacing the right information at the right time.<br /><br />By establishing a more natural and seamless interaction between doctors and knowledge systems, cognitive computing is paving the way to revolutionize care, by bringing better and more timely insights that help experts in taking the best possible decisions to help patients.</span><br /><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtJhed_lT3FMaY6x6dFelLApVt-3vdI5p3ob5rzVNuDvoR9rL4ePFQ2pFUkzuV05hZYg3Dwo1_ImXy6n8eUtbej0Kcqk8nxEUD40Bwvsn2bXPLBfWuNMRCJMIeBV3lm3Lfa-M3Udfs4aw/s1600/GuruBlueLensHackathonAwardGroupShot.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="303" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtJhed_lT3FMaY6x6dFelLApVt-3vdI5p3ob5rzVNuDvoR9rL4ePFQ2pFUkzuV05hZYg3Dwo1_ImXy6n8eUtbej0Kcqk8nxEUD40Bwvsn2bXPLBfWuNMRCJMIeBV3lm3Lfa-M3Udfs4aw/s400/GuruBlueLensHackathonAwardGroupShot.jpg" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">L-R: Martin Stephenson, Vanessa Lopez, Jiewen Wu,  Marco Luca Sbodio,<br />Pierpaolo Tommasi, Spyros Kotoulas, Guruduth Banavar, and Nuno Lopes.</td></tr></tbody></table><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt; line-height: 107%;"><br />[1] John F. Sowa. 1976. Conceptual graphs for a data base interface. IBM Journal of Research and Development 20, 4 (July 1976), 336-357. DOI=[<a href="http://dx.doi.org/10.1147/rd.204.0336">http://dx.doi.org/10.1147/rd.204.0336</a>]<br /><br />[2] Bizer, Christian; Heath, Tom; Berners-Lee, Tim (2009). "Linked Data—The Story So Far". International Journal on Semantic Web and Information Systems 5 (3): 1–22. [doi:10.4018/jswis.2009081901]. ISSN 1552-6283.<br /><br />[3] [<a href="https://www.youtube.com/watch?v=lx3MRzR_fxI">https://www.youtube.com/watch?v=lx3MRzR_fxI</a>]<br /><br />[4] Spyros Kotoulas, Vanessa Lopez, Marco Luca Sbodio, Martin Stephenson, Pierpaolo Tommasi, and Pol Mac Aonghusa. 2014. A linked data approach to care coordination. In Proceedings of the 25th ACM conference on Hypertext and social media (HT '14). ACM, New York, NY, USA, 77-87. DOI=[<a href="http://dx.doi.org/10.1145/2631775.2631807">http://dx.doi.org/10.1145/2631775.2631807</a>]<br /><br />[5] [<a href="http://researcher.watson.ibm.com/researcher/view_group.php?id=6439">http://researcher.watson.ibm.com/researcher/view_group.php?id=6439</a>]</span>

Editor’s note: This article was written by Marco Luca Sbodio , R esearch S taff M ember at IBM Research - Ireland , and originally appeared...

Tired of answering questions? Let a smart machine help.

7:00 PM N1000 0

<div align="left" class="MsoNormal" style="text-align: left;"><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;"></span></div><span lang="EN-US"><a href="http://researcher.ibm.com/researcher/view.php?person=jp-KAMIJOH"><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">Koichi Kamijo</span></a></span><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">, a scientist at IBM’s research lab in Tokyo was only 20 questions into a 300-question life insurance form. And he was already exhausted. There had to be an easier way to answer these questions. So, he got together with his colleagues, </span><span lang="EN-US"><a href="http://researcher.ibm.com/researcher/view.php?person=jp-RYOKAWA"><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">Ryo Kawahara</span></a></span><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">, </span><span lang="EN-US"><a href="http://researcher.ibm.com/researcher/view.php?person=jp-OSOGAMI"><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">Takayuki Osogami,</span></a></span><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;"> Masaki Ono and Shunichi Amano to come up with a cognitive tech answer to the cumbersome process of finding a life insurance product. And they identified two critical issues with the life insurance questionnaire that needed to be solved to make life plan simulator more user-friendly and useful for consumers and financial planners.</span> <br /><div align="left" class="MsoNormal" style="text-align: left;"><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt; mso-no-proof: yes;">                                                 </span><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;"></span></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEJdPJ17uLKTHo82p2igD77P1CRS2ogNOQQml39MSEdiTSOcSRYLcL4FyO9OMtV52rTAHA-DgcdX091mt3-SbBwKX3pR8jLVvAu82IkCwOe98-nVBX9_533N1XUaUJU8HUBdQMDYR2s3_U/s1600/Tokyo_team.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEJdPJ17uLKTHo82p2igD77P1CRS2ogNOQQml39MSEdiTSOcSRYLcL4FyO9OMtV52rTAHA-DgcdX091mt3-SbBwKX3pR8jLVvAu82IkCwOe98-nVBX9_533N1XUaUJU8HUBdQMDYR2s3_U/s1600/Tokyo_team.jpg" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: x-small;">L-R: Masaki Ono, Ryo Kawahara, Koichi Kamijo, Takayuki Osogami<br />and Shunichi Amano of IBM Research – Tokyo</span></td></tr></tbody></table><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;"></span><br /><a name="more"></a><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">Typically, consumers are asked to answer an average of 300 questions, from name, age, and marital status, to expected salary at the age of retirement, and education expenses for their children. The first issue Koichi and his team recognized: there were too many questions! Going through hundreds of questions is too much for people, even though they help financial planners understand their needs. The second issue he and his team realized they needed to solve was how to help financial planners also predict potential needs. With his colleagues’ expertise in machine learning, business analytics and natural language processing – along with ST-Life (Smile and Thanks Life), whose life plan simulator Dynamic Life Plan Navigator has been widely used by enterprises and organizations in Japan – they developed an insurance information lifecycle management platform to solve these critical issues. <span style="mso-spacerun: yes;"> </span></span><br /><div align="left" class="MsoNormal" style="text-align: left;"><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt; mso-no-proof: yes;">  </span><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;"></span></div><div class="separator" style="clear: both; text-align: center;"></div><div align="left" class="MsoNormal" style="text-align: left;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMXS4y1A49dw4Jns2tN1pwRcqYpnGh7yj2EvZn3iFBProaIAKvGHEpqR6XVwVokMY72yS0gLt8OLl14Y6pZkIFROB9HfBUhO4vtMPQ1S6X_0RPcXVBYHcAv4RgVLU5Bdw9WO2IJJDsE72K/s1600/life_sim.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMXS4y1A49dw4Jns2tN1pwRcqYpnGh7yj2EvZn3iFBProaIAKvGHEpqR6XVwVokMY72yS0gLt8OLl14Y6pZkIFROB9HfBUhO4vtMPQ1S6X_0RPcXVBYHcAv4RgVLU5Bdw9WO2IJJDsE72K/s1600/life_sim.jpg" /></a></div></div><div align="left" class="MsoNormal" style="text-align: left;"><br /> <br /><div align="left" class="MsoNormal" style="text-align: left;"><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">First, Koichi and his team combined two components on the Dynamic Life Plan Navigator: its parameter estimator and human response simulator. The parameter estimator minimized the burden of answering 300 questions down to only eight by applying machine learning and analytics technologies. Just by answering questions like marital status, gender, number of children, income level, date of birth, and home mortgage status, the system then analyzes other users’ answers to automatically fill in answers to the other 292 questions. According to the recent experiment, people who successfully went through life plan simulation questions nearly doubled.</span></div><div align="left" class="MsoNormal" style="text-align: left;"><br /></div><table align="right" border="1" cellpadding="0" cellspacing="0" class="MsoTableGrid" style="border-collapse: collapse; border: none; margin-left: 6.75pt; margin-right: 6.75pt; mso-border-alt: solid white 1.5pt; mso-border-insideh-themecolor: background1; mso-border-insideh: 1.5pt solid white; mso-border-insidev-themecolor: background1; mso-border-insidev: 1.5pt solid white; mso-border-themecolor: background1; mso-padding-alt: 0in 5.4pt 0in 5.4pt; mso-table-anchor-horizontal: margin; mso-table-anchor-vertical: paragraph; mso-table-left: right; mso-table-lspace: 9.0pt; mso-table-overlap: never; mso-table-rspace: 9.0pt; mso-table-top: 2.85pt; mso-yfti-tbllook: 1184;"> <tbody><tr style="mso-yfti-firstrow: yes; mso-yfti-irow: 0; mso-yfti-lastrow: yes;">  <td style="border: solid white 1.5pt; mso-border-themecolor: background1; padding: 0in 5.4pt 0in 5.4pt; width: 2.25in;" valign="top" width="216"><div align="left" class="MsoNormal" style="mso-element-anchor-horizontal: margin; mso-element-anchor-vertical: paragraph; mso-element-frame-hspace: 9.0pt; mso-element-left: right; mso-element-top: 2.85pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly; text-align: left;"><b style="mso-bidi-font-weight: normal;"><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">CREST   Grant</span></b></div><div align="left" class="MsoNormal" style="mso-element-anchor-horizontal: margin; mso-element-anchor-vertical: paragraph; mso-element-frame-hspace: 9.0pt; mso-element-left: right; mso-element-top: 2.85pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly; text-align: left;"><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">The preference estimation uses   a technology that IBM Research -Tokyo has researched and developed with a grant   from the </span><a href="http://www.jst.go.jp/kisoken/crest/en/index.html"><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">Japan   Science and Technology Agency’s Core Research for Evolutionary Science and   Technology (CREST)</span></a><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">.</span></div></td> </tr></tbody></table><div align="left" class="MsoNormal" style="text-align: left;"><span style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">Next, the human simulator can help discover a user’s potential preferences, such as insurance premium, coverage type, or what they consider important in choosing a product. The estimator creates several unconventional, two-option hypothetical questions based on the user’s data from the past. Then the simulator suggests an option. By repeating this process four to five times, the simulation system uncovers the user’s potential preferences, including the ones which the user has never imagined, such as job, partner and vacation destination preferences. </span></div></div><div align="left" class="MsoNormal" style="text-align: left;"><br /><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">More than 97 percent of the users who completed these hypothetical questions in the recent experiment did not feel it was a burden. And felt that they received an appropriate life insurance product recommendation.</span><b style="mso-bidi-font-weight: normal;"><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;"> </span></b><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOBxDlT-2xxmtYYIPxVNnJhwRMQp52Gkr-wut4hmJLCEiFoO9fRKdd-X3lTpXI2WbQROz9ATqg3LenKsyOgU2oXvvdjWlzRULv5dWgtchSZ8LoqxWXe3Qlpwnv8AIM58p0X2ZZc3-E8hQH/s1600/lifesim_question.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOBxDlT-2xxmtYYIPxVNnJhwRMQp52Gkr-wut4hmJLCEiFoO9fRKdd-X3lTpXI2WbQROz9ATqg3LenKsyOgU2oXvvdjWlzRULv5dWgtchSZ8LoqxWXe3Qlpwnv8AIM58p0X2ZZc3-E8hQH/s1600/lifesim_question.jpg" /></a></div><b style="mso-bidi-font-weight: normal;"><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">Protecting Your Life (Insurance) Choices</span></b></div><div align="left" class="MsoNormal" style="text-align: left;"><br /></div><div align="left" class="MsoNormal" style="text-align: left;"><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">Additionally, the research team created a bonus technology to help avoid posting inappropriate expressions and private information to online life insurance-related forums. In Japan, some users want to reference and reply to “people-like-me” comments in these forums that are often managed by volunteer life insurance agents and financial advisors. But to post a question can be a challenge because it can be difficult to determine what information to contribute, while not disclosing private information.</span></div><div align="left" class="MsoNormal" style="text-align: left;"><br /></div><div align="left" class="MsoNormal" style="text-align: left;"><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">The context-aware discussion forum technology the team built highlights potentially inappropriate expressions and private information to help users amend their posts before uploading them to a forum. Koichi and his team hope that it will also help prevent “flaming” (hostile interactions) in the forum due to improper comments. </span></div><div align="left" class="MsoNormal" style="text-align: left;"><br /></div><div align="left" class="MsoNormal" style="text-align: left;"><b style="mso-bidi-font-weight: normal;"><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">Try, try again</span></b></div><div align="left" class="MsoNormal" style="text-align: left;"><br /></div><div align="left" class="MsoNormal" style="text-align: left;"><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">Koichi used the tool again to do his life plan, and this time, it only took him five minutes to answer eight questions, overwrite some past answers, and receive a detailed life insurance plan and product based on future income and stock market performance. </span></div><div align="left" class="MsoNormal" style="text-align: left;"><br /></div><div align="left" class="MsoNormal" style="text-align: left;"><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">Koichi and his team are now working on applying the machine learning platform to other industries, such as banks and education. For example, like life insurance products, banks may consider using this platform to offer products to their customers that suit their savings goals. In education, such a platform may be applied to help teachers understand their students’ preferences and abilities by asking questions of their parents. If teachers can anticipate each student’s preference and ability based on an estimated value the platform’s analytics technology provides, they may be able to come up with a customized curriculum that effectively provides student support.<span style="mso-spacerun: yes;">  </span><span style="mso-spacerun: yes;">  </span></span></div><div align="left" class="MsoNormal" style="text-align: left;"><br /></div><div align="left" class="MsoNormal" style="text-align: left;"><span lang="EN-US" style="font-family: "helvneue light for ibm" , sans-serif; font-size: 13.0pt;">ST-Life continues to work with Koichi and his team to advance the technology. At IBM Japan, Koichi and his team plan to collaborate with IBM’s consulting unit to deploy the insurance information lifecycle management platform to the insurance industry. <span style="mso-spacerun: yes;"> </span></span></div>

big data cognitive computing ibm research tokyo natural language processing Tired of answering questions? Let a smart machine help.

Koichi Kamijo , a scientist at IBM’s research lab in Tokyo was only 20 questions into a 300-question life insurance form. And he was already...

Helping customers before they call

5:00 AM N1000 0

<br /><div class="MsoNormal"><b style="mso-bidi-font-weight: normal;"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif; font-size: 13.0pt; line-height: 107%;"></span></b></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkpC29mK3Ln3TCSeig_e69ZcTVonIFw4iM-5pju3lGmrKPKkyflG8yC6MG-9mFoQDOMH8jDDxslaQMTu9nZKpWQOnY3YEjKVRitChBYbLAHH5VNgTMRokStudZzEtR2hdiPV1LM-ukUt2v/s1600/fcalabrese.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkpC29mK3Ln3TCSeig_e69ZcTVonIFw4iM-5pju3lGmrKPKkyflG8yC6MG-9mFoQDOMH8jDDxslaQMTu9nZKpWQOnY3YEjKVRitChBYbLAHH5VNgTMRokStudZzEtR2hdiPV1LM-ukUt2v/s1600/fcalabrese.jpg" /></span></a></div><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="font-size: 13.0pt; line-height: 107%;">by <a href="http://researcher.watson.ibm.com/researcher/view.php?person=ie-FCALABRE">Francesco Calabrese</a>, manager of Smarter Urban Dynamics, IBM Research-Ireland</span><span style="font-size: 13.0pt; line-height: 107%;"> </span></span></div><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span></div><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif; font-size: 13.0pt; line-height: 107%;">As our smartphones get smarter, we’re using more Over-The-Top applications like WhatsApp, Viber or Skype, rather than our telco provider’s voice, and text. This downward trend means shrinking income for the telco, even though it’s estimated that data usage will grow beyond 20 exabytes per month in the coming years. When my team of social, mobile and decision theory researchers at IBM’s lab in Dublin noticed this OTT trend, we wanted to know, in broader terms: could we measure and predict the quality of a customer’s experience on a telco network in real time?</span></div><a name="more"></a><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span><br /><div class="MsoNormal"></div><table align="right" border="1" cellpadding="0" cellspacing="0" class="MsoNormalTable" style="border-collapse: collapse; border: none; margin-left: 6.75pt; margin-right: 6.75pt; mso-border-alt: solid white .5pt; mso-border-insideh: .5pt solid white; mso-border-insidev: .5pt solid white; mso-padding-alt: 0in 5.4pt 0in 5.4pt; mso-table-anchor-horizontal: margin; mso-table-anchor-vertical: paragraph; mso-table-left: right; mso-table-lspace: 9.0pt; mso-table-overlap: never; mso-table-rspace: 9.0pt; mso-table-top: 2.9pt; mso-yfti-tbllook: 1184;"> <tbody><tr style="mso-yfti-firstrow: yes; mso-yfti-irow: 0; mso-yfti-lastrow: yes;">  <td style="border: solid white 1.0pt; mso-border-alt: solid white .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 2.25in;" valign="top" width="216"><div class="MsoNormal" style="mso-element-anchor-horizontal: margin; mso-element-anchor-vertical: paragraph; mso-element-frame-hspace: 9.0pt; mso-element-left: right; mso-element-top: 2.9pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly;"><i style="mso-bidi-font-style: normal;"><span style="color: black; font-family: Helvetica Neue, Arial, Helvetica, sans-serif; font-size: 14.0pt; line-height: 107%; mso-bidi-font-family: "Tms Rmn"; mso-fareast-language: EN-IE;">Telcos need to deepen their   customer knowledge through smarter capabilities such as predictive analytics.</span></i></div><div class="MsoNormal" style="mso-element-anchor-horizontal: margin; mso-element-anchor-vertical: paragraph; mso-element-frame-hspace: 9.0pt; mso-element-left: right; mso-element-top: 2.9pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly;"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span></div><div class="MsoNormal" style="mso-element-anchor-horizontal: margin; mso-element-anchor-vertical: paragraph; mso-element-frame-hspace: 9.0pt; mso-element-left: right; mso-element-top: 2.9pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly;"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="color: black; font-family: "HelvNeue Light for IBM",sans-serif; font-size: 12.0pt; line-height: 107%; mso-bidi-font-family: "Tms Rmn"; mso-fareast-language: EN-IE;">– IBM   Institute for Business Value (IBV) study, <span style="mso-bidi-font-weight: bold;"><a href="http://www-935.ibm.com/services/us/gbs/thoughtleadership/restoringconnections/">Restoring   Connections</a></span></span><span style="font-size: 12.0pt; line-height: 107%;"></span></span></div></td> </tr></tbody></table><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="font-size: 13pt; line-height: 107%;">And we may have found the answer in predictive analytics. Our recent </span><i style="mso-bidi-font-style: normal;">ArXiv</i><span style="font-size: 13pt; line-height: 107%;">paper, </span><span style="color: #0000ee;"><span style="font-size: 17.3333339691162px; line-height: 18.546667098999px;"><u><a href="http://arxiv.org/pdf/1508.02884v1.pdf" target="_blank">Towards Real-time Customer Experience Prediction for Telec<span id="goog_1439388196"></span><span id="goog_1439388197"></span>ommunication Operators</a></u></span></span><span style="font-size: 13pt; line-height: 107%;">, tells an interesting story of telcos’ inability to anticipate customer problems. If telcos could predict issues in real time, they could then serve their customers more efficiently by anticipating problems and implementing fixes before a customer calls with an issue. The good news is that our study found that telcos have the data necessary to help – and keep – their customers. </span></span></div><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span></div><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif; font-size: 13.0pt; line-height: 107%;">So, we devised a way for a telco’s system to understand a bad customer experience so it could examine, and anticipate customer usage. This, in turn, would give the telco a chance to intervene in real time, in what the system considered a potentially bad experience in an automated way, such as a text to customers, or to send operators out to fix a technical issue. For telcos, this predictive analysis could not only mean improved customer satisfaction and reduced help calls, but less customer churn, and a better <a href="http://www.netpromoter.com/why-net-promoter/know">Net Promoter Score</a> – a telco’s customer experience grade.</span></div><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span></div><div class="MsoNormal"><b style="mso-bidi-font-weight: normal;"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif; font-size: 13.0pt; line-height: 107%;">Piloting through a forest of data</span></b></div><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span></div><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="font-size: 13.0pt; line-height: 107%;">We partnered with a major South African telco to test our ideas on some real, but historical and anonymized, data. They provided a data set of app usage, which we plugged into a big data architecture of network analysis, and machine learning software. We used the IBM <a href="http://www-01.ibm.com/software/data/now-factory/">Now Factory</a> network analytics “feed probe” to collect the data and push it into <a href="http://www-01.ibm.com/software/data/infosphere/hadoop/enterprise.html">Big Insights</a>’</span><span style="color: black; font-family: "HelvNeue Light for IBM",sans-serif; font-size: 13.0pt; line-height: 107%; mso-bidi-font-family: Arial; mso-fareast-language: EN-IE;"> </span><span style="font-size: 13.0pt; line-height: 107%;">machine learning algorithms. This allowed us to generate models of contextual relationships within the data. And then we used <a href="http://www-03.ibm.com/software/products/en/infosphere-streams">InfoSphere Streams</a> analytics platform to score such models on millions of events from the network and make predictions of customer experience. During the pilot, our application processed billions of records that included mobile app usage and performance, and network location. It then evaluated and visualized how these factors affected churn rate, customer care and Net Promoter Score.<span style="mso-spacerun: yes;"> </span></span></span></div><blockquote class="tr_bq"><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><i style="mso-bidi-font-style: normal;"><span style="font-size: 13.0pt; line-height: 107%;">For example, we uncovered an issue with a mobile application that was linked to a high percentage of retransmitted data, which resulted in many customer care calls during the monitored days, as well as for two subsequent weeks. If the telco had deployed our predictive solution in real time, it would have been able to identify the issue well in advance, and proactively communicate it to the customers, thus reducing the number of customer care calls.</span></i><span style="font-size: 13.0pt; line-height: 107%;"> </span></span></div><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span></div><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif; font-size: 13.0pt; line-height: 107%;">– <a href="http://researcher.ibm.com/researcher/view.php?person=ie-e.diaz-aviles">Ernesto Diaz-Aviles</a>, machine learning and data scientist, IBM Research-Ireland </span></div></blockquote><div class="MsoNormal"><span style="color: black; font-family: Helvetica Neue, Arial, Helvetica, sans-serif; font-size: 13.0pt; line-height: 107%; mso-bidi-font-family: "Tms Rmn"; mso-fareast-language: EN-IE;">The system discovered the issue by training what we call a Restrictive Random Forest model. It observed customer transactions in the telco’s data feed. It then generated an ensemble of decision trees that labeled each daily experience as either positive or negative, based on if a help call was made. This approach could allow the system to help prepare call center agents to manage a problem before a call, better manage a call that does come through, or tell technicians of a (potential) mechanical failure – before anyone experiences an outage. </span></div><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span></div><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif; font-size: 13.0pt; line-height: 107%;">My team, and our system, will continue to study the individual aspects of user behavior and the reasons for their calls. Our goal is to build personalized models for segments of users, an approach we expect to more-accurately predict the user’s mobile experience. So, maybe in the future, customers will make a call on their telco’s network for something other than reporting a complaint. </span></div><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span></div><div class="MsoNormal"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><i style="mso-bidi-font-style: normal;"><span style="font-size: 13pt; line-height: 107%;">For more, read: </span><span style="color: #0000ee;"><span style="font-size: 17.3333339691162px; line-height: 18.546667098999px;"><u><a href="http://arxiv.org/pdf/1508.02884v1.pdf" target="_blank">Towards Real-time Customer Experience Prediction for Telecommunication Operators</a></u></span></span><span style="font-size: 13pt; line-height: 107%;">, by </span></i><i style="mso-bidi-font-style: normal;"><span lang="EN-IE" style="font-family: "HelvNeue Light for IBM",sans-serif; font-size: 13.0pt; line-height: 107%; mso-ansi-language: EN-IE;"><a href="http://arxiv.org/find/cs/1/au:+Diaz_Aviles_E/0/1/0/all/0/1">Ernesto Diaz-Aviles</a>, <a href="http://arxiv.org/find/cs/1/au:+Pinelli_F/0/1/0/all/0/1">Fabio Pinelli</a>, <a href="http://arxiv.org/find/cs/1/au:+Lynch_K/0/1/0/all/0/1">Karol Lynch</a>, <a href="http://arxiv.org/find/cs/1/au:+Nabi_Z/0/1/0/all/0/1">Zubair Nabi</a>, <a href="http://arxiv.org/find/cs/1/au:+Gkoufas_Y/0/1/0/all/0/1">Yiannis Gkoufas</a>, <a href="http://arxiv.org/find/cs/1/au:+Bouillet_E/0/1/0/all/0/1">Eric Bouillet</a>, <a href="http://arxiv.org/find/cs/1/au:+Calabrese_F/0/1/0/all/0/1">Francesco Calabrese</a>, <a href="http://arxiv.org/find/cs/1/au:+Coughlan_E/0/1/0/all/0/1">Eoin Coughlan</a>, <a href="http://arxiv.org/find/cs/1/au:+Holland_P/0/1/0/all/0/1">Peter Holland</a>, <a href="http://arxiv.org/find/cs/1/au:+Salzwedel_J/0/1/0/all/0/1">Jason Salzwedel</a></span></i></span></div><div class="MsoNormal"><br /></div><div class="MsoNormal"><br /></div><div class="MsoNormal"><br /></div>

big data ibm research - Ireland machine learning telco Helping customers before they call

by Francesco Calabrese , manager of Smarter Urban Dynamics, IBM Research-Ireland As our smartphones get smarter, we’re using more Over-The...