An Introduction to Big Data Governance PDF Print E-mail
Database - Business Intelligence
Written by Sunil Soares   
Monday, 12 November 2012 00:00

Support MC Press - Visit Our Sponsors

 

Forums Sponsor

POPULAR FORUMS

Forums

Search Sponsor

POPULAR SEARCHES

Search

 

 

This article is an excerpt from the book Big Data Governance: An Emerging Imperative by Sunil Soares (MC Press, October 2012).

 

We are drowning in data today. This data comes from social media, telephone GPS signals, utility smart meters, RFID tags, digital pictures, and online videos, among other sources. IDC estimates that the amount of information in the digital universe exceeded 1.8 zettabytes (1.8 trillion gigabytes) in 2011 and is doubling every two years.1 Much of this data can be characterized as big data. Big data is generally referred to in the context of the "three Vs"—volume, velocity, and variety. We add another "V" for value. Let's consider each of these terms:

  • Volume (data at rest)—Big data is generally large. Enterprises are awash with data, easily amassing terabytes and petabytes of information, and even zettabytes in the future.
  • Velocity (data in motion)—Often time-sensitive, streaming data must be analyzed with millisecond response times to bolster real-time decisions.
  • Variety (data in many formats)—Big data includes structured, semi-structured, and unstructured data such as email, audio, video, clickstreams, log files, and biometrics.
  • Value (cost effectiveness)—Organizations are looking to gain insights from big data in a cost-effective manner. This is where open source technologies such as Apache® Hadoop® have become extremely popular. Hadoop, discussed in detail later in this book, is software that processes large data sets across clusters of hundreds or thousands of computers in a cost-effective manner.

 

Organizations must govern all this big data, which brings us to the subject of this book. We define big data governance as follows:

 

Big data governance is part of a broader information governance program that formulates policy relating to the optimization, privacy, and monetization of big data by aligning the objectives of multiple functions.

 

Let's decompose this definition into its main parts:

 

Big data is part of a broader information governance program.

 

Information governance organizations should incorporate big data into their existing information governance frameworks by doing the following:

  • Extend the scope of the information governance charter to include big data governance.
  • Broaden the membership of the information governance council to include power users of big data such as data scientists.
  • Appoint stewards for specific categories of big data such as social media.
  • Align big data with information governance disciplines such as metadata, privacy, data quality, and master data.

 

Big data governance is about policy formulation. 
 

Policy includes the written or unwritten declarations of how people should behave in a given situation. For example, a big data governance policy might state that an organization will not integrate a customer's Facebook profile into his or her master data record without that customer's informed consent.

 

Big data must be optimized.

  • Consider how organizations might apply the principles of the physical world to their big data. Companies have well-defined enterprise asset management programs to care for their machinery, aircraft, vehicles, and other assets. Similar to cataloging physical assets, organizations need to optimize their big data as follows:
  • Metadata—Build information about inventories of big data.
  • Data quality management—Cleanse big data just as companies conduct preventive maintenance on physical assets.
  • Information lifecycle management—Archive and retire big data when it no longer makes sense to retain these massive volumes.

 

Privacy of big data is important.

Organizations also need to establish the appropriate policies to prevent the misuse of big data. Organizations need to consider the reputational, regulatory, and legal risks involved when handling social media, geolocation, biometric, and other forms of personally identifiable information.

 

Big data must be monetized.

Monetization is the process of converting an asset such as data into money by selling it to third parties or by using it to develop new services. Traditional accounting rules do not allow companies to treat information as a financial asset on their balance sheets unless purchased from external sources. Despite this conservative accounting treatment, organizations now recognize that they should treat big data as an enterprise asset with financial value. For example, operations departments can use sensor data to increase equipment uptime based on preventive maintenance programs. Call centers can analyze agents' notes to reduce call volumes by understanding why customers call. In addition, retailers can use master data to power Facebook apps that drive customer loyalty.

 

Big data exposes natural tensions across functions.

Big data governance needs to harmonize competing objectives across multiple functions. For example, the wireless marketing department at a telecommunications carrier might be interested in using geolocation data to drive new revenue streams, such as when a subscriber receives coupons from retailers that are in close proximity. However, the wireline business might be concerned about the reputational hazard associated with reusing subscribers' geolocation data without their consent. Meanwhile, the network management team might want to use this information to address any issues with network performance, such as a large number of dropped calls at a specific wireless tower. Finally, the chief privacy officer might have concerns about the potential for regulatory backlash. In this situation, big data governance needs to bring all the parties together to determine whether the potential revenue upside from the new services outweighs the associated reputational and regulatory risks. The usage of geolocation data for internal network analytics is probably okay, but the other business uses might not be.

 

 

Case Study 1.1 reviews the unfortunate events surrounding the Mars Climate Orbiter. We would not consider this volume of data to be "big" by today's standards. However, NASA likely produced the navigation commands by crunching some very big numbers with complex mathematics. If commercial organizations do similar crunching of big data to score a risk, fraud, or propensity to buy, they might incorrectly reject credit card applications or miss customer churn events because scores are misunderstood or applied incorrectly.

 

Case Study 1.1: Big data governance and the Mars Climate Orbiter2,3,4

 

 

In a typical information governance project, the team identifies a business problem, develops a business case, obtains an executive sponsor, defines a technical architecture, and proceeds with the rest of the initiative. However, big data projects are different because of the following characteristics:

 

  • Projects are driven by early adopters.
  • The business problem needs to be discovered.
  • IT is often at the forefront with technologies such as Hadoop.
  • The business case has not been developed.
  • The characteristics of the data are unclear.

 

As of the publication of this book, governance has taken a backseat to the analytics and technologies associated with big data. However, as big data projects become mainstream, we anticipate that privacy, stewardship, data quality, metadata, and information lifecycle management will coalesce into an emerging imperative for big data governance.

Notes

1. The 2011 Digital Universe Study: Extracting Value from Chaos (IDC, 2011).

2. http://en.wikipedia.org/wiki/Mars_Climate_Orbiter.

3. "Mars Climate Orbiter Fact Sheet." http://mars.jpl.nasa.gov/msp98/orbiter/fact.html.

4. "Mars Climate Orbiter Mishap Investigation Board Phase I Report." November 1999.

 

 


Sunil Soares
About the Author:
Sunil Soares is the founder and managing partner of Information Asset, LLC, a consulting firm that specializes in data governance. Prior to this role, Sunil was director of information governance at IBM, where he worked with clients across six continents and multiple industries. Before joining IBM, Sunil consulted with major financial institutions at the Financial Services Strategy Consulting Practice of Booz Allen & Hamilton in New York. Sunil lives in New Jersey and holds an MBA in Finance and Marketing from the University of Chicago Booth School of Business.

This is Sunil’s fifth book about data governance. His first book, The IBM Data Governance Unified Process, details the almost 100 steps to implement a data governance program. This book has been used by several organizations as the blueprint for their data governance programs and has been translated into Chinese. Sunil's second book, Selling Information Governance to the Business, reviews the best practices to approach information governance by industry and function. Sunil's third book, IBM InfoSphere: A Platform for Big Data Governance and Process Data Governance, focuses on IBM's InfoSphere product. Sunil's fourth book, Big Data Governance, addresses the specific issues associated with the governance of big data.
 
Books by Sunil Soares are available at the MC Press Bookstore:

 

Data Governance Tools Data Goverance Tools

In this new release, data governance expert Sunil Soares walks you through all the available data governance software tools and shows you how to evaluate these tools for your program. Soares explains why your organization should consider tools as a critical component of its data governance program, and explores the various tool offerings. Soares clarifies how data governance tools integrate with big data technologies and shows you how the various software vendors stack up. Get this latest release today!

List Price $53.95...On Sale Now

Element Break
IBM InfoSphere: A Platform for Big Data Governance and Process Data Governance

IBM InfoSphere: A Platform for Big Data Governance and Process Data Governance

Use the knowledge presented in this book to understand big data support across the IBM InfoSphere portfolio, including DataStage, Streams, QualityStage, MDM, Guardium, Optim, and Data Explorer. Understand the integration between IBM Business Process Manager Express and IBM InfoSphere in terms of integrating information governance, big data, and business process management.

List Price $16.95...On Sale Now

Element Break
Big Data Governance

Big Data Governance

A growing number of books address the topic of big data, but none deals with the challenge of governing big data. Yet big data governance is a crucial enabler to derive maximum value from a big data program. Big Data Governance addresses this knowledge gap, examining the industry imperatives that are driving the convergence of these two major trends in Information Management and explaining not only the why but the how of governing big data.

List Price $59.95...On Sale Now

Element Break
Selling Information Governance to the Business

Selling Information Governance to the Business

Other books discuss the importance of selling to the business; this book actually dives into the details and mechanics of doing so. In it, you will find best practices supported by dozens of real-life case studies to implement an information governance program across a variety of industries, including banking, financial markets, insurance, healthcare, manufacturing, retail, government, telecom, utilities, and more.

List Price $49.95...On Sale Now

Element Break
The IBM Data Governance Unified Process

The IBM Data Governance Unified Process

This book describes a unified process targeted at Data Governance practitioners. It provides a rigorous explanation of the 14 steps and almost 100 sub-steps to implement Data Governance based on IBM products, services, and best practices over hundreds of client engagements.

List Price $24.95...On Sale Now

Read More >>
Last Updated on Monday, 12 November 2012 00:00
 
User Rating: / 0
PoorBest 
   MC-STORE.COM