Setting Data Policies, Standards, and Processes

Business Intelligence
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

In today's increasingly digital economy, data is the fuel that runs your organization's applications, business processes, and decisions.

Editor's note: This article is an excerpt from Chapter 5, "Setting Data Policies, Standards, and Processes," of The Chief Data Officer Handbook for Data Governance (MC Press, 2015).

The Chief Data Officer (CDO) needs to establish a framework to document the decisions from the data governance program. In this chapter, we will discuss how the chief data office can set data policies, standards, and processes.

Prescribe Terminology for Data Policies

A number of terms—such as data policies, standards, processes, guidelines, and principles—are used to describe data decisions. The CDO must prescribe a consistent terminology to describe these data decisions. In this book, we use a standard framework of data policies, standards, and processes. These artifacts may be found in many places, including in people's heads or embedded in broader policy manuals. Some organizations document their data policies in Microsoft Word or Microsoft PowerPoint and load the documents to Microsoft SharePoint or an intranet portal. We begin this chapter with definitions of key terms:

  • Data policies—Data policies provide a broad framework for how decisions should be made regarding data. Data policies are high-level statements and need more detail before they can be operationalized. Each data policy may be supported by one or more data standards.
  • Data standards—Data standards provide detailed rules on how to implement data policies.
  • Data processes—Data processes provide special instructions on how to implement data standards. Each data standard may be supported by one or more data processes.

Data policies, standards, and processes follow a hierarchy, as shown in Figure 1.


Figure 1: Hierarchy of data policies, standards, and processes

Establish a Framework for Data Policies

The CDO must drive a framework for data policies within the following six categories:

  1. By EDM category—Policies in this category may cover data ownership, data architecture, data modeling, data integration, data security and privacy, master data management, reference data management, metadata management, data quality management, and information lifecycle management.
  2. By data domain—Policies in this category may cover customer, product, vendor, equipment, and chart of accounts. This category also includes policies to deal with customer duplicates and product hierarchies.
  3. By critical data element—This data policy includes guidelines to identify critical data elements. Additional policies may cover specific attributes such as U.S. Social Security number, email address, phone number, and product identifier.
  4. By organization—Policies in this category deal with data issues that are specific to a given function or department, such as risk management or marketing. For example, risk management data policies may deal with the use of data to calculate value at risk. Marketing data policies may address customer segmentation, propensity scores, and contact preferences, such as do not call.
  5. By business process—Policies in this category may cover customer service and new product introduction. For example, the customer service policy may include a policy that phone representatives need to search for a customer record before creating a new one.
  6. By big data domain—Policies in this category include the use of big data such as Facebook, Twitter, equipment sensor data, facial recognition, chat logs, and Web cookies. We have called out this category separately because there are a number of unique challenges in dealing with these emerging data types.

The remainder of this chapter will review some examples of data policies, standards, and processes.

Data Quality Management Policy

Figure 2 shows a hierarchy of policy, standards, and processes for data quality management. The overall data policy includes standards for accountability, critical data elements, and the data quality scorecard. The accountability data standard also refers to the data issue resolution process.


Figure 2: Data quality management policy, standards, and processes


A sample policy for data quality management is shown below:

Data quality management is a discipline that includes the methods to measure and improve the quality and integrity of an organization's data. We must adhere to an enterprise-approved process to manage and improve the quality of business-critical data.


The data quality management standards are as follows:

  • Accountability—The data governance team must lead the overall data quality program. However, each data owner must assign one or more business data stewards to manage data quality for key systems and data domains. The responsibilities of the data steward include identifying critical data elements, creating business rules for data profiling, and resolving data issues.
  • Critical data elements—The data steward must identify critical data elements, which will be the focus of the data quality program. Critical data elements must constitute not more than 10 to 15 percent of the total attributes in a data domain or data repository. These critical data elements must be used to create business rules that will drive the data quality scorecard. For example, "email address should not be null" is a business rule that relates to the "email address" critical data element.
  • Data quality scorecard—The data governance team must manage a data quality scorecard to track key metrics by system and data domain. This scorecard must be updated on a monthly basis and will be circulated to key stakeholders.


Finally, the accountability data standard links to the data issue resolution process. This process states that the lead data steward must track data issues in a log that will be circulated to stakeholders on a periodic basis. This log must track the list of data issues, severity, assignee, date assigned, and current status.

Metadata Management Policy

Figure 3 shows a hierarchy of metadata management policy and standards. The overall metadata management policy refers to the data standards for business glossary, data stewardship, business rules, and data lineage and impact analysis. No data processes have been developed in this case.


Figure 3: Hierarchy of metadata management policy and standards


The metadata management policy is shown below:

We must have quality metadata stored within an enterprise metadata repository to manage business terms, technical metadata, business rules, data quality rules, master data rules, reference data, data lineage, and impact analysis.


The metadata management policy relates to the following metadata management standards:

  • Business glossary—The data governance team must maintain a business glossary with definitions of key business terms. These business definitions must be created and maintained for critical data elements using organizationally adopted naming and definition standards. The business glossary will also contain a data dictionary with the definitions of column and table names for key data repositories.
  • Data stewardship—Data owners must assign data stewards to manage business terms and other data artifacts, such as business rules.
  • Business rules—Business rules for critical data elements must be documented and kept up to date for each data repository and must be reflected within the business glossary.
  • Data lineage and impact analysis—The metadata repository must ingest metadata from key systems, including relational databases, data modeling tools, data integration platforms, reports, analytic models, and Hadoop. The lineage of data elements must be documented and should be up to date. Impact analysis must also be performed.

Reference Data Management Policy

Let's review a hierarchy of reference data management policies, standards, and processes. As shown in Figure 4, the reference data policy relates to the system of record and stewardship standards. The system of record standard, in turn, relates to the data processes for workflows and propagation.


Figure 4: Reference data management policy and standards



The reference data policy is shown below:

We must identify and manage critical reference data along with the code mappings across critical applications.


A brief description of the reference data standards is shown below:

  • System of record—A system of record shall contain an inventory of code tables along with a list of canonical or standard values and a mapping across systems. For example, "AL" is the state code for Alabama. The reference data system of record shall contain this information along with a mapping to state code "1" in another application because Alabama is the first state in an alphabetical list of states.
  • Stewardship—The EDM team must assign code tables to data owners or data stewards who will be responsible for adding, modifying, and deleting code values.


The system of record standard relates to the following data processes:

  • Workflows—Prebuilt workflows must be used to make changes to the code tables. For example, a European country such as "Kosovo" can only be added to the list of country codes based on a recommendation from European finance and approval from global finance.
  • Propagation—Reference data changes shall be propagated from the reference data hub to target applications across the enterprise. For example, when "Puerto Rico" is added in the state table, the reference data hub will propagate the change as state code "PR" in one application and as state code "51" in another application.

Email Address Policy

Email address may be a critical data element for companies that want to reach a large number of customers electronically. The following topics form part of an overall data policy for email address:

  • Completeness—The email address field should not be null.
  • Conformity—Email addresses must be in the format
  • Uniqueness—Email addresses must not be duplicated. This business rule addresses situations such as insurance agents entering their own email address in place of a customer's email address. The agents did this to avoid the insurer sending direct emails to the customer; however, the master data management (MDM) hub inadvertently matched different customers because they shared the same email address.
  • Timeliness—Customer service representatives should validate email addresses whenever they are on the phone with a customer. An email address is considered stale if it has not been validated in the previous 12 months.
  • Accuracy—The customer service department should identify any customers with bounced emails. Marketing must validate customer email addresses with external service providers such as Acxiom or Experian CheetahMail.

Facebook Data Policy

As part of the chief data office, the data governance team must also establish data policies for emerging types of data such as social media.

Establish Process for Data Policy Changes

As part of the chief data office, the data governance team must establish a process to manage changes to data policies, standards, and processes. The appropriate body, such as the data governance steering committee or data governance council, must approve these changes.

Editor's note: Learn more with the book The Chief Data Officer Handbook for Data Governance.

Sunil Soares

Sunil Soares is the founder and managing partner of Information Asset, LLC, a consulting firm that specializes in data governance. Prior to this role, Sunil was director of information governance at IBM, where he worked with clients across six continents and multiple industries. Before joining IBM, Sunil consulted with major financial institutions at the Financial Services Strategy Consulting Practice of Booz Allen & Hamilton in New York. Sunil lives in New Jersey and holds an MBA in finance and marketing from the University of Chicago Booth School of Business.

MC Press books written by Sunil Soares available now on the MC Press Bookstore.

Big Data Governance Big Data Governance
Discover not only the “why” but the “how” of governing big data.
List Price $59.95

Now On Sale

Data Governance Tools Data Governance Tools
See why tools are a critical component of a data governance program, and learn how to evaluate them.
List Price $59.95

Now On Sale

IBM InfoSphere: A Platform for Big Data Governance and Process Data Governance IBM InfoSphere: A Platform for Big Data Governance and Process Data Governance
Get to know the big data support across the IBM InfoSphere portfolio.
List Price $16.95

Now On Sale

Selling Information Governance to the Business Selling Information Governance to the Business
Learn best practices for implementing an information governance program across a variety of specific industries.
List Price $49.95

Now On Sale

The Chief Data Officer Handbook for Data Governance The Chief Data Officer Handbook for Data Governance
Implement a program that will manage data as an asset while delivering the trusted data your business initiatives require.
List Price $16.95

Now On Sale

The IBM Data Governance Unified Process The IBM Data Governance Unified Process
Learn the 14 steps to implementing data governance based on IBM products, services, and best practices.
List Price $24.95

Now On Sale



Support MC Press Online





  • White Paper: Node.js for Enterprise IBM i Modernization

    SB Profound WP 5539

    If your business is thinking about modernizing your legacy IBM i (also known as AS/400 or iSeries) applications, you will want to read this white paper first!

    Download this paper and learn how Node.js can ensure that you:
    - Modernize on-time and budget - no more lengthy, costly, disruptive app rewrites!
    - Retain your IBM i systems of record
    - Find and hire new development talent
    - Integrate new Node.js applications with your existing RPG, Java, .Net, and PHP apps
    - Extend your IBM i capabilties to include Watson API, Cloud, and Internet of Things

    Read Node.js for Enterprise IBM i Modernization Now!


  • Profound Logic Solution Guide

    SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation.
    Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects.
    The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the companyare not aligned with the current IT environment.

    Get your copy of this important guide today!


  • 2022 IBM i Marketplace Survey Results

    Fortra2022 marks the eighth edition of the IBM i Marketplace Survey Results. Each year, Fortra captures data on how businesses use the IBM i platform and the IT and cybersecurity initiatives it supports.

    Over the years, this survey has become a true industry benchmark, revealing to readers the trends that are shaping and driving the market and providing insight into what the future may bring for this technology.

  • Brunswick bowls a perfect 300 with LANSA!

    FortraBrunswick is the leader in bowling products, services, and industry expertise for the development and renovation of new and existing bowling centers and mixed-use recreation facilities across the entertainment industry. However, the lifeblood of Brunswick’s capital equipment business was running on a 15-year-old software application written in Visual Basic 6 (VB6) with a SQL Server back-end. The application was at the end of its life and needed to be replaced.
    With the help of Visual LANSA, they found an easy-to-use, long-term platform that enabled their team to collaborate, innovate, and integrate with existing systems and databases within a single platform.
    Read the case study to learn how they achieved success and increased the speed of development by 30% with Visual LANSA.


  • Progressive Web Apps: Create a Universal Experience Across All Devices

    LANSAProgressive Web Apps allow you to reach anyone, anywhere, and on any device with a single unified codebase. This means that your applications—regardless of browser, device, or platform—instantly become more reliable and consistent. They are the present and future of application development, and more and more businesses are catching on.
    Download this whitepaper and learn:

    • How PWAs support fast application development and streamline DevOps
    • How to give your business a competitive edge using PWAs
    • What makes progressive web apps so versatile, both online and offline



  • The Power of Coding in a Low-Code Solution

    LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed.
    Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

    • Discover the benefits of Low-code's quick application creation
    • Understand the differences in model-based and language-based Low-Code platforms
    • Explore the strengths of LANSA's Low-Code Solution to Low-Code’s biggest drawbacks



  • Why Migrate When You Can Modernize?

    LANSABusiness users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.
    In this white paper, you’ll learn how to think of these issues as opportunities rather than problems. We’ll explore motivations to migrate or modernize, their risks and considerations you should be aware of before embarking on a (migration or modernization) project.
    Lastly, we’ll discuss how modernizing IBM i applications with optimized business workflows, integration with other technologies and new mobile and web user interfaces will enable IT – and the business – to experience time-added value and much more.


  • UPDATED: Developer Kit: Making a Business Case for Modernization and Beyond

    Profound Logic Software, Inc.Having trouble getting management approval for modernization projects? The problem may be you're not speaking enough "business" to them.

    This Developer Kit provides you study-backed data and a ready-to-use business case template to help get your very next development project approved!

  • What to Do When Your AS/400 Talent Retires

    FortraIT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators is small.

    This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn:

    • Why IBM i skills depletion is a top concern
    • How leading organizations are coping
    • Where automation will make the biggest impact


  • Node.js on IBM i Webinar Series Pt. 2: Setting Up Your Development Tools

    Profound Logic Software, Inc.Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. In Part 2, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Attend this webinar to learn:

    • Different tools to develop Node.js applications on IBM i
    • Debugging Node.js
    • The basics of Git and tools to help those new to it
    • Using as a pre-built development environment



  • Expert Tips for IBM i Security: Beyond the Basics

    SB PowerTech WC GenericIn this session, IBM i security expert Robin Tatam provides a quick recap of IBM i security basics and guides you through some advanced cybersecurity techniques that can help you take data protection to the next level. Robin will cover:

    • Reducing the risk posed by special authorities
    • Establishing object-level security
    • Overseeing user actions and data access

    Don't miss this chance to take your knowledge of IBM i security beyond the basics.



  • 5 IBM i Security Quick Wins

    SB PowerTech WC GenericIn today’s threat landscape, upper management is laser-focused on cybersecurity. You need to make progress in securing your systems—and make it fast.
    There’s no shortage of actions you could take, but what tactics will actually deliver the results you need? And how can you find a security strategy that fits your budget and time constraints?
    Join top IBM i security expert Robin Tatam as he outlines the five fastest and most impactful changes you can make to strengthen IBM i security this year.
    Your system didn’t become unsecure overnight and you won’t be able to turn it around overnight either. But quick wins are possible with IBM i security, and Robin Tatam will show you how to achieve them.

  • Security Bulletin: Malware Infection Discovered on IBM i Server!

    SB PowerTech WC GenericMalicious programs can bring entire businesses to their knees—and IBM i shops are not immune. It’s critical to grasp the true impact malware can have on IBM i and the network that connects to it. Attend this webinar to gain a thorough understanding of the relationships between:

    • Viruses, native objects, and the integrated file system (IFS)
    • Power Systems and Windows-based viruses and malware
    • PC-based anti-virus scanning versus native IBM i scanning

    There are a number of ways you can minimize your exposure to viruses. IBM i security expert Sandi Moore explains the facts, including how to ensure you're fully protected and compliant with regulations such as PCI.



  • Encryption on IBM i Simplified

    SB PowerTech WC GenericDB2 Field Procedures (FieldProcs) were introduced in IBM i 7.1 and have greatly simplified encryption, often without requiring any application changes. Now you can quickly encrypt sensitive data on the IBM i including PII, PCI, PHI data in your physical files and tables.
    Watch this webinar to learn how you can quickly implement encryption on the IBM i. During the webinar, security expert Robin Tatam will show you how to:

    • Use Field Procedures to automate encryption and decryption
    • Restrict and mask field level access by user or group
    • Meet compliance requirements with effective key management and audit trails


  • Lessons Learned from IBM i Cyber Attacks

    SB PowerTech WC GenericDespite the many options IBM has provided to protect your systems and data, many organizations still struggle to apply appropriate security controls.
    In this webinar, you'll get insight into how the criminals accessed these systems, the fallout from these attacks, and how the incidents could have been avoided by following security best practices.

    • Learn which security gaps cyber criminals love most
    • Find out how other IBM i organizations have fallen victim
    • Get the details on policies and processes you can implement to protect your organization, even when staff works from home

    You will learn the steps you can take to avoid the mistakes made in these examples, as well as other inadequate and misconfigured settings that put businesses at risk.



  • The Power of Coding in a Low-Code Solution

    SB PowerTech WC GenericWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed.
    Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

    • Discover the benefits of Low-code's quick application creation
    • Understand the differences in model-based and language-based Low-Code platforms
    • Explore the strengths of LANSA's Low-Code Solution to Low-Code’s biggest drawbacks



  • Node Webinar Series Pt. 1: The World of Node.js on IBM i

    SB Profound WC GenericHave you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.
    Part 1 will teach you what Node.js is, why it's a great option for IBM i shops, and how to take advantage of the ecosystem surrounding Node.
    In addition to background information, our Director of Product Development Scott Klement will demonstrate applications that take advantage of the Node Package Manager (npm).
    Watch Now.

  • The Biggest Mistakes in IBM i Security

    SB Profound WC Generic The Biggest Mistakes in IBM i Security
    Here’s the harsh reality: cybersecurity pros have to get their jobs right every single day, while an attacker only has to succeed once to do incredible damage.
    Whether that’s thousands of exposed records, millions of dollars in fines and legal fees, or diminished share value, it’s easy to judge organizations that fall victim. IBM i enjoys an enviable reputation for security, but no system is impervious to mistakes.
    Join this webinar to learn about the biggest errors made when securing a Power Systems server.
    This knowledge is critical for ensuring integrity of your application data and preventing you from becoming the next Equifax. It’s also essential for complying with all formal regulations, including SOX, PCI, GDPR, and HIPAA
    Watch Now.

  • Comply in 5! Well, actually UNDER 5 minutes!!

    SB CYBRA PPL 5382

    TRY the one package that solves all your document design and printing challenges on all your platforms.

    Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product.

    Request your trial now!

  • Backup and Recovery on IBM i: Your Strategy for the Unexpected

    FortraRobot automates the routine tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:
    - Simplified backup procedures
    - Easy data encryption
    - Save media management
    - Guided restoration
    - Seamless product integration
    Make sure your data survives when catastrophe hits. Try the Robot Backup and Recovery Solution FREE for 30 days.

  • Manage IBM i Messages by Exception with Robot

    SB HelpSystems SC 5413Managing messages on your IBM i can be more than a full-time job if you have to do it manually. How can you be sure you won’t miss important system events?
    Automate your message center with the Robot Message Management Solution. Key features include:
    - Automated message management
    - Tailored notifications and automatic escalation
    - System-wide control of your IBM i partitions
    - Two-way system notifications from your mobile device
    - Seamless product integration
    Try the Robot Message Management Solution FREE for 30 days.

  • Easiest Way to Save Money? Stop Printing IBM i Reports

    FortraRobot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing.
    Manage your reports with the Robot Report Management Solution. Key features include:

    - Automated report distribution
    - View online without delay
    - Browser interface to make notes
    - Custom retention capabilities
    - Seamless product integration
    Rerun another report? Never again. Try the Robot Report Management Solution FREE for 30 days.

  • Hassle-Free IBM i Operations around the Clock

    SB HelpSystems SC 5413For over 30 years, Robot has been a leader in systems management for IBM i.
    Manage your job schedule with the Robot Job Scheduling Solution. Key features include:
    - Automated batch, interactive, and cross-platform scheduling
    - Event-driven dependency processing
    - Centralized monitoring and reporting
    - Audit log and ready-to-use reports
    - Seamless product integration
    Scale your software, not your staff. Try the Robot Job Scheduling Solution FREE for 30 days.