Sat, Sep
0 New Articles

Product Review: Vision Solutions' Data Manager

Product Reviews
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times
Many IT departments have had archiving on their project lists for some time, but it is continually bumped down the list by other items that are considered to be of a higher priority, such as new application development and bug fixes. Archiving is an administrative task that is seen as being on the "should do" list, but, because the business will still function without it—albeit, as described below, slowly and not in the most cost-effective manner—archiving never rises onto the "must do" list in some organizations.

Why does this matter? The obvious answer is that, despite the dramatic decline in disk costs over the past couple of decades, disk is still not free. Beyond that, there are performance implications to keeping more data on hand than necessary. Database backup tasks and batch jobs that run against the whole database take longer. Response times for online queries may also become unacceptable as the queries sift through excessive obsolete data.

Furthermore, more data is not always better than less data. Very old data may mislead business planners and strategists because trends are discontinuous. New product introductions by the company or its competitors, changing consumer tastes, new business models, new technologies, and other factors alter customers' buying behaviors, internal inventory flows, and other operating characteristics. Inferring and extrapolating trends using information that spans too long a time may be not only a slow process because of the volume of data, but also a misleading one.

Data Manager tackles this problem by moving old data from the primary database to either a secondary database on disk or off to tape. It is, in effect, an archiving engine. You provide it with a model that defines your databases and your archiving policies. On an ongoing basis, it then automatically archives data according those policies, without the need for operator intervention.

For example, you might tell Data Manager to move "old" data—data that is not needed on a regular basis, but which still may be required on short notice—from the primary database to a reference database on disk. You can also have it move "older" data from the reference database onto tape in order to reduce disk costs. Alternatively, you might set up just a single step from the primary database directly to tape. You define "old" and "older" in the model.

Comprehensive Archiving Policies

The "old" and "older" concepts are not as straightforward as they might seem. For example, you might want to archive all purchase orders more than three years old, but what does "three years old" mean? Standing purchase orders may be open for months or even years, and the various line items on one-time purchase orders may be shipped individually over a span of months. You don't want to archive an open purchase order or one that may still need to be referenced because it was filled only two months ago and it or the payment of it may still be open to dispute. Data Manager allows you to establish detailed archiving policies that accommodate these issues. For example, you can have it archive purchase orders that were created more than three years ago, but only if they are completely closed and nothing has been received on them for, say, at least six months.

Data Manager also honors all referential integrity rules. Even if a record meets all of the other archiving policies, Data Manager will not archive it if dependent records exist that are not scheduled for archiving.

To help ensure compliance with data security and management accountability regulations, Data Manager maintains a complete audit trail of all actions it performs. This audit log shows not only where Data Manager moved data, but also when and how the archiving criteria were changed over time.

ERP Database Archiving

ERP applications often provide purging and archiving features, but these features are usually not comprehensive, and worse, they normally require that the system be shut down while they run. In contrast, Data Manager provides comprehensive purging and archiving functionality. And Data Manager's processes can be run while systems are active. Many organizations find that this benefit alone delivers a return on their investment almost immediately.

Working with Data Manager is simple. First, you load the archiving model. Models have been built for most of the major ERP packages, including BPCS, MOVEX, System
21, World, and OneWorld, and a growing number can even be downloaded from Vision's Web site. These models can then be customized to accommodate your specific archiving policies and any tailoring that your organization performed on the ERP. Alternatively, models can be easily built from scratch.

Archive Modeling

This is not to suggest that Data Manager is only for environments running ERP software. Data Manager is entirely application-independent. Its archiving model is simply a means of defining files and links between files, archiving and purging policies that are to be applied to them, and identifying any unusual conditions that require exceptions to those policies. The model is completely independent and not directly aware of the applications that use the files that are included in it.

Archive Process Control functionality in Data Manager creates, in effect, development and test environments for the archiving model. You can build new models or modify existing ones in the development environment without affecting the purging and archiving activities going on in the production environment. Those activities continue to follow the policies in the existing, proven model. When you have finished creating or modifying a model and have thoroughly tested it, you can promote it to the production environment. Only then will the policies in the model under development replace those in production.

Hierarchical Archiving

Data Manager can implement either two-tier or three-tier archiving models. All IT people are familiar with the two-tiered model. Under it, old data is sent to tape and deleted from disk. The archived data can be retrieved from tape if necessary, but doing so is a very long and, assuming the tape cartridge is not mounted in the drive or automated tape library, labor-intensive process. Consequently, old data is often not archived until the IT staff is fairly certain that it will never, or at least exceptionally rarely, be called for again.


Figure 1: Here's an example of a three-tiered hierarchical archiving model. (Click image to enlarge.)

A three-tier model includes the option of archiving data to disk. For example, data that will not be needed for operational purposes but is still required for reporting and analysis purposes is archived to a reference disk rather than to tape. Doing so gets the data off the operational database so it doesn't slow down production applications but keeps it quickly available when needed for other purposes.

Older data, typically data that is obsolete but must still be kept for legal purposes or "just in case" is moved off disk completely and onto tape, just as it would be in the two-tiered model.
The three-tier model does not require that data always follow this two-step process. Depending on the nature of the data, you might specify that some of it is to move from the primary database to the reference disk once it reaches a certain age and then later onto tape, while other data may move directly from the primary disk to tape.

Copying Without Deleting

Data Manager allows you to copy data without deleting it from the primary database. This can be helpful when testing your model as you can defer the purging of production data until you are completely confident that the archiving rules are correct. Even then, there is no threat to your data as Data Manager allows you to easily restore archived data to the production database.

The ability to copy but not delete selected data provides an ancillary benefit. You can use this feature to easily create high-quality, up-to-date test databases with referential integrity fully intact. This allows developers to test new and updated applications under current real-world conditions.

Ease of Use and Administration

Vision Solutions has put considerable work into minimizing the administrative load required to install and run Data Manager. A simple, query-like interface allows you to model your purge and archive plan without the need for programming skills.

Data Manager is implemented in conjunction with Director, an integrated set of applications that proactively monitors, manages, and optimizes System i servers, databases, and application environments. By combining Data Manager with Director, you can free up more space by using the reorganization-while-active functionality, gain greater visibility into your System i, and keep your System i optimally tuned.

Data Manager and Director come with self-installing engines and, installed in this combination, they demonstrate their heritage in being built on the eight industry-standard pillars of autonomic computing as described below:

  1. Self-Aware—It manages itself with respect to resources, configurations, and parameters.
  2. Self-Configuring—It enables self-installation, -set up, and -configuration.
  3. Self-Optimizing—It monitors and manages itself, adjusting to the business needs and priorities.
  4. Self-Healing—It detects, anticipates, and can recover from routine and extraordinary events.
  5. Self-Protecting—It detects, identifies, and responds to anomalies in the system to maintain overall system performance and integrity.
  6. Self-Adapting—It is aware of itself and its environmental context and will manage its own resource allocation.
  7. Self-Managing—It monitors its system resource requirements and will regularly manage resource usage for optimal performance.
  8. Self-Anticipating: It understands its environment and forward resource requirements for various tasks, constantly "looking ahead" to ensure that tasks can be started and completed within defined windows of processing time or will reschedule as appropriate to optimize system use for the business.

For more information about Data Manager, contact Vision Solutions at one of the addresses below.


Vision Solutions, Inc.
17911 Von Karman Ave.
Irvine, CA 92614
Tel: 949.253.6500
Fax: 949.253.6501
Web: www.visionsolutions.com
Email: This email address is being protected from spambots. You need JavaScript enabled to view it.


Joel Klebanoff

Joel Klebanoff is a consultant, writer, and formerly president of Klebanoff Associates, Inc., a Toronto-based marketing communications firm. He has 30 years' experience in various IT capacities and now specializes in writing articles, white papers, and case studies for IT vendors and publications across North America. Joel is also the author of BYTE-ing Satire, a compilation of a year's worth of his columns. He holds a BS in computer science and an MBA, both from the University of Toronto.

MC Press books written by Joel Klebanoff available now on the MC Press Bookstore.

BYTE-ing Satire BYTE-ing Satire
Find out the hilarious answer to the eternal question: "Is technology more hindrance than help?"
List Price $14.95

Now On Sale



Support MC Press Online

$0.00 Raised:

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: