Mon, Sep
3 New Articles

Protect Data During Migrations to Virtual Systems

High Availability / Disaster Recovery
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

Migrating to a virtual system to consolidate servers can be a time-consuming, labor-intensive job that places your data at risk, but it doesn't have to be.


The benefits of using virtual systems to consolidate physical hardware have been widely discussed and generally accepted. Server consolidation can reduce hardware costs, software-licensing fees, power consumption, HVAC costs, and the burden of server management and maintenance. In addition, a single physical server containing multiple virtual severs can be easier to secure than multiple physical systems.


The benefits may be widely written about, discussed, and recognized, but the process of migrating from existing physical systems to consolidated virtualized servers doesn't receive as much attention in the media.


Such virtual migrations provide all of the excitement of any hardware migration. It will likely be impractical, and perhaps impossible, to take the disks attached to a physical server, attach them to a new virtual server, and then have the applications run on the virtual sever without problems. Consequently, it will be necessary to migrate both data and applications to the new server by some other means.


Depending on how it's performed, this migration may create a number of challenges and incur major costs.


The traditional way to migrate data was to dump it off the old system and then reload it on the new one. If the transfer medium is tape, this process can easily consume several hours, especially if it includes large files and databases. Even when using disk or electronic transfers, the unload/reload process can consume considerable time.


A key issue you should consider when using an unload/reload migration strategy is that from the moment the unload process begins until the moment the data and applications are loaded and are ready to run on the new system, all user activity must stop. Otherwise, transactions entered on the old server may not be copied to the new one. And if users begin working on the new virtual server before it's fully loaded, their applications may fail or new transactions may be corrupted as the copied data is loaded onto the system.


A lengthy shutdown might be an option for an organization that can afford to live without its systems for a weekend. However, that luxury isn't available to many organizations today. Worldwide operations, Internet-based e-commerce, and increased competition have dramatically narrowed and often eliminated maintenance windows to the point where most organizations now demand a less disruptive means of migrating data and applications.


Business disruptions aren't the only issue. Manually unloading and reloading data is an error-prone process. The wrong tape may be loaded onto the new system. Data may be damaged or lost in transit. And the load process may inadvertently corrupt the data. Depending on the nature of the data corruption, the error might not be found until the applications are restarted and then fail because of the corrupted data. Even worse, the data corruption might not be found until the numbers don't add up at month-end or customers call to complain about missing orders or errors in their accounts.


A company might choose to reduce the work involved in a tape-based migration by using a regular backup tape as the migration media. This introduces a small threat. The longer backup tapes are kept onsite, the higher the risk that they will be destroyed by a disaster. If the most recent backup tapes are destroyed, it will be necessary to recover data from an older generation of the tapes, resulting in the loss of more data than would otherwise be the case.


While the probability that a disaster will occur while you're migrating to a new system is exceptionally small, the reality is that disasters do occur and create substantial disruption to business operations. A healthy awareness of this possibility should serve as ample motivation to seek lower-risk alternatives.


Migrate-while-active solutions can eliminate many of the drawbacks of traditional migration techniques. These technologies vary depending on the vendor, but in general they create a mirror of the production databases, user files, system data, and applications on a new server.


Typically, as the name of this class of software implies, applications can remain active while the mirror is being created. The software automatically detects any updates applied during the migration process and copies them to the target system. This ensures that the target system will be up to date when the migration is complete.


Using this type of solution, it is usually not necessary to switch users to the new server as soon as the virtual sever mirrors the production server. Most migrate-while-active software solutions can keep the new virtual server synchronized with the old physical server for as long as necessary. That way, you can fully test the new server and switchover only when you are certain that it is ready to support operations.


Advanced migrate-while-active software also performs complex synchronization checks and fixes any out-of-sync data while it is creating the mirror and until the switchover is complete and the migrate-while-active software is turned off.

The HA Option

If you employ high availability (HA) software, you might not need to use a migrate-while-active solution to achieve the same benefits. In fact, if a vendor sells both migrate-while-active and HA software, the migrate-while-active solution may employ the core technologies of the HA software with few, if any, modifications. For instance, it is possible to use HA software for a complex migration; then, as an added byproduct of the migrate-while-active software, the customer can wind up with a fully functional HA solution already online!


HA software maintains a replica of a production server's data, applications, and system objects on a backup system. It also provides the ability to fully or partially automate the process of switching users to the backup system when necessary. This is exactly the functionality required to perform a migration from one server to another with little or no downtime.


Server virtualization is designed such that, to the software running on the virtual server, a virtual server is indistinguishable from a physical server. Thus, the HA software can be used to effect a migration from a physical system to a virtual one just as easily as if it were replicating between two physical servers or two virtual servers.


If you are already running HA software to maintain a backup server, you can simply designate the new virtual server as the backup and have the HA software create and maintain a new replica there. Then, when you are ready to assign the production role to the virtual server, you can use the HA software's "role swap" functionality to perform the switch.


After cutting over to the new virtual server, you might choose to use the old production hardware as the new backup system. If so, no further action is required as that is what a role swap does; it swaps the production and backup roles on the two systems.

If you want to ensure complete protection of your data throughout the migration process, you can maintain replicas on both servers simultaneously rather than discard the old backup system before creating a production mirror on the new virtual server. That way, there will never be a time when you are left without a ready-to-run, hot-standby backup of your production server.


Depending on the HA solution, the software might be able to perform this dual replication in either of two modes. In broadcast mode, the initial mirror on the new virtual server would be created from the production system. The HA software would then replicate data from the production system to both the backup system and the new virtual server simultaneously, keeping them both continuously synchronized with the production server.


The other option is to use a cascade mode. Under this topology, the mirror on the new virtual server is created from the backup system, rather than the production system. The HA software is then used to replicate ongoing updates from the backup system to the virtual system. Because the HA software replicates data in real-time or near real-time for both transmissions—from the old production server to the backup and from the backup to the new virtual server—there is little or no synchronization latency between the production server and the new virtual server.


Cascade mode offers the advantage of not adding any processing and disk I/O load to the production server.

Migration as an HA Opportunity

If you currently don't use an HA solution, server virtualization might provide a good opportunity to implement one. As described above, the HA software can serve as a data migration mechanism that will assure migration accuracy, eliminate the potential for human error, and reduce the migration workload.


A migrate-while-active solution will also provide all of these benefits, and, because it is dedicated solely to performing the migration, it usually costs less than an HA solution. But with an HA solution, you are left with an environment that is much more highly available than the one you started with.


After consolidating servers, if you retain one of the old systems for use as the backup server in your new HA environment, you might choose to add virtual server partitions to it, allowing it to serve as a backup for a number of production virtual servers. In doing so, you gain all of the benefits of virtualization for your backup server as well.


Because the old hardware was likely sized sufficiently to run only its expected load, it will likely not be adequately sized to handle the load of all of the organization's virtual servers. However, if maintenance can be scheduled for off-peak times and some operations can be curtailed during emergencies, then reducing the cost of maintaining an HA environment by virtualizing the backup servers on one of the old systems might be a viable option.


In addition, an HA solution often serves not only to recover from a disaster, but also to keep an application going while its virtual server's operating system is being upgraded, its databases are being backed up or reorganized, or other maintenance is being performed on the application. In this case, the backup server assumes a production role for only the applications running on that single virtualized server, not for all of the virtual production servers.


Furthermore, there is no reason why there has to be only one backup server. Clearly, if you retained all of the old hardware and used those systems as backups, you would forfeit the cost benefits expected from server consolidation. However, if you were, for example, consolidating six servers onto a single physical server running six virtual servers, you might choose to retain two of the old systems as backups. You could then set up three partitions on each of those systems and split the backup load between them. This would reduce your hardware requirements from six systems to only three.

Data Replication

A standalone data replicator offers another way to partially automate a migration from a physical to a virtual server. Data replicator functionality varies somewhat from vendor to vendor, but they are generally able to copy all data from one server onto another server and then keep the two servers synchronized by capturing all changes made on the source server and replicating them to the target server.


Like HA and migrate-while-active software, this automates the data migration and, therefore, reduces the workload and the potential for human error. And, because the data replicator can maintain synchronization indefinitely, you can take as long as you like to test out the new virtual server, confident that the business data on it will be fully up to date when you're ready to switchover. One additional benefit of this type of replication solution is that it can also perform simple transformations of data across different databases, which provides another degree of freedom in the migration process.


However, a data replicator doesn't offer some of the benefits of HA and migrate-while-active software. For example, depending on the replicator, it might replicate only database data, not the system variables and applications required to create a functioning, up-to-date server on the virtual machine.


In addition, switchover capabilities that automate most or all of the processes required to make the new virtual server the active production machine are generally standard features in HA software and, possibly, in migrate-while-active software, but they are lacking in standalone replicators.

Continuous Data Protection

Organizations often clean up their data before migrating to a new virtual server. This may include finding and deleting obsolete files and reorganizing databases. Doing so reduces the amount of data that must be migrated and helps to improve efficiency and reduce storage costs on the new virtual server.


These housekeeping activities can be a source of considerable human error. The reason for this is that the cleanup work can involve a high percentage of manual processes. Under the pressure of the work involved in the migration, an operator might accidentally delete a critical file, mistaking it for an obsolete one. A sophisticated system administration tool can reduce the possibility of these errors by providing facilities to help operators to find obsolete files, archive them, delete them, and "undo" the deletes if necessary.


Yet, even with such tools in place, accidents do happen. HA, migrate-while-active, and data replication software can do nothing to reverse an error such as an accidentally deleted file because the replicators in those products will faithfully duplicate the deletion on the target system in an instant. Tape backups are often the only way to recover individual data items that are lost or damaged in this way. However, if the tape is offsite, it can take considerable time to retrieve the tape and restore the data, creating significant downtime. What's more, if the deleted or damaged file is updated frequently throughout the day, recovering it from the previous night's backup tape may result in the loss of several updates.


Continuous Data Protection (CDP) can help to resolve these problems. CDP software captures changes made to production data and uses specialized techniques to copy them to a secondary server, creating a type of "undo" buffer. Depending on the CDP software, the backup machine may not have to run on the same platform as the production system. Instead, it may be possible to employ, for instance, a low-cost, low-powered Windows or Linux system for this purpose.


Because a true-CDP solution captures and copies data updates continuously, you can use it to restore data to its state at any point in time, not just to its state at sometime the previous night, as is the case when recovering from tape. And, because it is a disk-to-disk backup and recovery solution, CDP allows for very rapid data recovery compared to tape-based solutions.


Not all CDP products are true-CDP solutions. Near-CDP products transfer data updates to the backup server only at particular intervals, possibly just when a file is closed. With near-CDP, it might not be possible to recover data right up to the point it was deleted or damaged, but you can generally come closer to that point than would be the case with tape-based recovery.

Simple, Risk-Free Migrations

The old approach to migrations, whether those migrations are done to consolidate existing physical servers onto consolidated virtualized servers or for any other purpose, typically involved unloading data and applications onto tape or some other media, reloading them onto the new server, and then validating the results. This process was labor-intensive, complex, and error-prone. Worst of all, it required considerable downtime and unpredictable risk.


HA and migrate-while-active technologies can reduce the human resource requirement, simplify the process, eliminate most opportunities for human error, and virtually eliminate the required downtime. Furthermore, for those who do not yet experience the benefits of HA, using HA software for the migration can result in an HA-protected environment as a byproduct of the migration.



Henry Martinez

As Senior Vice President of Engineering, Henry Martinez has a broad spectrum of experience in the application of information technologies (IT), including traditional IT, consumer electronics, the utility industry, aerospace, and industrial automation. He has managed global, virtual R&D labs; launched high-technology startups; established domestic and international manufacturing operations; and negotiated and implemented technology alliances/transfers.


Prior to joining Vision Solutions, Henry has been Senior Vice President of New Product Development for CCC Network Systems, Vice President and General Manager of Excellon Automation's Laser Business Division, Director of Strategic Technologies for Leviton Manufacturing, and Research & Development and Unit Manager for Northrop Electronics. Martinez holds a BSEE from UCLA with graduate studies in computer science and has participated in Executive Management programs at Stanford University and the California Institute of Technology. Additionally, he is a member of the Institute of Electrical and Electronics Engineers (IEEE), has authored article in several technical publications, and holds two U.S. patents.


Henry may be reached at This email address is being protected from spambots. You need JavaScript enabled to view it.. For further information about Vision Solutions, visit http://www.visionsolutions.com/ or call 801.541.7769.



Support MC Press Online

$0.00 Raised:

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: