Planning for Successful Cloud Data Migrations

    By: Mark Van de Wiel on Jul 17, 2018

    HVR Homepage Feature-7.18.png

    By Mark Van de Wiel, CTO, HVR, an IOUG partner

    This post provides a set of best practices to facilitate cloud migrations with minimal downtime and minimal risk, irrespective of the type of cloud service you plan to adopt, focusing on the application data that has to be migrated. Generally, databases store application data and change continuously. Application servers, interfacing with the database, can be migrated more easily using a lift and shift approach because they don’t store the data.

    The amount of effort to be put into a cloud migration depends on the complexity of your IT infrastructure. To achieve minimal downtime, take a phased approach to cloud migration, in which system by system or application by application will be migrated, rather than an entire data center at the time. Organizations with hundreds if not thousands of applications, databases and systems need to prepare for a long-term cloud migration effort.

    In preparation for a cloud migration with minimal downtime, there are multiple considerations to take into account:

    • Network utilization, and related to that, performance. In an environment with data being transferred across a Wide Area Network (WAN), it is important to make optimum use of network resources, especially given some cloud providers will charge for data transfers into the cloud.

    • Security. To achieve minimal downtime, some if not all of the data will have to be transferred across the wire into the cloud. Do this securely - e.g. using SSL - to avoid data breaches.

    • Firewall requirements. Related to security, what are the requirements to open the firewall? Will the cloud have to be able to reach into the corporate data infrastructure, and if so, how can you limit exposure?

    • Scale. Especially as a large organization, with possibly hundreds of databases, you want to be able to perform multiple migrations at any one time, make sure performance is adequate, and the setup is manageable.

    As part of the migration to the cloud, you may consider changing the technology you use for your data platform. Maybe you plan to change technology version, or perhaps you consider changing the underlying technology altogether.

    Migrating to the cloud involves the following steps:

    1. Instantiate the target database.

    2. Create the target database structures, including user accounts, tables, indexes, constraints, stored procedures and any other database objects.

    3. Perform the initial data load. Consider data volume, and means of transferring the data, when planning this step. When using network communication, consider available network bandwidth and whether data can be compressed.

    4. Synchronize the database. To achieve minimum downtime, and assuming the instantiation will take a significant amount of time, there is a need to synchronize the database with changes since the initial load took place.

    5. Switch on-premises use of the application(s) to the cloud. The switch will involve downtime, but if well-planned, the downtime associated with the switchover will be minimal. At this point consider a continuation of synchronizing the databases, with data now flowing out of the cloud back into the on-premises database.

    Throughout the migration to the cloud it is important to perform tests to ensure your migration will be successful. For example, after the initial database load or as systems are synchronized, validate the data and attempt to mimic a load that matches the production load when the application migration is complete.

    HVR for Cloud Migrations

    The HVR solution, built for continuous, real-time data integration between database technologies in a heterogeneous environment, is ideally suited to manage cloud database migrations with minimal downtime. HVR supports a multitude of commonly-used relational database technologies, both as a source log-based Change Data Capture (CDC), and continuous integration. The HVR architecture prescribes a distributed setup in which one installation acts as the control center – the hub – and other installations are referred to as agents. Thanks to HVR’s rich support for database platforms and versions, a technology migration as part of the move to the cloud is trivial.

    HVR is used to perform the initial data load, CDC with continuous integration, and compare/repair. The use of initial load is integrated with CDC to align initial data load with continuous replication to avoid data loss. HVR provides multiple options when performing log-based CDC from Oracle, including capture from a physical standby database, and archive log only mode. To achieve optimum efficiency the HVR agent should be installed on the server(s) that perform(s) log-based CDC, whether that is the primary, the standby or an archive log only machine.

    With its heritage in database replication the HVR technology is also extremely resilient to any environment issues during the migration like temporary loss of network and database or server restarts.


    Mark_450x534.jpg


    Mark brings a strong background in data replication as well as real-time Business Intelligence and analytics to his role here at HVR. Most recently, at Actian Corporation, Mark led a team of high performing solution architects. Prior to that, Mark developed extensions to Oracle’s GoldenGate replication product.

    Throughout his 15+ year career, Mark has expanded his responsibilities from consultant to product management and business development. Mark holds an MSc in Industrial Engineering and Management Sciences from the Eindhoven University of Technology.

    When Mark isn’t thinking about data replication and analytics, he’s running marathons. In fact, he’s run more marathons than his age in years and boosts his total by running in the San Francisco marathon each year.


    Released: July 17, 2018, 1:53 pm | Updated: July 18, 2018, 12:25 pm
    Keywords: Feature


    Copyright © 2018 Communication Center. All Rights Reserved
    All material, files, logos and trademarks within this site are properties of their respective organizations.
    Terms of Service - Privacy Policy - Contact

    Independent Oracle Users Group
    330 N. Wabash Ave., Suite 2000, Chicago, IL 60611
    phone: 312-245-1579 | email: ioug@ioug.org

    IOUG Logo

    Copyright © 1993-2018 by the Independent Oracle Users Group
    Terms of Use | Privacy Policy