Exadata Deployment Basics

    By: Umair Mansoob on Mar 14, 2017

    Exadata Deployment Basics

    By Umair Mansoob  ◾  Edited by Mike Gangler

    The Oracle Exadata Database Machine was first introduced by Oracle in 2008 and it has become one of the most popular database platform to host Oracle databases. Exadata Machine is like a mini data center in itself, comprising of Database Servers, Storage Servers, InfiniBand Switches and Ethernet switch working together to deliver outstanding database performance.

    Over the years, Exadata went through different deployment processes, as Oracle tries to make it simpler and error free. This article will take you through different phases of Exadata deployment and clearly define responsibilities of each group involved, during each phase. At minimum, you should treat Exadata deployment as a small project and permanently dedicate resources for it.

    Basically, Exadata deployment will involve many phases, and it can take up to three weeks before you can log in to Oracle Database running on your Exadata Machine. It starts by planning for target Exadata Machine, which includes” capacity planning, key deployment decisions and licensing considerations.

    Once you have narrow-downed your Exadata configurations, you can download and run Oracle Exadata Deployment Utility to generate different configuration files. OEDA utility will also provide the list of software and patches that you need to download from Metalink. After downloading and copying all the software to target Exadata Machine, you can apply Elastic Configuration and deploy Oracle software to Exadata Machine. Finally, verify all the software and hardware components once Exadata deployment is completed.

    Overview

    Planning

    Exadata is a big investment, so it important that you plan Exadata deployment very carefully.  Following are some important considerations for your Exadata deployment.

    Capacity Planning:

    You might be buying Exadata Machine for a particular application, or you may be planning to consolidate databases to Exadata platform. In any case, you should perform a detailed analysis of your existing database or databases, in order to do capacity planning for your Exadata Machine.

    It is important to understand that Exadata offloads many processes to storage nodes, therefore requires less CPUs on compute nodes and also provides extraordinary compression ratio using Hybrid Columnar Compression, so it requires less storage. Capacity planning for Exadata is a little different from traditional hardware, but basics will still apply. You should gather information about CPU, IOPs and Physical Memory of an existing system and use them to size your Exadata Machine.

    Deployment Decisions:

    Once you have sized your Exadata Machine, it is time for you to make important deployment decisions. Exadata deployment decisions can have lasting impacts on your hosted databases. For example, changing ASM disk group redundancy will require recreating that disk group causing application outage. Therefore, it is important to consider Exadata deployment decisions in detail before moving to the next phase of Exadata Deployment Life Cycle, including:

    • ASM Redundancy Level,
    • Virtualization,
    • Network Isolation,
    • Role Separation and
    • Storage Type.

    ASM redundancy is probably the most important Exadata deployment consideration, since it will be hard to change it after the deployment. Though Oracle recommends using high redundancy level for Exadata Machine, you have an option to choose normal redundancy based on your requirements. If you are planning to deploy non-production databases, you might want to use normal redundancy to efficiently utilize expensive Exadata storage. It is also important to note that external ASM redundancy is not supported with Exadata systems.

    As many of you already know, virtualization is supported with the release of X5 and it has become an important deployment consideration for many customers. I will not get into details in this article, but there are many benefits of using virtualization with Exadata Machine including license cost savings, security and isolation. It is also important to understand that virtualizing Exadata Machine comes with administrative overhead.

    Now you have multiple clusters to maintain instead of one. Network isolation can also be important if you have consolidated numbers of databases to Exadata Machine. You can achieve network isolation using two options: private VLAN and InfiniBand partitioning. You may also want role separation: You can deploy Exadata with Oracle user being a sole owner of GRID and RDBMS software, or you can have role separation by having two separate owners for GRID and RDBMS software. You can choose either way based on your organization standards.

    Next, you need to decide what type of storage disk you want for your Exadata Machine. Exadata Machine comes with two storage disk options: Extreme Flash I/O and high capacity. As their names suggest, one is designed to provide with more performance and better I/O response time, whereas the other is designed to provide you with higher storage capacity.

    My recommendation will be to choose high capacity disks because new Exadata machines come with terabytes of flash cache and your application will be performing most of the read and write from flash cache. Similarly, you need to decide backup location for your databases running on Exadata Machine. Exadata storage is not cheap, but it’s best to use an Oracle-supported NFS mount, like ZFS or any other Oracle-supported backup solution.

    Licensing Considerations:

    When you buy Exadata Machine, you will also at least need to buy Oracle Database Enterprise Edition and Exadata Storage Software licenses. Given that Exadata compute nodes come with a higher number of CPUs, you are looking at significant investment to purchase Exadata Machine.

    As a customer, you have two options to control Exadata software licensing cost:

    1.  The Capacity on Demand (CoD) option to disable a subset of the cores on Exadata database servers to reduce licensing requirements. The maximum number of cores that can be disabled on an Exadata X5-2 Database Server is 60 percent.
    2. Exadata now supports Oracle Virtual Machines (OVM). OVM can be used to create virtual machines with specific number of cores to reduce licensing requirements, similar to the CoD option.

    You can also use a virtual machine to save money on licensing, since the options can be licensed on just the virtual machines using them instead of every physical core in the server. Additionally, if you are planning to use Oracle Advanced Compression or Oracle Advance Security as an option and you don’t have existing licenses for it, consider buying them for the following reasons:

    • You might be required to buy Oracle Advanced Security for compliance reasons, especially if your database contain credit card information and you need to encrypt data at rest and on the network.
    • Though Exadata comes with its own free compression called Hybrid Columnar Compression (HCC), it does not support online transaction processing (OLTP) operations.

    Oracle Advanced Compression can help you save on expensive storage and it can also improve performance by reducing I/O footprint.

    ODEDA Utility

    If you are one of early adapters of Exadata Machine, you probably remember the old Exadata configuration excel sheet used for Exadata installation. A few years back, Oracle introduced a Java-based utility called Oracle Exadata Deployment Assistant (OEDA) in an effort to simplify the Exadata installation process.

    You need to download latest OEDA version and generate Exadata configuration files before the delivery of Exadata Machine. Once downloaded, work on the network and database administrators to fill out the following sections:

    • IP addresses (Private & Public Network).
    • Domain Name System (DNS).
    • Network Time Protocol (NTP).
    • Cluster name.
    • Disk group name and redundancy.
    • Email information for alerts.
    • Oracle Configuration Manager (OCM).
    • Auto Service Request (ASR).
    • Grid Control Agent (OEM 12c).

    Oracle Exadata Deployment Assistant Downloads

    If you are able to successfully run OEDA utility, it should generate following configuration files for Exadata installation:

    • OEDA XML file.
    • Network Check Script.
    • Installing Template.
    • Pre-configuration Files. 

    Download Software

    This deployment phase can be completed in parallel with any other phase, as long as you have run OEDA utility. It is important to understand that you will not be able to deploy Exadata software without downloading all the necessary software and patches from My Oracle Support. You can get a complete list of software and patches from Exadata configuration file generated by OEDA utility.

    This phase consists of two parts: Download software and patches from My Oracle Support, then transfer them to Exadata Machine. You can download software and patches before you have possession of Exadata Machine, and once Exadata Machine is on your network, you can transfer them to your machine.

    Here is an example of software and patches from the Exadata configuration file:

    Software & Patches from Exadata Configuration

    Elastic Configuration

    By now, you should have the Oracle Exadata Database Machine shipped to your data center and it is ready for the next deployment phase. New Oracle Exadata machines are shipped with factory default IP addresses and these addresses are in the 172.16.2.1 to 172.16.7.254 range. Prior to connecting Oracle Exadata Database Machine to the network, ensure these IP addresses do not conflict with other addresses on the network

     I strongly recommend running the checkip.sh script before connecting the network to avoid problems, even if a check was performed as part of the planning process before the machine was delivered. This is a very important step and you only need to run this step from one node only. This step will use your Exadata configuration file to assign new IP addresses and reboot all Exadata nodes including storage nodes.

    Once this phase is complete, connect through SSH and run IBHOSTS to see if new IP addresses have been assigned. It is important that you don’t see any elastic keyword on the screen; otherwise, it shows that your elastic configuration is not complete.

    Example of running elastic configuration:

    ./applyElasticConfig.sh -cf customer_name-configFile.xml

    Elastic Configuration

    Now you can connect the Exadata Machine to your network.

    Software Installation

    During this phase, you install and configure Oracle software — including Oracle Grid and RDBMS on Exadata Machine. If you have ever installed Oracle Real Application Clusters in your organization, this phase will look very familiar to you. This phase is similar to installing Oracle Real Application Clusters on any other supported hardware configuration but more comprehensive.

    This process was developed by Oracle engineers to simplify and automate software installation process on Exadata Machine. This phase consists of 19 steps and you can run them one by one or you can run all of them together. I strongly recommend doing it one by one.

    The following steps are performed by the Oracle Exadata OneCommand Utility and actual steps for a specific deployment may differ depending on your Exadata configuration. For example, you have to perform an extra step “Update Nodes for Eighth Rack” only for eighth rack deployment.

    1. Validate Configuration File.
    2. Update Nodes for Eighth Rack.
    3. Create Virtual Machine.
    4. Create Users.
    5. Setup Cell Connectivity.
    6. Create Cell Disks.
    7. Create Grid Disks.
    8. Configure Alerting.
    9. Install Cluster Software.
    10. Initialize Cluster Software.
    11. Install Database Software.
    12. Relink Database with RDS.
    13. Create ASM Disk groups.
    14. Create Databases.
    15. Apply Security Fixes.
    16. Install Exachk.
    17. Setup ASR Alerting.
    18. Create Installation Summary.
    19. Re-secure Machine.

    Once you complete all the 19 steps successfully, you have almost completed your Exadata deployment. Please make sure to change all the root passwords when logging in for the first time. All the operating system users are configured with default passwords.

    Validate Installation

    As mentioned earlier, your Exadata deployment is almost complete, but you should validate your Exadata installation before you start deploying any database on it. Exadata Machine is like a mini data center, containing many different types of hardware, including database servers, storage and network switches. You should validate all the Exadata components including software and hardware

    You can start by reviewing log files from Exadata software deployment phase; each step will generate a log file. Secondly, review Exachk and the installation summary report generated by Oracle Exadata OneCommand Utility for any reported errors. If you find any issues during this phase, it is important to get My Oracle Support involved to resolve these issues.

    Conclusion

    Oracle have invested a lot of time and effort to simplify the Exadata deployment process, but there are still many moving parts and hardware components. You might find the Exadata deployment process complex, especially if this is the first time you are deploying Exadata Machine in your organization. I hope this article will help you understand the Exadata deployment process better and make you feel comfortable enough to choose the Oracle Exadata Database Machine as your Oracle database platform. Good luck!

    Released: March 14, 2017, 10:57 am | Updated: April 6, 2017, 8:33 am
    Keywords: Feature | #Exadata | Deployment | Oracle Exadata Database Machine | Primer

    Robin Chatterjee OECA
    By: Robin Chatterjee | Posted: March 22, 2017, 3:57 am

    Informative. One thing I notice we as DBAs tend to forget is the Datacenter planning aspects. Apart from OEDA oracle also has a spradsheet tool called OECA that helps with defining the DC parameters for Exadata including calcluation of Power cooling any other requirements .

    That probably needs to become part of or discussions when we talk abut Exadata deployment also :) .

    Robin Chatterjee OECA
    By: Robin Chatterjee | Posted: March 22, 2017, 3:57 am

    Informative. One thing I notice we as DBAs tend to forget is the Datacenter planning aspects. Apart from OEDA oracle also has a spradsheet tool called OECA that helps with defining the DC parameters for Exadata including calcluation of Power cooling any other requirements .

    That probably needs to become part of or discussions when we talk abut Exadata deployment also :) .


    Copyright © 2017 Communication Center. All Rights Reserved
    All material, files, logos and trademarks within this site are properties of their respective organizations.
    Terms of Service - Privacy Policy - Contact

    Independent Oracle Users Group
    330 N. Wabash Ave., Suite 2000, Chicago, IL 60611
    phone: 312-245-1579 | email: ioug@ioug.org

    IOUG Logo

    Copyright © 1993-2017 by the Independent Oracle Users Group
    Terms of Use | Privacy Policy