Page 1: Windows 2000
Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Second Edition (June 2001) Part Number 225083-002 Compaq Computer Corporation...
Page 2 All other product names mentioned herein may be trademarks of their respective companies. Compaq shall not be liable for technical or editorial errors or omissions contained herein. The information in this document is provided “as is” without warranty of any kind and is subject to change without notice.
Page 3: Table Of Contents
Symbols in Text.......................xvii Symbols on Equipment..................xviii Rack Stability ......................xix Getting Help ......................xix Compaq Technical Support ................xix Compaq Website....................xx Compaq Authorized Reseller................xx Chapter 1 Clustering Overview Clusters Defined ...................... 1-2 Availability ......................1-3 Scalability ........................ 1-3 Compaq Parallel Database Cluster Overview............1-4...
Page 4 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Chapter 2 Architecture Compaq ProLiant Servers ..................2-2 High-Availability Features of ProLiant Servers ..........2-3 Shared Storage Components ..................2-3 RA4000 Array....................2-4 RA4100 Array....................2-4 RA4000 Array Controller .................
Page 5 Overview of the Cluster Software................3-1 Microsoft Windows 2000 Advanced Server............3-2 Compaq Software ....................3-2 Compaq SmartStart and Support Software............3-2 Compaq System Configuration Utility ............. 3-3 Compaq Array Configuration Utility..............3-3 Fibre Channel Fault Isolation Utility ..............3-3 Compaq Insight Manager .................
Page 6 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Cluster Planning continued Disk Drive Planning....................4-18 Nonshared Disk Drives ................... 4-18 Shared Disk Drives ..................4-18 Network Planning ....................4-19 Windows 2000 Advanced Server Hosts Files for an Ethernet Cluster Interconnect ....................
Page 7 Contents vii Installation and Configuration continued Verifying the Hardware and Software Installation ..........5-38 Cluster Communications ................5-38 Access to Shared Storage from All Nodes............5-38 OSDs ......................5-38 Other Verification Tasks ................5-39 Power Distribution and Power Sequencing Guidelines ......... 5-39 Server Power Distribution ................
Page 8 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Cluster Management continued Adding a Cluster Node................... 6-21 Preparing the New Node................. 6-22 Preparing the Existing Cluster Nodes ............. 6-23 Installing the Cluster Software for Oracle8i ........... 6-23 Monitoring Cluster Operation ................
Page 9 Rerun and Validate Object Link Manager On All Affected Nodes ......A-4 Run Disk Management On All Nodes ..............A-5 Run and Validate the Array Configuration Utility On All Nodes......A-5 Perform Cluster Software and Firmware Checks ............A-6 Perform Cluster Hardware Checks ................A-6 Contact Your Compaq Support Representative ............A-7 Glossary Index...
Page 10 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide List of Figures Figure 1-1. Example of a six-node Compaq Parallel Database Cluster Model PDC/O1000..................... 1-2 Figure 2-1. Fibre Host Adapters in a two-node PDC/O1000 cluster....... 2-7 Figure 2-2.
Page 11 Contents xi Figure 5-4. Non-redundant Ethernet cluster interconnect using a crossover cable ......................... 5-13 Figure 5-5. Non-redundant Ethernet cluster interconnect using an Ethernet switch or hub....................5-14 Figure 5-6. Redundant Ethernet cluster interconnect for a two-node PDC/O1000 cluster ..................5-15 Figure 5-7.
Page 12: About This Guide
Purpose This administrator guide provides information about the planning, installation, configuration, implementation, management, and troubleshooting of the Compaq Parallel Database Cluster Model PDC/O1000 running Oracle8i software on the Microsoft Windows 2000 Advanced Server operating system. Audience The expected audience of this guide consists primarily of MIS professionals whose jobs include designing, installing, configuring, and maintaining Compaq Parallel Database Clusters.
Page 13: Scope
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Scope This guide offers significant background information about clusters as well as basic concepts associated with designing clusters. It also contains detailed product descriptions and installation steps.
Page 14: Referenced Manuals
Referenced Manuals For additional information, refer to documentation related to the specific hardware and software components of the Compaq Parallel Database Cluster. These related manuals include, but are not limited to: Documentation related to the ProLiant servers you are clustering (for...
Page 15: Supplemental Documents
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Oracle8i Parallel Server Concepts Oracle8i Parallel Server Administration, Deployment, and Performance Oracle Enterprise Manager Administrator’s Guide Oracle Enterprise Manager Configuration Guide Oracle Enterprise Manager Concepts Guide...
Page 16: Text Conventions
About This Guide xvii Text Conventions This document uses the following conventions to distinguish elements of text: User Input, GUI Text a user types or enters appears in boldface. Selections Items a user selects from a GUI, such as tabs, buttons, or menu items, also appear in boldface.
Page 17: Symbols On Equipment
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Symbols on Equipment These icons may be located on equipment in areas where hazardous conditions may exist. Any surface or area of the equipment marked with these symbols indicates the presence of electrical shock hazards.
Page 18: Rack Stability
Compaq Technical Support In North America, call the Compaq Technical Phone Support Center at 1-800-OK-COMPAQ. This service is available 24 hours a day, 7 days a week. For continuous quality improvement, calls may be recorded or monitored. Outside North America, call the nearest Compaq Technical Support Phone Center.
Page 19: Compaq Website
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Compaq Website The Compaq website has information on this product as well as the latest drivers and Flash ROM images. You can access the Compaq website by logging on to the Internet at www.compaq.com...
Page 20: Clustering Overview
Before examining the features and benefits of the Compaq Parallel Database Cluster Model PDC/O1000 (referred to here as the PDC/O1000), it is helpful to understand the concepts and terminology of...
Page 21: Clusters Defined
1-2 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i on Windows 2000 Administrator Guide Clusters Defined A cluster is an integration of software and hardware products that enables a set of loosely coupled servers and shared storage subsystem components to present a single system image to clients and to operate as a single system.
Page 22: Availability
Clustering Overview 1-3 The PDC/O1000 uses non-redundant Fibre Channel Fabric Storage Area Network (SAN) and non-redundant Fibre Channel Arbitrated Loop (FC-AL) SAN topologies for its shared storage I/O data paths. These two SAN topologies support the use of multiple non-redundant fabrics and loops, respectively.
Page 23: Compaq Parallel Database Cluster Overview
1-4 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i on Windows 2000 Administrator Guide Compaq Parallel Database Cluster Overview As traditional clustering technology has moved into the open systems of client/server computing, Compaq has provided innovative, customer-focused solutions. The PDC/O1000 moves client/server computing one step closer to the capabilities found in expensive, proprietary cluster solutions, at a fraction of the cost.
Page 24 Chapter Architecture The Compaq Parallel Database Cluster Model PDC/O1000 (referred to here as the PDC/O1000) is an integration of a number of different hardware and software products. This chapter discusses how each of the hardware products plays a role in bringing a complete clustering solution to your computing environment.
Page 25: Compaq Proliant Servers
2-2 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Gigabit Interface Converter-Shortwave (GBIC-SW) modules Fibre Channel cables Cluster interconnect components Ethernet NIC adapters Ethernet cables Ethernet switches or hubs The software products include: Microsoft Windows 2000 Advanced Server with Service Pack 1 or later...
Page 26: High-Availability Features Of Proliant Servers
Error Checking and Correcting (ECC) memory. The high-availability features of ProLiant servers are a critical foundation of Compaq clustering products. Table 2-1 lists the high-availability features found in many ProLiant servers. Table 2-1...
Page 27: Ra4000 Array
2-4 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide RA4000 Array The RA4000 Array is one shared storage solution for the PDC/O1000. Each non-redundant Fibre Channel Fabric or non-redundant Fibre Channel Arbitrated Loop (FC-AL) can contain one or more RA4000 Arrays. Each RA4000 Array contains one single-port RA4000 Array Controller.
Page 28: Ra4000 Array Controller
For more information about the RA4000 Array Controller, refer to the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide. Fibre Channel SAN Switch...
Page 29: Storage Hub
2-6 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide For further information, refer to these manuals provided with the Fibre Channel SAN Switch: Compaq StorageWorks Fibre Channel SAN Switch 8 Installation and Hardware Guide Compaq StorageWorks Fibre Channel SAN Switch 16 Installation and...
Page 30: Fc-Al Switch
The FC-AL Switch 8 supports eight ports. With the addition of the 3-port Expansion Module (PEM), the switch supports 11 ports. For further information, refer to the Compaq StorageWorks Fibre Channel Arbitrated Loop Switch (FC-AL Switch) User Guide. Fibre Host Adapters Fibre Host Adapters are the interface between the cluster nodes (servers) and the RA4000/RA4100 Arrays to which they are connected.
Page 31: Gigabit Interface Converter-Shortwave Modules
2-8 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Each non-redundant Fibre Channel Fabric or non-redundant FC-AL contains one dedicated Fibre Host Adapter in every cluster node. Across nodes, Fibre Host Adapters for the same Fibre Channel Fabric or FC-AL must be installed in the same server slot and connected to the same Fibre Channel SAN Switch, FC-AL Switch, or Storage Hub.
Page 32: Availability Features Of The Shared Storage Components
(cached) is safe even with equipment failure or power outage. For a complete list of features and accompanying descriptions, refer to the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide.
Page 33 2-10 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Non-Redundant Fibre Channel Fabric The PDC/O1000 supports non-redundant Fibre Channel Fabrics. A non-redundant Fibre Channel Fabric provides one I/O path between each cluster node and its Fibre Channel SAN Switch and one I/O data path between the Fibre Channel SAN Switch and each of its RA4000/RA4100 Arrays.
Page 34: Using Multiple Non-Redundant Fibre Channel Fabrics
Architecture 2-11 Figure 2-2 shows a two-node PDC/O1000 with one non-redundant Fibre Channel Fabric. RA4000/4100 RA4000/4100 RA4000/4100 Array Array Array Node 1 Node 2 Fibre Fibre Host Adapter Host Adapter ProLiant ProLiant Fibre Channel Server Server SAN Switch Switch (Cluster Interconnect) Client LAN Figure 2-2.
Page 35: Figure 2-3. Pdc/O1000 Cluster With Two Non-Redundant Fibre Channel Fabrics
2-12 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Adding a second non-redundant Fibre Channel Fabric to the PDC/O1000 involves duplicating the hardware components used in the first Fibre Channel Fabric. The maximum number of non-redundant Fibre Channel Fabrics you can install in a PDC/O1000 is restricted by the number of Fibre Host Adapters your Compaq ProLiant servers support.
Page 36: Maximum Distances Between Cluster Nodes And Shared Storage Subsystem Components In A Non-Redundant Fibre Channel Fabric
Architecture 2-13 Maximum Distances Between Cluster Nodes and Shared Storage Subsystem Components in a Non-Redundant Fibre Channel Fabric By using standard short-wave Fibre Channel cables and GBIC-SW modules, an RA4000/RA4100 Array can be placed up to 500 meters from the Fibre Channel SAN Switch, and the Fibre Channel SAN Switch can be placed up to 500 meters from the Fibre Host Adapter in each cluster node.
Page 37: Figure 2-5. Fibre Host Adapter-To-Fibre Channel San Switch I/O Data Paths
2-14 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Fibre Host Adapter-to-Fibre Channel SAN Switch Paths Figure 2-5 highlights the I/O data paths that run between the Fibre Host Adapter in each cluster node and the Fibre Channel SAN Switch. There is one I/O path for each Fibre Host Adapter.
Page 38: Figure 2-6. Fibre Channel San Switch-To-Array Controller I/O Data Path
RA4000/RA4100 Array. As with the Fibre Host Adapter-to-Fibre Channel SAN Switch data path, Compaq Insight Manager detects this fault, changes the affected RA4000/RA4100 Array’s status to “Failed,” and changes its condition to red.
Page 39: I/O Path Configurations In A Non-Redundant Fibre Channel Arbitrated Loop
2-16 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide I/O Path Configurations in a Non-Redundant Fibre Channel Arbitrated Loop Overview of FC-AL SAN Topology Fibre Channel standards define a multi-layered architecture for moving data across the storage area network (SAN). This layered architecture can be implemented using the Fibre Channel Fabric SAN or the Fibre Channel Arbitrated Loop (FC-AL) SAN topology.
Page 40: Figure 2-7. Two-Node Pdc/O1000 Cluster With One Non-Redundant Fc-Al
Architecture 2-17 Each non-redundant FC-AL consists of the following hardware: One Fibre Host Adapter in each node One Storage Hub or FC-AL Switch One or more RA4000/RA4100 Arrays, each containing one single-port RA4000 Array Controller A GBIC-SW module installed in each Fibre Host Adapter, each active port on the Storage Hub or FC-AL Switch, and each array controller Fibre Channel cables used to connect the Fibre Host Adapter in each node to the Storage Hub or FC-AL Switch and the Storage Hub or...
Page 41: Using Multiple Non-Redundant Fibre Channel Arbitrated Loops
2-18 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Using Multiple Non-Redundant Fibre Channel Arbitrated Loops The PDC/O1000 supports the use of multiple non-redundant FC-ALs within the same cluster. Physically, this means that within each cluster node, multiple Fibre Host Adapters are used to connect the nodes to different sets of RA4000/RA4100 Arrays.
Page 42: Figure 2-8. Pdc/O1000 Cluster With Two Non-Redundant Fc-Als
Architecture 2-19 Figure 2-8 shows a four-node PDC/O1000 cluster with two non-redundant FC-ALs. Each FC-AL has its own Fibre Host Adapter in every node, one Storage Hub or FC-AL Switch, and one or more RA4000/RA4100 Arrays. In Figure 2-8, the hardware components that constitute the second non-redundant FC-AL are shaded.
Page 43: Maximum Distances Between Cluster Nodes And Shared Storage Subsystem Components In A Non-Redundant Fc-Al
2-20 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Maximum Distances Between Cluster Nodes and Shared Storage Subsystem Components in a Non-Redundant FC-AL By using standard short-wave Fibre Channel cables and GBIC-SW modules, an RA4000/RA4100 Array can be placed up to 500 meters from the Storage Hub or FC-AL Switch, and the Storage Hub or FC-AL Switch can be placed up to 500 meters from the Fibre Host Adapter in each cluster node.
Page 44: Figure 2-10. Fibre Host Adapter-To-Storage Hub/Fc-Al Switch I/O Data Paths
RA4000/RA4100 Array, the status of the array controller in that RA4000/RA4100 Array changes to “Failed” and the condition is red. The red condition is reported to higher-level Insight Manager screens, and eventually to the device list. Refer to the Compaq Insight Manager Guide for details.
Page 45: Figure 2-11. Storage Hub/Fc-Al Switch-To-Array Controller I/O Data Path
2-22 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Storage Hub/FC-AL Switch-to-Array Controller Paths Figure 2-11 highlights the I/O data path that runs between the Storage Hub or FC-AL Switch and the RA4000 Array Controller in each RA4000/RA4100 Array of the FC-AL.
Page 46: Cluster Interconnect Options
TCP/IP. When configuring the Ethernet cluster interconnect, be sure to enable TCP/IP. NOTE: Refer to the technical white paper Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server (ECG062/0299) for detailed information about configuring redundant and non-redundant Ethernet cluster interconnects.
Page 47 2-24 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Non-Redundant Ethernet Cluster Interconnect Components The following components are used in a non-redundant Ethernet cluster interconnect: One Ethernet adapter in each cluster node Ethernet cables and a switch or hub...
Page 48 Ethernet cluster interconnect and the second port for the client LAN. Refer to the technical white paper Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server for detailed information about configuring redundant Ethernet cluster interconnects. This document is available at www.compaq.com/support/techpubs/whitepapers...
Page 49: Figure 2-12. Non-Redundant Ethernet Cluster Interconnect Using A Crossover Cable
2-26 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Ethernet Cluster Interconnect Diagrams Figure 2-12 shows the non-redundant Ethernet cluster interconnect components used in a two-node PDC/O1000 cluster. These components include a dual-port Ethernet adapter in each node. The top port on each adapter connects by Ethernet crossover cable to the top port on the adapter in the other node.
Page 50: Figure 2-13. Non-Redundant Ethernet Cluster Using An Ethernet Switch Or Hub
Architecture 2-27 Figure 2-13 shows another option for a non-redundant Ethernet cluster interconnect in a two-node PDC/O1000 cluster. These include an Ethernet adapter in each node connected by Ethernet cables to an Ethernet switch or hub. RA4000/4100 Array Storage Hub/Switch Ethernet Adapter Ethernet Adapter Storage Hub/Switch...
Page 51: Figure 2-14. Redundant Ethernet Cluster Interconnect For A Two-Node Pdc/O1000 Cluster
2-28 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Because Ethernet switches are required in PDC/O1000 clusters with three or more nodes, using an Ethernet switch instead of a crossover cable makes it easier to upgrade the cluster interconnect if more servers are added to the cluster.
Page 52: Local Area Network
Architecture 2-29 Local Area Network NOTE: For the PDC/O1000, the client LAN and the cluster interconnect must be treated as separate networks. Do not use either network to handle the other network’s traffic. Every client/server application requires a local area network, or LAN, over which client machines and servers communicate.
Page 53: Cluster Software Components
Chapter Cluster Software Components Overview of the Cluster Software The Compaq Parallel Database Cluster Model PDC/O1000 (referred to here as the PDC/O1000) combines software from several leading computer vendors. The integration of these components creates a stable cluster management environment in which the Oracle database can operate.
Page 54: Microsoft Windows 2000 Advanced Server
3-2 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Microsoft Windows 2000 Advanced Server This version of the PDC/O1000 supports only Microsoft Windows 2000 Advanced Server with Service Pack 1 or later. NOTE: The PDC/O1000 does not work in conjunction with Microsoft Cluster Server. Do not install Microsoft Cluster Server on any of the cluster nodes.
Page 55: Compaq System Configuration Utility
Support Software CD, is used to configure the hardware aspects of any disk drives attached to an array controller, including the non-shared drives in the servers and the shared drives in the Compaq StorageWorks RAID Array 4000s (RA4000 Arrays) or Compaq StorageWorks RAID Array 4100s (RA4100 Arrays).
Page 56: Compaq Insight Manager
3-4 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Compaq Insight Manager Compaq Insight Manager, loaded from the Compaq Management CD, is a software utility used to collect information about the servers in the cluster.
Page 57: Compaq Operating System Dependent Modules
For a detailed description of how the OSD layer interacts with Oracle8i Parallel Server, refer to the Oracle8i Parallel Server Setup and Configuration Guide. Also refer to “Installing Compaq OSDs” in Chapter 5, “Installation and Configuration.” The OSD software is found on the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD.
Page 58: Oracle8I Server
3-6 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Oracle8i Server Oracle8i Server is the database application software and must be installed on each node in the PDC/O1000. Refer to the documentation provided with the Oracle8i Server software for additional information.
Page 59: Oracle8I Certification
Oracle8i Parallel Server must demonstrate the successful completion of the Oracle8i Parallel Server for Windows 2000 Certification. Neither Oracle nor Compaq will support any implementation of Oracle8i Parallel Server that does not strictly conform to the configurations certified with this process. For a...
Page 60: Cluster Planning
Cluster Planning Before connecting any cables or powering on any hardware on your Compaq Parallel Database Cluster Model PDC/O1000 (referred to here as the PDC/O1000), it is important that you understand how all the various cluster components fit together to meet your operational requirements. The major...
Page 61: Site Planning
4-2 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Site Planning You must carefully select and prepare the site to ensure a smooth installation and a safe and efficient work environment. To select and prepare a location for...
Page 62: Capacity Planning For Cluster Hardware
NOTE: Certain restrictions apply to the server models and server configurations that are supported by the Compaq Parallel Database Cluster. For a current list of PDC/O1000-certified servers and details on supported configurations, refer to the Compaq Parallel Database Cluster Model PDC/O1000 Certification Matrix for Windows 2000.
Page 63: Planning Shared Storage Components For Non-Redundant Fibre Channel Fabrics
4-4 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Planning Shared Storage Components for Non-Redundant Fibre Channel Fabrics Several key components make up the shared storage subsystem for the PDC/O1000. Each non-redundant Fibre Channel Fabric in a PDC/O1000 uses...
Page 64: Planning Shared Storage Components For Non-Redundant Fibre Channel Arbitrated Loops
Each non-redundant Fibre Channel Arbitrated Loop (FC-AL) in a PDC/O1000 uses these hardware components: One Fibre Host Adapter in each node One Compaq StorageWorks Storage Hub (Storage Hub) or Compaq StorageWorks FC-AL Switch (FC-AL Switch) One or more RA4000 Arrays or RA4100 Arrays...
Page 65: Planning Cluster Interconnect And Client Lan Components
4-6 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Planning Cluster Interconnect and Client LAN Components PDC/O1000 clusters running Oracle8i Parallel Server can use a redundant or non-redundant Ethernet cluster interconnect. A redundant cluster interconnect is recommended because it provides fault tolerance along the entire cluster interconnect path.
Page 66: Planning Cluster Configurations For Non-Redundant Fibre Channel Fabrics
Cluster Planning 4-7 Planning the Client LAN Every client/server application requires a local area network (LAN) over which client machines and servers communicate. In the case of a cluster, the hardware components of the client LAN are no different than in a stand-alone server configuration.
Page 67: Sample Midsize Cluster With One Non-Redundant Fibre Channel Fabric
4-8 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Sample Midsize Cluster with One Non-Redundant Fibre Channel Fabric Figure 4-1 shows an example of a midsize PDC/O1000 cluster with one non-redundant Fibre Channel Fabric that contains four cluster nodes and three RA4000/RA4100 Arrays.
Page 68: Sample Large Cluster With One Non-Redundant Fibre Channel Fabric
Cluster Planning 4-9 Sample Large Cluster with One Non-Redundant Fibre Channel Fabric Figure 4-2 shows an example of a larger PDC/O1000 cluster with one non-redundant Fibre Channel Fabric that contains six cluster nodes and six RA4000/RA4100 Arrays. RA4000/4100 Arrays (6) Fibre Channel SAN Switch Node 2 Node 4...
Page 69: Planning Cluster Configurations For Non-Redundant Fibre Channel Arbitrated Loops
4-10 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Planning Cluster Configurations for Non-Redundant Fibre Channel Arbitrated Loops This section describes sample configurations for midsize and larger clusters that use non-redundant Fibre Channel Arbitrated Loops (FC-ALs).
Page 70: Sample Large Cluster With One Non-Redundant Fc-Al
Cluster Planning 4-11 The sample midsize cluster configuration shown in Figure 4-3 contains these hardware components: Four ProLiant servers (cluster nodes) One Fibre Host Adapter in each cluster node One Storage Hub or FC-AL Switch installed between the Fibre Host Adapters and the shared storage arrays Three RA4000/RA4100 Arrays One RA4000 Array Controller in each RA4000/RA4100 Array...
Page 71: Raid Planning
4-12 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide The larger configuration shown in Figure 4-4 contains these hardware components: Six ProLiant servers (cluster nodes) One Fibre Host Adapter in each cluster node One Storage Hub or FC-AL Switch to connect the RA4000/RA4100...
Page 72 RAID. When you implement RAID on shared storage arrays, you use the hardware RAID to perform such functions as making copies of the data or calculating checksums. Use the Compaq Array Configuration Utility to implement RAID on your logical disks.
Page 73: Supported Raid Levels
4-14 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Supported RAID Levels RAID provides several fault-tolerant options to protect your cluster’s shared data. However, each RAID level offers a different mix of performance, reliability, and cost.
Page 74: Raw Data Storage And Database Size
Cluster Planning 4-15 Raw Data Storage and Database Size Raw data storage is the amount of storage available before any RAID levels have been configured. It is called raw data storage because RAID volumes require some overhead. The maximum size of a database stored in a RAID system will always be less than the amount of raw data storage available.
Page 75: Selecting The Appropriate Raid Levels
4-16 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Selecting the Appropriate RAID Levels Many factors affect which RAID levels you select for your cluster database. These include the specific availability, performance, reliability, and recovery capabilities required from the database.
Page 76: Planning The Grouping Of Physical Disk Storage Space
A logical drive is what you see labeled as Disk 1, Disk 2, and so on, from the Disk Management utility. For information about RAID disk arrays and logical drives, refer to the information on drive arrays in the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100...
Page 77: Disk Drive Planning
4-18 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Use Disk Management to define one extended partition per RAID logical drive. Also using Disk Management, divide the extended partitions into logical partitions, each having its own drive letter. (Windows 2000 Advanced Server logical partitions are called “logical drives”...
Page 78: Network Planning
Ethernet Cluster Interconnect When an Ethernet cluster interconnect is installed between cluster nodes, the Compaq operating system dependent modules (OSDs) require a unique entry in the hosts and lmhosts files located at %SystemRoot%\system32\drivers\etc for each network port on each node.
Page 79: Client Lan
4-20 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Client LAN Physically, the structure of the client network is no different than that used for a nonclustered configuration. To ensure continued access to the database when a cluster node is evicted from the cluster, each network client should have physical network access to all of the cluster nodes.
Page 80: Installation And Configuration
Installation and Configuration This chapter provides instructions for installing and configuring the Compaq Parallel Database Cluster Model PDC/O1000 (referred to here as the PDC/O1000) for use with Oracle8i software. A PDC/O1000 is a combination of several individually available products. As you set up your cluster, have the following materials available during installation.
Page 81: Installation Overview
5-2 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Compaq StorageWorks Fibre Channel SAN Switch 16 Installation and Hardware Guide Compaq StorageWorks Fibre Channel SAN Switch Management Guide Compaq SmartStart Installation poster Compaq SmartStart and Support Software CD Microsoft Windows 2000 Advanced Server Administrator’s Guide...
Page 82: Installing The Hardware
Installing and configuring operating system software, including: SmartStart 4.9 or later Microsoft Windows 2000 Advanced Server with Service Pack 1 or later Installing and configuring the Compaq operating system dependent modules (OSDs), including: Using Oracle Universal Installer to install OSDs for an Ethernet cluster interconnect...
Page 83: Installing The Fibre Host Adapters
5-4 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Set up the hardware on one node completely, then set up the rest of the nodes identically to the first one. Do not load any software on any cluster node until all the hardware has been installed in all cluster nodes.
Page 84: Cabling The Fibre Host Adapters To The Storage Hub, Fc-Al Switch, Or Fibre Channel San Switch
Installation and Configuration 5-5 To install GBIC-SW modules: 1. Verify that a GBIC-SW module has been installed in each Fibre Host Adapter in a server. 2. Insert a GBIC-SW module into the port for the Fibre Host Adapter on its Storage Hub, FC-AL Switch, or Fibre Channel SAN Switch.
Page 85: Installing The Cluster Interconnect Adapters
5-6 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Figure 5-1 shows the Fibre Host Adapters in four nodes connected to one Storage Hub, FC-AL Switch, or Fibre Channel SAN Switch. Client LAN Fibre Fibre...
Page 86: Installing The Client Lan Adapters
Ethernet adapters into each cluster node. For recommended dual-port and single-port Ethernet adapters, see the Compaq Parallel Database Cluster Model PDC/O1000 Certification Matrix for Windows 2000 at www.compaq.com/solutions/enterprise/ha-pdc.html If you need specific instructions on how to install an Ethernet adapter, refer to the documentation of the Ethernet adapter you are installing or refer to the user guide of the ProLiant server you are using.
Page 87: Setting Up The Ra4000/Ra4100 Arrays
5-8 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Setting Up the RA4000/RA4100 Arrays Unless otherwise indicated in this guide, follow the instructions in the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide to set up shared storage subsystem components.
Page 88: Installing Gbic-Sw Modules For The Ra4000 Array Controller
Controller For the PDC/O1000, each RA4000/RA4100 Array contains one single-port Compaq StorageWorks RAID Array 4000 Controller (RA4000 Array Controller) and ships with GBIC-SW modules. Insert one module into the RA4000 Array Controller and the other module into the Storage Hub, FC-AL Switch, or Fibre Channel SAN Switch port for the controller.
Page 89: Installing Additional Fibre Channel Fabrics Or Fc-Als
5-10 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Installing Additional Fibre Channel Fabrics or FC-ALs At this point, you have installed the hardware for one non-redundant Fibre Channel Fabric or one non-redundant FC-AL. To add another non-redundant...
Page 90: Cabling The Ethernet Cluster Interconnect
Installation and Configuration 5-11 Figure 5-3 shows a PDC/O1000 with two non-redundant Fibre Channel Fabrics or two non-redundant FC-ALs. The hardware for the second Fibre Channel Fabric or FC-AL is shaded. NOTE: You can mix non-redundant Fibre Channel Fabrics and non-redundant FC-ALs in the same PDC/O1000 cluster.
Page 91 5-12 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide The following components are used in a redundant Ethernet cluster interconnect: Two Ethernet adapters in each cluster node Ethernet cables and switches or hubs For two-node PDC/O1000 clusters, you can use two 100-Mbit/sec Ethernet switches or hubs with cables to connect the servers.
Page 92: Cable
Installation and Configuration 5-13 Figure 5-4 shows the non-redundant Ethernet cluster interconnect components used in a two-node PDC/O1000 cluster. These components include a dual-port Ethernet adapter in each node. The top port on each adapter connects by Ethernet crossover cable to the top port on the adapter in the other node. The bottom port on each adapter connects by Ethernet cable to the client LAN switch or hub.
Page 93: Figure 5-5. Non-Redundant Ethernet Cluster Interconnect Using An Ethernet Switch Or Hub
5-14 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Figure 5-5 shows another option for a non-redundant Ethernet cluster interconnect in a two-node PDC/O1000 cluster. These include an Ethernet adapter in each node connected by Ethernet cables to an Ethernet switch or hub.
Page 94: Figure 5-6. Redundant Ethernet Cluster Interconnect For A Two-Node Pdc/O1000 Cluster
Ethernet cluster interconnect. For clusters of three or more nodes using a redundant Ethernet cluster interconnect, Compaq requires that you use two Ethernet 100-Mbit/sec switches to maintain good network performance across the cluster. If there are only two nodes in a cluster, you can use either Ethernet hubs or switches and standard Ethernet cables to connect the nodes.
Page 95: Cabling The Client Lan
5-16 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Cabling the Client LAN You can use any TCP/IP network to connect to a client LAN. The following procedure contains instructions for cabling an Ethernet client LAN.
Page 96: Installing Operating System Software And Configuring The Ra4000/Ra4100 Arrays
Installation and Configuration 5-17 Installing Operating System Software and Configuring the RA4000/RA4100 Arrays You will follow an automated procedure using Compaq SmartStart to install the operating system software and configure the shared storage on the RA4000/RA4100 Arrays. Guidelines for Clusters Installing clustering software requires several specific steps and guidelines that might not be necessary when installing software on a single server.
Page 97: Automated Installation Using Smartstart
5-18 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Be sure to set up unique IP addresses and node names for each node in the hosts and lmhosts files at %SystemRoot%\system32\drivers\etc. For an Ethernet cluster interconnect, one IP address and node name is for the cluster interconnect, and the other IP address and node name is for the client LAN.
Page 98 Installation and Configuration 5-19 Automated Installation Steps You will perform the following automated installation steps to install operating system software on every node in the cluster. 1. Power up the following cluster components in this order: RA4000/RA4100 Arrays Storage Hubs, FC-AL Switches, or Fibre Channel SAN Switches Ethernet hubs or switches 2.
Page 99 5-20 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 9. Enter unique IP addresses and node names for each node in the hosts and lmhosts files located at %SystemRoot%\system32\drivers\etc. Record this information: For the Ethernet cluster interconnect, one IP address and node name is for the redundant cluster interconnect, and the other IP address and node name is for the client LAN.
Page 100 IMPORTANT: Record the cluster interconnect node name, the client LAN node name, and the IP addresses assigned to them. You will need this information later when installing Compaq OSDs. 12. If you are installing node1, open Disk Management to create extended disk partitions and logical partitions within the extended partitions on all RA4000/RA4100 Arrays.
Page 101: Installing Compaq Osds
5-22 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Installing Compaq OSDs Use the Oracle Universal Installer (OUI) program to install the Compaq operating system dependent modules (OSDs) for Oracle8i Parallel Server. Compaq supplies a software package with the OUI that installs Compaq OSDs for an Ethernet cluster interconnect.
Page 102: Mounting Remote Drives And Verifying Administrator Privileges
Installation and Configuration 5-23 If you are using an Ethernet cluster interconnect, a cluster interconnect name and IP address were assigned to each cluster node. Ping the Ethernet cluster interconnect name of every node in the cluster. When using the client LAN name with the ping command, the IP address displayed by the ping utility should be the client LAN IP address.
Page 103: Installing The Ethernet Osds
5-24 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide To mount drives on remote nodes, enter a net use command from a C: prompt on the installing machine. Enter one instance of the command for each remote cluster node.
Page 104 Installation and Configuration 5-25 3. If autorun is enabled, the Oracle Universal Installer Welcome screen appears. If autorun is disabled, enter d:\setup at a C: prompt. Substitute the letter assigned to the CD drive for d. The Oracle Universal Installer Welcome screen appears.
Page 105 5-26 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 4. Click Next to continue. The License screen appears. 5. Click Accept to accept the terms of the license agreement. The File Locations screen appears.
Page 106 Installation and Configuration 5-27 6. Click Next to continue. The Installation Types screen appears.
Page 107 5-28 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 7. Select Typical to install the Ethernet OSDs into the default location, C:\Compaq\OPS. Skip to step 12. 8. Select Custom to install the Ethernet OSDs into a location of your...
Page 108 Installation and Configuration 5-29 9. If you want to accept the default location, skip to step 13. 10. If you want to specify an alternate location and you know the exact directory path, specify it by typing over the default location and skip to step 12.
Page 109 5-30 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 12. Browse to the location where you want to install the OSDs. Click OK. The Component Locations screen appears indicating the alternate location you specified.
Page 110 Installation and Configuration 5-31 13. Click Next to continue. The Cluster Members screen appears.
Page 111 5-32 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 14. The Cluster Members screen indicates the host name of the installing machine. Define the node names for the client LAN by adding the node names you specified in the hosts and lmhosts files for the client LAN for all other cluster nodes.
Page 112 Installation and Configuration 5-33 16. The Cluster Members screen now shows the node names you specified on the first Cluster Members screen with “_san” appended to each name, indicating it as a cluster interconnect node name. If necessary, retype the cluster interconnect node names to match the names you entered into the hosts and lmhosts files for the Ethernet cluster interconnect.
Page 113 5-34 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 18. If the information on the OSD Summary screen is not correct, click Previous to modify the previous screens. When the information is correct, click Next to continue. The Summary...
Page 114 Installation and Configuration 5-35 19. Click Install to install the components listed in the summary. At this point, the OUI verifies that the installing node can communicate with other cluster nodes through both the client LAN and cluster interconnect node names. If the OUI cannot communicate with all nodes, this process can take awhile.
Page 115: Installing Oracle Software
5-36 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Installing Oracle Software Using the Oracle8i Parallel Server Setup and Configuration Guide, follow the steps to install Oracle8i Enterprise Edition software on all cluster nodes, including:...
Page 116: Additional Notes On Configuring Oracle Software
For instructions on installing and using Object Link Manager, see the readme file on the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD at \Object Link Manager\readme.txt...
Page 117: Verifying The Hardware And Software Installation
5-38 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Verifying the Hardware and Software Installation Cluster Communications Use the ping utility to verify that each node in the cluster can communicate with every other node over the client LAN. Run the ping utility from the installing cluster node.
Page 118: Other Verification Tasks
Sequencing Guidelines It is recommended you connect most PDC/O1000 cluster components to a Compaq power distribution unit (PDU). PDUs connect to an uninterruptible power supply (UPS) or building power. The PDUs and UPSs are the only cluster components connected to a building power source. If there is no UPS, the PDU plugs into building power.
Page 119: Server Power Distribution
5-40 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Server Power Distribution Figure 5-7 shows an example of server power distribution for a three-node PDC/O1000 cluster. Power supply #1 of each server is connected to PDU #1.
Page 120: Power Sequencing
Installation and Configuration 5-41 Power Sequencing Be sure to power up the cluster components in the following order: 1. RA4000/RA4100 Arrays 2. Storage Hubs, FC-AL Switches, or Fibre Channel SAN Switches (Power is applied to a Storage Hub when the AC power cord is plugged in.) 3.
Page 121: Cluster Management
This chapter describes these management activities for the Compaq Parallel Database Cluster Model PDC/O1000 (referred to here as the PDC/O1000). The topics addressed in this chapter include:...
Page 122: Cluster Management Concepts
6-2 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Cluster Management Concepts Powering Off a Node Without Interrupting Cluster Services At some time during the life of your cluster you will need to perform an operation on a cluster node that will require it to be powered off.
Page 123: Managing Network Clients Connected To A Cluster
Cluster Management 6-3 4. Determine how critical it is to repair the problem. If the problem is not considered to be critical, wait until a non-peak time to service the problem. If the problem is critical but does not affect all cluster nodes, shut down the Oracle instances on the affected nodes and wait until a non-peak time to service the problem.
Page 124: Management Applications
6-4 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide The following log files are used: The Node Manager (NM), provided by Compaq, logs messages to C:\compaq\ops\nm.log By default the OracleCMService sends messages to C:\compaq\ops\cmsrvr.log Management Applications...
Page 125: Managing Shared Drives
The ACU is available on the SmartStart and Support Software CD, which is found in the Compaq Server Setup and Management pack. For additional information about the ACU, see the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide.
Page 126: Monitoring Non-Redundant Fibre Channel Arbitrated Loops
6-6 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Gigabit Interface Converter-Shortwave (GBIC-SW) modules Fibre Channel cables Monitoring Non-Redundant Fibre Channel Arbitrated Loops Each non-redundant FC-AL in a PDC/O1000 contains this hardware: One or more RA4100 Arrays or RA4000 Arrays...
Page 127: Remotely Managing A Cluster
They can operate on Windows NT Server, Windows 95, or Windows 2000. Before installing Compaq Insight Manager, Compaq Insight Manager XE, or Oracle8i Enterprise Manager, it is recommended that you read through the corresponding documentation to determine how to set up and configure each...
Page 128: Software Maintenance For Oracle8I
6-8 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Software Maintenance for Oracle8i Deinstalling the OSDs At some point, you might want to deinstall the operating system dependent modules (OSDs). Some example situations are: When a node is being permanently removed from the cluster...
Page 129 Cluster Management 6-9 4. If autorun is disabled, enter d:\setup at a C: prompt. Substitute the letter assigned to the CD-ROM drive for d. The Oracle Universal Installer Welcome screen appears.
Page 130 6-10 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 5. Click Deinstall Products. Next, a window containing an inventory of the installed components appears. The following shows the Inventory screen that is displayed when Ethernet is used for the cluster interconnect.
Page 131: Upgrading Oracle8I Server
Significant changes in Oracle8i Server will likely affect the operation of the cluster. Before upgrading to any new release of Oracle8i Server, consult your Compaq and Oracle service representatives. Make sure that the new release is certified and supported on the PDC/O1000, and get assistance to determine which procedure you should follow to perform the upgrade properly.
Page 132 6-12 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Upgrading the Ethernet OSDs To upgrade the Ethernet OSDs: 1. Insert the CD containing the new revision of the OSDs into the CD-ROM drive on the primary node. (This is the node that was used for the previous installation of the OSDs.)
Page 133: Deinstalling A Partial Osd Installation
Deinstalling a Partial OSD Installation While not a common occurrence, the Oracle Universal Installer might not be able to complete the installation of the Compaq Ethernet OSDs. An incomplete OSD installation could result for various reasons, for example: A client LAN cable is no longer seated tightly into its connector.
Page 134: Upgrading Oracle8I Server
6-14 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 4. Run the uninstall utility by entering: uninstallOSD 5. Restart the node. 6. Repeat steps 1 through 5 on all other nodes in the PDC/O1000 cluster.
Page 135: Adding Disk Drives To Increase Shared Storage Capacity
CAUTION: Failure to follow the rules described in the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide can result in loss of data.
Page 136: Adding An Ra4000/Ra4100 Array
6-16 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Adding an RA4000/RA4100 Array You must shut down the PDC/O1000 cluster to add an RA4000/RA4100 Array. The following steps summarize the procedure: 1. Shut down the Oracle instance on each cluster node.
Page 137: Replacing A Failed Fibre Host Adapter
Cluster Management 6-17 Replacing a Failed Fibre Host Adapter To replace a failed Fibre Host Adapter in a cluster node: 1. Shut down the Oracle instance on the node. 2. Shut down the Oracle services running on the node. 3. Shut down the server. 4.
Page 138: Replacing A Cluster Node
6-18 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Replacing a Cluster Node Before replacing a node, verify that the new configuration is supported by the PDC/O1000 cluster. To determine if your new server is supported, see the...
Page 139: Adding The Replacement Node
Although physically connected to the cluster, the node is not yet integrated into the cluster. 2. Start the replacement node with the Compaq SmartStart and Support Software CD in the CD-ROM drive. 3. Use the Compaq System Configuration Utility to configure the hardware settings of the node and its adapters.
Page 140 6-20 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 7. Install the Fibre Host Adapter or Fibre Host Adapters in the replacement node in the same server PCI slot locations used in the other cluster nodes.
Page 141: Adding A Cluster Node
Adding a Cluster Node Before adding a node, verify that the new configuration is supported by the PDC/O1000. To determine if your new server is supported, see the Compaq Parallel Database Cluster Model PDC/O1000 Certification Matrix for Windows 2000 at www.compaq.com/solutions/enterprise/ha-pdc.html IMPORTANT: With some possible exceptions, the servers in the cluster must be set up identically.
Page 142: Preparing The New Node
6-22 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide This can be a time-consuming and complex procedure. Be sure to read through all of the following directions as well as those in the Oracle documentation before starting.
Page 143: Preparing The Existing Cluster Nodes
Cluster Management 6-23 8. Using Fibre Channel cables, connect each Fibre Host Adapter in the new node to its Fibre Channel SAN Switch, FC-AL Switch, or Storage Hub. 9. Restart the new node. 10. Verify TCP/IP connectivity between the new node and the existing cluster nodes.
Page 144 6-24 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide To install the cluster software, perform the following steps: 1. To add a node to the cluster, you must disassemble the existing cluster and then reform it to include the new node. Do the following on all original cluster nodes (not the new node): a.
Page 145: Monitoring Cluster Operation
Compaq Insight Manager and Compaq Insight Manager XE can be used to gather low-level, in-depth hardware statistics for the server and networking hardware components used in the cluster.
Page 146: Troubleshooting
Chapter Troubleshooting This chapter addresses potential problems you might encounter as you install, configure, test, and operate your Compaq Parallel Database Cluster Model PDC/O1000 (referred to here as the PDC/O1000). This chapter is organized as follows: Basic troubleshooting tips Troubleshooting Oracle8i and operating system dependent module...
Page 147: Basic Troubleshooting Tips
7-2 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Basic Troubleshooting Tips Power Verify that the following are all powered on: ProLiant servers (cluster nodes) StorageWorks RAID Array 4000s (RA4000 Arrays) or StorageWorks RAID Array 4100s (RA4100 Arrays)
Page 148: Access To Cluster Components
The following software components must have the same revision level on each of the cluster nodes: Oracle8i Server Release 8.1.7 with Oracle8i Parallel Server Option Release 8.1.7 RA4000 Array Controller drivers Compaq Ethernet OSDs Microsoft Windows 2000 Advanced Server, including Service Pack 1 or later...
Page 149: Firmware Revisions
7-4 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide For information about currently supported software revisions for the PDC/O1000, refer to the Compaq Parallel Database Cluster Model PDC/O1000 Certification Matrix for Windows 2000 at www.compaq.com/solutions/enterprise/ha-pdc.html...
Page 150: Troubleshooting Oracle8I And Osd Installation Problems And Error Messages
Troubleshooting 7-5 Troubleshooting Oracle8i and OSD Installation Problems and Error Messages Potential Difficulties Installing the OSDs with the Oracle Universal Installer The Oracle Universal Installer (OUI) uses an inventory file to track which components of the OSDs have been installed on each of the cluster nodes.
Page 151: Unable To Start Oraclecmservice
7-6 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Determine whether or not you need to deinstall a partial OSD installation. An incomplete installation can result in a partial installation of the OSDs on the cluster. If this occurs, you must clean up after the partial installation (by removing it) before you can successfully run the OUI again.
Page 152: Unable To Start Oraclenmservice
Verify that the OracleNMService has properly started by checking the error log. The default location for the error log is C:\compaq\ops\nm.log. If not found in the default location, look in the same directory in which the OSD modules were installed.
Page 153: Initialization Of The Dynamic Link Library Nm.dll Failed
7-8 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Initialization of the Dynamic Link Library NM.DLL Failed If this failure occurs, stop the currently running OracleCMService and OracleNMService. Restart the OracleNMService, then the OracleCMService. From Administrative Tools, open the Services screen. Locate each service in the displayed list, highlight it, then stop it.
Page 154: Unable To Ping The Cluster Interconnect Or The Client Lan
Troubleshooting 7-9 When Using an Ethernet Cluster Interconnect Make sure that the Ethernet cluster interconnect components (adapters, cables, switches or hubs) are properly installed as specified in Chapter 5, “Installation and Configuration.” Make sure that the Ethernet cluster interconnect and the client LAN are connected to different Ethernet switches or hubs.
Page 155: Node Or Nodes Unable To Rejoin The Cluster
7-10 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Node or Nodes Unable to Rejoin the Cluster If you have an Ethernet cluster interconnect, use ping to ensure that the rejoined node or nodes can communicate with other nodes in the cluster.
Page 156: Troubleshooting Shared Storage Problems
RA4000/RA4100 Array in a stand-alone server configuration. For those issues, refer to the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide.
Page 157: Verifying Connectivity To A Non-Redundant Fibre Channel Arbitrated Loop
7-12 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Verifying Connectivity to a Non-Redundant Fibre Channel Arbitrated Loop To ensure that all I/O path connections in a non-redundant FC-AL are intact and functioning, verify the following: For each redundant FC-AL, one Fibre Host Adapter has been installed in the same PCI slot across all servers in the cluster.
Page 158 Troubleshooting 7-13 Determine whether or not the Compaq servers support the physical SCSI disks installed in the RA4000/RA4100 Arrays. Check the event log for disk I/O error messages or indications of problems with the communications transport. If the drives are still not recognized, it is recommended that you start investigating the problem from the lowest level of drive configuration.
Page 159: A Cluster Node Cannot Connect To The Shared Drives
7-14 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide A Cluster Node Cannot Connect to the Shared Drives Verify that a physical connection exists between each Fibre Host Adapter and its Fibre Channel SAN Switch, FC-AL Switch, or Storage Hub.
Page 160: Diagnosing And Resolving Shared Disk Problems
Diagnosing and Resolving Shared Disk Problems Introduction When one or more nodes in a Compaq Parallel Database Cluster Model PDC/O1000 (referred to here as the PDC/O1000) cannot properly join the cluster, you need to consider two possible causes: A problem related to the shared disk storage in the cluster’s...
Page 161: Figure A-1. Tasks For Diagnosing And Resolving Shared Disk Problems
A-2 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Figure A-1 identifies, in a flowchart format, the sequence of tasks you should perform to diagnose, isolate, and resolve shared disk problems in a PDC/O1000 cluster. A detailed description of each task is provided in the following sections of this appendix.
Page 162: Run Object Link Manager On All Nodes
“Restart All Affected Nodes in the Cluster” in this appendix for further instructions. For instructions on installing and using Object Link Manager, see the readme file: \Object Link Manager\readme.txt This file is located on the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD.
Page 163: Restart All Affected Nodes In The Cluster
A-4 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Restart All Affected Nodes in the Cluster If Object Link Manager does not run properly on one or more nodes or does not provide consistent information about Oracle partitions and symbolic links from all nodes, perform these steps: 1.
Page 164: Run Disk Management On All Nodes
Diagnosing and Resolving Shared Disk Problems A-5 Run Disk Management On All Nodes Perform these steps from each node: 1. From Disk Management, verify that each RA4000 Array or RA4100 Array is visible. 2. Verify that all shared drive information (drive letters, volume labels, number of partitions, partition sizes, etc.) is consistent across all nodes.
Page 165: Perform Cluster Software And Firmware Checks
A-6 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Perform Cluster Software and Firmware Checks Use the Compaq SmartStart and Support Software CD to perform these checks on the cluster software and firmware: 1. Verify that all necessary drivers have been installed. For example, verify...
Page 166: Contact Your Compaq Support Representative
Contact Your Compaq Support Representative If you are unable to resolve the problem, contact your Compaq support representative for assistance. In North America, call the Compaq Technical Support Phone Center at 1-800-OK-COMPAQ. Outside North America, call the nearest Compaq Technical Support Phone Center.
Page 167: Glossary
Glossary Availability A measure of how well a computer system can deliver services to its clients continuously. Availability is typically expressed as a percentage, with 100 percent being the best possible rating. Client LAN A local area network dedicated to servicing requests between network client machines and between the cluster nodes and the client machines.
Page 168 2 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Fibre Channel An IEEE standard for providing high-speed data transfer among workstations, mainframes, supercomputers, desktop computers, storage devices, and display devices. Fibre Channel A storage area network (SAN) topology that uses the Storage Hub or...
Page 169 Glossary 3 IP Address Internet Protocol Address. An address assigned to a network interface card, which computer entities use to locate and communicate with each other. IP addresses can be statically or dynamically assigned. Local Area Network. Load balancing To balance the amount of work (processing) undertaken by each node in a cluster.
Page 170 4 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Reliability The continuous integrity of a system (server, storage, network, or cluster). Resource A software or hardware entity upon which a client/server application or service is dependent. As it pertains to the PDC/O1000, a cluster resource must have the ability to be managed by the cluster and must reside on one of the cluster nodes.
Page 171: Index
Windows 2000 Advanced non-redundant Fibre Server 5-18 Channel Fabrics 4-4 availability certification defined 1-3 Oracle8i software 3-7 features of Compaq ProLiant CIM See Compaq Insight Manager Servers 2-3 client LAN adapters installation 5-7 cabling 5-16 overview 2-29 planning 4-7, 4-20...
Page 172 2 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide redundant 5-7, 5-16 cluster manager, starting 5-35 transmissions allowed 5-7 cluster nodes verifying adding a new node 6-21 communications 5-22 connectivity failure 7-14 verifying IP addresses 5-20...
Page 173 7-10 with 7-13 cluster cannot communicate using to set up RA4000/4100 with client 7-10 Arrays 5-8 cluster connectivity 7-9 Compaq authorized reseller xx Compaq Full Spectrum Fault Ethernet cluster interconnect 7-9 Management 3-4 FC-AL connectivity Compaq Insight Manager guidelines 7-12...
Page 174 4 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Compaq Options ROMPaq, Compaq StorageWorks RAID overview 3-4 Array 4000 Array Controllers Compaq Parallel Database Cluster, description 2-5 overview 1-4 Compaq StorageWorks RAID Compaq SmartStart and Support...
Page 175 FC-ALSs 5-10 multiple non-redundant FC- ALs 2-18 non-redundant FC-AL 2-16 error logs topology overview 2-16 for Oracle8i 6-4 FC-AL Switches See Compaq Ethernet adapters StorageWorks FC-AL Switches planning 4-6 description 2-7 Ethernet client LAN FFIU See Fibre Channel Fault...
Page 176 6 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Fibre Host Adapters See Compaq I/O data paths in a non-redundant StorageWorks Fibre Channel FC-AL 2-20 Host Bus Adapters I/O data paths in a non-redundant replacing 6-17...
Page 177 Oracle8i Server Enterprise rerunning and validating on Edition 3-5 affected nodes A-4 overview 3-5 running to check for shared OracleCMService disk problems A-3 starting 5-35 operating system dependent troubleshooting 7-6 modules See Compaq OracleNMService, operating system dependent troubleshooting 7-7 modules...
Page 178 8 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide OSDs See Compaq operating shared storage components for system dependent modules non-redundant Fibre OUI See Oracle Universal Installer Channel Fabrics 4-4 overview site planning 4-2 cluster management 6-1...
Page 179 Oracle8i 3-7 Arrays 6-14 installation verification for Fibre Host Adapters 6-17 Oracle8i 5-38 RJ-45 receptacle xviii installing for Windows 2000 ROMPaq See Compaq Options Advanced Server 5-17 Rompaq revisions 7-3 Storage Hubs See Compaq StorageWorks Fibre Channel Storage Hub SCU See Compaq System...
Page 180 10 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide cluster degradation 6-2 physical connections 7-2 cluster interconnect 7-8 power 7-2 Ethernet cluster software revisions 7-3 interconnect 7-9 FC-AL connectivity guidelines 7-12 uninterruptible power supplies Fibre Channel Fabric...

This manual is also suitable for:

Proliant 6500 Proliant 8500 6400r - proliant - 1 gb ram Pdc/o1000

Compaq 112726-001 - ProLiant - 6500 Administrator's Manual

About this Guide

Chapter 1 Clustering Overview

Chapter 2 Architecture

Chapter 3 Cluster Software Components

Chapter 4 Cluster Planning

Chapter 5 Installation and Configuration

Installation and Configuration

Chapter 6 Cluster Management

Cluster Management

Chapter 7 Troubleshooting

Appendix A Diagnosing and Resolving Shared Disk Problems

Quick Links

Windows 2000

Troubleshooting

Related Manuals for Compaq 112726-001 - ProLiant - 6500

Summary of Contents for Compaq 112726-001 - ProLiant - 6500