Prepare the cluster nodes for Oracle RAC

Before installation of RAC 11g R2 , you have to prepare the cluster nodes. Following must be performed.
User Accounts
The installation should be performed as the Local Administrator, the Local Administrator username and

password MUST be identical on all cluster nodes.
If a domain account is used, this domain account must be explicitly defined as a member of the Local
Administrator group on all cluster nodes.
For Windows 2008:

• Open Windows 2008 Server Manager
•  Expand the Configuration category in the console tree
•  Expand the Local Users and Groups category in the console tree
•  Within Groups, open the Administrator group
•  Add the desired user account as a member of the Administrator Group
•  Click OK to save the changes.

We must now configure and test the installation user's ability to interact with the other cluster nodes.

User Account changes specifically for Windows 2008:

1. Change the elevation prompt behavior for administrators to "Elevate without prompting" to allow for user
equivalence to function properly in Windows 2008:
Open a command prompt and type "secpol.msc" to launch the Security Policy Console management
utility.

•  From the Local Security Settings console tree, click Local Policies, and then Security Options
• Scroll down to and double-click User Account Control: Behavior of the elevation prompt for
  administrators.
• From the drop-down menu, select: "Elevate without prompting (tasks requesting elevation will
  automatically run as elevated without prompting the administrator)"
•  Click OK to confirm the changes.
•  Repeat the previous 5 steps on ALL cluster nodes.

 2. Ensure that the Administrators group is listed under "Manage auditing and security log":
Open a command prompt and type "secpol.msc" to launch the Security Policy Console management
utility.

•  Click on "Local Policies"
•  Click on "User Rights Assignment"
• Locate and double click the "Manage auditing and security log" in the listing of User Rights Assignments.
•  If the Administrators group is NOT listed in the "Local Security Settings" tab, add the group now.
•  Click OK to save the changes (if changes were made).
•  Repeat the previous 6 steps on ALL cluster nodes.

3. Disable Windows Firewall When installing Oracle Grid Infrastructure and/or Oracle RAC it is required to
turn off the Windows firewall. Follow these steps to turn off the windows firewall :

•  Click Start, click Run, type "firewall.cpl", and then click OK
• In the Firewall Control Pannel, click "Turn Windows Firewall on or off" (upper left hand corner of the window).
• Choose the "Off" radio button in the "Windows Firewall Settings" window and click OK to save the changes.
•  Repeat the previous 3 steps on ALL cluster nodes. 

After the installation is successful, you can enable the Windows Firewall for the public connections. However, to ensure correct operation of the Oracle software, you must add certain executables and ports to the Firewall exception list on all the nodes of a cluster.

**********************************************************************

Note:
The Firewall must be disabled for both Standard Profile and Domain Profile in the Windows registry.

You can change the EnableFirewall key value to '0' under the following keys on each RAC node if you receive error about firewall ennoblement even after performing the above step (3) .

'HKEY_LOCAL_MACHINE/System/CurrentControlSet/Services/SharedAccess/Parameters/FirewallPolicy/DomainProfile'

and

'HKEY_LOCAL_MACHINE/System/CurrentControlSet/Services/SharedAccess/Parameters/FirewallPolicy/StandardProfile'

**********************************************************************
Net Use Test
The "net use" utility can be used to validate the ability to perform the software copy among the cluster nodes.

•  Open a command prompt
• Execute the following (replacing C$ with the appropriate drive letter if necessary) repeat the command to ensure access to every node in the cluster from the local node replacing with the appropriate nodes in the cluster.

C:\Users\Administrator>net use \\remote node name\C$ The command completed successfully.

•  Repeat the previous 2 steps on ALL cluster nodes.

Remote Registry Connect
Validate the ability to connect to the remote nodes registry(s) as follows:

•  Open a command prompt and type "regedit"
•  Within the registry editor menu bar, choose File and select "Connect Network Registry"
•  In the Select Computer window enter the remote node name.
•  Click OK and wait for the remote registry to appear in the tree.
•  Repeat the previous 4 steps for ALL cluster nodes. 

Networking
NOTE: This section is intended to be used for installations NOT using GNS.
1. Determine your cluster name. The cluster name should satisfy the following conditions:

•  The cluster name is globally unique throughout your host domain.
•  The cluster name is at least 1 character long and less than 15 characters long.
• The cluster name must consist of the same character set used for host names: single-byte
• alphanumeric characters (a to z, A to Z, and 0 to 9) and hyphens (-).

 NOTE: It is a requirement that network interfaces used for the Public and Private Interconnect be consistently named (have the same name) on every node in the cluster. Common practice is to use the names Public and Private for the interfaces, long names should be avoided and special characters are NOT to be used.
2. Determine the public host name for each node in the cluster. For the public host name, use the primary host
name of each node. In other words, use the name displayed by the hostname command for example: racnode1.
It is recommended that NIC teaming is configured. Active/passive is the preferred teaming method
due to its simplistic configuration.

For Windows 2008:
Perform the following to rename the network interfaces:

•  Click Start, click Run, type "ncpa.cpl", and then click OK.
•  Determine the intended purpose for each of the interfaces (may need to view the IP configuration)
•  Right click the interface to be renamed and click "rename"
•  Enter the desired name for the interface.
• Repeat the previous 4 steps on ALL cluster nodes ensuring that the public and private interfaces have the same name on every node.

3. Determine the virtual hostname for each node in the cluster. The virtual host name is a public node name
that is used to reroute client requests sent to the node if the node is down. Oracle recommends that you
provide a name in the format -vip, for example: racnode1-vip. The virutal hostname must
meet the following requirements:

•  The virtual IP address and the network name must not be currently in use.
•  The virtual IP address must be on the same subnet as your public IP address.
• The virtual host name for each node should be registered with your DNS.

4. Determine the private hostname for each node in the cluster. This private hostname does not need to be
resolvable through DNS and should be entered in the hosts file (typically located in:
c:\windows\system32\drivers\etc). A common naming convention for the private hostname is -priv.<="" public="" style="text-align: left;">

 The private IP should NOT be accessible to servers not participating in the local cluster.

 The private network should be on standalone dedicated switch(es).

 The private network should NOT be part of a larger overall network topology.

 The private network should be deployed on Gigabit Ethernet or better.

It is recommended that redundant NICs are configured using teaming. Active/passive is the preferred
teaming method due to its simplistic configuration.
5. Define a SCAN DNS name for the cluster that resolves to three IP addresses (round-robin). SCAN VIPs
must NOT be in the c:\windows\system32\drivers\etc\hosts file. SCAN VIPs must be resolvable by DNS.
6. Even if you are using a DNS, Oracle recommends that you list the public IP, VIP and private addresses for
each node in the hosts file on each node. Configure the c:\windows\system32\drivers\etc\hosts file so that it is similar to the following example:
#public
192.168.26.11   P-HQ-CL-OR-11        P-HQ-CL-OR-11
192.168.26.12   P-HQ-CL-OR-12        P-HQ-CL-OR-12

#VIP
192.168.26.13    P-HQ-CL-OR-11-VIP        P-HQ-CL-OR-11-VIP
192.168.26.14    P-HQ-CL-OR-12-VIP        P-HQ-CL-OR-12-VIP

#private
10.11.11.1      P-HQ-CL-OR-11-PRIV                  P-HQ-CL-OR-11-PRIV
10.11.11.2      P-HQ-CL-OR-12-PRIV            P-HQ-CL-OR-12-PRIV

Network Ping Tests
There are a series of 'ping' tests that should be completed, and then the network adapter binding order should
be checked. You should ensure that the public IP addresses resolve correctly and that the private addresses are of the form 'nodename-priv' and resolve on both nodes via the hosts file.
* Public Ping test Pinging Node1 from Node1 should return Node1's public IP address Pinging Node2 from Node1
If any of the above tests fail you should fix name/address resolution by updating the DNS or local hosts files
on each node before continuing with the installation.
Network Interface Binding Order (and Protocol Priorities)
It is required that the Public interface be listed first in the network interface binding order on ALL cluster
nodes.
For Windows 2008:
Perform the follow tasks to ensure this requirement is met:

•  Click Start, click Run, type "ncpa.cpl", and then click OK.
• In the menu bar on the top of the window click "Advanced" and choose "Advanced Settings" (ForWindows 2008, if the "Advanced" is not showing, click 'Alt' to enable that menu item).
• Under the Adapters and Bindings tab use the up arrow to move the Public interface to the top of the Connections list.
•  Under Binding order for increase the priority of IPv4 over IPv6
•  Click OK to save the changes
•  Repeat the previous 5 steps on ALL cluster nodes

Disable DHCP Media Sense
Media Sense should be disabled. Media Sense allows Windows to uncouple an IP address from a card when
the link to the local switch is lost. You should disable this activity using the registry editor regedit.
Navigate to the Key HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters and
right click to create a new key of type DWORD. make sure that the Key is called DisableDHCPMediaSense, is of type DWORD and has a value of 1. For Windows 2008 we can check the status of DHCP Media Sense with the command:
netsh interface ipv4 show global
Disable SNP Features
On Windows 2003 SP2 and later platforms there are several network issues related to SNP features. These
issues are described in detail in Microsoft KB article 948496 and 951037. Perform the following tasks to take
proactive action on these potential issues:

•  Click Start, click Run, type "ncpa.cpl", and then click OK.
•  Right-click a network adapter object, and then click Properties.
•  Click Configure, and then click the Advanced tab.
•  In the Property list, click Receive Side Scaling, click Disable in the Value list, and then click OK.
•  In the Property list, click TCP/IP Offload, click Disable in the Value list, and then click OK.
•  Repeat steps 2 through 5 for each network adapter object.

The same can be accomplished on Windows 2008 by issuing the following commands:
netsh int tcp set global chimney=disabled and netsh int tcp set global rss=disabled
Validate these changes with the command: netsh interface ipv4 show global
Stopping Services
There can be issues with some (non-Oracle) services, which may already be running on the cluster nodes.
Typically a Microsoft Service: Distributed Transaction Coordinator (MSDTC) can interact with Oracle
software during install. It is recommended that this service is stopped and set to 'manual' start using
services.msc on both nodes.
Synchronizing the Time on ALL Nodes
There is a general requirement for Oracle RAC that the time on all the nodes be the same. With 11gR2 time
synchronization can be performed by the Clusterware using CTSSD (Cluster Time Synchronization Services
Daemon) or by using the Windows Time Service. If the Windows Time Service is being used, it MUST be
configured to prevent the time from being adjusted backwards. Perform the following steps to ensure the time
is NOT adjusted backwards using Windows Time Service:

•  Open a command prompt and type "regedit"
• Within the registry editor locate the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config key.
•  Set the value for MaxNegPhaseCorrection? to 0 and exit the registry editor.
•  Open a command prompt and execute the following to put the change into effect:

                    cmd> W32tm /config /update

•  Repeat steps 1 through 4 for ALL cluster nodes.

Environment Variables
Set the TEMP and TMP environment variables to a common location that exists on ALL nodes in the cluster.
During installation the Oracle Universal Installer (OUI) will utilize these directories to store temporary copies
of the binaries. If the location is not the same for both variables on ALL cluster nodes the installation will fail.
Most commonly these parameters are set as follows:
TMP=C:\temp TEMP=C:\temp
For Windows 2008:
To set the TEMP and TMP environment variables:

•  Log into the server as the user that will perform the installation
•  Open Computer Properties
•  Click the Advanced system settings link (on the left under tasks)
•  Under the Advanced tab, click the Environment Variables button
• Modify the TEMP and TMP variables under "User variables for Administrator" to the desired setting.
• Keep in mind, this path must be identical for both TMP and TEMP and they must be set to the same location on ALL cluster nodes.
•  Click OK to save the changes.
•  Repeat steps 1 through 6 for ALL cluster nodes.

Stage the Oracle Software
It is recommended that you stage the required software onto a local drive on Node 1 of your cluster.
Important. Ensure that you use only 32 bit versions of the Oracle Software on a 32bit OS and 64 bit versions
of the Oracle Software on a 64bit OS.

Cluster Verification Utility (CVU) stage check
Now you can run the CVU to check the state of the cluster prior to the install of the Oracle Software. Check if there is a newer version of CVU available on otn compared to the one that ships on the installation media
http://otn.oracle.com/rac