New ways to create disk groups in Oracle Database 11g Release 2

Guess what? As of Oracle Database 11g Release 2, we can create disk groups using the asmcmd command. Check out one of the new ways we can create a diskgroup using an XML file as a template. The mkdg ASMCMD command can accept an xml file as a parameter to create a disk group:

[oracle@jin1 ~]$ asmcmd mkdg dg_high.xml

[oracle@jin1 ~]$ cat dg_high.xml
<dg name="dg_high" redundancy="high">
  <fg name="fg1">
    <dsk string="ORCL:HIGH1" />
  </fg>
  <fg name="fg2">
    <dsk string="ORCL:HIGH2" />
  </fg>
  <fg name="fg3">
    <dsk string="ORCL:HIGH3" />
  </fg>
  <a name="compatible.asm" value="11.2"/>
  <a name="compatible.rdbms" value="11.2"/>
</dg>

And if we refresh our Automatic Storage Management main screen, we see the new diskgroup as shown here:

Let’s drop the dg_high disk group and show you how you can do this same thing with asmca command line interface.

Here’s the super hard syntax to drop the dg_high disk group:

ASMCMD> dropdg dg_high

Now, let’s create the disk group with asmca command line interface:

[oracle@jin1 ~]$ ksh cr_dg_high.txt

DiskGroup dg_high created successfully.

[oracle@jin1 ~]$ cat cr_dg_high.txt
asmca -silent -createDiskGroup \
-diskgroupname dg_high \
-redundancy high \
-disk 'ORCL:HIGH1' -disk 'ORCL:HIGH2' -disk 'ORCL:HIGH3' \
-failuregroup fg1 \
-failuregroup fg2 \
-failuregroup fg3 \
-au_size 64  \
-compatible.asm '11.2.0.0.0'  \
-compatible.rdbms '11.2.0.0.0'  \
-compatible.advm '11.2.0.0.0'

In the example with the asmca command line syntax, we added a little twist by adding the allocation extent to 64 MB and adding the compatible.advm attribute.

Majority of the ASM customers run external redundancy and have the protection provided to us by the storage array vendors. Let’s see our same example and show how to create an external redundant diskgroup. Here’s the XML file for the external redundancy disk group:

[oracle@jin1 ~]$ cat dg_external.xml
<dg name="dg_external" redundancy="external">
    <dsk string="ORCL:HIGH1" />
    <dsk string="ORCL:HIGH2" />
    <dsk string="ORCL:HIGH3" />
  <a name="compatible.asm" value="11.2"/>
  <a name="compatible.rdbms" value="11.2"/>
  <a name="compatible.advm" value="11.2"/>
</dg>

Using asmcmd, we will create the dg_external diskgroup:

[oracle@jin1 ~]$ asmcmd mkdg dg_external.xml

Alternatively, you can create the diskgroup in a single inline command as shown here:

ASMCMD> mkdg '<dg name="dg_external" redundancy="external">  <dsk string="ORCL:HIGH1" /> <dsk string="ORCL:HIGH2" /> <dsk string="ORCL:HIGH3" /> <a name="compatible.asm" value="11.2"/> <a name="compatible.rdbms" value="11.2"/>  <a name="compatible.advm" value="11.2"/> </dg>'

Obviously, nobody in their right mind would create a disk in this format. Wether you use asmca or asmcmd with an XML file or even the dreaded SQL*Plus, figure out what works best for you and stick with it. Depending on if you are a Unix System Administrator or Database Administrator, you may choose a different tool as your standard.

Posted by Charles Kim, Oracle ACE Director

Disable ocssd.bin daemon if you are not running ASM

ocssd.bin is required for ASM if you are not running RAC. If you are not running ASM, you will want to disable this process from starting up. You can use the following instructions from this blog.

bash-3.00$ ps -fu oracle
 UID PID PPID C STIME TTY TIME CMD
 oracle 9578 9567 0 14:39:17 pts/2 0:00 /bin/bash
 oracle 4129 2268 0 Sep 02 ? 4:54 /apps/oracle/product/GC/db10g/bin/ocssd.bin
..
.

Login as root and change your directory to /etc/init.d and issue the init.cssd command:

bash-3.00# ./init.cssd stop css
Shutting down CRS daemon.
Shutting down EVM daemon.
Shutting down CSS daemon.
Shutdown request successfully issued.
Shutdown has begun. The daemons should exit soon.
bash-3.00#

If you do not want ocssd.bin process to start back after the server reboot, type the following command:
bash-3.00# ./init.cssd disable css
Automatic startup disabled for system boot.

In Oracle 10.2 and above, Oracle recommends using the crsctl command to start/stop CSS. To stop the ocssd.bin daemon, change your directory to $ORACLE_HOME/bin and execute the following commands:

crsctl stop css
crsctl disable css

Posted by Charles Kim, Oracle ACE

Creating ASM Volumes and Cluster File Systems with Enterprise Mangers in 11g Release 2

Earlier in another blog post, I wrote on the subtle differences in how to start Enterprise Manager Database Control (dbconsole) in Oracle Database 11g Release 2

Let’s login to the EM Database Control. Your URL to login should be something like this:

https://jin1:1158/em

Scroll to the bottom of the screen to the instances section. You will see the database and ASM instance information as shown here:

Click on the +ASM1_[HOSTNAME] instance URL
The Automatic Storage Management Home page now includes sections for ASM Clustered File Systems (ACFS) and Volumes.

You will also notice a new tab dedicated to support the new ASM Cluster File System feature. The ASM Cluster File System tab will simplify your ability to create volumes, create cluster file systems and ACFS snapshots. In this blog post, we will only cover creating volumes and ASM cluster file systems.

Oracle ASM Dynamic Volume Manager (Oracle ADVM)

The new Oracle ASM Dynamic Volume Manager (Oracle ADVM) provides volume management services and a standard disk device driver interface. A volume device is constructed from an ASM volume. You can create one or more volumes devices within a disk group. Once you create a volume, you can create a file system using that volume. You can create an ASM cluster file system off of the ASM volume. Likewise, you can create an EXT3 or OCFS2 file system based on the ASM volume. Likewise, as Oracle ports ACFS to other Unix Platforms, you can create the OS native file systems using Oracle ASM disk group volumes. Oracle ACFS is layered on ASM through the ADVM interface.

Note:
Creating an OCFS2 file system on top of ASM Disk Group does not make sense, but it is do-able.

Requirements for ADVM
1. COMPATIBLE.ASM and COMPATIBLE.ADVM attributes must be set to 11.2.
2. Volume name can be a maximum of 11 alphanumeric characters (no dashes and first character must be alphabetic)

Let’s click on the new ASM Cluster File System tab and create ourselves an ASM Cluster File System.

You will see the following screen:

Notice that we already created the VOL_RPTS volume that is already mounted as /apps/oracle/acfsmounts/data_vol_rpts mount point. Let’s create a new volume and mount another file system. To start this process, click on the Create button on the right corner. You will be directed to the Create ASM Cluster File System screen. Before we can create an ASM Cluster File System, we must first create a volume. Click on the the Create ASM Volume button. From the Create ASM Volume screen,

Enter the values for Volume, Data Group, and Size.

Notice the size can be specified in MB, GB and TB.
If you want to see the SQL, click on the Show Command button

Click on the Return button to go back to the Create ASM Cluster File System Screen
You will notice that the Volume Device field is populated with the new Volume that you just created
Specify a label (optional) if you want and specify a Mount Point

Again, click on the Show Command button

Click on the Return button to back to the Create ASM Cluster File System screen
Click on the OK button to create the ACFS file system
You will be requested to enter OS credentials – This operating system user name and password will be used to perform ASM Cluster File System operations on this host.

You will receive a confirmation status on top of the screen indicating that the file system was successfully created:

Next, click on the VOL_LOGFILE Volume radio button and click on the Mount button on the top left portion of the screen. You will be routed to the Mount ASM Cluster File System: [VOLUME_NAME] screen.

Unfortunately, you will not be able to mount the file system from Enterprise Manager. Only thing you will be able to do is generate the command and ask your system administrator to mount the file system.

As the root user, issue the mount command shown from this screen.

[root@jin1 oracle]# /bin/mount -t acfs /dev/asm/vol_logfile-340 /apps/oracle/acfsmounts/data_vol_logfile

Now from the OS, you can see the output from the df -h command to see the new file system mounted:

[root@jin1 oracle]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                      122G   11G  106G   9% /
/dev/sda1              99M   12M   82M  13% /boot
tmpfs                 1.1G  154M  874M  15% /dev/shm
.host:/               163G  141G   23G  87% /mnt/hgfs
/dev/asm/vol_rpts-340
                      5.0G   49M  5.0G   1% /apps/oracle/acfsmounts/data_vol_rpts
/dev/asm/vol_logfile-340
                      2.0G   41M  2.0G   2% /apps/oracle/acfsmounts/data_vol_logfile

Click on the Return button to go back to the Mount Cluster File System screen. Click the Return button again to back to the ASM Cluster File System main screen.

Posted by Charles Kim, Oracle ACE

Creating ASM Snapshots with Enterprise Mangers in 11g Release 2

Earlier we went through the exercise to create ASM Volumes and Cluster File Systems within ASM disk groups. Another great feature introduced in Oracle Database 11g Release 2 is the capability to create snapshot images of the ASM cluster file systems. Oracle allows up to 63 read-only snapshots of their file system leveraging the Copy-on-write (COW) technology. When a snapshot request is submitted, COW technology stores only the meta-data about where the original data is stored is copied. The creation of the snapshot should be almost instantaneous since only the meta-data is being copied. The snapshot copy then tracks changed blocks to a snapshot storage location as writes are being performed. The original data that is being written to is copied into the designated area on the file system before the original data is overwritten. This is where the name copy-on-write comes from.

Read requests for unchanged data blocks go directly to the original file system. Read requests for changed data blocks are directed to the copied block on the snapshot storage volume. Please remember that the original data is only copied once. Subsequence change to the same block is not copied more than once.

EM provides full support for ASM snapshots. Let’s go EM Database Control and see how this is done. From the ASM Cluster File System tab, select a mount point and click on the Create Snapshot button.

You should se the Create Snapshot on ASM Cluster File System screen. Notice that the Snapshot Name column is auto-populated for you with shapshot_[date]_[time] format. You can change this to any kind of standard you may designate for your company.

Notice the Delete the oldest snapshot (62 remaining before the maximum limit is reached) check box. You will see this check box as you create your second and subsequent ASM snapshots. You will also see the oldest snapshot with the URL for easy convenience.

Click on the Show Command button to see the command line syntax that EM is about to submit:

For the command line DBAs, we can create snapshots copying and pasting from the following syntax:

/sbin/acfsutil snap create "snapshot_20090911_104833" /apps/oracle/acfsmounts/data_vol_logfile

Click on the Return button
From the Create Snapshot on ASM Cluster File System screen, click on the OK button. You will be routed to the ASM Cluster File System Host Credentials screen. Enter the username and password for oracle. If you want EM Database Control to remember the username and password, you can click on the option to save as preferred credential checkbox.

Click on the Continue button
You will be redirected to the ASM Cluster File System main page and will see a successful confirmation status:

We created our snapshot of the /apps/oracle/acfsmounts/data_vol_logfile file system. From the ASM Cluster File System main screen, click on the file system URL and look at some General information about that mount point.

You can see the general usage information and alerts for the file system. Usage information of the file system include Size (GB), Used (GB), Free (GB), Used (%), Allocated Space (GB) and Space Used by Snapshots (MB) or (GB).
Also in this screen, you will notice the Snapshots tab. Click on the Snapshots tab to see all the snapshot history for this ACFS file system. You can create additional snapshots or even delete existing snapshots from this screen:

And, of course, you can search for a specific snapshot using the search option on the snapshot name.

Posted by Charles Kim, Oracle ACE

Configure ACFS for auto-startup for non-RAC database servers

[root@rac9 bin]# ./acfsload start -s

[root@rac9 bin]# lsmod |grep -i ora
oracleacfs 781476 0
oracleadvm 212736 0
oracleoks 224864 2 oracleacfs,oracleadvm
oracleasm 46356 1

Create the file: /etc/init.d/acfsload

The file will contain the runlevel configuration, and the acfsload command. Place in the file the following lines

#!/bin/sh

# chkconfig: 2345 30 21
# description: Load Oracle ACFS drivers at system boot
/apps/oracle/product/11.2.0/grid/bin/acfsload start -s

[root@rac9 init.d]# chmod u+x /etc/init.d/acfsload
[root@rac9 init.d]# chkconfig –add acfsload
[root@rac9 init.d]# chkconfig –list acfsload
acfsload 0:off 1:off 2:on 3:on 4:on 5:on 6:off

Created by Charles Kim, Oracle ACE Director

Check for free space in ASM disks

You wanted to know how much space is actually being used at each of your ASM disks. You can use the following query to determine that.

set lines 255
col path for a35
col Diskgroup for a15
col DiskName for a20
col disk# for 999
col total_mb for 999,999,999
col free_mb for 999,999,999
compute sum of total_mb on DiskGroup
compute sum of free_mb on DiskGroup
break on DiskGroup skip 1 on report -

set pages 255

select a.name DiskGroup, b.disk_number Disk#, b.name DiskName, b.total_mb, b.free_mb, b.path, b.header_status
from v$asm_disk b, v$asm_diskgroup a
where a.group_number (+) =b.group_number
order by b.group_number, b.disk_number, b.name
/

set lines 122
set pages 66

Created by Charles Kim, Oracle ACE Director

ASMLIB Installation

This blog post will cover how to install ASMLIB and reveal the dependency order of the RPMs.

ASMLIB is only available for the linux operating system.  ASMLIB is available for the following flavors of Oracle Unbreakable Linux:

• SuSE Linux Enterprise Server 10
• Red Hat Enterprise Linux 5 AS
• Red Hat Enterprise Linux 4 AS
• SuSE Linux Enterprise Server 9
• Red Hat Enterprise Linux 3 AS
• SuSE Linux Enterprise Server 8 SP3
• Red Hat Advanced Server 2.1

Installing Oracle ASMLIB has dependency requirements. The RPMs must be installed in the following order:

1. oracleasm-support

2. oracleasm

3. oracleasmlib

If the System Administrators (SAs) attempt to apply RPMs in a different order than what is listed above, they will encounter the "error: Failed dependencies" message. In the example below, ASMLIB RPMs will be installed and configured in the correct dependency order:

[root@gc oracle]# rpm -ihv oracleasm-support-2.0.3-1.i386.rpm
Preparing… ########################################### [100%]
1:oracleasm-support ########################################### [100%]
[root@gc oracle]# rpm -ihv oracleasm-2.6.9-55.0.2.EL-2.0.3-1.i686.rpm
Preparing… ########################################### [100%]
1:oracleasm-2.6.9-55.0.2.########################################### [100%]
[root@gc oracle]# rpm -ihv oracleasmlib-2.0.2-1.i386.rpm
Preparing… ########################################### [100%]
1:oracleasmlib ########################################### [100%]

Alternatively, SAs can opt to install all three RPMs at once by listing all the RPMs as shown below:

[root@gc oracle]# rpm -ihv oracleasm-2.6.9-55.0.2.EL-2.0.3-1.i686.rpm \
oracleasmlib-2.0.2-1.i386.rpm \
oracleasm-support-2.0.3-1.i386.rpm
Preparing… ########################################### [100%]
1:oracleasm-support ########################################### [ 33%]
2:oracleasm-2.6.9-55.0.2.########################################### [ 67%]
3:oracleasmlib ########################################### [100%]

By listing all the RPMs on a single command line syntax, SAs do not have to be cognitive of dependency order.

We recommend that RPMs for ASMLIB 2.0 be downloaded from Oracle’s website. When downloading RPMs, SAs/DBAs must carefully select the appropriate RPMs specific for their linux kernel version and architecture. You can use the `uname -r` command to determine the exact version of the linux kernel specific to your environment.  RPMs specific to Redhat 4 can be downloaded from the website listed below:

http://www.oracle.com/technology/software/tech/linux/asmlib/rhel4.html

 If you have issues loading the oracleasm module, you will see errors in the /var/log/messages file that look similar to what is shown below:

Apr 24 22:24:42 gc modprobe: FATAL: Module oracleasm not found.
Apr 24 22:24:42 gc oracleasm: Unable failed

Created by Charles Kim, Oracle ACE Director

ASMCA Command Line Interface

In another post, we learned how to exploit all the ASMCA GUI features. asmca can be invoked with the -silent option and can effectively become a great command line interface.

You can create disk groups with the ASMCA configuration tool (ASMCA) or asmcmd command line interface as shown in this blog entry:
New ways to create disk groups in Oracle Database 11g Release 2

Another useful Oracle Database 11g Release 2 feature is the capability to add disk(s) to an existing disk group using asmca. In our example, we will add ORCL:DATA3 ASMLIB disk to the DATA diskgroup. Before we add the disk group, let’s look at the disks that make up the DATA disk group:

[oracle@jin1 ~]$ asmcmd lsdsk -G data
Path
ORCL:DATA1
ORCL:DATA2

Adding disks to an existing disk group:

Now, using ASM configuration assistant, let’s add a disk to the DATA disk group:

[oracle@jin1 ~]$ asmca -silent -adddisk -diskgroupname DATA -disk 'ORCL:DATA3'

Disks added successfully to diskgroup DATA

Let’s confirm that ORCL:DATA3 disk is added:

[oracle@jin1 ~]$ asmcmd lsdsk -G data
Path
ORCL:DATA1
ORCL:DATA2
ORCL:DATA3

Create ASM Volumes with asmca

You can leverage asmca to create volumes out of existing disk groups. Once you create volumes, you can use them to create ASM cluster file systesm. Let’s go through the process of creating a volume of the DATA disk group and creating an ASM cluster file system and mounting the file system for end users.

[oracle@jin1 tmp]$ ksh cr_asmca_volume.txt 

Volume vol_lob created successfully.

The contents of the file cr_asmca_volume.txt looks like this:

[oracle@jin1 tmp]$ cat cr_asmca_volume.txt
asmca -silent \
      -createVolume \
      -volumeName vol_lob \
      -volumeDiskGroup LOBDATA \
      -volumeSizeGB 4

Listing the /dev/asm directory, you can see that our volume that was create is vol_lob-484:

[oracle@jin1 tmp]$ ls -ltr /dev/asm
total 0
brwxrwx--- 1 root oinstall 252, 174081 Sep  9 00:45 vol_rpts-340
brwxrwx--- 1 root oinstall 252, 174082 Sep 11 06:58 vol_logfile-340
brwxrwx--- 1 root oinstall 252, 247809 Sep 12 13:29 vol_lob-484

Create ASM cluster file system (ACFS) with asmca

Using the vol_lob-484 volume, let’s create an ACFS file system:

[oracle@jin1 tmp]$ asmca -silent -createACFS -acfsVolumeDevice /dev/asm/vol_lob-484

ACFS creation on /dev/asm/vol_lob-484 completed successfully.
The ACFS should be mounted for use.

For the final step, login as root and mount the ASM file system:

[root@jin1 acfsmounts]# pwd
/apps/oracle/acfsmounts
[root@jin1 acfsmounts]# mkdir lobdata
[root@jin1 acfsmounts]# mount -t acfs /dev/asm/vol_lob-484 /apps/oracle/acfsmounts/lobdata

Verify that the newly created file system is mounted:

[root@jin1 acfsmounts]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                      122G   11G  106G   9% /
/dev/sda1              99M   12M   82M  13% /boot
tmpfs                 1.1G  154M  874M  15% /dev/shm
.host:/               163G  149G   15G  91% /mnt/hgfs
/dev/asm/vol_rpts-340
                      5.0G   49M  5.0G   1% /apps/oracle/acfsmounts/data_vol_rpts
/dev/asm/vol_logfile-340
                      3.0G  221M  2.8G   8% /apps/oracle/acfsmounts/data_vol_logfile
/dev/asm/vol_lob-484  4.0G   45M  4.0G   2% /apps/oracle/acfsmounts/lobdata

Posted by Charles Kim, Oracle ACE Director