Apr 26 2010

Install 11g Release 2 Grid Infrastructure for Standalone Server on Windows 7 for Sandbox

Category: 11g,Databaseittichai @ 9:08 pm

Oracle 11g Release 2 for Windows was just released this month. With the availability of the grid infrastructure in this version, I plan to install it on my Windows 7 desktop to see what it can do even if it is just on stand alone environment.

In order for database to use Automatic Storage Management (ASM), it requires the Grid Infrastructure. In addition to ASM, Grid Infrastructure will also provide Oracle Restart to manage the Oracle processes (database, listener, and ASM).

One of the first issues I’ve encountered is the new requirement that the clusterware files (OCR & Voting) must be on ASM. I have to admit even though I’ve done ASM on Solaris and Linux before, but never on Windows. Since this is mandatory, I will give it a try. And since I will use ASM for clusterware files, I plan to use it for database data files as well.

In order to use ASM, I’m required to provide the unformatted (raw) basic disks. I plan to use the existing disks without adding new physical ones. Fortunately in Windows 7, I can use the disk management (diskmgmt.msc) tool to shrink volume and create a new logical disk from claimed space. Note that you may have multiple physical disks on your machine, but ASM supports and recognizes only logical drives on the Basic disk (not Dynamic disk). Click here if you’re interested in differences between Basic and Dynamic disks.

Once data volume is shrunk, I can create a new volume and then a logical drive. The new drive must not be formatted or having a drive letter assigned to it. Here is the guidelines from Oracle document on “create disk partitions”.

To use ASM with direct attached storage or SAN, the disks must be stamped with a header. This can be accomplished by using either asmtool (command-line version) or asmtoolg (GUI version). Since we will install Oracle grid infrastructure in interactive mode, the asmtoolg will be called during the configuration. Somehow, if I tried to launch the asmtoolg outside Oracle grid infrastructure installation, I always encountered error with no disks found. However, within the Oracle grid infrastructure installation, there is no issue.

In general, the installation went well. I’ve encountered few issues which I’ve documented them in the documents below. The snapshots of steps here are for educational purpose only.

Windows 7 – Disk Preparation for ASM

Oracle 11g R2 Grid Infrastructure for Standalone Server Installation on Windows

Oracle 11g R2 Software Installation for Single Instance Database on Windows

Oracle 11g R2 Database Creation using ASM on Windows

Tags: , , , , , , , , , ,


Oct 13 2009

OOW09 Day 2

Category: Events,OOWittichai @ 12:04 am

Today all my sessions are pretty much centered around Oracle ASM. Even though all sessions started with basic information about ASM (what is, why, and so on), I found the Q&A time at the end more interesting. One of the sessions – “Extending Oracle ASM in Release 11.2 to Manage All Data” has the panel of the ASM development team for Q & A. It is awesome to see these people who are responsible for this cool product face-to-face.

ASM Development Team

Oracle ASM Development Team

A common question I’ve heard over and over is that how many diskgroups Oracle recommends in a deployment. The panel said that regardless of the number of databases, still only two diskgroups are recommended as the best practice – one diskgroup for work area (e.g., data files), other for recovery (e.g., flash recovery). The exception of more-than-two diskgroups is if organization has ILM strategy – putting disks of different tiers into different diskgroups based on requirement and strategy.

A quick note from the ASM development team while they’re talking about the ACFS snapshot is that they’re working on more of data services around ACFS such as storage replication. We should hear about it soon. :-)

Tags: , , , ,


Oct 05 2009

Online SAN Storage Migration for Oracle 11g RAC database with ASM

Category: 11g,Database,Migrationittichai @ 7:30 am

It has been awhile since my last post. My pathetic excuses are all pretty much mentioned here. :-)

Last month we’ve worked with the storage team to migrate the SAN storage of our Oracle 11gR1 database to a new one. The drive of migration is mainly for SAN consolidation which is, of course, ultimately for cost saving. In addition to migrating the ASM disk groups storing database’s data files, all clusterware files (OCR and voting disk) must be migrated too. The rebalance feature in Oracle ASM makes data migration very easy and seamless. And since the clusterware files have redundancy, they can be seamlessly migrated as well. With 11gR1, all migration tasks can be performed online.

Prerequisites/Assumptions:

- New SAN LUNs/disks are already visible to all RAC nodes. In case of the disks for ASM diskgroups, they are already discovered by ASM. The minimum numbers and permissions of the OCR and voting disks must be met.

- It is recommended to perform the migration tasks during off-peak hours or even better if during planned maintenance window period.

Note that the sample shown here is specific to my environment (11.1.0.7 on Solaris 10 with dual-pathing to Hitachi SAN, and OCR and voting disks are on raw devices).

SAN Migration of the ASM diskgroups

If you’re more comfortable with GUI, all tasks here can be accomplished using the Enterprise Manager.

1. Add new disks to ASM diskgroups.

ALTER DISKGROUP PMDW_DG1 ADD DISK
  '/dev/rdsk/c4t60060E80056FB30000006FB300000823d0s6' NAME PMDW_DG1_0003,
  '/dev/rdsk/c4t60060E80056FB30000006FB300000826d0s6' NAME PMDW_DG1_0004,
  '/dev/rdsk/c4t60060E80056FB30000006FB300000829d0s6' NAME PMDW_DG1_0005
REBALANCE POWER 11;

We go with the rebalance power of 11 which is full throttle because it is planned maintenance.

2. Check rebalance status from Enterprise Manager or v$ASM_OPERATION.

3. When rebalance completes, drop the old disks.

ALTER DISKGROUP PMDW_DG1 DROP DISK
   PMDW_DG1_0000,
   PMDW_DG1_0001,
   PMDW_DG1_0002
REBALANCE POWER 11;

When adding or removing several disks, it is recommend to add or remove all disks at once. This is to reduce the number of the rebalance operations that are needed for storage changes.

SAN Migration of the OCR Files

1. Backup all OCR-related files.

# {CRS_HOME}/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          2
         Total space (kbytes)     :     921332
         Used space (kbytes)      :       4548
         Available space (kbytes) :     916784
         ID                       :  776278942
         Device/File Name         : /dev/rdsk/c4t50060E800000000000002892000003F8d0s6
                      Device/File integrity check succeeded
         Device/File Name         : /dev/rdsk/c4t50060E800000000000002892000003F9d0s6
                       Device/File integrity check succeeded

Backup the /var/opt/oracle/ocr.loc file:

# cp ocr.loc ocr.loc.old

Manually backup OCR:

# {CRS_HOME}/bin/ocrconfig -manualbackup

2. As root, run the following commands to replace OCR files. This change can be performed on-line, and will be reflected across the entire cluster.

# {CRS_HOME}/bin/ocrconfig -replace ocr /dev/rdsk/c4t60060E80056FB30000006FB300001014d0s6

# {CRS_HOME}/bin/ocrconfig -replace ocrmirror /dev/rdsk/c4t60060E80056FB30000006FB300001015d0s6

3. Verify the new configuration.

Check new ocr.loc file updated:

# cat /var/opt/oracle/ocr.loc
#Device/file /dev/rdsk/c4t50060E800000000000002892000003F9d0s6 getting replaced by device /dev/rdsk/c4t60060E80056FB30000006FB300001015d0s6
ocrconfig_loc=/dev/rdsk/c4t60060E80056FB30000006FB300001014d0s6
ocrmirrorconfig_loc=/dev/rdsk/c4t60060E80056FB30000006FB300001015d0s6

Check OCR:

# {CRS_HOME}/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version                  :          2
Total space (kbytes)     :     921332
Used space (kbytes)      :       4548
Available space (kbytes) :     916784
ID                       :  776278942
Device/File Name         : /dev/rdsk/c4t60060E80056FB30000006FB300001014d0s6
Device/File integrity check succeeded
Device/File Name         : /dev/rdsk/c4t60060E80056FB30000006FB300001015d0s6
Device/File integrity check succeeded

Cluster registry integrity check succeeded

Logical corruption check succeeded

SAN Migration of the voting disks

1. Backup the voting disks.

Query the original locations:

# /opt/oracrs/bin/crsctl query css votedisk
0.     0    /dev/rdsk/c4t50060E800000000000002892000003FBd0s6
1.     0    /dev/rdsk/c4t50060E800000000000002892000003FCd0s6
2.     0    /dev/rdsk/c4t50060E800000000000002892000003FFd0s6

Backup voting disks using dd:

dd if={voting_disk_name} of={backup_file_name}

Example,
dd if=/dev/rdsk/c4t50060E800000000000002892000003FBd0s6 of=/tmp/voting1

2. Move voting disks.

Starting with 11.1 onwards, the voting disk migration can be performed on-line.

# /opt/oracrs/bin/crsctl delete css votedisk /dev/rdsk/c4t50060E800000000000002892000003FBd0s6
# /opt/oracrs/bin/crsctl add  css votedisk /dev/rdsk/c4t60060E80056FB30000006FB300001017d0s6

# /opt/oracrs/bin/crsctl delete css votedisk  /dev/rdsk/c4t50060E800000000000002892000003FCd0s6
# /opt/oracrs/bin/crsctl add css votedisk  /dev/rdsk/c4t60060E80056FB30000006FB300001018d0s6

# /opt/oracrs/bin/crsctl delete css votedisk  /dev/rdsk/c4t50060E800000000000002892000003FFd0s6
# /opt/oracrs/bin/crsctl add css votedisk  /dev/rdsk/c4t60060E80056FB30000006FB300001019d0s6

3. Verify the new configuration.

# /opt/oracrs/bin/crsctl query css votedisk
0.     0    /dev/rdsk/c4t60060E80056FB30000006FB300001017d0s6
1.     0    /dev/rdsk/c4t60060E80056FB30000006FB300001018d0s6
2.     0    /dev/rdsk/c4t60060E80056FB30000006FB300001019d0s6

Reference:
Metalink #428681.1:  OCR / Vote disk Maintenance Operations: (ADD/REMOVE/REPLACE/MOVE), including moving from RAW Devices to Block Devices

Tags: , , , , , ,


Mar 09 2009

asmcmd cp command on the test with transportable tablespace

Category: Databaseittichai @ 7:37 am

As you all know, the transportable tablepsace (TTS) enables users to unplug a set of tablespaces from one database, and plug it into another database. It is very efficient to move bulk data because only metadata and data files are involved in file movement – this can be much faster than export/import. More detailed information can be found in Oracle document.

We’ve been using TTS in 10g for many scenarios especially in the database refresh. However, we’ve never used it in database with ASM. For pre-11g with ASM, RMAN or DBMS_FILE_TRANSFER has to be used to move data out of ASM into file system or vice versa. With new feature of asmcmd cp command in 11g, I’d like explore whether or not this could be successfully used to move data files in/out ASM on our 11.1.0.7 database for the TTS export/import purpose.

Notes below are straight-forward step-by-step for TTS configuration. However, when copying data files, I use asmcmd cp instead. In the initial run, I’ve encountered ORA-01200 error during import saying that the actual copied file size is smaller than the correct size. It took me awhile to discover that when copying a data file larger than 2GB, the file was actually shrunk! This was filed as a bug by Oracle support. I will talk more about this at the end of this post. So all samples below are done with files smaller than 2GB. I’ve tested with 100 MB and 1.1 GB, both cases are working fine.

Assumption

The test_user has tables in the TTSASMCP_TS tablespace. One of them is testtb.

SQL> conn testuser
SQL> select count(*) from testtb;
COUNT(*)
----------
1000

Prerequisite

1. Make sure that this tablespace is self-contained for transportable.

SQL> exec dbms_tts.transport_set_check(TS_LIST=>'TTSASMCP_TS', incl_constraints=>TRUE);
PL/SQL procedure successfully completed.

If there are multiple transportable tablespaces, they can be listed with comma-separated in TS_LIST.

2. View whether or not there is any violation.

SQL> select * from transport_set_violations;
no rows selected

Start Transportable Tablespace Export using expdp

1. Make tablespace read-only.

SQL> alter tablespace TTSASMCP_TS read only;
Tablespace altered.

2. Locate data files for this tablespace.

col tablespace_name format a10
col file_name format a50

SQL> select tablespace_name, file_name from dba_data_files
where tablespace_name = 'TTSASMCP_TS';

TABLESPACE   FILE_NAME
----------   --------------------------------------------------
TTSASMCP_TS  +PROD_DG1/ORA11PD/datafile/ttsasmcp_ts.314.680796267

3. Create a directory to be used for export data pump.

SQL> create directory expdir as '/opt/oracle/admin/ORA11PD/exp';
Directory created.

SQL> grant read, write on directory expdir to system;
Grant succeeded

4. Perform export data pump.

$ expdp system directory=expdir dumpfile=ttsasmcp.dmp transport_tablespaces=TTSASMCP_TS include=triggers,constraints,grant

Export: Release 11.1.0.7.0 - 64bit Production on Friday, 06 March, 2009 9:51:12

Copyright (c) 2003, 2007, Oracle.  All rights reserved.
Password:

Connected to: Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
Starting "SYSTEM"."SYS_EXPORT_TRANSPORTABLE_01": system/******** directory=expdir dumpfile=ttsasmcp.dmp
transport_tablespaces=TTSASMCP_TS
Processing object type TRANSPORTABLE_EXPORT/PLUGTS_BLK
Processing object type TRANSPORTABLE_EXPORT/TABLE
Processing object type TRANSPORTABLE_EXPORT/POST_INSTANCE/PLUGTS_BLK
Master table "SYSTEM"."SYS_EXPORT_TRANSPORTABLE_01" successfully loaded/unloaded
******************************************************************************
Dump file set for SYSTEM.SYS_EXPORT_TRANSPORTABLE_01 is:
/opt/oracle/admin/ORA11PD/exp/ttsasmcp.dmp
******************************************************************************
Datafiles required for transportable tablespace TTSASMCP_TS:
+PROD_DG1/ORA11PD/datafile/ttsasmcp_ts.314.680796267
Job "SYSTEM"."SYS_EXPORT_TRANSPORTABLE_01" successfully completed at 14:10:08

5. Once completed, move the export’s metadata file and data file over to target database.

ASMCMD> cp +PROD_DG1/ora11pd/datafile/TTSASMCP_TS.314.680796267 /tmp
copying +PROD_DG1/ora11pd/datafile/TTSASMCP_TS.314.680796267 -> /tmp/TTSASMCP_TS.314.680796267

$ scp /tmp/TTSASMCP_TS.314.680796267 serverB:/tmp
$ scp /opt/oracle/admin/ora11pd/exp/ttsasmcp.dmp serverB:/opt/oracle/admin/ora11dv/exp/

6. Put the original tablespace back to read write mode.

SQL> alter tablespace TTSASMCP_TS read write;
Tablespace altered.

In this case, since both of our systems are on the same platforms, there is no need for platform conversion. If you’re interested in, information about cross-platform transportable tablespace can be found here and here.

Start Transportable Tablespace Import

1. Create a needed directory object for import data pump.

SQL> create directory expdir as '/opt/oracle/admin/ora11dv/exp';
Directory created.

SQL> grant read, write on directory expdir to system;
Grant succeeded

2. Move the data file to the data file location under ASM.

ASMCMD> cp /tmp/TTSASMCP_TS.314.680796267 +DEV_DG1/ora11dv/datafile/TTSASMCP_TS
copying /tmp/TTSASMCP_TS.314.680796267 -> +DEV_DG1/ORA11DV/datafile/TTSASMCP_TS

However, since ASM will automatically append file/incarnation pair to ensure uniqueness, you will need to list to get a new data file name.

ASMCMD> ls -s +DEV_DG1/ora11dev/datafile/TT*
Block_Size  Blocks      Bytes      Space  Name
16384    6401  104873984  106954752  TTSASMCP_TS.271.680796929

Please note that the following syntaxes won’t work. This is very inconvenient because it is common way of typing cp syntax in Unix. Hopefully it will be fixed in the later release.

cp /tmp/TTSASMCP_TS.314.680796267 +DEV_DG1/ora11dv/datafile/

or

cp /tmp/TTSASMCP_TS.314.680796267 +DEV_DG1/ora11dv/datafile/TTSASMCP_TS.314.680796267

You will receive this error

ORA-15046: ASM file name ‘+DEV_DG1/ora11dv/datafile/TTSASMCP_TS.314.680796267′ is not in single-file creation form.

3. Rum impdp

$ impdp system directory=expdir dumpfile=ttsasmcp.dmp transport_datafiles='+DEV_DG1/ora11dv/datafile/TTSASMCP_TS.271.680796929'

Import: Release 11.1.0.7.0 - 64bit Production on Friday, 06 March, 2009 14:25:34

Copyright (c) 2003, 2007, Oracle.  All rights reserved.
Password:

Connected to: Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
Master table "SYSTEM"."SYS_IMPORT_TRANSPORTABLE_01" successfully loaded/unloaded
Starting "SYSTEM"."SYS_IMPORT_TRANSPORTABLE_01": system/******** directory=expdir dumpfile=ttsasmcp.dmp
transport_datafiles=+DEV_DG1/ORA11DV/datafile/TTSASMCP_TS.271.680796929
Processing object type TRANSPORTABLE_EXPORT/PLUGTS_BLK
Processing object type TRANSPORTABLE_EXPORT/TABLE
Processing object type TRANSPORTABLE_EXPORT/POST_INSTANCE/PLUGTS_BLK
Job "SYSTEM"."SYS_IMPORT_TRANSPORTABLE_01" successfully completed at 14:25:50

Validation

SQL> conn testuser
SQL> select count(*) from testtb;
COUNT(*)
----------
1000

A bug found with asmcmd cp command

Normally when copying file, you must get the same file size :-) Sample here is with 104 MB file.

ASMCMD> ls -s tts*
Block_Size  Blocks      Bytes      Space  Name
16384    6401  104873984  106954752  TTSASMCP_TS.314.680796267

ASMCMD> cp TTSASMCP_TS.314.680796267 TTSASMCP_TS.backup
copying +PROD_DG1/ORA11PD/datafile/TTSASMCP_TS.314.680796267 -> +PROD_DG1/ORA11PD/datafile/TTSASMCP_TS.backup

ASMCMD> ls -s tts*
Block_Size  Blocks      Bytes      Space  Name
16384    6401  104873984  106954752  TTSASMCP_TS.314.680796267
16384    6401  104873984  106954752  TTSASMCP_TS.backup.313.680798709

Same with 1.1 GB.

ASMCMD> ls -s npi*
Block_Size  Blocks       Bytes       Space  Name
16384   70401  1153449984  1155530752  NPI_DATA.298.677341631

ASMCMD> cp NPI_DATA.298.677341631 NPI_DATA.backup
copying +PROD_DG1/ORA11PD/datafile/NPI_DATA.298.677341631 -> +PROD_DG1/ORA11PD/datafile/NPI_DATA.backup

ASMCMD>  ls -s npi*
Block_Size  Blocks       Bytes       Space  Name
16384   70401  1153449984  1155530752  NPI_DATA.298.677341631
16384   70401  1153449984  1155530752  NPI_DATA.backup.315.680916403

But for a file  larger than 2GB (21 GB in this sample), as you can see, the file size is smaller.

ASMCMD> ls -s co*
Block_Size   Blocks        Bytes        Space  Name
16384  1327249  21745647616  21747466240  CO_DATA.276.673697925

ASMCMD> cp CO_DATA.276.673697925 CO_DATA.backup
copying +PROD_DG1/ORA11PD/datafile/CO_DATA.276.673697925 -> +PROD_DG1/ORA11PD/datafile/CO_DATA.backup

ASMCMD> ls -s co*
Block_Size   Blocks        Bytes        Space  Name
16384  1327249  21745647616  21747466240  CO_DATA.276.673697925
16384    16529    270811136    272629760  CO_DATA.backup.313.680798663

I was told by Oracle support that this is a bug (#7828187) : Asmcmd CP Command Can Not Copy Files Larger Than 2 GB. Currently there is no patch available yet. The recommended work-around is to either use RMAN or dbms_file_transfer.copy_file to copy file.

Even though I still can’t use the asmcmd cp for a larger file copy until it is fixed, it is still very useful for making a backup of smaller files such as spfile.

If anyone encounters the same issue I’ve had, please  feel free to share your experience.

Tags: , , , , , , , , , ,