I/O Tuning with Different RAID Configurations
PURPOSE
-------
This document gives a general overview of RAID (Redundant Arrays of
Inexpensive Disks), the different levels of RAID and their uses, and the use
of RAID with Oracle databases.
SCOPE & APPLICATION
-------------------
This note is intended to provide a discussion on RAID configurations.
1. Overview of RAID configurations and Oracle
---------------------------------------------
RAID-0:
-------
RAID-0 offers pure disk striping. The striping allows a large file to be spread
across multiple disks/controllers, providing concurrent access to data because
all the controllers are working in parallel. It does not provide either data
redundancy or parity protection. In fact, RAID-0 is the only RAID level focusing
solely on performance. Some vendors, such as EMC, do not consider level 0 as
true RAID and do not offer solutions based on it. Pure RAID-0 significantly
lowers MTBF, since it is highly prone to downtime. If any disk in the array
(across which Oracle files are striped) fails, the database goes down.
RAID-1:
-------
With RAID-1, all data is written onto two independent disks (a "disk pair") for
complete data protection and redundancy. RAID-1 is also referred to as disk
mirroring or disk shadowing. Data is written simultaneously to both disks to
ensure that writes are almost as fast as to a single disk. During reads, the
disk that is the least busy is utilized. RAID-1 is the most secure and reliable
of all levels due to full 100-percent redundancy. However, the main disadvantage
from a performance perspective is that every write has to be duplicated.
Nevertheless, read performance is enhanced, as the read can come from either
disk. RAID-1 demands a significant monetary investment to duplicate each disk;
however, it provides a very high Mean time between failures (MTBF). Combining
RAID levels 0 and 1 (RAID-0+1) allows data to be striped across an array,
in addition to mirroring each disk in the array.
RAID-0 & RAID-1:
----------------
If RAID/0 is then combined with RAID/1 (mirroring) this then provides the resilience,
but at a cost of having to double the number of disk drives in the configuration.
There is another benefit in some RAID/1 software implementations in that the requested
data is always returned from the least busy device.
This can account for a further increase in performance of over 85% compared
to the striped, non-mirrored configuration.
Write performance on the other hand has to go to both pieces of the software
mirror. If this second mirror piece is on a second controller (as would
normally be recommended for controller resilience), this degradation can be as
low as 4 percent.
RAID-3:
-------
In a RAID 3 configuration, a single drive is dedicated to storing error correction
or parity data. Information is striped across the remaining drives.
RAID/3 dramatically reduces the level of concurrency that the disk
subsystem can support (I/O's per second) to a comparable software mirrored
solution . The worst case for a system using RAID/3, would be an OLTP
environment, where the number of rapid transactions is numerous and response
time is critical.
So to put it simply, if the environment is mainly read only (Eg Decision
Support) RAID/3 provides disk redundancy with read performance slightly
improved, but at the cost of write performance. Unfortunately, even decision
support databases still do a significant amount of disk writing since complex
joins, unique searches etc still do temporary work, thus involving disk
writing.
RAID-5:
-------
Instead of total disk mirroring, RAID-5 computes and writes parity for every
write operation. The parity disks avoid the cost of full duplication of the
disk drives of RAID-1. If a disk fails, parity is used to reconstruct data
without system loss. Both data and parity are spread across all the disks in
the array, thus reducing disk bottleneck problems. Read performance is improved,
but every write has to incur the additional overhead of reading old parity,
computing new parity, writing new parity, and then writing the actual data, with
the last two operations happening while two disk drives are simultaneously
locked. This overhead is notorious as the RAID-5 write penalty. This write
penalty can make writes significantly slower. Also, if a disk fails in a RAID-5
configuration, the I/O penalty incurred during the disk rebuild is extremely
high. Read-intensive applications (DSS, data warehousing) can use RAID-5 without
major real-time performance degradation (the write penalty would still be
incurred during batch load operations in DSS applications). In terms of storage,
however, parity constitutes a mere 20-percent overhead, compared to the
100-percent overhead in RAID-1 and 0+1.
Initially, when RAID-5 technology was introduced, it was labeled as the
cost-effective panacea for combining high availability and performance.
Gradually, users realized the truth, and until about a couple of years ago,
RAID-5 was being regarded as the villain in most OLTP shops. Many sites
contemplated getting rid of RAID-5 and started looking at alternative solutions.
RAID 0+1 gained prominence as the best OLTP solution for people who could afford
it. Over the last two years, RAID-5 is making a comeback either as hardware-based
RAID-5 or as enhanced RAID-7 or RAID-S implementations. However, RAID-5 evokes
bad memories for too many OLTP database architects.
RAID-6
-------
Raid 6 extends Raid 5 with additional parity block. So with the second parity strip, Raid 6 gives
redundancy or high availability feature. This gives added protection, as even if data is lost on one
disk, the second parity stripe will be there to ensure integrity. Raid 6 does not have performance
penalty for reads but does have performance impact for writes, as there is overhead for the
additional parity configuration. Furthermore, the performance varies with the storage setup and
architecture. So before implementation, thorough evaluation should be made and investigated.
RAID-S:
-------
RAID S is EMC's implementation of RAID-5. However, it differs from pure RAID-5
in two main aspects:
(1) It stripes the parity, but it does not stripe the data.
(2) It incorporates an asynchronous hardware environment with a write cache.
This cache is primarily a mechanism to defer writes, so that the overhead of
calculating and writing parity information can be done by the system, while it
is relatively less busy (and less likely to exasperate the user!). Many users of
RAID-S imagine that since RAID-S is supposedly an enhanced version of RAID-5,
data striping is automatic. They often wonder how they are experiencing I/O
bottlenecks, in spite of all that striping. It is vital to remember that in
RAID-S, striping of data is not automatic and has to be done manually via
third-party disk-management software.
RAID-7:
-------
RAID-7 also implements a cache, controlled by a sophisticated built-in real-time
operating system. Here, however, data is striped and parity is not. Instead,
parity is held on one or more dedicated drives. RAID-7 is a patented architecture
of Storage Computer Corporation.
2. Pro's and Cons of Implementing RAID technology
-------------------------------------------------
There are benefits and disadvantages to using RAID, and those depend on the
RAID level under consideration and the specific system in question.
In general, RAID level 1 is most useful for systems where complete redundancy
of data is a must and disk space is not an issue. For large datafiles or
systems with less disk space, this RAID level may not be feasible. Writes
under this level of RAID are no faster and no slower than 'usual'.
For all other levels of RAID, writes will tend to be slower and reads will be
faster than under 'normal' file systems. Writes will be slower the more
frequently ECC's are calculated and the more complex those ECC's are.
Depending on the ratio of reads to writes in your system, I/O speed may have a
net increase or a net decrease. RAID can improve performance by distributing
I/O, however, since the RAID controller spreads data over several physical
drives and therefore no single drive is overburdened.
The striping of data across physical drives has several consequences besides
balancing I/O. One additional advantage is that logical files may be created
which are larger that the maximum size usually supported by an operating
system. There are disadvantages, as well, however. Striping means that it is
no longer possible to locate a single datafile on a specific physical drive.
This may cause the loss of some application tuning capabilities. Also, in
Oracle's case, it can cause database recovery to be more time-consuming. If a
single physical disk in a RAID array needs recovery, all the disks which are
part of that logical RAID device must be involved in the recovery.
One additional note is that the storage of ECC's may require up to 20%
more disk space than would storage of data alone, so there is some disk
overhead involved with usage of RAID.
3. RAID and Oracle
------------------
The usage of RAID is transparent to Oracle. All the features specific to
RAID configuration are handled by the operating system and go on behind-
the-scenes as far as Oracle is concerned. Different Oracle file-types
are suited differently for RAID devices. Datafiles and archive logs can be
placed on RAID devices, since they are accessed randomly.
The database is sensitive to read/write performance of the Redo Logs
and should be on a RAID 1 , RAID 0+1 or no RAID at all since they are accessed
sequentially and performance is enhanced in their case by having the disk drive
head near the last write location.
Keep in mind that RAID 0+1 does add overhead due to the 2 physical I/O's.
Mirroring of redo log files, however,is strongly recommended by Oracle.
In terms of administration, RAID is far simple than using Oracle
techniques for data placement and striping.
Recommendations:
In general, RAID usually impacts write operations more than read operation.
This is specially true where parity need to be calculated (RAID 3, RAID 5, etc).
Online or archived redo log files can be put on RAID 1 devices.
You should not use RAID 5. 'TEMP' tablespace data files should also go on
RAID1 instead of RAID5 as well. The reason for this is that streamed
write performance of distributed parity (RAID5) isn't as good as that of
simple mirroring (RAID1).
Swap space can be used on RAID devices without affecting Oracle.
====================================================================================
RAID Type of RAID Control Database Redo Log Archive Log
File File File File
====================================================================================
0 Striping Avoid* OK* Avoid* Avoid*
------------------------------------------------------------------------------------
1 Shadowing OK OK Recommended Recommended
------------------------------------------------------------------------------------
0+1 Striping + OK Recommended OK Avoid
Shadowing (1)
------------------------------------------------------------------------------------
3 Striping with OK Avoid Avoid Avoid
Static Parity (2)
------------------------------------------------------------------------------------
5 Striping with OK Avoid Avoid Avoid
Rotating Parity (2)
------------------------------------------------------------------------------------
* RAID 0 does not provide any protection against failures. It requires a strong backup
strategy.
(1) RAID 0+1 is recommended for database files because this avoids hot spots and gives
the best possible performance during a disk failure. The disadvantage of RAID 0+1
is that it is a costly configuration.
(2) When heavy write operation involves this datafile
RELATED DOCUMENTS
-----------------
References:
1. RAID: High-Performance, Reliable Secondary Storage, (Chen, Peter etal, 1994) References
2. Six Storage Tips for 24x7 Availability (Devaj, Venkat S.), Oracle Magazine
