A physical standby is the most common disaster-recovery setup for Oracle: a second database, on another server or in another region, kept up to date by shipping and applying the primary's redo. If the primary is lost, the standby takes over in minutes.
This is the runbook I follow to build one on Oracle 19c, from installing the software on the new server to a tested switchover. Each step says where to run it and how to know it worked before you move on.
Environment
| Primary | Standby |
|---|
| Host | prodhost | stbyhost |
| DB_NAME | PRODCDB | PRODCDB (must be the same) |
| DB_UNIQUE_NAME / SID | PRODCDB | PRODSTBY |
| Oracle Home | /u01/app/oracle/product/19.0.0/dbhome_1 | same path |
| Datafiles / FRA | /oradata/PRODCDB · /fra | same paths |
| Version | 19c with the same Release Update on both | |
| Protection mode | Maximum Performance (ASYNC) | |
Before you start
- Both servers run the same OS family and version, with the Oracle preinstall package applied on the standby.
- Port 1521 is open both ways between the servers (redo flows one way, gap fetching the other).
- The standby has disk space for the whole database plus the FRA.
- The network can carry your peak redo rate. Check
dba_hist_sysmetric_summary for "Redo Generated Per Sec".
- Clocks are in sync (chrony/NTP) on both servers.
STEP 01
Install identical Oracle software on the standbySTANDBY
The standby must run the same Oracle version and the same Release Update as the primary. In 19c the software is an image: unzip it into the new Oracle Home, then run the installer in software-only mode.
▸ STANDBY
mkdir -p /u01/app/oracle/product/19.0.0/dbhome_1
cd /u01/app/oracle/product/19.0.0/dbhome_1
unzip -q /stage/LINUX.X64_193000_db_home.zip
./runInstaller -silent -ignorePrereqFailure \
oracle.install.option=INSTALL_DB_SWONLY \
UNIX_GROUP_NAME=oinstall INVENTORY_LOCATION=/u01/app/oraInventory \
ORACLE_HOME=/u01/app/oracle/product/19.0.0/dbhome_1 ORACLE_BASE=/u01/app/oracle \
oracle.install.db.InstallEdition=EE \
oracle.install.db.OSDBA_GROUP=dba oracle.install.db.OSOPER_GROUP=oper \
oracle.install.db.OSBACKUPDBA_GROUP=backupdba oracle.install.db.OSDGDBA_GROUP=dgdba \
oracle.install.db.OSKMDBA_GROUP=kmdba oracle.install.db.OSRACDBA_GROUP=racdba
# then, as root, run root.sh as the installer tells you
Then apply the same RU and one-off patches as the primary (see my 19c patching post) and compare:
▸ BOTH
$ORACLE_HOME/OPatch/opatch lspatches
✓ Done when: opatch lspatches shows exactly the same list on both servers.
STEP 02
Prepare the standby file systemsSTANDBY
The easiest standby has the same directory layout as the primary, so datafile paths need no conversion. Create the same paths, owned by oracle:oinstall:
▸ STANDBY
mkdir -p /oradata/PRODCDB /fra /u01/app/oracle/admin/PRODSTBY/adump
chown -R oracle:oinstall /oradata /fra /u01/app/oracle/admin
chmod 750 /oradata /fra
If a disk has to be mounted under a different path to match the primary, do it cleanly: stop whatever uses it, then remount, and make it permanent in /etc/fstab using the UUID:
▸ STANDBY
fuser -vm /olddata # who is using it? stop those services/databases properly
umount /olddata
mkdir -p /oradata
blkid /dev/sde1 # get the UUID
vi /etc/fstab # UUID=<uuid> /oradata ext4 defaults,nofail 1 2
mount -a && df -h /oradata
Different paths? That works too. Add db_file_name_convert and log_file_name_convert to the SPFILE SET clause in Step 06.
✓ Done when: df -h shows the same data and FRA mount points on both servers, with enough free space for the full database.
STEP 03
Prepare the primary databasePRIMARY
▸ PRIMARY
-- Must be ARCHIVELOG
SELECT log_mode, force_logging, flashback_on, db_unique_name FROM v$database;
ALTER DATABASE FORCE LOGGING; -- no NOLOGGING operation may bypass the standby
ALTER DATABASE FLASHBACK ON; -- recommended: lets you reinstate after a failover
ALTER SYSTEM SET log_archive_config='DG_CONFIG=(PRODCDB,PRODSTBY)' SCOPE=BOTH;
ALTER SYSTEM SET fal_server='PRODSTBY' SCOPE=BOTH;
ALTER SYSTEM SET standby_file_management=AUTO SCOPE=BOTH;
ALTER SYSTEM SET remote_login_passwordfile=EXCLUSIVE SCOPE=SPFILE; -- usually already set
Standby redo logs (SRLs). Create them on the primary too: the duplicate copies them to the standby, and they are ready if the roles ever switch. The rules are the same size as the online redo logs, and one more group than the online groups, per thread.
▸ PRIMARY
SELECT thread#, group#, bytes/1024/1024 AS size_mb FROM v$log; -- e.g. 3 groups of 1024 MB
ALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 11 ('/oradata/PRODCDB/srl11.log') SIZE 1024M;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 12 ('/oradata/PRODCDB/srl12.log') SIZE 1024M;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 13 ('/oradata/PRODCDB/srl13.log') SIZE 1024M;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 14 ('/oradata/PRODCDB/srl14.log') SIZE 1024M;
SELECT thread#, group#, bytes/1024/1024 AS size_mb, status FROM v$standby_log;
Take a backup first. These are online changes, but a fresh RMAN backup before any Data Guard build is simple insurance.
✓ Done when: force_logging = YES, 4 SRLs of the same size as the online logs, and log_archive_config lists both names.
STEP 04
Configure Oracle Net on both serversBOTH
During the duplicate, the standby instance is only in NOMOUNT, so it cannot register with the listener. It needs a static entry. The second entry (_DGMGRL) lets the broker restart the standby during a switchover.
▸ STANDBY
SID_LIST_LISTENER =
(SID_LIST =
(SID_DESC =
(GLOBAL_DBNAME = PRODSTBY)
(ORACLE_HOME = /u01/app/oracle/product/19.0.0/dbhome_1)
(SID_NAME = PRODSTBY))
(SID_DESC =
(GLOBAL_DBNAME = PRODSTBY_DGMGRL)
(ORACLE_HOME = /u01/app/oracle/product/19.0.0/dbhome_1)
(SID_NAME = PRODSTBY)))
Add the matching PRODCDB_DGMGRL static entry on the primary's listener too. Then reload both listeners with lsnrctl reload.
Use the same tnsnames.ora entries on both servers:
▸ BOTH
PRODCDB =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = prodhost)(PORT = 1521))
(CONNECT_DATA = (SERVER = DEDICATED)(SERVICE_NAME = PRODCDB)(UR = A)))
PRODSTBY =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = stbyhost)(PORT = 1521))
(CONNECT_DATA = (SERVER = DEDICATED)(SERVICE_NAME = PRODSTBY)(UR = A)))
▸ BOTH
tnsping PRODCDB
tnsping PRODSTBY
✓ Done when: tnsping works in both directions from both servers, and lsnrctl status on the standby shows service PRODSTBY with status UNKNOWN (static).
STEP 05
Start the standby instance in NOMOUNTPRIMARYSTANDBY
Copy the password file from the primary. Redo transport authenticates with it, so it must be identical. Rename it for the standby SID:
▸ PRIMARY
scp $ORACLE_HOME/dbs/orapwPRODCDB oracle@stbyhost:/u01/app/oracle/product/19.0.0/dbhome_1/dbs/orapwPRODSTBY
The standby only needs a one-line pfile to start. RMAN builds the real SPFILE from the primary in the next step.
▸ STANDBY
export ORACLE_SID=PRODSTBY
echo "db_name='PRODCDB'" > $ORACLE_HOME/dbs/initPRODSTBY.ora
sqlplus / as sysdba
SQL> STARTUP NOMOUNT; -- picks up $ORACLE_HOME/dbs/initPRODSTBY.ora
Test the connection that RMAN will use. Don't put the password on the command line, where it can be seen in shell history and in ps output; let the tool prompt for it:
▸ PRIMARY
sqlplus sys@PRODSTBY as sysdba # enter the password when prompted; expect "Connected to an idle instance"
✓ Done when: you can connect as SYS to both PRODCDB and PRODSTBY over the network from the primary.
STEP 06
Duplicate the primary to the standby with RMANPRIMARY
FROM ACTIVE DATABASE copies the database straight over the network, with no backup files to stage. USING COMPRESSED BACKUPSET and SECTION SIZE let several channels split large datafiles, which is much faster on big databases.
▸ PRIMARY
-- /home/oracle/dg/dup_stby.rman
RUN {
ALLOCATE CHANNEL p1 TYPE DISK; ALLOCATE CHANNEL p2 TYPE DISK;
ALLOCATE CHANNEL p3 TYPE DISK; ALLOCATE CHANNEL p4 TYPE DISK;
ALLOCATE AUXILIARY CHANNEL s1 TYPE DISK; ALLOCATE AUXILIARY CHANNEL s2 TYPE DISK;
ALLOCATE AUXILIARY CHANNEL s3 TYPE DISK; ALLOCATE AUXILIARY CHANNEL s4 TYPE DISK;
DUPLICATE TARGET DATABASE FOR STANDBY FROM ACTIVE DATABASE
USING COMPRESSED BACKUPSET SECTION SIZE 16G
SPFILE
SET db_unique_name = 'PRODSTBY'
SET fal_server = 'PRODCDB'
SET log_archive_dest_2 = ''
SET audit_file_dest = '/u01/app/oracle/admin/PRODSTBY/adump'
SET local_listener = '(ADDRESS=(PROTOCOL=TCP)(HOST=stbyhost)(PORT=1521))'
-- SET db_file_name_convert = '/oradata/PRODCDB/','/data2/PRODCDB/' -- only if paths differ
-- SET log_file_name_convert = '/oradata/PRODCDB/','/data2/PRODCDB/'
NOFILENAMECHECK
DORECOVER;
}
Run it inside tmux or screen so a dropped SSH session cannot kill a long copy. RMAN prompts for both passwords:
▸ PRIMARY
tmux new -s dup
rman target sys@PRODCDB auxiliary sys@PRODSTBY log=/home/oracle/dg/dup_stby.log
RMAN> @/home/oracle/dg/dup_stby.rman
# from another session
tail -f /home/oracle/dg/dup_stby.log
Why NOFILENAMECHECK? Because the paths are identical on purpose. It tells RMAN not to stop when the standby datafile names match the primary's (they are on different servers).
✓ Done when: the log ends with Finished Duplicate Db and the standby is MOUNTED with database_role = PHYSICAL STANDBY.
STEP 07
Create the Data Guard Broker configurationBOTHDGMGRL
The broker manages redo transport and apply for you, and turns a switchover into one command.
▸ BOTH
ALTER SYSTEM SET dg_broker_start=TRUE SCOPE=BOTH;
-- The broker manages LOG_ARCHIVE_DEST_n itself; make sure no manual standby destination is left
ALTER SYSTEM SET log_archive_dest_2='' SCOPE=BOTH;
▸ DGMGRL
dgmgrl sys@PRODCDB
DGMGRL> CREATE CONFIGURATION prod_dg AS
PRIMARY DATABASE IS PRODCDB CONNECT IDENTIFIER IS PRODCDB;
DGMGRL> ADD DATABASE PRODSTBY AS CONNECT IDENTIFIER IS PRODSTBY MAINTAINED AS PHYSICAL;
DGMGRL> ENABLE CONFIGURATION;
DGMGRL> SHOW CONFIGURATION;
Configuration - prod_dg
Protection Mode: MaxPerformance
Members:
PRODCDB - Primary database
PRODSTBY - Physical standby database
Fast-Start Failover: Disabled
Configuration Status:
SUCCESS (status updated 12 seconds ago)
Not using the broker? Set redo transport and apply by hand instead: log_archive_dest_2='SERVICE=PRODSTBY ASYNC NOAFFIRM VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=PRODSTBY' on the primary, then ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT; on the standby. Do not mix both methods.
✓ Done when: SHOW CONFIGURATION returns SUCCESS. A WARNING right after enabling usually clears within a minute or two.
STEP 08
Verify redo transport and applyPRIMARYSTANDBYDGMGRL
▸ PRIMARY
ALTER SYSTEM ARCHIVE LOG CURRENT; -- force a log switch to test shipping
▸ STANDBY
SELECT name, value, time_computed FROM v$dataguard_stats
WHERE name IN ('transport lag','apply lag');
SELECT name, role, action, thread#, sequence#
FROM v$dataguard_process
ORDER BY name; -- look for MRP0 = APPLYING_LOG and an RFS receiving redo
SELECT thread#, MAX(sequence#) AS last_applied
FROM v$archived_log WHERE applied = 'YES' GROUP BY thread#;
▸ DGMGRL
DGMGRL> SHOW DATABASE PRODSTBY;
DGMGRL> VALIDATE DATABASE PRODSTBY;
✓ Done when: transport lag and apply lag are a few seconds, MRP0 is APPLYING_LOG, and VALIDATE DATABASE says Ready for Switchover: Yes.
STEP 09
Test a switchover (recommended)DGMGRL
A standby you have never switched to is a standby you cannot fully trust. In a maintenance window:
▸ DGMGRL
DGMGRL> VALIDATE DATABASE PRODSTBY;
DGMGRL> SWITCHOVER TO PRODSTBY; -- PRODSTBY becomes primary
DGMGRL> SHOW CONFIGURATION;
DGMGRL> SWITCHOVER TO PRODCDB; -- and back
✓ Done when: both switchovers complete and SHOW CONFIGURATION returns SUCCESS with PRODCDB as primary again.
STEP 10
Day-2 housekeepingPRIMARYSTANDBY
Do not delete archive logs before the standby has applied them:
▸ BOTH
RMAN> CONFIGURE ARCHIVELOG DELETION POLICY TO APPLIED ON ALL STANDBY; -- on the primary
RMAN> CONFIGURE ARCHIVELOG DELETION POLICY TO APPLIED ON STANDBY; -- on the standby
- Monitor transport and apply lag, plus
SHOW CONFIGURATION status, from your monitoring tool or a cron job.
- Size the FRA on the standby for at least a couple of days of archive logs.
- Opening the standby read-only while it applies redo needs the Active Data Guard licence. A mounted standby (as built here) does not.
- Patch both sides together, following Oracle's standby-first patching notes.
✓ Done when: the deletion policy is set on both sides and lag monitoring is in place.
Troubleshooting
| Error | Cause | Fix |
|---|
| ORA-12528 listener: all handlers blocked | Standby in NOMOUNT; dynamic registration only | Add the static SID_LIST entry; use (UR=A) in tnsnames |
| ORA-01017 / ORA-16191 invalid username/password, redo transport auth failed | Password files differ | Copy orapw from the primary again (rename for the standby SID) |
| ORA-16047 DGID mismatch | db_unique_name not in log_archive_config on one side | Same DG_CONFIG list on both; check db_unique_name |
| ORA-16057 server not in Data Guard configuration | log_archive_config missing or wrong on the standby | Set log_archive_config on the standby |
| ORA-16766 Redo Apply is stopped | MRP stopped (error, or someone stopped it) | Check the standby alert log, then EDIT DATABASE PRODSTBY SET STATE='APPLY-ON'; |
| ORA-16853 apply lag exceeded threshold | Gap, slow I/O, or network | SELECT * FROM v$archive_gap;, check FAL and the network |
| ORA-16698 member has LOG_ARCHIVE_DEST_n set | Manual destination left in place | Clear log_archive_dest_2 before ADD DATABASE |
First place to look: the alert log on both sides, and SHOW DATABASE VERBOSE PRODSTBY in DGMGRL, which lists the exact error for each member.
Summary
Same software → same paths → primary ready (force logging, SRLs, DG_CONFIG) → static listener and TNS on both sides → password file → NOMOUNT → RMAN active duplicate → broker → verify lag → test switchover. Following that order, and checking each step's "done when" before moving on, turns a Data Guard build into a routine job.
Questions about your own standby build? Ask in the comments.
Tamilselvan Perumal · Oracle ACE Apprentice · Lead Product Engineer & Senior DBA