With ZDLRA you can clone your backups to tape using two ways. The first is using third-party software and the second is using Oracle Secure Backup (OSB). This integration from ZDLRA and OSB is the native way to do that.

Cloning backups to tape will help to offload backups from ZDLRA (reducing the space usage if you need to sustain long recovery windows), and add another layer of protection (since you can put tapes in a third site).

Here I will show how easy is to configure the OSB backup and how to integrate it into your backup policy.

TAPE and ZDLRA

Tape backups are recommended for long-term storage, going to years and decades of storage. For ZDLRA itself, when using OSB, it turns itself as Media Manager too. This means that all the operations are realized by ZDLRA, tape load, rewind, cleans. Everything.

Using OSB as a tape library for ZDLRA, the communication with tape is done trough SAN, with direct access. This means that every database node of ZDLRA have SAN card, and you can connect to tape directly, or using SAN switch. All the cards have two links (4 as total) and you can configure as failover access (in each node).

All the tasks for tape are done with OSB (load, unload, overwrite), so, we can use commands like obtool lsdev, obtool lsvol and others to check the library and tapes.

Oracle Secure Backup

Since OSB is external and exists outside the ZDLRA world there are some details that need be explained. As explained in previous posts (ZDLRA Internals, Virtual Full Backup), ZDLRA deconstructs the ingested backup and this database internal tables are not compatible with OSB. OSB not connect inside of the ZDLRA database and copy the backups.

So (copied directly from documentation), when Recovery Appliance executes a copy-to-tape job for a virtual full backup, it constructs the physical backup sets, and copies them to tape, and then writes the metadata to the recovery catalog. If desired, the Recovery Appliance can also copy successive incremental backups and archived redo log file backups to tape. Whereas the backup on the Recovery Appliance is virtual, the backup on tape is a non-virtual, full physical backup. The Recovery Appliance automatically handles requests to restore backups from tape, with no need for administrator intervention.

The info above is clear, but basically this means that ZDLRA constructs a new backupset and sends it to OSB to copy to tape. If you are sending a full backup to tape (and your backup has 14TB as an example) be aware of the time consumption to do that. Another point is that ZDLRA will automatically update the rman catalog for your database when the copy is finished at the tape, the backupset will appears inside the rman catalog.

One important point here it is there OSB manages totally the tape communication. If you receive one backupset to clone, you can’t go to ZDLRA and pause the copy for this backupset (you can pause the job that triggered the copy inside ZDLRA, but no the copy that already started). Consider that OSB as external software, where you send something to copy and the result will be OK or ERROR.

We can use some commands using obtool, but since it is integrated/managed by ZDLRA, it is not allowed to change the configuration. Commands like lsjob, lsdev, lsinventory, unlabelvol can be used.

Be careful when managing tapes because their communication is only from ZDLRA to OSB. If you clean one tape at OSB side, ZDLRA will not be aware of that and (if you have some not expired backupset there) you can lose some data.

Third-party Libraries

If you are not using OSB you can use external/third-party libraries to offload to tape. In this case, it needs to be compatible with RMAN. Basically, when you create the library inside ZDLRA, you point to the operational system client library (installed in each database node) and follow the normal procedure as OSB.

Be aware that you need to allow in your third-party software, that the client can send the backups to do the server. The start of the clone always will start at the ZDLRA side (and not called by third-party software).

CREATE_SBT_LIBRARY

In this post I will use OSB as a clone library, so, some details will be relevant just for OSB. But the process for creating the library will be similar (just adjust some parameters).

The first step is to create the library using the DBMS_RA.CREATE_SBT_LIBRARY:

 

SQL> BEGIN
  2      DBMS_RA.CREATE_SBT_LIBRARY(
  3          lib_name => 'osbsbt',
  4          drives => 4,
  5          restore_drives => 1,
  6          parms => 'SBT_LIBRARY=libobk.so'
  7      );
  8  END;
  9  /

PL/SQL procedure successfully completed.

SQL>

The parameters are:

• lib_name: Name that identifies the library inside ZDRLA. It is unique name
• drivers: Number of drivers that are available for backup usage
• restore_drivers: Number of the driver used for restore. Will be always a subset of the drivers If not specified, ZDLRA can use all drivers to do the restore.
• parms: here you can specify some parameters that are passed when to call the library.

Remember that the tape library needs to be compatible with RMAN? So, the parms and send attributes for the procedure are used as parameters for ALLOCATE CHANNEL when ZDLRA calls the tape library. If you need to define some specific parameters (like tapes groups, client names, or whatever), you need to define it here. This follows the same format as PARMS in rman ALLOCATE CHANNEL.

After that, you can create attributes for your tape using the CREATE_SBT_ATTRIBUTE_SET. These attributes are important if you want to segregate some backups. As an example, you can create one attribute to send some backups for specific media families. So, if you have tapes to store backups after some application (or company milestones) you can create specific attributes.

SQL> BEGIN
  2      DBMS_RA.CREATE_SBT_ATTRIBUTE_SET(
  3          lib_name => 'osbsbt',
  4          attribute_set_name => 'bronzerepfull',
  5          streams => 4,
  6          parms => 'ENV=(OB_MEDIA_FAMILY=zdlras1-osbmf)'
  7      );
  8  END;
  9  /

PL/SQL procedure successfully completed.

SQL>

Here, I defined that the attribute bronzerepfull for my library osbsbt. That means (for OSB) that will use the media family zdlras1-osbmf to store the backups. Again, the parms and send use the format for rman PARMS at ALLOCATE CHANNEL in rman.

One detail important it is that streams attribute needs to be less or equal than drivers value that you specified during CREATE_SBT_LIBRARY commands.

If you check the RASYS.RA_SBT_LIBRARY table the library is there:

SQL> col LIB_NAME format a30
SQL> col LAST_ERROR_TEXT format a30
SQL> SELECT LIB_NAME, LAST_ERROR_TEXT, STATUS FROM RA_SBT_LIBRARY;

LIB_NAME                       LAST_ERROR_TEXT                STATU
------------------------------ ------------------------------ -----
OSBSBT                                                        READY
REP$LIB_9043_ZDLRAS2_REP                                      READY

SQL>

if you are starting the ZDLRA and it already will come with OSB integration the steps above are not needed. Will be configured directly during the deployment phase of ZDLRA.

Clone to Tape Jobs

The clone to tape jobs inside of ZDLRA is managed entirely with DBMS_SCHEDULER and for protection policies or db_unique_name as a parameter. This means that unfortunately, you need to take care of the overlaps of the clone to tape jobs.

If, as an example, you defined the clone for BRONZE policies to start from 10:00 until 11:00 and the SILVER to start at 11:00. And your backup for BRONZE took more time to execute and pass over 11:00, you will see some contentions.

DBMS_RA.CREATE_SBT_JOB_TEMPLATE

The first step to clone to tape is to define what and how to do that. For ZDLRA we can clone full, incremental, and archivelogs backups. Besides that, we can define the templates for protection policies or database unique names.

SQL> BEGIN
  2      DBMS_RA.CREATE_SBT_JOB_TEMPLATE (
  3          template_name => 'BRONZE_FULL_ARCH',
  4          protection_policy_name => 'ZDLRA_BRONZE',
  5          attribute_set_name => 'bronzerepfull',
  6          backup_type => 'FULL,ARCH',
  7          priority => DBMS_RA.SBT_PRIORITY_LOW,
  8          window => INTERVAL '4' HOUR
  9      );
 10  END;
 11  /

PL/SQL procedure successfully completed.

SQL> BEGIN
  2      DBMS_RA.CREATE_SBT_JOB_TEMPLATE (
  3          template_name => 'BRONZE_ARCH',
  4          protection_policy_name => 'ZDLRA_BRONZE',
  5          attribute_set_name => 'bronzerepfull',
  6          backup_type => 'ARCH',
  7          priority => DBMS_RA.SBT_PRIORITY_LOW,
  8          window => INTERVAL '4' HOUR
  9      );
 10  END;
 11  /

PL/SQL procedure successfully completed.

SQL>

Basically, with CREATE_SBT_JOB_TEMPLATE, we define what we want to backup (database or policy) or which type of backup we want (full, archive).

Another important definition is the priority between the templates. So, in case your backup took more time than expected, the priority helps ZDLRA to automatically handle the overlaps of clones times. You can use the window to specify the maximum time that clone can runs, after that the backupset will put in the queue to be copied in the next run.

In the example above you can see that:

• BRONZE_FULL_ARCH: Template that defines that FULL and ARCHIVELOGS backups will be copied at the same time to tape. We will use the SBT attribute bronzerepfull that we defined previously. And have 4 hours to maximum runs and have LOW priority.
• BRONZE_ARCH: Template that defines that just ARCHIVELOGS backups will be copied at the same time to tape. We will use the SBT attribute bronzerepfull that we defined previously. And have 4 hours to maximum runs and have LOW priority.

Important to remember that ZDLRA always copies the last backupset available. This means that for a full backup, it will be always the last that was not copied since the last execution of your job. For archivelogs, it will be from the last clone to tape (even if the last copied just the archivelogs). One example is that you execute every day one copy per day for your database at 22:00, but you do two incremental backups (one 11:00 and other 19:00), just the full backup generated because of the incremental backup at 19:00 will be cloned to tape.

For archivelogs, the rule is copy all since the last clone to tape. This guarantees that you always have the archivelogs needed to recover the database. One point that needs attention is when using real-time redo, in this case, each backup of archivelog is unique (different than we execute filesperset 1000 – which will have one backupset with 1000 archivelogs inside). This means that if you generate 2000 archivelogs per day, you will need to clone these files one to one, and for tape (depending the tape type), can took long time because will be (inside tape) 2000 different files (think about the process for tape, where each file need a mark where start and end – this took a lot of time consumption for tape process).

It is possible to create a template with TAG’s too. In this case, just backups with specific TAGS (from rman) will be cloned to tape. This is useful when you need to clone specific backups for long retention schemas.

DBMS_SCHEDULER.CREATE_JOB

After creating the templates it is possible to schedule it using the database scheduler. Is used the normal scheduler, setting the desired template and the desired time that will be called.

As an example:

SQL> BEGIN
  2      DBMS_SCHEDULER.CREATE_JOB(
  3          job_name => 'SBTJOB_BRONZE_FULL_ARCH',
  4          job_type => 'PLSQL_BLOCK',
  5          job_action => 'dbms_ra.queue_sbt_backup_task(''BRONZE_FULL_ARCH'');',
  6          start_date => SYSDATE+(1/1440),
  7          enabled => TRUE,
  8          auto_drop => TRUE,
  9          repeat_interval => 'freq=WEEKLY; BYDAY=SUN; BYHOUR=20'
 10      );
 11  END;
 12  /

PL/SQL procedure successfully completed.

SQL>

We can see that it is a normal schedule that calls one PLSQL_BLOCK, and the block it is dbms_ra.queue_sbt_backup_task that will inform the internal ZDLRA library to generate the backupset and send to OSB. As you can see, the parameter for the function is the template name created before.

The start_date and repeat_internal are defined as is for the normal scheduler. In this particular example, the backup will be every week, Sunday at 20:00.

And translating everything this means that I will have, in tape, one full backup (plus archivelog) for each database that it is part of BRONZE protection policy. And this will occurs every Sunday at 20:00. The library will be OSB and the media family to store the backups will be zdlras1-osbmf.

Another example:

SQL> BEGIN
  2      DBMS_SCHEDULER.CREATE_JOB(
  3          job_name => 'SBTJOB_BRONZE_ARCH',
  4          job_type => 'PLSQL_BLOCK',
  5          job_action => 'dbms_ra.queue_sbt_backup_task(''BRONZE_ARCH'');',
  6          start_date => SYSDATE+(1/1440),
  7          enabled => TRUE,
  8          auto_drop => TRUE,
  9          repeat_interval => 'freq=HOURLY; BYMINUTE=40'
 10      );
 11  END;
 12  /

PL/SQL procedure successfully completed.

SQL>

And some examples of repeat intervals:

• repeat_interval => ‘freq=HOURLY; BYMINUTE=0’: Every hour as *:00
• repeat_interval => ‘FREQ=DAILY; INTERNAL=1; BYHOUR=20; BYMINUTE=15’: Every day at 20:15
• repeat_interval => ‘FREQ=DAILY; BYDAY=MON,TUE,WED,THU,FRI; BYHOUR=7,8,9,10,11,12,13,14,15,16,17,18; BYMINUTE=05’: Monday to Friday, from 08-18 at 05 minute.
• repeat_interval => ‘freq=DAILY; BYHOUR=22; BYDAY=MON,TUE,WED,THU,FRI,SAT,SUN’: Another way to specify every day at 22:00
• repeat_interval => ‘freq=WEEKLY; BYDAY=SAT; BYHOUR=23’: Every week at 20:00

Cloned Backups

After configure the scheduler, and it is called we can see this in rman:

RMAN> list backup of archivelog sequence 92;


List of Backup Sets
===================


BS Key  Size
------- ----------
10082   2.80M

  List of Archived Logs in backup set 10082
  Thrd Seq     Low SCN    Low Time            Next SCN   Next Time
  ---- ------- ---------- ------------------- ---------- ---------
  1    92      2520099    01/01/2020 22:31:51 2530992    01/01/2020 23:29:08

  Backup Set Copy #1 of backup set 10082
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    00:43:14     01/01/2020 23:29:24 YES        TAG20200101T232923

    List of Backup Pieces for backup set 10082 Copy #1
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    10083   1   AVAILABLE                           $RSCN_1920977_RTIM_1028557385_THRD_1_SEQ_92_CTKEY_10033_BACKUP

  Backup Set Copy #2 of backup set 10082
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    00:43:14     01/01/2020 22:46:09 YES        TAG20200101T232923

    List of Backup Pieces for backup set 10082 Copy #2
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    10338   1   AVAILABLE   zdlras2-osbmf-000003    RA_SBT_OR19DG_41954437_10086_66uku2ij_1_2_10082

RMAN>

And for datafile

RMAN> list backup of datafile 1;


List of Backup Sets
===================


BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ -------------------
9067    Incr 0  337.52M    SBT_TAPE    00:00:57     01/01/2020 19:07:12
        BP Key: 9068   Status: AVAILABLE  Compressed: YES  Tag: BKP-DB-INC0
        Handle: VB$_2127575003_9062I   Media:
  List of Datafiles in backup set 9067
  File LV Type Ckp SCN    Ckp Time            Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- ------------------- ----------- ------ ----
  1    0  Incr 2438840    01/01/2020 19:03:19 2440207      NO    +DATA/OR19DGS/DATAFILE/system.265.1028562437

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ -------------------
9210    Incr 1  136.00K    SBT_TAPE    00:00:10     01/01/2020 19:16:43
        BP Key: 9211   Status: AVAILABLE  Compressed: YES  Tag: BKP-DB
        Handle: VB$_2127575003_9206I   Media:
  List of Datafiles in backup set 9210
  File LV Type Ckp SCN    Ckp Time            Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- ------------------- ----------- ------ ----
  1    1  Incr 2441418    01/01/2020 19:15:20              NO    +DATA/OR19DGS/DATAFILE/system.265.1028562437

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ -------------------
9218    Incr 0  336.36M    SBT_TAPE    00:00:10     01/01/2020 19:16:43
        BP Key: 9219   Status: AVAILABLE  Compressed: YES  Tag: BKP-DB
        Handle: VB$_2127575003_9206_1   Media:
  List of Datafiles in backup set 9218
  File LV Type Ckp SCN    Ckp Time            Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- ------------------- ----------- ------ ----
  1    0  Incr 2441418    01/01/2020 19:15:20              NO    +DATA/OR19DGS/DATAFILE/system.265.1028562437

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ -------------------
9496    Incr 1  168.00K    SBT_TAPE    00:00:10     01/01/2020 20:26:41
        BP Key: 9497   Status: AVAILABLE  Compressed: YES  Tag: BKP-DB
        Handle: VB$_2127575003_9492I   Media:
  List of Datafiles in backup set 9496
  File LV Type Ckp SCN    Ckp Time            Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- ------------------- ----------- ------ ----
  1    1  Incr 2452089    01/01/2020 20:24:22              NO    +DATA/OR19DGS/DATAFILE/system.265.1028562437

BS Key  Type LV Size
------- ---- -- ----------
9500    Incr 0  336.36M
  List of Datafiles in backup set 9500
  File LV Type Ckp SCN    Ckp Time            Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- ------------------- ----------- ------ ----
  1    0  Incr 2452089    01/01/2020 20:24:22              NO    +DATA/OR19DGS/DATAFILE/system.265.1028562437

  Backup Set Copy #1 of backup set 9500
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    02:14:00     01/01/2020 20:26:41 YES        BKP-DB

    List of Backup Pieces for backup set 9500 Copy #1
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    9501    1   AVAILABLE                           VB$_2127575003_9492_1

  Backup Set Copy #2 of backup set 9500
  Device Type Elapsed Time Completion Time     Compressed Tag
  ----------- ------------ ------------------- ---------- ---
  SBT_TAPE    02:14:00     01/01/2020 22:40:31 NO         BKP-DB

    List of Backup Pieces for backup set 9500 Copy #2
    BP Key  Pc# Status      Media                   Piece Name
    ------- --- ----------- ----------------------- ----------
    10125   1   AVAILABLE   zdlras2-osbmf-000002,zdlras2-osbmf-000005 RA_SBT_OR19DG_41954437_10086_qjuktnrn_1_2_9500

RMAN>

For both cases look that we have the Backup Set Copy #2 in tape. And specifically for datafile, look that just the last full backup was cloned to tape.

These are just examples, in other posts, I will provide more examples about clone to tape.

Conclusion

Configurate ZDLRA to do clone to tapes and offload the backup is not complicated. But will require some attention because of the details. The most important is related to the job scheduler. It is DBA duties correctly to schedule it and avoids some overlap between the jobs.

Unfortunately, there is no easy way to do that. And it can be more complicated because you can add more database to your protection policies and this will increase the clone time.

Besides that, we have the fact that OSB (or third-part library) are handled as “external” by ZDLRA. This means that there is not much control beside start and stop coming from ZDLRA. No easy error identification, report and solution besides “try again”.

Clone to tape backup is one of activity that demands more attention for ZDLRA and needs to take care of a lot of details to have a good clone to tape plan that will lead to fewer errors in the future.

 

Disclaimer: “The postings on this site are my own and don’t necessarily represent my actual employer positions, strategies or opinions. The information here was edited to be useful for general purpose, specific data and identifications were removed to allow reach the generic audience and to be useful for the community. Post protected by copyright.”