A step-by-step guide to configure NetApp CDOT NDMP in combination with CommVault Simpana 10

There are some differences in setting up the CommVault Simpana NDMP iDataAgent in combination with NetApp Clustered Data OnTap (CDOT) compared to a 7-mode filer.

This post explains the configuration process to start backing up snapshots stored on a SnapVault NetApp running CDOT by using two-way NDMP.

In case you are using three-way NDMP to protect the data stored on the filer, some of the steps are not required. I will list those throughout the procedure.

If you need a better understanding about two-way NDMP vs. three-way NDMP, please refer to my blogpost called: “Lingo Explained: 2-way NDMP vs. 3-way NDMP”.

Before you start, ensure the CommVault Simpana software is running on a supported version. My environment is running CommVault Simpana 10 Service Pack 12 when I performed this integration. More information about the CommVault configuration prerequisites and – requirements can be found on CommVault BOL. The configuration process of NDMP on NetApp CDOT is outlined in a document called “Clustered Data OnTap 8.3 NDMP Configuration Express Guide“.

GENERAL RECOMMENDATIONS:

  1. Try to use controller-based protocols (such as CIFS & NFS) as much as possible. The filer is able to see the contents in these folders, which results in an increased flexibility in the restore process (compared to FC – or iSCSI LUN emulation).
  2. Create one NDMP subclient per volume instead of using one subclient containing all volumes. By splitting a higher level of performance can be achieved as data restores require less tapes to be mounted resulting in an increased recovery (limitation of the way NDMP works).
  3. NDMP backup data cannot be used in combination with Content Indexing (End User Search). If this is a business requirement, consider to use a pseudo client.
  4. Data archiving and stubbing is not supported on NDMP subclients. If this is a business requirement, consider to use a pseudo client.

PREREQUISITES:

  1. Before executing the steps outlined in this procedure, verify if all required DNS-entries are created.
  2. In case you are using 2-way NDMP, make sure the tape devices are presented to all controllers within the cluster before executing the procedure outlined below.

Continue reading

Lingo Explained: tape shoe-shining (and how to avoid?)

Shoe-shining or backhitching – a term used in backup context – is a repeated back-and-forth-motion a tape device makes when the data flow is interrupted or is too slow. The process itself is destructive for the tape device, as well as for the tape cartridge.

A tape cartridge is a sequential medium requiring a continuous flow of data to keep the internal band in motion. When the data stream gets interrupted or the backup server is sending data slower than the tape drive processor, the tape device is obligated to stop. The stop  causes an offset of the writer and when resuming the device is required to position the writer back to the spot containing the blocks of the last write operation. The process: stop (end-of-data-stream), reposition (finding the right spot again), write (continue data stream) and repeat is what is called shoe-shining.

Continue reading

CommVault Oracle Database backup fails with RMAN error: ORA-27211: Failed to load Media Management Library

We ran into the following error during the migration from Oracle backups taken with HP Data Protection (version 6.20) towards CommVault Simpana (version 10 service pack 8).

The first database backup executed with CommVault Simpana 10 throws the following error message. Please note the filesystem backups are perfectly conducted.
Description: RMAN Script execution failed with error [ORA-19554: error allocating device, device type: SBT_TAPE, device name: ]. Please check the Logs for more details.
Source: server, Process: ClOraAgent”

Continue reading

Creating the Enterprise Vault backup scripts

In most cases I see people re-using the backupscripts they once created for another customer. Not a lot of people are aware there is a powershell script available in the installation directory of Enterprise Vault that will do the task for you.

In the Templates directory in the Enterprise Vault installation directory (in my case E:\Program Files (x86)\Enterprise Vault\Reports\Templates\) you can find a script which is called Transform-Backup.ps1.

Continue reading

Data Protector backup job throws “Fallback to legacy filesystem backup was not allowed”

Wheb backing-up a Windows Client, sometimes the error “Fallback to legacy filesystem backup was not allowed” occurs. In most cases this is related to the CONFIGURATION object, but recently I noticed this also on the D-drive of a (SQL) server.

[Critical] From: VBDA@client “fqdnoftheclient [/D]” Time: 3/25/2014 8:16:09 AM
Fallback to legacy filesystem backup was not allowed. Aborting the backup.
[Critical] From: VBDA@client “fqdnoftheclient [/D]” Time: 3/25/2014 8:16:09 AM
[81:52] /D Not a valid mount point => aborting.

The software tries to start a backup by using the VSS integration and when this fails it fallbacks to the legacy method. If this is not defined in the backupjob properties, the backupjob will fail with the error mentioned above.

Continue reading

CommVault Oracle backup fails with error code 62:342

Out-of-the-blue we encountered that two of our Oracle databases started failing with the error code “62:342″ and description “Encountered an I/O error while performing the operation”. The odd thing is that we have more than 15 databases on the server and only two of them were having the issue.

Important to note is the fact that the server itself is a MediaAgent, which means the server has access to a shared tape device (in our case a VTL). The VTL itself is shared with approximately 90 MediaAgents and the storage policy used for database backup is associated with more than 250 databases. And… only 2 were failing…

Continue reading

Preserve tape media label & description in Backup Exec 2012

If you want to use the description field of a tape in Backup Exec to identify the backup media set. You will notice once tape is rewritten or formatted the description is lost.

These fields can come in handy for example when you are trying to identify tape 1 as week 1 or as month January.

To change this behaviour, follow this procedure:

  1. Use regedit to locate the following key HKEY_LOCAL_MACHINE\SOFTWARE\Symantec\Backup Exec for Windows\Adamm.
  2. within the ADAMM key create a DWORD value called: PreserveMediaDescriptions.
  3. Set the value to 1 to maintain the media descriptions during overwrites.
  4. Restart the Backup Exec services.

More information can be found in this technote.

Abort an unresponsive Data Protector job

Sometimes aborting a job by using the Data Protector Manager Console does not succeed.
In those situation, follow this procedure:

  • Go to the Data Protector Manager console > Monitor > Session-Id and take a not of the start time.
    [Normal] From: BSM@CellManager.lab.local “FILE_DAILY” Time: 5/23/2013 4:15:05 AM
    Backup session 2013/05/23-13 started.
  • Open a PowerShell Window and execute the following command: “get-process b* | sort-object StartTime | format-table Name,Id,StartTime”
    Name Id StartTime
    —- — ———
    bsm 8532 18/05/2013 4:15:07
    bsm 8996 19/05/2013 5:15:05
    bsm 11672 23/05/2013 4:15:05
  • Execute the following command to kill the process: “stop-process 8532“.

Execute the command “omnistat” or “get-process b* | sort-object StartTime | format-table Name,Id,StartTime” to verify the operation. Restart the Data Protector Management console to clear the monitor in the GUI.

Registry change after installing Backup Exec 2010/2012 on a Hyper-V 2008R2 node

Installation of Symantec Backup Exec 2012 is fully supported on Windows 2008R2 Hyper-V nodes as described in the Software Compatibility List. However after rebooting the node, Hyper-V throws the following errors. Important to note is that the cluster itself is online without any warnings or errors as described in this post. The errors are visible after live migrating a virtual machine to the impacted target host or when you attempt to start a virtual machine on the impacted node.

‘VM Name’ failed to start.

Microsoft Emulated IDE Controller (Instance ID {########-####-####-####-############}): Failed to Power on with Error: ‘A device attached to the system is not functioning.’

Failed to open attachment ‘Drive Letter:\path\Virtual Hard drivers\VMNAME_########-####-####-####-############.vhd’. Error: ‘A device attached to the system is not functioning’

Failed to open attachment ‘Drive Letter:\path\Virtual Hard drivers\VMNAME_########-####-####-####-############.vhd’. Error: ‘A device attached to the system is not functioning’

and

‘VM Name’’ Microsoft Emulated IDE Controller (Instance ID {########-####-####-####-############}): Failed to Power on with Error: ‘A device attached to the system is not functioning.’ (0x8007001F) (Virtual Machine ID: ########-####-####-####-############)

Some research resulted in the following conclusion:

  1. In the Kernel Mode the following components exist:
    • Fsdepends.sys: File System Dependency Manager Mini Filter Driver
    • Vhdmp.sys: VHD Miniport Driver
    • vdrvroot.sys: Virtual Drive Root Enumerator
  2. The VirtDisk.dll uses the above mentioned components for starting the VDS VHD API’s.
  3. And those are used for the Virtual Disk Service which is configured with a manual startup type…

.
Continue reading

Backup, Restore and/or Migrate HP Data Protector’s Internal Database

There are two ways allowing a backup and restore of the Data Protector Internal Database. The procedures described below can also be used for performing migrations. The internal database is located in the DB40 folder within the installation directory.

The first way is relatively easy: create a backup job in the Data Protector Manager Console and select the Internal Database in the selection tree. The downside of this method is the lack of control where the database is stored (on a tape, but which one? on disk, in a bunch of files!) and how fast can it be restored in case of disaster. If you did not implement any fallback mechanisms, a catalog operation will be required! Needless to say it will take some time to complete.

Dumping the data to disk can come in very handy as a safety copy during maintenance operation such as upgrades, patchings, etc.

Continue reading