I’m starting a new project to change the backup infrastructure for one of my customers. Currently the customer is using NetBackup 7.5 (installed on Windows) and we will incorporate the remote site in it’s centrally managed CommVault 10 environment to ease the handover to the operational teams and to efficiently use the capacity-based licenses.
To get a good understanding of the environment, including data sizings – we opted to execute the External Data Connector (EDC) Scripts for NetBackup 7.5 provided by CommVault Cloud Services.
Please note, you have the ability to install an “External Data Connector Agent” on one of your clients and use that one to feed the data from the NetBackup environment. However, I’m not in favour for this approach as it will pollute the CommServe Database. This blogpost describes how you can use the Cloud Applications of CommVault to stage the existing configuration without thrashing the production environment.
Recently one of our development servers crashed. Upon recovery we started to receive the following error messages when backing up the Microsoft SQL databases: “Unable to get the SQL Version for the server [DEVSERVER]. Please check if the SQL Services are running.”
It’s important to note:
- Before the crash the backups ran fine;
- the file “SQLServerInstanceList.txt” listed the following information: “(local);Clustered:No;Version:11.0.5058.0“;
- the SQLiDA log only mentions “Unable to get the SQL Version for the server“;
- the server has been restarted and the SQL iDataAgent has been reconfigured (removed keeping software in place and reconfigured).
We will start with a oneliner: “the cloud will be everything in the nearby future!“.
Enterprises are discovering the cloud and will slowly evolve to a hybrid environment. Many of them already took the first step with Office 365, OneDrive for Business and so on. As they are moving towards services and shared infrastructure, the need for data protection will remain untouched. However according to an EMC study (Reaching Solid Ground by Chris Ratcliffe) executed in December 2014, we notice only 13% of the organizations are confident to protect data stored in various locations. In one of my previous blog posts I mentioned we will see backup silo’s returning again. I strongly believe besides hyperconvergence (Software-Defined DataCenter), we will notice software and/or management console convergence for protecting virtual machines and data in the cloud, as on-premise.
Motivated by the above reasons, I looked into the CommVault Simpana Azure client which is available as of version 10 Service Pack 9 and is currently available as “Early Release“. My findings can be found at the end of this blogpost.
I recently started using Microsoft Azure to build my own personal playground. In the past I used to deploy everything on my desktop PC at home and honestly my resources were limiting me to do what I wanted. I thought about buying a server, but I was not really willing to do these kind of investments.
For other tech people out there, Microsoft has a “30 day/150 euro” free trial available. Go check it out!
One of the things I really wanted to do is test some things regarding ‘portability‘ from a backup perspective. My intention is to recover on premise machines to the public cloud (Microsoft Azure in this test case) by using backup & restore, migrate and synchronisation.
I was going to deploy a standard virtual machine and install everything manually, as it came to my attention a predefined CommVault Simpana Software appliance (version 10 SP8) was available in the Microsoft Azure marketplace. I decided to deploy this template, as the system will solely be used in a sandbox environment.
The configuration of a virtual network, storage account, resource group and any other shared services (such as Active Directory and DNS) are out of scope of this article.
The main reasons to perform a manual or custom install on a regular virtual machine are:
- The CommVault Simpana Software appliance uses a Microsoft SQL Server 2012 SP2 Express edition (version: 11.2.5058.0). The express edition is limited to a maximum of 1GB memory allocation for the database engine, a maximum size of 10GB per database and up to four cores or one socket.
- The CommVault Simpana SQL instance is locked down with only the sa-account (role = “sysadmin“) and the BUILTIN\Users (role = “public“). So in case you want full blown access onto the database, you need to request CommVault to unlock it or start hacking your way in.
- You want to change the installation path for the CommVault Simpana software. By default everything is installed onto the C-drive.
- Both versions support database mirroring which can be used as a disaster recovery mechanism for the CommVault Simpana backup environment.
- Update 3-MAY-2015: The commVault Operations Manager is not installed by default on the virtual machine. The Operations Manager allows some advanced features such as “Virtualize Me!” and “VM lifecycle management“. Manual installation is possible by using the Software Cache SetupAll.exe.
- Update 3-MAY-2015: the webconsole (http://localhost/webconsole) configuration still points to the template virtual machine hostname (“csmaexpress”). To get it to work, you need to change the following registry keys:
- “HKLM\Software\CommVault Systems\Galaxy\Instance01\Webconsole\ sZDM2WEBSERVERHOSTNAME“
- “HKLM\Software\CommVault Systems\Galaxy\Instance01\CustomReportsEngine\ sZCRENGINEWEBSERVERHOSTNAME”
- “HKLM\Software\CommVault Systems\Galaxy\Instance01\JobInfo\ sWebConsoleUrl“
to include the right hostname. Additionally also alter the webconsole configuration file “C:\Program Files\CommVault\Simpana\WebConsole\WEB-INF\classes\config.properties“.
Please note, CommVault does not allow the Simpana databases to be stored on a consolidated server. These need to be stored on the local system for best performance, business continuity (“chicken or the egg principle“) and disaster recovery tolerance.
Some time ago, I was confronted with some media related problems. The problem originated from tape device persistency not in place on various Linux systems resulting in tape drives being detected on other SCSI addresses.. resulting in hardware failures because the serial number “changed“.
To make sure we were able to map everything in the right order, we had to detect the serial numbers from an operating system level as we were not able to trust the info we saw in the CommVault Simpana Administrative Console. Luckily for us CommVault provides some tools on-the-box to assist you in the task. These tools are available on Linux and Windows Operating Systems.
- The tool ScanScsiTool is used to list the SCSI ports, WWNs, serial numbers and model/type of the tape different connected tape drives. Additionally the tool allows you to list the different disk drives, libraries and fiber channel cards as well. Basically this tool provides some very usefull information during a troubleshooting session and you want to collect the information directly from the hardware/OS. For more information click here.
- Another very usefull tool is ARMTool that gives you the ability to collect the tape labels, do some robotic operations (move tape, …) from the operating system level. Please note these operations are not recommended as CommVault is not aware of the changes performed (a manual rescan within Simpana is required). This implies as well that this tool cannot be executed when backups are running. For more information click here.
I have two CommCells which are accessed by about 20 persons. For the ease of access I activated Single SignOn within the configuration. The Single SignOn can be activated in the CommVault Simpana Administration Console in the section “Security” (“CommCell > Security > Name Servers > Domain > Properties”). Check or uncheck the box to activa/deactivate Single SignOn.
We ran into the following error during the migration from Oracle backups taken with HP Data Protection (version 6.20) towards CommVault Simpana (version 10 service pack 8).
The first database backup executed with CommVault Simpana 10 throws the following error message. Please note the filesystem backups are perfectly conducted.
“Description: RMAN Script execution failed with error [ORA-19554: error allocating device, device type: SBT_TAPE, device name: ]. Please check the Logs for more details.
Source: server, Process: ClOraAgent”
I got called this weekend to look at a CommVault restore of a Linux FileSystem which goes in pending modus with the error message mentioned below. This problem has been encountered on CommVault Simpana 9 Service Pack 14.
“Error Code: [32:254] Description: Read operation from media [tapelabel] could not be completed. The reason: Read operation cannot be completed as none of the resources available can be used. Please check the following: 1. There are drives with compatible recording format to the media being read. 2. There are resources (Drives, Libraries, MediaAgents, etc.) that are online and ready to be used. If you still have an issue, please contact customer support.”
Recently I performed a Disaster Recovery test of the CommVault Simpana 10 software. The Disaster Recovery test has been executed by using the SQL Mirroring functionality as described in this procedure on the CommVault website.
My environment is configured as follows:
- An active CommServe installed on server1 as instance1;
- An standby CommServe installed (services are stopped) on server 2 as instance1;
- A filesystem agent has been deployed on the StandBy CommServe as instance2. The agent is used to protect data locally by using the software installed on server1.
Yesterday they asked me to take a closer look at an error they were receiving during a restore operation. To be honest it took me some time to figure this one out. Mainly because the error description was not very clear, nor to the point.
The error happened on a CommVault Simpana 9 Service Pack 14 environment during a restore of an Oracle database.
The error thrown by the restore job indicated an unknown platform error occured: “Error Code: [62:506] – Description: Failed to mount media with barcode [BarcodeLabel], side [A_33692], into drive [myTapeDrive], in library [myTapeLibrary] on MediaAgent [myMediaAgent]. SCSI Operation: Move Media From Slot To Drive. Reason: Unknown platform error. Advice: Please upload log files from CommServe, MediaAgent and Client and contact your vendor’s support hotline with the job ID of the failed job.”