7 Linux/Unix Instances Backup
Making an application-consistent backup of Linux instances does not require any agent installation. Since the N2WS server is Linux based, backup scripts will run on it. Typically, such a script would use SSH to connect to the backed-up instance and perform application quiescence. However, this can also be accomplished using custom client software.
There is no parallel to VSS in Linux, so the only method available is by running backup scripts.
To create, test, and install backup scripts, you will need to connect to the N2WS server using SSH with
cpmuser. The only way to authenticate
cpmuseris by using the private key from the key pair you used when you launched the N2WS server instance. As long as your key is not compromised, no unauthorized person will be able to connect to the N2WS server.
cpmuser, you will be able to copy (using secure copy), create, and test your scripts.
cpmuseris the same user N2WS will use to run the scripts. If you need to edit your scripts on the N2WS Server, you can use Vim or nano editors. Nano is simpler to use.
Backup scripts should be placed in the path
/cpmdata/scripts. If the policy belongs to an N2WS user other than the root user, scripts will be located in a subfolder named like the user (e.g.,
/cpmdata/scripts/cpm_user1). This path resides on the CPM data volume and will remain there even if you terminate the N2WS server instance and wish to launch a new one. Backup scripts will remain on the data volume, together with all other configuration data. As
cpmuser, you have read, write, and execute permissions in this folder.
- All scripts should exit with the code zero
0when they succeed and one
1(or another non-zero code) when they fail.
- All scripts have a base name (detailed for each script in the coming sections) and may have any addition after the base name. The delimiter between the base part of the name and the file extension is a period (
.). For example,
before_policy1’ is the base name, ‘
v11.5’ is the optional additional part of the name, and ‘
bash’ is the file extension.
- Scripts can be written in any programming language: shell scripts, Perl, Python, or even binary executables.
- You only have to make sure the scripts can be executed and have the correct permissions.
There are three scripts for each policy:
before_<policy-name>[.optional_addition].<ext>script runs before backup begins. Typically, this script is used to move applications to backup mode. The before script typically leaves the system in a frozen state for a short time until the snapshots of the policy are fired. One option is to issue a freeze command to a file system like XFS.
after_<policy-name>[.optional_addition].<ext>script runs after all the snapshots of the policy fire. It runs within a few seconds after the before script. This script releases anything that may have been frozen or locked by the before script. This script accepts the success status of the before script. If the before script succeeded, the argument will be one
1. If it failed, crashed, or timed out, the argument will be zero
complete_<policy-name>[.optional_addition].<ext>script runs after all snapshots are completed. Usually, it runs quickly since snapshots are incremental. This script can perform cleanup after the backup is complete and is typically used for transaction log truncation. The script accepts one argument. If the entire backup was successful and all the previous scripts were successful, it will be one
1. If any issues or failures happened, it will be zero
0. If this argument is one
1, truncate logs.
You can use the output collected by N2WS to debug backup scripts. However, the recommended way is to run them independently of N2WS, on the N2WS Server machine using SSH. You can then view their outputs and fix what is needed. Once the scripts work correctly, you can start using them with N2WS. Assuming these scripts are using SSH, during the first execution you will need to approve the SSH key by answering yes at the command line prompt. If you terminate your N2WS Server and start a new one, you will need to run the scripts again from the command line and approve the SSH key.
Following is an example of a set of backup scripts that use SSH to connect to another instance and freeze a MySQL Database:
- The before script will flush and freeze the database.
- The after script will release it.
- The complete script will truncate binary logs older than the backup.
The scripts are written in Bash:
ssh -i /cpmdata/scripts/mysshkey.pem [email protected]_name.compute-1.amazonaws.com "mysql -u root –p<MySQL root password> -e 'flush tables with read lock; flush logs;'
if [$? -gt 0[; then
echo "Failed running mysql freeze" 1>&2
echo "mysql freeze succeeded" 1>&2
This script connects to another instance using SSH and then runs a MySQL command. Another approach would be to use a MySQL client on the N2WS Server, and then the SSH connection will not be necessary.
After that script is executed, the N2WS server will start the snapshots, and then call the next script:
if [ $1 -eq 0 ]; then
echo "There was an issue running first script" 1>&2
ssh -i /cpmdata/scripts/mysshkey.pem [email protected]_name.compute-1.amazonaws.com "date +'%F %H:%M:%S' > sql_backup_time; mysql -u root -p<MySQL root password> -e 'unlock tables;'"
if [ $? -gt 0 ]; then
echo "Failed running mysql unfreeze" 1>&2
echo "mysql unfreeze succeeded" 1>&2
This script checks the status in the first argument and then does two things:
- First, it saves an exact timestamp of the current backup of the frozen database to a file,
- Then, it releases the lock on the MySQL table.
After that, when all snapshots succeed, N2WS runs the complete script:
if [ $1 -eq 1]; then
cat /cpmdata/scripts/complete_sql_inner |ssh -i /cpmdata/scripts/mysshkey.pem [email protected]_name.compute-1.amazonaws.com "cat > /tmp/complete_ssh; chmod 755 /tmp/complete_ssh; /tmp/complete_ssh"
if [ $? -gt 0 ]; then
echo "Failed running mysql truncate logs" 1>&2
echo "mysql truncate logs succeeded" 1>&2
echo "There was an issue during backup - not truncating logs" 1>&2
It calls an inner script,
mysql -u root -p<MySQL root password> -e 'PURGE BINARY LOGS BEFORE "'"$butime"'"
These two scripts purge the binary logs only if the complete script receives one
1as the argument. They purge logs earlier than the time in the timestamp files.
If your N2WS configuration requires multiple users, which are separated from each other, you may wish to allow users to access N2WS using SSH to create and debug backup scripts:
- Create additional Linux users in the N2WS instance and allowing each user access to their own scripts folder only.
cpmuserwill need to be able to access and execute the scripts of all users. This can be achieved by assigning the user
cpmuseras the group of all user subfolders and scripts. Then, if given executable permissions for the group,
cpmuserwill be able to access and execute all scripts.
Some differences between the ‘old backup scripts’ and the new SSM scripts:
- SSM scripts are per instance and run on the instance, which can give better granularity and don’t require SSH.
- The old local scripts are per policy and can be good for a centralized approach or operations that function across instances.
- The old local scripts require the use of SSH for connectivity, thereby requiring connectivity between the CPM and the target.
- SSM, however, requires a setup that includes putting an IAM role on your instance, which you may not want to do.
To use SSM for Linux backups:
- 1.Install an SSM Agent on the target machine. Some Linux AMI, such as AWS Linux AMI, come with SSM already installed. To install an agent, see https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-manual-agent-install.html
- 2.An IAM role with the core SSM policy (AmazonSSMManagedInstanceCore) attached to an instance should be sufficient for using the agent. To attach an IAM instance profile to an EC2 instance, see https://docs.aws.amazon.com/systems-manager/latest/userguide/setup-launch-managed-instance.html
- 3.Similar to using normal backup scripts, as mentioned in section 7.2, create before/after/complete scripts.
- 1.For root user policies, scripts need to be under
/cpmdata/<instance_id>, for example, ‘
- 2.For non-root user policies, scripts need to be under the user folder, for example, ‘