In addition to creating and managing EBS snapshots, N2WS can store backups in Simple Storage Service (S3) and S3 Glacier, allowing you to lower backup costs when storing backups for a prolonged amount of time. N2WS allows you to create a lifecycle policy, where older snapshots are automatically moved from high-cost to low-cost storage tiers. A typical lifecycle policy would consist of the following sequence:
Store daily EBS snapshots for 30 days.
Store one out of seven (weekly) snapshots in S3 for 3 months.
Finally, store a monthly snapshot in S3 Glacier for 7 years, as required by regulations.
Configuring a lifecycle management policy in N2WS consists of the following sequence:
Defining how many EBS snapshots to keep.
Enabling and configuring Backup to S3.
Enabling and configuring Archive to S3 Glacier.
For detailed S3 storage class information, refer to https://aws.amazon.com/s3/storage-classes.
Using the N2WS Copy to S3 feature, you can:
Define multiple folders, known as repositories, within a single S3 bucket
Define the frequency with which N2WS backups are moved to a Repository in S3, similar to DR backup. For example, copy every third generation of an N2WS backup to S3.
Define backup retention based on time and/or number of generations per Policy.
N2WS stores backups in S3 as block-level incremental backups.
Only the copy of instance backups is supported.
Copy to S3 is supported for weekly and monthly backup frequencies only. Daily backup copies to S3 are not supported.
Copy to S3 is not supported for other AWS resources that N2WS supports, such as RDS and Aurora.
Snapshots consisting of ‘AMI-only’ cannot be copied to an S3 repository.
Due to AWS service restrictions in some regions, the root volume of instances purchased from Amazon Marketplace, such as instances with product code, may be excluded from Copy to S3. The data volumes of such instances, if they exist, will be copied.
Backup records that were copied to S3 cannot be moved to the Freezer.
A separate N2WS server, for example, one with a different “CPM Cloud Protection Manager Data” volume, cannot reconnect to an existing S3 repository.
To use the Copy to S3 functionality, the
cpmdata policy must be enabled. See N2WS User Guide for details on enabling the
Only a single S3 operation is possible on a policy at any given time. Additional executions of Copy to S3 backups will not occur if the previous execution is still running. Restore from S3 is always possible unless the backup itself is being cleaned up.
AWS accounts have a default limit to the number of instances that can be launched. Copy to S3 launches extra instances as part of its operation and may fail if the AWS quota is reached. See N2WS User Guide for details.
Copy and Restore of volumes to/from regions different from where the S3 bucket resides may incur long delays and additional bandwidth charges.
Instance names may not contain slashes (/) or backslashes (\) or the copy will fail.
N2W Software has the following recommendations to N2WS customers for help lowering transfer fees and storage costs:
When an ‘N2WSWorker’ instance is using a public IP (or NAT/IGW within a VPC) to access an S3 bucket within the same region/account, it results in network transfer fees.
Using a VPC endpoint instead will enable instances to use their private IP to communicate with resources of other services within the AWS network, such as S3, without the cost of network transfer fees.
For further information on how to configure N2WS with a VPC endpoint, see section Appendix A.
The Copy to S3 feature is similar in many ways to the N2WS Disaster Recovery (DR) feature. When Copy to S3 is enabled for a policy, copying EBS snapshot data to S3 begins at the completion of the EBS backup, similar to the way DR works. Copy to S3 can be used simultaneously with the DR feature.
Define an S3 Repository.
Define a Policy with a Schedule, as usual.
Configure the policy to include Copy to S3 by selecting the Lifecycle Management (Snapshot/S3/Glacier tab. Turn on the Backup to S3 toggle and complete the parameters.
If you are going to back up and restore S3 instances and volumes across accounts and regions, prepare a Worker Configuration using the Worker Configuration tab. See section 22.
Use the Backup Monitor and Recovery Monitor, with some additional controls, to manage S3 snapshots as usual.
cpmdata policy must exist before configuring an S3 Repository.
There can be multiple repositories in a single AWS S3 bucket.
In N2WS, select the S3 Repositories tab, and select New.
In the New S3 Repository screen, complete the following fields, and select Save when complete.
Name - Type a unique name for the new repository, which will also be used as a folder name in the AWS bucket. Only alphanumeric characters and the underscore are allowed.
Description - Optional brief description of the contents of the repository.
User – Select the user in the list.
Account - Select the account that has access to the S3 bucket.
AWS Region - Select the region in which the S3 bucket is located.
S3 Bucket Name - Type the name of the S3 bucket that exists in this region.
You can delete all snapshots copied to a specific S3 repository.
Select the Repositories tab, and then select a repository.
Configuring a Policy for Copy to S3 backups includes definitions for the following:
Name of the S3 Repository defined in N2WS.
Interval of AWS snapshots to copy.
Snapshot retention policy. Selecting the Delete instance snapshots from EBS after storing in S3 option minimizes the time that CPM holds any backup data in the EBS snapshots service. CPM achieves that by deleting any EBS snapshot immediately after copying it to S3.
It is possible to retain a backup based on both time and number of generations copied. If both Time Retention (Keep backups in S3 for at least x time) and Generation Retention (Keep backups in S3 for at least x generations) are enabled, both constraints must be met before old snapshots are deleted or moved to Glacier, if enabled.
In the left panel, select the Policies tab.
Select a Policy, and then select Edit.
Select the Lifecycle Management tab.
Select the number of EBS Backup Snapshot Generations to keep in the list.
Complete the following fields:
Backup to S3 – By default, Backup to S3 is disabled. Turn the toggle on to enable.
Store EBS snapshots in S3 based on the following settings:
Delete instance snapshots from EBS after storing in S3 –If selected, N2WS will automatically set the Backup to S3 every n (EBS) Backup Snapshot Generations to 1 and will delete snapshots from EBS after performing the Copy to S3 operation.
Backup to S3 every n (EBS) Backup Snapshot Generations – Select the maximum number of backup snapshot generations to keep. This number is automatically set to 1 if you opted to Delete instance snapshots from EBS after storing in S3.
In the Keep backups in S3 for at least lists, select the duration and/or number of backup generations to keep.
To Archive to Glacier, see section 21.4.
In the Storage settings section, choose the following parameters:
Select the Target Repository in the S3 bucket to move the backup to, or select New to define a new repository. If you define a new repository, select Refresh before selecting.
Choose an S3 Storage Class that meets your needs:
Standard - (Frequent Access) for Frequent access and backups.
Infrequent Access - For data that is accessed less frequently.
Intelligent Tiering - Automatic cost optimization for S3 copy. Intelligent Tiering incorporates the Standard (Frequent Access) and Infrequent Access tiers. It monitors access patterns and moves objects that have not been accessed for 30 consecutive days to the Infrequent Access tier. If the data is subsequently accessed, it is automatically moved back to the Frequent Access tier.
See information on S3 Storage Class charges below.
If Archive to Glacier is enabled, select the Archive Storage class.
You can recover an S3 backup to the same or different regions and accounts.
If you Recover Volumes Only, you can:
Select volumes and Explore folders and files for recovery.
Define Attach Behaviour
Define the AWS Credentials for access
Configure a Worker in the Worker Configuration tab.
Clone a VPC
If you recover an S3 Instance, you can specify the recovery encryption key:
If Use Default Volume Encryption Keys is enabled, the recovered volumes will have the default key of each encrypted volume.
If Use Default Volume Encryption Keys is disabled, all encrypted volumes will be recovered with the same key that was selected in the Encryption Key list.
To recover an S3 backup:
In the Backup Monitor tab, select a relevant backup that as a Lifecycle Status of 'Stored in S3', and then select Recover.
In the Restore from drop-down list of the Recover screen, select the name of the S3 Repository to recover from. If you have multiple N2WS accounts defined, you can choose a different target account to recover to.
In the Restore to Region drop-down list, select the Region to restore the S3 copy to. The source Region of the S3 copy is displayed in the Region column.
To follow the progress of the recovery, select Open Recovery Monitor in the ‘Recovery started’ message at the top right corner, or select the Recovery Monitor tab.
To abort a recovery in progress, in the Recovery Monitor, select the recovery item and then select Abort Recover from S3.
By default, Copy to S3 is performed incrementally for data modified since the previous snapshot was stored. However, you can force a copy of the full data for a single iteration to your S3 Repository. While configuring the Backup Targets for a policy with Copy to S3, select Force a single full Copy. See section 4.2.3.
You can set different retention rules in each Policy.
To update the S3 retention rules for a policy:
In the Policies column, select the target policy.
Select the Lifecycle Management tab.
Update the Keep backups in S3 for at least lists for time and generations, as described in section 21.3, and select Save.
Amazon S3 Glacier and S3 Glacier Deep Archive provide comprehensive security and compliance capabilities that can help meet regulatory requirements, as well as durable and extremely low-cost data archiving and long-term backup.
CPM allows customers to use the Amazon Glacier low-cost cloud storage service for data with longer retrieval times.
The CPM can now backup your data to a cold data cloud service on Amazon Glacier by moving infrequently accessed data to archival storage to save money on storage costs.
Following are some of the highlights of the Amazon pricing for Glacier:
Amazon charges per gigabyte (GB) of data stored per month on Glacier.
Objects that are archived to S3 Glacier and S3 Glacier Deep Archive have a minimum of 90 days and 180 days of storage, respectively.
Objects deleted before 90 days and 180 days incur a pro-rated charge equal to the storage charge for the remaining days.
For more information about S3 Glacier pricing, refer to sections ‘S3 Intelligent – Tiering’ / ‘S3 Standard-Infrequent Access’ / ‘S3 One Zone - Infrequent Access’ / ’S3 Glacier’ / ’S3 Glacier Deep Archive’ at https://aws.amazon.com/s3/pricing/
To configure archiving S3 backups to Glacier:
From the left panel, in the Policies tab, select a Policy and then select Edit.
Select the Lifecycle Management (Snapshot / S3 / Glacier) tab. See section 21.3.
Follows the instructions for Backup to S3. See section 21.3.1.
Turn on the Archive to Glacier toggle.
Complete the following parameters:
Move one expired S3 backup to Glacier every X period – Select the frequency of archiving.
Keep in Glacier for X period– Select the duration of the archive in Glacier.
Select the Archive Storage class:
Glacier - Designed for archival data that will be rarely, if ever, accessed.
Deep Archive - Solution for storing archive data that only will be accessed in rare circumstances.
Archived snapshots cannot be recovered directly from Glacier. The data must first be copied to S3 (‘retrieved’) before it can be accessed.
The process of retrieving data from Archive to S3 is automatically and seamlessly managed by N2WS. However, to recover an archived snapshot, the user should specify the following parameters:
Days to keep
Duration and cost of Instance recovery are determined by the retrieval tier selected. Depending on the Retrieval option selected, the restore completes in:
Expedited - 1-5 minutes
Standard - 3-5 hours
Bulk - 5-12 hours
To restore data from S3 Glacier:
Follow the steps for Recovering an S3 Backup. See section 21.3.2.
In the Backup Monitor, select a successful Glacier copy, and then select Recover.
In the Restore from drop-down list, select ‘Glacier’. N2WS will copy the data from Glacier to S3 and keep it for the expiration set in Keep data in S3 for x days.
In the Restore to Region list, select the target region.
Select a Retrieval option (Bulk, Standard, or Expedited) and then select Retrieve.
After a policy with Backup to S3 starts, you can:
Follow its progress in the Status column the Backup Monitor.
Abort the copy of snapshots to S3.
Stop S3 and Archive operations.
Delete S3 snapshots.
You can view the progress and status of S3 and archived backups in the Backup Monitor.
Select the Backup Monitor tab.
In the Lifecycle Status column, the realtime status of an S3 Copy is shown. Possible lifecycle statuses include:
Storing to S3 (n%)
Stored in S3
Not stored in S3 – Operation failed or was aborted by user.
Marked as archived – Some or all the snapshots of the backup were not successfully moved to Archive storage, either due to the user aborting the operation or an internal failure. However, the snapshots in the backup will be retained according to Archive retention policy, regardless of their actual storage.
Deleted from S3/Archive – Snapshots were successfully deleted from either S3 or Archive. See section 21.5.4.
Marked for deletion – The backup was scheduled for deletion according to the retention policy and will be deleted shortly.
The Copy to S3 portion of a Policy backup occurs after the non-S3 backups have completed.
To stop an S3 Copy in progress:
In the Backup Monitor, select the policy.
When the Lifecycle Status is ‘Storing to S3 ...’, select Abort Copy to S3 Snapshots.
If an S3 retention Cleanup is ‘In progress’, in the Policies tab, select the S3 policy and then select Stop S3 / Archive Operations to stop the Cleanup. See the Information in section 21 for the reasons you might want to stop the S3 Cleanup.
Stopping S3 Cleanup does not stop the non-S3 cleanup portion of the policy from completing. Only the S3 cleanup portion is stopped.
Stopping S3 Cleanup of a policy containing several instances will stop the cleanup process for a policy as follows:
N2WS will perform the cleanup of the current instance according to its retention policy.
N2WS will terminate all S3 Cleanups for the remainder of the instances in the policy.
N2WS will set the session status to Aborted.
N2WS user will get an ‘S3 Cleanup of your policy aborted by user’ notification by email.
To stop an S3 Cleanup in progress:
Determine when the S3/Archiving is taking place by going to the Backup Monitor
Select the policy and then select Log.
When the log indicates the start of the Cleanup, select Stop S3 /Archive Operations.
To delete only the snapshots copied to a specific S3 repository:
Select the S3 Repositories tab.
Select a repository, and then select Delete.