4 Defining Backup Policies
The backbone of the N2WS solution is the backup policy. A backup policy defines everything about a logical group of backed-up objects. A policy defines:
- What will be backed up - Backup Targets.
- How many generations of backup data to keep.
- When to back up – Schedules.
- Whether to use backup scripts.
- Whether VSS is activated (Windows Servers 2008, 2012, 2016, and 2019 only).
- Whether backup is performed via a backup agent (Windows only).
- The retry policy in case of failure.
- DR settings for the policy.
- Lifecycle events: Number of backup generations, Copy to S3, Archive to Glacier
The following sections explain the stages for defining an N2WS policy and its schedule. Schedule definition is addressed first as it one of the attributes of a policy and Scheduled Reports.
Schedules are the objects defining when to perform a backup
- Schedules are defined separately from policies and Scheduled Reports.
- One or more schedules can be assigned to both policies and Scheduled Reports.
Schedules can be managed in the Schedules tab found in the left panel. They can be added also during the creation of a new Policy.
Or, added when creating a new scheduled report in the Scheduled Reports tab of the Reports tab in the left panel.
You can define schedules to:
- Run for the first time at a date and time in the future.
- Run forever or have a specific date and time to stop.
- Repeat every ‘n’ minutes, hours, days, weeks, months.
- Selectively enable for certain minutes, hours, and days of the week, but not for weeks and months.
- Repeat every day of the week, or only run on certain days.
- Exclude running a report or policy during certain time ranges within the scheduled times.
For the root/admin user, if you have created additional managed users, you can select the user to whom the schedule belongs.
The same schedule can be used in all of the following:
- Policy backup operations.
- Recovery Scenarios for policies with schedules.
- Generating Scheduled Reports.
You can add a schedule from a number of places:
- In the Schedules tab, selectNew.
- While creating or editing a policy in the Policies tab, in the Policy Details tab, selectNew above the Schedules list.
- While creating or editing a scheduled report in the Scheduled Reports tab of the Reports tab, selectNew above the Schedules list.
To define a schedule:
- 1.Type a name for the schedule and an optional description.
- 2.In the First Run box, if the First Run is other than immediately, select Calendarto choose the date and time to first run this schedule.
- 1.If you want a daily backup to run at 10:00 AM, set Repeats Every to one day and the start time to 10:00 AM.
- 2.If you want an hourly backup to run at 17 minutes after the hour, set Repeats Every to one hour and First Run to
nnis the hour of the First Run.
- 3.If this schedule expires, turn on the Expires toggle and select the date and time the schedule ends. By default, schedules never expire.
- 4.In the Repeats Every list, select the frequency of the backups for this schedule. The possible units are months, weeks, days, hours, and minutes.
- 5.In the Enabled On section, select the day-of-week checkboxes on which to run the schedule.
- 7.Select Save.
You can override existing schedules and run backups and Scheduled Reports immediately:
When you configure an N2WS server, its time zone is set. See section 2.3. In the N2WS management application, all time values are in the time zone of the N2WS server. Schedule times, however, may be set and executed according to a specific time zone. A policy’s Backup Times will be according to the time zone.
In the N2WS database, times are saved in UTC time zone (Greenwich). So, if, at a later stage, you start a new N2WS server instance, configure it to a different time zone, and use the same CPM data volume as before, it will still perform backup at the same times as before.
While or after defining a schedule, you can set specific times when the schedule should not start a backup or generate a Scheduled Report. For example, you want a backup or report to run every hour, but not on Tuesdays between 01:00 PM and 3:00 PM. You can define that on Tuesdays, between these hours, the schedule is disabled.
You can define a disabled time where the finish time is earlier than the start time. The meaning of disabling the schedule on Monday between 17:00 and 8:00 is that it will be disabled every Monday at 17:00 until the next day at 8:00. The meaning of disabling the schedule for All days between 18:00 and 6:00 will be that every day the schedule will be disabled after 18:00 until 6:00.
For each schedule, you can define as many excluded time ranges as you need.
To define disabled times:
- 1.In the New Schedule screen, turn on the Exclude Time Ranges toggle.
- 2.In the Day list, select a day of the week to exclude from the schedule.
- 3.In the Start Time and End Time lists, select the start and end time of the exclusion.
- 4.Select Apply after each definition. To add another time range, selectNew.
- 5.Select the checkboxes of the excluded time ranges to enable, and then select Save.
Policies are the main objects defining backups. A policy defines:
- What to back up
- How to back it up
- When to perform the backup by assigning schedules to the policy
Policy definitions are spread among the following tabs:
- Policy Details – Basic policy details: name, user, account, enabled, schedules, auto-target removal, backup generations, and description
- Backup Targets – Choose and configure backup targets. Application consistency is applied at the instance level.
- More Options – Backup options, such as activate Linux backup scripts, define successful backup, retry parameters, and number of failures to trigger an alert
- DR – Enable DR
- Lifecycle Management (Snapshot /S3 / Glacier) – Configure Lifecycle operations, including the number of backup generations, Copy to S3, and archive to Glacier.
A policy named
cpmdatamust exist for backing up Windows instances and the N2WS server, DR, and Copy to S3. For the
cpmdatapolicy, the relevant tabs are Policy Details and DR.
To define a new policy:
- 1.In the left panel, select Policies.
- 2.In theNew menu, select AWS. The Policy Details tab opens.
- 3.In the Name box, type a name for the policy.
- 4.For the root/admin user, if you have created additional managed users, select the policy owner in the User box.
- 5.If you have more than one account, in the Account list, select the account that the policy is associated with. The account cannot be modified after the policy has been created.
- 6.In the Auto Target Removal list, specify whether to automatically remove resources that no longer exist. If you enable this removal, if an instance is terminated, or an EBS volume deleted, the next backup will detect that and remove it from the policy. Choose yes and alert if you want the backup log to include a warning about such removal.
- 7.To use application-consistent scripts as the default for CPM data, in the
cmpdatapolicy, select Application Consistent.
- 8.In the Description box, optionally type a description.
- 10.To add Backup Targets later, in the Policy Details tab, select Save. The new policy is included in the list of policies in the Policies screen. To add Backup Targets, select the new policy, selectEdit and then select the Backup Targets tab.
Backup targets define what a policy is going to back up. You define backup targets by selecting the Backup Targets tab of the Create Policy screen.
Following are the types of backup targets:
- Instances – This is the most common type. You can choose as many instances as you wish for a policy up to your license limit. For each instance, allowed under your license, define:
See section 4.2.3.
- Whether to back up all its attached volumes, some, or none.
- Whether to take snapshots (default for Linux), take snapshots with one initial AMI (default for Windows), or just create AMIs.
- EBS Volumes – If you wish to back up volumes, not depending on the instance they are attached to, you can choose volumes directly. This can be useful for backing up volumes that may be detached part of the time or moved around between instances (e.g., cluster volumes).
- RDS Databases – You can use N2WS to back up RDS databases using snapshots. There are advantages to using the automatic backup AWS offers. However, if you need to use snapshots to back up RDS, or if you need to back up databases in sync with instances, this option may be useful.
- Aurora Clusters – Even though Aurora is part of the RDS service, Aurora is defined in clusters rather than in instances. Use this type of backup target for your Aurora clusters and Aurora Serverless.
- Aurora cluster storage is calculated in increments of 10 GiB with the respect to the license. For example, if you have over 10 GiB of data but less than 20 GiB, your data is computed as 20 GiB.
- Keep in mind that clusters can grow dynamically and may reach the storage limits of your license. If the total storage is approaching your license limit, N2WS will issue a warning.
- For Aurora Serverless, capacity units will appear in the Class column when adding RDS Clusters as Backup Targets.
- Redshift Clusters – You can use N2WS to back up Redshift clusters. Similar to RDS, there is an automatic backup function available, but using snapshots can give an extra layer of protection.
- DynamoDB Tables – You can use N2WS to back up DynamoDB tables. The recommended best practice is a backup limit of 10 DynamoDB tables per policy.
- When defining your backup targets, keep in mind that DynamoDB table storage is calculated in increments of 10 GiB with the respect to the license. For example, if you have over 10 GiB of data but less than 20 GiB, your data is computed as 20 GiB.
- Tables can grow dynamically and may reach the storage limits of your license. If the total storage is approaching your license limit, N2WS will issue a warning.
- Elastic File Systems (EFS) – You can use N2WS to back up and restore your EFS snapshot data to AWS using AWS Backup service. Configuration options include backup vault, IAM role, transition to cold storage, and expiration.
- FSx File Systems – Use N2WS to back up and restore your FSx File Systems to the same region and the same account. Following are the FSx types:
For AWS FSx backup limitations, see section 4.2.7.For recovery options, see section 10.11.Also, see https://aws.amazon.com/fsx.
- Lustre (Linux) file system
- Windows file system with managed Active Directory (Win-AD)
- Windows file system with self-managed Active Directory (Win-SMAD)
- NetApp ONTAP file system
- S3 Bucket Sync – You can use N2WS to synchronize a source S3 bucket to a destination bucket. Since a snapshot is not created, there is no recovery or movement to the Freezer. The buckets require configuration. See section 9.8.
In the Add Backup Targets drop-down menu, select the relevant resource type. A window containing a list of the available resources for the selected type and region opens. To view additional columns and rows, move the scroll bars as needed.
When adding backup targets of the resource type to the policy:
- You will see all the backup targets of the requested type that reside in the current region, except the ones already in the policy.
- You can select targets in another region by choosing from the region drop-down list.
- If there are many targets, you have the ability to:
- Filter by typing part of a resource name in the search box and select Search. To clear a search, select x.
- Sort by a column by selecting its heading. Select it again to change the sort direction.
- Scroll between pages.
- For each backup target, the Policies column shows the policy, or the number of policies, the target is already in. Select the link to see which policies the target is in.
To add a backup target to the policy:
- 1.Select a region in the drop-down list. The resources in the selected region are shown.
- 2.To filter the resources in the region, enter all or part of a resource name in the Search resources box and selectSearch.
- 3.Select the checkbox of the desired target resources.
- 4.Select Add selected. The selected objects are added to the policy’s backup target list.
- 5.Repeat as many times as needed, from multiple regions if relevant.
- 6.Select Close when finished. The selected targets are listed in the Backup Targets tab.
- 7.For Instances, Volumes, EFS, and S3 Bucket Sync targets, selectConfigure and complete the backup options.
- 8.In the Backup Targets screen, select Save to save the listed selections to the policy.
In the case of EC2 instances, you can set options for any instance.
By default, Copy to S3 is performed incrementally. To ensure the correctness of your data, you can force a copy of the full data for a single iteration to your S3 Repository. While defining the Backup Targets for a policy with Copy to S3, select Force a single full copy to S3.
To configure an instance:
- 1.Select a policy, select the Backup Targets tab, and then select an instance.
- 2.SelectConfigure. The Policy Instance and Volume Configuration screen opens.
- 3.In the Which Volumes list, choose whether to back up All Volumes attached to this instance, or include or exclude some of them. By default, N2WS will back up all the attached storage of the instance, including volumes that are added over time.
- 5.If All Volumes was not selected, in the volumes table, clear or select the volume checkboxes.
- 6.In the Backup Options list, select one of the following:
- 1.Snapshots Only - Default for Linux-based instances.
- 2.Snapshots with Initial AMI - Take an initial AMI and then snapshots. Default for Windows-based instances.
- 7.When Copy to S3 is enabled for the policy, to have a full copy of the data copied to your S3 Repository, select Force a single full copy to S3.
- 8.Select Apply.
- 9.Continue to add instances from other regions, and select Close when finished with the resource type. Control returns to the Backup Targets tab list.
- 10.At the bottom of the Backup Targets tab list, select Save to update the policy for all listed targets.
If you choose to just create AMIs:
- N2WS will create AMIs for that instance instead of taking direct snapshots. App-aware backup per agent does not apply for AMI creation.
- You can choose whether to reboot the instance during AMI creation or not to reboot, which leaves a risk of data corruption. As opposed to AMI creation in the EC2 console, the default in N2WS is no reboot.
Single or Initial AMIs are meant to be used mainly for Windows instance recovery.
- N2WS will keep a single AMI for each instance with this setting. A single AMI will contain only the root device (boot disk).
- N2WS will rotate single AMIs from time to time. It will create a new AMI and delete the old one. N2WS aims to optimize costs by not leaving very old snapshots in your AWS account.
- By default, N2WS will rotate single AMIs every 90 days. That value can be configured in theServer Settings > General Settings > Cleanup tab to any number of days, or to '0' if you prefer no rotation at all.
AMIs will be copied across regions and accounts for DR.
To see more policy options, select the More Options tab for a policy in the Policies tab. The options consist of:
- Activating Linux backup scripts and defining script timeout and output
- Defining backup success, retries, and failures for alerting
- Activate Linux Backup Scripts – This option is turned off by default. If you select Activate Linux Backup Scripts, the following options appear:
- Scripts Timeout – Timeout (in seconds) to let each script run. When a backup script runs, after the timeout period, it will be killed, and a failure will be assumed. The default is 30 seconds.
- Collect Scripts Output – N2WS can collect the output of backup scripts to the standard error (
stderr). This may be useful for debugging. It can also be used by a script to log the operations it is performing and write useful information. This output is captured, saved in the N2WS database, and can be viewed from the Recovery Panel screen. To turn this option on, choose Collect. The default option is Ignore.
- Backup Successful when - This indicates whether a backup needs its scripts/VSS to complete, to be considered a valid backup. This has a double effect:
The possible values are:
- For retries, a successful backup will not result in a retry;
- For the automatic retention management process, a backup that is considered successful is counted as a valid generation of data.
- Success Without Any Issues – If scripts and/or VSS are defined for this policy, the backup will be considered successful only if everything succeeds. If backup scripts or VSS fails and all snapshots succeed, the backup is not considered successful. You can still recover from it, but it will cause a retry (if any are defined), and the automatic retention management process will not count it as a valid generation of data. This is the stricter option and is also the default.
- Snapshot Succeeded with Possible VSS or Script Failure – This is the less strict option and can be useful if scripts or VSS fail often, which can happen in a complex environment. Choosing this option accepts the assumption that most applications will recover correctly even from a crash-consistent backup.
- Retry information - The next three options deal with what to do when a backup does not succeed:
- Number of Retries – The maximal number of retries that can be run for each failed backup. The default is three. After the retries, the backup will run again at the next scheduled time.
- Wait Between Retries (minutes) – Determines how much time N2WS will wait after a failure before retrying. Backup scripts and VSS may sometimes fail or timeout because the system is busy. In this case, it makes sense to substantially extend the waiting time until the next retry when the system may be more responsive.
- Failures to Trigger Alert – The minimum number of failures to trigger an alert.
To enable Disaster Recovery for the policy:
- 1.Select the DR tab for a policy in the Policies tab screen.
- 2.Select Enable DR.
- 3.Select the DR Frequency for backups, DR Backup Timeout, and Target Regions.
- 4.To enable Cross Account DR Backups, select the checkbox, and:
- 1.Choose the To Account and DR Account Target Regions in the lists.
- 2.To Keep Original Snapshots, select the checkbox.
The Lifecycle Management tab for a policy allows you to do the following:
- Set retention policy for snapshots: Keep n (Native) Backup Generations.
FSx ONTAP can be configured to backup only part of volumes, for example an EC2 instance, regardless of the owner Storage Virtual Machine (SVM).
To configure a file system:
- 1.Select a policy and then select the Backup Targets tab.
- 2.In the Add Backup Targets menu, select FSx File Systems.
- 3.Select an ONTAP policy.
Configure. The ONTAP FSx and Volume Configuration screen opens.
5. In the Which Volumes list, choose whether to back up All Volumes attached to this instance, or include or exclude individual volumes.
6. Select Apply and then Save.
Following are limitations on backups of FSx for Lustre:
- Backups are not supported on scratch file systems because these file systems are designed for temporary storage and shorter-term processing of data.
- Backups are not supported on file systems linked to an Amazon S3 bucket because the S3 bucket serves as the primary data repository, and the Lustre file system does not necessarily contain the full data set at any given time. For further information, see https://docs.aws.amazon.com/fsx/latest/LustreGuide/using-backups-fsx.html
The following actions are available for both AWS and Azure policies.
In the Policies tab, select a policy and then select
Backup Times to open the Planned Backups window, which is ordered by Time Zone. You can change the Time Period from the default ‘Next Day’ to a number of days, weeks, or the next month.
Backup Times are relevant to the schedules of the selected policy.
An ad hoc backup will execute the selected Policy and create backups of all its targets.
To run a backup immediately:
- 1.In the left panel, select the Policies tab and then select a policy in the list.
- 2.To start the backup, selectRun ASAP.
- 3.To follow the progress of the backup, select the Open Backup Monitor link in the ‘Backup started’ messageat the top right corner, or select the Backup Monitor tab.
- 1.To update the list, selectRefresh.
- 2.You can switch between showing backup records 'in the Freezer' by turning on and off thetoggle key and backup records 'not in the Freezer' by turning on and off thetoggle key in the Showarea to the right of the filters.
- 4.To view or download backup details, selectLog. SelectRefresh as needed.
To delete a particular snapshot in AWS or to delete all AWS snapshots after a successful run:
- 1.SelectView Snapshots.
- 2.Select one or more backups, and then select Delete.
To delete a policy, select it in the Policies table, and then select Delete. In the Delete Policy confirmation dialog box, type ‘delete all’ and then select Delete.