Getting started with the Cloud Sync service Edit on GitHub Request doc changes

Getting started with the Cloud Sync service includes preparing your environment, signing up for a free trial, creating your first sync relationships, and then paying for your relationships after your free trial ends.

Preparing your environment

Before you use the Cloud Sync service, you must verify that your environment is set up correctly.

When you set up your first sync relationship in Cloud Sync, you must add a NetApp data broker. Cloud Sync uses the data broker software to sync data from a source to a target (this is called a sync relationship).

You can set up the source and target in a sync relationship as follows:

Source location Supported target locations

CIFS server 1

  • Blob container

  • CIFS server

  • S3 bucket

  • StorageGRID Webscale

NFS server 1

  • Amazon EFS

  • Blob container

  • NFS server

  • S3 bucket

  • StorageGRID Webscale

Amazon EFS

  • NFS server

  • S3 bucket

Amazon S3 bucket

  • Amazon EFS

  • CIFS server

  • NFS server

Azure Blob container

  • CIFS server

  • NFS server

StorageGRID Webscale

  • CIFS server

  • NFS server

1 Cloud Sync supports any CIFS or NFS server, including the NetApp Cloud Volumes Service.

Source and target requirements

The following requirements must be met for the source and target in a sync relationship:

CIFS and NFS servers
  • CIFS and NFS servers can be in any location that has a connection to the data broker.

    For example, if an NFS server is in your data center and the data broker is in AWS, then you need a network connection (VPN or Direct Connect) from your network to the VPC.

  • The file server must allow the data broker host to access the exports.

  • Cloud Sync supports NFS version 3 and later, and SMB version 1 and later.

  • If you want to sync NFS data from an ONTAP system, you must ensure that access to the NFS export list for an SVM is enabled (vserver nfs modify -vserver svm_name -showmount enabled).

    The default setting for showmount is enabled starting with ONTAP 9.2.
Amazon EFS

Network connectivity between the EFS service and the data broker.

Amazon S3 buckets

Network connectivity between the S3 bucket and the data broker.

Azure Blob containers

Network connectivity between the Azure Blob container and the data broker.

StorageGRID Webscale

Cloud Sync supports StorageGRID Webscale using the S3 protocol. A network connection must be available between StorageGRID Webscale and the data broker.

Data broker requirements

The data broker can reside in AWS, in Azure, or in your network. Cloud Sync walks you through the steps to add a data broker when you create a relationship. You should be aware of the following requirements:

Networking requirements
  • The data broker needs an outbound internet connection so it can poll the Cloud Sync service for tasks. You can configure the data broker to use a proxy server, if needed.

  • Port 443 must be open on the data broker host for outbound traffic to AWS.

  • In AWS, all regions are supported except for GovCloud (US).

  • In Azure, all regions are supported except for the US Gov and US DoD regions.

Host requirements

If you choose the data broker for Azure or an on-premises data broker, then you must provide a host that meets the following requirements:

Operating system

64-bit Red Hat Enterprise Linux 7 or 64-bit CentOS 7

RAM

8

CPU

4 cores

Free disk space

10

Software

IBM J9 virtual machine (JVM) must not be installed on the host

If you choose the data broker for AWS, then Cloud Sync deploys a data broker instance using a CloudFormation template.

Example configuration

The following image shows network connections for a sync relationship from an NFS server to an S3 bucket, with the data broker in AWS:

This graphic shows each component: The NetApp data broker communicates with an NFS server in a data center to synchronize data to an S3 bucket. The VPC has an outbound internet connection and the data center has a VPN connection to the VPC.

Creating your first sync relationship

When you create a sync relationship, the Cloud Sync service copies files from the source to the target. After the initial copy, the service syncs any changed data every 24 hours.

Steps
  1. Go to NetApp Cloud Central.

  2. Sign up or log in and then start a free trial of Cloud Sync.

  3. After you log in, review details about using the service after the free trial ends, and then click OK.

  4. On the Select Source & Target page, choose a source and target.

    The following steps provide an example of how to create a sync relationship from an NFS server to an S3 bucket.

  5. Review the details about how the service works and then click Continue.

  6. On the NFS Server page, enter the IP address or fully qualified domain name of the NFS server that you want to sync to AWS.

  7. On the Data Broker page, follow the prompts to create a data broker in AWS, to install the data broker in Azure, or to install the data broker an on-premises Linux host.

    If you choose AWS, the AWS CloudFormation Console loads in a separate browser tab. The Cloud Sync service uses a CloudFormation template to quickly provision the data broker instance. The Cloud Sync service does not have access to your AWS credentials because you are logging in directly to AWS.

    The following video shows how to launch the data broker instance in AWS:

    The instance takes approximately 5 to 10 minutes to deploy. You can monitor the progress from the AWS CloudFormation console, or you can return to the Cloud Sync service website, which automatically refreshes when the instance is available.

    The following video shows how to install the data broker software on an on-premises Linux host:

  8. After you install the data broker, click Continue.

    The following image shows a successfully deployed instance in AWS:

    This screenshot shows a successfully deployed instance. The interface shows details about the instance including AWS networking.

  9. On the Directories page, optionally filter source objects and then select a top-level directory or subdirectory.

    Filtering source objects enables you to define how source files and folders are synced and maintained in the target location. For example, you can choose to never delete files from the target location, even if they were deleted from the source, which is useful for archiving.

    If you want to sync more than one directory on the NFS server, then you must create additional sync relationships after you are done.

    The following video shows how to filter source objects and select a directory:

  10. On the S3 Bucket page, choose the target location:

  11. On the Relationship Tags page, enter up to 9 relationship tags and then click Continue.

    The Cloud Sync service assigns the tags to each object that it syncs to the S3 bucket.

  12. Review the details of the sync relationship and then click Create Relationship.

  13. After the Cloud Sync service successfully creates the relationship, click View in Dashboard to view details about the data sync relationship.

    This screenshot shows details about a sync relationship including the NFS source and directory and the data broker instance and the S3 bucket.

Result

Congratulations, you have created your first sync relationship. You can manage the relationship at any time by:

  • Performing an immediate data sync

  • Changing the schedule
    Relationships are automatically configured to sync data every 24 hours, but you can change the schedule to as frequently as 5 minutes.

  • Deleting the relationship
    This action does not delete the data broker instance and it does not delete data from object storage.

This screenshot shows the actions that you can perform on a sync relationship from an NFS server to an S3 bucket.

Sending data back to the original server or to a different server

After you sync data from the source to a target destination and then run analytics on that data, you might want to send the results back to your server. Or, you might want to sync data that originated in the public cloud to your private cloud.

About this task

You must create a new sync relationship to sync data back to the source destination.

Steps
  1. Click Create New Sync Relationship.

  2. Choose a source and target.

    This screenshot shows the relationship from an S3 bucket to an NFS server.

  3. Follow the steps in the wizard to create the relationship.

Paying for your sync relationships after your free trial ends

There are two ways to pay for sync relationships after your 14-day free trial ends. The first option is to pay-as-you-go. The second option is to purchase licenses directly from NetApp.

With the pay-as-you-go option, the Cloud Sync service charges hourly based on the number of sync relationships that you create. For pricing details, go to the Cloud Sync service page. To use this option, you must subscribe to the Cloud Sync service from the AWS Marketplace.

The other option is to pay for your sync relationships up front by purchasing licenses. Each license enables you to create up to 20 sync relationships. After you purchase a license, you must add it to Cloud Sync. For further details about licenses, see How Cloud Sync licenses work.

You can use both options together. For example, if you have 25 sync relationships, you can pay for the first 20 sync relationships using a license and then pay-as-you-go with the remaining 5 sync relationships.

Subscribing to the Cloud Sync service from the AWS Marketplace

You should subscribe to the Cloud Sync service from the AWS Marketplace, which enables you to pay for sync relationships at an hourly rate.

Steps
  1. Go to the License Settings page.

  2. Click Subscribe and then click Continue.

  3. Subscribe from the AWS Marketplace, and then log back in to the Cloud Sync service to complete the registration.

    The following video shows the process:

Purchasing licenses and adding them to the Cloud Sync service

To pay for your sync relationships up front, you must purchase one or more licenses and add them to the Cloud Sync service.

Steps
  1. Purchase a license by contacting NetApp.

  2. Go to the License Settings page and add the license.