Getting Started

Deploying the RamenDR Starter Kit Pattern

Prerequisites

An OpenShift cluster
- To create an OpenShift cluster, go to the Red Hat Hybrid Cloud console.
- Select OpenShift -> Red Hat OpenShift Container Platform -> Create cluster.
A GitHub account with a personal access token that has repository read and write permissions.
The Helm binary, for instructions, see Installing Helm
Additional installation tool dependencies. For details, see Patterns quick start.

It is desirable to have a cluster for deploying the GitOps management hub assets and a separate cluster(s) for the managed cluster(s).

Preparing for deployment

Procedure

Fork the ramendr-starter-kit repository on GitHub. You must fork the repository because your fork is updated as part of the GitOps and DevOps processes.

Clone the forked copy of this repository.

$ git clone git@github.com:your-username/ramendr-starter-kit.git

Go to your repository: Ensure you are in the root directory of your Git repository by using:
```
$ cd /path/to/your/repository
```

Run the following command to set the upstream repository:

$ git remote add -f upstream git@github.com:validatedpatterns/ramendr-starter-kit.git

Verify the setup of your remote repositories by running the following command:

$ git remote -v

Example output

origin	git@github.com:kquinn1204/ramendr-starter-kit.git (fetch)
origin	git@github.com:kquinn1204/ramendr-starter-kit.git (push)
upstream	git@github.com:validatedpatterns/ramendr-starter-kit.git (fetch)
upstream	git@github.com:validatedpatterns/ramendr-starter-kit.git (push)

Decide which install variant to use and set main.variant in values-global.yaml before install. The default is odf. For variant purposes and verification differences, see Install variants on the pattern overview page.
```
main:
  variant: odf
```
Make a local copy of secrets template outside of your repository to hold credentials for the pattern.
Do not add, commit, or push this file to your repository. Doing so may expose personal credentials to GitHub.
Run the following commands:
```
$ cp values-secret.yaml.template ~/values-secret.yaml
```

Populate this file with secrets, or credentials, that are needed to deploy the pattern successfully:

$ vi ~/values-secret.yaml

Edit the vm-ssh section to include the username, private key, and public key. To ensure the seamless flow of the pattern, the value associated with the privatekey and publickey has been updated with path. For example:
```
  - name: vm-ssh
    vaultPrefixes:
    - global
    fields:
    - name: username
      value: 'cloud-user'
    - name: privatekey
      path: '/path/to/private-ssh-key'
    - name: publickey
      path: '/path/to/public-ssh-key'
```
Paste the path to your locally stored private and public keys. If you do not have a key pair, generate one using ssh-keygen.

Edit the cloud-init section to include the userData block to use with cloud-init. For example:

  - name: cloud-init
    vaultPrefixes:
    - global
    fields:
    - name: userData
      value: |-
        #cloud-config
        user: 'cloud-user'
        password: 'cloud-user'
        chpasswd: { expire: False }

Edit the aws section to refer to the file containing your AWS credentials:

  - name: aws
    fields:
      - name: aws_access_key_id
        ini_file: ~/.aws/credentials
        ini_key: aws_access_key_id
      - name: aws_secret_access_key
        ini_file: ~/.aws/credentials
        ini_key: aws_secret_access_key
      - name: baseDomain
        value: aws.example.com
      - name: pullSecret
        path: ~/pull_secret.json
      - name: ssh-privatekey
        path: ~/.ssh/privatekey
      - name: ssh-publickey
        path: ~/.ssh/publickey

Edit the openshiftPullSecret section to refer to the file containing your OpenShift pull secret:

  - name: openshiftPullSecret
    fields:
      - name: .dockerconfigjson
        path: ~/pull_secret.json

If you selected the drpartner-s4 variant, add the two Vault secrets required for hub vp-s4-storage. Paths must match the chart overrides in variants/drpartner-s4/values-drpartner-s4.yaml (s4UICredentials.vaultKey and s4APICredentials.vaultKey). The drpartner-minimal variant does not deploy vp-s4-storage and does not need these secrets.

  - name: s4-ui-credentials
    vaultPrefixes:
      - global
    fields:
      - name: UI_USERNAME
        value: s4admin
        onMissingValue: error
      - name: UI_PASSWORD
        onMissingValue: generate
        override: true
        vaultPolicy: advancedPolicy

  - name: s4-api-credentials
    vaultPrefixes:
      - global
    fields:
      - name: AWS_ACCESS_KEY_ID
        value: s4admin
        onMissingValue: error
      - name: AWS_SECRET_ACCESS_KEY
        onMissingValue: generate
        override: true
        vaultPolicy: advancedPolicy

External Secrets merges both Vault entries into Kubernetes Secret s4-credentials in the vp-s4-storage namespace. Defaults in values-secret.yaml.template use s4admin for the UI user and access key ID; the password and secret key can be generated when onMissingValue: generate and vaultPolicies.advancedPolicy are set in your secrets file.

If you plan to use bring your own cluster (BYOC) instead of Hive-provisioned managed clusters, add kubeconfig secrets for each regional DR cluster. See Using bring your own cluster (BYOC).

  - name: ocp-primary_cluster_kubeconfig
    fields:
      - name: kubeconfig
        path: /path/to/ocp-primary/kubeconfig

  - name: ocp-secondary_cluster_kubeconfig
    fields:
      - name: kubeconfig
        path: /path/to/ocp-secondary/kubeconfig

Create and switch to a new branch named my-branch, by running the following command:
```
$ git checkout -b my-branch
```
The pattern will infer the baseDomain of your cluster based on the clusterDomain which is tracked by the pattern operator. Previously, this required the pattern to be forked to be useful - but this is no longer the case (you may still wish to change other settings in the RDR chart’s values file, such as aws.region settings. This file is at hub/rdr/values.yaml. If you do make customizations to this or other files, it is necessary to fork the pattern so that the changes will be seen by ArgoCD. If you made any changes to this or any other files tracked by git, git add them and then commit the changes by running the following command:
```
$ git commit -m "any updates"
```
Push the changes to your forked repository:
```
$ git push origin my-branch
```

The preferred way to install this pattern is by using the ./pattern.sh script. By default, Red Hat Advanced Cluster Management (RHACM) Hive provisions the primary and secondary managed clusters on AWS. If you already have regional DR clusters, use BYOC and ./pattern.sh make install-byoc instead of ./pattern.sh make install.

Using bring your own cluster (BYOC)

BYOC (bring your own cluster) lets you use existing OpenShift clusters as the regional DR pair (ocp-primary and ocp-secondary) instead of waiting for Hive to provision new managed clusters on AWS. This option is available for all install variants and is the typical path for drpartner-minimal (Hive/BYOC bring-up without Ramen DR CRs).

When to use BYOC

Use BYOC when:

You already operate two OpenShift clusters that will serve as the primary and secondary DR sites.
You want to avoid Hive cluster provisioning time and AWS cluster creation costs for the managed pair.
You are validating partner CSI plumbing on clusters you provisioned outside the pattern.
You are installing on a platform other than AWS

The default install path (Hive provisioning) remains appropriate when you want the pattern to create and lifecycle-manage the managed clusters for you.

BYOC cluster requirements

Before install, the install-byoc target runs ansible/playbooks/validate_byoc.yml against the kubeconfigs in your secrets file. Clusters must meet these checks:

Reachability — Hub and both regional clusters are reachable from the install environment.
OpenShift Container Platform version — Hub is OpenShift Container Platform 4.18 or later. Primary and secondary must match the hub minor version (same minor is required for RamenDR and OpenShift Data Foundation).
Non-overlapping networks — Pod and service CIDRs must not overlap between the hub, primary, and secondary clusters.

You are responsible for making sure that the managed clusters are suitable for running Virtual Machines. The Hive provisioner will use m5.metal instances on AWS. Different clouds and bare metal have different requirements which are hard to generalize, so the playbook does not. (Previous versions would look for "metal" instance types but that only works on AWS.)

Configuring BYOC

Provision and register your regional DR clusters with Red Hat Advanced Cluster Management (RHACM). Managed cluster names must match the names in

overrides/values-cluster-names.yaml (default: ocp-primary and ocp-secondary).

Set byoc: true in overrides/values-cluster-names.yaml and align clusterOverrides with your cluster names, versions, and regions:

byoc: true

clusterOverrides:
  primary:
    name: ocp-primary
    version: 4.22.1
  secondary:
    name: ocp-secondary
    version: 4.22.1

Deploying the pattern by using the pattern.sh file

Commit and push byoc: true and any clusterOverrides changes to your fork so Argo CD sees them during install.

To deploy the pattern by using the pattern.sh file, complete the following steps:

Log in to your cluster by following this procedure:
1. Obtain an API token by visiting https://oauth-openshift.apps.<your-cluster>.<domain>/oauth/token/request.
2. Log in to the cluster by running the following command:
  $ oc login --token=<retrieved-token> --server=https://api.<your-cluster>.<domain>:6443
  Or log in by running the following command:
  $ export KUBECONFIG=~/<path_to_kubeconfig>
Deploy the pattern to your hub cluster.
1. For the default Hive provisioning path, run:
  $ ./pattern.sh make install
2. For BYOC, after setting byoc: true and adding regional cluster kubeconfigs to your secrets file, run:
  $ ./pattern.sh make install-byoc
  The install-byoc target loads secrets (when configured), runs the BYOC validation playbook, then installs the pattern the same way as install.

Verification

Verify that the Operators have been installed on the hub cluster. Navigate to Operators → Installed Operators page in the OpenShift Container Platform web console on the Hub cluster (in the "local-cluster" view),
Figure 1. RamenDR Hub Operators
Verify that the primary and secondary managed clusters are available in Red Hat Advanced Cluster Management (RHACM). For the default Hive path, cluster creation can take close to an hour on AWS. For BYOC, clusters should already be registered before install completes. On the hub cluster, navigate to All Clusters in the OpenShift Container Platform web console:
Figure 2. RamenDR Clusters
Wait some time for everything to deploy to all the clusters. It might take up to another hour from when the managed clusters finish building. You can track the progress through the Hub ArgoCD UI from the nines menu, especially the "opp-policy" and the "regional-dr" applications. Most of the critical resources are in the regional-dr application (at present, the opp-policy app may show missing/out-of-sync, and the regional-dr app may show OutOfSync - even when both are healthy. We are working on a fix, track bug progress here):
Figure 3. RamenDR Starter Kit Applications
For the odf variant (default), the Virtual Machines will be deployed and the Disaster Recovery Placement Control (DRPC) will show that resources are now protected. This screen can be reached via All Clusters → Data Services → Disaster Recovery → Protected Applications on the hub cluster. Normally it will be faster to synchronize Kubernetes objects than Application volumes. When these indicators both show Healthy it is safe to trigger a failover:
Figure 4. RamenDR Starter Kit Applications
For the odf variant, you might want to see the VMs themselves running. They will be on the primary cluster in the Virtualization → VirtualMachines area. The pattern configures 4 RHEL9 VMs by default:
Figure 5. RamenDR Starter Kit Trigger Failover, part 1
For the odf variant, clicking the Failover option will bring up a modal dialog that indicates where the failover will move the workload, and when the last known good state of the workload is. Click the Initiate button to begin the failover:
Figure 6. RamenDR Starter Kit Trigger Failover, part 2
For the odf variant, while the failover is happening, you can watch the progress of it in the activity area. When it is done, it will say (with a discovered application) that it is necessary to clean up application resources to allow replication to start in the other direction. Notice that the primary cluster should have changed:
Figure 7. RamenDR Starter Kit Failover Cleanup
For the odf variant, the pattern provides a script to do this cleanup. Invoke it with your Hub cluster KUBECONFIG set and run ./pattern.sh scripts/cleanup-gitops-vms-non-primary.sh:
Figure 8. RamenDR Starter Kit Failover Cleanup
For the odf variant, after a few minutes, the resources should show healthy and protected again (the PVCs take a few minutes to synchronize):
Figure 9. RamenDR Starter Kit Reprotected

Verifying partner variants

After sync, expect the following differences for the partner install variants.

`drpartner-s4`

Hub operators include Multicluster Orchestrator (Ramen), OpenShift Virtualization, and OADP. Submariner is disabled in opp-policy.
Hub vp-s4-storage is present and creates buckets via s4Role.buckets.
No ODF StorageSystem, MirrorPeer, or odf-dr application.
The regional-dr application runs with ramen.infrastructureEnabled: true. It creates DRClusters, a single 2m-novm DRPolicy (no 2m-vm), and upserts hub s3StoreProfiles with ensureBuckets: false. It does not deploy DRPC or VMs.
The opp-policy application injects caCertificates on S3 profiles only.

`drpartner-minimal`

Same partner operators and plumbing as drpartner-s4, but without vp-s4-storage or Submariner (submariner.enabled: false in opp-policy).
The regional-dr application runs with both ramen.resourcesEnabled and ramen.infrastructureEnabled set to false. No DRPolicy, DRClusters, S3 profile work, or DRPC/VM workloads are deployed.

Edit this page Open a documentation issue