Prerequisites

• Podman is installed on your system.
• You have the OpenShift Container Platform installation program and the pull secret for your cluster. You can get these from Install OpenShift on AWS with installer-provisioned infrastructure.
• Red Hat Openshift cluster running in AWS.

It is also possible to deploy the RAG-LLM Gitops pattern to Azure. Since these docs focus mostly on the AWS deployment, it’s recommended that you reference RAG-LLM pattern on Microsoft Azure for more details about installing this pattern on Azure.

Procedure

1. Create the installation configuration file using the steps described in Creating the installation configuration file.

   Note: Supported regions are us-east-1 us-east-2 us-west-1 us-west-2 ca-central-1 sa-east-1 eu-west-1 eu-west-2 eu-west-3 eu-central-1 eu-north-1 ap-northeast-1 ap-northeast-2 ap-northeast-3 ap-southeast-1 ap-southeast-2 and ap-south-1. For more information about installing on AWS see, Installation methods.

2. Customize the generated install-config.yaml creating one control plane node with instance type m5.2xlarge and 3 worker nodes with instance type m5.2xlarge. A sample YAML file is shown here:

   additionalTrustBundlePolicy: Proxyonly
   apiVersion: v1
   baseDomain: aws.validatedpatterns.io
   compute:
     - architecture: amd64
       hyperthreading: Enabled
       name: worker
       platform:
         aws:
           type: m5.2xlarge
       replicas: 3
   controlPlane:
     architecture: amd64
     hyperthreading: Enabled
     name: master
     platform:
       aws:
         type: m5.2xlarge
     replicas: 1
   metadata:
     creationTimestamp: null
     name: kevstestcluster
   networking:
     clusterNetwork:
       - cidr: 10.128.0.0/14
         hostPrefix: 23
     machineNetwork:
       - cidr: 10.0.0.0/16
     networkType: OVNKubernetes
     serviceNetwork:
       - 172.30.0.0/16
     platform:
       aws:
         region: us-east-1
     publish: External
     pullSecret: "<pull-secret>"
     sshKey: |
       ssh-ed25519 <public-key> someuser@redhat.com    
   

3. Fork the rag-llm-gitops git repository.

4. Clone the forked repository by running the following command:

   $ git clone git@github.com:your-username/rag-llm-gitops.git
   

5. Go to your repository: Ensure you are in the root directory of your git repository by using the following command:

   $ cd rag-llm-gitops
   

6. Create a local copy of the secret values file by running the following command:

   $ cp values-secret.yaml.template ~/values-secret-rag-llm-gitops.yaml
   

   Note: For this demo, editing this file is unnecessary as the default configuration works out of the box upon installation.

7. Add the remote upstream repository by running the following command:

   $ git remote add -f upstream git@github.com:validatedpatterns/rag-llm-gitops.git
   

8. Create a local branch by running the following command:

   $ git checkout -b my-test-branch main
   

9. By default the pattern deploys the EDB Postgres for Kubernetes as a vector database. To deploy Elasticsearch, change the global.db.type parameter to the ELASTIC value in your local branch in values-global.yaml. For more information see, Deploying a different databases to change the vector database.

10. By default instance types for the GPU nodes are g5.2xlarge. Follow the Customize GPU provisioning nodes to change the GPU instance types.

11. Run the following command to push my-test-branch (including any changes) to the origin remote repository:

    $ git push origin my-test-branch
    

12. Ensure you have logged in to the cluster at both command line and the console by using the login credentials presented to you when you installed the cluster. For example:

    INFO Install complete!
    INFO Run 'export KUBECONFIG=<your working directory>/auth/kubeconfig' to manage the cluster with 'oc', the OpenShift CLI.
    INFO The cluster is ready when 'oc login -u kubeadmin -p <provided>' succeeds (wait a few minutes).
    INFO Access the OpenShift web-console here: https://console-openshift-console.apps.demo1.openshift4-beta-abcorp.com
    INFO Login to the console with user: kubeadmin, password: <provided>
    

13. Add GPU nodes to your existing cluster deployment by running the following command:

    $ ./pattern.sh make create-gpu-machineset
    

    Note: You may need to create a file config in your home directory and populate it with the region name.

    1. Run the following:

    vi ~/.aws/config
    

    2. Add the following:

    [default]
    region = us-east-1
    

14. Adding the GPU nodes should take about 5-10 minutes. You can verify the addition of these g5.2xlarge nodes in the OpenShift web console under Compute > Nodes.

15. Install the pattern with the demo application by running the following command:

    $ ./pattern.sh make install
    

    Note: This deploys everything you need to run the demo application including the Nividia GPU Operator and the Node Feature Discovery Operator used to determine your GPU nodes.

Verify the Installation

1. In the OpenShift web console go to the Workloads > Pods menu.

2. Select the rag-llm project from the drop down.

3. Following pods should be up and running.

   

Launch the application

• Click the Application box icon in the header, and select Retrieval-Augmented-Generation (RAG) LLM Demonstration UI

  

• It should launch the application

  

Generate the proposal document

• The demo generates a proposal document using the default provider Mistral-7B-Instruct; a model available on Hugging Face. It is a fine-tuned version of the base Mistral-7B model.

• Enter any company name for example Microsoft.

• Enter the product as RedHat OpenShift AI

• Click the Generate button, a project proposal should be generated. The project proposal also contains the reference of the RAG content. The project proposal document can be Downloaded in the form of a PDF document.