AI Generation with LLM and RAG

Validation status:

Tested

CI status:

Links:

• Install
• Help & Feedback
• Report Bugs

Document Generation Demo with LLM and RAG

Introduction

This deployment is based on the validated pattern framework, using GitOps for seamless provisioning of all operators and applications. It deploys a Chatbot application that harnesses the power of Large Language Models (LLMs) combined with the Retrieval-Augmented Generation (RAG) framework.

The application uses either the EDB Postgres for Kubernetes operator (default), or Redis, to store embeddings of Red Hat product documentation, running on Red Hat OpenShift Container Platform to generate project proposals for specific Red Hat products.

Pre-requisites

• Podman
• Red Hat Openshift cluster running in AWS. Supported regions are us-west-2 and us-east-1.
• GPU Node to run Hugging Face Text Generation Inference server on Red Hat OpenShift cluster.
• Create a fork of the rag-llm-gitops git repository.

Demo Description & Architecture

The goal of this demo is to demonstrate a Chatbot LLM application augmented with data from Red Hat product documentation running on Red Hat OpenShift. It deploys an LLM application that connects to multiple LLM providers such as OpenAI, Hugging Face, and NVIDIA NIM. The application generates a project proposal for a Red Hat product

Key Features

• LLM Application augmented with content from Red Hat product documentation.
• Multiple LLM providers (OpenAI, Hugging Face, NVIDIA).
• Vector Database, such as EDB Postgres for Kubernetes, or Redis, to store embeddings of Red Hat product documentation.
• Monitoring dashboard to provide key metrics such as ratings.
• GitOps setup to deploy e2e demo (frontend / vector database / served models).

Figure 1. Overview of the validated pattern for RAG Demo with Red Hat OpenShift

Figure 2. Logical diagram of the RAG Demo with Red Hat OpenShift.