AI Generation with LLM and RAG
TestedDocument Generation Demo with LLM and RAG
Introduction
This deployment is based on the validated pattern framework, using GitOps for seamless provisioning of all operators and applications. It deploys a Chatbot application that harnesses the power of Large Language Models (LLMs) combined with the Retrieval-Augmented Generation (RAG) framework.
The pattern uses the Red Hat OpenShift AI to deploy and serve LLM models at scale.
The application uses either the EDB Postgres for Kubernetes operator (default), or Redis, to store embeddings of Red Hat product documentation, running on Red Hat OpenShift Container Platform to generate project proposals for specific Red Hat products.
Pre-requisites
- Podman
- Red Hat Openshift cluster running in AWS. Supported regions are us-west-2 and us-east-1.
- GPU Node to run Hugging Face Text Generation Inference server on Red Hat OpenShift cluster.
- Create a fork of the rag-llm-gitops git repository.
Demo Description & Architecture
The goal of this demo is to demonstrate a Chatbot LLM application augmented with data from Red Hat product documentation running on Red Hat OpenShift AI. It deploys an LLM application that connects to multiple LLM providers such as OpenAI, Hugging Face, and NVIDIA NIM. The application generates a project proposal for a Red Hat product.
Key Features
- Leveraging Red Hat OpenShift AI to deploy and serve LLM models powered by NVIDIA GPU accelerator.
- LLM Application augmented with content from Red Hat product documentation.
- Multiple LLM providers (OpenAI, Hugging Face, NVIDIA).
- Vector Database, such as EDB Postgres for Kubernetes, or Redis, to store embeddings of Red Hat product documentation.
- Monitoring dashboard to provide key metrics such as ratings.
- GitOps setup to deploy e2e demo (frontend / vector database / served models).
Figure 1. Overview of the validated pattern for RAG Demo with Red Hat OpenShift
Figure 2. Logical diagram of the RAG Demo with Red Hat OpenShift.
