RAG with text and images

End-to-End RAG with Weaviate

JP Hwang

Senior Developer Educator, Weaviate

What about the images?

 

  • Images after docling: <!-- image -->
  • Image context is lost

 

  • New approach: treat the entire document as an image

hai_ai-index-report-2025_chapter2_excerpts_8_of_8.jpg

End-to-End RAG with Weaviate

RAG with images

image_rag.png

End-to-End RAG with Weaviate

Multi-modal embedding models

cat_in_hat.png

End-to-End RAG with Weaviate

Encoding images

encoding_images1.png

End-to-End RAG with Weaviate

Encoding images

encoding_images2.png

End-to-End RAG with Weaviate

Encoding images

encoding_images.png

End-to-End RAG with Weaviate

Augmenting and generating with images

 

multi_modal_prompt.png

End-to-End RAG with Weaviate

Let's practice!

End-to-End RAG with Weaviate

Preparing Video For Download...