We’ve talked a lot about optimizing text, but if your documents contain images or tables, it’s important to take a few extra steps to ensure these elements are also usable by your agent. Non-textual content like images, charts, and tables can hold valuable information, but without careful preparation, an LLM can ignore or misinterpret the information they contain.

Let’s start with images. If your documents contain any images, which might be a photo of a product or even very stylized text, it’s a great idea to convert that to plain text before uploading your file. Botpress will pre-process your files anyway once you upload them, so the best way to get consistent answers from your files is to convert them yourself.

Let’s take this stylized restaurant menu for example. Before converting it to a plaintext file, the information that the LLM has to work with looks like this, after it’s been parsed. Instead, if we convert it to markdown before uploading it (or using the built-in Botpress rich text editor), then we get much more reliable results.

Next, let’s talk about tables and structured data. If you’re including tables in your documents, remember that before performing RAG, your files are converted to markdown. Here, you have two options. You can designate a built-in Botpress table as a knowledge base, so your information is structured, or you can use a markdown-formatted table like so.

Optimizing non-textual content means processing images with OCR, adding descriptions for complex visuals, and ensuring tables are presented in a way that your AI agent can use. Here, our goal is to make the entire dataset—including both text and non-text content—easy for an LLM to read.

‍