Table of Contents

Image Description

Learn what the Management Portal offers today for processing images in data pipelines.

Overview

Data pipelines turn documents into searchable text with the Text Extraction Data Pipeline Stage (FoundationaLLM) plugin. That stage selects a content text extraction plugin, configured as a stage dependency, based on the file type.

The standard FoundationaLLM data pipeline plugin package (.NET) lists two image-related content text extraction plugins:

Plugin Purpose Parameter File types
Image Content Text Extraction (FoundationaLLM) Extract text from image files with an LLM. AIModelObjectId: the AI model used to extract text. JPG, JPEG, GIF, PNG, WEBP, TIFF, BMP
Image Metadata Text Extraction (FoundationaLLM) Extract a description from image files with an LLM. AIModelObjectId: the AI model used to extract the description. JPG, JPEG, GIF, PNG, WEBP, TIFF, BMP

Current Limitation

In the current release, the text extraction stage does not process image files. It handles only PDF, DOCX, PPTX, and XLSX files (through dependency plugins) and plain text, code, and data files (read directly). Image files are reported as an unsupported file type for that work item, even when an image plugin is selected as a dependency, and the image plugins themselves are not yet implemented.

As a result, images in a data source are not described or indexed by data pipelines today.

Configure the Text Extraction Stage

Although image processing isn't available yet, this is how you choose content text extraction plugins for a stage:

  1. Open a data pipeline for editing and select the Stages tab.
  2. Select the text extraction stage on the canvas, or select Add Stage to create one.
  3. In the stage dialog, select Text Extraction Data Pipeline Stage (FoundationaLLM) in the Plugin dropdown.
  4. Under Dependencies, select the content text extraction plugins to use (for example, PDF, DOCX, PPTX, and XLSX). You can select more than one.
  5. If a dependency has parameters, set them under {dependency} Dependency Parameters.
  6. Optionally set the stage's MaxContentSizeCharacters parameter. Extracted text longer than this limit is discarded with a warning.

See Creating Data Pipelines for the full editor walkthrough.