Pebblo Documentation

OpenSource Safe DataLoader for Gen AI applications

Project maintained by daxa-ai Hosted on GitHub Pages — Theme by mattgraham

Overview

Pebblo enables developers to safely load data and promote their Gen AI app to deployment without worrying about the organization’s compliance and security requirements. The project identifies semantic topics and entities found in the loaded data and summarizes them on the UI or a PDF report.

Pebblo Overview

Benefits

  1. Identify semantic topics and entities in your data loaded in RAG applications
  2. Accelerate time-to-production by effortlessly meeting your organization’s data compliance requirements
  3. Mitigate security risks arising from data poisoning and emerging threats.
  4. Comply with regulations such as the EU AI Act with custom reports and data records
  5. Support for a wide range of Gen AI development frameworks and data loaders

Components

Pebblo has two components.

  1. Pebblo Daemon - a REST api application with topic-classifier, entity-classifier and reporting
  2. Pebblo Safe DataLoader - a thin wrapper to Gen-AI framework’s data loaders

Pebblo Safe DataLoader currently support Langchain framework. Support for other frameworks like LlamaIndex, Haystack will be added in the upcoming releases.

Documentation