DataBonsai

DataBonsai is a Python library that utilizes large language models (LLMs) to perform data cleaning tasks. It provides a range of tools, including data classification, transformation and extraction, as well as validation of LLM output. Supports batching to save tokens and has retry logic to handle rate limiting and transient errors.

Demand group:

Data Scientist: Can quickly classify and clean large amounts of data for further analysis.

Developers: Ability to integrate into applications and automate data preprocessing processes.

Enterprise users: Improve data processing efficiency and reduce costs through automated data cleaning.

Example of usage scenario:

Classification and sentiment analysis of social media comments.

Automatic archiving and topic classification of news articles.

Collation and extraction of customer feedback data for product improvement.

Product features:

Data Classification: Use LLMs to classify data into predefined categories.

Data transformation: Transform data through prompts.

Data Extraction: Extract data into a structured format based on patterns.

Batch processing: Save tokens and classify a batch of data by sending patterns and examples only once.

Retry logic: Built-in retry logic for handling API-related errors.

Progress Bar: Provides progress feedback when processing large amounts of data.

Automatic batching: Automatically adjust batch sizes to optimize token usage and error handling.

Usage tutorial:

1. Install the DataBonsai library.

2. Create an .env file containing the API key in the project root directory.

3. Set up the LLM provider and category.

4. Use the categorize function to classify a single piece of data.

5. Use the categorize_batch function to classify data batches.

6. Use the applytocolumn_autobatch function to automatically batch the DataFrame or list.

7. Monitor the progress bar to understand the current processing progress.

8. When you encounter an error, adjust the batch size or use a better LLM model as needed.