The Challenge

Integrating GenAI required a new approach

ClickHouse wanted to integrate generative AI to power natural language query suggestions in ClickHouse Cloud. This presented three key challenges:

• Quality. Query suggestions needed to be highly accurate based on table schema and context. Off-the-shelf solutions weren't optimized for this use case.
• Product velocity. Rapid iteration was critical during the development process. The team needed to quickly test different prompts, models, and approaches.
• Scale. Tooling needed to be adaptable enough to handle their scale and evolving needs, without adding more dependencies.

We didn't have a solid framework in place for things like prompt testing, model monitoring, and iteration. It felt like taking shots in the dark without the right tools.

The Solution

Agile development of GenAI features

Autoblocks provided the infrastructure ClickHouse needed to build and refine its query generation AI product. With Autoblocks, ClickHouse can:

• Collaboratively share insights about user behavior,
• Monitor model behavior and quickly debug issues,
• Understand the efficacy of the retrieval mechanism (RAG),
• Track key metrics like latency and token usage, and
• Streamline prompt engineering and testing.

The brilliance of Autoblocks is its adaptability. We just plugged it into a few places in our existing codebase, and it immediately gave us a ton of value.

The Results

• 10x faster prototyping of the query generation product
• 2x improvement in the accuracy of queries
• Production roll-out in just 3 months