The Challenge

Integrating GenAI required a new approach

ClickHouse wanted to integrate generative AI to power natural language query suggestions in ClickHouse Cloud. This presented three key challenges:

• Quality. Query suggestions needed to be highly accurate based on table schema and context. Off-the-shelf solutions weren't optimized for this use case.
• Product velocity. Rapid iteration was critical during the development process. The team needed to quickly test different prompts, models, and approaches.
• Scale. Tooling needed to be adaptable enough to handle their scale and evolving needs, without adding more dependencies.

"We didn't have a solid framework in place for things like prompt testing, model monitoring, and iteration. It felt like taking shots in the dark without the right tools." - Zach Naimon, Product

The Solution

Agile development of GenAI features

Autoblocks provided the infrastructure ClickHouse needed to build and refine its query generation AI product. With Autoblocks, ClickHouse can:

• Collaboratively share insights about user behavior,
• Monitor model behavior and quickly debug issues,
• Understand the efficacy of the retrieval mechanism (RAG),
• Track key metrics like latency and token usage, and
• Streamline prompt engineering and testing.

"The brilliance of Autoblocks is its adaptability. We just plugged it into a few places in our existing codebase, and it immediately gave us a ton of value." - Alexey Korepanov, Engineering

The Impact

• 10x faster prototyping of the query generation product
• 2x improvement in the accuracy of queries
• Production roll-out in just 3 months