Stop messing with rigid platforms that create unnecessary dependencies. Autoblocks adapts to any codebase or tech stack, so you retain full control over your AI system.

Adapts to your AI system and workflows

Co-founder

Autoblocks supercharged our product velocity. It helps us understand user behavior, test changes to our AI, and ship with confidence. 

AI product development is incredibly collaborative and iterative. Give developers & PMs the tools they need to efficiently improve AI products.

Designed for product teams to collaborate

```python
from autoblocks.tracer import AutoblocksTracer
from openai import OpenAI

client = OpenAI()

tracer = AutoblocksTracer()

tracer.send_event("ai.request", properties=params)
completion = client.chat.completions.create(**params)
tracer.send_event("ai.response", properties=completion)
```

Python SDK

```typescript
import OpenAI from 'openai';
import { AutoblocksTracer } from '@autoblocks/client';

const openai = new OpenAI();

const tracer = new AutoblocksTracer();

tracer.sendEvent('ai.request', {
    properties: params,
});
const response = await openai.chat.completions.create(params);
tracer.sendEvent('ai.response', {
  properties: response,
});
```

TypeScript SDK

Tracer SDK

```python
from autoblocks.tracer import AutoblocksTracer
from openai import OpenAI

from src.prompts import TextSummarizationMinorVersion
from src.prompts import TextSummarizationPromptManager

openai = OpenAI()

tracer = AutoblocksTracer()
manager = TextSummarizationPromptManager(
    TextSummarizationMinorVersion.v0,
)

with manager.exec() as prompt:
        params = dict(
            model=prompt.params.model,
            temperature=prompt.params.temperature,
            max_tokens=prompt.params.max_tokens,
            messages=[
                dict(
                    role="system",
                    content=prompt.render.system(
                        language_requirement=prompt.render.util_language(
                            language="Spanish",
                        ),
                        tone_requirement=prompt.render.util_tone(
                            tone="formal",
                        ),
                    ),
                ),
                dict(
                    role="user",
                    content="\n\n".join(
                        [
                            prompt.render.user(document=document)
                            for document in ("doc1", "doc2", "etc")
                        ],
                    ),
                ),
            ],
        )

        tracer.send_event(
            "ai.request",
            properties=params,
        )

        response = openai.chat.completions.create(**params)

        tracer.send_event(
            "ai.response",
            prompt_tracking=prompt.track(),
            properties=response
        )
```

```typescript
import OpenAI from 'openai';
import { AutoblocksTracer } from '@autoblocks/client';
import { AutoblocksPromptManager } from '@autoblocks/client/prompts';

const openai = new OpenAI();

const tracer = new AutoblocksTracer();
const manager = new AutoblocksPromptManager({
  id: 'text-summarization',
  version: {
    major: '1',
    minor: '0',
  },
});

await manager.init();

const response = await manager.exec(async ({ prompt }) => {
    const params: ChatCompletionCreateParamsNonStreaming = {
      model: prompt.params.model,
      temperature: prompt.params.temperature,
      messages: [
        {
          role: 'system',
          content: prompt.render({
            template: 'system',
            params: {
              languageRequirement: prompt.render({
                template: 'util/language',
                params: {
                  language: 'Spanish',
                },
              }),
              toneRequirement: prompt.render({
                template: 'util/tone',
                params: {
                  tone: 'formal',
                },
              }),
            },
          }),
        },
        {
          role: 'user',
          content: ['doc1', 'doc2', 'etc']
            .map((doc) => {
              return prompt.render({
                template: 'user',
                params: {
                  document: doc,
                },
              });
            })
            .join('\n\n'),
        },
      ],
    };

    tracer.sendEvent('ai.request', {
      properties: params,
    });

    const response = await openai.chat.completions.create(params);

    tracer.sendEvent('ai.response', {
      properties: {
        response,
      },
      promptTracking: prompt.track(),
    });

    return response;
  });

```

Prompt SDK

```python
from typing import List

from autoblocks.testing.run import run_test_suite

from my_project.tasks import function_to_test
from my_project.evaluators import IsProfessionalTone
from my_project.evaluators import Correctness
from my_project.test_cases import TestCase
from my_project.test_cases import gen_test_cases


def test_fn(test_case: TestCase) -> List[Result]:
    return function_to_test(test_case)


run_test_suite(
    id="my-test-suite",
    test_cases=gen_test_cases(),
    evaluators=[
        IsProfessionalTone(),
        Correctness(),
    ],
    fn=test_fn,
)
```

```typescript
import { runTestSuite } from '@autoblocks/client/testing';
import { genTestCases, type TestCase } from './test-cases';
import { IsProfessionalTone, Correctness } from './evaluators';
import { functionToTest } from './my-app';


await runTestSuite<TestCase, Flashcard[]>({
  id: 'my-test-suite',
  testCases: genTestCases(),
  evaluators: [new IsProfessionalTone(), new Correctness()],
  fn: (args: { testCase: TestCase }) => functionToTest(args.testCase),
});

```

Testing SDK

Designed from the ground up with developer ergonomics in mind. Autoblocks empowers you to efficiently manage and improve AI products, without adding friction to your workflow.

Flexible developer tools

Expert interviews with teams from

Evaluation and testing solution for product teams building with LLMs. Measure how changes impact quality. Optimize, together.

Improve the reliability
of LLM-based products

Turbocharge your local testing & experimentation process to always put your best foot forward.

Testing & Evaluation

Configure online evaluations and guardrails to ensure a safe, trustful user experience.

Monitoring & Guardrails

Get down to the root cause of bugs and rapidly prototype solutions.

Debugging

Connect AI product state to user outcomes. Proactively uncover opportunities for improvement.

AI Product Analytics

Enable prompt collaboration, while ensuring your code doesn’t break.

Prompt Management

Optimize each part of your context pipeline to drive accurate and relevant outputs.

RAG & Context Engineering

One platform to continuously improve all your AI products

Intercom

Retool

OpenAI

Airtable

Rewind

Fireflies

Gamma

copy.io

Capsule

Data Indy

MultiOn

Kraftful

Independent Consultant

Midpage

laurel

Summit

Chatbase

Consumer

The Gamma team improved the quality of their AI-generated content, leading to a significant product experience boost as evidenced by user feedback.

Improved multi-modal output quality by 57%

Enterprise

The ClickHouse team was able to launch their query assistant in just 5 weeks instead of the initially planned 16 weeks.

Increased product velocity by 68%

Healthcare

A Series C virtual clinic was investing valuable engineering resources in managing an in-house solution. With the help of Autoblocks, they were able to free up developer bandwidth to concentrate on enhancing the member experience.

Decreased engineering overhead by 42%

Powered by Autoblocks

Engineered to satisfy the most rigorous privacy and security requisites.

Scale with security

Quickly adapt to the breakneck speed of AI. Unlike rigid tools that create unnecessary dependencies, Autoblocks adapts to any codebase or tech stack.

Scales with you, securely

Product

After evaluating several vendors, it became apparent that Autoblocks offers the most enterprise-grade platform in the market.

Improve your
LLM Product Reliability

Trusted by top AI product teams

Bring your entire team together with human-driven testing & evaluation

Better tests. Better products.

Curate high-quality test datasets

Experiment, collaboratively

Align evaluation metrics

Integrate easily with any codebase, any framework

Empower subject-matter experts and non-engineer stakeholders

Everything you need to ensure the reliability of your LLM-based products

Testing & Evaluation

Monitoring & Guardrails

Debugging

AI Product Analytics

Prompt Management

RAG & Context Engineering

Scale with Security

Improve your LLM Product Reliability

Trusted by top AI product teams

Bring your entire team together with human-driven testing & evaluation

Better tests. Better products.

Curate high-quality test datasets

Experiment, collaboratively

Align evaluation metrics

Integrate easily with any codebase, any framework

Empower subject-matter experts and non-engineer stakeholders

Everything you need to ensure the reliability of your LLM-based products

Testing & Evaluation

Monitoring & Guardrails

Debugging

AI Product Analytics

Prompt Management

RAG & Context Engineering

Scale with Security

Improve your
LLM Product Reliability