TracePilot AI with AutoGen and custom orchestrators

Trace any async orchestration system — AutoGen, custom loops, or your own framework — using wrapOpenAI for LLM calls and wrapToolCall for everything else.

TracePilot’s rule is simple: if your code makes an async call, you can wrap it. Use tp.wrapOpenAI for any call to OpenAI’s chat completions API, and tp.wrapToolCall for any other async operation you want to trace — web searches, database queries, external API calls, file reads, or anything else. This works with AutoGen, custom agent loops, event-driven systems, or any orchestration pattern you’ve built.

Core pattern

Every wrapped call returns { result, spanId }. Pass spanId as parentSpanId to subsequent calls to build a parent-child span tree in the dashboard. The result is the original return value, unchanged.

import { TracePilot } from 'tracepilot-sdk';
import OpenAI from 'openai';

const tp = new TracePilot(process.env.TRACEPILOT_API_KEY!);
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

async function runAgent(userInput: string) {
  await tp.startTrace('my-agent');

  const messages = [{ role: 'user' as const, content: userInput }];

  // Wrap any async call — result is unchanged, spanId links spans together
  const { result, spanId } = await tp.wrapOpenAI(
    () => openai.chat.completions.create({ model: 'gpt-4o-mini', messages }),
    messages,
    undefined, // parentSpanId — undefined means this is the root span
    1,
  );

  console.log(result.choices[0].message.content);
}

Patterns

Sequential steps

Each step passes its spanId as parentSpanId for the next step, creating a chain of linked spans.

async function sequentialAgent(query: string) {
  await tp.startTrace('sequential-agent');

  const step1Messages = [{ role: 'user' as const, content: query }];

  // Step 1 — Plan
  const { result: plan, spanId: planSpanId } = await tp.wrapOpenAI(
    () =>
      openai.chat.completions.create({
        model: 'gpt-4o',
        messages: step1Messages,
      }),
    step1Messages,
    undefined,
    1,
  );

  // Step 2 — Fetch data based on the plan
  const { result: data, spanId: dataSpanId } = await tp.wrapToolCall(
    'fetch-data',
    () => fetchFromDatabase(plan.choices[0].message.content!),
    planSpanId, // child of the planning step
    2,
  );

  // Step 3 — Synthesize using fetched data
  const step3Messages = [
    ...step1Messages,
    { role: 'assistant' as const, content: plan.choices[0].message.content! },
    { role: 'user' as const, content: `Use this data: ${JSON.stringify(data)}` },
  ];

  const { result: answer } = await tp.wrapOpenAI(
    () =>
      openai.chat.completions.create({
        model: 'gpt-4o',
        messages: step3Messages,
      }),
    step3Messages,
    dataSpanId, // child of the data-fetch step
    3,
  );

  return answer.choices[0].message.content;
}

async function fetchFromDatabase(query: string): Promise<Record<string, unknown>> {
  // Replace with your real database client
  return { records: [], query };
}

Parallel steps

When steps are independent, run them in parallel with Promise.all. Each gets its own span linked to the same parent, so they appear as siblings in the dashboard.

async function parallelAgent(topic: string) {
  await tp.startTrace('parallel-research-agent');

  const rootMessages = [{ role: 'user' as const, content: `Research: ${topic}` }];

  // Root planning span
  const { result: plan, spanId: rootSpanId } = await tp.wrapOpenAI(
    () =>
      openai.chat.completions.create({
        model: 'gpt-4o-mini',
        messages: rootMessages,
      }),
    rootMessages,
    undefined,
    1,
  );

  // Run two independent tool calls in parallel — both are children of rootSpanId
  const [{ result: webData }, { result: dbData }] = await Promise.all([
    tp.wrapToolCall(
      'web-search',
      () => searchWeb(topic),
      rootSpanId, // same parent
      2,
    ),
    tp.wrapToolCall(
      'database-lookup',
      () => lookupInternalData(topic),
      rootSpanId, // same parent
      3,
    ),
  ]);

  // Final synthesis step using all gathered data
  const synthMessages = [
    ...rootMessages,
    {
      role: 'user' as const,
      content: `Web: ${JSON.stringify(webData)}\nDB: ${JSON.stringify(dbData)}`,
    },
  ];

  const { result: summary } = await tp.wrapOpenAI(
    () =>
      openai.chat.completions.create({
        model: 'gpt-4o',
        messages: synthMessages,
      }),
    synthMessages,
    rootSpanId,
    4,
  );

  return summary.choices[0].message.content;
}

async function searchWeb(query: string) {
  return { results: [`Web result for: ${query}`] };
}

async function lookupInternalData(query: string) {
  return { records: [`Internal record for: ${query}`] };
}

Nested tool calls

Tools can call other tools. Each nested call gets the parent tool’s spanId, building a multi-level tree in the dashboard.

async function nestedAgent(task: string) {
  await tp.startTrace('nested-agent');

  const messages = [{ role: 'user' as const, content: task }];

  // Root LLM call
  const { result: decision, spanId: rootSpanId } = await tp.wrapOpenAI(
    () =>
      openai.chat.completions.create({
        model: 'gpt-4o',
        messages,
      }),
    messages,
    undefined,
    1,
  );

  // Parent tool call
  const { result: report, spanId: reportSpanId } = await tp.wrapToolCall(
    'generate-report',
    () => generateReport(decision.choices[0].message.content!),
    rootSpanId,
    2,
  );

  // Child tool calls — nested under generate-report
  const [{ result: formatted }, { result: saved }] = await Promise.all([
    tp.wrapToolCall(
      'format-markdown',
      () => formatAsMarkdown(report as string),
      reportSpanId, // child of generate-report
      3,
    ),
    tp.wrapToolCall(
      'save-to-storage',
      () => saveToStorage(report as string),
      reportSpanId, // child of generate-report
      4,
      true, // isDestructive — marks this span with a ⚠ badge
    ),
  ]);

  return { formatted, saved };
}

async function generateReport(content: string): Promise<string> {
  return `# Report\n\n${content}`;
}

async function formatAsMarkdown(content: string): Promise<string> {
  return content.trim();
}

async function saveToStorage(content: string): Promise<{ id: string }> {
  // Replace with your real storage client
  return { id: `doc_${Date.now()}` };
}

AutoGen compatibility

AutoGen uses async function calls internally. Wrap at the same level — intercept the OpenAI call inside your AutoGen agent’s reply function, or wrap the tool functions your agent registers.

import { ConversableAgent } from 'autogen';

// Wrap the underlying OpenAI call inside your AutoGen reply function
const tracedReplyFunction = async (
  messages: { role: string; content: string }[],
  parentSpanId?: string,
  stepOrder?: number,
) => {
  const { result, spanId } = await tp.wrapOpenAI(
    () =>
      openai.chat.completions.create({
        model: 'gpt-4o-mini',
        messages,
      }),
    messages,
    parentSpanId,
    stepOrder,
  );

  return { content: result.choices[0].message.content, spanId };
};

// Register it on your AutoGen agent
const agent = new ConversableAgent({
  name: 'traced-agent',
  llmConfig: { model: 'gpt-4o-mini' },
});

// Use tracedReplyFunction wherever your agent generates replies

AutoGen support is available for any async call pattern. If AutoGen exposes a custom reply function hook in your version, use tracedReplyFunction there. Refer to your AutoGen version’s docs for the exact hook name.

Building a custom orchestration loop

If you’re running your own agent loop, wrap every call inside the loop body and thread spanId values forward.

async function agentLoop(goal: string, maxSteps = 5) {
  await tp.startTrace('agent-loop');

  let messages = [{ role: 'user' as const, content: goal }];
  let parentSpanId: string | undefined;

  for (let step = 1; step <= maxSteps; step++) {
    // LLM decides what to do next
    const { result: decision, spanId } = await tp.wrapOpenAI(
      () =>
        openai.chat.completions.create({
          model: 'gpt-4o',
          messages,
        }),
      messages,
      parentSpanId,
      step,
    );

    const content = decision.choices[0].message.content!;

    // Check if the agent has finished
    if (content.includes('[DONE]')) {
      break;
    }

    // Execute the tool the agent requested
    const toolName = parseToolName(content); // your parsing logic
    const toolArgs = parseToolArgs(content);

    const { result: toolResult, spanId: toolSpanId } = await tp.wrapToolCall(
      toolName,
      () => executeTool(toolName, toolArgs),
      spanId, // child of this step's LLM span
      step,
    );

    // Feed the tool result back into the conversation
    messages = [
      ...messages,
      { role: 'assistant' as const, content },
      { role: 'user' as const, content: `Tool result: ${JSON.stringify(toolResult)}` },
    ];

    parentSpanId = toolSpanId; // next iteration links to this tool call
  }
}

function parseToolName(content: string): string {
  // Replace with your parsing logic
  return content.split(':')[0].trim();
}

function parseToolArgs(content: string): Record<string, unknown> {
  return {};
}

async function executeTool(
  name: string,
  args: Record<string, unknown>,
): Promise<unknown> {
  // Dispatch to your tool implementations
  return { tool: name, args };
}

The key rule: wrap any async call you want to trace, and pass parentSpanId to link related spans. Even if you only wrap some calls, those spans still appear in the dashboard — you don’t need to wrap everything at once.