Taalk Help Center

Test Plugins and Agents with Sample Data

Sample Data lets you try a plugin on real calls before a change goes live, so you know it still does the right thing for every kind of call.

Why it matters

Before: when someone changed a plugin's AI questions or its JS, the only way to know it worked was to save it, wait for real calls, and hope. If it broke, the customer usually noticed first, for example when their CRM got "Spoke to Contact" for a voicemail. Debugging meant digging through call logs by hand.

With Sample Data:

Without it With it
Change a prompt or JS and hope Change it, click Check all, see what moved
Find out from the customer that something broke Find out before you save
Testing a voicemail means placing a call that goes to voicemail Change one field, click Try it
"Does it handle a dropped call?" is a guess You can see which outcomes have samples and which don't
Only the person who wrote the JS can check it Anyone on the team runs the same checks the same way

Everyone in your workspace sees the same Sample Data, so a case one teammate adds is there for everyone.

What it doesn't do: it doesn't place calls, send anything to customer systems, or test how the agent talks on the phone. It tests the step after the call: given what happened, did the plugin decide correctly and send the right thing?

How it works

  1. Real calls become test cases. Paste call links, or pull the plugin's recent calls. Each one is saved with what happened on that call and what the plugin sent, which becomes the expected result.
  2. One click checks everything. Open the plugin and every sample runs again in a few seconds, in your browser. Nothing is sent and no AI is used. Each sample shows pass or fail, with a one-line reason such as Taalk_ContactOutcome: expected Unable to Contact, got Spoke to Contact.
  3. It shows what you haven't tested. Samples are grouped by the outcome they expect. When an outcome has no sample, for example Left a Message, the panel says so and links to From calls.
  4. "What if" without making calls. Open a sample, change what the AI concluded or a call fact like the voicemail length, and click Try it. Variations does this for you: it changes one thing at a time and shows which changes move the result.

Read the Sample Data panel

Open a plugin: Outreach › AI Agents › Plugins, then scroll to Sample Data.

  • The status line says whether every checked sample passes, and when it was checked. If the plugin has 30 samples or fewer, they're checked as soon as the page opens. Click Check all to check again, for up to 100 samples.
  • Needs attention appears at the top when something is off:
    • samples that no longer give the result they expect. Open one to see what changed: either the plugin broke, or the expected result is out of date;
    • samples with no expected result, which can never fail.
  • Each action lists its samples by the outcome they expect, with pass and fail counts. Groups with a failure open by themselves.
  • Dashed outcomes have no sample yet. Click From calls on that line and paste a call that ended that way.
  • Stops, nothing sent groups samples where the action should stop on purpose, for example a step that only acts on Spoke to Contact.

Each row has buttons to open the source call, run Variations, Edit, Duplicate and delete.

Import real calls

  1. In Sample Data, click From calls.
  2. Paste call links or call IDs, one per line (up to 25), and click Find runs. Or click Use recent calls to check the latest finished calls of the agents that use this plugin and pull up to 10 where it ran.
  3. Each run is listed with its action, the live result and Match or Differs: whether the preview replays the live call exactly. Runs that differ are unchecked, so they don't become wrong tests.
  4. Click Save. Each sample expects what the live call did.

When a call was classified wrong, edit that sample's expected result to the right answer. It fails until the plugin is fixed.

Test with different data: Quick edit

Open a sample with Edit, or Duplicate to keep the original. The dialog opens on Quick edit:

  • AI answers: one field per question in the action's extraction prompt. Questions with listed answers become dropdowns.
  • Call facts the JS uses: only the facts this action's JS reads, such as callStatus, durationVM or durationAfterTransfer. Durations are in milliseconds.
  • Expected result: what the sample should give. Changing answers does not change it; set it yourself.
  • Try it: runs the data through the action as it is now and shows what it gives, and whether that matches. If the new result is right, click Use this result as expected.

Click Save to keep it. The Input and Expected tabs still hold the full data as JSON if you need it.

When a sample fails

  1. Read the reason on the row, for example UTCReason: expected Answered - Dropped, got Other.
  2. Open the sample in Quick edit and find the AI answer or call fact that feeds that field.
  3. Change it to what the call really said and click Try it.
    • It matches now: the sample's saved data was wrong. Save it.
    • The reason changes to another field: keep going. Fix that one too and try again.
    • It still fails on the same field: the plugin is making the decision. Look at the call facts and the JS.

Try variations

Click Variations on a sample. Taalk changes one thing at a time, such as each AI answer or a call fact like a voicemail, a live transfer or a busy line, and runs the action again. You see which changes move the result, for example list_accepted = false → Unable to Contact. Save the ones that matter as samples. Nothing is sent and the plugin is not changed.

A variation changes one thing, so it tests the rules near that call's outcome. Run it on one call of each outcome to cover the main rules.

Run while you edit an action

Open an action on the plugin page. The Run with Sample Data bar at the bottom runs your unsaved edits:

  • Run preview runs one sample and shows each step, the checks, the webhook request (not sent) and a side-by-side with the live call.
  • Run all runs every sample for that action.
  • Run with AI (off by default) runs the extraction prompt on the sample's transcript instead of the saved AI answers. It approximates live calls. Each sample is one AI call, so Run all asks first.

Use samples in agent test calls

  1. Open an agent. Add samples in Sample Data in the Knowledge and Skills section.
  2. Click Test, open the advanced options, and pick a sample in Load sample next to Contact data. Samples from the agent's plugins are listed too.
  3. Click Save as sample to keep the contact data you typed.

Good to know

  • A browser preview doesn't call your webhook and can differ slightly from a live call. Read the JS step's warnings as well as pass or fail.
  • When two actions on a plugin ask the AI the same questions at the same moment, the call log keeps one set of answers. A sample imported from such a call can then hold the other action's answers and fail even though the live call was right. From calls flags these as Differs. To fix one, set the right AI answers in Quick edit.
  • Samples imported from calls hold real contact details. They stay in your workspace.
  • Each sample can be up to 256 KB.
  • Developers can push and pull Sample Data through the API (/api/plugins/{id}/fixtures and /api/agents/{id}/fixtures), up to 100 samples per request.

Next

Follow the hands-on Tutorial: Fix a Plugin Without Losing the Call.