How to Test a Haptik Chatbot

What Is a Haptik Chatbot?

Haptik builds Intelligent Virtual Assistants in a low-code Conversation Studio, a drag-and-drop canvas of conversational flows. The classic bot is intent-driven and scripted; the current generation adds Contakt, Haptik's generative AI suite driven by large language models. Testing the two halves means testing two different failure surfaces.

What a real Haptik chatbot is made of:

• Skills, FAQs, and Small Talk assembled in Conversation Studio, plus pre-built industry Smart Skills for retail, telecom, insurance, and more.
• An NLU layer that detects intent and steps and collects entities, with confidence scores exposed in the debug window.
• Contakt, the LLM-backed layer that answers from an ingested knowledge base for self-service and agent assist.
• Deployment channels including WhatsApp, RCS, Instagram, Messenger, and web, with multilingual support across 100-plus languages.

Because Contakt generates answers rather than reading them from a script, the same question phrased two ways can produce two different answers, only one of which is grounded. For the test types that apply to any bot, our library of chatbot test cases is a useful starting point.

Why Do Haptik Chatbots Pass the Test Bot but Fail in Production?

The Test Bot is a good build-time check. It is also a single conversation, typed by the builder, close to the path they had in mind. Most production failures live in the inputs nobody typed there, and Haptik's own debug guide names most of them.

What Should You Test in a Haptik Chatbot?

Whether the bot handles order tracking, insurance queries, or lending support, the same four dimensions decide whether it is ready for real users. Each needs a different kind of scenario, not just another manual Test Bot session.

How Do You Test a Haptik Chatbot, Step by Step?

Testing happens in three layers: what you check by hand in Conversation Studio, how you drive the deployed bot over the messaging API, and how you score that bot against the messy real-user traffic the first layer never simulates.

# A thin adapter exposes a simple message-in, text-out endpoint over Haptik's API
# POST /chat  ->  { "reply": "..." }
curl -X POST "https://your-haptik-adapter/chat" \
  -H "Content-Type: application/json" \
  -d '{"message": "track my order 48213"}'

# The adapter calls Haptik's synchronous webhook, decodes the reply,
# and returns { "reply": "Your order is out for delivery." }

Testing a Haptik Chatbot With TestMu AI's Agent Testing

Connecting Agent Testing to a Haptik bot does not require a platform SDK. It calls the adapter endpoint directly, adapting to its request shape with a body template and a response path, then layers three inputs on top before generating a full scenario set.

# Point the evaluator at the Haptik adapter endpoint
testmu-a2a test \
  --agent https://your-haptik-adapter/chat \
  --body-template '{"message": "{{message}}"}' \
  --response-path "reply" \
  --spec "Retail support bot that tracks orders, handles returns, and answers product FAQs" \
  --count 30

• Upload the same knowledge base or PRD the bot was built from, and the platform parses it for behavioral requirements and scenario seeds.
• Paste in the bot's instructions, so the evaluation baseline matches what the bot is actually told to do.
• Add focus areas that weight scenario generation toward what matters most this run, such as multilingual handling and Contakt grounding.

From there, autonomous evaluators run the generated scenarios against the endpoint and score each on the 9 quality metrics. A Contakt answer is checked against what the bot should actually know, not just whether it returned text, and every run rolls up into a Green, Yellow, or Red verdict, the same model our overview of AI agent testing walks through in depth.

Which Metrics Matter for a Haptik Chatbot?

A clean Test Bot session tells you one path worked. It does not tell you whether the answer was honest, complete, or safe on inputs you did not try. Agent Testing scores each conversation on 9 quality metrics that apply to any chat bot, tuned for the ways a multilingual, generative-answer bot goes wrong.

• Hallucination detection catches answers Contakt invented when its knowledge base returned weak or no grounding.
• Context awareness checks whether the bot held state across a long conversation, not just a two-turn Test Bot session.
• Completeness and root-cause understanding check whether the bot solved the real problem or a related one that looked close.
• Tone consistency and bias detection check for the same response quality regardless of the user's language or phrasing.

Every score also carries a confidence level, High, Medium, or Low, based on how many scenarios actually ran, so a Green built on ten scenarios is never mistaken for a Green built on three hundred. That distinction is what separates a demo that passed from a bot that is ready.

How Do You Red-Team a Haptik Chatbot?

The four dimensions cover a user trying to get help. Red-teaming covers a user trying to break the bot on purpose: override its instructions, exfiltrate data, or coax out PII. Haptik bots are exposed here because they collect PII and run business logic across public channels like WhatsApp. The testmu-a2a-cli runs adversarial probes against the endpoint, scoring categories including prompt injection, jailbreak, data exfiltration, and PII leakage at basic, intermediate, or advanced intensity.

testmu-a2a redteam \
  --agent https://your-haptik-adapter/chat \
  --intensity advanced \
  --spec "Haptik retail bot that collects order and contact details"

# Target specific attack categories only
testmu-a2a redteam \
  --agent https://your-haptik-adapter/chat \
  --categories prompt-injection,jailbreak,pii-leakage

The output is a letter grade, A+ to F, per category rather than one pass or fail, because a bot can resist a direct jailbreak and still leak data through a cleverly framed request. For any bot that collects customer PII, that per-category grade is the gap between a vulnerability you found and one an attacker finds.

How Do You Run Haptik Chatbot Tests in CI/CD?

A Skill edit, a knowledge base update, or a model change ships the moment you publish. The testmu-a2a-cli adds the endpoint-level sweep to your pipeline: it installs with pip, authenticates from CI secrets instead of an interactive login, and writes JUnit XML that GitHub Actions, GitLab CI, and Jenkins read natively.

# .github/workflows/haptik-chatbot-tests.yml
name: Haptik Chatbot Tests
on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - name: Install TestMu A2A CLI
        run: pip install testmu-a2a-cli
      - name: Run Haptik chatbot regression suite
        env:
          TESTMU_USERNAME: ${{ secrets.TESTMU_USERNAME }}
          TESTMU_ACCESS_KEY: ${{ secrets.TESTMU_ACCESS_KEY }}
        run: |
          testmu-a2a test \
            --agent ${{ vars.HAPTIK_ADAPTER }} \
            --body-template '{"message":"{{message}}"}' \
            --response-path "reply" \
            --spec "Haptik retail bot that tracks orders and answers product FAQs" \
            --format junit --output results.xml
      - name: Publish results
        uses: dorny/test-reporter@v1
        if: always()
        with:
          name: Haptik Chatbot Test Results
          path: results.xml
          reporter: java-junit

Gate on exit code: 0 for all scenarios passed, 1 for any failure. Run the full evaluation for Skill, knowledge base, or model changes, and a smaller smoke set for a change that does not touch bot behavior. The discipline is the same one covered in our guide to AI agent regression testing.

What Won't Automated Testing Catch?

A Green verdict means the bot passed on the scenarios you ran, with the personas you tested, on the metrics you configured. Here is what it does not cover:

• A low-confidence Green is not a cleared launch. It means too few scenarios ran to trust the score yet, so run more, especially on Contakt grounding and injection resistance, before you ship.
• Channel-specific behavior needs its own smoke test. WhatsApp session windows, template rules, and rich elements differ from the studio preview and from web.
• Every target language needs coverage. A Green in English says little about Hindi or Hinglish, so weight scenarios toward the languages your users actually use.
• Non-determinism means one clean run is not proof. Run the scenario set enough times to see the range of answers the bot actually gives.
• A stale spec gives you a stale verdict. If the bot's behavior changed and the prompt describing correct behavior did not, some regressions slip through.

None of this is a reason to skip testing. It is a reason to keep testing after launch, not just before it.

Conclusion

Take the Haptik chatbot you already have passing in the Test Bot. Point it at a harder question than you typed there: what happens when a user is angry, switches to Hinglish mid-sentence, asks something Contakt was never grounded on, or tries to talk the bot out of its rules. The Test Bot and debug logs catch the step and entity bugs. TestMu AI's Agent Testing catches the rest, running the deployed bot through hundreds of personas and adversarial probes and handing back a go-live verdict before a real user finds the gap first.

Point the evaluator at your endpoint, check the setup docs, and run your first evaluation before the next Skill or model change ships.