A/B testing chat assistant versions on live conversations

The scenario splits users between different assistant versions and answers each with their version, to fairly compare which one performs better.

  • Comparing versions on real conversations
  • Each user is locked to one version
  • Decisions by numbers, not by gut
  • Remembers the conversation flow