AI Agent Integration

Track AI model performance, intent classification, and errors from your LLM-powered agents.

Overview

AI analytics events allow you to track how your AI/LLM components are performing. These events integrate with WhatsApp analytics to show the complete user journey.

ai.classification - When your AI classifies user intent
ai.generation - When your AI generates a response
ai.error - When the AI fails or times out

`ai.classification`

Fired when your AI classifies user intent.

{
  "type": "track",
  "event": "ai.classification",
  "userId": "+254712345678",
  "properties": {
    "input_text": "I want to file nil returns",
    "detected_intent": "nil_filing",
    "confidence": 0.99,
    "latency_ms": 450,
    "prompt_tokens": 120,
    "completion_tokens": 15,
    "model": "llama3.2:3b"
  },
  "context": {
    "channel": "whatsapp"
  }
}

Properties

Property	Type	Description
`detected_intent`	string	The classified intent (e.g., "nil_filing", "pin_registration")
`confidence`	float	Confidence score 0-1
`latency_ms`	int	Time taken for inference in milliseconds
`model`	string	Model identifier (e.g., "llama3.2:3b")
`prompt_tokens`	int	Input tokens used (for cost tracking)

`ai.generation`

Fired when your AI generates a text response.

{
  "type": "track",
  "event": "ai.generation",
  "userId": "+254712345678",
  "properties": {
    "prompt_type": "clarification",
    "output_length": 45,
    "latency_ms": 800,
    "model": "llama3.2:3b"
  }
}

`ai.error`

Fired when the AI encounters an error.

{
  "type": "track",
  "event": "ai.error",
  "userId": "+254712345678",
  "properties": {
    "error_type": "json_parse_error",
    "recovery_attempt": 1,
    "recovered": false,
    "fallback_action": "agent_handoff"
  }
}

Error Types

timeout - Model took too long to respond
json_parse_error - Output wasn't valid JSON
network - Network error reaching model
context_length - Input exceeded model context

Dashboard Metrics

Once you send AI events, you'll see these metrics in the WhatsApp Analytics dashboard:

Metric	Description
AI Classifications	Total number of intent classifications
AI Accuracy	Average confidence score across classifications
Avg Latency	Average inference time in milliseconds
Error Rate	Percentage of AI requests that failed
Top Intents	Most common user intents detected
Latency Distribution	Histogram of inference times