How to Configure AI Auto-Reply in Heedback
· 8 min read · Heedback Team
Most support teams answer the same types of questions over and over — billing inquiries, setup instructions, troubleshooting steps that are already documented somewhere. Heedback’s AI Auto-Reply feature drafts responses by pulling from your knowledge base articles, so your team spends less time writing and more time solving the issues that actually require human judgment.
The AI doesn’t send replies on its own. It generates a draft that an agent reviews, edits if needed, and sends. This keeps your team in control while cutting response time significantly.
Prerequisites
Before enabling AI Auto-Reply, you need:
- A Pro plan on Heedback (AI features are part of the Pro tier).
- An API key from either OpenAI or Anthropic. You’ll enter this in your organization settings — Heedback never stores your key beyond your encrypted org config.
- A knowledge base with published articles. The AI generates answers by searching your articles using retrieval-augmented generation (RAG). Without articles, it has nothing to draw from. Even 10–15 well-written articles covering your most common questions will make a noticeable difference.
Step 1: Connect Your AI Provider
Navigate to Settings → AI Configuration in your Heedback dashboard at app.heedback.io. You’ll see two supported providers:
- OpenAI — paste your API key from platform.openai.com. Heedback uses GPT-4o by default for the best balance of quality and speed.
- Anthropic — paste your API key from console.anthropic.com. Heedback uses Claude for response generation.
Choose the provider you prefer, enter your key, and click Save. Heedback will validate the key immediately and confirm the connection.
You can switch providers at any time without losing any configuration. The AI drafts are generated on demand, so there’s no migration or retraining involved.
Step 2: Enable Auto-Reply in Your Inbox Settings
With your AI provider connected, go to Settings → Inbox and toggle AI Auto-Reply on. You can configure:
- Auto-draft on new conversations — when enabled, every new incoming message automatically triggers an AI draft. The draft appears in the conversation thread as a suggestion, clearly labeled as AI-generated.
- Confidence threshold — set how confident the AI must be before generating a draft. A higher threshold means fewer but more accurate drafts. Start with the default (medium) and adjust based on your experience.
- Tone and style — provide optional instructions like “Be concise and friendly” or “Use formal language” to shape how the AI writes. These instructions apply to all drafts across your organization.
Step 3: Understand How RAG Works Behind the Scenes
When a customer sends a message, Heedback’s AI pipeline does the following:
- Analyzes the customer’s question to understand intent and extract key terms.
- Searches your knowledge base using semantic search to find the most relevant articles. This isn’t simple keyword matching — the AI understands meaning, so a question about “canceling my plan” will match an article titled “How to manage your subscription.”
- Generates a draft response grounded in the matched articles. The AI cites specific information from your docs rather than inventing answers, which dramatically reduces hallucination risk.
- Presents the draft to your agent in the conversation thread with a clear “AI Draft” label and links to the source articles it referenced.
This retrieval-augmented approach means the AI is only as good as your knowledge base. If an answer exists in your articles, the AI will find it. If it doesn’t, the AI will either indicate low confidence or skip the draft entirely — it won’t fabricate information.
Step 4: Review and Send AI Drafts
When you open a conversation with an AI draft, you’ll see the suggested response in the reply box. From here:
- Send as-is if the draft is accurate and complete. One click.
- Edit before sending to add personalization, adjust tone, or append additional context the AI couldn’t know (like account-specific details).
- Dismiss the draft if it’s off-target. This is useful feedback — see the next step.
Most teams find that 60–70% of AI drafts need only minor edits or none at all, especially for well-documented topics. The remaining drafts either need significant editing or dismissal, which is expected for complex or novel issues.
Step 5: Train the AI Through Corrections
Every time you edit or dismiss an AI draft, you’re providing implicit feedback that improves future performance. Heedback uses this feedback loop in two ways:
- Knowledge gap detection — if the AI frequently fails on a specific topic, Heedback surfaces that topic as a “suggested article” in your knowledge base dashboard. Writing that article closes the gap for both AI and self-service.
- Style calibration — consistent edits (e.g., always making responses shorter, always adding a greeting) help the system learn your team’s preferred communication style over time.
You can also directly improve results by enriching your knowledge base. When you notice the AI struggling with a category of questions, write or update the relevant articles. The improvement is immediate — new articles are indexed in real time.
Tips and Best Practices
- Start with a solid knowledge base. AI Auto-Reply amplifies what’s already documented. If your articles are thin or outdated, the AI drafts will reflect that. Invest in your top 20 articles first.
- Don’t skip the review step. The AI is a drafting assistant, not an autonomous agent. Always have a human review before sending, especially in the first few weeks while you calibrate confidence thresholds.
- Monitor your AI usage costs. Each draft consumes tokens from your OpenAI or Anthropic account. Heedback shows estimated token usage in Settings → AI Configuration so you can track spend.
- Use tone instructions strategically. Generic instructions like “be helpful” add little value. Specific ones like “Keep responses under 3 sentences when possible” or “Always end with an offer to help further” produce noticeably better drafts.
- Combine with quick replies. For questions the AI handles poorly but humans answer easily, a quick reply template is often more efficient than waiting for AI improvement.
Related Features
- Knowledge Base — the foundation that powers AI Auto-Reply. Better articles mean better drafts.
- Support Inbox — where AI drafts appear and agents manage conversations.
- AI Memory — lets the AI remember context from previous interactions for more personalized responses.
- Embeddable Widget — the primary channel where customers submit messages that trigger AI drafts.