Support & Documentation

Everything you need to get started with AI Ticket Classifier

Getting Started

1Install the App

Install AI Ticket Classifier from the Atlassian Marketplace. You'll need Jira administrator permissions to install apps.

2Get a Claude API Key

The App requires your own Anthropic (Claude) API key:

Anthropic: Get a key at console.anthropic.com/settings/keys

Note: API usage costs are charged by Anthropic based on your usage. Classification typically uses 500-1500 tokens per ticket.

3Configure Settings

Open the App in Jira, go to the Settings tab, and:

Paste your Claude API key
Click Test to verify the connection
Select which projects to enable classification for
Configure classification fields (issue type, priority, assignee)

Your API key is stored encrypted in Forge Storage and never sent to our servers.

4Train the Classifier

Before the classifier can work, it needs to learn from your existing tickets:

Go to the Training tab in the Admin Dashboard
Select a project to train on
Choose how many historical tickets to analyze (recommended: 100-500)
Click Start Training
Wait for the AI to learn your team's classification patterns

Training typically takes 2-5 minutes depending on the number of tickets.

Configuration

Classification Fields

You can configure which fields the classifier suggests values for:

Issue Type: Bug, Story, Task, Epic, etc.
Priority: Highest, High, Medium, Low, Lowest
Assignee: Team member best suited based on expertise
Labels: Custom labels based on ticket content

Confidence Thresholds

Set minimum confidence levels for auto-applying classifications:

Auto-apply: Classifications above this threshold are applied automatically (default: 85%)
Suggest: Classifications between suggest and auto-apply thresholds are shown as suggestions (default: 50%)
Skip: Classifications below the suggest threshold are not shown

Project Settings

Each project can have its own configuration:

Enable or disable classification per project
Choose which fields to classify
Set project-specific confidence thresholds
Configure auto-apply behavior

How Classification Works

The Classification Process

A new ticket is created in a configured project
The App reads the ticket's summary and description
Ticket content is sent to Claude API along with learned classification patterns
Claude returns suggested issue type, priority, and assignee with confidence scores
Based on your threshold settings, suggestions are either auto-applied or shown for review

Training

The classifier learns from two sources:

Historical tickets: Initial training uses your existing tickets to learn patterns
Feedback: When team members accept or correct classifications, the model improves

You can retrain the classifier at any time from the Admin Dashboard (Premium tier).

Feedback Loop

After a ticket is classified, team members can:

Accept: Confirm the classification is correct (reinforces the pattern)
Correct: Change the classification to the right value (teaches the model)
Dismiss: Remove the suggestion without feedback

Admin Dashboard

Analytics Overview

The Admin Dashboard shows:

Classification accuracy: Percentage of correct classifications over time
Volume metrics: Number of tickets classified per day/week/month
Category breakdown: Distribution of issue types, priorities, and assignees
Confidence distribution: How confident the classifier is on average
Feedback stats: Acceptance rate and correction patterns

Managing Training Data

From the Admin Dashboard you can:

View current training status and pattern count
Retrain the classifier with new historical data (Premium)
Reset training data to start fresh
Export analytics data

Frequently Asked Questions

Is my data safe?

Yes. The App runs on Atlassian's secure Forge platform. Classification patterns are stored in Forge Storage within Atlassian's infrastructure. Ticket content is sent to the Claude API only for classification. No data is stored on our servers.

How much does the AI cost?

AI costs depend on your Anthropic usage. Claude typically processes a ticket classification for $0.001-0.005 per ticket. For a team classifying 1,000 tickets per month, expect roughly $1-5/month in API costs. Check Anthropic's pricing page for current rates.

How accurate is the classification?

Accuracy depends on the quality and quantity of your training data. With 200+ well-categorized historical tickets, most teams see 75-90% accuracy within the first week. Accuracy improves over time as the feedback loop refines the model.

Can I undo a classification?

Yes. Classifications applied by the App can be changed just like any other field edit in Jira. The original values are preserved in Jira's issue history. You can also correct classifications through the feedback interface, which also helps improve future accuracy.

Why isn't my API key working?

Check that:

The key is copied correctly with no extra spaces
Your Anthropic account has available credits/quota
The key hasn't been revoked or expired
You have the correct API key format (starts with sk-ant-)

Use the Test button in Settings to verify your connection.

Does it work with all Jira project types?

AI Ticket Classifier works with all standard Jira Cloud project types including Scrum, Kanban, and company-managed projects. It classifies any issue type available in your project's configuration.

Can I use this with Jira Data Center?

No, AI Ticket Classifier is only available for Jira Cloud. It's built on Atlassian Forge, which is a cloud-only platform.

What happens when I uninstall the app?

All data stored in Forge Storage (classification patterns, feedback, configuration) is automatically deleted when you uninstall the App. Your Jira tickets and any classifications already applied to them remain unchanged.

Troubleshooting

Classifications are not appearing

Verify the project has classification enabled in Settings
Check that your API key is valid (use the Test button)
Ensure the classifier has been trained for the project
Check that you haven't exceeded your monthly classification limit (Free tier)

Classification accuracy is low

Train with more historical tickets (recommend 200+)
Provide feedback on classifications to improve the model
Ensure your historical tickets are well-categorized
Consider retraining after major changes to your workflow (Premium)

App is slow

Classification typically takes 2-5 seconds per ticket
Training on large datasets (500+ tickets) may take several minutes
Check your internet connection
Anthropic API response times may vary during peak usage

Contact Support

Need help?

We're here to help! Contact us through:

Email: [email protected]
Marketplace: Use the "Get Support" link on our Marketplace listing

Please include:

Your Jira Cloud URL
Steps to reproduce the issue
Any error messages you see
Screenshots if helpful
Your subscription tier (Free, Standard, or Premium)