Sandbox looked fine, production drifted: baselines for Jira configuration
Also published on Atlassian Community
In this guide
The workflow change passed every test in the sandbox. Two weeks after the release, a team reports that a transition does not behave the way it did in the sandbox. The deployment log says it succeeded. Somebody now has to work out what production looks like today, what it looked like on release day, and what was approved.
ConfigRadar is a configuration change audit app for Jira Cloud from Radrly. This guide follows one release on a timeline, with the places where production can drift away from what you tested, the native checks for each, and then how a baseline in the app keeps the approved state.
Before the release: the sandbox is a different site
A sandbox can differ from production before anyone changes anything.
Release tracks. Atlassian apps and sandboxes are on the continuous track by default. If production is on the bundled track, it gets most changes monthly, and a sandbox on the preview track gets bundled changes a month before production. So an option you configured in the sandbox may not exist in production yet, which is exactly what one recent Community thread about an automation user condition ran into. An organization admin can check the release track of both in Atlassian Administration (What are release tracks?).
Stale data. A sandbox copied from production months ago does not contain the changes made in production since. Atlassian's recommended workflow for complex deployments asks you to copy all spaces from production to the sandboxes before you test.
Release day: what the deployment carries, and what it leaves to you
On Premium and Enterprise, an organization admin can deploy configuration from a sandbox (beta): Atlassian Administration > Apps > Sandboxes, then More actions > Create deployment. The scan lists changes since your last deployment, selects dependencies, and after deployment the log offers a report with Summary.csv and RequiresAttention.csv (Deploy configuration changes from a sandbox).
Read the limits before you rely on it (Data deployed from sandboxes):
- Only company-managed spaces are supported.
- Some operations are left for you to repeat by hand in production, among them removing a custom field option, changing a custom field type and removing a status from an existing workflow. New filters arrive as private.
- Objects deleted in the sandbox are not deployed.
- Atlassian asks you not to change the destination by hand until the deployment completes.
Every manual step is a place where production and the sandbox can part ways. Without the deployment feature, the whole release is manual: teams copy a scheme before the change, rebuild it in production and compare by eye.
The week after: quick fixes nobody wrote down
After go-live, quick fixes land straight in production: a field made optional again, a group added to a permission, a transition renamed. Settings > System > Audit log records many configuration events, and you set its retention under Audit Log Settings (Audit activities in Jira). What the log does not hold is the approved state. To answer "how far are we from what we released?" you read every entry since release day and rebuild the comparison yourself.
A baseline: the approved state, stored
ConfigRadar takes a normalised daily snapshot of Jira configuration (workflows and workflow schemes, custom fields and contexts, field configurations, screens, permission schemes, space role actors, statuses and more; 22 object types once a site admin accepts version 2.0.0). Reorder-only noise does not create a change.
Baselines and drift are Advanced:
- Save a baseline in production once the release is approved. The New baseline dialog also takes an As of date, which captures each object as it was at the end of that day, as long as retention still holds that version. If it does not, the app names the earliest date that works.
- Read the drift. ConfigRadar lists every object that differs from the baseline, with its diff. Added and deleted objects count as drift.
- Ignore the noise. Each baseline can ignore up to 50 paths, for example /description, and the drift count follows the same rules.
- Compare spaces to a template. A baseline can also be a template space plus up to 50 member spaces, compared on workflow scheme, permission scheme, work types, space type and a few space settings. Work type screen schemes and field configurations are not part of that comparison.
Baselines are not deleted by retention. The Advanced dashboard gadget shows how many objects have drifted from the latest baseline, and the Advanced PDF auditor pack can include drift against a baseline you pick.
For each drifted object you open the change in the timeline to see who made it and when, and How to revert gives up to 20 steps with the Jira admin path. Revert guidance is in both editions; ConfigRadar never changes configuration itself.
One limit to know: a baseline is a snapshot of the same site, so ConfigRadar compares production with its own approved state and does not compare a sandbox with production.
Quick answers
Why can a Jira Cloud sandbox differ from production before anyone changes anything?
Atlassian apps and sandboxes are on the continuous release track by default. If production is on the bundled track, it gets most changes monthly, so an option configured in the sandbox may not exist in production yet. A sandbox copied from production months ago also lacks the changes made in production since.
What does a sandbox deployment leave to be done by hand in production?
Deploying configuration from a sandbox (beta, Premium and Enterprise) supports only company-managed spaces. Some operations must be repeated by hand in production, among them removing a custom field option, changing a custom field type and removing a status from an existing workflow. New filters arrive as private, and objects deleted in the sandbox are not deployed.
What is a configuration baseline in ConfigRadar?
A baseline (Advanced) is a saved snapshot of the approved Jira configuration, for example after a release. ConfigRadar lists every object that differs from it, with its diff, counts added and deleted objects as drift, and can ignore up to 50 paths per baseline. Baselines are not deleted by retention.
Can ConfigRadar compare a sandbox with production?
No. A baseline is a snapshot of the same site, so ConfigRadar compares production with its own approved state. For each drifted object, How to revert gives up to 20 steps with the Jira admin path; ConfigRadar never changes configuration itself.