Note: If you have made a non-Pipeline deployment that you would like to roll back, please see this article.
Understanding rollbacks
“Rollbacks”, as they are generally understood (a reversal of a Production or WIP deployment), are not consequence-free, and choosing the correct problem-solving method for your Pipeline should reflect this.
Best practice, where possible, is always to fix forward. If you find that you (or your team) regularly need to roll back a Production release, taking a closer look at your process is advisable.
We would recommend you read "How to prevent the need for a rollback", check out one of our webinars for some practical starting points or reaching out to our in app chat to talk about rollbacks.
For example, if a feature branch has been reviewed, approved, passed all quality gates, and released into Production, but only then highlighted as having issues severe enough to demand a rollback, it is likely your testing or approval processes need reviewing.
An important thing to keep in mind is that some things can’t be deleted over the Metadata API (such as Record Types), and indeed shouldn’t be deleted over the API - there could be serious implications for your data if Record Types were deleted.
Therefore, it's always better to think about how your process can prevent the need for a rollback before planning to carry one out.
When to do a rollback in your pipeline
Before rolling back, ask yourself:
This PR has been deployed to Production see Scenario 2
This PR is still in the pipeline waiting to go into a environment Scenario 1
After doing a rollback it's always important to understand how to avoid this scenario occurring again
Important note: Ensure that the auto-delete feature branches checkbox is unchecked during a rollback.
Scenario 1: Rolling back a feature not yet deployed to Production
A feature branch may require more components or may have some removed, pending validation, testing, or feedback.
In this example, a feature branch has been deployed through the Pipeline from DEV to INT to UAT and a pull request has automatically opened against Production.
During UAT, your team might identify that the feature:
Needs to be updated by adding or removing components.
Is not working as it should, and needs to be removed from SIT and UAT.
You then have two choices:
Fix the feature in progress (Recommended)
Delete the feature from your Pipeline in order to start again from the beginning. (Not Recommended)
Fix the feature in progress (Recommended)
Fix the feature in progress (Recommended)
This is always the preferred option as it reduces the drift between the feature branch and org that can be an issue when abandoning a feature.
We have a an article about how to fix your deployment using the "Commit more changes" feature in Gearset pipelines:
Edit or fix a user story in progress in pipelines.
Rollback the feature from your Pipeline in order to start again from the beginning (Not recommended)
Rollback the feature from your Pipeline in order to start again from the beginning (Not recommended)
While it’s never ideal, sometimes rolling back a change that has entered your pipeline is necessary.
If you do need to do this, it’s important to review what happened and identify ways to prevent this scenario from happening again.
Rolling back in pipelines takes unnecessary time and can create drift between your feature branches and orgs. This often leads to the need for re-syncing your environments.
Take a look at our section on how to prevent the need for a rollback.
How to roll back a change not yet in Production
In this example, I’m rolling back a change that is currently sitting against Production, restoring it to my production state.
Specifically, I’m rolling back my PR: Richard-29May-LegacyUser.
For simplicity’s sake, this PR contains only one change: Legacy_User__C
.
1. First, we are now going to open a new manual comparison (from Compare and deploy page) as follows:
Source: Main/Master branch
Target: Feature branch name (In my case Pipelines-Demo/Richard-29May-LegacyUser
)
Note: Make sure you DO NOT select the promotion branch. You can spot the promotion branch by looking for "_-_EnvironmentName
" at the end of the promotion's branch name.
2. In your manual comparison, select the changes you are trying to roll back.
Be careful as there may be changes in the comparison that are not related to this PR and should not be added to your rollback.
3. Deploy these changes into your feature branch.
This has updated your current feature branch with the removal of these changes.
4. Open your pipeline and select one of your dev sandboxes.
Then change the feature branch to the feature branch we have just deployed to (in my case I'm changing the feature branch to Richard-29May-LegacyUser
)
Once done, select "Create pull request".
5. You will now have a new PR open against the first environment in your pipeline that will be deploying the rollback of your changes.
Continue to promote this down your pipeline until it reaches the original PR open against Prod.
This process will remove these changes from all the environments in your pipeline.
6. Once this new PR reaches the original deployment you should see the changes no longer exist in the PR.
You can then click the rubbish bin icon (on your Prod environment) to delete this PR and feature branch.
7. Now that you have successfully rolled back your PR from the pipeline, it's a good idea to look into how to prevent the need for a rollback in the future to avoid this scenario from occurring again.
Scenario 2: Rolling back a feature in your Pipeline which has been deployed to Production
Have you already merged a feature branch all the way through your Pipeline and deployed it to Production, and now you need to correct Production?
Before deciding how to resolve an issue with a Production deployment ask yourself these questions:
Is it affecting Production users?
If it is, then rolling back immediately is the best option.
If it is not, does the change need rolling back, or is it something we can correct by fixing forward?
Rolling back a Production deployment immediately
Rolling back a Production deployment immediately
If you find yourself in a situation where you need to urgently remove the components deployed to Production from the org you can follow these steps. Because this involves a direct deployment to Production it should only be used in exceptional circumstances.
This is a two-stage process, the actual removal of the unwanted components from the org and the branch changes required to get things in sync. For a shorter path please see the section "Using a hotfix branch to roll back".
Open the CI job dashboard and locate the Production CI job.
Open the history.
Open the most recent deployment details, and click “Roll back”.
Select the components (taking care with some of the MDAPI nuances as listed here, not every component can be deleted using a deployment) you would like to remove from Production.
Proceed through problem analysis and deploy your changes to Production.
From the deployment summary, choose the option to “Deploy to new target”.
Create a hotfix branch from main and commit your package.
Open a pull request from your hotfix branch to the main branch.
After approval, merge your hotfix branch to the main branch. This will bring the main branch and Production back in sync. If the target of the hotfix is already in sync with the main branch, then you will see zero components to validate. This isn’t anything to worry about, and you can merge the branch with no ill effect.
Gearset will back-propagate this hot fix through your Pipeline. Merge it to your environments to make sure they are also up to date with Production.
Review what went wrong to cause the need for a rollback and close any gaps in your process, it's a good idea to look into how to prevent the need for a rollback in the future to avoid this scenario from occurring again.
Rolling back a specific feature
Rolling back a specific feature
If there is a specific feature that you need to remove from the main branch/Production you can do so by following these steps. To use this method you will need to make sure that the original feature branch still exists.
Start by navigating to your deployment history page. Locate the user story/ticket number in your deployment history.
Roll back the original commit to the feature branch. This will essentially 'revert' the changes and add an extra commit to your branch.
After the fix has been committed, open a pull request to the main branch.
After all the quality gates and review processes have passed successfully, promote the fix to Production.
Gearset will back-propagate this hotfix through your Pipeline. Merge it to your environments to make sure they are also up to date with Production.
Review which features were affected by your rollback and update any associated tickets.
Review what went wrong to cause the need for a rollback and close any gaps in your process, it's a good idea to look into how to prevent the need for a rollback in the future to avoid this scenario from occurring again.
Deploying a fix to Production via a hotfix branch
Deploying a fix to Production via a hotfix branch
If you find yourself in a situation where you want to fix something that has just been deployed to Production very quickly, but without directly altering the Production org to do so, then you should use a hotfix branch.
Develop your fix in a Sandbox, and then open a hotfix branch.
Commit your fix to the hotfix branch using Gearset.
After the fix has been committed, open a pull request to the main branch.
After all the quality gates and review processes have passed successfully, promote the fix to Production.
Gearset will back-propagate this hotfix through your Pipeline. Merge it to your environments to make sure they are also up to date with Production.
Review which features were affected by your rollback and update any associated tickets.
Review what went wrong to cause the need for a rollback and close any gaps in your process, it's a good idea to look into how to prevent the need for a rollback in the future to avoid this scenario from occurring again.
How to prevent the need for a rollback
After resolving the immediate need to roll back or fix forward a change in your Pipeline we recommend asking the following questions:
How could we have found this issue earlier?
How can we close this gap in our process going forwards?
We have some tips to help you get started with this immediately:
Use the recommended settings for your Pipeline.
Create Unit Testing and Monitoring jobs for all your orgs, use these results to run daily org scans and identify the following before they ever enter the Pipeline:
Apex code coverage dropping below a set threshold.
Apex code with failing tests.
Code present in your orgs violating your standard or custom rule sets.
Set Gearset as an unavoidable status check on your source control provider, requiring a successful validation before changes can be promoted. Check out these articles for some inspiration: