While using CI jobs (both validation only and deployment) as well using pipeline feature of Gearset platform, user can face LIMIT_EXCEEDED fatal error thrown by Salesforce. The chances of deployments failing with this error is more when the multiple validation or deployment requests are triggered by Gearset against the same target Salesforce org in shorter period of time.
Gearset triggers validation requests to Salesforce in the following situations:
Upon raising the pull request against the branch that tracks the SF Org.
User manually running 'Validation Only CI job'.
Upon merging the PR to the target branch.
And similarly Gearset triggers deployment requests to Salesforce in the following situations:
Upon merging the PR to the target branch or by clicking on 'Promote Changes' in the pipeline setup.
User manually running 'Deployment CI job'
Gearset issues such validation and deployment requests to Salesforce org and relies on Salesforce to manage the queue of requests to execute one after the other in the order of requests made on First in First Out basis. However, this is not always the case and hence paves the way for deployment failures.
Problem of dequeuing
Let us take an example where:
first pull request (PR1) is raised against a target branch which kicks-off validation run in the SF org.
While this PR1 validation is running, another developer raises a second pull request (PR2) against the same target branch, Gearset will trigger PR2 validation run request and the request will be queued by Salesforce. The validation run for PR2 will start once the validation run for the first pull request is completed.
Upon completion of PR1 validation developer merges the PR in Git, Gearset again issues a validation run request for merged changes before doing the quick deploy.
Going with similar pattern of pull requests getting lined up either for deployment or for validation, when one of the PRs is ready for quick deploy, Salesforce instead of queuing the quick deploy request, it will prioritise quick deploy requests to go through even when the validation job of other PR is currently running in the Org. This is what we know as dequeuing problem.
As the next package validation invalidates quick deploy of previous successful validation run, the quick deploy is meant to fail, however, Salesforce does retry of doing this quick deploy 4 to 5 times while other validation is still in progress eventually ending up with following error:
"LIMIT_EXCEEDED: Try deploying again. Your deployment exceeded its maximum retry limit. If you keep seeing this error, please contact Salesforce Customer Support."
This issue is also listed as one of the Salesforce Known Issues.
Why Salesforce prioritise certain deployments?
Salesforce protects the load on the infrastructure, and deployments which are potentially heavy. Therefore deployments are typed by their expected load as SMALL, MEDIUM of FFX (full, including Apex tests).
So a Quick deploy would not be FFX, but a Full-deploy would be FFX.
A PR validation including Apex tests would be FFX, but a PR validation without tests run would be SMALL or MEDIUM.Now, parallel deployments of type FFX will never be allowed. So if there are two FFX jobs queued, they will be sequential.
But a SMALL/MEDIUM could be picked-up while a FFX is executing. But the FFX has a lock, so the SMALL/MEDIUM deploy cannot start. Now it depends timing-wise whether the LIMIT_EXCEEDED will be raised. As the platform retries 5 times, with a certain interval (unknown to us).
If the FFX job is still running, and another SMALL/MEDIUM (e.g. Quick deploy) is submitted by Gearset shortly following the moment the FFX started running the likeliness it will fail big.
How can we prevent such deployment failures?
Option 1 - One way to avoid LIMIT_EXCEEDED deploy error is to keep validation and deployment runs against the same Org staggered over period of time instead of making multiple requests in shorter period of time. Gearset user can monitor if any jobs are running against the same target and wait until current validation/deployment run is complete before issuing next validation/deployment request from Gearset.
Option 2 - Please check with our Support Engineer via our App chat who can investigate this error more in detail and possibly discuss workarounds that may suit your use case.