Problem:

When first configuring each of the continuous integration (CI) jobs that make up your automated pipeline, you need to set a metadata filter. This dictates which metadata types each CI job will consider for each run.

However, since the initial configuration of your filter, you may come to realise that your filter is missing certain metadata types, which now need to be considered by each job. These types also need to be added to your repository.

Depending on the number of types you're looking to add, our advice varies. This document walks you through these two scenarios.

Scenario 1: A few missing metadata types and environments are in sync

If it's only a few metadata types, then you can feasibly add these types to your pre-existing CI jobs and pipeline. The key to success here is how in sync your org environments are. To ensure that each environment is as much in sync as possible, it's recommended that you follow this process post sandbox refresh. (If you're unable to ensure your environments are in sync, we'd recommend following scenario 2 instead).

When ready, follow these steps:

Configure and save your new filter.
1. Navigate to the Compare & deploy page and open the comparison filter option.
2. Select Manage custom filters... and select all the metadata types you'd like your CI job to consider. This selection will be the new all encompassing filter.
3. In the bottom left, select the dropdown arrow and select Save as new filter.
4. Give your new filter a recognisable name and save it.
Repopulate Main from Production, using the new filter.
1. From the same Compare & deploy page, run a comparison between Production (as the source) and your Main branch (as the target). Use your new filter selection.
2. Select all the New and Changed components, bringing them into the package.
3. Proceed to the next page. You may see a number of problem analysers triggering for various reasons. Pay attention to these suggestions, but for the most part you can deselect/ignore them. We're looking to seed your repository with all the metadata, bringing it in sync with your org.
4. Commit all these components into your Main branch.
Apply that newly saved filter to each environment in your pipeline.
1. From the continuous integration dashboard, select any of the CI jobs that make up part of your pipeline. Once expanded, select Edit settings...
2. Navigate to the Metadata filter tab and within the dropdown box, you should now be able to find and select the filter you just saved.
3. With the new filter selected, save the job.
4. Repeat this process for the rest of the CI jobs that make up your pipeline.
Re-sync each environment in your pipeline.
1. Navigate to your final Pipeline environment on the Deployment pipelines page and select this sync feature (the one with description: "Create sync PRs from main to all your upstream environments" when you hover over it)
2. Create sync pull requests for each and every static environment. Note that if a static environment already has an open sync PR, it won't appear on the list.
  
  Also note that if a static environment is already in sync with main, you'll see below message appearing on the pop-up window:
  Failed to open sync PR: There are no commit differences between the environment <nxxx> and "gs-pipeline/main_-_<xxx>". The environment <xxx> is already up to date. - where <xxx> in this context refers to an environment name (see an example of the error message on below screenshot).
3. A pull request will be opened between Main and the environment you're doing it on. The promotion of this PR will sync all these newly added metadata components from your Main branch into your environment branch. This in turn will trigger a run of the CI job. As your org is in sync with Production, we wouldn't expect this run to actually deploy anything to the org. But please be aware that it will, if the environments aren't in sync.

Scenario 2: A significant number of missing metadata types and/or environments aren't in sync

If there are a significant number of metadata types that need to be added to your filters, or your environments aren't in sync, then we'd suggest starting with new CI jobs and environment branches. We wouldn't recommend deleting your old CI jobs and branches until you're confident you're done with them. Any in-flight work or currently open PRs will still be present in these branches, so we should only delete them once we're done with this transition.

When ready to begin, follow these steps:

Configure and save your new filter.
1. Navigate to the Compare & deploy page and open the comparison filter option.
2. Select Manage custom filters... and select all the metadata types you'd like your CI job to consider. This selection will be the new all encompassing filter.
3. In the top right, select the dropdown arrow and select Save as new filter.
4. Give your new filter a recognisable name and save it.
Repopulate Main from Production, using the new filter.
1. From the same Compare & deploy page, run a comparison between Production (as the source) and your Main branch (as the target). Use your new filter selection.
2. Select all the New and Changed components, bringing them into the package.
3. Proceed to the next page. You may see a number of problem analysers triggering for various reasons. Pay attention to these suggestions, but for the most part you can deselect/ignore these suggestions. We're looking to seed your repository with all the metadata, bringing it in sync with your org.
4. Commit all these components into your Main branch.
Disable existing CI jobs and remove them from the pipeline.
1. Navigate to the Deployment pipelines page and select any of the static jobs that make up part of your pipeline.
2. Toggle the CI job from Job enabled to Job disabled.
3. Repeat this process for each static environment in your pipeline.
4. Then on the Deployment pipelines page select Edit environments (top right corner), and delete each job from your pipeline (by selecting Delete environment button). Please note that this action is only removing each CI job from the pipeline — it's not actually deleting the CI job. This is intentional as we don't yet want to delete any jobs as we'll need them for the next step.
Duplicate existing CI jobs and create new environment branches.
1. Navigate to the Continuous integration dashboard where you'll still see each of the now disabled CI jobs.
2. Select one of the CI jobs, expanding it to reveal more options, and select Duplicate job. This option will allow you to create a new CI job but nearly all the settings from the original job will be pre-configured to save you time.
3. For this new job, you'll need to once again select the source and target. For the source, select your Git Provider, original repository and initially select your Main branch. Right underneath it you'll see the option + Create new branch from Main — select that and name the new branch in accordance with the target environment for this CI job. If we're duplicating the UAT job, we'd likely name the new branch according to that. (E.g. UAT-Branch). The target in this example would be the UAT org.
4. Next navigate to the Metadata filter tab and apply the newly saved metadata filter to the CI job.
5. Be sure to check all the configuration tabs to make sure it matches what you would expect, and save the job.
6. Repeat this process for each of the original CI jobs in your pipeline.
Add these CI jobs back into your pipeline and connect them.
1. Navigate to the Deployment pipelines page and select + Add environment, then Add static environment.
2. Select Choose from an existing CI job and select one of the CI jobs that you've just created. Click Save to add it to your pipeline.
3. Repeat this process for each CI job in your pipeline.
4. Finally, connect each CI job within the pipeline, as they were originally. This restores your pipeline to how it used to be, but now the filters and repository reflect what they should.

Post configuration considerations for scenario 2

I still have in-flight work open in a PR against one of the old environment branches. What do I do about these?

As you now have whole new environment branches, you'll need to bring these PRs and their associated feature branches over to the newly updated pipeline. The original feature branch for each of these PRs should still be active and usable within your repository, so you simply need to open a new PR from each original feature branch to the first environment in your pipeline. This process brings the feature back to the beginning of the new pipeline, which you can them promote through until it resumes its original location.

I had configured automatic Jira status updates for each stage of my pipeline, do I need to reconfigure these?

Yes, you will need to reconfigure these. Please follow our documentation on this:
Updating a Jira ticket's status in pipelines

At what point am I safe to delete the old disabled CI jobs?

Each of these CI jobs contains a history of all deployments that it did. Once you are certain that you no longer require this history, you're safe to delete these CI jobs from the Continuous integration dashboard.

At what point am I safe to delete the old environment branches?

Similar to the CI jobs, each of these branches will contain a history of any previous commits. As well as this, if there is any in-flight work that still needs to be transitioned over to the newly created environment branches, then the branch still needs to be kept. Once you're certain you don't need to transition any further in-flight work from them, or need their history, then you're safe to delete them directly within your Git Provider.

Setting up your first Gearset Pipeline

Keep your environment branches in-sync

Long term projects in Gearset Pipelines

How to realign/fix an out of sync Pipeline

How to Deploy CPQ with Gearset Pipelines