locationsNotCovered gets empty objects

Summary

I'm trying to get uncovered lines of code from a "validate" job result.

Running the same commands on my machine works like a charm but running them from the docker runner on the server doesn't.

Steps To Reproduce

Make a package.xml with at least one meta to deploy (doesn't matter which one)

<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <types>
        <members>*</members>
        <name>ApexPage</name>
    </types>
    <version>63.0</version>
</Package>

Run a validation with sf project deploy validate -x ./package.xml --json -g --async -l RunLocalTests > ./dry-run-id.json
Run sf project deploy resume -i JOB_ID --json > ./validate-resume.json (replace JOB_ID by the deployment id)
Get the result with sf project deploy report -i JOB_ID --json > ./dry-run-result.json (replace JOB_ID by the deployment id)

Check validate-resume.json and dry-run-result.json file. They should both contains test results with uncovered lines.

Expected result

Example part of the file about codeCoverage, especially the locationsNotCovered section.

{
    "id": "some_sf_id",
    "locationsNotCovered": [
        {
            "column": 0,
            "line": 11,
            "numExecutions": 0,
            "time": -1
        },
        {
            "column": 0,
            "line": 26,
            "numExecutions": 0,
            "time": -1
        }
    ],
    "name": "some_trigger",
    "namespace": null,
    "numLocations": 15,
    "numLocationsNotCovered": 2,
    "type": "Trigger"
},

Actual result

In the dry-run-result.json file all locationsNotCovered sections are broken like that (validate-resume.json is fine)

{
    "id": "some_sf_id",
    "locationsNotCovered": [
        {},
        {}
    ],
    "name": "some_trigger",
    "namespace": null,
    "numLocations": 15,
    "numLocationsNotCovered": 2,
    "type": "Trigger"
},

Additional information

I tried to reproduce in several ways without success. It only happens inside my gitlab docker runner.

I tried :

locally by executing directly the commands (windows with powershell)
in a local docker runner using the exact same docker image (windows with docker desktop)

Running the report command locally when my gitlab job fails and using the exact same JOB_ID is giving me the correct result.

I tried to look in the source code to see if I was missing some flags but it doesn't seems to exist any that would kind of limit the depth of the JSON file.

System Information

sf doctor command result on the local docker runner (should be identical with the gitlab docker runner).

{
  "status": 0,
  "result": {
    "versionDetail": {
      "architecture": "linux-x64",
      "cliVersion": "@salesforce/cli/2.98.6",
      "nodeVersion": "node-v22.18.0",
      "osVersion": "Linux 6.6.87.2-microsoft-standard-WSL2",
      "rootPath": "/usr/lib/node_modules/@salesforce/cli",
      "shell": "bash",
      "pluginVersions": [
        "@oclif/plugin-autocomplete 3.2.33 (core)",
        "@oclif/plugin-commands 4.1.29 (core)",
        "@oclif/plugin-help 6.2.31 (core)",
        "@oclif/plugin-not-found 3.2.60 (core)",
        "@oclif/plugin-plugins 5.4.45 (core)",
        "@oclif/plugin-search 1.2.27 (core)",
        "@oclif/plugin-update 4.7.1 (core)",
        "@oclif/plugin-version 2.2.31 (core)",
        "@oclif/plugin-warn-if-update-available 3.1.44 (core)",
        "@oclif/plugin-which 3.2.39 (core)",
        "@salesforce/cli 2.98.6 (core)",
        "agent 1.23.3 (core)",
        "apex 3.6.19 (core)",
        "api 1.3.3 (core)",
        "auth 3.7.9 (core)",
        "data 4.0.46 (core)",
        "deploy-retrieve 3.22.33 (core)",
        "info 3.4.73 (core)",
        "limits 3.3.60 (core)",
        "marketplace 1.3.8 (core)",
        "org 5.9.9 (core)",
        "packaging 2.18.0 (core)",
        "schema 3.3.73 (core)",
        "settings 2.4.36 (core)",
        "sobject 1.4.65 (core)",
        "telemetry 3.6.48 (core)",
        "templates 56.3.54 (core)",
        "trust 3.7.109 (core)",
        "user 3.6.29 (core)",
        "sfdx-git-delta 6.16.0 (user)"
      ]
    },
    "sfdxEnvVars": [],
    "sfEnvVars": [
      [
        "SF_DISABLE_AUTOUPDATE",
        "true"
      ],
      [
        "SF_IMPROVED_CODE_COVERAGE",
        "true"
      ],
      [
        "SF_ORG_METADATA_REST_DEPLOY",
        "true"
      ],
      [
        "SF_DOMAIN_RETRY",
        "300"
      ],
      [
        "SF_DISABLE_TELEMETRY",
        "true"
      ],
      [
        "SF_CONTAINER_MODE",
        "true"
      ],
      [
        "SF_SKIP_NEW_VERSION_CHECK",
        "true"
      ],
      [
        "SF_CONTENT_TYPE",
        "JSON"
      ],
      [
        "SF_UPDATE_INSTRUCTIONS",
        "Use \"npm update --global @salesforce/cli\" to update npm-based installations."
      ]
    ],
    "proxyEnvVars": [],
    "cliConfig": {
      "arch": "x64",
      "bin": "sf",
      "binAliases": [
        "sfdx"
      ],
      "cacheDir": "/root/.cache/sf",
      "channel": "stable",
      "configDir": "/root/.config/sf",
      "dataDir": "/root/.local/share/sf",
      "dirname": "sf",
      "flexibleTaxonomy": true,
      "home": "/root",
      "isSingleCommandCLI": false,
      "name": "@salesforce/cli",
      "nsisCustomization": "scripts/nsis.nsi",
      "platform": "linux",
      "root": "/usr/lib/node_modules/@salesforce/cli",
      "shell": "bash",
      "theme": {
        "bin": "blueBright",
        "command": "blueBright",
        "dollarSign": "green",
        "flag": "green",
        "flagDefaultValue": "blueBright",
        "flagOptions": "blueBright",
        "flagRequired": "red",
        "sectionHeader": "blue",
        "topic": "blueBright",
        "json": {
          "brace": "whiteBright",
          "bracket": "whiteBright",
          "comma": "green",
          "key": "blueBright",
          "string": "greenBright",
          "number": "blue",
          "boolean": "redBright",
          "null": "blackBright"
        }
      },
      "topicSeparator": " ",
      "updateConfig": {
        "node": {
          "options": [
            "--no-deprecation"
          ]
        },
        "s3": {
          "bucket": "dfc-data-production",
          "host": "https://developer.salesforce.com",
          "templates": {
            "target": {
              "baseDir": "<%- bin %>",
              "manifest": "<%- channel === 'stable' ? '' : 'channels/' + channel + '/' %><%- platform %>-<%- arch %>",
              "unversioned": "<%- channel === 'stable' ? '' : 'channels/' + channel + '/' %><%- bin %>-<%- platform %>-<%- arch %><%- ext %>",
              "versioned": "<%- channel === 'stable' ? '' : 'channels/' + channel + '/' %><%- bin %>-v<%- version %>/<%- bin %>-v<%- version %>-<%- platform %>-<%- arch %><%- ext %>"
            },
            "vanilla": {
              "baseDir": "<%- bin %>",
              "manifest": "<%- channel === 'stable' ? '' : 'channels/' + channel + '/' %>version",
              "unversioned": "<%- channel === 'stable' ? '' : 'channels/' + channel + '/' %><%- bin %><%- ext %>",
              "versioned": "<%- channel === 'stable' ? '' : 'channels/' + channel + '/' %><%- bin %>-v<%- version %>/<%- bin %>-v<%- version %><%- ext %>"
            }
          }
        }
      },
      "userAgent": "@salesforce/cli/2.98.6 linux-x64 node-v22.18.0",
      "valid": true,
      "version": "2.98.6",
      "warned": false,
      "windows": false,
      "_base": "@oclif/core@4.5.1",
      "_commands": {},
      "_topics": {},
      "commandPermutations": {
        "validPermutations": {}
      },
      "pluginLoader": {
        "options": {
          "root": "/usr/lib/node_modules/@salesforce/cli/dist"
        },
        "errors": [],
        "plugins": {},
        "pluginsProvided": false
      },
      "topicPermutations": {
        "validPermutations": {}
      },
      "nodeEngine": ">=18.6.0"
    },
    "pluginSpecificData": {
      "@salesforce/plugin-deploy-retrieve": [
        {}
      ],
      "@salesforce/plugin-auth": [
        {
          "isUsingGenericKeychain": false,
          "sfCryptoV2Support": true,
          "cryptoVersion": "unknown"
        }
      ]
    },
    "diagnosticResults": [
      {
        "testName": "salesforcedx plugin isn’t installed",
        "status": "pass"
      },
      {
        "testName": "you don't have any linked plugins",
        "status": "pass"
      },
      {
        "testName": "[@salesforce/plugin-deploy-retrieve] sourceApiVersion matches apiVersion",
        "status": "warn"
      },
      {
        "testName": "[@salesforce/plugin-trust] can ping: https://registry.npmjs.org",
        "status": "pass"
      },
      {
        "testName": "[@salesforce/plugin-trust] can ping: https://registry.yarnpkg.com",
        "status": "pass"
      },
      {
        "testName": "[@salesforce/plugin-trust] can ping: https://registry.npmjs.org/",
        "status": "pass"
      },
      {
        "testName": "using latest or latest-rc CLI version",
        "status": "pass"
      },
      {
        "testName": "can access: https://test.salesforce.com",
        "status": "pass"
      },
      {
        "testName": "can access: https://appexchange.salesforce.com/services/data",
        "status": "pass"
      },
      {
        "testName": "can access: https://developer.salesforce.com/media/salesforce-cli/sf/channels/stable/sf-win32-x64-buildmanifest",
        "status": "pass"
      },
      {
        "testName": "[@salesforce/plugin-auth] CLI supports v2 crypto",
        "status": "pass"
      }
    ],
    "suggestions": [
      "Check https://github.com/forcedotcom/cli/issues for CLI issues posted by the community.",
      "Check http://status.salesforce.com for general Salesforce availability and performance.",
      "Neither sourceApiVersion nor apiVersion are defined. The commands that deploy and retrieve source use the max apiVersion of the target org in this case. The issue isn't a problem, as long as it's the behavior you actually want.",
      "using npm registry https://registry.npmjs.org/ from npm config"
    ],
    "logFilePaths": [
      "/opt/ci-scripts/1754984540366-diagnosis.json",
      "/opt/ci-scripts/1754984540366-diagnosis.json"
    ],
    "commandExitCode": 0
  },
  "warnings": []
}

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

@Bartheleway

Hello @Bartheleway 👋 None of the versions of sf you shared match the latest release.

Shared: 2.98.6, 2.98.6
Latest: 2.100.4

Update to the latest version of Salesforce CLI (docs) and confirm that you're still seeing your issue.
You can also try the rc and nightly releases! (docs)

After updating, share the full output of sf version --verbose --json

@Bartheleway

Hey @Bartheleway, unfortunately these "only happens in my CI" issues are notoriously difficult to track down. What is the name of the Docker container you are using in Gitlab?

If the same Docker container works from your local machine, something must be up with the Gitlab one. Could you please provide us with the following information executed from inside your Gitlab CI runner.

The output of sf doctor
Add the --dev-debug flag to your sf project deploy report -i JOB_ID --json > ./dry-run-result.json command and provide the full output

@Bartheleway

Hello @Bartheleway 👋 None of the versions of sf you shared match the latest release.

Shared: 2.98.6, 2.98.6
Latest: 2.101.5

Update to the latest version of Salesforce CLI (docs) and confirm that you're still seeing your issue.
You can also try the rc and nightly releases! (docs)

After updating, share the full output of sf version --verbose --json

This issue has not received a response in 7 days. It will auto-close in 7 days unless a response is posted.

@iowillhoit

@iowillhoit, yes I agree. I would not be here if it was easier 😄

I did run the sf project deploy report -i JOB_ID --json > ./dry-run-result.json in a debug job and got the expected output 😮

There is one additional factor to check: Command is executed right after it finishes to make the validation on SF side, so maybe I just need to wait 1min or so?

The updated sf doctor from Gitlab runner

{
  "status": 0,
  "result": {
    "versionDetail": {
      "architecture": "linux-x64",
      "cliVersion": "@salesforce/cli/2.103.7",
      "nodeVersion": "node-v22.18.0",
      "osVersion": "Linux 5.14.0-570.32.1.el9_6.x86_64",
      "rootPath": "/usr/lib/node_modules/@salesforce/cli",
      "shell": "bash",
      "pluginVersions": [
        "@oclif/plugin-autocomplete 3.2.34 (core)",
        "@oclif/plugin-commands 4.1.32 (core)",
        "@oclif/plugin-help 6.2.32 (core)",
        "@oclif/plugin-not-found 3.2.65 (core)",
        "@oclif/plugin-plugins 5.4.46 (core)",
        "@oclif/plugin-search 1.2.28 (core)",
        "@oclif/plugin-update 4.7.4 (core)",
        "@oclif/plugin-version 2.2.32 (core)",
        "@oclif/plugin-warn-if-update-available 3.1.46 (core)",
        "@oclif/plugin-which 3.2.39 (core)",
        "@salesforce/cli 2.103.7 (core)",
        "agent 1.24.4 (core)",
        "apex 3.6.19 (core)",
        "api 1.3.3 (core)",
        "auth 3.9.0 (core)",
        "data 4.0.53 (core)",
        "deploy-retrieve 3.22.41 (core)",
        "info 3.4.81 (core)",
        "limits 3.3.64 (core)",
        "marketplace 1.3.8 (core)",
        "org 5.9.23 (core)",
        "packaging 2.19.5 (core)",
        "schema 3.3.78 (core)",
        "settings 2.4.42 (core)",
        "sobject 1.4.68 (core)",
        "telemetry 3.6.55 (core)",
        "templates 56.3.60 (core)",
        "trust 3.7.113 (core)",
        "user 3.6.34 (core)",
        "sfdx-git-delta 6.18.2 (user)"
      ]
    },
    "sfdxEnvVars": [],
    "sfEnvVars": [
      [
        "SF_CONTAINER_MODE",
        "true"
      ],
      [
        "SF_DISABLE_TELEMETRY",
        "true"
      ],
      [
        "SF_DISABLE_AUTOUPDATE",
        "true"
      ],
      [
        "SF_DOMAIN_RETRY",
        "300"
      ],
      [
        "SF_SKIP_NEW_VERSION_CHECK",
        "true"
      ],
      [
        "SF_CONTENT_TYPE",
        "JSON"
      ],
      [
        "SF_ORG_METADATA_REST_DEPLOY",
        "true"
      ],
      [
        "SF_IMPROVED_CODE_COVERAGE",
        "true"
      ],
      [
        "SF_UPDATE_INSTRUCTIONS",
        "Use \"npm update --global @salesforce/cli\" to update npm-based installations."
      ]
    ],
    "proxyEnvVars": [],
    "cliConfig": {
      "arch": "x64",
      "bin": "sf",
      "binAliases": [
        "sfdx"
      ],
      "cacheDir": "/root/.cache/sf",
      "channel": "stable",
      "configDir": "/root/.config/sf",
      "dataDir": "/root/.local/share/sf",
      "dirname": "sf",
      "flexibleTaxonomy": true,
      "home": "/root",
      "isSingleCommandCLI": false,
      "name": "@salesforce/cli",
      "nsisCustomization": "scripts/nsis.nsi",
      "platform": "linux",
      "root": "/usr/lib/node_modules/@salesforce/cli",
      "shell": "bash",
      "theme": {
        "bin": "blueBright",
        "command": "blueBright",
        "dollarSign": "green",
        "flag": "green",
        "flagDefaultValue": "blueBright",
        "flagOptions": "blueBright",
        "flagRequired": "red",
        "sectionHeader": "blue",
        "topic": "blueBright",
        "json": {
          "brace": "whiteBright",
          "bracket": "whiteBright",
          "comma": "green",
          "key": "blueBright",
          "string": "greenBright",
          "number": "blue",
          "boolean": "redBright",
          "null": "blackBright"
        }
      },
      "topicSeparator": " ",
      "updateConfig": {
        "node": {
          "options": [
            "--no-deprecation"
          ]
        },
        "s3": {
          "bucket": "dfc-data-production",
          "host": "https://developer.salesforce.com/",
          "templates": {
            "target": {
              "baseDir": "<%- bin %>",
              "manifest": "<%- channel === 'stable' ? '' : 'channels/' + channel + '/' %><%- platform %>-<%- arch %>",
              "unversioned": "<%- channel === 'stable' ? '' : 'channels/' + channel + '/' %><%- bin %>-<%- platform %>-<%- arch %><%- ext %>",
              "versioned": "<%- channel === 'stable' ? '' : 'channels/' + channel + '/' %><%- bin %>-v<%- version %>/<%- bin %>-v<%- version %>-<%- platform %>-<%- arch %><%- ext %>"
            },
            "vanilla": {
              "baseDir": "<%- bin %>",
              "manifest": "<%- channel === 'stable' ? '' : 'channels/' + channel + '/' %>version",
              "unversioned": "<%- channel === 'stable' ? '' : 'channels/' + channel + '/' %><%- bin %><%- ext %>",
              "versioned": "<%- channel === 'stable' ? '' : 'channels/' + channel + '/' %><%- bin %>-v<%- version %>/<%- bin %>-v<%- version %><%- ext %>"
            }
          }
        }
      },
      "userAgent": "@salesforce/cli/2.103.7 linux-x64 node-v22.18.0",
      "valid": true,
      "version": "2.103.7",
      "warned": false,
      "windows": false,
      "_base": "@oclif/core@4.5.2",
      "_commands": {},
      "_topics": {},
      "commandPermutations": {
        "validPermutations": {}
      },
      "pluginLoader": {
        "options": {
          "root": "/usr/lib/node_modules/@salesforce/cli/dist"
        },
        "errors": [],
        "plugins": {},
        "pluginsProvided": false
      },
      "topicPermutations": {
        "validPermutations": {}
      },
      "nodeEngine": ">=18.6.0"
    },
    "pluginSpecificData": {
      "@salesforce/plugin-deploy-retrieve": [
        {
          "sourceApiVersion": "63.0"
        }
      ],
      "@salesforce/plugin-auth": [
        {
          "isUsingGenericKeychain": false,
          "sfCryptoV2Support": true,
          "cryptoVersion": "unknown"
        }
      ]
    },
    "diagnosticResults": [
      {
        "testName": "salesforcedx plugin isn’t installed",
        "status": "pass"
      },
      {
        "testName": "you don't have any linked plugins",
        "status": "pass"
      },
      {
        "testName": "[@salesforce/plugin-trust] can ping: https://registry.npmjs.org/",
        "status": "pass"
      },
      {
        "testName": "[@salesforce/plugin-trust] can ping: https://registry.yarnpkg.com/",
        "status": "pass"
      },
      {
        "testName": "[@salesforce/plugin-trust] can ping: https://registry.npmjs.org/",
        "status": "pass"
      },
      {
        "testName": "using latest or latest-rc CLI version",
        "status": "pass"
      },
      {
        "testName": "[@salesforce/plugin-deploy-retrieve] sourceApiVersion matches apiVersion",
        "status": "pass"
      },
      {
        "testName": "can access: https://test.salesforce.com/",
        "status": "pass"
      },
      {
        "testName": "can access: https://appexchange.salesforce.com/services/data",
        "status": "pass"
      },
      {
        "testName": "can access: https://developer.salesforce.com/media/salesforce-cli/sf/channels/stable/sf-win32-x64-buildmanifest",
        "status": "pass"
      },
      {
        "testName": "[@salesforce/plugin-auth] CLI supports v2 crypto",
        "status": "pass"
      }
    ],
    "suggestions": [
      "Check https://github.com/forcedotcom/cli/issues for CLI issues posted by the community.",
      "Check http://status.salesforce.com/ for general Salesforce availability and performance.",
      "using npm registry https://registry.npmjs.org/ from npm config"
    ],
    "commandExitCode": 0
  },
  "warnings": []
}

@Bartheleway

yes I agree. I would not be here if it was easier 😄

lol, fair point 😆 Sorry for the delayed response @Bartheleway. It is entirely possible that there is a race condition here. Did you try adding a sleep before calling the sf project deploy report?

As said initially, my command flow is like this

sf project deploy validate -x ./package.xml --json -g --async -l RunLocalTests > ./dry-run-id.json
sf project deploy resume -i JOB_ID --json > ./validate-resume.json (replace JOB_ID by the deployment id)
sf project deploy report -i JOB_ID --json > ./dry-run-result.json

Which means:

First I send package to org and get deploy id in dry-run-id.json file
Second I wait until org validation is done and put result in validate-resume.json file
Third (which is just a workaround and should not be necessary), I get again the result of the validation in dry-run-result.json

Second step is getting result right after validation ends on org. Second and third step are run just "seconds" apart but offer different result where they should not. How can it be that result from two different calls differ especially if 1st one succeed, 2nd failed and later 3rd succeed again?

@Bartheleway

Hi @Bartheleway, sorry for late response! Is this issue still occurring with the latest sf version? If so, could you provide updated reproduction steps or details and we can investigate further. Thanks!

Hi, yes I still have the same issue in v2.141.6 & node v26.5.0. Not the latest but if you haven't done anything, it should not fix by itself especially when we see that I reported that in v2.98 ...

Oddly the incomplete result is located in dry-run-result.json where it was, according to my previous posts, located in validate-resume.json. I don't know if the issue moved from one command to another or if I just inverted the two file when trying to sumup what happened.

You have the reproduction steps above and here is a small node script that you can execute when drop on the root of your sf project. I didn't provided the package.xml file as any would do as long as it contains something.

import { readFile } from 'node:fs/promises'
import { exec } from 'node:child_process'
import { promisify } from 'node:util'

const execAsync = promisify(exec)

async function run () {
	const projectDeployCmd = await execAsync('sf project deploy validate -x ./package.xml --json -g --async -l RunLocalTests > ./dry-run-id.json || true')

	const rawResult = await readFile(resolve(process.env.CI_PROJECT_DIR, './dry-run-id.json'))
	const projectDeployResult = JSON.parse(rawResult)
	const jobId = projectDeployResult.result.id

	await execAsync(`sf project deploy resume -i ${jobId} --json > ./validate-resume.json || true`)
	await execAsync(`sf project deploy report -i ${jobId} --json > ./dry-run-result.json || true`)
}

run()

@Bartheleway

Hi @Bartheleway, I think the issue related to how SOAP vs REST return the output. I noticed your CI env has SF_ORG_METADATA_REST_DEPLOY=true set. Could you confirm — when you run the commands locally on your machine (where it works), do you also have SF_ORG_METADATA_REST_DEPLOY=true set?

Hi, on my local machine I indeed have SF_ORG_METADATA_REST_DEPLOY=false. This could be a hint toward explaination. I doubt it a bit as my previous comments state that running the docker image in my local machine doesn't produce the issue. I can double check.

@Bartheleway

@Bartheleway, could you do one more test to verify? Run the same 3 commands locally twice --- once with SF_ORG_METADATA_REST_DEPLOY=true and once without env set. Then compare the locationsNotcovered in report output result-rest.json vs result-soap.json. Let me know how it goes:)

Navigation Menu

Repository navigation

locationsNotCovered gets empty objects #3357

Summary

Steps To Reproduce

Expected result

Actual result

Additional information

System Information

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

locationsNotCovered gets empty objects #3357

Description

Summary

Steps To Reproduce

Expected result

Actual result

Additional information

System Information

Activity

github-actions commented on Aug 12, 2025

github-actions commented on Aug 12, 2025

iowillhoit commented on Aug 14, 2025

github-actions commented on Aug 14, 2025

github-actions commented on Aug 22, 2025

Bartheleway commented on Aug 28, 2025

iowillhoit commented on Oct 2, 2025

Bartheleway commented on Oct 7, 2025

soridalac commented on Aug 6, 2026

Bartheleway commented on Aug 7, 2026

soridalac commented on Oct 19, 2026

Bartheleway commented on Oct 20, 2026

soridalac commented on Oct 20, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions