"Code scanning cannot determine the alerts introduced or fixed by this pull request"

[rfay]

Recently I see failures always from codeql, "Code scanning cannot determine the alerts introduced or fixed by this pull request"

Although I have only the go language configured:

    strategy:
      fail-fast: true
      matrix:
        language: [ 'go' ]

(See https://github.com/drud/ddev/blob/a5edc5dde1a630658d7acfded2bfd11b53811f25/.github/workflows/codeql.yml#L32-L37)

it still complains that it can't find a python run.

Is this because it's a forked PR perhaps? Anyway, it's pretty unuseful behavior.

Result is in https://github.com/drud/ddev/pull/4118/checks?check_run_id=7822079521

[aeisenberg]

Early on, there was an analysis for python. See ddev/ddev@112ea75. The way that code scanning works is that it compares the latest analysis for the branch you are merging into with the analysis for the PR you are running (for each language). Code scanning then compares the two analyses to determine which alerts are new, fixed, or already existing.

Since there was once an analysis uploaded for python (presumably by accident) into the main branc, code scanning is looking for a python analysis in your PR, but it's not finding one.

The simplest thing to do is just delete the python analysis from your repo. See the GitHub API: https://docs.github.com/en/rest/code-scanning#delete-a-code-scanning-analysis-from-a-repository

[rfay]

Thanks. It's not clear to me how to know the ANALYSIS_ID that needs to be deleted.

Or did you perhaps delete it? I don't see this happening now.

I used gh api -H "Accept: application/vnd.github+json" /repos/drud/ddev/code-scanning/analyses |jq | less to see any analyses that had python... but there are none now, see gist

Originally the action had auto-detected python usage, and yes I had turned that off because it wasn't useful on this project.

[aeisenberg]

Hmmm...I didn't do anything. Your Code Scanning results pages are still showing a missing python analysis. I am not sure why it is not showing up when you list all the analyses. Let me ask internally.

[aeisenberg]

Ah...the python analyses do indeed exist, you just need to paginate your request. Try this instead:

gh api -H "Accept: application/vnd.github+json" /repos/drud/ddev/code-scanning/analyses --paginate | jq | less

And you'll see the analyses for python. Something like this will work:

gh api -H "Accept: application/vnd.github+json" -X delete /repos/drud/ddev/code-scanning/analyses -F "ref=refs/heads/master" -F "tool=CodeQL" -F "analysis_key=github/workflows/codeql.yml:analyze" -F "environment={\"language\":\"python\"}"

This will delete the most recent analysis for python on the main branch. See the docs for how to delete all of the analyses.

[rfay]

gh api -H "Accept: application/vnd.github+json" -X delete /repos/drud/ddev/code-scanning/analyses -F "ref=refs/heads/master" -F "tool=CodeQL" -F "analysis_key=github/workflows/codeql.yml:analyze" -F "environment={\"language\":\"python\"}" gets a 403.

`gh api \

--method DELETE
-H "Accept: application/vnd.github+json"
/repos/drud/ddev/code-scanning/analyses/31312199`

(using the python analysis found with gh api -H "Accept: application/vnd.github+json" /repos/drud/ddev/code-scanning/analyses --paginate | jq | less almost works, but gets gh: Analysis is last of its type and deletion may result in the loss of historical alert data. Please specify confirm_delete. (HTTP 400)

"Analysis is last of its type and deletion may result in the loss of historical alert data. Please specify confirm_delete"

How do I specify confim_delete ? :) I see it as a query parameter in https://docs.github.com/en/rest/code-scanning#delete-a-code-scanning-analysis-from-a-repository but there's no example how to use it.

I tried gh api --method DELETE -H "Accept: application/vnd.github+json" --field confirm_delete=true /repos/drud/ddev/code-scanning/analyses/31312199 but without success, same message gh: Analysis is last of its type and deletion may result in the loss of historical alert data. Please specify confirm_delete. (HTTP 400)

[aeisenberg]

Not sure why that isn't working for you. Maybe try the curl variant of the API (just replace <TOKEN> with your token:

curl \
  -X DELETE \
  -H "Accept: application/vnd.github+json" \ 
  -H "Authorization: token <TOKEN>" \
  https://api.github.com/repos/drud/ddev/code-scanning/analyses/31312199?confirm_delete

[rfay]

Well, I used the curl to get rid of that one, but there are more, and

curl -X DELETE -H "Accept: application/vnd.github+json" -H "Authorization: token $GITHUB_TOKEN" https://api.github.com/repos/drud/ddev/code-scanning/analyses/31308286?confirm_delete

gets

{
  "message": "Analysis specified is not deletable.",
  "documentation_url": "https://docs.github.com/rest/reference/code-scanning#delete-a-code-scanning-analysis-from-a-repository"
}

I see that that one has "deletable": false,:

  {
    "ref": "refs/heads/master",
    "commit_sha": "125f700de5ae37d9686359a4a6bff4aea9b0f2a3",
    "analysis_key": ".github/workflows/codeql.yml:analyze",
    "environment": "{\"language\":\"python\"}",
    "category": ".github/workflows/codeql.yml:analyze/language:python",
    "error": "",
    "created_at": "2022-06-03T23:27:27Z",
    "results_count": 0,
    "rules_count": 34,
    "id": 31308286,
    "url": "https://api.github.com/repos/drud/ddev/code-scanning/analyses/31308286",
    "sarif_id": "bad68c30-e394-11ec-90b4-161f2c5671f5",
    "tool": {
      "name": "CodeQL",
      "guid": null,
      "version": "2.9.2"
    },
    "deletable": false,
    "warning": ""
  },

In fact, there are several more python items that are not deletable, and there are 16,900 of these python ones still out there.

Is this a lost cause? Is there any way to wipe it all out and start over? I thought probably codeql had some value but am not seeing it at this point.

[aeisenberg]

I apologize that you are having so much difficulty with this and I understand your frustration. Managing and deleting old analyses is something we are discussing internally to make this process easier. I'm discussing internally to see what the best way forward is.

[rfay]

Thanks. I think there's a bug here too... the OP describes misbehavior, not just something I can fix by deleting one analysis, and there's nothing that was done wrong in the use of this action.

So:

• Please fix it so it only compares to the current or recent analyses on the target branch.
• Please fix it so it only compares to analyses that are currently enabled on the target branch.

[thapabishwa]

gh api -X GET -H "Accept: application/vnd.github+json" /repos/xyz-org/abc-repo/code-scanning/analyses | jq '.[].id' | xargs -I {} gh api \
  --method DELETE \
  -H "Accept: application/vnd.github+json" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
"/repos/xyz-org/abc-repo/code-scanning/analyses/{}?confirm_delete"

I ran this command couple of times until this stopped returning code-scanning results

gh api -X GET -H "Accept: application/vnd.github+json" /repos/xyz-org/abc-repo/code-scanning/analyses