Displaying compiler diagnostics via SARIF uploads

[LebedevRI]

Select Topic Area

Product Feedback

Body

Hello.

There's currently a bit of a push to switch
from old-school textual compiler diagnostics,
to a more user-friendly diagnostics powered by SARIF,
so that they can be nicely integrated into IDE/tools.

Github already provides a way to upload results of an
analysis in SARIF format, and they will be displayed as
new security issues. I've tried, and this is //almost// sufficient
to also handle compiler diagnostics, but not quite.

There's just one thing missing:

Unfortunately, the issues from SARIF are only posted
as in-line comments IFF the lines they appear on are in the diff.
This is, unfortunately, very obviously wrong.
Consider: https://godbolt.org/z/9MqY9rrP9

(I'm also not sure if only new issues are posted, or all are?)
(In the same vein, it would be nice if the CodeQL analysis
itself had an -Werror switch, i.e. fail if any issues were found...)

This does not seem like an ovely complex issue,
and solving that seems like the only blocker from Github side,
so, please, consider supporting displaying in-line diags on lines not-in-diff? :)
(There's an edge-case: some files might not exist in the repo at all.
I'm not sure how their diags should be handled.)

Concretely, i think this can be viewed as the following enhancements:

1. Adding fail-if-any-issues-found (aka -Werror) option to github/codeql-action/upload-sarif,
   to fail the step if the uploaded SARIF had any issues,
   regardless of whether or not they are preexisting or not.
2. Same for github/codeql-action/analyze
3. Support showing SARIF issues on the code not in the diff
4. Support showing SARIF issues on the files not in the diff, but in the repo (!)
5. Support showing all other SARIF issues, i.e. those that do not correspond to the files in the repo.
   They would probably need to be posted as normal comments in the PR.
6. Perhaps an option to always show all issues from uploaded SARIF regardless of whether or not they are preexisting

There's also the tooling question - there is no nice way
to get the compiler to produce those SARIF's in a predictable way,
and then they need to be merged into a single SARIF.
I've hacked something together: darktable-org/rawspeed#798
but probably build systems might need to handle that...

Roman.

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[hvitved]

Ping @chrisgavin , who's on call.

[LebedevRI]

Concretely, i think this can be viewed as the following enhancements:

1. Adding fail-if-any-issues-found (aka -Werror) option to github/codeql-action/upload-sarif,
   to fail the step if the uploaded SARIF had any issues,
   regardless of whether or not they are preexisting or not.
2. Same for github/codeql-action/analyze

I think these two things are now implemented.
There's now repo's /settings/security_analysis

Protection rules
Check runs failure threshold
Select the alert severity level for code scanning check runs to fail.

3. Support showing SARIF issues on the code not in the diff

This might or might not be affected by
https://github.blog/changelog/2025-09-25-pull-request-files-changed-public-preview-now-supports-commenting-on-unchanged-lines/

4. Support showing SARIF issues on the files not in the diff, but in the repo (!)
5. Support showing all other SARIF issues, i.e. those that do not correspond to the files in the repo.
   They would probably need to be posted as normal comments in the PR.
6. Perhaps an option to always show all issues from uploaded SARIF regardless of whether or not they are preexisting

There's also the tooling question - there is no nice way to get the compiler to produce those SARIF's in a predictable way, and then they need to be merged into a single SARIF. I've hacked something together: darktable-org/rawspeed#798 but probably build systems might need to handle that...

Roman.