Accessibility issues with the Copilot on VSCode

[aamir51]

Hello all.
I received an invitation to the github copilot today, and I was very excited. However after installing the extension I must say I was very disappointed.
The accessibility is almost non-existent for screen-reader users. The suggestions are not read with screen readers (at least with NVDA), and even if you open the suggestions in a new tab, I did not find a way to except the suggested solutions.
Can someone look in to it please? thank you. The copilot is certainly a very exciting project, and we(screen-reader users) are just as excited as anyone else. However, if you can't use it, there is a very little point in getting excited about it
best wishes

[isidorn]

@aamir51 thank you very much for your feedback and sorry for the slow response.

We have been working in the previous milestone to make Copilot more accessible. Can you try out the experience now in VS Code insiders https://code.visualstudio.com/insiders/ Or in VS Code stable from Thursday

The idea is that now there will be an audio cue when there is Copilot suggestion present and you can press Cmd + K, Cmd + I to interact with the suggestions.
We are really looking forward to feedback here on how to improve this further.

fyi @hediet

[aamir51]

Hello @isidorn
I just tried what you suggested, however I don't hear anything when I press alt+i or alt+k. I do however hear a sound when I navigate to the body of a function for example
please let me know if you need anymore information
thanks

[isidorn]

@aamir51 can you try again by pressing ctrl+i, ctrl+k. This is a chord, so it should be pressed one after the other.

[aamir51]

Hello @isidorn. pressing ctrl+i brings up a suggestion list with python keywords nothing related to copilot. pressing ctrl+k however does nothing. the suggestions list does show up if I open a document in a new copilot tab with ctrl+enter

[jooyoungseo]

Please allow me to chime in. @aamir51 -- when Copilot suggestion is available, it's presented as a tooltip element. In order to put your focus to the tooltip area, press ctrl+k, ctrl+i. Keep pressing ctrl key, and press k, followed by i, and then release ctrl key. Once your focus moves to the suggestion tooltip area, use screen reader's browse mode. In NVDA, you need to use NVDA+Spacebar to switch between focus and browse mode. @isidorn -- I noticed that your answer above suggested ctrl+i, ctrl+k. It's the other way around.

@isidorn -- I have an accessibility suggestion for VSCode Copilot extension. For screen reader users, it's better to navigate the copilot suggestions in a separate read-only tab instead of the tooltip. As you know, currently, users can open a side-by-side copilot tab via "Open GitHub Copilot, Control+Enter" command. Although this read-only tab is fully navigable by using standard arrow keys, it has two accessibility issues:

• Issue 1: In the Copilot separate tab, there is no easy way to accept currently focused suggestion because there is no assigned or configurable keyboard shortcut. Users have to manually find and click the "Accept Suggestion" button, which is extra challenging and confusing to screen reader users. This echoes what @aamir51 reported above.

• Issue 2: Also in the Copilot tab, there is no easy way to jump to the next and previous suggestion blocks. Unlike the Copilot tooltip suggestion, pressing either alt+] or alt+[ does not work when using the side-by-side Copilot tab. Currently, each Copilot suggestion block is separated by "=======" divider. Screen reader users have to tediously navigate each suggestion chunk using line-by-line reading. Please add configurable shortcut keys for the Copilot tab just like the Copilot tooltip so that we can quickly jump between suggested code blocks.

[mikolysz]

I think it would be really helpful if we had a keyboard shortcut to announce the current copilot suggestion. Also, cycling between suggestions with Option + left and right bracket should definitely announce them automatically. Since cycling between suggestions is clearly a user-initiated action, I can't see a use case where a screen reader user wouldn't want the new suggestion announced.

I'd personally like suggestions to be announced as they appear, though I see why others might not, so I suggest allowing this behavior as a configuration option.

[Marc2p]

Hello,
I recently started testing Copilot and like many people, I'm amazed at the results.
I am a blind person, screen reader user. I've found some accessibility issues in the Copilot extension for Vs Code that I'd like to report, hoping to get a quick fix.

First of all, it would be nice to improve the way you navigate Copilot's suggestions.
Currently, you hear a sound, then you have to press ctrl+K, then ctrl+I, then navigate through the suggestions by pressing alt+¿ or alt+'
Suggestions are not automatically spoken by the screen reader.
The problem with this is that time is wasted. If you keep typing after hearing the sound, the copilot suggestions will probably disappear and nothing will happen when you press ctrl+K and ctrl+I.

Also, if intellisense suggestions appear, I think they take precedence over copilot suggestions, so pressing tab will insert the intellisense suggestion, also preventing navigation of copilot suggestions when pressing ctrl+K and ctrl+I

Lastly, if the Copilot tab is opened by pressing ctrl+enter, there is no easy way to accept or reject a solution, or even to quickly navigate between suggestions.

It would be interesting if work could be done on these issues in order to take advantage of the full potential of such a valuable tool when it comes to saving time, under the same conditions as a person without visual impairment.

I am currently using a machine running Windows 11 in its latest version, as well as the latest stable versions of NVDA, Visual Studio Code and Copilot

Best regards

[hamidrezaabroshan]

Any update about this issue?

[jooyoungseo]

CC @meganrogge

[meganrogge]

With the suggestions/insights of @jooyoungseo, we are actively working to improve this experience this iteration and next.

Some of the issues to follow along with:

• [Accessibility] div.monaco-tokenized-source requires aria-live="assertive" microsoft/vscode#167348
• When in tabFocusMode, assign a different keybinding for inline suggestion acceptance microsoft/vscode#174360
• Next suggestion isn't read microsoft/vscode#174362

cc @isidorn

[julienbenaouda]

Hello

Any updates on this?
I am also visually impaired and using a screen reader (Jaws in my case).
I just installed the extension and did some testing, but seems that ctrl+k-ctrl+i isn't working, the tooltips are not read out. I can use ctrl+enter to read the suggestions in a separate tab, but there still seems to be no way to reach the "accept suggestion" button.

[isidorn]

We are actively working on improving this experience. Are you using the latest VS Code and latest version of the Copilot extension?
Accept suggestion command should now be accessible and @kimadeline should have more details since she fixed this.

[kimadeline]

Hi @julienbenaouda!

The tooltips being unreachable are a known problem, however it needs to be fixed on the VS Code side. There is an open issue to track this: microsoft/vscode#144060.

When you open suggestions in a separate tab, this tab supports keyboard navigation: You can use alt+[ and alt+] to navigate to the previous/next suggestion in the panel, and you can use ctrl+/ to accept the selected suggestion.

We added that support this month, so you will need Copilot version 1.83.41 or later to be able to use keyboard navigation in the Copilot panel.

[julienbenaouda]

Thanks @kimadeline and @isidorn works nicely!

[yncat]

I just started using copilot today, and this addition is soooo helpful for me. What a nice timing and what a coincidence!

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[mlama007]

Hello @aamir51, we just released a guide on using GitHub Copilot in Visual Studio Code with a screen reader. This guide covers instructions specific to Windows on a desktop. For macOS or Linux, some commands and shortcuts may differ.