Copilot hallucinating / using "own" memory instead actual data

[a-hulman]

Select Topic Area

Product Feedback

Copilot Feature Area

VS Code

Body

Hello,
I understand that Copilot cannot "save" my preferences, but it could actually modify some local workspace copilot setting to avoid telling Copilot every new chat to:

1. search in entire workspace, not just file(s) in context
2. perform actions with thoroughness, not negligence
3. use actual data, not some "own"

Today I discovered quite a disturbing fact - while asking copilot which version of PrimeNG library is used in the workspace, the answer was 13.4, because my packages.json says so....
Guess what - packages.json says 17.18. So the information was completely made up despite having the actual file that should be read. When I asked how come, Copilot answered that he used "popular historical ferquently used data" to speed up response....

So how extremely exact do I have to be, to get correct information???
And how much can I trust each answer, if it can be simply fabricated...

[Akash1134]

Hi @a-hulman,

Thank you for sharing this feedback about GitHub Copilot's behavior. You've highlighted an important issue with Copilot providing incorrect information despite having access to accurate data in your workspace files.

The behavior you described - where Copilot incorrectly stated your package.json contained PrimeNG version 13.4 when it actually had version 17.18 - and Copilot's explanation that it used "popular historical frequently used data" instead of reading the actual file, represents a limitation in how AI assistants sometimes operate.

Documented Approaches That May Help

According to GitHub's official documentation, these techniques may help improve accuracy:

1. Be specific in your prompts: The GitHub Docs on prompt engineering suggest being explicit - try "Read the exact content of package.json and tell me the PrimeNG version"

2. Indicate relevant code: From the official documentation: "If you are using Copilot Chat in your IDE, open the file or highlight the code that you want Copilot to reference. You can also use keywords to manually supply context to Copilot Chat."

3. Use @workspace in VS Code: The documentation notes you can add the @workspace chat participant in VS Code to expand the context Copilot uses

Your suggestion about persisting workspace-level Copilot settings is interesting and would be valuable feedback to share directly with GitHub through their feedback channels.

Documentation Resources

• Prompt engineering for GitHub Copilot
• GitHub Copilot Chat cheat sheet

Thank you again for raising this issue. It helps the community understand the current limitations and best practices when working with Copilot.

Best regards,