Support on-device models via LiteRT

[seehorne]

Is your feature request related to a problem? Please describe.
Currently, all models for all tools run on the server, which can contribute to latency, since the server must act as a middle man for all input and output. Consequently, it would be beneficial to be able to run models on-device when possible.

Describe the solution you'd like

• Look into LiteRT and LiteRT-LM. These are platforms supported by google for running models on edge devices.
• Determine how to connect these to ProgramAT app (recall, we use React Native, not Swift)
• Update Copilot instructions to support generating tools that use on-device model
• Update server to support running tools where the model does not run on server but rather on-device.
• Make some on-device tools to test it works (you do not have to, nor should you, include these tools with the PR for this issue)
• Update Copilot instructions with logic for when a tool should be made to run on device, informed by the above tests
• Either refactor the "Conversation" to be able to use the image given to the on-device model when an On Device tool is run in Take Photo mode or disable "Conversation" for on-device tools and document this
• Write up/document all infrastructure changes needed to make this possible

Some things to keep in mind

• The way tools run involves the server mocking a branch switch in your repository and using an exec() call to run the tool itself. Any on-device activity will need to fit into this flow
• iOS and Testflight do not allow hot-loading new code in the app itself. We are able to have a variety of outputs (speech, haptics, earcons, etc.) by having them latent in the app and the server messages the app to call them forward. You should consider a similar approach to knowing when to leverage an on-device model.

Additional context
Note that, since this is a frontend change, testing it would require building the app yourself and using the metro dev server (i.e. you would need a Mac and XCode)

Also, this issue is particularly technically involved, and should not be undertaken without first taking time to get familiar with the codebase. This issue is probably also best for someone with pre-existing development experience.