Skip to content

Nearest Exit Locator #95

Description

@yushanwe

Feature Description
Locate the nearest exit door and guide the user towards it

Problem It Solves
Difficulty locating the nearest exit in large buildings

Proposed Solution
Nearest Exit Locator

Implementation details

Assume each generated tool implements one user-facing task. If this issue enumerates multiple stages, implement them in the listed order and pass earlier results to later stages when needed. Planner stages contain only a goal and capability. Each model-backed stage MUST map to its own copilot_llm_call(...) using that capability as task_category. For any LLM or VLM operation, generated tools must call the existing backend capability interface through the existing client with from model_router_client import copilot_llm_call. Choose only from these task categories: general_reasoning, ocr, object_detection_localization, structured_visual_understanding, spatial_reasoning, navigation, camera_motion, or temporal_reasoning. Never use visual_reasoning. The backend applies the ordered execution policy and sufficiency policy for each declared capability. Generated tools must not choose implementations, models, providers, detector backends, or fallback order. Do not implement detection, OCR, VLM, LLM, model loading, or provider calls inside generated tool files. Generated tools must not create routers, capability registries, detector/OCR/LLM wrappers, new model-router clients, provider-specific DEFAULT_MODEL constants, COCO_CLASSES, .pt model loading/discovery logic, or direct provider calls.

Alternatives Considered

Example usage
If I am inside a building, identify the nearest exit door and guide me toward it

Live Mode
no

Live Query

Additional Context
Large building settings, such as malls or airports
Unless otherwise specified, in streaming mode, any verbal/text response should be limited to 15 words. No such limit applies to one-shot output.

Task Stages

Stage 1

  • Goal: Identify the nearest exit door.
  • Capability: object_detection_localization

Stage 2

  • Goal: Guide me toward the detected exit door.
  • Capability: navigation

Write the code for this tool inside the tools folder. Assume the tool implements one user-facing task. If the issue enumerates multiple stages, implement those stages sequentially in the generated tool and pass intermediate outputs from earlier stages into later stages. For any issue with a Task Stages section, each model-backed stage must map to its own copilot_llm_call(...), and each call must use exactly the listed stage capability as task_category. Do not collapse multiple stage capabilities into a single model call. For any LLM or VLM operation, generated tools must call the existing Copilot-routed backend interface through the existing client with from model_router_client import copilot_llm_call, then copilot_llm_call(task_category=..., messages=..., images=..., metadata=...) for the relevant stage capability. Use only these task_category values: general_reasoning, ocr, object_detection, map_web, spatial_relationship, navigation, camera_motion, video. Never use visual_reasoning. Do not import litellm, call litellm.completion(), create new model-router clients, create ModelRouter classes, resolve model names, resolve API keys, import detector libraries/provider SDKs/YOLO, define COCO_CLASSES, hardcode model names, load .pt files, or call YOLO(...). The model router only selects models for capabilities; do not ask it to execute stages, manage workflows, pass outputs between stages, or orchestrate pipelines.

Metadata

Metadata

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions