AWF cli-proxy DIFC proxy probe failed (diagnosis=unknown) on GitHub Enterprise Cloud data residency (*.ghe.com)

[davidkarlsen]

Summary

On a GitHub Enterprise Cloud data-residency tenant (*.ghe.com), an agentic workflow with engine: copilot fails before the agent runs: the AWF firewall's awf-cli-proxy sidecar can't complete its health probe to the external DIFC proxy, so the firewall fails fast and the agent is never invoked. The same workflow is expected to work on github.com.

Environment

• gh-aw v0.80.9; firewall v0.27.7; mcpg v0.3.27
• Standard GitHub-hosted ubuntu-24 runner
• GitHub Enterprise Cloud, data residency (*.ghe.com)
• engine: copilot with permissions: { copilot-requests: write } (org-billed inference)

Symptom

squid + api-proxy come up healthy, and the DIFC proxy reports ready:

CLI proxy ready on port 18443

…but the sidecar can't reach it:

[cli-proxy] connecting to external DIFC proxy at host.docker.internal:18443
[cli-proxy] DIFC proxy probe failed (attempt 1..6/10, diagnosis=unknown)
dependency failed to start: container awf-cli-proxy is unhealthy
[ERROR] AWF firewall failed to start: awf-cli-proxy could not connect to the
        external DIFC proxy ... The agent was never invoked.

Per containers/cli-proxy/entrypoint.sh, diagnosis=unknown means the probe (gh api rate_limit through the proxy) returned something that is neither connection-refused nor timeout — i.e. the proxy is reachable but the forwarded gh api call returns an unexpected result. That points at the DIFC proxy not resolving the enterprise API host on *.ghe.com.

Ruled out

1. Host port allowlist — added 18443 to --allow-host-ports (was 80,443,8080). No change. (A dedicated cli-proxy→host:18443 rule already exists, so this was expected to be a no-op — confirmed.)
2. Enterprise host env — set workflow-level GH_HOST / GITHUB_HOST / GITHUB_ENTERPRISE_HOST to the *.ghe.com host. No change. These reach the cli-proxy via --env-all, but the mcpg DIFC proxy is launched by actions/setup/sh/start_difc_proxy.sh with a curated env, so workflow env never reaches it.

Likely cause

The mcpg DIFC proxy started by start_difc_proxy.sh doesn't appear to be enterprise-host-aware, so the probe's forwarded gh api hits the wrong API host on *.ghe.com. Probably the same area as #21407 (compiler doesn't inject GH_HOST/telemetry for *.ghe.com) and github/gh-aw-firewall#1300 (enterprise GITHUB_API_URL → model-list 400).

Questions

1. Is engine: copilot (token steering / DIFC proxy) supported on *.ghe.com data-residency tenants today?
2. If so, what makes the mcpg DIFC proxy use the enterprise API host so the probe succeeds?
3. Either way — could the probe surface a clearer error than diagnosis=unknown (e.g. the actual gh api status/body)?

[davidkarlsen]

Update: reproduced on the latest prereleases — gh-aw v0.81.2, firewall 0.27.9, mcpg v0.3.29 — with the identical failure (DIFC proxy probe failed … diagnosis=unknown, firewall fails fast, agent never invoked). So an image upgrade does not fix it; this isn't already-resolved-on-main.

Two obvious mitigations were also retried, neither helped:

• Added 18443 to --allow-host-ports (was 80,443,8080).
• Set workflow-level GH_HOST / GITHUB_HOST / GITHUB_ENTERPRISE_HOST to the enterprise host. These reach the cli-proxy via --env-all, but not the mcpg DIFC proxy launched by actions/setup/sh/start_difc_proxy.sh with its curated env — which is consistent with the proxy not being enterprise-host-aware.

Runner throughout was a standard GitHub-hosted ubuntu-24.

[pelikhan]

@lpcox i don't see your agent message here

[lpcox]

Created companion tracking issues to address this end-to-end:

• DIFC proxy is not enterprise-host-aware on GHEC data-residency (*.ghe.com) gh-aw-mcpg#8202 — DIFC proxy enterprise-host awareness (root cause)
• Compiler does not propagate enterprise host into curated DIFC-proxy env on *.ghe.com #41911 — compiler/env propagation: inject the enterprise host into the curated DIFC-proxy env
• Improve awf-cli-proxy DIFC probe diagnostics (replace opaque diagnosis=unknown) gh-aw-firewall#5615 — awf-cli-proxy probe diagnostics: replace the opaque diagnosis=unknown with the actual HTTP status/body (ask Add workflow: githubnext/agentics/weekly-research #3)