Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,25 @@ uv run copilot-experiments show --last
uv run copilot-experiments analyze --last
```

To use a local checkout of this tool from any other directory, run the console script through
`uvx --from` instead of syncing this repository first:

```bash
# Use the path to your local github-copilot-lab checkout.
export COPILOT_EXPERIMENTS_REPO=/path/to/github-copilot-lab

uvx --from "$COPILOT_EXPERIMENTS_REPO" copilot-experiments init my-experiments
cd my-experiments
uvx --from "$COPILOT_EXPERIMENTS_REPO" copilot-experiments run --dry-run
uvx --from "$COPILOT_EXPERIMENTS_REPO" copilot-experiments show --last
```

In PowerShell, set
`$env:COPILOT_EXPERIMENTS_REPO = "C:\path\to\github-copilot-lab"` and use
`uvx --from $env:COPILOT_EXPERIMENTS_REPO copilot-experiments ...`. This installs from the
current local checkout into uv's cache. If you are iterating on the tool and need to force uv to
rebuild from the working tree, add `--no-cache` before `--from`.

Real runs require Copilot auth (`COPILOT_GITHUB_TOKEN`, `GH_TOKEN`, `GITHUB_TOKEN`, or `gh auth
login`) and a Pier-supported execution backend such as Docker.

Expand Down
18 changes: 18 additions & 0 deletions docs/authoring-experiments.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,24 @@ uv run copilot-experiments show --last
uv run copilot-experiments analyze --last --trial 1
```

If you are working from a standalone experiment repo and want to use a local checkout of the
`copilot-experiments` tool, replace `uv run copilot-experiments ...` with the form
`uvx --from <tool-repo> copilot-experiments ...`:

```bash
export COPILOT_EXPERIMENTS_REPO=/path/to/github-copilot-lab

uvx --from "$COPILOT_EXPERIMENTS_REPO" copilot-experiments run --dry-run
uvx --from "$COPILOT_EXPERIMENTS_REPO" copilot-experiments run
uvx --from "$COPILOT_EXPERIMENTS_REPO" copilot-experiments show --last
uvx --from "$COPILOT_EXPERIMENTS_REPO" copilot-experiments analyze --last --trial 1
```

In PowerShell, use
`$env:COPILOT_EXPERIMENTS_REPO = "C:\path\to\github-copilot-lab"` and pass
`--from $env:COPILOT_EXPERIMENTS_REPO`. If you are iterating on the tool and need to force uv to
rebuild from the working tree, add `--no-cache` before `--from`.

`--dry-run` validates Pier configs and path normalization without starting a sandbox. The legacy
Python experiment path still has an ephemeral mock dry-run, but Pier is the primary authoring
model.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ This repository contains **GitHub Copilot research experiments** built with the
- Keep code ruff-formatted/linted and add focused tests for behavior changes.

## Commands

If you are using a local checkout of the unpublished harness, replace
`uv run copilot-experiments` with
`uvx --from <path-to-github-copilot-lab> copilot-experiments`.

```bash
uv sync
uv run copilot-experiments run [--dry-run]
Expand Down
28 changes: 28 additions & 0 deletions src/copilot_experiments/templates/experiment_repo/README.md.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,34 @@ results/ # derived SQLite index for queries (gitignored)

## Usage

The quickest way to use an unpublished local checkout of the harness is `uvx --from`. Set the
environment variable to the path of your `github-copilot-lab` checkout, then run the CLI from this
experiment repo:

```bash
export COPILOT_EXPERIMENTS_REPO=/path/to/github-copilot-lab

# validate Pier job configs without starting Docker or spending credits
uvx --from "$COPILOT_EXPERIMENTS_REPO" copilot-experiments run --dry-run

# run for real through Pier (requires Copilot auth and a supported Pier backend)
uvx --from "$COPILOT_EXPERIMENTS_REPO" copilot-experiments run
uvx --from "$COPILOT_EXPERIMENTS_REPO" copilot-experiments show --last

# explore results
uvx --from "$COPILOT_EXPERIMENTS_REPO" copilot-experiments list
uvx --from "$COPILOT_EXPERIMENTS_REPO" copilot-experiments inspect <job-name> --trial 1
uvx --from "$COPILOT_EXPERIMENTS_REPO" copilot-experiments analyze <job-name> --trial 1
```

In PowerShell, use
`$env:COPILOT_EXPERIMENTS_REPO = "C:\path\to\github-copilot-lab"` and pass
`--from $env:COPILOT_EXPERIMENTS_REPO`. If you are iterating on the harness and need to force uv
to rebuild from the working tree, add `--no-cache` before `--from`.

If you prefer to install the harness into this repo's virtual environment, use the normal uv
workflow:

```bash
uv sync

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ requires-python = ">=3.12"
dependencies = [
# The experiment harness. Until published to PyPI, install from git (or replace
# with a local editable path: `copilot-experiments @ file:///path/to/repo`).
# To use an unpublished local checkout without syncing this repo, run:
# uvx --from /path/to/github-copilot-lab copilot-experiments ...
# Add --no-cache before --from when iterating on the harness working tree.
"copilot-experiments @ git+https://github.com/dbroeglin/github-copilot-lab.git",
]

Expand Down
Loading