Overall
We make the user do a bunch of stuff and provide a bunch of values, but we don't help orient them in context, and it feels like we're asking for things out of order and in a contextual vacuum. As a first-time user:
- What on earth is a profile, and how should I think about it?
- Key concept: tightly-scoped profiles. Think of these like "BitBucket at work", "GitHub at home", etc. -- this drives how to scope a profile.
- A profile consists of X, which configures foo, Y which points to the credentials for bar, and Z which frobulates the fizzblat
- Reviewer entity: can be you, can be a separate github user account, can be a github app depending on repo and org permission settings
- Reviewer entities are the things that REQUEST_CHANGES, APPROVE, or COMMENT on pull requests
- Etc.
- Don't ask for things like "credential refs" (what's a credential ref???) if the user hasn't created anything related to credentials yet. They have nothing to point to, and they likely don't understand the concept yet.
- Managing credentials is completely outside the
init flow
- As a first-time user, why are you asking me about GitHub Apps vs PATs? Isn't that related to security? What are you using these for??? What am I specifying right now‽
Getting granular
Thoughts while editing default profile:
- Users won't know what a "credential ref" is -- the term has never been introduced in the init flow.
- Erm, how do I know whether to use a PAT or an App?
- Git host is more like "scope"
- More generally "default" is the "fallback", I think?
- If it hadn't been already set up, I wouldn't know how/what to enter for git host... https://? My org's URL? etc. Provide likely examples. github.com is ok, but doesn't help the user think about more complex/specified cases like they might find at work.
After working thru those options, I am presented with:
- I don't know what to do here -- it's not clear what "Reviewer credential auth mode" is/means, or how it relates to a profile from the previous screen
- And why am I specifying PAT again? Didn't I already do that?
- And again we have a credential ref -- what does that mean?
LLM provider portion:
- I mostly understand this page, because I understand LLM providers.
- It's a LITTLE strange that selecting "Anthropic" will allow you to select Codex CLI later on, but I guess there are third-party harnesses -- but in reality, harness is orthogonal to "authenticating with an LLM provider behind the scenes". Can you actually use Codex in Claude Code or Fireworks.ai in Codex?
- We should put a
(recommended) on the model tier suggestion for the "built-in default"
- I'm not sure "built-in default" is the right wording -- maybe it should be "model tier specified by the reviewer prompt"? I don't think
cr has built-in defaults, does it? This should be driven by the index.yaml files...
After completing cr init I get this message:
Initialized profile default
Next: cr set-credential --ref codereview/default-reviewer --key git_token --stdin
- Shouldn't I be able to initialize another profile without running
init again?
- Why do I have to set-credential in the terminal? Why can't I do this during
init following a "paste my secret" / "read my secret from clipboard" flow like our other CLIs do?
- This is especially bad because in the first step of creating/editing a profile, we're asked about credential refs, and this text makes it seem like these are pre-requisites or at least required to use the utility successfully
Overall
We make the user do a bunch of stuff and provide a bunch of values, but we don't help orient them in context, and it feels like we're asking for things out of order and in a contextual vacuum. As a first-time user:
initflowGetting granular
Thoughts while editing
defaultprofile:After working thru those options, I am presented with:
LLM provider portion:
(recommended)on the model tier suggestion for the "built-in default"crhas built-in defaults, does it? This should be driven by the index.yaml files...After completing
cr initI get this message:initagain?initfollowing a "paste my secret" / "read my secret from clipboard" flow like our other CLIs do?