Build pipeline security

[pcrockett]

Since supply chain attacks seem to be all the rage these days, what's our stance on our build pipeline?

I see a few missing best practices. However at the same time I think we're in pretty good shape:

good things

• The only third-party GitHub actions we use come from docker, github, and ruby orgs. Well-established, reputable.
• Other inputs into our pipeline (besides our own code, Ruby dependencies, and the GitHub runner itself) are Ubuntu packages, and @DannyBen's own Rush repository.
• The Rush repository pulls a few dependencies from GitHub releases and pipx (always installing latest versions of things).
• I don't see who all has push access to the repository, however I believe it's a small number. Which is good, because they can all theoretically gain access to repository secrets (Docker credentials) via a malicious PR.

So I'm not terribly concerned. Room for improvement though.

unanswered questions

I'm not a Ruby dev, so excuse me if any of these are obvious:

• How are Ruby dependency updates managed? I see no Dependabot or Renovate config, so I wonder what the dependency update routine is.
• I see no lockfile committed to the repository. I would expect that for a library, but Bashly is a CLI program. That is unexpected -- what am I missing?

do we care?

Since there's now a good amount of downstream software being built with Bashly, do we think it would be wise to tighten things up a bit, exercise a bit more rigor over the inputs to our build pipeline?

[DannyBen]

Hey Phil, thanks for raising.

To start from the bottom:

do we care?

Yes. Tighter security that does not interfere with development and does not add unreasonable maintenance overhead is welcome.

My thoughts about the issues you mentioned:

Use of rush in GitHub Actions

This can easily be avoided if needed. It is a convenience wrapper to bring shfmt and check-jsonschema

GitHub write permissions

Several people have permissions. You, me and 4 more. I think we can remove some that are no longer active.

Furthermore, I should probably review the secrets and consider limiting scope where appropriate.

RubyGems push permissions

You did not mention this (unless this is what you meant by push permissions) - only me.
This is a different concern, if we want to think about continuity.

Ruby dependencies update

Dependabot is noisy and creates an overhead. Not used for dependency updates, by design.
It is however configured to alert when any dependency has vulneraability.

I sometimes check outdated gems, and if needed, update the gemspec with higher optimistic versions.

I recently added the new cooldown feature by rubygems to the Gemfile: source 'https://rubygems.org', cooldown: 7
This only impacts developers of the gem, not consumers.

Lock file

but Bashly is a CLI program. That is unexpected

I think you are probably more used to seeing lockfiles in compiled binaries. In this case, you want to use a completely frozen manifest for compilation.

For interpreted CLIs that use shared libraries, versioning is done via a manifest that allows a range of versions (aka optimistic versioning) on the user's computer.
In Ruby, this is expressed by the gemspec. You will notice that for some gems I was more strict than others:

  s.add_dependency 'colsole', '~> 1.0'
  s.add_dependency 'completely', '~> 0.7.0'
  s.add_dependency 'gtx', '~> 0.1.1'
  s.add_dependency 'listen', '~> 3.9'
  # ...

So no, lock file should not be committed - in fact, it will do more harm than good, as it will most certainly force a development environment that has older versions than what the user may have.

[DannyBen]

I hardened the docker workflows to use an environment that requires my approval (16d00c6)

[pcrockett]

I missed RubyGems push permissions -- is that in a GitHub workflow? Or do you do that locally?

Lockfiles

The "compiled" / "not compiled" distinction doesn't seem to be true, at least not anymore. These days I find it very difficult to find any end-user applications without a lockfile, regardless whether you're running Rust, Go, JavaScript, or Python. The "whether to use a lockfile" conversation usually hangs on whether or not you're building a library or application. In the ecosystems I know, applications always have lockfiles.

The key, however, is an auto-update mechanism. As you rightly said, lockfiles make things get stale. But not if you have a good auto-update mechanism. Lockfiles and auto-updates are like peas and carrots. They go well together.

Best practice

Interestingly enough, Bundler's documentation recommends always committing a lockfile, regardless whether you're building an end-user application or a gem:

As a result, you SHOULD check your Gemfile.lock into version control, in both applications and gems. If you do not, every machine that checks out your repository (including your production server) will resolve all dependencies again, which will result in different versions of third-party code being used if any of the gems in the Gemfile(5) or any of their dependencies have been updated.

But I'm not a believer in blindly following best practices. It is best to understand the practice before deciding whether to adopt it or not.

The main problem a lockfile solves: Whatever we test in CI or on a dev machine, is not guaranteed to be the same as what gets baked into the Dockerfile or gem. And if I understand correctly, when we pull the Bashly gem down to our dev machines, we will be re-resolving dependencies again, which will NOT correspond with what was tested in CI.

Other important problems get solved there too -- For example if a dependency was compromised, we can tell whether or not that version was shipped in any release. And it prevents brief compromises in our dependencies from ever reaching end-user machines as well.

Dependabot noise

I'm with you on this one. However with some tweaks to settings, an auto-update mechanism can be very not-noisy. Some ideas:

• Monthly updates
• Bundle all dependency updates into a single PR (so only one update per month besides security updates)
• Use Renovate instead, and turn on auto-merge. Since we have robust CI, and cooldown enabled, we can fairly reliably merge dependency update PRs with zero interaction.

What I can do

I propose an experiment: If you're open to adding a lockfile and turning on auto-updates, I can support. My day job is platform engineering; my job exists to make developers' lives better. Whatever friction you encounter with such a setup, I'd be glad to help resolve.

[DannyBen]

I missed RubyGems push permissions -- is that in a GitHub workflow? Or do you do that locally?

Yes. run gem push locally.

Lockfiles

Committing them does not solve any problem, it just adds more problems.
Gemfile.lock has zero impact on the user's release. Users who do gem install bashly get their versions from the gemspec, not the Gemfile / Gemfile.lock.

So by this logic, the "best practice" recommendation should also require gem developers to use fixed versions in the gemspec.

The gem - like any other gem - is expected to work with various versions of its dependencies. This is why semver exists.
We should not care which version of rspec developers use as long as it is known to be compatible with our code.

Claiming this mechanism for security reasons is throwing the baby with the bath water.

The main problem a lockfile solves: Whatever we test in CI or on a dev machine

This is not solving anything, it is creating a problem.
You want CI and any developer running the test to fail if the app no longer works with the newer-but-still-in-range gems.

This gemspec definition - s.add_dependency 'colsole', '~> 1.0' - expects any 1.x version to work. If it fails, it means that the developer of this gem did not follow "non breaking change" rule, and in this case, it is best for a CI to catch it as early as possible. And when it does, the solution would not be to add a lockfile, but to either lock the version harder in the gemspec, or fix the problem, or report a bug to the upstream.

Imagine that every tool you install that needs curl also needs its own specific locked version of curl. That is what committing a lockfile is akin to.

regardless whether you're running Rust, Go, JavaScript, or Python. The "whether to use a lockfile" conversation usually hangs on whether or not you're building a library or application. In the ecosystems I know, applications always have lockfiles.

• Rust, Go: Compiled, the dependencies are not installed separately on the user's computer. Totally different dependency story.
• Javascript: Javascript is different, and it is probably the worst at managing dependencies. The sheer amount of dependency managers that come to improve on their predecessor should tell you that. It is not going to be my example of how to do things right. Besides, to my knowledge - it is the same deal. Lock files do not impact end users (assuming a CLI app), and it has a manifest that defines versions as tightly or loosely as it wants
• Python: Same as Ruby. It has a dependency manifest with tight or loose versioning. Committing a lockfile may create the same problems here.

These days I find it very difficult to find any end-user applications without a lockfile

You probably did not look at any repo for a Ruby gem. I cannot find one with a lcokfile committed. First 5 examples.

• rspec
• amazing_print
• sqlite3-ruby
• tty-markdown
• sinatra

Dependabot updates

There is no need. If we use optimistic versioning, the cadence in which we need to update dependencies is slow and manageable.
I love automation, and tools that automate - but in this case, it only caused maintenance noise without any benefit.

---

What I can do

Thank you for that. I am open to listening and learning of any possible improvements, but these particular remaining issues - lockfile and dependabot - I have not seen any reason to implement, only reasons to avoid.

[pcrockett]

libraries shouldn't have lockfiles.

rspec, amazing_print, sqlite3-ruby, tty-markdown, sinatra -- all libraries.

applications, however:

• gitlab
• mastodon
• discourse
• homebrew
• diaspora
• shopify

to be fair, i did find a couple that don't have lockfiles that i can see: RuboCop and Vagrant.

in any case, the point is: the VAST majority of applications in 2026, regardless of ecosystem, use a lockfile + autoupdate workflow. i use it extensively, have for years. i RELY on the fact that the same code gets deployed reliably, everywhere. lockfiles to me aren't burdens; they make the software development lifecycle so much easier. i'd hate to lose them.

one has to wonder: if so many people find value in this way of doing things, perhaps lockfiles don't "just add problems."

[DannyBen]

Seems like you have a solution looking for a problem.

Lock files in ruby gems are for developers, not users. They do not solve anything for gem release safety, and they do not solve any problem that any developer encountered.

You are again comparing the wrong things. Your examples:
Shopify, Diaspora, Discourse, mastodon, Gitlab - these are web applications. In these, adding the lockfile serves a different purpose.
I don't know why Homebrew included their lock file - they are the exception, not the norm.

You would normally find lock files in things where the repository itself is cloned by end users, like web applications, or in compiled applications, or sometimes in frameworks that use their own local package space (node_modules).

Anywhere else, the manifest (with wildcard / optimistic versions) are the single source of truth, and the lock file is a gitignored artifact.

You said you are not a ruby developer - so maybe at least we can agree that those who are actually developing this repository know what is good for them better than those who don't? If any ruby developer working on this gem can point to a problem caused by not having the lock file committed, I am open to continue this discussion.

[pcrockett]

ok. so "for developers, not users" is what i should have focused on for my understanding.

i did more research. i've learned a lot. and wow. WOW.

if the code runs in CI and we cut a new release, and then an hour later one of bashly's transitive dependencies pushes out a new version, that new version gets shipped directly to bashly users without ever seeing our CI pipeline. because users install via bundler / RubyGems.

even if our gemspec used = instead of ~> for all of its version constraints, this would still be true. because we don't have control over the transitive dependencies, just the direct ones.

is that correct? if so... wow. let's just hope all the malicious actors out there keep focusing on attacking npm package maintainers instead of ruby gem maintainers.

---

fwiw i found why homebrew provides a lockfile:

Homebrew’s runtime Ruby dependencies are vendored in-tree and pinned in Library/Homebrew/Gemfile.lock, rather than resolved live from RubyGems during normal brew execution. Updates to those dependencies are reviewed like other Homebrew changes, with Dependabot cooldowns providing an additional delay for newly published gem releases. This reduces the package manager’s own dependency chain as an attack surface, a recurring weak point in ecosystems where package-manager plugins or runtime libraries are resolved dynamically at execution time.

source

so in other words, homebrew thinks a little more like me.

[DannyBen]

If I am not mistaken, homebrew is not installed as a gem, but rather as a cloned repo. All of "brews" concept is "lets bring the repo and compile here". So it makes sense that they bring the lockfile, now that you mention it, and assuming I recall correctly.

Yes - I believe you are right, even if the direct dependencies are pinned with "=", sub-dependencies may still be different.

The trendy panic from supply chain contamination is understandable, but if we are there, then you can also never fully trust any installation from open source. Why do you trust that pacman/apt/dnf/brew bring uncontaminated versions? We have seen this happen.

I think the protection measures should be careful not to nullify the concept of version ranges, semver and optimistic versioning.
Protection should focus on how to prevent malicious actors from deploying to central package managers, and not on the distribution of corrupted packages. Target the source, not the plumbing.