chroot support

[kingces95]

Hello codespaces!

I expect to be able to create a chroot in a codespace without having to modify how /workspaces is mounted and once inside the chroot I expect subprocess to work. Actually, I need to modify the /workspaces mount and when I get into the chroot subprocess don't work.

sudo mount -i -o remount,exec,dev /workspaces/

I'm not quite sure what this does, but it does allow me to use debootstrap to create the environment. I can enter the chroot environment, but operations in the environment that make use of mounted directories have issues. For example, subprocess don't work.

cat < <(echo test)
bash: /dev/fd/63: No such file or directory

StackOverflow tells me the issue is related to the way the /proc is mounted, but not being familiar with UNIX mounting configurations, I'm unsure what the problem is. Can change the parameters I pass to debootstrap to make things work? Have I bumped up against some inherit limitation in codespaces?

I've created this little 7 step reproduction that can easily be spun up in a codespace. kingces95/chroot-mounts: Reporduce subprocess failures in codespace chroot environment. (github.com).

1 $ sudo mount -i -o remount,exec,dev /workspaces/
2 $ sudo apt-get install -y binutils debootstrap
3 $ CHROOT=/workspaces/chroot
4 $ mkdir -p "${CHROOT}"
5 $ sudo debootstrap stable "${CHROOT}" http://deb.debian.org/debian/
6 $ sudo chroot "${CHROOT}"
7 $ cat < <(echo test)
bash: /dev/fd/63: No such file or directory

---

My overarching goal is to standup a bash environment my teammates can use to reproduce integration tests failures. I'd like my environment to be hosted by either Codespaces, GitHub actions, or WSL2 on our product DevBox. Each of those three Unix environments is slightly different. I want to hide those differences by creating a chroot within in each of those environments and then develop within the chroot.

For example, GitHub Actions has a pre-installed version of az whereas the other environments don't. I lazily install the az version I want but if one is already present, then I make use of it. If it's the wrong version, then things fail. I can fix such such issues one by one (e.g. it wouldn't be hard to detect if the pre-installed version is unexpected, uninstall it, and re-install the correct one) but if I could use chroot to create an environment without any pre-installed tooling then there are fewer cases to consider.

Thanks!

[joshspicer]

Hey! I thinking this will take a bit more thinking for me to wrap my head around, but as we discussed offline perhaps trying a directory that is not /workspaces may yield more predictable results (given how we treat /workspaces as special).

@Chuxel, you are the strongest linux guru I know 😛, do you have any thoughts on this?

[joshspicer]

Speaking offline, moving away from /workspaces allowed @kingces95 to create the chroot like they had wanted.

@kingces95 is still looking for some guidance on the subprocess bit (https://unix.stackexchange.com/questions/708066/subprocess-launched-inside-a-chroot-created-on-a-codespace-with-debootstrap-fail)

[Chuxel]

I've never tried doing this myself, but a few thoughts...

First, try enabling privileged (--priviledged in runArgs or in your Docker Compose file) if you have not. There are certain capabilities that are disabled by default for security reasons. (Some discussion here).

Second, since containers use chroot themselves, you're already in a namespaced environment. For docker-in-docker, there's additional mounts and cgroup tweaks that are made to get docker to work within a container even beyond privileged. See the code here https://github.com/microsoft/vscode-dev-containers/blob/d98fef4fd8ae814c6056db8d936009fef97bed4b/script-library/docker-in-docker-debian.sh#L337

Third, the workspace folder is a mount done by the container engine, so that could also be causing some trouble (again, not sure). Locally the workspaceMount property can be used to customize the mount point, but that's not available in Codespaces AFAIK. I have, however, used overlayFS on top of this mount point before, so that may or may not be related.

[Chuxel]

Oh, three other points:

1. You should be able to create a customized dev container that contains everything that you need if you have not tried this yet.
2. You can use this dev container in GitHub Actions directly via the new dev container Action. https://github.com/devcontainers/ci/blob/main/docs/github-action.md and there's an OSS CLI for other scenarios (https://github.com/devcontainers/cli)
3. FWIW - overhead in WSL2 for containers if the source code is on that side is pretty minimal. You could use WSL + Remote - Containers with the same dev container config. See https://code.visualstudio.com/docs/remote/containers#_open-a-wsl-2-folder-in-a-container-on-windows You can even just run the Linux docker engine directly in WSL2 w/o Docker Desktop if you prefer.

[kingces95]

Chuxel, thanks for all the pointers. Starting there I eventually came to understand the problem was simply that /proc was not mounted. Running the following in the chroot solved the subprocess problem.

sudo mount --types proc /proc proc/

Thanks for the pointer on how to run a GitHub Action using the codespace image. If I hit an insurmountable obstacle with chroot I now have something to fall back to. Thanks!