Consider adding a standalone shell engine that doesn't rely on Knitr or Jupyter

[ndusek]

Quarto CLI seems to be very useful for creating basic Linux and Bash shell documentation. It would be nice if Quarto supported syntax highlighting and execution of shell code without the need for the full Knitr or Jupyter stacks.

What would it take to create a standalone execution engine just for the shell?

[mcanouil]

What about Windows user? 🤔
It seems a lot of work and not for all users, especially because some are using PowerShell, BASH, etc.
And where to stop, an engine for each programming language?

In my opinion, Quarto is not about building engines, but is about the publishing/editing/writing side.
This is why, they are using existing tools to perform the computations.

This being said, if you are aware of some existing tool such as Jupyter/knitr/etc., please do share to Quarto team.
You could also try to build what you want and ask Quarto how it could be used inside Quarto.

Final note, please note that knitr/R or Jupyter/Python are not installed nor provided by Quarto, they are external tools.

[MHellmund]

This is not exactly what you asked for, but the nice thing about jupyter is that there exist jupyter kernels for a lot of languages.

See this impressive list: https://github.com/jupyter/jupyter/wiki/Jupyter-kernels

In principle, they should all be usable for code execution with Quarto. And syntax highlighting is done by pandoc, anyway:

$ pandoc --list-highlight-languages | wc -l
148

A bash kernel is here: https://github.com/takluyver/bash_kernel

[mcanouil]

Same applies for both computing engines.
Here, he does not want any of the engines.

[jamesowers-cohere]

Just adding my +1 to this issue and two cents. I want to make clear right up top that I'm not suggesting that the Quarto project should create an engine for every language. I'm suggesting that the ability to run shell code (particularly bash) with a persistent shell between code blocks is a very common thing to want to do, very valuable, and there are other similar projects that do it well (e.g. Atuin). Simply having some documentation which demonstrates how to do this with existing methods would be really really helpful.

I currently maintain an internal quarto website project with many active contributors, and a couple of hundred pages. It would be really lovely if there was a simple "bash" engine (or well supported route). The following approaches don't work well for us:

• jupyter notebooks with the python kernel but using ipython magic
  • dev overhead of additional syntax
  • and the requirement to understand the idiosyncrasies of shell startup scripts etc
  • doesn't maintain state between cells
• knitr bash cells
  • doesn't maintain state between cells
• jupyter notebooks with the bash kernel
  • has bugs in relation to error handling (detailed below)

The issue with https://github.com/takluyver/bash_kernel (aside from the fact there's been no active contribution since 2023) is the part which auto restarts the kernel.

This means that if you have code in your .qmd like this:

```{bash}
exit 1
echo "This echo doesn't get printed - this is expected"
```

```{bash}
echo "But this echo does get printed - this is unexpected! The recipe should have failed and not continued."
```

Whilst I expect the quarto render process to fail and exit at exit 1 it does not! It proceeds to the next cell... this is very unexpected and breaks my use case!

Here's a full file to aid reproduction of the issue if necessary:

bash_kernel_gotchas.qmd

---
title: Jupyter bash kernel gotchas
author: "James Owers-Bardsley"
jupyter: bash
format: html
---


## These things do not make the recipe fail (but you would expect them to)

### Using `exit 1` in a bash cell doesn't make the recipe fail
The bash kernel tries to restart itself if it finds itself in a bad state. This means that if you run `exit 1` in a bash cell, the kernel will restart and the recipe will continue running.

```{bash}
exit 1
echo "This echo doesn't get printed - this is expected"
```

```{bash}
echo "But this echo does get printed - this is unexpected! The recipe should have failed and not continued."
```

### Using `set -e` in a bash cell
For the same reason as above, if you run `set -e` in a bash cell and then write a command that errors, it will not make the recipe fail. The cell will error, but the bash kernel will restart and the recipe will continue running.
```{bash}
set -e
this_does_not_exist
echo "This echo should not be printed"
```

```{bash}
echo "But this echo does get printed - this is unexpected! The recipe should have failed and not continued."
```


### Command fails but is not the last command in the cell
By default the bash kernel does not have `set -e` enabled, so if a command fails but is not the last command in the cell, the recipe will not fail. This is different from how the python kernel works.

```{bash}
this_does_not_exist
echo "This echo should not be printed"
```

Here's a screenshot of what happens when that qmd is run - as far as quarto is concerned there was no error!

To reiterate, I don't think it's your responsibility to fix or maintain https://github.com/takluyver/bash_kernel. But if you do not provide an example qmd in your docs which demonstrates how I can write documentation and use a persistent bash shell, then it falls to me to:

1. work it out myself, and
2. fix and maintain https://github.com/takluyver/bash_kernel (or create and maintain my own bash_kernel)

If you agree with my claim that the ability to create a qmd file which runs shell code (particularly bash) with a persistent shell between code blocks is a very common thing to want to do, and very valuable, then you have an opportunity to make your project much more obviously valuable and usable to more people by adding a simple example to your docs! I would be enormously grateful for such an addition.

[jamesowers-cohere]

P.S. Very happy to open a new issue here since, although related, my ask is much more of the flavour of "Please show me how to do this in the docs" rather than create a new engine. Though the difficulty arises if there is way to do what I'm asking for with what is currently available! Not relying upon knitr nor jupyer would just be a "nice to have" for me.

[mcanouil]

FYI, Quarto 1.9 brought a new type of extensions, namely engine extensions which could allow you to solve this, see engine extensions documentation.

Note that Quarto does not support multiple engines per document, so having a bash engine means no R/knitr, julia, or Jupyter/Python capabilities (unless your engine supports all of them eventually).