| 1 |
// vim: textwidth=99 |
| 2 |
/* |
| 3 |
Meta note: This file is loaded as a .rs file by rustdoc only. |
| 4 |
*/ |
| 5 |
/*! |
| 6 |
|
| 7 |
A more detailed version of the [warning at the top level](super#warning) about the `quote`/`join` |
| 8 |
family of APIs. |
| 9 |
|
| 10 |
In general, passing the output of these APIs to a shell should recover the original string(s). |
| 11 |
This page lists cases where it fails to do so. |
| 12 |
|
| 13 |
In noninteractive contexts, there are only minor issues. 'Noninteractive' includes shell scripts |
| 14 |
and `sh -c` arguments, or even scripts `source`d from interactive shells. The issues are: |
| 15 |
|
| 16 |
- [Nul bytes](#nul-bytes) |
| 17 |
|
| 18 |
- [Overlong commands](#overlong-commands) |
| 19 |
|
| 20 |
If you are writing directly to the stdin of an interactive (`-i`) shell (i.e., if you are |
| 21 |
pretending to be a terminal), or if you are writing to a cooked-mode pty (even if the other end is |
| 22 |
noninteractive), then there is a **severe** security issue: |
| 23 |
|
| 24 |
- [Control characters](#control-characters-interactive-contexts-only) |
| 25 |
|
| 26 |
Finally, there are some [solved issues](#solved-issues). |
| 27 |
|
| 28 |
# List of issues |
| 29 |
|
| 30 |
## Nul bytes |
| 31 |
|
| 32 |
For non-interactive shells, the most problematic input is nul bytes (bytes with value 0). The |
| 33 |
non-deprecated functions all default to returning [`QuoteError::Nul`] when encountering them, but |
| 34 |
the deprecated [`quote`] and [`join`] functions leave them as-is. |
| 35 |
|
| 36 |
In Unix, nul bytes can't appear in command arguments, environment variables, or filenames. It's |
| 37 |
not a question of proper quoting; they just can't be used at all. This is a consequence of Unix's |
| 38 |
system calls all being designed around nul-terminated C strings. |
| 39 |
|
| 40 |
Shells inherit that limitation. Most of them do not accept nul bytes in strings even internally. |
| 41 |
Even when they do, it's pretty much useless or even dangerous, since you can't pass them to |
| 42 |
external commands. |
| 43 |
|
| 44 |
In some cases, you might fail to pass the nul byte to the shell in the first place. For example, |
| 45 |
the following code uses [`join`] to tunnel a command over an SSH connection: |
| 46 |
|
| 47 |
```rust |
| 48 |
std::process::Command::new("ssh") |
| 49 |
.arg("myhost") |
| 50 |
.arg("--") |
| 51 |
.arg(join(my_cmd_args)) |
| 52 |
``` |
| 53 |
|
| 54 |
If any argument in `my_cmd_args` contains a nul byte, then `join(my_cmd_args)` will contain a nul |
| 55 |
byte. But `join(my_cmd_args)` is itself being passed as an argument to a command (the ssh |
| 56 |
command), and command arguments can't contain nul bytes! So this will simply result in the |
| 57 |
`Command` failing to launch. |
| 58 |
|
| 59 |
Still, there are other ways to smuggle nul bytes into a shell. How the shell reacts depends on the |
| 60 |
shell and the method of smuggling. For example, here is Bash 5.2.21 exhibiting three different |
| 61 |
behaviors: |
| 62 |
|
| 63 |
- With ANSI-C quoting, the string is truncated at the first nul byte: |
| 64 |
```bash |
| 65 |
$ echo $'foo\0bar' | hexdump -C |
| 66 |
00000000 66 6f 6f 0a |foo.| |
| 67 |
``` |
| 68 |
|
| 69 |
- With command substitution, nul bytes are removed with a warning: |
| 70 |
```bash |
| 71 |
$ echo $(printf 'foo\0bar') | hexdump -C |
| 72 |
bash: warning: command substitution: ignored null byte in input |
| 73 |
00000000 66 6f 6f 62 61 72 0a |foobar.| |
| 74 |
``` |
| 75 |
|
| 76 |
- When a nul byte appears directly in a shell script, it's removed with no warning: |
| 77 |
```bash |
| 78 |
$ printf 'echo "foo\0bar"' | bash | hexdump -C |
| 79 |
00000000 66 6f 6f 62 61 72 0a |foobar.| |
| 80 |
``` |
| 81 |
|
| 82 |
Zsh, in contrast, actually allows nul bytes internally, in shell variables and even arguments to |
| 83 |
builtin commands. But if a variable is exported to the environment, or if an argument is used for |
| 84 |
an external command, then the child process will see it silently truncated at the first nul. This |
| 85 |
might actually be more dangerous, depending on the use case. |
| 86 |
|
| 87 |
## Overlong commands |
| 88 |
|
| 89 |
If you pass a long string into a shell, several things might happen: |
| 90 |
|
| 91 |
- It might succeed, yet the shell might have trouble actually doing anything with it. For example: |
| 92 |
|
| 93 |
```bash |
| 94 |
x=$(printf '%010000000d' 0); /bin/echo $x |
| 95 |
bash: /bin/echo: Argument list too long |
| 96 |
``` |
| 97 |
|
| 98 |
- If you're using certain shells (e.g. Busybox Ash) *and* using a pty for communication, then the |
| 99 |
shell will impose a line length limit, ignoring all input past the limit. |
| 100 |
|
| 101 |
- If you're using a pty in cooked mode, then by default, if you write so many bytes as input that |
| 102 |
it fills the kernel's internal buffer, the kernel will simply drop those bytes, instead of |
| 103 |
blocking waiting for the shell to empty out the buffer. In other words, random bits of input can |
| 104 |
be lost, which is obviously insecure. |
| 105 |
|
| 106 |
Future versions of this crate may add an option to [`Quoter`] to check the length for you. |
| 107 |
|
| 108 |
## Control characters (*interactive contexts only*) |
| 109 |
|
| 110 |
Control characters are the bytes from `\x00` to `\x1f`, plus `\x7f`. `\x00` (the nul byte) is |
| 111 |
discussed [above](#nul-bytes), but what about the rest? Well, many of them correspond to terminal |
| 112 |
keyboard shortcuts. For example, when you press Ctrl-A at a shell prompt, your terminal sends the |
| 113 |
byte `\x01`. The shell sees that byte and (if not configured differently) takes the standard |
| 114 |
action for Ctrl-A, which is to move the cursor to the beginning of the line. |
| 115 |
|
| 116 |
This means that it's quite dangerous to pipe bytes to an interactive shell. For example, here is a |
| 117 |
program that tries to tell Bash to echo an arbitrary string, 'safely': |
| 118 |
```rust |
| 119 |
use std::process::{Command, Stdio}; |
| 120 |
use std::io::Write; |
| 121 |
|
| 122 |
let evil_string = "\x01do_something_evil; "; |
| 123 |
let quoted = shlex::try_quote(evil_string).unwrap(); |
| 124 |
println!("quoted string is {:?}", quoted); |
| 125 |
|
| 126 |
let mut bash = Command::new("bash") |
| 127 |
.arg("-i") // force interactive mode |
| 128 |
.stdin(Stdio::piped()) |
| 129 |
.spawn() |
| 130 |
.unwrap(); |
| 131 |
let stdin = bash.stdin.as_mut().unwrap(); |
| 132 |
write!(stdin, "echo {}\n", quoted).unwrap(); |
| 133 |
``` |
| 134 |
|
| 135 |
Here's the output of the program (with irrelevant bits removed): |
| 136 |
|
| 137 |
```text |
| 138 |
quoted string is "'\u{1}do_something_evil; '" |
| 139 |
/tmp comex$ do_something_evil; 'echo ' |
| 140 |
bash: do_something_evil: command not found |
| 141 |
bash: echo : command not found |
| 142 |
``` |
| 143 |
|
| 144 |
Even though we quoted it, Bash still ran an arbitrary command! |
| 145 |
|
| 146 |
This is not because the quoting was insufficient, per se. In single quotes, all input is supposed |
| 147 |
to be treated as raw data until the closing single quote. And in fact, this would work fine |
| 148 |
without the `"-i"` argument. |
| 149 |
|
| 150 |
But line input is a separate stage from shell syntax parsing. After all, if you type a single |
| 151 |
quote on the keyboard, you wouldn't expect it to disable all your keyboard shortcuts. So a control |
| 152 |
character always has its designated effect, no matter if it's quoted or backslash-escaped. |
| 153 |
|
| 154 |
Also, some control characters are interpreted by the kernel tty layer instead, like CTRL-C to send |
| 155 |
SIGINT. These can be an issue even with noninteractive shells, but only if using a pty for |
| 156 |
communication, as opposed to a pipe. |
| 157 |
|
| 158 |
To be safe, you just have to avoid sending them. |
| 159 |
|
| 160 |
### Why not just use hex escapes? |
| 161 |
|
| 162 |
In any normal programming languages, this would be no big deal. |
| 163 |
|
| 164 |
Any normal language has a way to escape arbitrary characters in strings by writing out their |
| 165 |
numeric values. For example, Rust lets you write them in hexadecimal, like `"\x4f"` (or |
| 166 |
`"\u{1d546}"` for Unicode). In this way, arbitrary strings can be represented using only 'nice' |
| 167 |
simple characters. Any remotely suspicious character can be replaced with a numeric escape |
| 168 |
sequence, where the escape sequence itself consists only of alphanumeric characters and some |
| 169 |
punctuation. The result may not be the most readable[^choices], but it's quite safe from being |
| 170 |
misinterpreted or corrupted in transit. |
| 171 |
|
| 172 |
Shell is not normal. It has no numeric escape sequences. |
| 173 |
|
| 174 |
There are a few different ways to quote characters (unquoted, unquoted-with-backslash, single |
| 175 |
quotes, double quotes), but all of them involve writing the character itself. If the input |
| 176 |
contains a control character, the output must contain that same character. |
| 177 |
|
| 178 |
### Mitigation: terminal filters |
| 179 |
|
| 180 |
In practice, automating interactive shells like in the above example is pretty uncommon these days. |
| 181 |
In most cases, the only way for a programmatically generated string to make its way to the input of |
| 182 |
an interactive shell is if a human copies and pastes it into their terminal. |
| 183 |
|
| 184 |
And many terminals detect when you paste a string containing control characters. iTerm2 strips |
| 185 |
them out; gnome-terminal replaces them with alternate characters[^gr]; Kitty outright prompts for |
| 186 |
confirmation. This mitigates the risk. |
| 187 |
|
| 188 |
But it's not perfect. Some other terminals don't implement this check or implement it incorrectly. |
| 189 |
Also, these checks tend to not filter the tab character, which could trigger tab completion. In |
| 190 |
most cases that's a non-issue, because most shells support paste bracketing, which disables tab and |
| 191 |
some other control characters[^bracketing] within pasted text. But in some cases paste bracketing |
| 192 |
gets disabled. |
| 193 |
|
| 194 |
### Future possibility: ANSI-C quoting |
| 195 |
|
| 196 |
I said that shell syntax has no numeric escapes, but that only applies to *portable* shell syntax. |
| 197 |
Bash and Zsh support an obscure alternate quoting style with the syntax `$'foo'`. It's called |
| 198 |
["ANSI-C quoting"][ansic], and inside it you can use all the escape sequences supported by C, |
| 199 |
including hex escapes: |
| 200 |
|
| 201 |
```bash |
| 202 |
$ echo $'\x41\n\x42' |
| 203 |
A |
| 204 |
B |
| 205 |
``` |
| 206 |
|
| 207 |
But other shells don't support it — including Dash, a popular choice for `/bin/sh`, and Busybox's |
| 208 |
Ash, frequently seen on stripped-down embedded systems. This crate's quoting functionality [tries |
| 209 |
to be compatible](crate#compatibility) with those shells, plus all other POSIX-compatible shells. |
| 210 |
That makes ANSI-C quoting a no-go. |
| 211 |
|
| 212 |
Still, future versions of this crate may provide an option to enable ANSI-C quoting, at the cost of |
| 213 |
reduced portability. |
| 214 |
|
| 215 |
### Future possibility: printf |
| 216 |
|
| 217 |
Another option would be to invoke the `printf` command, which is required by POSIX to support octal |
| 218 |
escapes. For example, you could 'escape' the Rust string `"\x01"` into the shell syntax `"$(printf |
| 219 |
'\001')"`. The shell will execute the command `printf` with the first argument being literally a |
| 220 |
backslash followed by three digits; `printf` will output the actual byte with value 1; and the |
| 221 |
shell will substitute that back into the original command. |
| 222 |
|
| 223 |
The problem is that 'escaping' a string into a command substitution just feels too surprising. If |
| 224 |
nothing else, it only works with an actual shell; [other languages' shell parsing |
| 225 |
routines](crate#compatibility) wouldn't understand it. Neither would this crate's own parser, |
| 226 |
though that could be fixed. |
| 227 |
|
| 228 |
Future versions of this crate may provide an option to use `printf` for quoting. |
| 229 |
|
| 230 |
### Special note: newlines |
| 231 |
|
| 232 |
Did you know that `\r` and `\n` are control characters? They aren't as dangerous as other control |
| 233 |
characters (if quoted properly). But there's still an issue with them in interactive contexts. |
| 234 |
|
| 235 |
Namely, in some cases, interactive shells and/or the tty layer will 'helpfully' translate between |
| 236 |
different line ending conventions. The possibilities include replacing `\r` with `\n`, replacing |
| 237 |
`\n` with `\r\n`, and others. This can't result in command injection, but it's still a lossy |
| 238 |
transformation which can result in a failure to round-trip (i.e. the shell sees a different string |
| 239 |
from what was originally passed to `quote`). |
| 240 |
|
| 241 |
Numeric escapes would solve this as well. |
| 242 |
|
| 243 |
# Solved issues |
| 244 |
|
| 245 |
## Solved: Past vulnerability (GHSA-r7qv-8r2h-pg27 / RUSTSEC-2024-XXX) |
| 246 |
|
| 247 |
Versions of this crate before 1.3.0 did not quote `{`, `}`, and `\xa0`. |
| 248 |
|
| 249 |
See: |
| 250 |
- <https://github.com/advisories/GHSA-r7qv-8r2h-pg27> |
| 251 |
- (TODO: Add Rustsec link) |
| 252 |
|
| 253 |
## Solved: `!` and `^` |
| 254 |
|
| 255 |
There are two non-control characters which have a special meaning in interactive contexts only: `!` and |
| 256 |
`^`. Luckily, these can be escaped adequately. |
| 257 |
|
| 258 |
The `!` character triggers [history expansion][he]; the `^` character can trigger a variant of |
| 259 |
history expansion known as [Quick Substitution][qs]. Both of these characters get expanded even |
| 260 |
inside of double-quoted strings\! |
| 261 |
|
| 262 |
If we're in a double-quoted string, then we can't just escape these characters with a backslash. |
| 263 |
Only a specific set of characters can be backslash-escaped inside double quotes; the set of |
| 264 |
supported characters depends on the shell, but it often doesn't include `!` and `^`.[^escbs] |
| 265 |
Trying to backslash-escape an unsupported character produces a literal backslash: |
| 266 |
```bash |
| 267 |
$ echo "\!" |
| 268 |
\! |
| 269 |
``` |
| 270 |
|
| 271 |
However, these characters don't get expanded in single-quoted strings, so this crate just |
| 272 |
single-quotes them. |
| 273 |
|
| 274 |
But there's a Bash bug where `^` actually does get partially expanded in single-quoted strings: |
| 275 |
```bash |
| 276 |
$ echo ' |
| 277 |
> ^a^b |
| 278 |
> ' |
| 279 |
|
| 280 |
!!:s^a^b |
| 281 |
``` |
| 282 |
|
| 283 |
To work around that, this crate forces `^` to appear right after an opening single quote. For |
| 284 |
example, the string `"^` is quoted into `'"''^'` instead of `'"^'`. This restriction is overkill, |
| 285 |
since `^` is only meaningful right after a newline, but it's a sufficient restriction (after all, a |
| 286 |
`^` character can't be preceded by a newline if it's forced to be preceded by a single quote), and |
| 287 |
for now it simplifies things. |
| 288 |
|
| 289 |
## Solved: `\xa0` |
| 290 |
|
| 291 |
The byte `\xa0` may be treated as a shell word separator, specifically on Bash on macOS when using |
| 292 |
the default UTF-8 locale, only when the input is invalid UTF-8. This crate handles the issue by |
| 293 |
always using quotes for arguments containing this byte. |
| 294 |
|
| 295 |
In fact, this crate always uses quotes for arguments containing any non-ASCII bytes. This may be |
| 296 |
changed in the future, since it's a bit unfriendly to non-English users. But for now it |
| 297 |
minimizes risk, especially considering the large number of different legacy single-byte locales |
| 298 |
someone might hypothetically be running their shell in. |
| 299 |
|
| 300 |
### Demonstration |
| 301 |
|
| 302 |
```bash |
| 303 |
$ echo -e 'ls a\xa0b' | bash |
| 304 |
ls: a: No such file or directory |
| 305 |
ls: b: No such file or directory |
| 306 |
``` |
| 307 |
The normal behavior would be to output a single line, e.g.: |
| 308 |
```bash |
| 309 |
$ echo -e 'ls a\xa0b' | bash |
| 310 |
ls: cannot access 'a'$'\240''b': No such file or directory |
| 311 |
``` |
| 312 |
(The specific quoting in the error doesn't matter.) |
| 313 |
|
| 314 |
### Cause |
| 315 |
|
| 316 |
Just for fun, here's why this behavior occurs: |
| 317 |
|
| 318 |
Bash decides which bytes serve as word separators based on the libc function [`isblank`][isblank]. |
| 319 |
On macOS on UTF-8 locales, this passes for `\xa0`, corresponding to U+00A0 NO-BREAK SPACE. |
| 320 |
|
| 321 |
This is doubly unique compared to the other systems I tested (Linux/glibc, Linux/musl, and |
| 322 |
Windows/MSVC). First, the other systems don't allow bytes in the range [0x80, 0xFF] to pass |
| 323 |
<code>is<i>foo</i></code> functions in UTF-8 locales, even if the corresponding Unicode codepoint |
| 324 |
does pass, as determined by the wide-character equivalent function, <code>isw<i>foo</i></code>. |
| 325 |
Second, the other systems don't treat U+00A0 as blank (even using `iswblank`). |
| 326 |
|
| 327 |
Meanwhile, Bash checks for multi-byte sequences and forbids them from being treated as special |
| 328 |
characters, so the proper UTF-8 encoding of U+00A0, `b"\xc2\xa0"`, is not treated as a word |
| 329 |
separator. Treatment as a word separator only happens for `b"\xa0"` alone, which is illegal UTF-8. |
| 330 |
|
| 331 |
[ansic]: https://www.gnu.org/software/bash/manual/html_node/ANSI_002dC-Quoting.html |
| 332 |
[he]: https://www.gnu.org/software/bash/manual/html_node/History-Interaction.html |
| 333 |
[qs]: https://www.gnu.org/software/bash/manual/html_node/Event-Designators.html |
| 334 |
[isblank]: https://man7.org/linux/man-pages/man3/isblank.3p.html |
| 335 |
[nul]: #nul-bytes |
| 336 |
|
| 337 |
[^choices]: This can lead to tough choices over which |
| 338 |
characters to escape and which to leave as-is, especially when Unicode gets involved and you |
| 339 |
have to balance the risk of confusion with the benefit of properly supporting non-English |
| 340 |
languages. |
| 341 |
<br> |
| 342 |
<br> |
| 343 |
We don't have the luxury of those choices. |
| 344 |
|
| 345 |
[^gr]: For example, backspace (in Unicode lingo, U+0008 BACKSPACE) turns into U+2408 SYMBOL FOR BACKSPACE. |
| 346 |
|
| 347 |
[^bracketing]: It typically disables almost all handling of control characters by the shell proper, |
| 348 |
but one necessary exception is the end-of-paste sequence itself (which starts with the control |
| 349 |
character `\x1b`). In addition, paste bracketing does not suppress handling of control |
| 350 |
characters by the kernel tty layer, such as `\x03` sending SIGINT (which typically clears the |
| 351 |
currently typed command, making it dangerous in a similar way to `\x01`). |
| 352 |
|
| 353 |
[^escbs]: For example, Dash doesn't remove the backslash from `"\!"` because it simply doesn't know |
| 354 |
anything about `!` as a special character: it doesn't support history expansion. On the other |
| 355 |
end of the spectrum, Zsh supports history expansion and does remove the backslash — though only |
| 356 |
in interactive mode. Bash's behavior is weirder. It supports history expansion, and if you |
| 357 |
write `"\!"`, the backslash does prevent history expansion from occurring — but it doesn't get |
| 358 |
removed! |
| 359 |
|
| 360 |
*/ |
| 361 |
|
| 362 |
// `use` declarations to make auto links work: |
| 363 |
use ::{quote, join, Shlex, Quoter, QuoteError}; |
| 364 |
|
| 365 |
// TODO: add more about copy-paste and human readability. |