Here-docs and generating files

A here-document redirects a block of literal text into a command's standard input. Everything between the << marker and a line holding only the terminator is the input:

name=deb1
retain=7

cat <<EOF
host      = $name
retention = $retain days
doubled   = $(( retain * 2 ))
EOF

host      = deb1
retention = 7 days
doubled   = 14

Variables, command substitutions and arithmetic all expand, exactly as they would inside double quotes. EOF is a convention rather than a keyword: any word does, and something specific to the job (CONFIG, SQL) reads better in a script that has several.

Quoting the terminator turns expansion off

Write the opening word in quotes and the block becomes literal, which is what you want whenever the text contains $ that belongs to something else:

cat <<"EOF"
host      = $name
doubled   = $(( retain * 2 ))
EOF

host      = $name
doubled   = $(( retain * 2 ))

Quoting either the whole word or any part of it (<<'EOF', <<"EOF", <<\EOF) does the same thing. This is the form to use for a block of shell, awk or SQL you are writing out rather than running, since otherwise every $1 in it is expanded by the wrong shell before it ever reaches the file.

Generating a file

The redirection goes on the command, so > file and a here-doc combine into the standard way a setup script writes configuration:

host=deb1
port=8080

cat > app.conf <<EOF
# generated by setup.sh, do not edit
listen = $host:$port
workers = $(( 2 * 2 ))
EOF

cat app.conf

# generated by setup.sh, do not edit
listen = deb1:8080
workers = 4

>> appends instead. For a file you need root to write, the shape that works is sudo tee file <<EOF, because sudo cat > /etc/file opens the file as you: the redirection is performed by your shell before sudo runs anything. That is the same rule as anywhere else redirection meets sudo, and Pipes and redirection has it in full.

Any command that reads standard input can be fed this way, not just cat:

sort <<EOF
deb3
deb1
deb2
EOF

deb1
deb2
deb3

`<<-` strips indentation, but only tabs

A here-doc inside a function or a loop is unindentable, because the terminator has to sit at the start of its own line. <<- fixes that by stripping leading tabs from every line of the block, the terminator included:

deploy() {
	cat <<-EOF
		host = deb1
		port = 8080
	EOF
}

deploy

host = deb1
port = 8080

`<<<` passes one string

A here-string is the same idea for a single line, and it saves a process against echo x | cmd:

tr a-z A-Z <<< "deb1"
grep -c host <<< "host = deb1"

read -r first _ <<< "deb1 web server"
echo "first field: $first"

DEB1
1
first field: deb1

The read case is the one worth remembering. echo "$line" | read -r first _ puts read in a subshell, so $first is empty by the time you use it, which is the same trap the Loops lesson covers for while read. A here-string keeps it in the current shell.

Exercises

Write a motd file containing the literal text Welcome, $USER rather than the expanded name.
Answer
```
cat > motd <<'EOF'
Welcome, $USER
EOF
```
Quoting the terminator is the whole answer. Escaping every $ inside an unquoted here-doc also works and is a maintenance problem the first time somebody adds a line.
Why does this print nothing?
```
cat <<EOF
hello
  EOF
```
Answer

The terminator is indented, so it is content rather than a terminator, and the shell reads on to the end of the file looking for one. It warns about that and abandons the command. Either move EOF to column one, or use <<-EOF and indent everything with tabs.
A script generates an awk program into a file and every $1 in it comes out empty. What happened?

Answer

The here-doc terminator was unquoted, so the shell expanded $1 as its own positional parameter before awk ever saw the file. Quote the terminator, and the block is written out byte for byte.

What's next

Two lessons remain before the capstone: cleaning up with trap and mktemp when a script exits however it exits, and the flags in set -euo pipefail, which Exit codes and error handling sets up.