awk

Split lines into fields and compute over them

Updated 2026-08-12

awk reads its input one record at a time (a line, by default), splits each record into fields, and runs a small program against them. Where grep answers "which lines match?" and sed answers "how do I transform matching lines?", awk answers "how do I pull fields out of a line and compute something with them?" You want it once a task stops being about lines and starts being about columns: totals, averages, tallies, reordering, reformatting. It is not the tool for JSON, which has no columns and no fixed idea of where its lines go: jq is the equivalent there.

Every program is a list of pattern { action } pairs

An awk program is a series of pattern { action } pairs. For every input record, awk tests each pattern in order; if a pattern matches (or there is no pattern, meaning "always"), it runs the associated action. Leave off the action and the default is { print }, so awk '/error/' file is a complete, working program: print every line matching /error/. This is why so many awk one-liners look shorter than the equivalent grep/sed combination once you know the shorthand.

awk '/error/' app.log          # pattern only: same job as grep
awk '{ print $1 }' app.log     # action only: same job as cut
awk '/error/ { print $1 }' app.log   # both: grep + cut in one pass

How awk splits a record into fields

Each input line is a record, split on whitespace by default into fields: $1 is the first field, $2 the second, and so on, $0 is the whole record, and $NF is always the last field regardless of how many fields a line has. NF (number of fields) and NR (number of records seen so far, i.e. the line number) are built-in variables updated automatically for every record. Because field splitting is automatic, tasks that need cut -f2 | ... in one tool often need nothing more than { print $2 } in awk.

The default field separator (FS) is "any run of whitespace," which quietly handles both single spaces and columns padded with extra spaces. Set a different one with -F (or FS= in a BEGIN block) for structured data: -F: for /etc/passwd-style files, -F, for CSV, or a regex like -F'[0-9]+' when the separator itself varies. OFS (output field separator, a single space by default) controls how fields are rejoined when you print them individually or rebuild $0 by reassigning a field.

BEGIN and END: code that runs once

Most of an awk program runs once per record, but BEGIN { ... } runs once before any input is read (good for setting FS/OFS or printing a header) and END { ... } runs once after the last record (good for printing totals). A running total accumulated in a plain variable across every record, then printed in END, is awk's signature move:

awk -F, 'NR > 1 { sum += $3 } END { print sum }' employees.csv

END runs even if BEGIN set things up but no records ever arrived, and any variable you never explicitly initialise starts as both 0 and "", so sum += $3 works from the very first line without a sum = 0 line first.

Variables, arrays, and control flow

awk variables need no declaration and no type: assign a string or a number, and later use the same variable as the other, and awk converts as needed. Arrays are associative (keyed by string, not just integer index), which makes them the natural structure for counting and grouping: count[$1]++ builds a frequency table of field 1 across every record in a single pass, no external sort | uniq -c needed. awk also has the if/else, for, and while you'd expect from a general-purpose language, plus a ternary ?:, so logic that would need a sed hold-space trick or a shell loop around grep often fits in one awk line instead.

print is the quick option: it writes its arguments joined by OFS and terminated by ORS (a newline, by default), with sensible defaults for both. printf gives up those defaults in exchange for control: field widths, decimal places, zero-padding, no trailing newline unless you write \n yourself. Use printf the moment output needs to line up in columns or a number needs a fixed number of decimal places.

Debian's awk is mawk, not gawk

Debian's awk is a symlink managed by update-alternatives, and on a fresh install it usually points at mawk, a smaller and faster implementation that covers the POSIX language well (everything on this page runs under it). apt install gawk pulls in the GNU implementation, which adds extensions such as gensub(), asort(), and in-place editing (-i inplace). If a script you find online uses one of those and errors out with "calling undefined function," that's almost certainly a gawk-only extension running under mawk.

Combining awk with grep and sed

The three tools compose naturally in a pipeline: grep narrows the lines, sed reshapes text within a line, and awk extracts and computes over fields, often with sort/uniq -c closing the loop. See Pipes and redirection for why chaining small, single-purpose tools like this works as well as it does.

Sample files used on this page

Every example below was run against these files. Recreate them to follow along.

employees.csv a header row plus 6 records: name,department,salary

name,department,salary
alice,engineering,95000
bob,design,72000
carol,engineering,110000
dave,sales,68000
erin,design,81000
frank,engineering,88000

app.log 8 syslog-style lines, 3 of them errors

Jul  5 09:14:20 deb1 app[312]: info: service starting
Jul  5 09:14:21 deb1 app[312]: info: listening on port 8080
Jul  5 09:14:22 deb1 app[312]: error: connection refused
Jul  5 09:14:23 deb1 app[312]: warning: retrying in 5s
Jul  5 09:14:28 deb1 app[312]: error: connection refused
Jul  5 09:14:33 deb1 app[312]: info: connected
Jul  5 09:15:01 deb1 app[313]: error: connection refused
Jul  5 09:15:05 deb1 app[313]: info: connected

access.log 11 fields per line - combined log format plus a trailing response time

198.51.100.7 - - [05/Jul/2026:09:12:01 +0000] "GET /index.html HTTP/1.1" 200 1024 0.021
203.0.113.5 - - [05/Jul/2026:09:12:03 +0000] "GET /style.css HTTP/1.1" 200 512 0.008
203.0.113.5 - - [05/Jul/2026:09:12:07 +0000] "GET /app.js HTTP/1.1" 200 2048 0.014
198.51.100.7 - - [05/Jul/2026:09:12:15 +0000] "POST /login HTTP/1.1" 401 128 0.045
203.0.113.9 - - [05/Jul/2026:09:12:22 +0000] "GET /index.html HTTP/1.1" 200 1024 0.019
198.51.100.7 - - [05/Jul/2026:09:12:40 +0000] "POST /login HTTP/1.1" 401 128 0.052
203.0.113.9 - - [05/Jul/2026:09:13:02 +0000] "GET /missing HTTP/1.1" 404 0 0.003
198.51.100.7 - - [05/Jul/2026:09:13:10 +0000] "GET /dashboard HTTP/1.1" 200 4096 0.081

config.conf a comment line plus four key=value settings

# Server configuration
host=localhost
port=8080
debug=false
timeout=30

passwd.txt an excerpt in /etc/passwd format, for the -F: examples

root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
user:x:1000:1000:user:/home/user:/bin/bash
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin

allowed.txt the names treated as already approved, for the two-file example

alice
bob
carol
79 outputs, collapsed by default

Fields and records

Every line is a record, split automatically into numbered fields.

Print the whole record

awk '{ print }' config.conf

With no pattern, the action runs on every record. print with no arguments is shorthand for print $0, the whole line.

Show output 
# Server configuration
host=localhost
port=8080
debug=false
timeout=30

Print a single field

awk '{ print $1 }' access.log

$1 is the first whitespace-separated field. Here it's the client IP at the start of each access log line.

Show output 
198.51.100.7
203.0.113.5
203.0.113.5
198.51.100.7
203.0.113.9
198.51.100.7
203.0.113.9
198.51.100.7

Print two fields together

awk '{ print $1, $9 }' access.log

A comma between arguments to print joins them with a single space (the default OFS). Field 9 here is the HTTP status code, since the quoted request string splits into three of its own fields.

Show output 
198.51.100.7 200
203.0.113.5 200
203.0.113.5 200
198.51.100.7 401
203.0.113.9 200
198.51.100.7 401
203.0.113.9 404
198.51.100.7 200

Print the last field with $NF

awk '{ print $NF }' access.log

NF is always the number of fields on the current line, so $NF is the last field regardless of how many fields a line has. Here it's the request duration in seconds.

Show output 
0.021
0.008
0.014
0.045
0.019
0.052
0.003
0.081

Count the fields on each line

awk '{ print NF }' access.log

Every access log line here has 11 whitespace-separated fields, because the quoted request ("GET /path HTTP/1.1") counts as three of them.

Show output 
11
11
11
11
11
11
11
11

Watch the field count change with the data

awk '{ print NF, $0 }' config.conf

config.conf has no spaces in most lines, so NF is 1 there, but the comment line has spaces and splits into 3 fields. Field splitting depends entirely on what's in the line, not what you expect.

Show output 
3 # Server configuration
1 host=localhost
1 port=8080
1 debug=false
1 timeout=30

Line numbers and counting records

NR tracks how many records have been read so far.

Number every line

awk '{ print NR, $0 }' config.conf

NR (number of records) increments before every action runs, giving a cat -n-style line number for free.

Show output 
1 # Server configuration
2 host=localhost
3 port=8080
4 debug=false
5 timeout=30

Count records total

awk 'END { print NR }' app.log

NR still holds its final value inside END, so printing it there is a one-line wc -l.

Show output 
8

Print a specific line

awk 'NR==2' app.log

A bare expression is a pattern: true for line 2 only, so with no action the default { print } fires just once.

Show output 
Jul  5 09:14:21 deb1 app[312]: info: listening on port 8080

Print a range of lines

awk 'NR==2,NR==4' app.log

A comma between two patterns makes a range: every record from the first match through the second, inclusive.

Show output 
Jul  5 09:14:21 deb1 app[312]: info: listening on port 8080
Jul  5 09:14:22 deb1 app[312]: error: connection refused
Jul  5 09:14:23 deb1 app[312]: warning: retrying in 5s

Print the last line

awk 'END { print }' app.log

$0 still holds the most recent record once the main loop finishes, so print inside END prints the last line without knowing the file's length in advance.

Show output 
Jul  5 09:15:05 deb1 app[313]: info: connected

Selecting records with patterns

A pattern with no action is a complete program: it prints whatever matches.

Filter lines like grep, with a bare pattern

awk '/error/' app.log

No action means the default { print }, so this is a full working program: print every record matching /error/, the same job grep error app.log does.

Show output 
Jul  5 09:14:22 deb1 app[312]: error: connection refused
Jul  5 09:14:28 deb1 app[312]: error: connection refused
Jul  5 09:15:01 deb1 app[313]: error: connection refused

Invert a match

awk '!/error/' app.log

! negates the pattern, printing every record that does NOT match, the awk equivalent of grep -v.

Show output 
Jul  5 09:14:20 deb1 app[312]: info: service starting
Jul  5 09:14:21 deb1 app[312]: info: listening on port 8080
Jul  5 09:14:23 deb1 app[312]: warning: retrying in 5s
Jul  5 09:14:33 deb1 app[312]: info: connected
Jul  5 09:15:05 deb1 app[313]: info: connected

Combine a pattern with an action

awk '/error|warning/ { print NR, $0 }' app.log

Once a pattern matches, the action can do more than the default print, here adding the line number in front.

Show output 
3 Jul  5 09:14:22 deb1 app[312]: error: connection refused
4 Jul  5 09:14:23 deb1 app[312]: warning: retrying in 5s
5 Jul  5 09:14:28 deb1 app[312]: error: connection refused
7 Jul  5 09:15:01 deb1 app[313]: error: connection refused

Match a regex against a specific field

awk -F, '$1 ~ /^[ab]/' employees.csv

~ tests a field against a regex instead of the whole line. !~ is the negated form. This matches names starting with a or b.

Show output 
alice,engineering,95000
bob,design,72000

Filter on a numeric field

awk -F, 'NR>1 && $3 > 90000' employees.csv

NR>1 skips the header row before the numeric comparison runs. Without it, the header sneaks through, see the next example for why.

Show output 
alice,engineering,95000
carol,engineering,110000

Watch a header row sneak past a numeric filter

awk -F, '$3 > 90000' employees.csv

Without the NR>1 guard, $3 on the header line is the string "salary", which isn't numeric-looking, so awk falls back to a string comparison. "salary" sorts after "90000" lexically, and the header line matches.

Show output 
name,department,salary
alice,engineering,95000
carol,engineering,110000

Combine two conditions with &&

awk -F, '$2=="engineering" && $3 > 90000 { print $1 }' employees.csv

&& requires that both conditions hold. String comparison with == and numeric comparison with > combine freely.

Show output 
alice
carol

Combine two conditions with ||

awk -F, '$2=="design" || $2=="sales" { print $1 }' employees.csv

Either condition is enough to match, the awk equivalent of grep -E "design|sales" scoped to one field.

Show output 
bob
dave
erin

Match an exact field value

awk -F, '$1=="alice"' employees.csv

== on a field requires an exact match, unlike ~ which tests a regex. Faster and clearer when you are not matching a pattern.

Show output 
alice,engineering,95000

Changing the field separator

FS controls how a line splits into fields; -F sets it from the command line.

Split fields on commas

awk -F, 'NR>1 { print $1, $3 }' employees.csv

-F, sets the field separator to a comma instead of whitespace, so $1/$2/$3 line up with the CSV's own columns.

Show output 
alice 95000
bob 72000
carol 110000
dave 68000
erin 81000
frank 88000

Split fields on colons, /etc/passwd-style

awk -F: '{ print $1, $NF }' passwd.txt

Colon-separated files are common on Debian: /etc/passwd, /etc/group, and similar. $1 is the username, $NF the login shell.

Show output 
root /bin/bash
daemon /usr/sbin/nologin
user /bin/bash
www-data /usr/sbin/nologin

Set FS in a BEGIN block instead of -F

awk 'BEGIN{FS=","} NR>1{print $1}' employees.csv

-F and setting FS in BEGIN do the same thing. Prefer BEGIN when the separator is part of a longer script rather than a quick one-liner.

Show output 
alice
bob
carol
dave
erin
frank

Split on a regex separator

printf "a1b2c33d\n" | awk -F'[0-9]+' '{ print $1, $2, $3, $4 }'

When FS is more than one character, awk treats it as a regex. Here any run of digits, of any length, is a separator.

Show output 
a b c d

Control the output separator with OFS

awk -F, 'BEGIN{OFS="|"} NR>1 { $1=$1; print }' employees.csv

Reassigning any field (even to itself) makes awk rebuild $0 from the fields, joined with OFS. Without that reassignment, $0 stays untouched and still has the original commas.

Show output 
alice|engineering|95000
bob|design|72000
carol|engineering|110000
dave|sales|68000
erin|design|81000
frank|engineering|88000

BEGIN and END blocks

Code that runs once before the first record, or once after the last.

Run a program with no input file at all

awk 'BEGIN{print "starting report"}'

BEGIN runs before awk tries to read any input, so a BEGIN-only program never touches stdin or a file.

Show output 
starting report

Print a header before the data

awk -F, 'BEGIN{print "name\tsalary"} NR>1{print $1"\t"$3}' employees.csv

The header line comes from BEGIN, which runs once, before record processing starts.

Show output 
name	salary
alice	95000
bob	72000
carol	110000
dave	68000
erin	81000
frank	88000

Accumulate a running total, print it in END

awk -F, 'NR>1{sum+=$3} END{print sum}' employees.csv

sum starts at 0 automatically (an unset variable is both 0 and "" until assigned), so no separate initialisation line is needed.

Show output 
514000

Compute an average in END

awk -F, 'NR>1{sum+=$3; n++} END{printf "%.2f\n", sum/n}' employees.csv

A sum and a count accumulate separately and are combined once at the end. printf controls the decimal places; print sum/n would show far more digits.

Show output 
85666.67

END still runs when nothing matched

awk '/nonexistent/{c++} END{print c+0}' app.log

c is never touched if the pattern never matches, so print c alone would print an empty string, not 0. Adding 0 forces numeric context, printing the 0 you wanted.

Show output 
0

Associative arrays: counting and grouping

awk arrays are keyed by string, which makes them a natural fit for tallying and grouping in one pass.

Count occurrences of a field value

awk -F, 'NR>1{count[$2]++} END{for (d in count) print d, count[d]}' employees.csv | sort

count[$2]++ builds a frequency table keyed by department, all in a single pass, no external sort | uniq -c needed.

Show output 
design 2
engineering 3
sales 1

Sum a field, grouped by another field

awk -F, 'NR>1{sum[$2]+=$3} END{for (d in sum) print d, sum[d]}' employees.csv | sort

Same idea as counting, but accumulating a value instead of a count: total salary per department.

Show output 
design 153000
engineering 293000
sales 68000

Group and sum values from a log file

awk '{bytes[$1]+=$(NF-1)} END{for (ip in bytes) print ip, bytes[ip]}' access.log | sort

Bytes transferred (second-to-last field) totalled per client IP (first field). sort afterwards just orders the output, awk doesn't guarantee array iteration order.

Show output 
198.51.100.7 5376
203.0.113.5 2560
203.0.113.9 1024

Tally values in a different field

awk '{count[$9]++} END{for (c in count) print c, count[c]}' access.log | sort

The same counting pattern applied to HTTP status codes instead of IPs. Field 9 lines up with the status code because the quoted request eats three fields first.

Show output 
200 5
401 2
404 1

Test array membership with in

awk -F, 'NR>1{seen[$2]=1} END{print ("engineering" in seen) ? "found" : "missing"}' employees.csv

key in array tests for a key without creating it as a side effect, unlike referencing array[key] directly, which would.

Show output 
found

Count distinct keys with length()

awk -F, 'NR>1{d[$2]=1} END{print length(d)}' employees.csv

length() on an array returns its element count, here the number of distinct departments.

Show output 
3

Build a word-frequency table

printf "the quick fox the lazy fox the dog\n" | awk '{for(i=1;i<=NF;i++) count[$i]++} END{for (w in count) print count[w], w}' | sort -rn

A for loop over every field on the line, counting each word seen. Piping through sort -rn ranks the results afterwards.

Show output 
3 the
2 fox
1 quick
1 lazy
1 dog

printf and output formatting

printf trades print's convenient defaults for exact control over layout.

Left- and right-align columns

awk -F, 'NR>1{printf "%-8s %-12s %8s\n", $1, $2, $3}' employees.csv

%-8s left-aligns a string in an 8-character field, %8s right-aligns. This is how awk builds aligned tabular output without an external formatter.

Show output 
alice    engineering     95000
bob      design          72000
carol    engineering    110000
dave     sales           68000
erin     design          81000
frank    engineering     88000

Zero-pad a number

awk -F, 'NR>1{printf "%03d %s\n", NR-1, $1}' employees.csv

%03d pads a number to at least 3 digits with leading zeroes, useful for generating sortable sequence numbers.

Show output 
001 alice
002 bob
003 carol
004 dave
005 erin
006 frank

printf doesn't add a newline for you

awk '{printf "%s", $0}' config.conf

Unlike print, printf never appends anything automatically. Every line here runs into the next, with no \n between them.

Show output 
# Server configurationhost=localhostport=8080debug=falsetimeout=30

Add the newline back explicitly

awk '{printf "%s\n", $0}' config.conf

The fix for the previous example: write \n into the format string yourself.

Show output 
# Server configuration
host=localhost
port=8080
debug=false
timeout=30

String functions

Text-manipulation functions available in every action, not just BEGIN/END.

Get a field's length

awk -F, 'NR>1{print $1, length($1)}' employees.csv

length() with no argument returns the length of $0; with an argument, the length of that string.

Show output 
alice 5
bob 3
carol 5
dave 4
erin 4
frank 5

Extract a substring by position

echo "order-4821-confirmed" | awk '{print substr($0, 7, 4)}'

substr(string, start, length) pulls out a fixed-position chunk, here the 4-digit order number starting at character 7.

Show output 
4821

Find a substring's position with index

awk -F, 'NR>1{print $1, index($1,"a")}' employees.csv

index(string, target) returns the 1-based position of the first match, or 0 if it isn't found at all.

Show output 
alice 1
bob 0
carol 2
dave 2
erin 0
frank 3

Change case with toupper and tolower

echo "Hello World" | awk '{print toupper($0)}'
echo "Hello World" | awk '{print tolower($0)}'

Each takes a string and returns a new one, leaving its argument untouched.

Show output 
HELLO WORLD
hello world

Replace the first match with sub()

echo "foo bar foo" | awk '{sub(/foo/,"FOO"); print}'

sub(regex, replacement) edits $0 in place (or a named field, if given a third argument), replacing only the first match. It's awk's equivalent of a single-shot sed substitution.

Show output 
FOO bar foo

Replace every match and count them with gsub()

echo "foo bar foo baz foo" | awk '{n=gsub(/foo/,"FOO"); print n, $0}'

gsub() is sub()'s global counterpart, matching sed's g flag, and it returns the number of replacements made.

Show output 
3 FOO bar FOO baz FOO

Split a string into an array

awk 'BEGIN{n=split("a:b:c",parts,":"); print n, parts[1], parts[3]}'

split(string, array, separator) is the manual version of field splitting, useful when the string to split isn't $0 itself. It returns the number of pieces produced.

Show output 
3 a c

Build a formatted string with sprintf

awk -F, 'NR>1{line=sprintf("%-8s $%d",$1,$3); print line}' employees.csv

sprintf formats like printf but returns the string instead of printing it, so it can be stored in a variable and used later.

Show output 
alice    $95000
bob      $72000
carol    $110000
dave     $68000
erin     $81000
frank    $88000

Extract matched text with match(), RSTART, and RLENGTH

echo "order-4821-confirmed" | awk '{if (match($0,/[0-9]+/)) print substr($0,RSTART,RLENGTH)}'

match() sets RSTART and RLENGTH to where the regex matched, which substr() can then use, useful when the interesting text isn't at a fixed position.

Show output 
4821

Concatenate strings and fields

awk -F, 'NR>1{print $1 " works in " $2}' employees.csv

Placing values next to each other with nothing between them concatenates. No + or explicit operator needed.

Show output 
alice works in engineering
bob works in design
carol works in engineering
dave works in sales
erin works in design
frank works in engineering

Control flow: conditionals and loops

if/else, a ternary, for, and while, for logic that doesn't fit a single expression.

Branch per record with if/else

awk -F, 'NR>1{if ($3>90000) print $1,"senior"; else print $1,"standard"}' employees.csv

Standard if/else, useful once the logic is more than a single pattern can express cleanly.

Show output 
alice senior
bob standard
carol senior
dave standard
erin standard
frank standard

Use a ternary for a short branch

awk -F, 'NR>1{print $1, ($3>90000?"high":"normal")}' employees.csv

condition ? a : b is a compact alternative to if/else when the branch is a single value.

Show output 
alice high
bob normal
carol high
dave normal
erin normal
frank normal

Loop over every field with for

echo "a b c d" | awk '{for(i=1;i<=NF;i++) print i, $i}'

A C-style for loop indexing $i from 1 to NF visits every field on the line, regardless of how many there are.

Show output 
1 a
2 b
3 c
4 d

Loop with while

awk 'BEGIN{i=1; while(i<=5){printf "%d ", i; i++}; print ""}'

while is the other loop form, useful when the exit condition doesn't fit for's init/test/increment shape naturally.

Show output 
1 2 3 4 5

Build a repeated-character string in a loop

awk 'BEGIN{for(i=0;i<20;i++) s=s "="; print s}'

Concatenating onto s inside a loop is awk's equivalent of Python's "=" * 20, no repeat operator exists, so a loop does the job.

Show output 
====================

Trim a record down to N fields

echo "a b c d e" | awk '{NF=3; print}'

Assigning to NF truncates (or, if larger, extends) the record. Because NF changed, awk rebuilds $0 from the remaining fields, the same rebuild that reassigning any single field triggers.

Show output 
a b c

Blank out a field but keep its place

awk -F, -v OFS=, 'NR>1{$2=""; print}' employees.csv

Setting a field to an empty string keeps its position (and delimiter) instead of removing it entirely, unlike deleting an array element.

Show output 
alice,,95000
bob,,72000
carol,,110000
dave,,68000
erin,,81000
frank,,88000

Working with multiple files

NR counts across every file; FNR resets for each one.

Tell files apart with FNR and FILENAME

awk 'FNR==1{print FILENAME, "FNR="FNR, "NR="NR}' app.log access.log

FNR (file NR) restarts at 1 for every new input file, while NR keeps counting across all of them. FILENAME names whichever file is currently open.

Show output 
app.log FNR=1 NR=1
access.log FNR=1 NR=9

Count records across every file

awk 'END{print NR}' app.log access.log

With two files given, NR in END is the combined total, 8 lines from app.log plus 8 from access.log.

Show output 
16

Stop reading a file early with nextfile

awk 'FNR==1{print FILENAME; nextfile}' app.log access.log employees.csv

nextfile skips straight to the next input file, useful for peeking at just the first line (or first match) of each file in a batch without reading the rest.

Show output 
app.log
access.log
employees.csv

Cross-reference two files

awk -F, 'NR==FNR{allowed[$1]=1; next} FNR>1 && !($1 in allowed){print $1}' allowed.txt employees.csv

NR==FNR is only true while reading the first file, so that block builds a lookup table from it; next skips straight to the following record. The second block then reports names from the second file that aren't in that table.

Show output 
dave
erin
frank

CSV and tabular data

Field splitting makes awk a natural fit for CSV-shaped files, provided fields don't contain embedded commas.

Skip a header row

awk -F, 'NR>1' employees.csv

NR>1 alone, as a pattern with no action, prints every record after the first. The single most common guard in CSV-processing awk scripts.

Show output 
alice,engineering,95000
bob,design,72000
carol,engineering,110000
dave,sales,68000
erin,design,81000
frank,engineering,88000

Reorder columns

awk -F, -v OFS=, 'NR>1{print $1,$3,$2}' employees.csv

Printing fields in a different order, with OFS set to match the input delimiter, rewrites the CSV's column order.

Show output 
alice,95000,engineering
bob,72000,design
carol,110000,engineering
dave,68000,sales
erin,81000,design
frank,88000,engineering

Turn a CSV into an aligned table

awk -F, 'NR==1{next} {printf "%-6s %-12s %s\n",$1,$2,$3}' employees.csv

next skips straight to the next record without running the rest of the script, a tidy way to drop the header before a printf-formatted body.

Show output 
alice  engineering  95000
bob    design       72000
carol  engineering  110000
dave   sales        68000
erin   design       81000
frank  engineering  88000

Apply a calculation to a column

awk -F, -v OFS=, 'NR>1{$3=sprintf("%.0f",$3*1.10); print}' employees.csv

Reassigning $3 to a computed value, then letting the field reassignment trigger a rebuild of $0, applies a 10% raise across every row.

Show output 
alice,engineering,104500
bob,design,79200
carol,engineering,121000
dave,sales,74800
erin,design,89100
frank,engineering,96800

Analysing an access log

Log lines are field-rich by nature; awk is usually the fastest way to turn them into numbers.

Extract the timestamp from bracketed text

awk -F'[][]' '{print $2}' access.log

Using [ and ] themselves as the field separator (inside a character class, so they're literal, not a range) splits the line around the bracketed date, landing it in $2.

Show output 
05/Jul/2026:09:12:01 +0000
05/Jul/2026:09:12:03 +0000
05/Jul/2026:09:12:07 +0000
05/Jul/2026:09:12:15 +0000
05/Jul/2026:09:12:22 +0000
05/Jul/2026:09:12:40 +0000
05/Jul/2026:09:13:02 +0000
05/Jul/2026:09:13:10 +0000

Strip the leading quote from the HTTP method

awk '{gsub(/"/,"",$6); print $6}' access.log

The quoted request splits across three default fields; $6 is the method with a leading " still attached. gsub on a specific field, not $0, cleans up just that one.

Show output 
GET
GET
GET
POST
GET
POST
GET
GET

Filter to 4xx responses

awk '$9 ~ /^4/{print $1, $9}' access.log

Field 9 is the status code once the quoted request has consumed fields 6-8. Matching ^4 catches both 401s and 404s in one pattern.

Show output 
198.51.100.7 401
198.51.100.7 401
203.0.113.9 404

Average response size and duration

awk '{bytes+=$(NF-1); time+=$NF} END{printf "avg_bytes=%.0f avg_time=%.3f\n", bytes/NR, time/NR}' access.log

Using $(NF-1) and $NF for bytes and duration is more robust than a fixed field number, since it doesn't depend on how many words the quoted request happens to contain.

Show output 
avg_bytes=1120 avg_time=0.030

Rank status codes by frequency

awk '{count[$9]++} END{for (c in count) print count[c], c}' access.log | sort -rn

The same counting pattern as before, piped through sort -rn to put the most frequent status code first.

Show output 
5 200
2 401
1 404

Practical one-liners and pipelines

awk rarely works alone; it's usually one stage in a chain with grep, sed, and sort.

Deduplicate a field while keeping first-seen order

awk '!seen[$1]++' access.log

seen[$1]++ is 0 (falsy) the first time a value appears and truthy after, so !seen[$1]++ matches only the first occurrence of each value, preserving original order, unlike sort -u which reorders.

Show output 
198.51.100.7 - - [05/Jul/2026:09:12:01 +0000] "GET /index.html HTTP/1.1" 200 1024 0.021
203.0.113.5 - - [05/Jul/2026:09:12:03 +0000] "GET /style.css HTTP/1.1" 200 512 0.008
203.0.113.9 - - [05/Jul/2026:09:12:22 +0000] "GET /index.html HTTP/1.1" 200 1024 0.019

Narrow with grep, then extract fields with awk

grep error app.log | awk '{print $1, $2, $3}'

A common division of labour: grep filters the lines worth looking at, awk pulls out just the timestamp fields from what's left.

Show output 
Jul 5 09:14:22
Jul 5 09:14:28
Jul 5 09:15:01

Reshape with sed, then filter and extract with awk

sed 's/refused/denied/' app.log | awk '/error/{print $3, $6, $7}'

sed rewrites text in place in the stream; awk downstream still matches and extracts fields from the edited text, not the original.

Show output 
09:14:22 error: connection
09:14:28 error: connection
09:15:01 error: connection

Extract a field, then rank it with sort and uniq

awk '{print $9}' access.log | sort | uniq -c | sort -rn

The alternative to counting inside awk with an array: hand the extracted field to the standard sort | uniq -c | sort -rn tallying pipeline instead.

Show output 
      5 200
      2 401
      1 404

Read an environment variable

REPORT_TITLE="Weekly Summary" awk 'BEGIN{print ENVIRON["REPORT_TITLE"]}'

ENVIRON is an array of the process's environment variables, keyed by name, handy for parameterising a script without -v.

Show output 
Weekly Summary

Load a script from a file

printf 'BEGIN { FS = ","; OFS = "\t"; print "name", "dept" }\nNR > 1 { print $1, $2 }\n' > report.awk
awk -f report.awk employees.csv

-f reads the program from a file instead of the command line. Worth it once a script has more than a line or two, or gets reused across files, the same trade-off as sed -f.

Show output 
name	dept
alice	engineering
bob	design
carol	engineering
dave	sales
erin	design
frank	engineering