How to Use Awk Command in Bash

Learn how to use awk command in bash for text processing and searching by following this article.

Awk is a scripting language widely used to process text in the command line. This command allows you to select various parts of texts based on the patterns you provide as input. 

In addition, you can search for the words or modify specific columns of texts using the awk command.

Prerequisites

For this tutorial, you’ll need a Linux machine. The machine should have root privilege or sudo user access. In addition, you don’t have to install the awk command to learn how to use awk command in bash, as it is installed by default. 

How to Use Awk Command in Bash

In this section, we’ll discuss how to use awk command in bash for text processing. The basic syntax of the awk command looks something like this:

awk ‘{action}’ file_name.extension

In the {action} part of the argument, we assign the task that needs to be done on the file name. This file name is mentioned as a second argument to the awk command.

Create a File to Use Awk Command in Bash

Before learning how to use awk command in bash, create a file first. For this tutorial, we’ve created a sample file called information.txt using the nano or vim editor.

The file looks something like this:

Print the Contents Using the Awk Command

The first action you can perform with the awk command is to print it. Similar to the cat command, we can use the print action to view the file’s contents on the BASH terminal. For this step, type:

awk '{print $0}' information.txt

The output should look something like this:

Use Command in Bash to Print with Line Numbers

To view the same content with the line number, you’ll use the NR variable between the print command and the $0 argument. Specifically, type:

awk '{print NR,$0}' information.txt 

Output:

Print a Specific Column Using the Awk Command

If you want to print a specific column using the awk command, you’ll use the $1 argument instead of $0. For instance, type:

awk '{print $1}' information.txt

You should get a similar output:

Similarly, for any column, you would change the field number. For example, to print column 4, you would use $4 as shown below:

awk '{print $4}' information.txt

In addition, you can print multiple columns by passing the comma-separated field numbers. Specifically, type:

awk '{print $1, $4}' information.txt

This will print column 1 and column 4 from the information.txt file.

Alternatively, you can use the NF variable to print the last column in a record. For instance, type:

awk '{print $NF}' information.txt 

You should get a similar output:

Print Specific Rows from Column Using the Awk Command

Along with columns, you can also print specific rows from that particular column by using the awk command. For this step, you’ll pipe the head command as shown below:

awk '{print $2}' information.txt | head -1 

To print more than one line, you’ll change the argument passed to the head command. For instance, to print 3 lines, you’d type:

awk '{print $2}' information.txt | head -3 

You should get a similar output:

Use Awk Command in Bash to Match Patterns

Apart from printing the specific columns and rows, you can also search text using the pattern and then process it. For instance, to search for names starting with the letter S, you would write:

awk '/^S/' information.txt

Output:

The command selected the entries starting with the letter S. The carrot symbol (^) indicates the beginning of the pattern. Similarly, to limit a character at the end of the pattern, you would use the dollar symbol ($).

In addition, you can also use the ! symbol. This is NOT a symbol, meaning that any symbol types after this one will not be matched. For instance, to search for entries that do NOT end in 0, you’d type:

awk '! /0$/' information.txt 

The output should look something like this:

Use Regular Expressions with Awk Command

To make text searching more advanced, you can also input regular expressions with actions in the awk command. For instance, if you want to search for ‘an’ pattern, you’d type:

awk ' /an/{print $0}' information.txt 

For any regular expression, you’d define it between the pair of slashes (//). For example, to search for information on all people from Berlin, you’d type:

awk '/Berlin/' information.txt 

In addition, you can also specify columns from which you want the particular data. For this step, you’d write field numbers as shown below:

awk '/Berlin/{print $1, $2}' information.txt 

If you have entries containing special characters, you’ll have to use an extra backward slash along with that character. For example, if you want to search for N/A, you’ll modify your command like this:

awk '/N\/A$/' information.txt 

Use Awk Command in Bash With Operators

For advanced data filtering, you can also use the comparison operators in the awk command. For instance, if you want to get a list of all those users whose age is above 30, you’d write:

awk '$3 > 30 { print $0 }' information.txt

You should get a similar output:

Pipe the Output in Awk Command

If you want to save the output from the awk command in a separate file, use the redirection operator. Specifically, type:

awk '$3 > 30 { print $0 }' information.txt >> newUsers.txt

The first part is pattern, and the second part is action. Usually, if you don’t give a pattern, the action applies to the entire file content. 

Print Data with Different Field Separators

The default field separator for the awk command is single space. If your file contains a different field separator such as, you can modify the awk command to consider that field separator. Specifically, type:

awk -F':' '{ print $1 }' information.txt

Example: Use Awk Command in Bash to List your Favorite Commands

To list your 10 favorite commands using awk, you’d type:

history | awk '{print $2}' | sort | uniq -c | sort -rn | head

Output:

This will first take all the commands from history and pass them to the awk command. The awk command will pick only the second column, sort it, pick unique entries, sort them again, and show the top 10 entries out of all the entries. 

That’s it for a quick guide on how to use awk command in bash. Awk is a great resource for Linux users for text manipulation in documents. This article has covered several ways in which you can use this command. You can also use this command in shell scripts if you are an expert user.

If you have any questions, feel free to let us know in the comments below.

If this guide helped you, please share it.