How To Filter The Output of Commands
Razvan Ludosanu
Founder, learnbackend.dev
Published: 12/12/2023
Filtering the output with grep
To filter a command's output, you can feed the data it writes to the standard output to the grep command using the pipe operator (|) as follows:
Where:
- command is the command you want to filter the output of.
- pattern is either a string pattern or regular expression.
Alternatively, to filter the content of a file, you can pass it as an argument of the grep command as follows:
Where:
- file is the path of the file you want to filter the output of.
For example, this command will filter the server logs at the information level contained in the logs file by only matching and outputting the lines containing the INFO string:
Note that this command can also be written like this:
Specifying multiple patterns at once
To specify multiple patterns at once, you can use the grep command with the -e flag as follows:
Note that the -e flag can be repeated multiple times, for each pattern.
For example, this command will only output the Docker images starting with an n or u character using two regular expressions:
You can learn more about matching multiple patterns at once with our other articles on how to grep multiple strings, words, and patterns and how to grep across multiple lines.
Excluding lines based on a pattern
To invert the search and return lines that do not include a specific pattern or exclude a specific string, you can use the grep command with the -v flag as follows:
For example, this command will exclude all the listed entries with a .js or .json file extension:
Where:
- -E turns on Extended Regex Expressions for grep. For more information, see our section on common gotchas with grep.
You can learn more about excluding patterns with our article on how to exclude patterns or files with grep.
Easily filter commands using Warp Block Filtering feature
If you’re using Warp as your terminal, you can easily filter outputs and logs using the Warp Block Filtering feature:
To apply a dynamic filter to a block, you can either click on the filter icon in the top right corner of a block or press Option+Shift+F, and type in a string or pattern to match. You can learn more about this feature on the official Warp Block Filtering page.
Filtering and formatting the output with awk
The awk command is a powerful tool used for pattern scanning and text processing.
Similar to grep, you can filter the output of a command based on a pattern using the following awk command:
Where:
- command is the command you want to filter the output of.
- pattern is a string pattern or regular expression.
- print is an action (a command) used to define what to do with the filtered output.
For example, this command will filter the server logs at the information level contained in the logs file by only matching and printing the lines containing the INFO string:
Outputting specific columns
To extract and print specific columns of the output, you can use the print action of the awk command followed by the columns numbers as follows:
Where:
- $N is the number of the column to output.
For example, this command will only display the permissions and names of the files listed by the ls -l command, which are respectively the first and ninth columns:
Replacing patterns in the output
To replace the first occurrence of a pattern in the output, you can use the sub action of the awk command as follows:
Where:
- pattern is a regular expression.
- string is a replacement string.
- column is the number of the column the pattern should be replaced in.
For example, this command will substitute the domain names of the email addresses with three * characters:
Alternatively, to replace all the occurrences of a pattern in the output, you can use the gsub action.
For example, this command will substitute every single letter of the passwords with a * character:
Removing duplicate lines with uniq
To remove duplicate lines from an output, you can use uniq command in combination with the sort command as follows:
For example, this command will first sort the content of the logs files alphabetically using the sort command, then deduplicate and count each line using the uniq -c command:
Showing the first lines with head
To show the first 10 lines of a command output, you can use the head command as follows:
Alternatively, you can specify a file path using the following syntax:
Showing the first N lines
To show a specific number of lines instead, you can use the -n flag as follows:
For example, this command will sort the content of the logs file in reverse order and only output the first 2 lines:
Showing the last lines with tail
To show the last 10 lines of a command output, you can use the tail command as follows:
Alternatively, you can specify a file path using the following syntax:
Showing the last N lines
To show a specific number of lines instead, you can use the -n flag as follows:
For example, this command will only show the last 3 lines of the logs file containing the ERROR string:
Note that if the number is prefixed with a plus sign (+), the tail command will display the output from that specified line number onwards.
For example, this command will output the last two lines of the logs file:
And this command will output the content of the logs file from the second line onwards:
Written by
Razvan Ludosanu
Founder, learnbackend.dev
Filed Under
Related Articles
Grep Multiple Strings
How to filter lines and extract specific information from the output of commands or text files based on string patterns and regular expressions with grep.
How to Make Grep Case Insensitive
By default, grep is case sensitive
Grep Across Multiple Lines
Guide on several cases of using grep across multiple lines
Grep In a Directory
Learn how to use grep to search for words and phrases within a directory and all its subdirectories, a specific directory, all files, and other variations.
Exclude With Grep
Excluding unwanted key terms or directories when using grep
Grep Count
Efficiently count lines or occurrences in a file.
