Using pipes on Linux to get a lot more done

One of the things that I have always loved about Unix and then Linux is how it allows me to connect a series of commands together with pipes and get a lot of work done without a lot of effort. I can generate the output that I need in the form that I need it. It’s not just the existence of the pipes themselves, but the flexibility of the Linux commands. You can run commands, select portions of the output, sort the results or match on specific strings and you can pare the results down to just what you want to see.

In this post, we’re going to look at a couple commands that demonstrate the power of the pipe and how easily you can get commands to work together.

Checking chkrootkit stats

This first example command starts with using sudo to run the chkrootkit command. This command checks for signs of rootkits on your system by using a detailed process to detect signatures that are related to known rootkits. The command will easily generate well over 100 lines of output. To get a very useful summary of what it finds, however, you can run a command like this:

$ sudo chkrootkit | awk '{print $(NF-1) " " $NF}' | sort | uniq -c
      1 a while...
      2 enp0s25: PF_PACKET(/usr/sbin/NetworkManager)
      1 is `/'
     21 not found
      3 nothing deleted
      2 nothing detected
     56 nothing found
     41 not infected
      3 not tested
      1 PF_PACKET sockets
      1 pts/0 bash
      1 suspect files
      1 TTY CMD
      1 /usr/lib/.build-id /usr/lib/debug/.dwz
      1 /var/run/utmp !

The commands above run chkrootkit as root, select only the last two strings in each line of output, sort the results and then count how many times each of the two-string results were returned. While this is in no way a substitute for looking at the complete output, it can tell you a lot about the state of the system with respect to findable rootkits.

In the output above, we can easily see that most of this output is likely what we’d hope to see. The “nothing deleted” and “nothing detected” are nice, but noting 41 “not infected” messages is clearly good news. The single “suspect files” message is actually “no suspect files” and required a second look at the original output to confirm.

The awk expression in the overall command is displaying the last two fields. Since NF is awk‘s way of expressing the number of fields, $NF is the value of the final field and $(NF-1) is the value of the preceding field. The blank within the quotes keeps these fields from being jammed together.

The sort command then sorts all of the output alphanumerically while the final command, uniq -c, counts how many times each output line appears in the each sequential group in the overall output.

Note that it helps to be familiar with where all the stats came from. Looking at the chkrootkit output directly, you would likely see many lines like these

ROOTDIR is `/'
Checking `amd'... not found
Checking `basename'... not infected
Checking `biff'... not found
Checking `chfn'... not infected
Checking `chsh'... not infected
Checking `cron'... not infected
Checking `crontab'... not infected
Checking `date'... not infected
Checking `du'... not infected
Checking `dirname'... not infected
Checking `echo'... not infected
Checking `egrep'... not infected
Checking `env'... not infected
Checking `find'... not infected
Checking `fingerd'... not found
Checking `gpm'... not found
Checking `grep'... not infected

These lines all indicate the chkrootkit did a lot of checking on possible command infections, but found no problems. A little later, you might see lines like these indicating that possible signs of malware were not found on the system.

Searching for sniffer's logs, it may take a while... nothing found
Searching for HiDrootkit's default dir... nothing found
Searching for t0rn's default files and dirs... nothing found
Searching for t0rn's v8 defaults... nothing found

The piped command provides a useful summary and can be easily turned into a script so you don’t have to type it or even remember it every time you want to use it.

#!/bin/bash
sudo chkrootkit | awk '{print $(NF-1) " " $NF}' | sort | uniq -c

The command took only about 30 seconds to run on my system and makes it very easy for me to run these checks routinely.

User processes

To generate a list of the process IDs associated with processes that some particular user is running, you can use a command like this one:

$ ps aux | grep nemo | grep -v grep | awk '{print $2}'
903665
903674
903680
903695
903703

This piped command uses ps aux to list all running processes, narrows the output down to only those processes being run by the user nemo, excludes the “grep nemo” command (since it’s not being run by nemo) and then reduces the listing to just the process IDs. If you want to just see the number of processes instead all the process IDs, add another pipe and a wc -l command.

$ ps aux | grep nemo | grep -v grep | awk '{print $2}' | wc -l
5

If you are looking at a user logged in on the system console, you might want to pipe the output to the column command so that you see all the processes on a single screen.

$ ps aux | grep shs | grep -v grep | awk '{print $2}' | column
4508    4620    4728    4764    4802    4856    4884    4893    5030    6003
4515    4621    4729    4770    4814    4868    4886    4895    5046    897442
4528    4623    4737    4774    4819    4869    4888    4896    5049    897447
4538    4650    4741    4775    4821    4878    4889    4897    5092    897455
4541    4703    4742    4781    4827    4879    4890    4992    5258    904175
4543    4710    4745    4788    4839    4880    4891    5016    5276    904178
4545    4723    4753    4790    4846    4881    4892    5021    5997    904179

Wrap-Up

As you can see from the commands above, pipes can help you turn the output of Linux commands into a form that displays just what you want to know. And, regardless of how complex these commands turn out to be, you can save them as aliases or scripts so that you don’t have to recreate them every time you need to use them.