As I have mentioned several times in my previous Linux tutorials, nearly everything in Linux is a file, and very often they are text files. For instance, all of the configuration files in Linux are text files. To reconfigure an application in Linux, we simply need to open the configuration file, change the text, re-save, and then restart the application and our reconfiguration is applied.
With so many text files, manipulating text becomes crucial in managing Linux and Linux applications. In this tutorial, we'll look at several of the commands and techniques for manipulating text in Linux.
For demonstration purposes, we'll use files from the world's best Network Intrusion Detection System (NIDS), Snort.
Step 1: Cat That File
As demonstrated in an earlier tutorial, cat is probably the most basic text display command. Let's cat the Snort config file found in /etc/snort.
kali > cat /etc/snort/snort.conf
As you can see, the snort.conf is displayed on our screen until it comes to the end of the file. Not the most convenient or practical way to work with this file.
Step 2: Take the Head
If we just want to view the beginning of a file, we can use the head command. This command displays the first 10 lines of a file, by default.
kali > head /etc/snort/snort.conf
If we want to see more or less than the default 10 lines, we can tell head how many lines we want to see by putting the number of lines we want (with the - switch) between the command and the file name.
kali > head -20 /etc/snort/snort.conf
Here we can see that only the first 20 lines of snort.conf are displayed.
Step 3: Grab That Tail
Similar to the head command, we view the last lines of a file by using the tail command. Let's use it on the snort.conf.
kali > tail /etc/snort/snort.conf
Notice that it displays some of the last "includes" of the rules files, but not all of them. Let's now see if we can display all the rule "includes" by grabbing the last 20 lines of the snort.conf.
kali > tail -20 /etc/snort/snort.conf
Now we can view nearly all the rule includes all on one screen.
Step 4: Numbering Those Lines
Sometimes—especially with very long files—we may want the file displayed with line numbers. This is probably the case with the snort.conf, as it has over 600 lines. This makes it easier to reference changes and come back to the same place within a file. To display a file with line number, we simply type:
kali > nl /etc/snort/snort.conf
Note that each line now has a number making referencing much easier.
Step 5: I Grep That
After cat, grep is probably the most widely used text manipulation command. It's a filtering command; in other words, it enables us to filter the content of a file for display. If for instance, we wanted to see all the instances of where the word "output" occurs in our snort.conf file, we could ask cat to only display those lines where it occurs by typing:
kali > cat /etc/snort/ snort.conf | grep output
This command will first grab the snort.conf and then "pipe" it (|) to grep which will take it as input and then look for the occurrences of the word "output" and only display those lines. Grep is a powerful and essential command for working in Linux, as it can save us hours searching for every occurrence of a word or command.
Step 6: I Sed That Works
The sed command essentially allows us to search for occurrences of a word or text pattern and then do some work on it. The name comes from the concept of a stream editor and is a contraction of those two words. In its most basic form, sed operates like the find and replace function in Windows. Let's search for the word "mysql" in the snort.conf file using grep.
kali > cat /etc/snort/snort.conf | grep mysql
We can see that the grep command found two occurrences of the word mysql.
Let's say we want sed to replace every occurrence of mysql with MySQL (remember, Linux is case sensitive) and then save the new file to snort2.conf.
We could do this by typing:
kali > sed s/mysql/MySQL/g /etc/snort/snort.conf > snort2.conf
This command says, "search (s) for the word mysql and replace it with the word MySQL globally (i.e. wherever you find it in the file)."
kali > cat /etc/snort/snort2.conf | grep MySQL
Now, when we grep snort2.conf for mysql, we see that none were found and when we grep for MySQL, we find two occurrences of MySQL.
If we just want to replace only the first occurrence of the word mysql, we could leave out the trailing g (for global) and it would only replace the first occurrence.
kali > sed s/mysql/MySQL/ snort.conf > snort2.conf
The sed command can also be used to find and replace any specific occurrence of a word. For instance, if I want to only replace the second occurrence of the word mysql, I can simply place the number of the occurrence at the end of the command and sed will only replace the second occurrence of the word "mysql" with "MySQL".
kali > sed s/mysql/MySQL/3 snort.conf > snort2.conf
Step 7: Less is More
Although cat is good utility to display files and create small files, it certainly has its shortcomings when displaying large files. When we cat the snort.conf. it scrolls through numerous pages until it comes to the end. Not very practical.
For working with larger files, we have two other utilities, more and less. More came first and is the utility that the man (manual) pages use. Let's open snort.conf with more command.
kali > more /etc/snort/snort.conf
Notice that more displays only the first page and displays for us in the lower left corner how much of the file is shown (2% in this case).
To see additional lines or pages, we can use the ENTER key or PAGE UP or PAGE DN to move an entire page. To exit more, simply type "q" for quit.
less is very similar to more, but with additional functionality, hence the common Linux aficionado quip "Less is more".
Let's open snort.conf with less.
kali > less /etc/snort/snort.conf
Notice near the bottom left of the screen that less has highlighted the path to the file.
If we hit the / key, less will enable us to search for terms in the file. For instance, when setting up snort we need to determine how and where to send our output. If we need to find that section of the configuration file, we could simply type;
/output
This will immediately take us to the first occurrence of output and highlight it.
We can then look for the next occurrence of output, by typing "n" for next.
As you can see, less took us to the next occurrence of the word output, highlighted all the search terms and in this case took us directly to the output section of snort. How convenient!
Stay Tuned for More
That's it for this lesson, but there are many more to come, so check out our section on learning Linux basics to stay up to date. If you have any questions about this article, ask away in the comments below.