# 10 Linux Commands - every developer should know. If you have some experience in building enterprise applications, chances are you have used a Linux system. The Linux command-line interface (CLI) is an essential feature that developers appreciate. The CLI is powerful and flexible, allowing developers to perform complex tasks quickly and efficiently. In this post, I would like to list down 10 most useful commands which every developer should know. I am intentionally omitting some of the basic but very widely used commands e.g. `ls, cd,mkdir, rm, cp, mv` and so on with an assumption that they are too basic and should be already known to most of the regular users. ## 1 . tail (display the last lines of a file) `tail` is a Linux command that displays the last part of a file or multiple files. It can be useful for monitoring log files or keeping track of changes to a file in real time. Here are some commonly used options for `tail` and some examples of how to use them: 1. `-n` or `--lines`: This option specifies the number of lines to display. **Display the last 5 lines of a file:** 1. ```bash tail -n 5 file1.txt ## will display first 5 lines for file1.txt ``` 2. `-f` or `--follow`: This option will output the last part of a file in real-time as it is being written to. This is commonly used for monitoring log files. **Display new log entries as they are added to the file.** ```bash tail -f /var/log/messages ## will start straming file from end. ``` 3. `-q` or `--quiet`: This option suppresses the display of file headers when displaying multiple files. **Display only the last 5 lines of each file without showing the filename headers.** ```bash tail -q -n 5 file1.txt file2.txt ## will show content from file1,file2 stacked with out name(file1.txt,file2.txt) ``` 4. `-c` or `--bytes`: This option specifies the number of bytes to display. **Display the last 100 bytes of** `file1.txt` 1. ```bash tail -c 100 file1.txt ## shows 100 bytes starting from end ``` 5. `-v` or `--verbose`: This option displays the name of each file before the output. Display "==> file1.txt <==" before the last 5 lines of the file. 1. ```bash tail -v file1.txt ## ==> file1.txt <== followed by the content of file ``` These are just a few examples of how `tail` can be used. By combining these options and using `tail` with other commands, you can create powerful and flexible file monitoring and processing tools. ## 2 . head (display the first lines of a file) `head` is a command in Linux used to display the first few lines of a file. By default, it displays the first 10 lines of a file. Here are some examples of using `head` with different options: 1. Display the first 5 lines of a file: ```bash head -n 5 file.txt ``` 1. Display the first 20 bytes of a file: ```bash head -c 20 file.txt ``` 1. Display the first line of multiple files: ```bash head -n 1 file1.txt file2.txt file3.txt ``` 1. Display the first few lines of a file, with a message indicating the file name: ```bash head -v file.txt ``` 1. Display the first 5 lines of a file, and include the line numbers: ```bash head -n 5 -v file.txt ``` Here, the options used are: * `-n`: specify the number of lines to display. * `-c`: specify the number of bytes to display. * `-v`: display the name of the file before the output. These are just a few examples of using `head` with different options. There are many other options available that can be used to customize the output of this command which you can explore with `tail --help`. ## 3 . cat (concatenate and display files) The `cat` command is used to concatenate and display files. It can also be used to create new files by combining existing files. Here are some examples of using the `cat` command with various options: 1. Display contents of a single file: ```bash cat file.txt ``` 1. Combine multiple files and display their contents: ```bash cat file1.txt file2.txt file3.txt ``` 1. Display line numbers along with file contents: ```bash cat -b file.txt 1 In web applications, it's common to have long-running processes that need to be executed asynchronously. 2 When these processes take a long time to complete, 3 it's important to provide feedback to the user so 4 that they know what's happening and can cont ``` 1. Append one file to another: ```bash cat file1.txt >> file2.txt ``` 1. Create a new file by combining existing files: ```bash cat file1.txt file2.txt > newfile.txt ``` In the first example, `cat` is used to display the contents of a single file. In the second example, `cat` is used to combine multiple files and display their contents. In the third example, the `-n` option is used to display line numbers along with file contents. In the fourth example, the `>>` operator is used to append the contents of one file to another. In the fifth example, the `>` operator is used to create a new file by combining the contents of two existing files. ## 4 . diff ( compare two files line by line) The `diff` command in Linux is used to compare two files line by line and show the differences between them. It can also be used to compare two directories and their contents. Here are some common options used with the `diff` command: * `-u` or `--unified`: shows the differences in a unified format, making it easier to read and understand. * `-c` or `--context`: shows the differences in a context format, which includes a few lines before and after each difference to provide more context. * `-r` or `--recursive`: compares directories and their contents recursively. * `-i` or `--ignore-case`: ignores case differences in the files being compared. * `-w` or `--ignore-all-space`: ignores all whitespace differences in the files being compared. Here are some examples of using the `diff` command with different options: Example 1: Compare two files using the default output format ```bash diff file1.txt file2.txt ``` Example 2: Compare two files using the unified output format ```bash diff -u file1.txt file2.txt ``` Example 3: Compare two directories and their contents recursively ```bash diff -r dir1/ dir2/ ``` Example 4: Compare two files ignoring case differences ```bash diff -i file1.txt file2.txt ``` Example 5: Compare two files ignoring all whitespace differences ```bash diff -w file1.txt file2.txt ``` In these examples, `file1.txt` and `file2.txt` are files being compared, and `dir1/` and `dir2/` are directories being compared. The output of the `diff` command shows the differences between the files or directories. By using different options, the output format and the behavior of the `diff` command can be customized. ## 5 . patch (apply a diff file to a file or directory) `patch` is a command-line tool that allows you to apply a patch file to a file or directory. A patch file is a file that contains the differences between two versions of a file or directory. When you apply a patch file, the changes in the patch file are applied to the file or directory, resulting in a new version of the file or directory. Here are some examples of how to use the `patch` command with different options: **Example 1: Apply a patch to a file** Suppose you have a patch file called `patch.diff` and a file called `file.txt`. To apply the patch to the file, you can use the following command: ```bash patch file.txt patch.diff ``` This command will apply the changes in the patch file to the file. **Example 2: Apply a patch to a directory** Suppose you have a patch file called `patch.diff` and a directory called `dir/`. To apply the patch to the directory and all its files, you can use the following command: ```bash patch -p1 < patch.diff ``` The `-p1` option tells `patch` to strip one level of the directory path from the files in the patch file. This is necessary because the patch file contains the full path to the files. **Example 3: Create a patch file** Suppose you have two files called `file1.txt` and `file2.txt` and you want to create a patch file that contains the differences between the two files. To create the patch file, you can use the following command: ```bash diff -u file1.txt file2.txt > patch.diff ``` This command will create a patch file called `patch.diff` that contains the differences between `file1.txt` and `file2.txt`. **Example 4: Ignore whitespace changes** Suppose your patch file contains changes that only affect whitespace. To ignore these changes when applying the patch, you can use the following command: ```bash patch -l file.txt patch.diff ``` The `-l` option tells `patch` to ignore changes in whitespace. **Example 5: Apply a patch with dry-run** Suppose you want to see what changes a patch file will make before applying it. To do this, you can use the following command: ```bash patch --dry-run file.txt patch.diff ``` The `--dry-run` option tells `patch` to simulate the patching process without actually making any changes to the file. This can be useful for testing a patch file before applying it. These are just a few examples of how to use the `patch` command. `patch` provides many options for customizing the patching process, so be sure to consult the manual (`man patch`) for more information. ## 6 . tar (extract files from the archive/create an archive from files ) `tar` is a command-line utility in Linux that is used to create and extract archives from one or more files or directories. The name "tar" stands for "tape archive," as it was originally designed to write data to tape drives. Here are some of the commonly used options with `tar`: * `-c` (create) - create a new archive * `-x` (extract) - extract files from an archive * `-v` (verbose) - display progress and filenames while processing * `-f` (file) - specify the filename of the archive * `-z` (gzip) - compress or uncompress the archive using gzip Here are five examples of using `tar` with different options: 1. Create a new archive of all files in the current directory: ```bash tar -cvf archive.tar * ``` 1. Extract all files from an archive: ```bash tar -xvf archive.tar ``` 1. Extract all files from a gzip-compressed archive: ```bash tar -xzvf archive.tar.gz ``` 1. Create a gzip-compressed archive of a directory: ```bash tar -czvf archive.tar.gz directory/ ``` 1. Extract a specific file from an archive: ```bash tar -xvf archive.tar file.txt ``` These are just a few examples of how `tar` can be used to create and extract archives. The utility offers many more options and can be used in more complex ways for a variety of use cases. ## 7 . sed (stream editor for filtering and transforming text) `sed` (stream editor) is a extremly powerful tool used for filtering and transforming text. It can be used to modify text files in place, or to process text output from other commands. `sed` works by reading input line by line, applying a set of rules (expressions) to each line, and then printing the modified result to the output. Here are 10 examples of `sed` commands with explanations of their options: 1. Replace all occurrences of a string in a file with another string: ```bash sed -i 's/old_string/new_string/g' file.txt ``` The `-i` option is used to modify the file in place. The `s` command replaces the first occurrence of `old_string` with `new_string`. The `g` flag at the end specifies that all occurrences should be replaced. 2. Remove all occurrences of a string in a file: ```bash sed -i '/string_to_remove/d' file.txt ``` The `/string_to_remove/d` expression deletes all lines that contain `string_to_remove`. The `-i` option is used to modify the file in place. 3. Add a line to the beginning of a file: ```bash sed -i '1i\new_line' file.txt ``` The `1i\` command inserts `new_line` at line 1 of the file. The `-i` option is used to modify the file in place. 4. Add a line to the end of a file: ```bash sed -i '$a\new_line' file.txt ``` The `$a\` command appends `new_line` to the end of the file. The `-i` option is used to modify the file in place. 5. Remove empty lines from a file: ```bash pythonCopy codesed -i '/^$/d' file.txt ``` The `/^$/d` expression deletes all lines that are empty (contain only whitespace). The `-i` option is used to modify the file in place. 6. Replace the first occurrence of a string in a file on a specific line number: ```bash sed -i '3s/old_string/new_string/' file.txt ``` The `3s/old_string/new_string/` expression replaces the first occurrence of `old_string` with `new_string` on line number 3. The `-i` option is used to modify the file in place. 7. Remove all whitespace from the beginning and end of each line: ```bash sed -i 's/^[[:space:]]*//;s/[[:space:]]*$//' file.txt ``` The `s/^[[:space:]]*//` expression removes all whitespace from the beginning of each line. The `s/[[:space:]]*$//` expression removes all whitespace from the end of each line. The `-i` option is used to modify the file in place. 8. Remove a specific line from a file: ```bash sed -i '5d' file.txt ``` The `5d` expression deletes line number 5 from the file. The `-i` option is used to modify the file in place. 9. Replace a string in a file only on lines that match a specific pattern: ```bash sed -i '/pattern/s/old_string/new_string/' file.txt ``` The `/pattern/` expression matches all lines that contain `pattern`. The `s/old_string/new_string/` expression replaces the first occurrence of `old_string` with `new_string` on all matching lines. The `-i` option is used to modify the file. 10. Remove all consecutive duplicate lines from a file: ```bash sed -i '$!N; /^\(.*\)\n\1$/!P; D' file.txt ``` In conclusion, `sed` is a versatile command-line tool that can be used to perform a wide range of text-processing tasks. Its ability to read and modify files line by line makes it particularly useful for working with large text files, and its powerful regular expression capabilities provide a flexible and efficient way to search for and manipulate text. With practice, developers can become proficient at using `sed` to streamline their text processing workflows and automate common tasks. ## 8 . awk (pattern scanning and processing language) AWK is a programming language designed for text processing and data extraction. It is especially useful for processing files that contain structured data, such as CSV files, log files, and configuration files. AWK provides a range of built-in functions for manipulating strings, performing arithmetic operations, and manipulating data structures such as arrays. Here are 10 examples of how to use AWK, along with explanations of the options used: 1. Print the first field of each line in a file: ```bash awk '{print $1}' filename.txt ``` This command tells AWK to print the first field of each line in the file `filename.txt`. The `$1` variable refers to the first field of the current line. 1. Print the second field of each line in a file: ```bash awk '{print $2}' filename.txt ``` This command tells AWK to print the second field of each line in the file `filename.txt`. The `$2` variable refers to the second field of the current line. 1. Print the last field of each line in a file: ```bash awk '{print $NF}' filename.txt ``` This command tells AWK to print the last field of each line in the file `filename.txt`. The `$NF` variable refers to the last field of the current line. 1. Print all fields of each line in a file: ```bash awk '{print $0}' filename.txt ``` This command tells AWK to print the entire line for each line in the file `filename.txt`. The `$0` variable refers to the entire line. 1. Print the total number of lines in a file: ```bash awk 'END {print NR}' filename.txt ``` This command tells AWK to print the number of records (`NR`) at the end of the file. The `END` keyword specifies that this action should be taken after all lines have been processed. 1. Print the total number of fields in a file: ```bash awk '{print NF}' filename.txt | sort -rn | head -1 ``` This command tells AWK to print the number of fields for each line in the file `filename.txt`. The output is then piped to `sort -rn` to sort the results in reverse numerical order, and `head -1` is used to print the first line (i.e., the line with the highest number of fields). In conclusion, AWK is a powerful tool for processing text files and extracting data. By learning AWK, developers can streamline their workflows and automate common text-processing tasks. ## 9 . grep (search for a pattern in a file) `grep` is a command-line tool used for searching text in files or directories. It can search for a specific pattern in a file and return the lines that contain that pattern. Here is the basic syntax of the `grep` command: ```bash grep [OPTIONS] PATTERN [FILE...] ``` In this syntax, `OPTIONS` are the different options available for the `grep` command, `PATTERN` is the text pattern to search for, and `FILE` is the file or files to search in. Here are 7 examples of using `grep` command with some of the commonly used options: 1. Search for a pattern in a file: ```bash grep "pattern" file.txt ``` 2. Search for a pattern in multiple files: ```bash grep "pattern" file1.txt file2.txt ``` 3. Search for a pattern in all files in a directory: ```bash grep "pattern" * ``` 4. Search for a pattern in a file, ignoring case: ```bash grep -i "pattern" file.txt ``` 5. Search for a pattern in a file, with line numbers: ```bash grep -n "pattern" file.txt ``` 6. Search for a pattern in a file, showing only the matching text: ```bash grep -o "pattern" file.txt ``` 7. Search for a pattern in all files in a directory recursively: ```bash grep -r "pattern" directory/ ``` 8. Find all the files which do not have a specific pattern: ```bash grep -v "pattern" file.txt ``` The `-v` option in `grep` is used to invert the search and display only the lines that do not contain the specified pattern These are just a few examples of the many options available with `grep`. The `grep` command can be a powerful tool for searching text in files and directories, and can be used in combination with other commands to perform complex operations on text files. ## 10\. cut (remove sections from each line of a file) `cut` is a command-line utility in Linux that is used to extract parts of a file or a stream of data by specifying a delimiter. Here is a brief overview of the options available for `cut`: ```bash -d - Specify a delimiter character -f - Specify a list of fields to extract ``` Here are 7 examples of using `cut` with different options: 1. Extract the first field of each line of a file: ```bash cut -d',' -f1 file.txt ``` This command uses `,` as the delimiter and extracts the first field of each line of `file.txt`. 1. Extract the first 3 characters of each line of a file: ```bash cut -c1-3 file.txt ``` This command extracts the first three characters of each line of `file.txt`. 1. Extract the last field of each line of a file: ```bash cut -d',' -fNF file.txt ``` This command uses `,` as the delimiter and extracts the last field of each line of `file.txt`. 1. Extract the 2nd and 3rd fields of each line of a file: ```bash cut -d',' -f2,3 file.txt ``` This command uses `,` as the delimiter and extracts the 2nd and 3rd fields of each line of `file.txt`. 1. Extract the characters between the 10th and 20th positions of each line of a file: ```bash cut -c10-20 file.txt ``` This command extracts the characters between the 10th and 20th positions of each line of `file.txt`. 1. Extract the first and last fields of each line of a file: ```bash cut -d',' -f1,NF file.txt ``` This command uses `,` as the delimiter and extracts the first and last fields of each line of `file.txt`. 1. Extract the fields in reverse order of each line of a file: ```bash cut -d',' --output-delimiter=' ' -f$(awk -F, '{print NF; exit}')-1 file.txt ``` This command uses `,` as the delimiter and extracts the fields of each line of `file.txt` in reverse order, separated by a space. The `awk` command is used to get the number of fields in the file and pass it to `cut` to extract the fields in reverse order. 1. Extract fields 2 through 4 of a file using a tab delimiter: ```bash bashCopy codecut -d$'\t' -f2-4 file.txt ``` This command uses a tab character as the delimiter and extracts fields 2 through 4 of each line of `file.txt`. In conclusion, `cut` is a versatile and useful command-line utility for extracting fields or characters from a file or a stream of data. It is often used in conjunction with other commands such as `awk` and `sed` to manipulate and process text files. By mastering `cut`, developers can become more efficient at working with data and automating tasks on a Linux system. ## Honorable mention While the commands covered in this post are certainly useful for developers, it's important to note that there are many more Linux commands worth exploring and becoming familiar with. Due to the limitations of length, I did not cover them here. For example, `curl` and `wget` are powerful tools for downloading files from the web, `top` and `ps` provide information on system processes, `kill` is useful for terminating processes, and `ping` and `traceroute` are essential for checking network connectivity. `ssh` and `scp` are valuable for remote login and file transfer, respectively. By using the `--help` option or referring to Linux command manuals, developers can continue to learn about new commands and improve their workflow. I welcome any feedback or comments from our readers and hope that this post has helped expand your knowledge of commonly used Linux commands. **Happy Learning**. ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1678035160245/a58277e4-d9eb-49c2-a15d-8f615ff2adb5.png align="center")