uniq command in Linux with examples

 The linux uniq command is basically used to remove all repeated lines in a file. This command is used when a line is repeated multiple times and replace these multiple lines with one line. This command is designed to work on sorted files.

Purpose : Remove repetitious lines from the sorted ‘input-file’ and send unique lines to the ‘output-file’. If no ‘output-file’ is specified, the output of the command is send to standard output. if no ‘input-file’ is specified, the command takes input from standard output.

Syntax :

$ uniq [option] [input-file] [output-file]

Options :

-c : Precede each output line by the number of times it occur.
-d : Display the repeated lines.
-u : Display the lines that are not repeated.

Examples :
Create a sample input file.

$ cat sample

Input :
This is a test file for the uniq command.
It contains some repeated lines.
It contains some repeated lines.
And some are different.
It contains some repeated lines.
It contains some repeated lines.

1. Removing duplicate lines when file is not sorted.

$ uniq sample

Output :
This is a test file for the uniq command.
It contains some repeated lines.
And some are different.
It contains some repeated lines.

2. Removing duplicate files after sorting.

$ sort sample -> sample1
$ uniq sample1

Output :
And some are different.
It contains some repeated lines.
This is a test file for the uniq command.

3. Finding number of times the lines are repeated.

$ uniq -c sample1

Output :

1 And some are different.
4 It contains some repeated lines.
1 This is a test file for the uniq command.

4. Finding repeated lines.

$ uniq -d sample1

Output :

It contains some repeated lines.

5. Finding lines which are unique and store in another file.

$ uniq -u sample1 out
$ cat out

Output :

And some are different.
This is a test file for the uniq command.