✂️ Remove duplicates from a slice in Go

If you want to remove duplicate values from a slice in Go, you need to create a function that:

• Iterates over the slice.
• Checks if a given value of the slice is in the set of the result values. If not, it adds the value to the resulting slice. If so, it skips the value (we already have it in the output slice).
• Returns new output slice with duplicates removed.

The most important part of the unique() function is to create a set of seen values, which is a map of type map[string]bool, so it can check if the element is already in the resulting slice.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

package main

import "fmt"

func unique(s []string) []string {
    inResult := make(map[string]bool)
    var result []string
    for _, str := range s {
        if _, ok := inResult[str]; !ok {
            inResult[str] = true
            result = append(result, str)
        }
    }
    return result
}

func main() {
    fmt.Println(unique([]string{"abc", "cde", "efg", "efg", "abc", "cde"}))
}

Output:

[abc cde efg]

As an output, you get a new slice with all values unique. The function works in the same way for slices of different types. You just need to replace all occurrences of type string with another selected type.

With the release of Generics in Go 1.18, you can write a universal function to remove duplicates without having to re-implement it every time a slice is of a different type. Check out how to do it here.