Benchmark: Methods to remove duplicates from array

Script Preparation code:

var array = [];
for (let i = 0; i < 100000; i++) {
   array.push(Math.floor((Math.random() * 10) + 1));
}

Tests:

Using indexOf
array.filter((item, index) => array.indexOf(item) != index);
Using lastIndexOf
array.filter((item, index) => array.lastIndexOf(item) != index);
Using a Set
[...new Set(array)]

Rendered benchmark preparation results:

Suite status: <idle, ready to run>

Previous results

Test case name	Result
Using indexOf
Using lastIndexOf
Using a Set

Fastest: N/A

Slowest: N/A

Latest run results:

Run details: (Test run date: one month ago)

User agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36

Browser/OS: Chrome 130 on Mac OS X 10.15.7

View result in a separate tab

Test name	Executions per second
Using indexOf	161.9 Ops/sec
Using lastIndexOf	141.8 Ops/sec
Using a Set	725.6 Ops/sec

Autogenerated LLM Summary (model llama3.2:3b, generated 2 months ago):

LLMs can make mistakes. Check important info.

Let's break down the provided benchmark and explain what's being tested, compared, and their pros and cons.

**Benchmark Definition**

The benchmark is designed to measure the performance of different methods for removing duplicates from an array. The script preparation code generates an array of 100,000 random integers, which is then used as input for the benchmarking tests.

**Test Cases**

There are three test cases:

1. **Using indexOf**: This method uses `array.indexOf(item) != index` to check if an item exists in the array. If it does, `indexOf` returns its index; otherwise, it returns -1.
2. **Using lastIndexOf**: This method uses `array.lastIndexOf(item) != index` to check if an item exists in the array. Like `indexOf`, it returns its index if found or -1 if not found.
3. **Using a Set**: This method uses `[...new Set(array)]` to create a new set from the original array. Since sets only allow unique values, this approach removes duplicates.

**Comparison**

The benchmark compares the performance of these three approaches:

* Using `indexOf` and `lastIndexOf` are similar in that they both iterate over the array to find the index of each item.
* Using a Set is different because it uses a hash table under the hood, which provides faster lookup times but may require more memory.

**Pros and Cons**

1. **Using indexOf**: This approach has good performance for small arrays since `indexOf` can stop iterating as soon as it finds a match. However, its performance degrades significantly with large arrays due to the linear search.
2. **Using lastIndexOf**: Similar to `indexOf`, this approach has similar strengths and weaknesses. Its performance is better than `indexOf` for very large arrays since it can take advantage of the array's internal indexing.
3. **Using a Set**: This approach provides excellent performance for removing duplicates, especially when working with large datasets. However, it requires more memory since it needs to store all unique elements.

**Library and Special JS Features**

In this benchmark, no external library is used. The JavaScript engine itself handles the processing of arrays and sets.

There are no special JavaScript features or syntax being tested in this benchmark. All standard features are being used.

**Other Alternatives**

Some alternative approaches to removing duplicates from an array include:

* Using `filter()` with a predicate function that checks for uniqueness, e.g., `array.filter((item, index) => array.indexOf(item) === index)`.
* Using a custom implementation using bitwise operations or other algorithms.
* Using external libraries like Lodash's `uniq` function.

Keep in mind that the choice of approach depends on the specific use case and requirements.

Related benchmarks:

Methods to remove duplicates from array (version: 0)

Comparing performance of: Using indexOf vs Using lastIndexOf vs Using a Set

Created: 3 years ago by: Guest

Jump to the latest result

Using indexOf

Using lastIndexOf

Using a Set

Suite status: <idle, ready to run>

Experimental features:

Fastest: N/A

Slowest: N/A

Autogenerated LLM Summary (model llama3.2:3b, generated 2 months ago):