Skip to content
Surf Wiki
Save to docs
technology/algorithms

From Surf Wiki (app.surf) — the open knowledge base

Sorting algorithm

Algorithm that arranges lists in order

Sorting algorithm

Summary

Algorithm that arranges lists in order

[[Merge sort

In computer science, a sorting algorithm is an algorithm that puts elements of a list into an order. The most frequently used orders are numerical order and lexicographical order, and either ascending or descending. Efficient sorting is important for optimizing the efficiency of other algorithms (such as search and merge algorithms) that require input data to be in sorted lists. Sorting is also often useful for canonicalizing data and for producing human-readable output.

Formally, the output of any sorting algorithm must satisfy two conditions:

  1. The output is in monotonic order (each element is no smaller/larger than the previous element, according to the required order).
  2. The output is a permutation (a reordering, yet retaining all of the original elements) of the input.

Although some algorithms are designed for sequential access, the highest-performing algorithms assume data is stored in a data structure which allows random access.

History and concepts

From the beginning of computing, the sorting problem has attracted a great deal of research, perhaps due to the complexity of solving it efficiently despite its simple, familiar statement. Among the authors of early sorting algorithms around 1951 was Betty Holberton, who worked on ENIAC and UNIVAC. Bubble sort was analyzed as early as 1956. Asymptotically optimal algorithms have been known since the mid-20th century new algorithms are still being invented, with the widely used Timsort dating to 2002, and the library sort being first published in 2006.

Comparison sorting algorithms have a fundamental requirement of n log n - 1.4427n + O(log n) comparisons. Algorithms not based on comparisons, such as counting sort, can have better performance.

Sorting algorithms are prevalent in introductory computer science classes, where the abundance of algorithms for the problem provides a gentle introduction to a variety of core algorithm concepts, such as big O notation, divide-and-conquer algorithms, data structures such as heaps and binary trees, randomized algorithms, best, worst and average case analysis, time–space tradeoffs, and upper and lower bounds.

Sorting small arrays optimally (in the fewest comparisons and swaps) or fast (i.e. taking into account machine-specific details) is still an open research problem, with solutions only known for very small arrays (

Classification ==

Sorting algorithms can be classified by:

  • Computational complexity
    • Best, worst and average case behavior in terms of the size of the list. For typical serial sorting algorithms, good behavior is O(n log n), with parallel sort in O(log2 n), and bad behavior is O(n2). Ideal behavior for a serial sort is O(n), but this is not possible in the average case. Optimal parallel sorting is O(log n).
    • Swaps for "in-place" algorithms.
  • Memory usage (and use of other computer resources). In particular, some sorting algorithms are "in-place". Strictly, an in-place sort needs only O(1) memory beyond the items being sorted; sometimes O(log n) additional memory is considered "in-place".
  • Recursion: Some algorithms are either typically recursive or typically non-recursive, while others may typically be both (e.g., merge sort).
  • Stability: stable sorting algorithms maintain the relative order of records with equal keys (i.e., values).
  • Whether or not they are a comparison sort. A comparison sort examines the data only by comparing two elements with a comparison operator.
  • General method: insertion, exchange, selection, merging, etc. Exchange sorts include bubble sort and quicksort. Selection sorts include cycle sort and heapsort.
  • Whether the algorithm is serial or parallel. The remainder of this discussion almost exclusively concentrates on serial algorithms and assumes serial operation.
  • Adaptability: Whether or not the presortedness of the input affects the running time. Algorithms that take this into account are known to be adaptive.
  • Online: An algorithm such as Insertion Sort that is online can sort a constant stream of input.

Stability

An example of stable sort on playing cards. When the cards are sorted by rank with a stable sort, the two 5s must remain in the same order in the sorted output that they were originally in. When they are sorted with a non-stable sort, the 5s may end up in the opposite order in the sorted output.

Stable sorting algorithms sort equal elements in the same order that they appear in the input. For example, in the card sorting example to the right, the cards are being sorted by their rank, and their suit is being ignored. This allows the possibility of multiple different correctly sorted versions of the original list. Stable sorting algorithms choose one of these, according to the following rule: if two items compare as equal (like the two 5 cards), then their relative order will be preserved, i.e. if one comes before the other in the input, it will come before the other in the output.

Stability is important to preserve order over multiple sorts on the same data set. For example, say that student records consisting of name and class section are sorted dynamically, first by name, then by class section. If a stable sorting algorithm is used in both cases, the sort-by-class-section operation will not change the name order; with an unstable sort, it could be that sorting by section shuffles the name order, resulting in a nonalphabetical list of students.

More formally, the data being sorted can be represented as a record or tuple of values, and the part of the data that is used for sorting is called the key. In the card example, cards are represented as a record (rank, suit), and the key is the rank. A sorting algorithm is stable if whenever there are two records R and S with the same key, and R appears before S in the original list, then R will always appear before S in the sorted list.

When equal elements are indistinguishable, such as with integers, or more generally, any data where the entire element is the key, stability is not an issue. Stability is also not an issue if all keys are different.

Unstable sorting algorithms can be specially implemented to be stable. One way of doing this is to artificially extend the key comparison so that comparisons between two objects with otherwise equal keys are decided using the order of the entries in the original input list as a tie-breaker. Remembering this order, however, may require additional time and space.

One application for stable sorting algorithms is sorting a list using a primary and secondary key. For example, suppose we wish to sort a hand of cards such that the suits are in the order clubs (♣), diamonds (♦), hearts (♥), spades (♠), and within each suit, the cards are sorted by rank. This can be done by first sorting the cards by rank (using any sort), and then doing a stable sort by suit:

Within each suit, the stable sort preserves the ordering by rank that was already done. This idea can be extended to any number of keys and is utilised by radix sort. The same effect can be achieved with an unstable sort by using a lexicographic key comparison, which, e.g., compares first by suit, and then compares by rank if the suits are the same.

Comparison of algorithms

This analysis assumes that the length of each key is constant and that all comparisons, swaps and other operations can proceed in constant time.

Legend:

  • n is the number of records to be sorted.
  • Comparison column has the following ranking classifications: "Best", "Average" and "Worst" if the time complexity is given for each case.
  • "Memory" denotes the amount of additional storage required by the algorithm.
  • The run times and the memory requirements listed are inside big O notation, hence the base of the logarithms does not matter.
  • The notation log2 n means (log n)2.

Comparison sorts

Below is a table of comparison sorts. Mathematical analysis demonstrates a comparison sort cannot perform better than O(n log n) on average.

NameBestAverageWorstMemoryStableIn-placeMethodOther notes
HeapsortNoYesSelectionAn optimized version of selection sort. Performs selection sort by constructing and maintaining a max heap to find the maximum in O(\log n) time.
IntrosortNoYesPartitioning & SelectionUsed in several STL implementations. Performs a combination of Quicksort, Heapsort, and Insertion sort.
Merge sortYesNoMergingHighly parallelizable (up to O(log n) using the Three Hungarians' Algorithm).
In-Place Merge SortYesYesMergingVariation of Mergesort which uses an O(n \log n) in-place stable merge algorithm, such as rotate merge or symmerge.
Tournament sortYesNoSelectionAn optimization of Selection Sort, which uses a tournament tree to select the min/max.
Tree sortYesNoInsertionWhen using a self-balancing binary search tree.
Block sortYesYesInsertion & MergingCombine a block-based in-place merge algorithm with a bottom-up merge sort.
SmoothsortNoYesSelectionAdaptive variant of heapsort based on the Leonardo sequence instead of a binary heap.
TimsortYesNoInsertion & MergingMakes n-1 comparisons when the data is already sorted or reverse sorted.
Patience sortingNoNoInsertion & SelectionFinds all the longest increasing subsequences in O(n log n).
CubesortYesNoInsertionMakes n-1 comparisons when the data is already sorted or reverse sorted.
QuicksortNoYesPartitioningQuicksort can be done in-place with O(log n) stack space.
FluxsortYesNoPartitioning & MergingAn adaptive branchless stable introsort.
CrumsortNoYesPartitioning & MergingAn in-place, but unstable variant of Fluxsort.
Library sortNoNoInsertionSimilar to a gapped insertion sort.
ShellsortNoYesInsertionSmall code size. Complexity is influenced by the gap sequence used. Pratt's sequence is worst case \Theta(n \log^2 n) which is the best known. Tight bounds for the average and worst case remain open problems.
Comb sortNoYesExchangingFaster than bubble sort on average.
Insertion sortYesYesInsertionO(n + d), in the worst case over sequences that have d inversions.
Bubble sortYesYesExchangingTiny code size.
Cocktail shaker sortYesYesExchangingA bi-directional variant of Bubblesort.
Gnome sortYesYesExchangingTiny code size.
Odd–even sortYesYesExchangingCan be run on parallel processors easily.
Strand sortYesNoSelection
Selection sortNoYesSelectionTiny code size. Noted for its simplicity and small number of element moves. Makes exactly n-1 swaps.
Exchange sortNoYesExchangingTiny code size.
Cycle sortNoYesSelectionIn-place with theoretically optimal number of writes.

Non-comparison sorts

The following table describes integer sorting algorithms and other sorting algorithms that are not comparison sorts. These algorithms are not limited to Ω(n log n) unless meet unit-cost random-access machine model as described below.

  • Complexities below assume n items to be sorted, with keys of size k, digit size d, and r the range of numbers to be sorted.
  • Many of them are based on the assumption that the key size is large enough that all entries have unique key values, and hence that n ≪ 2k, where ≪ means "much less than".
  • In the unit-cost random-access machine model, algorithms with running time of n \cdot \frac{k}{d}, such as radix sort, still take time proportional to Θ(n log n), because n is limited to be not more than 2^\frac{k}{d}, and a larger number of elements to sort would require a bigger k in order to store them in the memory.
NameBestAverageWorstMemoryStablen ≪ 2kNotes
Pigeonhole sortn + 2^kn + 2^k2^kCannot sort non-integers.
Bucket sort (uniform keys)n+kn^2 \cdot kn \cdot kAssumes uniform distribution of elements from the domain in the array.
Bucket sort (integer keys)n+rn+rn+rIf r is , then average time complexity is .{{cite book
Counting sortn+rn+rn+rIf r is , then average time complexity is .
LSD Radix Sortn \cdot \frac{k}{d}n \cdot \frac{k}{d}n \cdot \frac{k}{d}n + 2^d\frac{k}{d} recursion levels, 2d for count array.
MSD Radix Sortnn \cdot \frac{k}{d}n \cdot \frac{k}{d}n + 2^dStable version uses an external array of size n to hold all of the bins.
MSD Radix Sort (in-place)nn \cdot \frac{k}{1}n \cdot \frac{k}{1}2^1d=1 for in-place, k/1 recursion levels, no count array.
Spreadsortnn \cdot \frac{k}{d}n \cdot \left( {\frac{k}{s} + d} \right)\frac{k}{d} \cdot 2^dAsymptotic are based on the assumption that n ≪ 2k, but the algorithm does not require this.
Burstsortn \cdot \frac{k}{d}n \cdot \frac{k}{d}n \cdot \frac{k}{d}Has better constant factor than radix sort for sorting strings. Though relies somewhat on specifics of commonly encountered strings.
Flashsortnn+rn^2nRequires uniform distribution of elements from the domain in the array to run in linear time. If distribution is extremely skewed then it can go quadratic if underlying sort is quadratic (it is usually an insertion sort). In-place version is not stable.
Postman sortn \cdot \frac{k}{d}n \cdot \frac{k}{d}n+2^dA variation of bucket sort, which works very similarly to MSD Radix Sort. Specific to post service needs.
Recombinant sortHashing, Counting, Dynamic Programming, Multidimensional data

Samplesort can be used to parallelize any of the non-comparison sorts, by efficiently distributing data into several buckets and then passing down sorting to several processors, with no need to merge as buckets are already sorted between each other.

Others

Some algorithms are slow compared to those discussed above, such as the bogosort with unbounded run time and the stooge sort which has O(n2.7) run time. These sorts are usually described for educational purposes to demonstrate how the run time of algorithms is estimated. The following table describes some sorting algorithms that are impractical for real-life use in traditional software contexts due to extremely poor performance or specialized hardware requirements.

NameBestAverageWorstMemoryStableComparisonOther notes
Bead sortWorks only with positive integers. Requires specialized hardware for it to run in guaranteed time. There is a possibility for software implementation, but running time will be , where S is the sum of all integers to be sorted; in the case of small integers, it can be considered to be linear.
Merge-insertion sortMakes very few comparisons worst case compared to other sorting algorithms.
"I Can't Believe It Can Sort"Notable primarily for appearing to be an erroneous implementation of either Insertion Sort or Exchange Sort.
Spaghetti (Poll) sortPollingThis is a linear-time, analog algorithm for sorting a sequence of items, requiring O(n) stack space, and the sort is stable. This requires n parallel processors. See .
Sorting network(stable sorting networks require more comparisons)Order of comparisons are set in advance based on a fixed network size.
Bitonic sorterAn effective variation of Sorting networks.
BogosortRandom shuffling. Used for example purposes only, as even the expected best-case runtime is awful.{{citation
LinearSortParody sorting algorithm to show the risk of overly relying on Big O notation - runs a merge sort and then sleeps until a fixed constant amount of time has elapsed from the function call, thus (wastefully) guaranteeing a fixed runtime below the hardcoded minimum.
Stooge sort}}}Slower than most of the sorting algorithms (even naive ones) with a time complexity of Can be made stable, and is also a sorting network.
SlowsortA multiply and surrender algorithm, antonymous with divide-and-conquer algorithm.
Franceschini's method{{Cite journaldoi = 10.1007/s00224-006-1311-1title = Sorting Stably, in Place, with O(n log n) Comparisons and O(n) Movesjournal = Theory of Computing Systemsvolume = 40issue = 4pages = 327–353last1 = Franceschinifirst1 = G. }}

Theoretical computer scientists have invented other sorting algorithms that provide better than O(n log n) time complexity assuming certain constraints, including:

  • Thorup's algorithm, a randomized integer sorting algorithm, taking O(n log log n) time and O(n) space.
  • AHNR algorithm,{{Cite conference | book-title = Proceedings of the twenty-seventh annual ACM symposium on Theory of computing
  • A randomized integer sorting algorithm taking O\left(n \sqrt{\log \log n}\right) expected time and O(n) space.

Memory usage patterns and index sorting

When the size of the array to be sorted approaches or exceeds the available primary memory, so that (much slower) disk or swap space must be employed, the memory usage pattern of a sorting algorithm becomes important, and an algorithm that might have been fairly efficient when the array fit easily in RAM may become impractical. In this scenario, the total number of comparisons becomes (relatively) less important, and the number of times sections of memory must be copied or swapped to and from the disk can dominate the performance characteristics of an algorithm. Thus, the number of passes and the localization of comparisons can be more important than the raw number of comparisons, since comparisons of nearby elements to one another happen at system bus speed (or, with caching, even at CPU speed), which, compared to disk speed, is virtually instantaneous.

For example, the popular recursive quicksort algorithm provides quite reasonable performance with adequate RAM, but due to the recursive way that it copies portions of the array it becomes much less practical when the array does not fit in RAM, because it may cause a number of slow copy or move operations to and from disk. In that scenario, another algorithm may be preferable even if it requires more total comparisons.

One way to work around this problem, which works well when complex records (such as in a relational database) are being sorted by a relatively small key field, is to create an index into the array and then sort the index, rather than the entire array. (A sorted version of the entire array can then be produced with one pass, reading from the index, but often even that is unnecessary, as having the sorted index is adequate.) Because the index is much smaller than the entire array, it may fit easily in memory where the entire array would not, effectively eliminating the disk-swapping problem. This procedure is sometimes called "tag sort".

Another technique for overcoming the memory-size problem is using external sorting, for example, one of the ways is to combine two algorithms in a way that takes advantage of the strength of each to improve overall performance. For instance, the array might be subdivided into chunks of a size that will fit in RAM, the contents of each chunk sorted using an efficient algorithm (such as quicksort), and the results merged using a k-way merge similar to that used in merge sort. This is faster than performing either merge sort or quicksort over the entire list.

Techniques can also be combined. For sorting very large sets of data that vastly exceed system memory, even the index may need to be sorted using an algorithm or combination of algorithms designed to perform reasonably with virtual memory, i.e., to reduce the amount of swapping required.

References

References

  1. (2013-10-13). "Meet the 'Refrigerator Ladies' Who Programmed the ENIAC".
  2. (Dec 17, 2001). "Frances E. Holberton, 84, Early Computer Programmer". NYTimes.
  3. Demuth, Howard B.. (1956). "Electronic Data Sorting". Stanford University.
  4. (2009). "Introduction To Algorithms". The MIT Press.
  5. (1983). "An {{math". Proceedings of the fifteenth annual ACM symposium on Theory of computing.
  6. (2008). "Ratio Based Stable In-Place Merging". Theory and Applications of Models of Computation.
  7. Sedgewick, Robert. (1 September 1998). "Algorithms In C: Fundamentals, Data Structures, Sorting, Searching, Parts 1-4". Pearson Education.
  8. (1978). "Implementing Quicksort programs". [[Comm. ACM]].
  9. (2001). "Introduction To Algorithms". The MIT Press.
  10. Nilsson, Stefan. (2000). "The Fastest Sorting Algorithm?". [[Dr. Dobb's]].
  11. {{Introduction to Algorithms
  12. (3 October 2021). "Is this the simplest (and most surprising) sorting algorithm ever?".
  13. (February 2002). "Randomized Sorting in O(n log log n) Time and Linear Space Using Addition, Shift, and Bit-wise Boolean Operations". Journal of Algorithms.
  14. (2002). "Integer sorting in O(n√(log log n)) expected time and linear space".
  15. Wirth, Niklaus. (1986). "Algorithms & Data Structures". Prentice-Hall.
  16. {{harvnb. Wirth. 1986
  17. {{harvnb. Wirth. 1986
  18. "Tim Peters's original description of timsort".
  19. "OpenJDK's TimSort.java".
  20. "sort – perldoc.perl.org".
  21. [http://java.sun.com/j2se/1.3/docs/api/java/util/Arrays.html#sort(java.lang.Object%5B%5D) Merge sort in Java 1.3], Sun. {{Webarchive. link. (2009-03-04)
  22. {{harvnb. Wirth. 1986
  23. {{harvnb. Wirth. 1986
  24. (2009). "Introduction to Algorithms". The MIT Press.
  25. (1997). "Introspective Sorting and Selection Algorithms". Software: Practice and Experience.
  26. {{harvnb. Wirth. 1986
  27. "Exchange Sort Algorithm".
  28. "Exchange Sort".
  29. "tag sort Definition from PC Magazine Encyclopedia".
  30. [[Donald Knuth]], ''[[The Art of Computer Programming]]'', Volume 3: ''Sorting and Searching'', Second Edition. Addison-Wesley, 1998, {{ISBN. 0-201-89685-0, Section 5.4: External Sorting, pp. 248–379.
  31. [[Ellis Horowitz]] and [[Sartaj Sahni]], ''Fundamentals of Data Structures'', H. Freeman & Co., {{ISBN. 0-7167-8042-9.
  32. (2023). "Sorting with Predictions".
Wikipedia Source

This article was imported from Wikipedia and is available under the Creative Commons Attribution-ShareAlike 4.0 License. Content has been adapted to SurfDoc format. Original contributors can be found on the article history page.

Want to explore this topic further?

Ask Mako anything about Sorting algorithm — get instant answers, deeper analysis, and related topics.

Research with Mako

Free with your Surf account

Content sourced from Wikipedia, available under CC BY-SA 4.0.

This content may have been generated or modified by AI. CloudSurf Software LLC is not responsible for the accuracy, completeness, or reliability of AI-generated content. Always verify important information from primary sources.

Report