Problem Solving: Maths for big-O notation

From Wikibooks, open books for an open world
Jump to navigation Jump to search

PAPER 1 - ⇑ Theory of computation ⇑

← Comparing algorithms Maths for Big-O Notation Order of complexity →

From the Specification : Big-O Notation
  • Linear time, polynomial time, exponential time.
  • Order of complexity.

Big O Notation (also known as Big-O Notation) is a mathematical way of describing the limiting behaviours of a function. In other words, it is a way of defining how efficient an algorithm is by how "fast" it will run.


[edit | edit source]

You can work out the time that an algorithm takes to run by timing it:

Dim timer As New Stopwatch()
For x = 1 to 1000000000
   'count to one billion!
' Get the elapsed time as a TimeSpan value. 
Dim el As TimeSpan = stopWatch.Elapsed

' Format and display the TimeSpan value. 
Dim formattedTime As String = String.Format("{0}:{1}:{2}.{3}", el.Hours, el.Minutes, el.Seconds, el.Milliseconds / 10)
Console.WriteLine( "Time Elapsed: " + formattedTime)
   Code Output
Time Elapsed: 0:0:02:3249

However, this isn't always suitable. What happens if you run some code on a 33 MHz processor, and the same code on a 3.4 GHz processor? Timing a function tells you a lot about the speed of a computer, and very little about the speed of an algorithm.

Refining algorithms

[edit | edit source]

We might want to write a program to calculate the sum of all the numbers between 0 and a variable N, where N in this case == 7483647. To solve this, you might write a solution that has a loop that cycles through each number all the way up to N, adding it to another variable sum. The code might look a little like thisː

dim N as integer = 7483647
dim sum as double= 0
for i = 1 to N
   sum = sum + i

It certainly works, but as N gets larger the program runs more slowly. In fact, there is a relationship between the value of N and the speed of the algorithm. If you were to double N, the algorithm would take twice as long and if you were to treble N, the code would take three times as long. We can describe the speed of this relationship using big O notationː . That is the speed of the algorithm is related to the number of items being processed, in our case N. This is known as linear time.

We can of course write this is a much faster wayː

dim N as integer = 7483647
dim sum as double =  N * (1 + N) / 2

This version of the algorithm gets the same result, but doesn't use a loop. It completes in the same amount of time whatever value of N you give it. We can describe this algorithm using big O notation as beingː . That is, it completes in a constant time.

Order of Complexity

[edit | edit source]
Notation Name Example
constant Determining if a number is even or odd; using a constant-size lookup table
logarithmic Finding an item in a sorted array with a binary search or a balanced search tree as well as all operations in a Binomial heap.
linear Finding an item in an unsorted list or a malformed tree (worst case) or in an unsorted array; Adding two n-bit integers by ripple carry.
linearithmic, loglinear, or quasilinear Performing a Fast Fourier transform; heapsort, quicksort (best and average case), or merge sort
quadratic Multiplying two n-digit numbers by a simple algorithm; bubble sort (worst case or naive implementation), Shell sort, quicksort (worst case), selection sort or insertion sort
polynomial or algebraic Tree-adjoining grammar parsing; maximum matching for bipartite graphs
exponential Finding the (exact) solution to the travelling salesman problem using dynamic programming; determining if two logical statements are equivalent using brute-force search
factorial Solving the travelling salesman problem via brute-force search; generating all unrestricted permutations of a poset; finding the determinant with expansion by minors.