Algorithm

Generally, an algorithm is a systematic list of instructions for accomplishing some task, and the task can be anything that has a recognizable end-point (or result). Often some of the specific steps in the procedure are to be repeated until the task is done. Normally, there are different algorithms for the same task, some better than others. A cooking recipe is one kind of algorithm. Some recipes for making potato salad, for example, have "peel the potato" before "boil the potato", while some have the "boil" step before the "peel" step, but they all call for those steps to be repeated for however many potatoes there are, and they all end when the potato salad is ready to eat.

Formalized algorithms

Algorithms are essential to the way computers process information, because a computer program is essentially an algorithm that tells the computer what specific steps to perform, in what specific order, to carry out a specific task, such as calculating the employees' paychecks or printing the students' report cards.

For any such computational process, the algorithm must be completely laid down: the way it applies in all possible circumstances that could arise must be specified. That is, any conditional steps must be systematically dealt with, case-by-case; the criteria for each case must be clear (and computable)

Because an algorithm is a precise list of precise steps, the order of operations will almost always be important. Instructions are usually assumed to be listed explicitly, and are described as starting 'from the top' and going 'down to the bottom', an idea that can be discussed more formally in terms of flow of control.

All the assumptions mentioned so far relate to ordinary ideas of systematic commands (the basis of imperative programming). They serve to pin down the idea of a 'mechanical' description of a task. Unique to formalized algorithms is the assignment operation, setting the value of a variable. See an example below. It derives from the intuition of 'memory' as a scratchpad.

Implementing algorithms

Once a formal description has been obtained, an algorithm is a well-defined method or procedure: for solving a problem, such as a problem in mathematics; or otherwise relating to the manipulation of information.

Algorithms are now most often implemented as computer programs but can also be implemented otherwise, for example as electric circuits or mechanically. They may be performed directly by humans: think for example of an abacus.

The general study of algorithms is a central part of computer science, going further than in programming languages that are designed for practical implementation. Some people restrict the definition of algorithm to procedures that eventually finish. One may also include procedures that could run forever without stopping, since a computer may be required to carry out an ongoing task.

Example

As an example of an algorithm, here is a simple one. Imagine you have a random, unsorted list of numbers. Our goal is to find the highest number in this list. First upon thinking about the solution, you will realize that you must look at every number in the list. Upon further thinking, you will realize that you need to look at each number only once. Taking this into account, here is a simple algorithm to accomplish this:

• Pretend the first number in the list is the largest number.
• Look at the next number, and compare it with this largest number
• Only if this next number is larger, then keep that as the new largest number
• Repeat steps 2 and 3 until you have gone through the whole list.

Most of the time, algorithms are written in computer code. Here is a more formal notation in a pseudocode that is similar to most programming languages:

	Given: a list _List_ of length _Length_ 
	 
	counter = 0
	largest = List[counter]
	while counter < Length:
	    if List[counter] > largest:
	        largest = List[counter]
	    counter = counter + 1
	print largest

As it happens, most people that implement algorithms want to know how much of a particular resource (such as time or storage) a given algorithm requires. Methods have been developed for the analysis of algorithms to obtain such quantitative answers, and after reading that section, you will determine that this algorithm has a time requirement of O n), where the big O notation was used and n stands for the length of the list.

History

The word algorithm is a corruption of early English algorisme, which came from Latin algorismus, which came from the name of an iranian scientist Abu Ja'far Mohammed ibn Musa al-Khwarizmi (ca. 780 - ca. 845). He was the author of the book Kitab al-jabr w'al-muqabala Rules of Restoration and Reduction) which introduced algebra to people in the West. The word algebra itself originates from al-Jabr from the book title. The word algorism originally referred only to the rules of performing arithmetic using Arabic numerals but evolved into algorithm by the 18th century. The word has now evolved to include all definite procedures for solving problems or performing tasks.

The lack of mathematical rigor in the "well-defined procedure" definition of algorithm posed some difficulties for mathematicians and logicians of the 19th and early 20th centuries. This problem was largely solved with the description of the Turing machine, an abstract model of a computer described by Alan Turing, and the demonstration that every method yet found for describing "well-defined procedures" advanced by other mathematicians could be emulated on a Turing machine (a statement known as the Church-Turing thesis).

Nowadays, a formal criterion for an algorithm is that it is a procedure implementable on a completely-specified Turing machine or one of the equivalent formalisms. Turing's initial interest was in the halting problem: deciding when an algorithm describes a terminating procedure. In practical terms computational complexity theory matters more: it includes the puzzling problem of the algorithms called NP-complete, which are generally presumed to take more than polynomial time.

Classes of algorithms

Algorithms come in different flavours. A recursive algorithm is one that calls itself repeatedly until certain condition matches, which is a central way in functional programming. A greedy algorithm works by making a series of simple decisions that are never reconsidered. A divide-and-conquer algorithm recursively reduces an instance of a problem to one or more smaller instances of the same problem, until the instances are small enough. A dynamic programming algorithm works bottom-up by building progressively larger solutions to subproblems arising from the original problem, and then uses those solutions to obtain the final result. Many problems (such as playing chess) can be modeled as problems on graphs. A graph exploration algorithm specifies rules for moving around a graph and is useful for such problems. Probabilistic algorithms are those that make some choices randomly. Algorithms are usually discussed with the assumption that computers execute each instruction of an algorithm at a time. Those computers are sometimes called serial computer and algorithm serial algorithm to make a distinction from parallel algorithms, which take advantage of computer architectures where several processors can work on a problem at the same time.

Genetic algorithms attempt to find solutions to problems by mimicking biological evolutionary processes, with a cycle of random mutations, reproduction and "survival of the fittest". In genetic programming, this approach is extended to evolve the algorithms themselves.

References

• Donald E Knuth: The Art of Computer Programming, Vol 1-3, Addison Wesley 1998. The standard reference.
• Gaston H. Gonnet and Ricardo Baeza-Yates: Example programs from Handbook of Algorithms and Data Structures, http://www.dcc.uchile.cl/~rbaeza/handbook/. Free source code for lots of important algorithms.
• Dictionary of Algorithms and Data Structures, http://www.nist.gov/dads/. "This is a dictionary of algorithms, algorithmic techniques, data structures, archetypical problems, and related definitions."

More on Algorithm

See also: Programming, Software