ISC or Intensive Scientific Computing needs several hardware and software technologies. For a given scientific application, the choice between the different possible technics to get the best performance in term of speed and precision depends on several factors :

- application dependent : parallelism ratio, grain, compute/communication ratio,...

- environment dependent : system or network of systems, interconnect, OS, available librairies...

Before analyzing the different solution classes, depending on the problem type and size, it is useful to mention several basics.

Firstly, Amdhal law, often used  for marketing purposes,  is critical  when attacking the optimization of a piece of software (optimization and not parallization, as the Amdhal law applies for any process composed of two parts running at different speeds, for instance serial and parallel, serial and vector, serial and parallel-vector...). The law can be  expressed in 2 ways, starting with the simpler one :

- 1 - for a program composed of two distinct parts, one running much slowly than the other, the influence of the slowest one can be dramatic for overall performance.

Far from being appliable only to computer science,  this is valid for any process having different execution speeds. As an example, a car trip will be dramatically affected in term of average speed by slow motion in the traffic jams during the trip.

This translates for parallelism (but would work for vector also) : 

- 2-  Speedup = 1 / (Ps + (1-Ps)/Np), with:  Ps = percentage execuing in scalar mode et Pp = percentage executing in parallel mode (1-Ps = Pp), and Np being the number of processors. This can also be expressed using the execution times on one (T1) or Np processors (Tnp):

Tnp  = T1 (Pp/Np +Ps)

 The direct (and painful !) consequence can be found in the following table :