Master/Slave assignment optimization for high performance computing in an EC2 cloud using MPI

Florian Schatz, Sven Koschnicke, Niklas Paulsen, Manfred Schimmler

Abstract


For high performance computing, cloud services offer highly scaleable infrastructures on demand. Without requiring a great deal of maintenance and financial resources, which a datacenter would need, it is possible to obtain a huge set of computing instances from, e.g., the Amazon Elastic Computing Cloud (Amazon EC2). The downsides of cloud computing are that the set of computing instances is arbitrary and that communication speed varies greartly and affects the running time of algorithms.
We present a master/slave selection algorithm for the EC2 cloud service based on a benchmark to counteract this disadvantage. The benchmark measures communication speed and the variance of communication in a NxN network consisting of N cloud instances. With the results of the benchmark, we perform a master/slave selection algorithm to maximize the number of messages between master and slaves. The results show that our proposed method greatly increases efficiency.

Keywords


Master/Slave selection;High Performance Computing; Cloud Computing; MPI; Communication Benchmark

Full Text:

PDF


DOI: http://dx.doi.org/10.5296/npa.v4i1.1290

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.

To make sure that you can receive messages from us, please add the 'macrothink.org' domain to your e-mail 'safe list'. If you do not receive e-mail in your 'inbox', check your 'bulk mail' or 'junk mail' folders.

Copyright © Macrothink Institute ISSN 1943-3581