Distributed computing has become very common in large scientific communities because of the needs of exchanging scientific data and sharing computing resources which would otherwise not be affordable by a single research institute. Cloud computing brings a number of advantages (cost being one) that now makes it a platform of choice for many research activities.
Distributed computing (grid vs. cloud computing)
Distributed computing started with the widespread adoption of computing networks in the scientific community already in the 70’s. Since then it has evolved and assumed many different identities, from remote job submission to mainframes, to meta-computing, to clusters of commodity computers and servers, to distributed clusters, peer-to-peer computing, grid computing and now cloud computing.
The fundamental difference between grid computing and cloud computing is that grids are built from heterogeneous computing elements, often small, medium size commodity clusters and departmental computer centres and several layers of middleware are needed to create a virtual distributed computer centre. Cloud computing is instead provided on a pay per use basis by a limited number of very large and homogeneous data centres with virtualized services.
While grid infrastructures have captured the requirements of several specific scientific communities, smaller groups and many domains with significant application potential have struggled in terms of getting their requirements met. Another issue is the need to ensure sustainability and continued usage beyond funding at EU or national level. R&D projects at EU level typically last for two to three years. When the project ends, the machines might be left idle, at best they may store data of value to scientists. Industry adoption of grid has not taken off as widely as expected and grid computing has, to date, failed to come up with a convincing business model.
By contrast, cloud computing is gaining fertile ground, particularly in commercial settings and these belt-tightening times. Cloud computing delivers resources when and as needed so businesses and researchers can really focus on their core activities. The more convincing business model offered by cloud computing could be extended to include virtualized forms of payment and service tokens, and in exchange of services, researchers could be requested to share their data in open access, thus benefiting the wider community, as recently recommended by a high level expert group on scientific data in their report to the European Commission. There is a real opportunity to fulfill the potential of cloud computing and facilitate people who don’t need a complex infrastructure but do need more than their PC to speed up the time it takes to do the number crunching.
These were among the driving forces behind our proposal for a project to develop a cloud computing infrastructure for researchers and enterprises at EU level. This ultimately led to the funding of VENUS-C (Virtual Multidisciplinary Environments Using Cloud Infrastructures), which started on 1 June 2010 with a budget of €8.8m: €4.5m from the European Commission for two years, with Microsoft providing access to its cloud computing platform, Windows Azure, free of charge for three.
The project’s value-add lies in connecting infrastructure experts (like Microsoft and its European innovation centres; Italian software company and project coordinator Engineering; Barcelona Supercomputing Center; Sweden’s Royal Institute of Technology) with user communities to develop, test and validate a “scientific cloud” for Europe.
The project kicked off with a compelling set of user scenarios spanning seven applications across four thematic areas: biomedicine, civil engineering, civil protection and emergencies and data for science with a focus on marine biodiversity. An open call will broaden the scope of VENUS-C by bringing on board up to twenty additional applications.
It is unlikely that cloud computing can play a part in every facet of scientific research that requires some form of distributed computing. Nor will clouds replace grids across the board, at least in the near term. Where cloud computing in general and VENUS-C in the European landscape in particular will make a difference is in pushing its potential to tackle real-world challenges more quickly and more cost-effectively without the need for complex technical know-how and on-going support. Another important advantage of cloud computing is bound up with the pressing need to address energy costs. New environment-friendly data centres bring important opportunities both technically and economically.
VENUS-C brings an opportunity to leverage these economies of scale and transform the cost base in both scientific and commercial settings. This need for a new cost base may ultimately determine the future of grid. Scientific computing is already moving to the cloud. VENUS-C has a key role to play, now and in the future.