Being rolled out by the VENUS-C * project, the infrastructure marks a pioneering attempt to implement a user-centric approach to the cloud, putting the needs of end-user communities - such as researchers and SMEs - at the forefront of development, and providing scalable and interoperable cloud resources that combine both open source and commercial solutions to offer the best of both worlds.
'Most researchers have never had access to supercomputer networks and have relied on desktop resources. They are the "long tail" of science: they number in the thousands for each traditional supercomputer user,' explains Andrea Manieri, the VENUS-C project director from ICT group Engineering Ingegneria Informatica in Italy. 'Cloud computing empowers them in a number of different ways, enabling them not only to do better science by accelerating discovery but also new science they could not have done before.'
By providing access via the Internet to distributed computation, software, storage and other resources, cloud computing is becoming increasingly essential to many disciplines of modern science. It is especially crucial for data-intensive research, with shared cloud resources able to crunch out calculations in minutes that would take days or weeks on a desktop PC, or otherwise require extremely expensive supercomputing or high-performance computing systems.
Supported by EUR 4.5 million in funding from the European Commission, the VENUS-C team sought to harness these benefits in a more scalable, interoperable and easy-to-use way than existing commercial and open source systems. The resulting infrastructure integrates easily with users' working environments and provides on-demand access to cloud resources as and when needed.
Innovatively, their approach was guided by the requirements of end-users themselves: 27 teams of researchers from across Europe working in seven different scientific fields, from bioinformatics and drug discovery to civil engineering and civil protection; and 15 selected pilot projects that received seed money from VENUS-C following an open call that attracted 60 proposals from 17 countries. The pilot teams' cloud computing requirements steered the design of the infrastructure.
'Cloud computing is a fast moving field. Cloud standards are by and large still immature and the number of standards organisations working independently is large,' Mr. Manieri explains. 'Providing a common layer to access heterogeneous infrastructures has proved technically challenging. But as a result our approach to the interoperability layer tackles current challenges with our users firmly in mind.'
The VENUS-C team defined common ways of naming resources and properties so that the same information is interpreted in the same way, irrespective of the infrastructure, and they focused on three key areas of standards interoperability, including job submission leveraging established protocols (BES/JSDL) and cloud data storage (the so-called 'Cloud Data Management Interface').
Democratising science with real-world applications
''Our approach to "openness" has been to tie together user requirements so as to deliver them with solutions and services that are more efficient and cost effective in their everyday work, the sharing of best practices through seed fund allocation, and leveraging open standards developments,' explains Ignacio Blanquer from the Universidad Politecnica de Valencia in Spain, community manager at VENUS-C. 'The cloud has the potential to democratise science by providing powerful computing and data analysis to any researcher.'
For example, members of the project experimented with BLAST, a data-intensive tool used by biologists to find regions of local similarity in amino-acid sequences of different proteins or the nucleotides of DNA sequences. Using the VENUS-C infrastructure on Microsoft’s Windows Azure cloud platform, the experiment cost less than EUR 600 and took just one week to process data that would have taken one to two years on a single PC.
'The advantage of using VENUS-C BLAST compared with renting cloud resources and deploying high-performance computing or high-throughput versions of BLAST is that deployment efforts are minimised and client impact is also minimal, since users don’t have to log-in on a different machine,' Prof. Blanquer explains.
Researchers at the University of Newcastle in the United Kingdom are using VENUS-C infrastructure to provide cloud services on their 'e-Science Central' platform which also combines software as a service accessible through a simple web browser and social networking functionalities for user interaction.
'It illustrates two important characteristics of cloud-based scientific systems: spreading tasks over 300 Windows Azure cores with a higher than 90 % efficiency, and user input through a web service allowing multiple users to invoke the same instantiation of the service at the same time. This interactive model is far different from the traditional batch approach used in supercomputing facilities,' Prof. Blanquer notes.
In Greece, researchers at Aristotle University are spearheading an innovative use of the cloud for assessing and tracking trends in social media to help businesses and policymakers better understand the concerns and interests of citizens, while another team at the same university is using VENUS-C infrastructure for earthquake impact assessment.
'Our involvement in VENUS-C offers a prime opportunity to access unprecedented resources only when and where necessary for earthquake impact estimation and related information dissemination, without worrying about how to build and maintain the corresponding infrastructure and operational tools,' says Costas Papazachos, a geophysicist who is heading the pilot project.
Meanwhile at the University of the Aegean, Kostas Kalabokidis, a senior geography researcher, has adopted a pioneering approach to using cloud computing for predicting wildfires. The system uses weather, topography, vegetation data and digital images to predict the spread of forest fires, giving fire fighters a one-hour head start on the blaze. It also provides forecasts of the risk of fires up to five days in advance, integrating Microsoft Bing Maps, Silverlight and Windows Azure.
'While there are many fire risk algorithms around the world, our tool is different because it provides a quantitative and systematic approach, based on geographic information systems. It can predict fires at an hourly rate,' explains Prof. Kalabokidis.
Other researchers in Spain, Denmark, Italy and the UK are using VENUS-C infrastructure for complex data analysis in the bioinformatics and biomedicine domains, for civil engineering applications, for cosmological calculations and for biology to study the dynamic movement of cells at the molecular level.
Outside of academia, companies and SME partners are also reaping important benefits from the technology.
VENUS-C received research funding from the European Union’s Seventh Framework Programme (FP7). Project partner Microsoft is providing computing and human resources.
* 'Virtual multidisciplinary environments using cloud infrastructures'