Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - POP (Performance Optimisation and Productivity)

Teaser

High performance Computing is becoming a fundamental tool for the progress of science and engineering and, as such, is becoming more and more important for increasing economic competitiveness. The growing complexity of parallel computers is leading to a situation where...

Summary

High performance Computing is becoming a fundamental tool for the progress of science and engineering and, as such, is becoming more and more important for increasing economic competitiveness. The growing complexity of parallel computers is leading to a situation where developers and users do not fully understand the underlying issues affecting the performance of their applications which often results in an inefficient use of infrastructure. Even when application developers are aware of the need to improve the performance and efficiency of their own code, they often do not have sufficient insight as to the root causes of their bottlenecks to be able to address them properly. This may lead to blind attempts to restructure code resulting in an overall lack of efficiency. The objective of POP (Performance Optimisation and Productivity) Centre of Excellence in Computing Applications is to help developers and subsequently businesses address their performance needs so that they can ultimately become more competitive POP is based on the premise that machines will become more and more complex as we approach exascale. The complexity, scale and variability in behavior for current HPC systems already exposes an important distinction between how developers and users think that applications and systems behave and how they actually behave.

POP offers a service to asses performance of a wide range of computing applications in a wide range of configurations from a few dozen processors to many thousands of processors. POP provides its Customers with detailed analysis of the issues affecting the performance of their code as well as suggests the most optimal way to alleviate these issues. POP targets application developers and users from all domains, including infrastructure operators, academic and industrial users. The estimated number of these applications in Europe is projected at 1500 codes. POP has the ambition of analyzing over 150 of these types of codes within the project lifetime.

However, POP does not stop at the service. POP is working to generate and expand the market. In our target customers (the communities of code developers, users and resource operators) POP aims to provide a recognized high value service that will save on developer effort and make the most of available resources while minimising cost. Based on the services from POP, application developers will make better-informed decisions to drive their work in the more productive direction. Infrastructure operators will ensure that their systems are provisioned in the most efficient way to maximise their Return on Investment. End-users (both academic and industrial) will stop wasting money and time by using applications operating below their optimal levels. The POP project also aims at establishing a link among those customers, promoting networking activities and ensuring that the POP users (customers) are offered the mechanisms to communicate among them as well as perform a key driving role in the activities of the Centre.

POP is one of the nine CoEs that was selected under the e-Infrastructure call (H2020-EINFRA-2014-2015/H2020-EINFRA-2015-1). POP started on 1 October 2015 and will run until 31 March 2018. The project has received EUR 4 million funding from the European Union‘s Horizon 2020 research and innovation programme.

For more information, visit the project website: https://pop-coe.eu/

Work performed

In the first months of the project, we focused on to defining and implementing the procedures required to manage the Center of Excellence. The organization of the Center of Excellence has been established and successfully validated tough the first year.

During the first twelve months, our main focus in the POP Community Development was on identifying and attracting potential POP users and maintaining and managing the community of actual and potential POP users. Initially all partners used their personal contacts to identify early users of the POP service. Additionally, we have advertised POP on a number of mailing lists, and identified potential customers by web searches, researching conference attendees, identifying software authors etc. However, our main focus has been on advertising POP at events where we can meet potential users face-to-face. In the first year we attended a total of 17 events. We presented POP at the majority of these events through one or more presentations or posters. Events have been identified because either they have a cross-sectoral focus on HPC (e.g. PRACE Days, International Supercomputing, TERATEC Forum), or they focus on an application area where HPC is heavily used (e.g. NAFEMS UK for CAE, Mobile World Congress for the mobile telecoms industry, a UK workshop on materials science etc.). This approach has given us good exposure to academic researchers and larger companies but it is harder to encounter SMEs at this sort of event. Up to now we have 185 potential leads contacted and 54 users that already requested a POP service. We note that so far 21% of our studies have been on code that is distributed commercially. Going forward we intend to continue with this general approach. However, we are also going to make a special effort to identify and contact ISVs, especially if they are SMEs.

We want users and potential users of POP to feel that they are part of a community rather than just customers of a service. To this end, we’ve launched a regular newsletter and have worked closely with WP7 to ensure a steady supply of content for the blog and social media. We have launched the POP Helpdesk to provide longer-term support for previous users who may have questions about services that they have previously received.

Additionally, we have begun to consider approaches to develop a strategy to ensure that POP is self- sustainable after the project ends, even though this work was not originally scheduled in the first project period. While we have yet to come to any definitive conclusions we have identified a number of possible models (e.g. subscription or consultancy model), each with their pros and cons. The choices will be set out in the next project period.

During the first year of POP, we received 70 service requests and by early November 2016 40 services have been completed. Most of them correspond to the basic performance Audit where the code is diagnosed with respect to the efficiency achieved (see Figure 1 below).

We are implementing the Proof-of-Concept (PoC) service activities to demonstrate to users how the optimization techniques suggested in WP4 can be applied. In particular, this includes complex code refactoring or applying advanced parallelisation techniques that can be performed on the source code or on kernels extracted from it or mock-up mini-applications.

We’ve created 5 questionnaires to collect the feedback on the services; one questionnaire for each of the three types of POP services (one for each type of service: Performance Audit, Performance Plan, and Proof-of-Concept), one questionnaire regarding Performance Tools, and one questionnaire to evaluate our suggestions for performance improvements with the aim or measuring global return on investment. We have sent the appropriate questionnaires to all customers who benefitted from a POP service as of the end of September 2016: 31 Perf romance Audit, 1 Performance Plan, and 3 Proof-of-Concept. Additionally, we’ve conduct

Final results

The first results of the POP performance analysis services are promising. All audits have provided useful suggestions for performance improvements to the clients. The three closed PoCs resulted in improvements in production runs, in a near ideal speedup of the concerned parts of the code and in much better scalability. So far all POP customers were very satisfied or satisfied with the provided services according to the satisfaction questionnaires and interviews.

POP identified 185 potential users, received 70 requests from 54 users in the first year of the project and completed 40 services by early November 2016.

Short-term project impact concerning increased code performance in terms of speed and quality or better return on investments (ROI) can be measured at a later stage of the project once modified codes were running in production for a given period of time.

The long term potential impacts of the project are listed in Table 1 below.

Website & more info

More info: http://pop-coe.eu/.