#	Pagina
attuale pagina	/open-h2020/projects/205931/results.html
-1	/open-h2020/per-topic/fo/list/index.html

Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - ELASTEST (ElasTest: an elastic platform for testing complex distributed large software systems)

Teaser

Summary

Large distributed software is promised to be fault-tolerant, elastic (it scales in and out) and cost-efficient, among other features. However, it is difficult to assess that distributed software complies with the quality demanded by users. For example, whenever developers want to validate non-functional features such as scalability, fault-tolerance or data consistency, they need to create complex testing architectures customizing the cloud orchestration mechanisms and to manage test scalability by themselves. Things become even more complex when trying to reproduce real-world operational conditions.

ElasTest proposes to apply the â€œdivide-and-conquerâ€ principle, which is commonly used for architecting complex software, to testing by developing a novel test orchestration theory and toolbox enabling the creation of complex test suites as the composition of simple testing units. This test orchestration mechanism is complemented with tools that include:

* Capabilities for collecting metrics (see Figure 1) and reproducing real-world operational conditions.
* Reusable testing services including Browsers, Emulation of Sensors and Actuators, Monitoring, Security Checks, Big Data Analysis, Cost Modeling, etc.
* Generating testing recommendations through cognitive computing and machine learning mechanisms.

All in all, ElasTest will make any software development team capable of delivering software faster, more confidently, more securely and with fewer defects.

Work performed

ElasTest has created an innovative cloud architecture for testing large distributed systems that follows some of the principles of microservices allowing us during the second period to make it elastic and to assess the quality of large software systems. At a very high-level perspective these are the main parts of such architecture:

* ElasTest Core Components: constitute the enablers of the platform and are responsible of providing management mechanisms for the platform, as well as managing test jobs and the software under test.
* ElasTest Test Support Services: reusable services providing different tools to testers. They are intended to be used from the test code itself and are provided as a service.
* ElasTest Test Engines: offer additional capabilities to help in the testing process that can be used by the platform itself or by testers.
* ElasTest integrations with external tools: specific integrations of ElasTest with external tools like continuous integration servers or test management tools.

The main contributions in this reporting period have been the following:

1) Architecture definition focused on the following principles: extensible, cloud-agnostic, and independent components with clear boundaries
2) State-of-the-art, both from an academic and industrial perspective:
2.1) From an academic perspective, a systematic literature review (reported in D2.2 [1]) of the concepts testing in the cloud, testing of the cloud and testing of the cloud in the cloud.
2.2) From an industrial perspective, scouting the market for potential tools that could overlap with ElasTest, or communities with whom we could partnership. As a result, a compendium of 415 tools has been built (reported in D2.2 [1]). To the best of our knowledge, thereâ€™s no tool with the goals that have been set for ElasTest. Most of the tools focus on a single testing problem.
3) First version of the ElasTest platform already available. In December 22nd, 2017 the first beta was made available. Several internal version were released afterwards while progressing towards a more mature platform, until May 2018, where we released our first non-beta version.
4) Continuous Integration environment. Since September 2017 the project has been working towards providing this Continuous Integration environment, and components have been developing end-to-end tests to help assess the quality of the platform. Indeed, one of the objectives of Milestone 5, testing ElasTest with ElasTest, due in M24, is almost complete.
5) Validation through demonstrators using the preliminary versions of the ElasTest platform. This preliminary experiment was useful for providing a feedback concerning both the platform functionality and the validation design, detecting bugs in the platform that were solved, and adding features that were needed to support one of the demonstrators.
6) Dissemination. The project took dissemination very seriously since the very beginning of the project. One of the objectives set out in the DoA is to build a community around the project that could sustain ElasTest beyond the end of the project. This is a hard task to achieve, hence the consortium decided to start early the dissemination activities. Since the beginning, 14 scientific contributions have been published, the partners attended more than 20 events, 19 press releases have been published, a collaboration with the H2020 STAMP project has been started, 21 blog entries, 495 tweets and 344 followers on Twitter, 22 videos on YouTube with a total of 867 views, 10 presentations in SlideShare with 1,713 views, and more than 5,000 visitors to the projectâ€™s webpage.

Final results

Progress beyond the state of the art is expected in the following areas:

1) ElasTest Monitoring Service (EMS), ElasTest Security Service (ESS), ElasTest Device Emulator Service (EDS) and ElasTest User Impersonation Service (EUS). These components that are going to provide advances over the state of the art on their respective research areas on the second half of the project.
2) Test orchestration: this concept revolves around the idea of building complex test suites from simple test cases. Based on conversations with companies that devote a lot of resources to testing test orchestration could be a disrupting feature saving a lot of effort devoted currently to testing.
3) Test recommendation. Being sure that the tests cover all the functional and non-functional aspects of the software can be difficult when managing a big set of test cases. Test recommenders provided by ElasTest intend to help testers and developers to identify those tests that are missing from the test cases and provide examples from previous projects.
4) Additional research:
4.1) CNR and URJC collaboratively built a systematic literature review of testing related to the cloud that was submitted to a high impact journal.
4.2) URJC, TUB and FOKUS presented a preliminary work on testing 5G infrastructures. Giving that most operators are migrating from hardware to software assets, testing activities might have a big impact on this area.

For the rest of the project we expect to push research forward on all the above mentioned areas, by means of high impact publications, and transferring the results to companies that could be interested in reducing testing costs and improving software quality with pilot projects. The project, since the beginning, is building a list of companies that at some point were interested in ElasTest and will reach them out when the research components mature and are ready to use.

The project is expected to have an economic impact on the cost of testing and testing-related activities, like troubleshooting (finding the root cause of a failed test). Current testing approaches avoid end-to-end testing as much as possible, due to the costs of implementing these end-to-end tests, and the infrastructure needed to run them. ElasTest allows even the most complex test scenarios to be developed or run with a fraction of the efforts.