Ing. Matěj Klíma, Ph.D.

In path-based testing, various test coverage criteria can be used to generate test paths from a system model. The question of the realistic economic effectiveness of these individual criteria deserves further investigation, as the answer strongly depends on the presence of defects in a system, topology of the model of a system under test, and other factors. This study presents an open benchmark testbed for measuring the effectiveness of test paths in detecting artificially introduced defects in a tested system. This framework offers a good level of scalability for various experiments in this field. To document its functionality and added value, an example use case comparing the effectiveness of test paths satisfying Edge, Edge-pair, Test Depth Level 3 and Prime Path coverage for the detection of 75 artificial defects is presented.

Contemporary Internet-of-Things (IoT) systems are hindered by several reliability-related issues, especially, the dynamic behavior of IoT systems caused by limited and often unstable network connectivity. Several intuitive ad-hoc approaches can be employed to test this behavior; however, the effectiveness of these approaches in detecting defects and their overall testing costs remain questionable. Therefore, we present a new specialized path-based technique to test the processes of an IoT system in scenarios wherein parts of these processes are influenced by limited or disrupted network connectivity. The proposed technique can be scaled using two levels of test coverage criteria to determine the strengths of the test cases. For this purpose, we propose two algorithms for generating test cases to implement the technique: an ant colony optimization-based search and a graph-traversal-based test case composition. We compared the efficiency of the proposed approach with possible solutions obtained using a standard path-based testing approach based on prime paths computed by a set-covering algorithm. We consider the total number of test case steps as the main proxy for test effort in experiments employing 150 problem models. For the less intensive of the two used test -coverage criteria, EachBorderOnce, an ant colony optimization-based algorithm, produced test sets with the same averaged number of steps as the graph traversal-based test-case composition; however, this algorithm performed with averaged number of steps 10% lower than a prime paths-based algorithm. For the more intensive test coverage criterion, AllBorderCombinations, these differences favoring the ant colony optimization-based algorithm were 18% and 25%, respectively. For these two types of defined test coverage criteria, the ant colony optimization -based search, graph-traversal-based algorithm, and standard path-based testing approach based on prime paths achieved the best results for 93 and 78, 14 and 24, and 13 and 17 models for AllBorderCombinations and EachBorderOnce criterion, respectively. Therefore, to guarantee the best test set, all compared algorithms are combined in a portfolio strategy that yields the best results based on the potential of the produced test sets to detect simulated defects caused by limited network connectivity. Additionally, this portfolio strategy also yields test sets, implying the lowest test effort for experimental problem instances.

A considerable number of patients with COVID-19 suffer from respiratory problems in the post-acute phase of the disease (the second-third month after disease onset). Individual telerehabilitation and telecoaching are viable, effective options for treating these patients. To treat patients individually, medical staff must have detailed knowledge of their physical activity and condition. A sensor network that utilizes medical-grade devices can be created to collect these data, but the price and availability of these devices might limit such a network's scalability to larger groups of patients. Hence, the use of low-cost commercial fitness wearables is an option worth exploring. This article presents the concept and technical infrastructure of such a telerehabilitation program that started in April 2021 in the Czech Republic. A pilot controlled study with 14 patients with COVID-19 indicated the program's potential to improve patients' physical activity, (85.7% of patients in telerehabilitation versus 41.9% educational group) and exercise tolerance (71.4% of patients in telerehabilitation versus 42.8% of the educational group). Regarding the accuracy of collected data, the used commercial wristband was compared with the medical-grade device in a separate test. Evaluating z-scores of the intensity of participants' physical activity in this test, the difference in data is not statistically significant at level p = 0.05. Hence, the used infrastructure can be considered sufficiently accurate for the telerehabilitation program examined in this study. The technical and medical aspects of the problem are discussed, as well as the technical details of the solution and the lessons learned, regarding using this approach to treat COVID-19 patients in the post-acute phase.

Software code is present on multiple levels within current Internet of Things (IoT) systems. The quality of this code impacts system reliability, safety, maintainability, and other quality aspects. In this paper, we provide a comprehensive overview of code quality-related metrics, specifically revised for the context of IoT systems. These metrics are divided into main code quality categories: Size, redundancy, complexity, coupling, unit test coverage and effectiveness, cohesion, code readability, security, and code heterogeneity. The metrics are then linked to selected general quality characteristics from the ISO/IEC 25010:2011 standard by their possible impact on the quality and reliability of an IoT system, the principal layer of the system, the code levels and the main phases of the project to which they are relevant. This analysis is followed by a discussion of code smells and their relation to the presented metrics. The overview presented in the paper is the result of a thorough analysis and discussion of the author's team with the involvement of external subject-matter experts in which a defined decision algorithm was followed. The primary result of the paper is an overview of the metrics accompanied by applicability notes related to the quality characteristics, the system layer, the level of the code, and the phase of the IoT project.

For Internet of Things (IoT) systems operating in areas with limited network connectivity, reliable and safe functionality must be ensured. This can be done using special test cases which are examining system behavior in cases of network outage and restoration. These test cases have to be optimal when approached from the testing effort viewpoint. When approached from the process viewpoint, in the sense that a business process supported by a tested system might be affected by a network outage and restoration, test cases can be automatically generated using a suitable model-based testing (MBT) technique. This technique is currently available in the open freeware Oxygen MBT tool. In this paper, we explain the principle of the technique, a process model of the tested system that may be affected by limited network connectivity, and support for this specialized MBT technique on the Oxygen platform.

With the rapid growth of the contemporary Internet of Things (IoT) market, the established systems raise a number of concerns regarding the reliability and the potential presence of critical integration defects. In this paper, we present a PatrIoT framework that aims to provide flexible support to construct an effective IoT system testbed to implement automated interoperability and integration testing. The framework allows scaling from a pure physical testbed to a simulated environment using a number of predefined modules and elements to simulate an IoT device or part of the tested infrastructure. PatrIoT also contains a set of reference example testbeds and several sets of example automated tests for a smart street use case.

Quality and reliability metrics play an important role in the evaluation of the state of a system during the development and testing phases, and serve as tools to optimize the testing process or to define the exit or acceptance criteria of the system. This study provides a consolidated view on the available quality and reliability metrics applicable to Internet of Things (IoT) systems, as no comprehensive study has provided such a view specific to these systems. The quality and reliability metrics categorized and discussed in this paper are divided into three categories: metrics assessing the quality of an IoT system or service, metrics for assessing the effectiveness of the testing process, and metrics that can be universally applied in both cases. In the discussion, recommendations of proper usage of discussed metrics in a testing process are then given.

The current development of the Internet of Things (IoT) technology poses significant challenges to researchers and industry practitioners. Among these challenges, security and reliability particularly deserve attention. In this paper, we provide a consolidated analysis of the root causes of these challenges, their relations, and their possible impacts on IoT systems’ general quality characteristics. Further understanding of these challenges is useful for IoT quality engineers when defining testing strategies for their systems and researchers to consider when discussing possible research directions. In this study, twenty specific features of current IoT systems are discussed, divided into five main categories: (1) Economic, managerial and organisational aspects, (2) Infrastructural challenges, (3) Security and privacy challenges, (4) Complexity challenges and (5) Interoperability problems.

The recent active development of Internet of Things (IoT) solutions in various domains has led to an increased demand for security, safety, and reliability of these systems. Security and data privacy are currently the most frequently discussed topics; however, other reliability aspects also need to be focused on to maintain smooth and safe operation of IoT systems. Until now, there has been no systematic mapping study dedicated to the topic of interoperability and integration testing of IoT systems specifically; therefore, we present such an overview in this study. We analyze 803 papers from four major primary databases and perform detailed assessment and quality check to find 115 relevant papers. In addition, recently published testing techniques and approaches are analyzed and classified; the challenges and limitations in the field are also identified and discussed. Research trends related to publication time, active researchers, and publication media are presented in this study. The results suggest that studies mainly focus only on general testing methods, which can be applied to integration and interoperability testing of IoT systems; thus, there are research opportunities to develop additional testing methods focused specifically on IoT systems, so that they are more effective in the IoT context.

Testing business processes and workflows in information systems, while aiming to cover all possible paths, requires high efforts demanding considerable costs. In this paper, we propose an algorithm generating a path-based test cases from the system model, based on weighted directed graph. The approach brings an alternative to the currently established test requirements concept. The algorithm reflects various levels of priorities of particular functions in the tested system, previously defined by the test designer. When compared to simulated naive approaches based on reverse reduction of test set, our proposed algorithm produces more efficient test cases in terms of number of the total test steps, whilst keeping the same level of test coverage of the priority functions of the tested system.