Serviços customizados
Inglês (pdf)
Artigo em XML
Referências do artigo
Enviar este artigo por email
Citado por SciELO 
Similares em
SciELO 
Permalink
Introduction
Nowadays, there is a trend towards the use of mobile technology and smart devices and thousands of mobile applications are created every day worldwide (Beltran, 2017). These can be considered as software developed to be run on devices such as tablets, phones or smart watches that have a suitable operating system. However, studies show (Bluumi, 2018), (Mediavilla, 2019), (Rivero, 2018) that there are a large number of installed applications that are deleted from smart devices after a very short time after being downloaded.
This is because sometimes these applications do not respond to a specific need and do not solve the user's problem. The degree of user acceptance will depend on the particular characteristics that each user considers to be important. From the point of view of Software Engineering (SE), one of the main characteristics that an application must have to be successful among users is that it must be of high quality (Enriquez, y otros, 2016). It is therefore of vital importance to be able to measure this quality by testing the applications built.
Software quality is the degree to which the software product incorporates a set of features in a way that ensures compliance with customer requirements and needs (25000, 2001). The Cuban Standard (NC) ISO 25010: 2016 (ISO/IEC, 2011) lists the quality characteristics of a product and its sub-characteristics, which are to be taken into account when evaluating the properties of a given software product and establishes the system for product quality evaluation.
Among these features, one of the most important in the development of mobile applications is usability. This attribute is defined as the ease of use, whether of a web page, a computer application or any other system that interacts with a user. (Enriquez, y otros, 2016), (elásquez, y otros, 2018). However, in the analysis of the consulted literature (Mogollon, 2020), (Enriquez, y otros, 2016), (HINOJOSA, 2016), (SEMINARIO, 2020), it was identified that:
The quality of applications for mobile devices is usually poor, because the development processes that are carried out are very accelerated and this causes tests to be omitted or performed incorrectly, since they are considered too complex, difficult to automate, costly and time-consuming.
Standards and models are not yet defined in a consistent manner, such that they outline practical guidance on the attributes, metrics and rules needed to adequately measure the usability of mobile applications.
Currently, existing guidelines and methods were created for usability analysis of traditional desktop applications, which makes it difficult to adopt them for mobile application development.
Usability tests alone are not sufficient for the realization of a user-friendly system, as these tests do not solve interface design problems.
Measuring usability is a complex process in the field of mobile applications, because it requires techniques and methods designed specifically for each context.
Because the factors related to usability are diverse, these standards or models do not cover all aspects of usability. They are also not adequately integrated into current IS practices and generally lack the IT tools to support it (HINOJOSA, 2016).
In the Electronic Government Center (CEGEL) attached to the Faculty 3 of the University of Informatics Sciences (UCI) solutions oriented to e-Government are developed, adjusting to the specific needs of each client and promoting the generation of replicable models. (UCI, 2020).
In this center is the project LexCuba registered by the Ministry of Science, Technology and Environment (CITMA) which aims to bring the electronic government to a greater integration with the population, through mobile applications that make it possible to carry out all procedures and consult essential services without the need to visit the offices of the corresponding entities and/or ministries.
Once the projects are finalized by the development team, they are tested by the center's quality group. Within this software quality verification and validation process, in addition to functional tests, usability tests are performed to identify vulnerabilities that may be present.
When evaluating the usability of this type of solution, there is no checklist that establishes the guidelines to be taken into account for a correct evaluation and, therefore, it is tested empirically. In surveys applied to programmers of the LexCuba project and to members of the center's quality group, it was identified that out of a total of 15 respondents, the following were identified:
90% know what usability is and 100% assume that they do not formally evaluate it because they do not have any existing model or guide for it.
95% of people recognize the following as disadvantages of not taking usability into account in the development of mobile applications: delays in the product delivery schedule due to the resolution of non-conformities, complicated and inefficient designs, and dissatisfied users due to not taking into account the target audience.
Only 10% of respondents are aware that a guide to evaluate usability exists in the ICU, but they do not use it because it focuses more on web applications than mobile applications.
Only 30% of programmers apply usability evaluation techniques in the later stages of software development where usability problems become very costly and 70% do not apply it at all.
95% of respondents believe that usability should be tracked throughout the software development process, but in practice only 40% track it up to the analysis and design discipline and 60% do not consider it necessary.
Therefore, in order to solve the problem, the general objective was established: to develop a guide for the management of usability testing in mobile applications, in order to reduce the number of non-conformities during the testing process.
Computational Methods or Methodology
The guide for managing usability testing in mobile applications aims to evaluate the usability of mobile applications developed in the center CEGEL which will serve as a common thread to reduce the number of usability nonconformities detected by the center's quality group. For the development of this study, a series of attributes were identified as the basis for measuring the usability of mobile applications: effectiveness, efficiency, satisfaction, ease of learning, errors, content, portability, security and context. Since the attributes cannot be measured directly, it was necessary to associate different metrics with each other (Castillo, 2020) for the evaluation of the same.
A metric is a numerical or nominal value assigned to characteristics or attributes of an object computed from a set of observable data and consistent with intuition (De Marco, 1986). Some of the metrics identified were the following: tasks solved in a limited time, time taken to complete a task, level of difficulty, time used to finish a task the first time etc.
Standards-based usability models
Within the scope of this research, a study of some of the models commonly used to evaluate usability in mobile applications was carried out see (Fig.1). Para la selección del modelo For model selection, the number of attributes was taken into account, as well as the objectives, contributions and limitations of each attribute (Ronny Scherer, 2019), (Jesús Aguirre, 2020), (Bruno BERTAGNOLLI, 2018). From the analysis corresponding to the study, the following model was selected PACMAD for gathering significant attributes from different usability models such as Nielsen, Tognazzini and Pierotti among others, to create a more complete model, the seven attributes of the model are: efficiency, effectiveness, ease of learning, recall, satisfaction, errors and cognitive load.
The model identifies three factors that can affect the overall usability of a mobile application: user, tasks and context of use. It combines the ISO 9241-11 model with the Nielsen model and adds the cognitive load attribute, which is a very important aspect because it evaluates the behavior of the mobile application when the user is multitasking. In addition, it was designed to address the limitations of existing usability models in the context of mobile devices (Hoz, y otros, 2020) and it is one of the most attribute-rich. Figure 1
Source: Own elaboration.
Each of the models studied has limitations (Chaabane, y otros, 2019), (Ronny Scherer, 2020), (Afua van Haasteren, 2019) The QUIM model could have been selected because it has the same number of attributes as the PACMAD model, but this one for usability evaluation still needs to be validated. (Enriquez, y otros, 2016). The PACMAD model is limited by the fact that it does not establish usability guidelines for evaluating a mobile application, Therefore, the authors of the research adapted the usability guidelines proposed by the National Center for Software Quality in Cuba CALISOFT based on the Cuban standard ISO 25010: 2016 to each of the attributes of the PACMAD model.
Table 1 shows a summary of the number of guidelines defined by attribute, with a total of 60 guidelines. As an example, Table II shows a partial list of the guidelines proposed for usability evaluation.
Fuente: Elaboración propia
Effectiveness is the attribute with the lowest number of defined guidelines (three), while the other attributes have 6 or more. This is because effectiveness, unlike the other attributes, depends on the adequate fulfillment of the tasks within the mobile application, i.e. that the application responds to the user's need, so it depends more on the functionality than on the design of the user interface. In this sense, in the other attributes we can find elements that can contribute to improve efficiency, error management, recall capacity, satisfaction and cognitive load from the design point of view. Table 2
Source: Own elaboration.
The elements present in mobile technology that must be considered when evaluating usability in this type of solution were also established, such as: the mobile context, the type of mobile application to be evaluated, the type of mobile application to be evaluated, and the type of application to be used (hybrid, native, web), connection speed and network performance, screen size and resolution, landscape and portrait displays, operating system or platform version, type of device where the application will be tested, hardware performance, battery and/or power consumption, and interruptions due to messages, calls and data connection.
Activities proposed in the guide
The activities proposed as part of the guide are as follows see (Figure 2) are in accordance with the life cycle of a mobile application development project and reach the Testing phase, taking into account that the vulnerabilities detected during the acceptance phase would imply an increase of time and cost in their resolution, divide the text into two columns of equal length
Source: Own elaboration.
The following is a description of each of the activities in Figure. 2 corresponding to the stages of a software development process.
Inicio
In the start-up phase or stage, usability management should be planned. Here the activities and tasks related to usability should be established in the project schedule. This activity has as input the initiation report and as output the project schedule. The roles involved in this activity are: the Project Manager and the Quality Manager. Then, the project team must be trained in topics related to usability in mobile applications. The responsibilities and tasks of each role are explained and the factors that influence each usability sub-characteristic and its classification by engineering disciplines are exposed. This activity has as input the project schedule and as output the updated project schedule, and the Act of Commitment. The roles involved are: project manager, development team, architect, analyst, quality managers and testers.
Modelado de negocio
In this phase, the conceptual model of the mobile application must be created, for which the business documentation, if provided by the client, must be received as input for the activity. Here the input is the business documents provided by the client and the output is the Conceptual Model. The roles involved in this activity are: Project Manager, Analyst and Architect. Then the technological environment must be prepared for usability testing. The tools are installed and configured and the documentation and everything necessary for the management of the usability RNF must be available.
This activity must have the Conceptual Model as input and the installed and configured tools as output. The roles involved are: Project Manager, Analyst and ArchitectThen the business elements should be reviewed with the customer. Here the factors to be taken into account for usability management are analyzed and the business elements are examined to detect and correct errors in time. The input of the activity is the Conceptual Model and the outputs are: Meeting Minutes, Business Acceptance Record and revised and updated Conceptual Model Artifacts. The roles involved are: Project Manager, Analyst and Client.
Requisitos
In the requirements stage, initially the Non Functional Requirements (NFR) of usability must be described, for this purpose the formal document with the detailed description of the usability NFR is written. The input in this activity is the Conceptual Model and the output is the Software Requirements Specification. The role involved is the Analyst.
Then the Formal Technical Review (FTR) of Requirements is performed where checklists will be used to measure the technical completeness of the requirements, and that there are no inconsistencies with the artifacts to which they relate. The inputs in this activity are: The Software Requirements Specification and the Checklist. The output is the updated Checklist and the Requirements Defect List. The roles involved are: Analyst, Quality Manager, Project Manager and Reviewer Team.
Finally, at this stage the artifacts of the requirements discipline must be reviewed with the client so that there are no inconveniences in the future and once everything is verified, the Act of Acceptance of the requirements is signed. The input is the documents of the project file and the output is the Minutes of the meeting and the signed Act of Acceptance of the requirements. The roles involved are: Project Manager, Analyst and Client.
Análisis y diseño
In this stage, the system and its architecture must be initially designed. In this activity the architecture and design views are modeled, taking into account the usability RNFs. It has as input the Software Requirements Specification and as outputs the Design Model and the Architecture Views. The roles involved are: Analyst and Architect. Then the architectural RTF is performed.
For the RTF, checklists will be used to measure the technical completeness of the design diagrams and architecture views, taking into account usability issues. As inputs you have the Design Model, the Architecture Views and the Checklist. As outputs you have the Updated Checklist and the Architecture Defect List. The roles involved are: Analyst, Architect and Reviewer Team. Finally, at this stage, changes in the architecture must be managed to ensure the quality of the mobile application in terms of usability. The input to this activity is the Architecture Defect List and the output is the updated Architecture View. The roles involved are: Quality Manager, Project Manager and Architect.
Implementación
In this phase the requirements must be implemented. All the functionalities of the system are developed and the usability RNFs must be taken into account, in addition to the usability guidelines proposed in the guide. The inputs are: The Analysis and Design Artifacts, Usability Patterns and the Usability Guidelines proposed in the guide. The output is the coded components of the product. The roles involved are: Development Team, Architect and Analyst. Then unit tests must be performed taking into account the usability and in this way evaluate the fulfillment of the usability NFR. The inputs of this activity are the coded Components of the product and the outputs, detected Non-Conformities (NC) and updated Product Components. The roles involved are: Development Team and Quality Manager.
Prueba
In this phase the product must be tested. The objective of the testing and evaluation process is to verify that the developed mobile application meets functional and non-functional requirements such as usability. The inputs in this activity are integrated components of the product and test documents. The outputs are the List of detected NCs and corrected product components. The roles involved are: Analyst, Development Team and Test Team.
Results and discussion
In order to assess the contribution of the guide to the reduction of the number of usability non-conformities based on the proposed usability guidelines, during the testing process of mobile applications of the Electronic Government Center CEGEL, the Delphi and survey methods were applied to obtain the experts' criteria. Seven experts participated, all with more than five years of experience in the software industry, with the ability to make recommendations, detect inadequacies and evaluate aspects of the proposed solution.
The categories of Relevance, Relevance and Coherence were used for the assessment. The procedure was carried out by applying the scale of Very Adequate (MA), Fairly Adequate (BA), Adequate (A), Not Adequate (PA), Inadequate (I).
The analysis of the results shows that all the categories are evaluated as Very Adequate or Fairly Adequate, validating the contribution of the guide to the solution of the problem. For the categories, in all cases, the degree of agreement was above 85%, which is considered a satisfactory result. The experts did not cast votes on the Poorly Adequate or Inadequate scale. From the votes cast by the experts, the following percentage results are obtained in the Very Adequate and Fairly Adequate ranges: Table 3
Fuente: Elaboración propia
In addition, a case study was carried out in the LexCuba project of the Electronic Government Center. (CEGEL) of the UCI specifically to the Civix mobile application to assess the effect of the implementation of the guide. This analysis made it possible to obtain the differences with respect to the number of NCs identified in different versions of the product in the testing process of these applications.
In a first version of the development, the application of the guide was not taken into account. During the testing process of the Civix project, a total of 20 usability NCs were identified. Most of the NCs identified referred to misspellings due to lack of accentuation of some words, error messages not corresponding to the action performed, misaligned forms, distorted images, too much content information on the same screen, the application in landscape and portrait mode of the phone are not displayed correctly, too much space is left between the left margin and the names of the tables, etc.
From the implementation of the functionalities planned in the scope of the second version of the product, the guide was applied taking into account the proposed usability guidelines. The application of the guide ensured the definition, management, development, follow-up and traceability of the non-functional usability requirement from the early stages of development to testing. Once the development of the second version was completed, 5 usability NCs of the Civix project were identified in the testing process. This evidences the decrease in the number of usability NCs identified and therefore the application of the guide contributes to reduce the number of nonconformities identified.
Finally, the Iadov technique was applied to evaluate customer satisfaction with respect to the usefulness and applicability of the guide, where a survey was applied to 10 experts, including members of various ICU development centers. When analyzing the responses collected from the application of the surveys in the Iadov logic table, a group satisfaction index of 0.90 was obtained, which translates into a clear satisfaction with the use of the guide for the management of usability tests and the proposed usability guidelines.
In the experts' criteria regarding the usefulness of the guide, there was 100.00% agreement with the qualification "Very Adequate". Regarding the applicability of the proposed guide and guidelines, there was a 90.00% agreement with the qualification "Very Adequate" and the other 10.00% qualified it as "Fairly Adequate".
Conclusions
It has been possible to conclude that the guide and the proposed usability guidelines were validated by scientific methods through surveys to the selected experts and case studies. The use of the guide favors the reduction of the number of non-conformities identified in the development of mobile applications. The experts' criteria endorse the coherence and relevance of the guide, being consistent with the requirements of the research problem.
