Artículo original

Testing strategy for organizations developing web portals

Estrategia de pruebas para organizaciones desarrolladoras de portales web

0000-0003-2631-6612Alicia Chacón Rodríguez^{1  *}  , 0000-0002-6534-132XMaidelyn Pinero Gonzalez ²   , 0000-0001-5101-7804Aymara Marin Diaz³  , 0000-0002-3138-011XYaimi Trujillo Casañola⁴ 

¹ University of Informatics Sciences, Cuba. San Antonio Highway Km 2 1/2. [e-mail]

² University of Informatics Sciences, Cuba. San Antonio Highway Km 2 1/2. [e-mail]

³ University of Informatics Sciences, Cuba. San Antonio Highway Km 2 1/2. [e-mail]

⁴ University of Informatics Sciences, Cuba. San Antonio Highway Km 2 1/2. [e-mail]

ABSTRACT

Web portals in the last decade have gained a high level of popularity among users who usually surf the Internet. This has led to customers becoming more rigorous every day and demanding innovative products of the highest quality and attractiveness. To do this, these products are tested, which are key activities in software development, since they help detect defects that would otherwise go unnoticed until the software is deployed. However, various studies prove that in most of these products only functional tests are carried out. This is a problem because the detection of the defect is far from the moment in which it is introduced, which affects the costs of correction and lengthens the schedules of the project, without covering the structural and non-functional tests . This article describes a testing strategy that encompasses functional, non-functional, structural, and change-associated testing based on web portals. The strategy takes into account good practices documented in internationally recognized models, norms and standards. It incorporates the experience gained in testing these types of products. It integrates what to test from the tiered test type and how to test from describing objectives, typical test objects, test bases, approaches and responsibilities, typical defects and failure types, techniques, and strategies. The results of the expert assessment and a case study are shown for a better understanding of the proposal.

Key words: quality; defects; strategy; portal; tests; website

RESUMEN

Los portales web en la última década han ganado un elevado nivel de popularidad entre los usuarios que usualmente navegan en Internet. Esto ha conllevado a que los clientes sean cada día más rigurosos y exijan productos innovadores de altísima calidad y atracción. Para ello, estos productos son sometidos a pruebas, las cuales constituyen actividades claves en el desarrollo de software, puesto que ayudan a detectar defectos que, de otro modo, pasarían desapercibidos hasta que el software sea desplegado. Sin embargo, diversos estudios comprueban que en la mayoría de estos productos solo se realizan pruebas funcionales. Esto supone un problema pues se aleja la detección del defecto del momento en que se introduce, lo que incide en los costos de corrección y alargan los cronogramas del proyecto, sin cubrir las pruebas estructurales y no funcionales. El presente artículo describe una estrategia de pruebas que abarca pruebas funcionales, no funcionales, estructurales y asociadas al cambio en función de portales web. La estrategia tiene en cuenta buenas prácticas documentas en modelos, normas y estándares reconocidos internacionalmente. Incorpora la experiencia adquirida en la realización de pruebas a estos tipos de productos. Se integra el qué probar a partir del tipo de prueba por niveles y el cómo probar a partir de describir los objetivos, objetos de pruebas típicos, bases de pruebas, enfoques y responsabilidades, defectos y tipos de fallas típicos, técnicas y estrategias. Se muestran los resultados de la valoración de expertos y un estudio de casos para mejor comprensión de la propuesta.

Palabras-clave: calidad; defectos; estrategia; portal; pruebas; web

Introduction

Currently, new technologies are always present and within the reach of most people and companies, so the need to have a website increases day by day. Not only for those who want to make money online, but for all those companies or people who offer a service or have a business. (^{Erazo, 2012}, ^{Magaly, Rolando et al., 2020})

Companies increasingly choose to use web portals, whose fundamental characteristic is to serve as a gateway to offer the user, in an easy and integrated way, access to a series of resources and services related to the same theme. Includes: links, search engines, forums, documents, applications, electronic purchase, etc. (^{Coloma, Pino et al., 2018}, ^{Magaly, Rolando et al., 2020}). Mainly, an Internet portal is aimed at solving specific information needs on a particular selected topic, organizing and distributing content to meet the needs of its users/consumers (^{Dominguez, 2006}, ^{Fajardo, Escalona et al., 2018}).

There are different types of web portals among them (^{Erazo, 2012}, ^{Pinho, Franco et al., 2018})

1. General Portals (Mega Portals or Horizontal Portals)

2. Specialized Portals.

3. Corporate Portals

4. Vertical Portals (Vortals)

It is important to clarify that, despite having different classifications, they have elements in common, they all offer resources, information services divided and organized depending on the type of public they are aimed at; allow personalization of information, which is displayed in an easy-to- use interface with a pleasant design (^{Santana Bonilla, Eirín Nemiña et al., 2017}). Therefore, the proposed strategy can be applied regardless of these classifications.

Therefore, as these software systems are becoming more and more important in society, the demand for their quality increases. According to the Institute of Electrical and Electronics Engineers (IEEE), software quality is the degree to which a system, component, or process meets specified requirements and the needs or expectations of the customer or user (IEEE 1990, ^{A. I. Vlasov, 2022}). Which depends largely on the tests that have been carried out during its development. Testing is therefore one of the most important activities in software development. Without them, the software would contain many defects that would cause failures that could only be detected when the product is installed and used in a real production environment, thus causing the impact of these failures to be serious and the cost to solve the defects that cause them is large (^{Boehm and Basili, 2007}, ^{Diaz, Casañola et al., 2020}, ^{Srivastava, Kumar et al., 2021}).

However, despite the importance of testing, in many organizations they are not given enough attention or are simply omitted from software product development planning. Many times, the pressure to have a finished product on a set date causes activities, mistakenly considered as expendable, to be eliminated or reduced in planning. In this way, the aim is to reduce costs and be able to have a product that can be delivered on time, although the drawbacks are usually much greater than the advantages of simply delivering “something”. Thus, eliminating testing activities, instead of accelerating software development, means that these products can never be considered truly finished, since it is possible that defects continually appear that must be resolved once the software has been deployed. In this context, a defect can be defined as an unexpected behavior of the system, which does not correspond to its requirements and specifications (^{Fernández, 2015}, ^{Chen, 2021}).

The tests are very expensive so they are left for the later stages of the project and do not cover all types of recommended tests. On the other hand, tests are the most important resource for evaluating the quality of software (^{Vásquez Romero, 2018}, ^{Vera, Valdivia et al., 2020}); however , quality and testing are not the same thing. Quality is built into software throughout the engineering process, if it's not there when testing begins, it won't be there when it's finished. For this reason, they should focus on prevention and control activities from the beginning of software development. As stated in the principles of evidence. (^{David Flores Mendoza, 2019})

Like all software, web portals are based on principles such as: graphic design, navigability, web positioning, user load, performance efficiency, security, among others (^{Toledo, 2019}). However, with the high demand for these products and the limited time for their production, developers choose to use methodologies with an agile approach, where functional and non-functional requirements such as reliability, functionality, security, scalability, usability, among others, they cannot be omitted since this lack would directly lead to the failure of the project (^{Pasini, Ramón et al., 2006}, ^{Santana Méndez and Trujillo Casañola, 2010}, ^{Guamán Barbecho, 2019}, ^{Zhao Huanga, 2021}).

In the bibliographic review carried out, the authors were able to verify that there are problems in the detection of defects in the early stages of development of web portals (^{Aldana La ́O and Ramos Medina, 2009}, ^{Ruiz Tenorio, 2010}, ^{Vásquez Romero, 2018}). evidences the scant attention paid to aspects as relevant as making an interactive design focused on maintaining the receiver's attention, which adapts to the specific characteristics of each user, and the efficiency of quickly displaying requested information, have been part of the background, ignoring Thus, quality characteristics set out in the ISO 25010 standard, causing deliveries of products with numerous defects in both functional and non-functional tests, but also a proposal is not made of what to test and how to test in the different stages of the development cycle to guarantee more attractive and efficient products to the user(ISO/IEC 2011, ^{Sánchez, 2021}).

In order to verify the above, an analysis of the defects detected in testing stages, at the national level, was carried out on this type of product. An interview was conducted with the National Center for Software Quality (CALISOFT), where the questions were aimed at knowing what were the most common defects in the portals and the quality characteristics of the product, according to the ISO / IEC 25010: 2016 Standard, which most affected were based on the number of associated defects in their tests. It was stated that the most affected characteristics in the evaluations are: functional adequacy, performance efficiency and usability(ISO/IEC, 2016, ^{González, Diaz et al., 2021}).

On the other hand, the data of the tests carried out at the University of Informatics Sciences in the last five years were analyzed. According to these data, the most affected characteristics are: functional suitability and usability. In the tests to evaluate the functional adequacy characteristic, the validation type defects were the most manifested. In the usability evaluations, the defects of visibility, aesthetics and design and those associated with consistency and standards are highlighted.

Analyzing these findings, it can be established that many of these defects are introduced in the early stages of the software. (^{Zamuriano, 2004}, ^{Ruiz Tenorio, 2010}). The following table shows some of the most common defects in websites as the cause that originates them.

Table 1 Some of the most common defects in websites. 

Disciplines software development	test level	Types of defects	Cause of defects
Requirements	Component, System	Validation defects can be introduced Defects related to Response Time	Due to an incorrect survey or absence of the requirements (functional or non-functional) Deliberate deviation from requirements (due to time pressure, advances without authorization) Documentation errors
Analysis and design	System	Correspondence between artifacts Aesthetics and design Visibility	Inadequate communication between team members Design Errors Insufficient evidence Documentation errors
Implementation	System	Not checking the web in different browsers and resolutions Navigability.	Non-compliance with standards related to a web portal facing the Internet

Font: Own elaboration.

Then, it can be established that, if tests are designed throughout the life cycle of the portals, with specific activities to deal with the introduction of these types of defects, the final quality of the product developed within the national territory will be improved, as well as the delivery times to customers and thus the satisfaction of end users.

Therefore, it is established as a problem to be solved for this research: What and how to test throughout the life cycle of portals, taking into account national experiences and good practices recommended in the most used models, norms and standards? The objective of the research is to develop a testing strategy to evaluate web portals, which allows detecting defects closer to the moment they are introduced and reducing delivery times to the client and the satisfaction of end users.

Methods or Computational Methodology

For the development of this research, the methods mentioned below were used. In addition, a brief explanation of the purposes for which they were used is provided.

Theoretical methods:

1. Dialectical method for the critical study of previous works and to use them as a source of reference and comparison of results.

2. The analytical-synthetic method was used for the study of the bibliography about the most used quality models internationally.

3. The deductive hypothetical for the identification of the problematic situation and the solutions.

Empirical methods:

1. Interview to obtain information in order to argue the problematic situation and the validation of the results.

2. The survey to obtain the experiences of the organizations.

3. The participant observation to obtain the necessary information for the approach of the problem, as well as to carry out the confrontation of the obtained results.

4. Statistical methods to assess the effect of the proposal.

5. The experimental method to check the usefulness of the data obtained from the implementation of the general testing strategy.

From the point of view of an investigation, a strategy must consider the following stages: Diagnosis, Approach of the general objective, Strategic Planning, Instrumentation and Evaluation. For this research, an alignment of these stages with the characteristics of a testing strategy was carried out. The general objective of the proposed strategy is identified, which would be contained in the organization's quality policy based on the results of the diagnosis. Strategic planning is carried out by defining the objectives by test levels and later the instrumentation is described where it is explained what to test from the type of test by levels and how to test from describing the objectives, typical test objects, test bases , approaches and responsibilities, defects and types of typical failures and techniques and strategies(^{Diaz, Casañola et al., 2019}, Diaz, Casañola et al., 2020).

Initially, the development organization must take into account in its test policy the quality activities related to the tests. Feedback from these activities should be used to detect changes in risks so that planning can be adjusted. These activities may include scope, objectives, risks, planning for analysis, design, implementation, execution, and testing activities (2018, ^{Board-ISTQB® 2018}, ^{Pinho, Franco et al., 2018}). For the drafting of the policy, the use of brainstorming is proposed where members of the organization's management who can make decisions in the processes, those responsible for directing the quality processes and some actors of these processes participate, thus improving quality. of these products, decrease the delivery time to customers and increase the satisfaction of end users.

After establishing the policy, the test strategy to be executed is designed. The proposal is based on the one designed by the authors (^{Diaz, Casañola et al., 2020}), see Table 2 . Taking into account that this strategy is for all software products, it is complemented by proposing activities based on the aforementioned needs of the portals.

Table 2 Instrumentation of the test strategy. 

	Component	Integration	System	Acceptance
Objectives	Reduce risk Verify the functional and non-functional requirements of the component Find component defects Prevent defects from escaping to other levels of testing	Reduce risk Verify functional and non-functional behaviors of interfaces Create confidence in the quality of interfaces Find defects in the interaction between components Prevent defects from escaping to other levels of testing Verify integration between systems	Reduce risk Verify that the functional and non-functional behaviors of the system Create confidence in the quality of the system Fault Prevent defects from escaping production	Validation that the system is complete and will function as intended Verify that the functional and non- functional behavior of the system Satisfy user, operation and legal or regulatory requirements
Typical test objects	Components/ classes/ units/ modules Code	Subsystems Databases Infrastructure Interfaces	Applications System configuration	Applications Systems System and data configuration
test base	Component Requirements Detailed design Code	Software and systems design sequence diagrams Specifications of interfaces and communication protocols Use cases Architecture	Specification of software and system requirements. Risk analysis reports Use cases, user stories System behavior models System and user manuals	business processes user requirements Regulations, legal contracts Software Requirements Specification Use cases, user stories System, user and installation manuals.
Approaches and responsibilities	It's from the developer, it focuses on test automation, usually white box. These tests must be carried out by the project team.	Component integration testing is the responsibility of the developer. Integration testing between systems is the responsibility of the tester. These tests must be carried out by the project team.	Testers whose fundamental task is the detection of defects from the design of the test. An independent testing team is proposed to the development team.	They are the responsibility of customers, users and system operators. For this level, if there is clarity about what has been agreed between the parties, it is proposed that only the development team and the client intervene.
Typical defects and failures	Validation	Response time	Validation Response time Correspondence between artifacts Aesthetics and design Visibility	Response time Not checking the web in different browsers and resolutions Navigability.
test types	Functional testing Structural Regression and confirmation tests	non functional Structural Regression and confirmation tests	Functional testing Non functional Regression and confirmation tests.	Functional testing Non functional Regression and confirmation tests.
Organization (Strategies and tests)	Analytics (requirements) methodical It is recommended at this level to carry out a complete review of both the functional and non-functional requirements. For structural tests, the use of the method to verify the correct use of coding standards. Carry out confirmation tests for all the defects detected in the previous tests and confirmation tests based on the risk analysis. At this level, the automation of all the proposed tests is proposed.	Analytics (requirements and risks) methodical based on models For this level, a review of the functional and non-functional requirements with an analytical approach is recommended. In the structural tests, compliance with standards and optimization of functions must be verified, as well as the correct integration between the components through architecture models. Confirmation tests must be carried out on all detected defects and confirmation based on a risk analysis.	Analytics methodical based on models Reactive or based on experience It is proposed to start with a complete review of the functional and non-functional requirements. Taking into account the available testing time, it can be done through an analysis of requirements or risks. Functional testing should be started in order to ensure some stability in the product before starting non-functional testing. Models made for the correct understanding of the business of the product can be used through operational flows for functional tests. Regression and confirmation tests should be used to verify that all detected defects were correctly corrected and that no new ones were introduced in said process.	Analytics methodical advisory At this level, unlike the rest, it is recommended to start with a method of functional and non-functional testing to validate that what has been agreed with the client is correctly built. Then you can continue with an analysis of the risks that the client considers. Regression and confirmation tests are proposed to be carried out, but it is necessary to be clear about the scope of the agreed contract, since there may be customer requests that are not non-conformities but requests for change. The consultative approach is proposed since at this level it is considered that the client should be the one to guide the tests.

Font: Own elaboration.

There is a group of specific characteristics in the development of national web portals, they have the characteristics that are content-oriented and focused on the user interface, favoring visual creativity and the incorporation of multimedia. Many of the sites are directed by objectives of content, promotion of products or services (^{Calderón Montero and Hernández León, 2007}, ^{Alvarez, Aldana et al., 2018}). In this type of products, the majority of dissatisfactions either by the user/consumer or in the tests that are carried out are those related to validation, aesthetics and design and performance efficiency for them, the researchers propose as a result of the investigation activities more specific linked to detecting these most common defects in this type of products, such as review of requirements to verify that they contain in their specifications everything required by the client, enhance communication (essential requirement) in the development team, implementation of good practices of quality activities to mitigate future risks and errors, so that defects closer to the stage where they are introduced are detected, which will reduce correction costs from the very beginning of the product.

Results and Discussion

The proposal will be applied to the University of Informatics Sciences, in which software products are developed, including web portals for different businesses (educational, health, sports, industries...) and for the development of these products, methodologies based on in user stories. The quality policy that was identified from the focus group was:

Quality activities are carried out to prevent, control and improve the quality of products. These activities represent revisions of the specifications, the design, the coding with the aim of preventing possible failures that may exist in the web portals. As a strategy, it was selected to combine the analytical strategies: based on requirements , based on experience and methodical.

Table 3 Description of activities. 

Feature: Functional Suitability
Test levels	Most common defects	Quality activities for its prevention	Roles and responsibilities	Observations
Component	Validation	Exercise 1: Review of requirements	external quality reviewer project analyst	In the formal technical reviews (RTF) it must be verified that all the requirements must be related to functionality, there must be a good definition (data type, limit values, input/output conditions, response to all kinds of situations), everything necessary for its understanding by the developer
Integration		Activity 2: Strengthen Analyst - Developer link .	Project analyst and developer	Good communication between both pillars of the team should be encouraged, since if there is poor communication between them, the product's objectives will not be achieved, influencing errors such as validation errors.
Integration		Exercise 3: Perform inspections on every code update	Project analyst and developer	Every time an embedded element on the screen such as buttons and forms that includes the insertion of data is updated or modified, the developer must communicate so that it can be tested if everything complies with the above stated in the requirements, and does not influence its correct operation. . (interface tests)
Feature: Performance Efficiency
Risks			Requirements
Lack of resources on the server side Network with inadequate bandwidth Poor or weak operating system capabilities Poorly designed functionality and other hardware or software conflicts that can lead to degraded software performance			The infrastructure elements must be reviewed to ensure that they do not contain any known vulnerabilities, as well as the administrative tools used for the maintenance of the different components. Requests to the application must be met in less than 5 seconds. Requests to the DB must be attended to in less than 3 seconds Define performance efficiency requirements from the beginning of the software.
Integration System or Acceptance	Response time	Activity 4: Identify risks associated with software resources. Activity 5: Define efficiency requirements.	Analyst	From the beginning of the software, possible deficiencies must be defined in order to influence the performance of the system, as well as establish requirements that take into account the characteristics associated with temporal behavior, the use of resources and capacity.
Feature: Software Usability
Risks			Requirements
Insecure communication with the client (dissatisfaction) Inadequate documentation Confusing information architecture (if the user does not find what they are looking for, they leave) Vulnerable software products High quality costs due to rework because the further the detection of defects is from the stage where they are introduced, the more expensive their elimination.			Define and correctly write non-functional requirements (usability) Indicate to the user in which part of its structure it is located, that is, if it shows breadcrumbs Provide all the information involved in performing a certain action to the user (forms, descriptions of required fields, confirmation or error messages) Carry out inspections of the documentation, there must be similarity with the product that is being developed.
Integration System	Aesthetics and design Visibility Navigability.	Activity 6: Define a design according to the needs of the user Activity 7: Establish project quality standards linked to usability.	Analyst, project leader, user, developer.	For this characteristic, it is essential to link the user from the beginning of the development cycle, since the perfection of the product starts from him. The project manager must ensure that basic internal and unit tests are carried out with the user during development, as a prerequisite for Quality Control tests (verify that the screens and messages are clear to the user)
		Activity 8: Define if the product will be visible internationally.
Note: A good survey of both functional and non-functional requirements must always be taken into account, since the development of the product is part of them. In addition, when creating user stories, take into account the proposal of a user interface prototype, as well as make clear the messages and specifications that must be shown to the user.

Font: Own elaboration.

It must be taken into account that these proposed activities are associated to discover and mitigate the most common defects related to the development of web portals. For the evaluation of the given proposal, it was necessary to carry out a survey that would allow obtaining expert criteria. For the selection of these, the curricular analysis technique was applied. 15 experts participated with more than ten years in the software industry and from different software development organizations nationwide.

Expert selection process

An assessment of the possible experts was carried out, the following were considered as the initial selection criteria: knowledge related to quality and quality management in software projects, work experience in the software industry of 8 years or more, practical experience as the main factor. Taking these criteria into account, a questionnaire for curricular knowledge was carried out. As a result, 15 experts were selected at the national level, from the institutions: DESOFT, XETID, ICU and CUJAE. After selecting the experts, taking into account the knowledge about the quality characteristics of the product and the practical experience they have about the value of complying with them, the Delphi method was applied to take advantage of the common elements in the group of experts. Anonymity was preserved through the use of communication flows that allow the participation of experts even if they are geographically dispersed (^{Trujillo Casañola, Febles Estrada et al., 2015}, ^{Díaz and Castañola, 2019})

Satisfactory results were obtained from the analysis of the answers given, since all the categories were evaluated as Very High or High, validating the contribution of good practices associated with minimizing defects in the development of these products, as well as activities necessary to incorporate from the beginning of the development of the software, as fundamental for the satisfaction of the users and acceptance of the product. A mode of High or Very High was obtained and the experts did not cast votes on the scale of Low or None. From the votes cast by the experts, the following results are obtained (Table 3):

Table 3 Percentage of the experts' criteria. 

Criterion	Percent	Fashion
Relevance	97.4	5
Relevance	94	5
Coherence	86	4
Comprehension	90	5
Accuracy	64.7	4

Font: Own elaboration.

Based on these results, it can be ensured that the experts agree that the activities and good practices proposed have a positive impact on the prevention and control associated with these types of errors and that they must be incorporated from the beginning of the software development, being necessary to carry out a correct management of product requirements so that they do not negatively affect the operation of the system and its acceptance.

Conclusions

1. Both functional and non-functional requirements must be correctly defined from the beginning of software development to obtain a design and architecture that correspond to the behavior of the product during its development to minimize the defects or failures that these may cause.

2. Carrying out software tests is important to reduce the risk of failure in operation, however, many times they are carried out after the product is finished and only functional tests are carried out.

3. The results reaffirm the need to carry out early tests in the development of the product, complemented with testing strategies and activities to obtain positive criteria at the end of the product.

References

(2018) "Certified Tester Foundation Level Syllabus." International Software Testing Qualifications Board (ISTQB).

A. I. Vlasov, B. V. A., and Juravleva, L. V. (2022). "Quality estimation method in advanced software systems." AIP Conference Proceedings 2467.

Aldana La, ́O.N. and Ramos Medina, J. L. (2009) "Propuesta de procedimiento general para pruebas de liberación de los productos de la Facultad 1.".

Alvarez, A. R., Aldana, L. M. and Rodríguez, A. E. G. (2018). "PROPUESTA DE REQUISITOS DE USABILIDAD Y EFICIENCIA DE DESEMPEÑO PARA NORMA RAMAL: REQUISITOS DE CALIDAD PARA SISTEMAS INFORMÁTICOS Y PRODUCTOS DE SOFTWARE." Informática 2018: 8.

Board-ISTQB®, I. S. T. Q. (2018). Probador Certificado del ISTQB®.

Boehm, B. W. and Basili, V. R. (2007). Software Defect Reduction Top 10 List. Software Engineering: Barry W. Boehm's Lifetime Contributions to Software, Development, Management, and Research. R. W. Selby. California, Estados Unidos: 34 (31): 135-137.

Calderón Montero, M. and Hernández León, R. A. (2007). Propuesta de procedimiento para la captura de requisitos en los proyectos de portales de la UCI.

Chen, Z. (2021). "An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications." IEEE/ACM 43rd International Conference on Software Engineering (ICSE): pp. 674-685.

Coloma, M. A. V., Pino, Á. M. B. and Mora, D. E. L."MÉTRICA DE CALIDAD EN LOS PORTALES WEB."

Coloma, M. A. V., Pino, Á. M. B. and Mora, D. E. L. (2018). "MÉTRICA DE CALIDAD EN LOS PORTALES WEB."

David Flores Mendoza, I. M., Tovar, P (2019). Creación de una metodología óptima y eficiente para la implementación de pruebas de software, Instituto Politécnico Nacional de México.

Diaz, A. M., Casañola, Y. T. and Hidalgo, D. B. (2019). "Apuntes para gestionar actividades de calidad en proyectos de desarrollo de software para disminuir los costos de corrección de defectos." Revista chilena de ingeniería vol. 27 Nº 2: pp. 319-327.

Diaz, A. M., Casañola, Y. T. and Hidalgo, D. B. (2020). "Estrategia de pruebas para organizaciones desarrolladoras de software." Revista Cubana de Ciencias Informáticas Vol. 14, No. 3: 22.

Díaz, A. M. and Castañola, Y. T. (2019). Aseguramiento de la Calidad del Proceso y el Producto. Maestría de Calidad en Software: 67.

Dominguez, M. A. C. S., Jorge, A.M. (2006). A Web-Based System to Monitor the Quality of Meta-Data in Web Portals. IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IATW'06).

Erazo, C. F. R. (2012) "DISEÑO, DESARROLLO E IMPLEMENTACIÓN DEL PORTAL WEB DE LA EMPRESA DE AUTO VENTAS “AUTO FÁCIL”, APLICANDO LA HERRAMIENTA DE DESARROLLO WEB OPEN SOURCE DRUPAL." Repositorio Digital Universidad Técnica del Norte.

Fajardo, I. V., Escalona, Y. R. and Morales, R. (2018). "Portal web para GEDEME." Revista Publicando 5(14 (3)): 497-501.

Fernández, M. Á. F. (2015). Aplicación de técnicas de pruebas automáticas basadas en propiedades a los diferentes niveles de prueba del software.

González, M. P., Diaz, A. M., Casañola, Y. T. and Hidalgo, D. B. (2021). "Buenas prácticas para prevenir los riesgos de la eficiencia del desempeño en los productos de software." Revista Cubana de Ciencias Informáticas Vol. 15, No. 1: Pág. 89-113.

Guamán Barbecho, E. P. (2019). "Análisis y evaluación de la calidad del sitio web de la Universidad del Azuay."

IEEE (1990). "IEEE Standard Glossary of Software Engineering Terminology (IEEE Std 610.12-1990)." IEEE Computer Society, Los Alamitos. CA.

ISO/IEC (2011). ISO/IEC 25010:2011 Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models, Switzerland: 44.

ISO/IEC (2016). ESTÁNDAR INTERNACIONAL ISO/IEC 25023 Sistemas e ingeniería de software Calidad de sistemas y software Requisitos y Evaluación (SQuaRE) - Medición de la calidad del producto del sistema y software. Switzerland: 61.

Magaly, L. M. N., Rolando, M. R. J., Fernando, M. R. R. and Marcela, P. S. C. (2020). Mensajería cliente-servidor aplicando sockets en las herramientas GEANY IDE 1.31, PHYTON 3.7 y POSTGRESQL 9.5 en el sistema operativo CENTOS 7. Conference Proceedings.

Pasini, A. C., Ramón, H. D. and Thomas, P. J. (2006). Testing en aplicaciones web. XII Congreso Argentino de Ciencias de la Computación.

Pinho, C., Franco, M. and Mendes, L. (2018). "Web portals as tools to support information management in higher education institutions: A systematic literature review." International Journal of Information Management 41: 80-92.

Ruiz Tenorio, R. (2010). "Las pruebas de software y su importancia en las organizaciones."

Sánchez, K. M. M. (2021). Análisis y evaluación del sistema Eproductions ERP para el cumplimiento de los requisitos de calidad de la Norma ISO/IEC 25010, en la Empresa Viaxperta S.A. Licenciatura en Sistemas de Información, Universidad de Guayaquil

Santana Bonilla, P. J., Eirín Nemiña, R. and Marín Suelves, D. (2017). "Análisis y evaluación de portales institucionales en España. Los casos de Canarias, Galicia y Valencia."

Santana Méndez, W. and Trujillo Casañola, Y. (2010). Proceso de desarrollo para portales Web.

Srivastava, N., Kumar, U., Singh, P. J. J. o. I. E. and Engineering, E. (2021). "Software and performance testing tools." 2(01): 1-12.

Toledo, M. A. (2019). "HERRAMIENTAS PARA TESTING DE APLICACIONES WEB." Conexiones 1(5): 140-144.

Trujillo Casañola, Y., Febles Estrada, A. and León Rodríguez, G. (2015). "Modelo para valorar las organizaciones desarrolladoras de software al iniciar la mejora de procesos."

Vásquez Romero, D. P. (2018). "Estrategia de pruebas de software en un proyecto full stack para minimizar el impacto sobre los clientes y usuarios en una empresa de telecomunicaciones."

Vera, Y. P., Valdivia, J. J. G., Quentasi, S. M. Z., Yana, D. M. C. and Apaza, R. E. C. (2020). "Design Thinking en la Planificación de Pruebas de Software." Innovación y Software 1(2): 40-51.

Zamuriano, R. F. (2004). Las inspecciones de software y las listas de comprobación, Tesis de Maestría, Instituto Superior Politécnico, José Antonio Eceverría ….

Zhao Huanga, J. M. (2021). "Gender differences in user perception of usability and performance of online travel agency websites." ScienceDirect 66.

Received: February 02, 2022; Accepted: March 22, 2022

^*Corresponding author. ( [e-mail])

The authors of this article authorize the distribution and use of their article.

Conceptualization: Aymara Marin Diaz

Data curation: Yaimí Trujillo Casañola

Formal analysis: Maidelyn Piñero González

Research: Alicia Chacón Rodríguez

Methodology: Alicia Chacón Rodríguez y Maidelyn Piñero González

Project Administration: Yaimí Trujillo Casañola

Resources: Alicia Chacón Rodríguez

Software: Alicia Chacón Rodríguez y Maidelyn Piñero González

Supervision: Aymara Marin Díaz y Yaimí Trujillo Casañola

Validation: Aymara Marin Díaz y Yaimí Trujillo Casañola

Visualization: Alicia Chacón Rodríguez y Maidelyn Piñero González

Editing - original draft: Alicia Chacón Rodríguez

Writing - proofreading and editing: Maidelyn Piñero González y Aymara Marin Díaz

No funding was necessary for the development of the research.