Original article

The evaluation of the impact of the introduction and generalization of results of pedagogic investigations

0000-0003-3095-4924José Luis Lissabet Rivero^{1  *}  , 0000-0002-5804-3615Tania de la Caridad Rosabal Ferrer¹ 

¹Universidad de Granma, CRAI, Campus Blas Roca, Manzanillo, Cuba.

ABSTRACT

In the article is presented a result of the Project Investigation Development Evaluation of the impact of the introduction and generalization of the scientific results in the University of Granma" that responds to the problem of the relationship among the processes of obtaining of results, their generalization in application contexts and the evaluation of the impact. The objective is to present a methodology to the generalization of scientific results and the evaluation of the impact. A theoretical investigation was developed, from a qualitative focus. The obtained results allowed elevating the quality of the investigation process and the transformations wanted in the educational levels.

Key words: Methodology; Evaluation; Introduction; Generalization; Impact; Scientific results.

Introduction

Cuban higher education is currently engaged in maintaining its model of a modern, humanistic, universalized, scientific, technological, innovative university, integrated with society and deeply committed to the construction of a prosperous and sustainable socialism, a university characterized by the formation of values and by the quality assurance of its substantive processes, in order to achieve a graduate who possesses personal qualities, culture and professional skills that allow him/her to perform with social responsibility, and to promote his/her education for life.

One of the challenges to overcome in the pedagogical careers, to achieve the above, is to have research in the field of Pedagogical Sciences that lay the foundations to promote a continuous increase in the quality and effectiveness in the comprehensive training of education professionals.

Therefore, for the research carried out in the field of Pedagogical Sciences, it is necessary to take as a starting point the study of the processes of introduction, generalization and evaluation of the impact of the scientific research results obtained by the professionals of the pedagogical careers.

This process must be characterized by the quality of the solution provided to the investigated problem, recognizing that any process of this nature has a methodological process that takes place from the execution of the scientific tasks of the Research and Development Project (R+D+i) to obtain the result, passing through the evaluation of the effect produced experimentally, to the evaluation of the impact produced with its generalization in the social practice.

So, what we are talking about is to achieve this chain in the research of the Pedagogical Sciences and thus achieve the efficiency of the Research Development Projects developed by professionals. According to ^{García (2017}), it expresses the relationship between the resources used and the results achieved. It is a measure of the productivity of the execution process, of the efficient use of the financial, human and material resources allocated.

These demands that, on the one hand, pose the process of obtaining scientific results, and on the other hand, their experimental application and evaluation of the results obtained with their generalization, generates a contradiction between the increasing number of scientific results obtained and the insufficient socialization of the results to transform educational practice, which allows revealing a gap (^{García, 2017}), between: the educational direction in the different instances, from the educational institutions, and the use of the scientific results to base and lead the transformations that must take place in the different educational levels.

This contradiction has its genesis in obtaining new solutions to the problems that the Research Development Projects have contributed, which have been produced, evaluated and introduced in a given sociocultural context, and subsequently these techno-scientific results have not been generalized (application context) in other sociocultural contexts, which has brought as a consequence the limitations in the quality of education, manifested in the results obtained in learning and in the integral formation of the personality of the learners.

The article presents a result obtained in one of the scientific tasks of the R+D+i Project "The evaluation of the impact of the introduction and generalization of scientific research results at the University of Granma" (^{Lissabet, J. et al., 2019}), where the solution to the problem expressed in the contradiction that manifests itself between the processes of obtaining scientific results, their generalization in different application contexts and the corresponding evaluation of the impact they should produce at different educational levels is addressed.

Therefore, the objective of the article is to present a methodology to develop the process of generalization of scientific results and the evaluation of their impact in pedagogical careers.

Development

In science, the term "methodology" has been understood, explained and interpreted from multiple theoretical positions, which vary depending on the plane from which their definitions are established; thus the use of this term is very diverse, from its different meanings, in the specific case of Educational Sciences, the term methodology as a scientific result can be used for various purposes, in this R+D+i Project, it is used, following ^{De Armas & Valle (2011}), with the purpose of "Guiding the realization of activities of educational practice". (p. 45)

Hence, it is assumed the definition of methodology provided by ^{De Armas & Valle (2011}), which is understood as: "A theoretical contribution that is based on a series of phases that allow guiding the ways to direct stages of the scientific research process, they are predominantly indicative, prescriptive and normative of the procedure in the field of pedagogical research". (p. 47)

The methodology requires the explanation of how it operates in practice, how the methods, procedures and techniques are combined, how the requirements are taken into account in the development of the process and the steps or actions that are followed to achieve the proposed objectives.

The methodology for developing the process of generalization of scientific research results and impact evaluation in pedagogical research (^{Lissabet et al., 2019}, p. 36), presents the following structure:

I.-

1. General objective.

The objective of the methodology is to improve the methodological tools to develop the process of generalization of scientific research results and the evaluation of the impact of research at the different educational levels, and thus:

1. To promote the planning and organization of the process of generalization of the scientific results by the members of the Project.

2. To strengthen the preparation of the executing entities for the assimilation and implementation of the scientific results.

3. To promote the planning and organization of the process of evaluation of the impact of the generalization of the scientific results.

4. To promote the preparation of the executing entities for the evaluation of the impact of the generalization of the scientific results.

5. To favor the evaluation of the process and the results achieved by the participants and the social environment in order to provide the necessary feedback and subsequent decision-making.

II.

1. Conceptual apparatus that supports the methodology.

There are two issues in the management of science (^{Núñez, 1999}, p. 84) that are cardinal when talking about results and their impacts: their introduction and generalization. The introduction of scientific-technical results is understood as:

The process of implementation by the executing entities and researchers of the scientific results in the pedagogical process, with the purpose of testing them and proving their usefulness in improving the quality of education. (^{Arencibia, 2005}, p. 26).

On the other hand, the generalization of scientific and technological results is considered as:

Process of assimilation and implementation by the Organisms of the Central Administration of the State (OACE), National Entities, Provincial and Municipal Governments, Companies, Budgeted Units and Cooperatives, of those results of the Science, Technology and Innovation activity already tested and useful in another entity that contribute to maintain or raise the efficiency, effectiveness, quality and competitiveness in the fulfillment of productions and services. (^{Ministry of Science, Technology and Environment, 2020}, p. 3).

The process of introduction of the results of research in the Educational Sciences corresponds to the context of evaluation of science and the process of generalization of scientific-technical results obtained in research in the Educational Sciences corresponds to the context of application.

Taking into account these aforementioned aspects, in the research in Educational Sciences, a contradiction is manifested between: the contexts of production, context of evaluation and context of application of pedagogical science, such contradiction is concretized in: the increasing number of scientific results and the insufficient socialization of the results to impact educational practice, which allows revealing a gap between: the educational direction in the different instances from the educational institutions and the use of scientific results to base and lead the transformations that should take place in the different educational levels.

This contradiction has its genesis in the new knowledge and educational technologies that research has contributed, which have been produced, evaluated and introduced in a given sociocultural context, and subsequently these techno-scientific results have not been generalized (context of application) in other very different contexts, which has resulted in limitations in the quality of education, manifested in the results obtained in learning and in the integral formation of the personality of the learners.

In recent times, and as a result of the accelerated development of society in all its spheres, the term "impact" of scientific and technological results in the scientific, technological, environmental and social spheres has been gaining strength in the context of science management (^{Núñez, 1999}, p. 93).

The impact of science and technological innovation is understood as:

The change or set of lasting changes that occur in society, the economy, science, technology, and the environment, improving their indicators, as a result of the execution of R&D&I actions that introduce added value to products, services, processes, and technologies. (^{Ministry of Science Technology and Environment, 2014}, p. 1).

Assuming the above definition, and from the purpose of educational research, to contribute to educational change, it makes sense to consider that the impacts of research in the Educational Sciences (^{Arencibia, 2005}, p. 25) should be sought in the results of educational change, defined as: "The transformations intentionally achieved in conceptions, attitudes and practices of the school educational community that contribute to the developmental education of people". Generally speaking, impact is the change that occurs in the short, medium or long term as an effect of knowledge and familiarization, application and dissemination of the results obtained in people, their professional performance, or in their contexts of action.

This necessarily leads to understand that the impacts of the results of educational research are produced gradually, some of them can be obtained almost immediately, in the very short term, those are the ones that have to do, generally with questions of form in the object of research, for example, with questions of school organization, hygiene, some teaching methodologies, among others.

Other impacts are observable in the medium term, although, in the author's opinion, it is not possible to limit this space in time for all cases. Reference is made to results aimed at achieving changes in the performance of the subjects, but only at the level of their practices and even in the external manifestations of their attitudes, for example, with results that deal with learning, motivational, orientation, communication strategies, among others.

There are other long-term impacts, which have to do with transformations at the level of the conceptions of the school educational community, referred to the results of fundamental, theoretical research, which offer results aimed at the development of competencies, formation of values, personality qualities, among others.

Among the main areas of social impact of research in Educational Sciences proposed by ^{Arencibia (2005}, p. 29) are:

1.-

1. Development of the actors involved in the educational process.

1. This is the fundamental area of the social impact of the results of research in the Educational Sciences, referring to transformations that propitiate:

2. The development of developmental learning.

3. The formation of an integral general culture in the students.

4. The development of an integral personality, which will allow all people to defend the conquests of socialism, values and national identity.

5. Increase of theoretical consensus around the Educational Sciences.

6. Development of the Educational Sciences.

7. Improvement of the training and improvement of the teaching staff.

2.-

1. Educational policy and system management.

Changes or modifications are introduced in the educational policy, due to the incorporation of research results, which constitute transformations in social and interpersonal relationships, the system of values that regulate them, the methods and styles of management.

3.

1. Educational Institutions.

Transformations are produced as a consequence of the adoption of educational research results in educational institutions, these are in the direction of school institutions or the territory, in the improvement of living conditions in them, methodological work in its different forms, planning and dissemination through the mass media and Information and Communications Technology.

The evaluation of the impact of the generalization of scientific results (^{Lissabet et. al., 2019}, p. 23), is understood as the process aimed at identifying, predicting, evaluating and reporting the correspondence of the objectives in the specific social environment, and its fundamental purpose is to assess the process and the results achieved by the participants and in the social context, and to provide the necessary feedback, which in the hands of decision-makers can enrich the policies and management of education and, above all, its links with the sustainable development of society.

Thus, the evaluation of the impact of the generalization of scientific results (^{Lissabet et. al., 2017}, p. 26) is used to:

• Evaluate compliance with educational policy

• Decide on management strategies

El enfoque metodológico para la evaluación del impacto (^{García, 2017}, p. 22), que se propone es el siguiente:

1. Identification of impacts

2. Identification of areas of impact of change.

3. Identification of the variables to be studied

4. Accuracy of the indicators of the variables to be studied.

5. Accuracy of the algorithm for the study of the variables.

The evaluation of the impact of the generalization of scientific results is a way to ensure the necessary transformations, to measure the social effect on the participants and their contexts of action based on the development of personal, professional, economic, environmental and social potentialities.

The impact on the educational system (^{Garcia, 2017}, p. 24) is manifested in:

• Satisfaction of training needs

• Efficient performance

• Social commitments

• The solution of performance problemical problems by scientific means.

• Increased initiative in calling on the professionals in their environment.

III.-

1. Stages that make up the methodology.

The methodology for developing the generalization and impact assessment of pedagogical research consists of the following stages:

First stage: organization and planning of the generalization and impact assessment process.

The purpose of this stage is: to specify the activities or tasks to be carried out and the resources required to achieve the proposed objectives within a given time, and to organize and harmonize the human, material and financial resources available to generalize the result and evaluate its impact.

Actions:

1. Identify the type of result to be generalized, its origin and its scope.

2. Determine the entity or unit that will generalize the result.

3. Determine the resources necessary for the generalization of the result.

4. Determine the level of generalization of the result.

5. Determine the level of the National Education System at which the results will be generalized.

6. Determine the possible outputs of the generalized results.

7. Determine the types of impact expected to be obtained with the generalization of the result.

8. Specify the start and end date of the generalization of the result.

Second stage: preparation of the executing entities to develop generalization and impact assessment.

The objective of this stage is to prepare managers and users of the executing entities to assimilate and implement the results to be generalized.

Actions:

1. Establish agreements with the executing entities to ensure the actions of generalization of the result.

2. Apply instruments to check the preparation of the managers and users of the executing entities for the assimilation and implementation of the results to be generalized.

3. Apply different ways to prepare managers and users of the executing entities for the assimilation and implementation of the results to be generalized.

Third stage: implementation of the obtained scientific result.

The objective of this stage is to develop actions, together with the users, that allow the implementation of the scientific result and obtain the expected transformations.

Actions:

1. Determine the pathways to be used to generalize the result.

2. Implement the planned ways to generalize the result.

3. Monitor the development of the processes involved in the generalization of the result.

4. Perform the necessary feedback of the processes and results that are being obtained.

5. Make timely decisions in the event of undesired transformations.

In order to execute the actions of this stage of implementation of the obtained scientific result, it is proposed to apply a methodological procedure to generalize the scientific result and evaluate the impact:

1. Determine the types of results to generalize.

A result is the product of an activity in which scientific methods and procedures have been used, which allows offering a solution to a problem, it takes the form of scientific knowledge or a material production or its combination and solves a certain economic, social or environmental need. According to the identification of the essence of the problem that solves the result to generalize can be:

• Diagnosis

• Regulatory

• Teaching

• Didactic

• Methodological

• Material

• Technological

Determine the origin of the result to be generalized

The Projects Associated with Programs (^{Ministry of Science, Technology and Environment, 2020}, p. 13) are those that are an organic part of a given program and their objectives respond to it, at any of the established levels: national, sectorial and territorial.

1. Provide the necessary feedback on the processes and results being obtained.

2. The process of generalization of the result is planned, from the design stage of the R&D&I Project, according to the scope of the problem to be solved and the specific objectives, the periods of control and evaluation of the obtaining, introduction, generalization and evaluation of the impact of the expected results are specified.

3. Specify the start and end date of the generalization of the result.

Fourth stage: impact assessment.

The objective of this stage is: to carry out actions to evaluate the internal impact and prepare the conditions to receive the external evaluation, in order to achieve the necessary feedback of the processes and the corresponding decision making.

Actions:

1. Define the frame of reference for impact and its measurement. 2.

2. Define the specific indicators for the internal evaluation of the impact of the generalization of the results.

Internal evaluation, which is carried out by the researchers themselves and which should be conceived in every research project as part of its theoretical-methodological design and should use certain general indicators and other more specific ones.

It is in the research itself where the referential framework of the impact is defined, therefore it is here where a first assessment of these should be made, specifying what types of impact are produced, what specific indicators were used and what results were obtained by using them.

The specific indicators proposed by the authors for the internal evaluation of the social impact of the results of research in Educational Sciences expected to be achieved in the medium and long term with the generalization of technoscientific results are the following:

1. Appropriation of the knowledge system (concepts, laws, theories and vision of the world) in the different subjects.

2. Assimilation of the system of specific competencies of the Teaching Disciplines.

3. Assimilation of the system of professional competences of the careers.

4. Assimilation of the system of intellectual, teaching and practical competencies of the different Teaching Disciplines.

5. The use of procedures and strategies for the conscious assimilation of contents.

6. The formation of professional modes of action.

7. The formation of behavioral norms, feelings, value orientations and values.

8. The motivation for learning, self-esteem and expectations,

9. The achievement of meaningful relationships between the contents learned and life, in the socio-cultural context in which it is developed and in the contribution that educational technology transmits.

1. Define the deadlines (short, medium, long) to achieve the impact of the generalization of the results.

2. Define specific indicators for external evaluation of the impact of the generalization of results.

IV.

1. - Evaluation of the achievement of the objective of the methodology.

The evaluation of the achievement towards the objective of the methodology is proposed to be carried out through the results provided by its external evaluation, which is the one carried out by experts, essentially taking into account the general indicators for impact evaluation proposed by ^{García (2017}), these are:

• Degree of completeness in the application of the results in social practice.

• Strategic scope in the development of management policy by the government as a result of research.

• Awards granted and national and international recognition.

• Multiplier effect of the research by deriving other related research and publications.

• Requests from companies, national and territorial agencies and organizations to integrate research results into national, territorial and local development macro-projects.

• Level of reception and incorporation of nationally and internationally recognized publications in the scientific community and other institutions.

• Linkage with updating processes of teaching programs at different educational and postgraduate levels. (p. 27)

In the process of implementing the result in the executing entities, instruments will be applied by the external evaluators to check:

1. The result of the planning and organization of the process of generalization of the scientific results.

2. The preparation of the executing entities for the assimilation and implementation of the scientific results.

3. The planning and organization of the process of evaluation of the impact of the generalization of the scientific results.

4. The preparation of the executing entities for the evaluation of the impact of the generalization of the scientific results.

5. The results achieved by the participants and in the social environment to provide the necessary feedback.

6. Timely decision making.

7. Apply instruments to verify compliance with the indicators of the impact of the generalization of the results.

8. Carry out the necessary feedback of the processes and decision making.

Conclusions

In the scientific-research activity in the field of Educational Sciences, conceived as a social process, the correspondence between the contexts of production, evaluation and application of scientific results must be taken into consideration, in order to identify, predict, evaluate and inform the correspondence between the results obtained and the impact produced in the formation of the professional in the pedagogical careers.

The methodology to develop the process of generalization of scientific results and the evaluation of their impact is based on epistemological foundations that allow revealing and arguing the system of categories, components and their relationships, and the intrinsic organic link between them, revealing as an essential function of the research process the solution of the contradiction between the process of obtaining scientific results, their generalization in different contexts of application and the evaluation of the impact they should produce in the different educational levels.

The application in the methodology has as fundamental objective to value the process and the results achieved by the participants and in the social context, to propitiate the necessary feedback and the subsequent decision making, in a way that guarantees the attention to the diversity of individuals, contexts and conditions that intervene in the pedagogical careers.

The social impact of research in the Educational Sciences is expressed in the development of the actors involved in the training process of professionals in pedagogical careers, which is, in the development of socio-cultural contexts and in the educational change reflected in students, teachers, families, managers and the community.

Referencias bibliográficas

Arencibia, V. (2005). La investigación educativa desde un enfoque dialéctico. Congreso PEDAGOGÍA 2005. Cuba.

De Armas, N. & Valle, A. (2011). Resultados científicos en la investigación educativa. Pueblo y Educación.

Ministerio de Ciencia Tecnología y Medio Ambiente (2014). Bases para la organización y evaluación de la actividad científica y tecnológica a partir del impacto de sus resultados. La Habana: PubliCien.

Ministerio de Ciencia Tecnología y Medio Ambiente (2020). Indicaciones metodológicas para la Actividad del Sistema de Programas y Proyectos de Ciencia, Tecnología e Innovación. La Habana: PubliCien .

García, G. (2017). Efectos e impactos de las investigaciones en Ciencias de la Educación. Conferencia Magistral Memorias del Evento Cubano de Desarrollo Local. Universidad de Granma. Bayamo. Granma.

Lissabet, J. et al. (2017). Innovación y Sistematización Didáctica para la Formación de Profesionales Universitarios, a través de la formación de pregrado y posgrado en Ciencias Pedagógicas. Informe del Proyecto de I+D+i Institucional. Universidad de Granma.

Lissabet, J. et al. (2019). La evaluación del impacto de la introducción y generalización de los resultados científicos investigativos en la Universidad de Granma. Informe de Proyecto de I+D+i Institucional. Universidad de Granma.

Matos, E. & Cruz, L. (2011). La práctica investigativa, una experiencia en la formación doctoral en ciencia pedagógica. Universidad de Oriente.

Núñez, J.(1999). La ciencia y la tecnología como procesos sociales. Lo que la educación científica no debiera olvidar. Félix Varela.

Received: September 18, 2021; Accepted: January 25, 2022

^*Autor para la correspondencia:[e-mail]