Custom services
text in 
English (pdf)
Article in xml format
Article references
Send this article by e-mail
Cited by SciELO 
Similars in
SciELO 
Permalink
Introduction
It is known that the term meta-analysis refers to the systemic and integral analysis of a series of statistical results of individual researches on the same subject, in order to find relevant conclusions (Sao 2013). The performing of all meta-analysis involves the passing through different stages: formulation of the problem to be researched, search of information, evaluation of methodological quality of the studies, coding of possible moderating variables, measurement of results, analysis and interpretation of data, and its publication (Sánchez 2010).
Meta-analytic studies have been used in different scientific research, but most have been developed in psychology, medicine and social sciences as an alternative method of scientific review (Martín 1995). In livestock and agricultural science, the use of meta-analytical techniques has not had a great proliferation in comparison with other areas of knowledge (St-Pierre 2001), although, in recent years, there has been an increase of their use (Signorini et al. 2012; Jensen et al 2015, Busanello et al. 2016, Maccarana et al. 2016, Zimmermann et al. 2016, Dezordi et al. 2017 and Dinnen et al. 2017).
In the studies consulted, however, meta-analysis is carried out, without taking into account the stage of the evaluation of the methodological quality of the studies. Zeballos (2013) states that evaluation is a fundamental requirement to minimize biases and may be statistically related to the results of meta-analysis.
There are two instruments for assessing the methodological quality of the studies: checklists and scales, being the latter the most used (Martín 2014). Specifically, the evaluation scales meet multiple criteria, define a scoring system for each criterion, and make it possible to attribute a global quality score to each study, which considers all the aspects that affect it. Currently, there are several scales, but they have been developed mainly for randomized clinical trials, and not in other types of studies (Cascaes et al. 2013).
The objective of the study was to propose a specific scale, which allows to evaluate the methodological quality of research about chemical composition and nutritional value of silages based on forages, to later include the best evaluation studies in a meta-analysis.
Materials and Methods
Design. For the preparation of the quality scale, the PEDro scale, developed by Verhagen et al. (1998) was used as reference. This scale was developed for its use in randomized clinical trials, and is responsible for measuring the credibility of the trial or internal validity, and for determining if it has enough statistical information to be able to interpret it (Fuentes 2016).
To develop the new scale proposal, some criteria of the PEDro scale were taken as a basis, referring to sample taking, randomization of treatments to the experimental units and the statistical information reported from the experiments. However, those referred to the studies of blind experiments were discarded because they are characteristic of clinical researches in humans.
In addition, other criteria were included, related to the characteristics of forage for silage, description of the silage preparation process, chemical analysis procedures used to determine the key variables, the statistical methods used in the experimentation, and the information or not about key variables for silage studies based on forage. These criteria were selected from an extensive review of the literature, with the aim of identifying an instrument from which potential criteria and relevant domains could be obtained (Burgos et al. 2011).
Methodology. To identify the criteria that made up the scale, 80 articles were analyzed, referring to the evaluation of nutritional quality of tropical forage silages, intended for feeding of ruminants. For the validation of the tool developed, another 40 articles with the same subject were used.
For the search and selection of articles, EBSCO, Scielo, Science Direct and Researchgate databases were used. The search strategy consisted on the use of the terms: silage, nutritive value, quality, chemical composition, and ruminant. These words were combined with the Boolean operator "AND" or "OR".
Statistical analysis. To validate the scale, reliability between two evaluators was determined using the intraclass correlation coefficient (ICC), with the statistical package Epidat version 4.2 (Hervada et al. 2016)
Results and Discussion
Table 1 shows the criteria used for the scale proposal, which will be the base for evaluating methodological quality of studies on chemical composition and nutritional value of forages for ruminants.
Figure 1 Scale proposal for evaluating methodological quality of studies on chemical composition and nutritional value of forage silages for ruminants to be included in the meta-analysis
The evaluator that uses the scale for assessing the quality of the study to be included in the meta- analysis will only provide points, when the criterion is clearly fulfilled, so the following should be taken into account:
Criterion 1. It is fulfilled if the article describes the characteristics of forage used for silage, for example: species and grass variety, cut age, soil type, climatic variables of the geographical region, use of irrigation and fertilization levels.
Criterion 2. It is fulfilled if the study reports about the sampling method used, schedule, cut height, equipment used and forage amount. It should also inform about the conservation of forage sample for its transfer to the chemical laboratory and its processing (drying, milling and storage).
Criterion 3. It is fulfilled if silage preparation process is properly described. This includes that forage was previously pre-dried, type of silo used, particle size, compaction method, use of additives and inclusion levels, sealing form and silage time.
Criterion 4. It considers the random designation the study, and if the evaluated source clearly informs that treatments were assigned to the experimental units of the product to be ensiled at random.
Criterion 5. It is fulfilled if the sampling in silos and the number of samples analyzed by treatment are described
Criterion 6. It is fulfilled if the methods of chemical analysis used for determining the variables under study are properly reported.
Criterion 7. It is fulfilled if the statistical method used during the experimentation process is adequately detailed.
Criterion 8. It is fulfilled if the results of the main variables of the chemical composition of forages used for ensiling are reported: dry matter [DM], organic matter [OM] or ash, crude protein [CP], fiber fractionating, soluble carbohydrates [SCH] and forage buffer capacity. This criterion does not receive points if less than 50 % of these variables are reported. This means that at least three of the aforementioned variables must be reported so that the evaluator gives points for this criterion.
Criterion 9. It is fulfilled if the results of the main variables of the chemical composition and nutritional value of the ensiled products are reported: DM, OM, ashes, PC, fiber fractioning, SCH, or degradability [DM, OM, CP]. No point is given by this criterion, if less than 50% of these variables are reported. This means that at least three of the aforementioned variables must be reported so that the evaluator gives points for this criterion.
Criterion 10. It is fulfilled when the results of the main variables of the fermentation that occurred during the silage process are reported, which are pH, lactic acid, short-chain fatty acids [acetic acid, propionic acid, and butyric acid] or ammoniacal nitrogen. Points are given for this criterion, if at least 75 % of these variables are reported. This means at least 3 variables.
Criterion 11. It is fulfilled if the statistical comparison of treatment means is developed and the test used or the comparison of a treatment with a control condition is reported. The comparison can be made by a hypothesis test, which provides a probability value.
Criterion 12. It is fulfilled if the value of treatment means and variability statistics for measured key variables is reported. The latter include standard deviations, standard errors, coefficient of variation, confidence intervals, interquartile range (or other quantity ranges) and average ranges.
Based on the scale proposal developed, the methodological quality of studies related to nutritional value of silages for ruminants can be evaluated, based on 12 criteria. Manterola and Otzen (2015) consider that a study is rigorous, when it meets 100 % of the criteria, while a study has poor quality if its score is less than 75 % of the total criteria that make up the instrument used. Being consistent with the above, the new scale proposal evaluates a study of good methodological quality and well designed, if it receives a score equal to or greater than 9 (75%).
Validating the scale with two independent evaluators resulted in a CCI of 0.81, with a confidence interval between 0.67 and 0.89 for a significance level of 0.05. According to Simancas and Arévalo (2017), CCI values superior to 0.75 indicate an excellent reliability. Therefore, the value obtained in the validation confirms the high reliability of the designed instrument.
Regarding the description of the 40 included studies, it is noteworthy that only 10% meet the fourth criterion, with respect to the randomization of treatments to the experimental units. In addition, the evaluators only observed fulfillment with the second criterion in 45 % of the total studies. That is, not all authors describe the process of sample taking of forage used to ensile.
The pilot study, with 40 articles and two evaluators, allowed to validate the reliability of the scale, with excellent level of reproducibility or reliability of measurement between different observers. Therefore, it is concluded that the new scale design allows to evaluate the methodological quality of studies related to chemical composition and nutritional value of silages for ruminants. It, thus, becomes an appropriate tool, which will minimize biases and improve the quality of the selected studies to develop meta-analyzes on the subject.
