SciELO - Scientific Electronic Library Online

 
vol.20 issue3Dr. Francisco Valdés Lazo: an exceptional physicianMorphologic relationship between the hyoid bone and the viscerocranium in a sample of skeletons author indexsubject indexarticles search
Home Pagealphabetic serial listing  

My SciELO

Services on Demand

Journal

Article

Indicators

  • Have no cited articlesCited by SciELO

Related links

  • Have no similar articlesSimilars in SciELO

Share


Revista Habanera de Ciencias Médicas

On-line version ISSN 1729-519X

Rev haban cienc méd vol.20 no.3 La Habana May.-June 2021  Epub June 16, 2021

 

Biomedical basic sciences

Comparison between nasopharyngeal swabs and saliva as reliable specimens for the diagnosis of SARS-CoV-2

Comparación de la confiabilidad del diagnóstico del SARS-CoV-2 entre muestras de hisopado y saliva

Cristobal González-Losada1  2  * 
http://orcid.org/0000-0002-7256-2649

Luis Gabriel González-Lodeiro2 
http://orcid.org/0000-0002-5580-9339

Abraham Ismael Beato Canfux3 
http://orcid.org/0000-0002-0751-7737

Julio Raúl Fernández2 
http://orcid.org/0000-0002-0858-8465

Hamlet Camacho2 
http://orcid.org/0000-0001-8668-034X

Dania Vazquez-Blomquist2 
http://orcid.org/0000-0001-5011-2850

Gerardo Enrique Guillén Nieto2 
http://orcid.org/0000-0003-3098-0970

1Universidad de Ciencias Médicas de La Habana, Instituto de Ciencias Básicas y Preclínicas "Victoria de Girón", Departamento de Inmunología. La Habana, Cuba.

2Centro de Ingeniería Genética y Biotecnología. La Habana, Cuba.

3Hospital Militar Central “Dr. Luis Díaz Soto”. La Habana, Cuba.

ABSTRACT

Introduction:

The SARS-CoV-2 virus is a positive-strand RNA virus. The virus can also be detected in many different specimens as throat swabs, nasal swabs, sputum, saliva, blood, etc.

Objective:

The aim of this paper is to compare the reliability of different types of specimen collection, saliva and swabs samples for the detection of SARS-CoV-2.

Material and Methods:

A sample of 22 COVID-19 positive patients was selected. Paired samples from saliva, nasopharyngeal, oropharyngeal and nasopharyngeal + oropharyngeal swabs were collected on the 7th day after diagnosis. The hyssops and medium employed was IMPROSWAB and IMPROVIRAL NAT Medium, Germany. The sample evaluation was conducted through RT-PCR. The results were compared using Fisher’s exact test and ROC curve. The gold standard proposed in this paper was the nasopharyngeal + oropharyngeal swabs specimen.

Results:

The gold standard method detected 10 true positive cases, of which oropharyngeal swabs, nasopharyngeal swabs and saliva only detected three positive cases. Significant differences (Fisher’s exact test p = 0.003) were detected in the comparison between saliva and the gold standart proposed. The ROC curve analysis showed that saliva had an area under the curve of 0.650, with a 30% of sensibility. However, the nasopharyngeal and nasopharyngeal + oropharyngeal samples had an area under curve of 0.950 and 1.000, respectively, with a sensibility of 90% and 100%, respectively.

Conclusion:

Saliva samples are not a reliable specimen for SARS-CoV-2 RNA detection. In turn, the most reliable specimens are nasopharyngeal and nasopharyngeal + oropharyngeal samples collected by swabbing.

Keywords: Saliva; COVID-19; SARS-CoV-2; sensibility; RT-PCR; nasopharyngeal swab; oropharyngeal swab

RESUMEN

Introducción:

El SARS-CoV-2 es un virus ARN positivo. Este virus puede ser detectado en diferentes tipos de secreción como hisopada bucal, nasal, esputo, saliva, sangre, etc.

Objetivo:

El objetivo de este estudio es comparar la confiabilidad de diferentes tipos de muestras, saliva y exudado, en la detección de SARS-CoV-2.

Material y Métodos:

Una muestra de 22 pacientes con diagnóstico de Covid-19 fue estudiada. Se tomaron muestras pareadas de saliva y exudado nasofaríngeo y orofaríngeo en cada paciente. Se emplearon los hisopos y medios de la firma alemana IMPROVE®. Los resultados de las determinaciones por RT-PCR se compararon mediante test de Fisher (test de la probabilidad exacta de Fisher) y cada sets de muestras fue evaluada individualmente y luego comparadas por curvas ROC. El estándar de oro propuesto fue el doble hisopado nasofaríngeo/orofaríngeo.

Resultados:

El método de oro propuesto detectó 10 casos positivos. La coincidencia de detección entre todos los sets de muestras fue de 3 casos (30%). Se obtuvieron diferencias significativas (Fisher p = 0.003) en la comparación de los casos detectados en saliva vs el estándar de oro. El análisis de curvas ROC mostró un área bajo la curva de 0.650 (30% de sensibilidad) para la saliva. En el caso del hisopado nasofaríngeo y el estándar de oro mostraron un área bajo la curva de 0.95 y 1.00, respectivamente, con una sensibilidad del 90% y 100%, respectivamente.

Conclusiones:

La saliva no es una muestra confiable para la detección de SARS-CoV-2. La muestra más confiable para el diagnóstico fue el hisopado nasofaríngeo y el doble hisopado.

Palabras claves: saliva; COVID-19; SARS-CoV-2; sensibilidad; RT-PCR; hisopado nasofaríngeo; hisopado orofaríngeo

Introduction

The SARS-CoV-2 virus (severe acute respiratory syndrome coronavirus 2) is a positive-strand RNA virus. This virus is the cause of the coronavirus disease 2019 (COVID-19).1 On 11 September 2020, the World Health Organization (WHO) declared an international health emergency (pandemic) as the virus is highly contagious and spreads rapidly throughout the world.2,3

On 4 September 2020, the number of confirmed cases with SARS-CoV-2 exceeded 30 million, with more than 950 000 deaths.3 The most frequent symptoms observed in patients were fever above 38 degrees, dry cough, fatigue, dyspnea, and myalgia and to a lesser extent, headache, abdominal pain, nausea, vomiting, and diarrhea.4

As recommended by the WHO, real-time reverse transcriptase-polymerase chain reaction assay (RT-PCR) of nasopharyngeal swabs has been used to confirm the clinical diagnosis.5,6,7 The virus can also be detected in throat swabs, nasal swabs, sputum, saliva, blood, feces, urine and bronchoalveolar lavage fluid specimens although respiratory tract samples tend to test positive for the virus more often.5,6,7,8,9

Although the international recommendation is employed (the sample collection by nasopharyngeal swabs), saliva has gained relevance as a sample for diagnosis. The saliva sample collection is a non-invasive and relatively simple collection process for those who are not well trained in the health area, which represents a specific advantage in this regard. However, the value of saliva as a viable sample is inconsistent.10,11,12,13,14,15

The objective of this paper is to compare the reliability of different types of specimen collection (saliva and swabs samples) for the detection of SARS-CoV-2.

Material and Methods

Patients in the study

An exploratory study was conducted in June 2020 including 22 positive COVID-19 patients from Havana province that were admitted to the “Dr. Luis Diaz Soto” General Hospital. Each participant signed an informed consent.

Due to the low incidence of COVID-19 positive patients in Cuba, a decision was made. Samples will be recollected on the 7th day after the positive diagnosis (hospital admission) to ensure the greatest number of positive cases.

A sample of 22 Covid-19 positive patients was selected (sample evaluation was conducted through RT-PCR). These cases were classified as severe and critical who did not present any consciousness alterations and were not included in any clinical trial yet.

Paired samples from saliva and swabs (nasopharyngeal, oropharyngeal and nasopharyngeal-oropharyngeal altogether) were taken from these patients. Patients were required to present themselves without eating and brushing their teeth, between seven and eight a.m.

All patients were treated under the same protocol established by the Cuban Ministry of Public Health (Ministerio de Salud Pública de Cuba, MINSAP),15 starting the day they were admitted to hospital.

Sample collections (saliva and swabs)

Saliva samples were self-collected by patients by letting saliva get out of the mouth into a sterile tube (Corning, 50 mL). Corning tubes were located at the lip corner and then the patient leaned over the tube with the mouth wide open until completing 5 mL of saliva, resulting in a physiological secretion process. Patients were told not to spit.

The swab sample collection was carried out following international guidelines.15 Nasopharyngeal, oropharyngeal and nasopharyngeal + oropharyngeal swabs samples were collected with hyssops from the respective anatomical zones. Each sample set was stored in separate containers. In the case of nasopharyngeal + oropharyngeal samples, both hyssops were kept in the same container to increase the sample utility. This specimen was considered as the gold standart for diagnosis.

All specimens were sent to a Biosafety Level 2 Laboratory for SARS-CoV-2 RT-PCR testing.

Commercial hyssops from Germany (IMPROVE®, IMPROSWAB: Microbial Transport swabs for oropharyngeal and nasopharyngeal use) were used to conduct the swabbing process. The medium employed to transport the hyssops with oropharyngeal and nasopharyngeal secretion was IMPROVE®, IMPROVIRAL NAT Medium (Germany). Data such as age and sex of the patients included were also collected.

SARS-CoV-2 RNA detection

Oropharyngeal and nasopharyngeal swabs were collected at the hospital following standard procedures. Afterwards, samples were transported to a Biosafety Level 2 Laboratory for SARS-CoV-2 plus for the SARS-CoV-2 viral nucleic acid detection through RT-PCR. Swabs were placed into a collection tube with a virus preservation solution. Total RNA was extracted with QIAamp® Viral RNA Mini kit using Qiacube automated system (Qiagen, USA).

A 70 bp long fragment from the Equine Arteritis Virus (EAV) genome (LightMix® Modular EAV RNA Extraction Control, from TIB MOLBIOL Syntheselabor GmbH, Berlin, Germany) was added before purification as an internal extraction control. The former was amplified with specific primers and detected with an LC670 labeled hydrolysis probe in the qPCR. A 20μL qPCR reaction was set up. It contained 5 μL of RNA and 15 μL of the components of LightCycler Multiplex RNA Virus Master (Roche, USA) and LightMix® Modular Sarbecovirus E-gene or Wuhan CoV RdRP-gene primer and probes systems (TIB MOLBIOL Syntheselabor GmbH, Berlin, Germany).

A real-time one-step RT-qPCR analysis of viral RNA was performed in a LightCycler® 480 II system, with thermal cycling at 50 °C for 10 min for reverse transcription, followed by 95 °C preincubation for the 30s and then 45 cycles of 95 °C for 5 s, 60 °C for 30s. Multiplex detection was carried out according to the manufacturer's instructions. Negative and positive controls (≈ Ct 30) were included. The criteria for the success of qPCR detection were the EAV internal control amplification (Ct 27-33) and the amplification of E or RdRP genes at Ct ≤40.

Statistical procedures

Every sample evaluated through RT-PCR where the SARS-Cov-2 virus was detected were considered as true positive cases, either for saliva or swabs.

The percentage of the cases detected for each type of specimen was calculated, as well as for the cases where the virus was detected by both specimens at the same time. This data was organized in a table and the Fisher´s test (exact probability) was applied to evaluate the differences between the cases detected in each specimen. P-value <0.01 was considered statistically significant.

RT-PCR SARS-CoV-2 results were compared using a ROC curve analysis and MedCalc Statistical Software (version 13.3). The P-value <0.05 was statistically significant in the ROC curve comparisons. Individual analyses of each specimen were considered as an area of reliable detection if p <0.5 following the same procedure.

Ethical approval

The Institutional Ethics Board approved this study and informed consents were obtained from all patients.

Results

The median age of the patients was 46.9 years (with an age range between 18 to 87 years) and 12 (54.5%) were women. Of the 22 samples studied, 10 were positive to SARS-CoV-2 by RT-PCR detection (true positive group employing gold standard) for 45.5 % of non-negativization on the seventh day after admission.

The ten truly positive cases detected employing the gold standard proposed showed 30% 3 of coincidence with the cases detected in other specimens studied (Table 1). The comparison between detected cases in saliva samples and those spotted with the gold standard showed significant differences among detection numbers (p = 0.003).

Ten positive cases were detected by using swabs, while saliva samples allowed to detect three out of ten truly positive cases. Significant differences were found when comparing the detection of positive cases from saliva and swab specimens.

Table 1 True positive cases detected in each sample sets 

*Fisher’s exact test (p = 0.003)

Individual ROC curve analysis for each sample set

Saliva was the least sensitive specimen in terms of detection of truly positive cases (Table 2). The use of ROC curves yielded no significant differences from 0.5. Besides the gold standard, the most reliable specimens were nasopharyngeal swabs with an area under the curve of 0.950 which was significantly different from 0.5.

Table 2 ROC curve analysis 

Comparison of ROC curve between saliva specimen and oropharyngeal, nasopharyngeal and nasopharyngeal + oropharyngeal samples.

The comparison of the ROC curve of the specimens showed significant differences in the detection of cases. The difference between saliva specimens and the gold standard was of 0.350, with a significant level of 0.004. However, no significant differences were observed in the comparison between saliva and oropharyngeal swabs. The area under the curve was of 0.200 in that case (Table 3).

Table 3 Pairwise comparison of ROC curves 

Discussion

For the diagnosis of suspicious cases, a combination of clinical, epidemiological, imaging, and laboratory data are used.13 However, as discussed above, RT-PCR is a highly sensitive method that detects the unique genetic sequence SARS-CoV-2 RNA.7 Target genes include envelope (E), nucleocapsid (N), spike (S), transmembrane (M), helicase (Hel), open reading frame 1ab (ORF1ab), RNA-dependent RNA polymerase (RdRp), and hemagglutinin-esterase (HE).6 Despite that, the sensitivity of these tests can vary according to the moment of the natural cycle of the disease where the samples were taken, samples collection, and transport.6,7,8,9 The occurrence of false-negative RT-PCR results implies the decision to discharge SARS-CoV-2 positive patients from hospitals,13 resulting in possible transmission events.14

Since the detection of the first cases of COVID-19 in Cuba, the WHO recommendations for sample collection were followed. Nasopharyngeal + oropharyngeal swabs were collected to further detect viral RNA through RT-PCR.15) Combined oropharyngeal + nasopharyngeal swabs for each patient were taken at the same time and preserved together in one container. This procedure is recommended in order to increase the amount of viral material (optimal collection)15 as well as to guarantee optimal samples for diagnosis. These samples provide higher reliability for the detection of SARS-CoV-2 cases; for that reason, this procedure was proposed as the gold standard in the present study.

Despite nasopharyngeal or oropharyngeal swabs have been highly recommended as biological samples for SARS-CoV-2 testing, the use of saliva has been proposed as a way of testing suspects, particularly those who are not hospitalized.9,10 The reason for this recommendation is that SARS-CoV-2 can be present in saliva because of trespassing of secretions located in the upper respiratory tract. These secretions can pass from blood to the gingival crevicular liquid or to the colonization of major and minor salivary glands, with the subsequent liberation of viruses into saliva.11

The use of saliva has its advantages when compared with the swabbing process. Saliva can be easily collected and the process is pain-free and not invasive, as opposed to nasopharyngeal or oropharyngeal swabs which can generate bleeding, hematomas, and mucous erosion among other disadvantages. All of these are particularly problematic in patients with hemorrhagic conditions.11,16 Nevertheless, the recommended way of sample taking for the diagnosis of COVID-19 continues to be the hyssops use for swabbing.17

Several studies have shown contradictions. The inconsistency in the results might be related to differences in aspects such as the condition of patients (symptomatic/asymptomatic), the severity of the disease, or the methodology followed to collect saliva samples.18,19,20,21

Our research shows that, in terms of viability and reliability, swabs (Nasopharyngeal + Oropharyngeal) were capable of detect a greater amount of cases (10/22) compared to saliva. In turn, saliva samples only allowed detecting three cases, showing a very low sensibility of only 30 % of true positive cases.

Furthermore, regardless of the small sample size, the present study indicates that the sample obtained from nasopharyngeal swabs is the most reliable of the three specimen evaluated (nasopharyngeal, oropharyngeal and saliva) and compared to the gold standard.

The Center for Disease Control and Prevention (CDC) recommends taking samples for molecular diagnosis through nasopharyngeal or oropharyngeal swabs. The reason for this recommendation is that nasopharyngeal swabs are more suitable for the sample taking process, owing to the presence of a higher concentration of the virus.16 This way, the test sensitivity increases, and the likelihood of obtaining a false-negative result in a patient with the virus is reduced.15

Samples from oropharyngeal swabs showed no significant differences with saliva specimens. This result does not necessarily mean that the sample type is not reliable because the result can be biased with the small sample size. Nevertheless, the individual analysis of ROC curves indicated that the detection area was significantly different from 0.5, which is a criterion of validity and reliability of the analyzed samples.

According the present research, the most statistically reliable sample types were nasopharyngeal swabs and nasopharyngeal + oropharyngeal swabs as opposed to saliva. However, the best specimen for the detection of SARS-CoV-2 remains the nasopharyngeal + oropharyngeal swabs (the gold standard proposed).

The main limitation of the present study lies in the small number of positive cases included; a larger sample would make this study more robust.

Conclusion

Saliva is not a reliable specimen for SARS-CoV-2 RNA detection. In turn, the most reliable specimens are nasopharyngeal and nasopharyngeal + oropharyngeal samples collected by swabbing.

Acknowledgments

We acknowledge the “Dr. Luis Díaz Soto” General Hospital for patient recruitment and sampling. We also thank the patients who consented to participate in the study.

References

1. Wu F, Zhao S, Yu B. A new coronavirus associated with human respiratory disease in China. Nature [Internet]. 2020 [Cited 27/08/2020];579:265-9. Available in: Available in: https://doi.org/10.1038/s41586-020-2008-3Links ]

2. Singh S. COVID-19: A Deadly Virus. IJMSCR [Internet]. 2020 [Cited 27/08/2020];2(2):1-5. Available in: Available in: https://www.journalofmedical.com/archives/2020.v2.i2.30Links ]

3. World Health Organization. Coronavirus disease 2019 (COVID-19) [Internet]. Geneva: World Health Organization; 2020 [Cited 27/08/2020]. Available in: Available in: https://www.covid19.who.intLinks ]

4. Wang D, Hu B, Hu C, Zhu F, Liu X, Zhang J, et al. Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in Wuhan, China. JAMA [Internet]. 2020 [Cited 27/08/2020];323(11):1061-9. Available in: Available in: http://doi.org/10.1001/jama.2020.1585Links ]

5. Péré H, Podglajen I, Wack M, Flamarion E, Mirault T, Goudot G, et al. Nasal swab sampling for SARS-CoV-2: a convenient alternative in times of nasopharyngeal swab shortage. J Clin Microbiol [Internet]. 2020 [Cited 27/08/2020];58:e00721-20. Available in: Available in: http://doi.org/10.1128/JCM.00721-20Links ]

6. Tang YW, Schmitz JE, Persing DH, Stratton CW. Laboratory diagnosis of COVID-19: current issues and challenges. J Clin Microbiol [Internet]. 2020 [Cited 27/08/2020];(58):e00512-20. Available in: Available in: https://doi.org/10.1128/JCM.00512-20Links ]

7. Loeffelholz MJ, Tang YW. Laboratory diagnosis of emerging human coronavirus infections - the state of the art. Emerg Microbes Infect [Internet]. 2020 [Cited 27/08/2020];9(1):747-56. Available in: Available in: http://doi.org/10.1080/22221751.2020.1745095Links ]

8. Pan Y, Zhang D, Yang P, Poon LLM, Wang Q. Viral load of SARS-CoV-2 in clinical samples. Lancet Infect Dis [Internet]. 2020 [Cited 27/08/2020];20:[Aprox. 2p.]. Available in: Available in: http://doi.org/10.1016/S1473-3099(20)30113-4Links ]

9. Wang W, Xu Y, Gao R, Lu R, Han R, Wu G, et al. Detection of SARS-CoV-2 in Different Types of Clinical Specimens. JAMA [Internet]. 2020 [Cited 27/08/2020];323(18):1843-4. Available in: Available in: http://doi.org/10.1001/jama.2020.3786Links ]

10. Czumbel LM, Kiss S, Farkas N, Mandel I, Hegyi AE, Nagy AK, et al. Saliva as a Candidate for COVID-19 Diagnostic Testing: A Meta-Analysis Front Med[Internet]. 2020 [Cited 27/08/2020];7:465. Available in: Available in: http://doi.org/10.3389/fmed.2020.00465Links ]

11. Melián RA, Calcumil HP, Boin BC, Carrasco SR. Detección de COVID-19 (SARS-CoV-2) mediante la saliva: Una alternativa diagnóstica poco invasiva. Int J Odontostomat [Internet]. 2020 [Cited 27/08/2020];14(3):316-20. Available in: Available in: https://dx.doi.org/10.4067/S0718-381X2020000300316Links ]

12. Meng, L, Hua F, Bian Z. Coronavirus disease 2019 (COVID-19): Emerging and Future Challenges for Dental and Oral Medicine. J Dent Res [Internet]. 2020 [Cited 27/08/2020];99(5):481-7. Available in: Available in: https://www.doi.org/10.1177/0022034520914246Links ]

13. Lan L, Xu D, Ye G, Xia C, Wang S, Li Y, et al. Positive RT-PCR test results in patients recovered from COVID-19. JAMA [Internet]. 2020 [Cited 27/08/2020];323(15):1502-3. Available in: Available in: http://doi.org/10.1001/jama.2020.2783Links ]

14. Lio T, Gong D, Xiao J, Hu J, He G, Rong Z, et al. Cluster infections play important role in the rapid evolution of COVID-19 trasmission: A systematic review. Int J Infect Dis [Internet]. 2020 [Cited 27/08/2020];99:374-80. Available in: Available in: http://doi.org/10.1016/j.ijid2020.07.073Links ]

15. World Health Organization. Laboratory Guidelines for Detection and Diagnosis of the Novel Coronavirus (2019-nCov) Infection [Internet]. Geneva: World Health Organization; 2020 [Cited 27/08/2020]. Available in: Available in: https://www.paho.org/en/documents/laboratory-guidelines-detection-and-diagnosis-novel-coronavirus-2019-ncov-infectionLinks ]

16. Khurshid Z, Asiri FYI, Wadaani H. Human Saliva: Non-Invasive Fluid for Detecting Novel Coronavirus (2019-nCoV). Int J Environ Res Public Health [Internet]. 2020 [Cited 27/08/2020];17(7):2225. Available in: Available in: http://doi.org/10.3390/ijerph17072225Links ]

17. Center for Disease Control and Prevention. Interim Guideline for Collecting, Handling, and Testing Clinical Specimens for COVID-19 [Internet]. Atlanta: Center for Disease Control and Prevention; 2020 [Cited 27/08/2020]. Available in: Available in: https://www.cdc.gov/coronavirus/2019-ncov/lab/guideline-clinical-specimens.htmlLinks ]

18. Iwasaki S, Fujisawa S, Nakakubo S, Kamada K, Yamashita Y, Fukumoto T, et al. Comparison of SARS-CoV-2 detection in nasopharyngeal swab and saliva. J Infect [Internet]. 2020 [Cited 27/08/2020];81(2):e145-7. Available in: Available in: http://doi.org/10.1016/j.jinf.2020.05.071Links ]

19. Wyllie AL, Fournier J, Casanovas Massana A, Campbell M, Tokuyama M, Vijayakumar P, et al. Saliva is more sensitive for SARS-CoV-2 detection in COVID-19 patients than nasopharyngeal swabs. J Infect [Internet]. 2020 [Cited 27/08/2020];81(2):e148-9. Available in: Available in: http://doi.org/10.1016/j.inf.2020.05.071Links ]

20. Williams E, Bond K, Zhang B, Putland M, Williamson DA. Saliva as a non-invasive specimen for detection of SARS-CoV-2. J Clin Microbiol [Internet]. 2020 [Cited 27/08/2020];58(8):e00776-20. Available in: Available in: http://doi.org/10.1128/JMC.00776-20Links ]

21. Becker D, Sandoval E, Amin A. Saliva is less sensitive than nasopharyngeal swabs for COVID-19 detection in the community setting [Internet]. New York: MedRxiv; 2020 [Cited 27/08/2020] Available in: Available in: https://doi.org/10.1101/2020.05.11.20092338Links ]

Authors’ contributions

3CGL: Data curation; formal analysis; investigation; resources; validation; visualization; roles/writing - original draft; writing - review & editing.

4LGGL: Investigation.

5AIBC: Investigation; resources; validation; writing - review & editing.

6JRF: Investigation; methodology; resources; validation; visualization; writing - review & editing.

7HC: Investigation; methodology; resources; validation; writing - review & editing.

8DVB: Investigation; methodology; resources; validation; visualization; writing - review & editing.

9GEGN: Conceptualization; Investigation; methodology; project administration; resources; supervisión; validation; visualization; writing - review & editing.

Received: October 27, 2020; Accepted: April 22, 2021

*Corresponding author: chriss.losada@gmail.com

The authors declare that they have no conflict of interest.

No funding was received for this work.

All authors participated in the discussion of the results and have read, reviewed and approved the final text of the article.

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License