Document Type : Research articles


1 Associate Professor of Counseling, Faculty of Educational Sciences and Psychology, University of Mohaghegh Ardabili, Ardabil, Iran

2 PhD in Counseling, Department of Counseling, Faculty of Educational Sciences and Psychology, University of Mohaghegh Ardabili, Ardabil, Iran

3 Department of Statistics, Faculty of Science, University of Mohaghegh Ardabili, Ardabil, Iran

4 PhD Candidate of Counseling, Department of Counseling, Faculty of Educational Sciences and Psychology, University of Mohaghegh Ardabili, Ardabil, Iran

5 PhD Candidate of Counseling, Department of Counseling, Faculty of Educational Sciences and Psychology, University of Isfahan, Isfahan, Iran


Background: Considering the scope of the epidemic of the COVID-19 pandemic and the psychological consequences known to be associated with infection, it seems necessary to provide a screening tool for determining the presence of psychological symptoms among the population in the healthcare sector for prevention and timely provision of psychological interventions.
Objectives: This study aimed to design and standardize a comprehensive screening test and assess clinical aspects of psychological symptoms associated with COVID-19.
Methods: This is a descriptive survey-based study. The study was permitted in two phases. The statistical population included all physicians, nurses, and staff who worked in COVID-19 care wards of hospitals in Ardabil (Iran) in 2021 and were in direct contact with these patients. A total of 200 participants in Phase I and 98 in Phase II were selected via the purposive sampling method. An initial researcher-made questionnaire was used to assess the psychological symptoms of the participants in eight and seven categories in Phase I and Phase II, respectively. The data were analyzed using SPSS26 and R software.
Results: Delphi method and second-order confirmatory factor analysis verified the validity of the questionnaire. Findings related to measuring the reliability of the questionnaire in phase I showed that although the questionnaire was reliable based on inter-rater and intra-rater, the test-retest reliability method based on Cohen's kappa coefficients showed no reliability for items 12 and 49-54. In phase II, the unreliable items were removed and the study was permitted to be conducted again with new samples. The results of intra-rater reliability also demonstrated that an intraclass correlation coefficient for each of the seven scales of the questionnaire was greater than 0.75. In addition, the results of inter-rater reliability showed that Cronbach's alpha coefficient for each scale of the questionnaire was greater than 0.70. Furthermore, high amounts of sensitivity and specificity as well as high area under the receiver operating characteristic curve verified the good accuracy of the final questionnaire.
Conclusion: Based on the findings, it can be said that the final proposed questionnaire with seven scales (55 items) was a tool with good validity, reliability, and accuracy to assess the psychological symptoms associated with COVID-19. However, since the participants of this study consisted of only medical personnel, the generalization of the results to the general population needs further investigation.


  1. Grover S, Sahoo S, Mehra A, Avasthi A, Tripathi A, Subramanyan A, et al. Psychological impact of COVID-19 lockdown: an online survey from India. Indian J Psychiatry. 2020;62(4):354.‏
  2. doi: 10.4103/psychiatry.indianJPsychiatry_427_20. [PubMed: 33165368].
  3. Ustun G. Determining depression and related factors in a society affected by COVID-19 pandemic. Int J Soc Psychiatry. 2021;67(1):54-63. doi: 10.1177/0020764020938807. [PubMed: 32605422].
  4. Lee SA. Coronavirus anxiety scale: a brief mental health screener for COVID-19 related anxiety. Death Stud. 2020;44(7):393-401. doi: 10.1080/07481187.2020.1748481. [PubMed: 32299304]‏.
  5. Qiu J, Shen B, Zhao M, Wang Z, Xie B, Xu Y. A nationwide survey of psychological distress among Chinese people in the COVID-19 epidemic: implications and policy recommendations. Gen Psychiatr. 2020;33(2):e100213. doi: 10.1136/gpsych-2020-100213. [PubMed: 32215365].
  6. Ausin B, Gonzalez-Sanguino C, Castellanos MÁ, Muñoz M. Gender-related differences in the psychological impact of confinement as a consequence of COVID-19 in Spain. J Gend Stud. 2021;30(1):29-38. doi: 10.1080/09589236.2020.1799768.
  7. Vadivel R, Shoib S, El Halabi S, El Hayek S, Essam L, Bytyçi DG, et al. Mental health in the post-COVID-19 era: challenges and the way forward. Gen Psychiatr. 2021;34(1):e100424. doi: 10.1136/gpsych-2020-100424. [PubMed: 33644689].
  8. Tang F, Liang J, Zhang H, Kelifa MM, He Q, Wang P. COVID-19 related depression and anxiety among quarantined respondents. Psychol Health. 2021;36(2):164-78. doi: 10.1080/08870446.2020.1782410. [PubMed: 32567952].
  9. Shah K, Kamrai D, Mekala H, Mann B, Desai K, Patel RS. Focus on mental health during the coronavirus (COVID-19) pandemic: applying learnings from the past outbreaks. Cureus. 2020;12(3):e7405. doi: 10.7759/cureus.7405. [PubMed: 32337131].
  10. Bao Y, Sun Y, Meng S, Shi J, Lu L. 2019-nCoV epidemic: address mental health care to empower society. Lancet. 2020;395(10224):37-8. doi: 10.1016/S0140-6736(20)30309-3. [PubMed: 32043982].
  11. Chen Q, Liang M, Li Y, Guo J, Fei D, Wang L, et al. Mental health care for medical staff in China during the COVID-19 outbreak. Lancet Psychiatry. 2020;7(4):15-6. doi: 10.1016/S2215-0366(20)30078-X. [PubMed: 32085839].
  12. Ryu S, Chun BC, of Epidemiology KS. An interim review of the epidemiological characteristics of 2019 novel coronavirus. Epidemiol Health. 2020;42:e2020006. doi: 10.4178/epih.e2020006. [PubMed:32023775].
  13. Tian F, Li H, Tian S, Yang J, Shao J, Tian C. Psychological symptoms of ordinary Chinese citizens based on SCL-90 during the level I emergency response to COVID-19. Psychiatry Res. 2020;288:112992. doi: 10.1016/j.psychres.2020.112992. [PubMed: 32302816].
  14. Wang Q, Xu R, Volkow ND. Increased risk of COVID‐19 infection and mortality in people with mental disorders: analysis from electronic health records in the United States. World Psychiatry. 2021;20(1):124-30. doi: 10.1002/wps.20806. [PubMed: 33026219]‏.
  15. Sarailoo M, Matin S, Vosoughi M, Dargahi A, Gholizadeh H, Damavandi MR, et al. Investigating the relationship between occupation and SARS-CoV2. Work. 2021;68(1):27-32. doi: 10.3233/WOR-205066. [PubMed: 33459683].
  16. Alalawi M, Makhlouf M, Hassanain, O, Abdelgawad AA, Nagy M. Healthcare workers’ mental health and perception towards vaccination during COVID-19 pandemic in a pediatric cancer hospital. Sci Rep. 2023;13(1):329. doi: 10.1038/s41598-022-24454-5. [PubMed: 36609572].
  17. Scott HR, Stevelink SA, Gafoor R, Lamb D, Carr E, Bakolis I, et al. Prevalence of post-traumatic stress disorder and common mental disorders in health-care workers in England during the COVID-19 pandemic: a two-phase cross-sectional study.
  18. Lancet Psychiatry. 2023;10(1):40-9. doi: 10.1016/S2215-0366(22)00375-3. [PubMed: 36502817]. ‏
  19. Antonijevic J, Binic I, Zikic O, Manojlovic S, Tosic‐Golubovic S, Popovic N. Mental health of medical personnel during the COVID‐19 pandemic. Brain Behav. 2020;10(12):e01881. doi: 10.1002/brb3.1881. [PubMed:33070475]. ‏
  20. Spoorthy MS, Pratapa SK, Mahant S. Mental health problems faced by healthcare workers due to the COVID-19 pandemic–A review. Asian J psychiatry. 2020;51:102119. doi: 10.1016/j.ajp.2020.102119. [PubMed: 32339895].
  21. Moradi MH, Khademi R. Covid-19 & Psychopathology: drawing the scientific map of international researches in the web of science citation databases. J Clin Psychol. 2021;13(2):23-32.
  22. Damirchi ES, Mojarrad A, Pireinaladin S, Grjibovski AM. The role of self-talk in predicting death anxiety, obsessive-compulsive disorder, and coping strategies in the face of coronavirus disease (COVID-19). Iranian J psychiatry. 2020;15(3):182-8. doi: 10.18502/ijps.v15i3.3810. [PubMed: 33193766].
  23. Zandian H, Sarailoo M, Dargahi S, Gholizadeh H, Dargahi A, Vosoughi M. Evaluation of knowledge and health behavior of university of medical sciences students about the prevention of COVID-19. Work. 2021;68(3):543-9. doi: 10.3233/WOR-203395. [PubMed: 33612505].
  24. Keith TZ. Multiple regression and beyond: An introduction to multiple regression and structural equation modeling. Routledge; 2014.
  25. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas.1960;20(1):37-46. doi: 10.1177/001316446002000104.
  26. Lee SA. Coronavirus Anxiety Scale: A brief mental health screener for COVID-19 related anxiety. Death stud. 2020;44(7):393-401. doi: 10.1080/07481187.2020.1748481. [PubMed: 32299304]
  27. Lorenz L, Bachem RC, Maercker A. The adjustment disorder–new module 20 as a screening instrument: Cluster analysis and cut-off values. Int J Occup Environ Med. 2016;7(4):215-20. doi: 10.15171/ijoem.2016.775. [PubMed: 27651082].
  28. Potvin L, Lasker J, Toedter L. Measuring grief: a short version of the perinatal grief scale. J Psychopathol Behav Assess. 1989;11(1):29-45. doi: 10.1007/BF00962697.
  29. Woody SR, Steketee G, Chambless DL. Reliability and validity of the Yale-Brown obsessive-compulsive scale. Behav Res Ther. 1995;33(5):597-605. doi: 10.1016/0005-7967(94)00076-v. [PubMed: 7598684].
  30. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86(2):420-8. doi: 10.1037//0033-2909.86.2.420. [PubMed: 18839484].
  31. Weir JP. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond
  32. Res. 2005.19(1):231-40. doi: 10.1519/15184.1. [PubMed: 15705040].
  33. Anderson RM, Heesterbeek H, Klinkenberg D, Hollingsworth TD. How will country-based mitigation measures influence the course of the COVID-19 epidemic? Lancet. 2020;395(10228):931-4. doi: 10.1016/S0140-6736(20)30567-5. [PubMed: 32164834].
  34. Mrklas K, Shalaby R, Hrabok M, Gusnowski A, Vuong W, Surood S, et al. Prevalence of perceived stress, anxiety, depression, and obsessive-compulsive symptoms in health care workers and other workers in Alberta during the COVID-19 pandemic: cross-sectional survey. JMIR Mental Health. 2020;7(9):e22408. doi: 10.2196/22408. [PubMed: 32915764].
  35. Nikopoulou VA, Holeva V, Parlapani E, Karamouzi P, Voitsidis P, Porfyri GN, et al. Mental health screening for COVID-19: A proposed cutoff scores for the Greek version of the Fear of COVID-19 Scale (FCV-19S). Int J Ment Health Addict. 2020;1-14. doi: 10.1007/s11469-020-00414-w. [PubMed: 33199975].
  36. Seçer İ, Ulaş S, Karaman-Özlü Z. The effect of the fear of COVID-19 on healthcare professionals’ psychological adjustment skills: mediating role of experiential avoidance and psychological resilience. Front Psychol. 2020;11:561536. doi: 10.3389/fpsyg.2020.561536. [PubMed:33192830].
  37. Wańkowicz P, Szylińska A, Rotter I. Assessment of mental health factors among health professionals depending on their contact with COVID-19 patients. Int J Environ Res Public Health. 2020;17(16):5849. doi: 10.3390/ijerph17165849 ‏. [PubMed: 32806699].
  38. Lee SA, Neimeyer RA. Pandemic Grief Scale: A screening tool for dysfunctional grief due to a COVID-19 loss. Death Stud. 2022; 46(1):14-2. doi: 10.1080/07481187.2020.1853885. [PubMed: 33349159].
  39. Caycho-Rodriguez T, Vilca LW, Vivanco-Vidal A, Saroli-Aranibar D, Carbajal-Leon C, Gallegos WL, et al. Assessment of dysfunctional grief due to death from COVID-19 in Peru: adaptation and validation of a Spanish version of the pandemic grief scale. Trends in Psychol. 2021;29(4):595-616. doi: 10.1007/s43076-021-00091-1.
  40. Rabow MW, Huang CH, White-Hammond GE, Tucker RO. Witnesses and victims both: Healthcare workers and grief in the time of COVID-19. J Pain Symptom Manage. 2021;62(3):647-56. doi: 10.1016/j.jpainsymman.2021.01.139. [PubMed: 33556494].