Somayeh Fazaeli¹, Mehdi Yousefi^2,3,* and Mohsen Shokoohizadeh⁴

¹ Assistant Professor, Department of Health Information Technology, School of Paramedical Sciences and Rehabilitation, Mashhad University of Medical Sciences, Mashhad, Iran

² Associate Professor, Department of Health Economics and Management Science, Mashhad University of Medical Sciences, Mashhad, Iran

³ Associate Professor, Social Determinants of Health Research Center, Mashhad University of Medical Sciences, Mashhad, Iran

⁴ MSc in Health Information Technology, Imam Reza Hospital, Mashhad University of Medical Sciences, Mashhad, Iran

* Corresponding author: Mehdi Yousefi, Department of Health Economics and Management Science, Mashhad University of Medical Sciences, Mashhad, Iran. Tel: +989183449165; Email: yousefimh@mums.ac.ir

Received 2022 June 19; Revised 2022 July 15; Accepted 2022 November 04.

 

Abstract Background: Information dashboards are useful tools for up-to-date decision-making by visualizing data. Objectives: This study aimed to report the development of a dashboard in the emergency department (ED) during COVID-19 in a big hospital in Iran. Methods: The authors developed a dashboard by user-centered design (UCD) methodology in four phases, namely specification of the context of use, specification of the requirements, creation of design solutions, and evaluation. Indicators were determined by reviewing previous studies and interviewing focus groups with an expert panel. The Power BI Desktop software was used for the development of the dashboard. Users’ comments about the dashboard were collected. The dashboard was then developed and revised according to the users’ feedback and suggestions. Finally, user satisfaction was evaluated. Results: The authors identified 30 indicators for COVID-19 ED, classified as input, output, and process indicators. The final version of the dashboard was implemented in 2021, and then 28 ED and managerial staff participated in the evaluation of the dashboard. The average score of the system usability scale of the dashboard was 84.10 points, and the situation awareness index was 3.97, which indicates “good” usability and situation awareness. Conclusion: This dashboard presented key managerial and clinical indicators for decision-making in ED. Future studies can be designed to develop dashboards for accidents and burns EDs and create emergency information dashboards for several hospitals for better management in times of crisis.   Keywords: COVID-19, Dashboard, Emergency department, Hospitals

 

1. Background

The new coronavirus disease that originated in 2019 (COVID-19) created a new challenge for health systems (1) and posed a large threat and work overload to emergency departments (EDs) worldwide (2). Totally, in Iran, there have been 7,557,024 confirmed cases of COVID-19 with 144,559 associated deaths, as reported to the World Health Organization (3).

Hence, the COVID-19 pandemic caused a high influx of patients into hospitals, which greatly overstretched the provision of services to patients in these centers. (4, 5). In such circumstances, the optimal management of resources and workforce is one of the most critical responsibilities of hospital managers, who also need access to up-to-date evidence about patients and available resources
(6-9). This is the case especially in EDs where information systems are needed to support communication and care coordination (10). Many studies have used Information Technology (IT) for various purposes, including prediction, disease control, disease diagnosis, patient management, and equipment (10-12). Previous studies have investigated the usability of electronic patient tracking systems in EDs (13). However, there has not been a focus on designing an integrated system based on end-users’ views to facilitate clinical practice and help to manage EDs (14).

Among the uses of IT in healthcare organizations, especially in times of crisis, information dashboards have a special application and position due to their unique characteristics. The use of information dashboards in the ED allows the efficient management of information for the optimal organization of patients and equipment in EDs based on up-to-date evidence (15). In the emergency room, patients’ information from different departments, such as radiology and laboratory, is stored in information dashboards. In this method, it is possible to visually and instantaneously display patient information, services provided, equipment, and facilities available in the emergency room for managers and service providers, including physicians and nurses (16).

Moreover, it allows managers to have an overview of ward trends, be able to anticipate and meet ward needs shortly, and prevent overcrowding of patients and their dissatisfaction (17).

Due to the various capabilities of information dashboards, some of which were mentioned above, at the beginning of the spread of COVID-19, they were used in different countries for various purposes, such as making a diagnosis, treatment, and management of COVID-19 patients (18-21).

2. Objectives

The present study aimed to develop a dashboard in an ED during COVID-19 in a big hospital.

 

3. Methods

This is a descriptive developmental applied
study for designing and evaluating a COVID-19 ED dashboard in a major hospital. In the first stage, a team consisting of representatives of the emergency information dashboard stakeholders was formed, including four physicians, four nurses, two administrators, and three quality improvement team members. In addition, two dashboard designers and the hospital information system (HIS) managers were part of the team. This study employed the focus group with an expert panel to be undertaken in XXXX. It was conducted in an ED that recorded XXX patients’ turnover annually and used HIS. Since May 14, 2020, the team started visualizing the ED process. The visualization dashboard was designed and developed based on the needs of end-users through a user-centered design (UCD) process. The research team designed the user-centered visualization dashboard using the following steps, as outlined in (22):

1.     Context of use specification, which includes determining who will work with the dashboard, their purpose for using the dashboard, and the circumstances in which they use the dashboard.

2.     Requirements specification, which incorporates ascertaining the requirements that need to be determined and fulfilled for the successful implementation of the dashboard in the organization.

3.     Design solutions creation, which includes the design of various parts of the dashboard, from prototyping to completing the design.

4.     Design evaluation, which includes evaluating the usability of the dashboard and software testing.

Five meetings were held with the participation of stakeholders and dashboard design officials, each lasting about 45 to 80 min. In these meetings, which were held in the form of face-to-face meetings, the main stakeholders were asked through a semi-structured researcher-made questionnaire about their information needs in the ED, the use of this information for them, and the amount of necessary access to this information for each stakeholder. Afterward, index identification was compiled for all suggested indices in the focus group. At the end of these meetings, users’ gave their final comments on the indicators needed to better perform their duties in the department and the reason for requiring these indicators, and the amount of access to the indicators. Their comments were written, summarized, and voted using the nominal group technique. Indicators that received more than 75% of the votes were included in the dashboard. Additionally, the study team classified indicators into three classes of input, process, and output. The assistant in charge of HIS checked access to the data according to the information sources in the HIS. The prototype of the requested indicators was designed in the form of a dashboard and displayed in a session for all medical staff. If the indicators needed to be modified or merged, this was performed and after the final approval of the index, it was inserted in the final sample of the dashboard. The dashboard’s platform was developed and deployed on three servers with Windows 10, each with a 1200-GB hard drive, 30-GB memory, and two 4-core Intel Xeon 2.4 GHz (Gigahertz) processors.

Dashboard servers extracted the mentioned indicators from the HIS, picture archive, and communication system servers. A Windows communication base was used as the visualization tool. The dashboard prototype was developed using the Power BI Desktop software. During the design of the dashboard, feedback was constantly received from end-users and was applied to the dashboard during the following months.

 

3.1. Analysis

Dashboard users were evaluated to ensure the usability of the dashboard. The inclusion criteria for selection were currently working ED physicians, nurses, supervisors, and managerial staff. Final users were recruited from September 6 to October 6, 2021, and they responded to 20 questions on a questionnaire. The first 10 questions investigated the usability of the dashboard derived from the System Usability Scale (SUS) used in a study by Pal and Vanijja (23). Half of the questions had a positive tone (odd items), and the other half possessed a negative one (even items). Answers on the Likert scale ranged from 1 (completely disagree) to 5 (agree). The SUS score spectrum ranged from 0 as the worst score to 100 being the best score. Scores above 68 were considered above average level, while scores below 68 were considered below average level (23).

According to the answers, the average score for odd and even questions was calculated separately. In addition, the SUS score was calculated according to the following formula:

 

X=25-(Total of even questions), y= (Total of odd questions)-5. Then, SUS =(X+Y)*2.5.

Following this method, the highest score in this test was 100. The closer the number used as an adjective rating scale to interpret the SUS scores is to 100, the higher the score for the desired product (site or application) is. However, the average SUS score was 68. Scores below 68 indicate problems in the design that need more research to solve them as quickly as possible, while scores above 68 indicate the need for minor modifications in the design (24).

The next 11 questions were derived from the situation awareness index (SAI). The SA implies a person’s awareness regarding peculiar circumstances, which arise through their interaction with the environment. An adequate level of SA is known to affect subsequent decisions and actions positively (25). The SAI score was calculated according to the following formula:

 

SAI={Q11+Q12+Q13+Q14+Q15+Q16+(6-Q17)+Q18+Q19+Q20}/10, where Q is the question number (16).

 

3.2. Ethical Considerations

The study was conducted under the Helsinki Declaration. Verbal informed consent was obtained from the participants to participate in the study, and the confidentiality of the data was maintained.

 

4. Results

Table 1 presents the characteristics of the team members that participated in the evaluation of the dashboard. Table 2 illustrates the final indicators required to be included in the emergency dashboard according to the surveys conducted in the focus-group meetings with the presence of the research team. Figures 1 and 2 depict examples of the main dashboard page. In the second version, changes were made to increase the usability of the dashboard based on end-users’ opinions. Some of these changes included adding the number of staff

 

ED: Emergency department;  CPR: Cardiopulmonary resuscitation

 

 

present in the ED, a more understandable display of patients at each triage level, rearranging the indicators on the screen, patients waiting to be transferred to the ward, date and time of updating, and the current status of patients.

Based on the findings, the average scores for odd questions were 4.56, 4.77, 4.89, and 4.81, while those for even questions were 2.02, 2.04, 1.03, 3.01, and 2.09. The SUS was also calculated according to the following formula:

X=25-(Total of even questions=14.81), y=(Total of odd questions=23.83)-5. Then SUS=(X+Y)* 2.5=33.64*2.5=84.1 (Table 3).

The SUS indicates the "good-excellent" usability of the dashboard. It also shows that the users used it very frequently feeling the dashboard was easy to learn, and that final users were very confident in using the dashboard.

The overall SAI score was 3.97. The top five scaled items were “Concentration support”
(4.43 points), “Division of attention” (4.36), “Spare mental capacity support” (4.21), “Variability representation” (4.21), “Arousal support” (4.21), and “Information quantity provided” (3.6) (Table 4).

 

 

 

 

 

5. Discussion

The fast spread of COVID-19 forced healthcare managers to use IT to respond to rapidly changing needs. In this regard, this study used a multidisciplinary visualization team with a rapid UCD approach to develop and implement a dashboard system in ED for COVID-19 in a tertiary hospital. Proper design of information dashboards depends on careful attention to the main functions and performance indicators, which was addressed well in the current dashboard (20). The interviews resulted in the identification of indicators according to the key performance indicators of the ED and the opinion of experts in this department. The indicators included data on the number of patients referred to the triage and the ED, classified by severity, primary diagnosis, age, gender, and average waiting time. Patients received different services, the number of which were provided to them. Other information included bed occupancy rate, the number of ED personnel, as well as the number of discharged and deceased patients during the last 24 h, which were also used in other dashboards related to COVID-19 (12, 26, 27). The indicators in this dashboard were classified according to the classification used by Yoo et al. (16).

The use of information dashboards to manage information in epidemics has already been considered by researchers. For example, dashboards were designed for COVID-19 (28, 29) and severe acute respiratory syndrome (30), which focused on clinical data analysis. In the current dashboard design, both clinical and managerial data were considered, and the dashboard design was successfully implemented. The designed dashboard data was uploaded directly from the HIS without the need for manual data entry by the operator. The graphic and statistical concepts considered by the end-users were continuously identified and upgraded with the opinion of dashboard design experts, thereby obtaining the satisfaction of the end-users. This was done in parallel with Yoo et al.  (31) and Dixit et al. (20) studies. The use of HIS data as a dashboard data source has been emphasized in other studies (32, 33).

UCD has been frequently used in developing software to meet the user’s needs and goals and deliver a usable system. Furthermore, UCD is an approved approach that is increasingly adopted for E-Health projects (34). This study applied UCD methods to develop a dashboard that adherently engaged users. Drawing the initial (accurate) visualization for end-users is better in providing a holistic approach since users’ needs are quickly identified on the initial dashboards designed based on users’ needs and gradually become more complete based on user feedback (20).

This study also used SUS and obtained a high score from physicians, nurses, and managerial users of the dashboard. This high score could imply the ‘high acceptability’ with ‘good usability’ of this dashboard, which corroborates findings from a study conducted by Fareed et al. They developed a dashboard about infant mortality, and in their usability evaluation, the median task completion success rate was 83%, and the median system usability score was 68 (35). Similarly, in the present study, a high usability score was recorded. In another study, Bersani et al. reported that dashboards effectively improve patient care (36).

Based on previous experiences, failure to pay attention to the demands of end-users and their lack of participation in the development of software that is ultimately used by them will cause reluctance to use the relevant system (37, 38). Due to the importance of paying attention to users’ opinions in this study, after evaluating the dashboard, users’ comments were evaluated. The average SUS score of the dashboard indicated “good” usability. Further, the score of SA in this study was higher than that in the study by Yoo et al. (3.87) (31). In addition, the ED staff stated that the dashboard shows the status of the ED effectively and that they can better focus on changes in the ED by using the dashboard designed in the ED.

 

5.1. Limitations

The present study was conducted in an ED of a big hospital in one province of the country. Therefore, the findings in this study cannot be generalized to other departments or hospitals. Hence, studies that are more extensive in this regard are recommended. Another limitation is that the indicators used in the dashboard have been prepared using a stakeholder survey and may need to be revised for global application. These indicators are often considered by specialists in the very crowded emergency rooms of a COVID-19 teaching hospital. Therefore, there may be differences in the prioritization of the indicators when designing a dashboard for smaller hospital emergencies. Furthermore, this study did not evaluate the effect of the visualization dashboard on the effectiveness of care being provided to the patients.

 

6. Conclusion

Visualization of electronic health records in the form of dashboards is a salient supervision tool for health managers, which is handy in reaching out for valuable information required for making an evidence-based decision. Regarding clinical applications, knowing the statistics of patients with different degrees of disease and services that they have received or are waiting to receive adds to the accuracy of clinical decisions regarding specific services or prioritization for different patients. It also assists therapists in deciding and predicting the services needed ahead based on the statistics.

This dashboard is designed for the ED of a hospital that is responsible for admitting patients in the event of an epidemic crisis. If such a situation persists or similar cases of an epidemic occur, such dashboards are also fully usable. In addition, 80% of the indicators designed in this dashboard can be used in all hospital EDs in common situations.

The short time required to design dashboards and their high flexibility in meeting users’ information needs and personalizing it based on user feedback has been considered a suitable solution for managing information in times of crises, such as the COVID-19 pandemic. Finally, to develop a functional dashboard, it is necessary to receive frequent feedback from system users, keep their information needs up-to-date, and ensure the quality of dashboard information.

 

Footnotes

Conflicts of Interest: None.

Funding: This research is part of a large project (Grant Code No. 990315).

Authors’ Contributions: MY, SF, MSH: Concept and Study design. MSH, AA: Data collection. MY, MSH: Analysis. SF: Writing the article. MY, AA: Critical revision of the article. SF, MY, MSH, AA: Final approval of the article.

Ethical Approval: The study protocol was approved by the Research Ethics Committee of Tehran University of Medical Sciences, Tehran, Iran (Ethical code: IR.TUMS.NIHR.REC.1400.012).

 

References

1.      Yousef M, Fazaeli S, Masoudi S, Arfae-Shahidi N, Ebrahimi Z, Shokoohizade M, et al. Comparative study of the physician specialty and laboratory services requested for patients with COVID-19: evidence from a large hospital. J Isfahan Med Sch. 2021;39(610):2021.

2.      Mantica G, Riccardi N, Terrone C, Gratarola A. Non-COVID-19 visits to emergency departments during the pandemic:
the impact of fear. Public Health. 2020;183:40-1. doi: 10.1016/j.puhe.2020.04.046. [PubMed: 32417567].

3.      WHO. WHO Coronavirus (COVID-19) Dashboard. 2022. Available from : https://covid19.who.int/table

4.      Sayadi M, Moghbeli F, Mehrjoo H, Mahaki M. A Linear Study of the Spread of COVID19 in China and Iran. J Frontiers in Health Informatics. 2020;9(1):32.

5.      Yousefi M, Arani AA, Sahabi B, Kazemnejad A, Fazaeli S. Household health costs: Direct, indirect and intangible. Iran J Public Health. 2014;43(2):202-9. [PubMed: 26060744].

6.      Bong C-L, Brasher C, Chikumba E, McDougall R, Mellin-Olsen J, Enright A, et al. The COVID-19 pandemic: effects on low-and middle-income countries. Anesth Analg. 2020;131(1):86-92. doi: 10.1213/ANE.0000000000004846. [PubMed: 32243287].

7.      Hartnett KP, Kite-Powell A, DeVies J, Coletta MA, Boehmer TK, Adjemian J, et al. Impact of the COVID-19 pandemic on emergency department visits—United States, January 1, 2019–May 30, 2020. MMWR Morb Mortal Wkly Rep. 2020;69(23):699-704. doi: 10.15585/mmwr.mm6923e1. [PubMed: 32525856].

8.      Ebrahimipour H, Olyani S, Rezazadeh A, Khorsand Vakilzadeh A, Fazaeli S, Jafari M, et al. Effect of" Iran's health system evolution" and" tariff change" based on relative values book on performance of obstetrics and gynecology department: a case study in a big hospital. Iran J Obstet Gynecol Infertil. 2017;20(8):15-25. doi: 10.22038/IJOGI.2017.9586.

9.      Fazaeli S, Khodaveisi T, Vakilzadeh AK, Yousefi M, Ariafar A, Shokoohizadeh M, et al. Development, Implementation, and User Evaluation of COVID-19 Dashboard in a Third-Level Hospital in Iran. Appl Clin Inform. 2021;12(05):1091-100. doi: 10.1055/s-0041-1740188. [PubMed: 34879405].

10.   Selck FW, Decker SL. Health information technology adoption in the emergency department. Health Serv Res. 2016;51(1):32-47. doi: 10.1111/1475-6773.12307. [PubMed: 25854423].

11.   Hu Z, Ge Q, Li S, Boerwinkle E, Jin L, Xiong M. Forecasting and Evaluating Multiple Interventions for COVID-19 Worldwide. Front Artif Intell. 2020;3:1-11. doi: 10.3389/frai.2020.00041. [PubMed: 33733158].

12.   Wong A, Otles E, Donnelly JP, Krumm A, McCullough J, DeTroyer-Cooley O, et al. External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Intern Med. 2021;181(8):1065-1070. doi: 10.1001/jamainternmed.2021.2626. [PubMed: 34152373].

13.   Guarrera TK, McGeorge NM, Clark LN, LaVergne DT, Hettinger ZA, Fairbanks RJ, et al. Cognitive engineering design of an emergency department information system. Cognitive systems engineering in healthcare Boca Raton, FL: CRC Press, Taylor & Francis; 2015.

14.   Wang X, Kim TC, Hegde S, Hoffman DJ, Benda NC, Franklin ES, et al. Design and evaluation of an integrated, patient-focused electronic health record display for emergency medicine. Appl Clin Inform. 2019;10(04):693-706. doi: 10.1055/s-0039-1695800. [PubMed: 31533171].

15.   Almasi S, Rabiei R, Moghaddasi H, Vahidi-Asl M. Emergency department quality dashboard; a systematic review of performance indicators, functionalities, and challenges. Arch Acad Emerg Med. 2021;9(1):e47-e. doi: 10.22037/aaem.v9i1.1230. [PubMed: 34405145].

16.   Bhar Layeb S, Omrane Aissaoui N, Hamouda C, Zeghal F, Moujahed H, Zaidi A. Performance indicators and dashboard for an emergency department of a teaching hospital. Tunis Med. 2021;99(4):435-40. [PubMed: 35244928].

17.   Randell R, Alvarado N, McVey L, Ruddle RA, Doherty P, Gale C, et al., editors. Requirements for a quality dashboard: Lessons from National Clinical Audits. AMIA Annu Symp Proc. 2020;2019:735-44. [PubMed: 32308869].

18.   Mukhtar H, Ahmad HF, Khan MZ, Ullah N. Analysis and Evaluation of COVID-19 Web Applications for Health Professionals: Challenges and Opportunities. Healthcare (Basel). 2020;8(4):1-26. doi: 10.3390/healthcare8040466. [PubMed: 33171711].

19.   Ivanković D, Barbazza E, Bos V, Brito Fernandes Ó, Jamieson Gilmore K, Jansen T, et al. Features Constituting Actionable COVID-19 Dashboards: Descriptive Assessment and Expert Appraisal of 158 Public Web-Based COVID-19 Dashboards. J Med Internet Res. 2021;23(2):e25682-e. doi: 10.2196/25682. [PubMed: 33577467].

20.   Dixit RA, Hurst S, Adams KT, Boxley C, Lysen-Hendershot K, Bennett SS, et al. Rapid development of visualization dashboards to enhance situation awareness of COVID-19 telehealth initiatives at a multihospital healthcare system. J Am Med Inform Assoc. 2020;27(9):1456-61. doi: 10.1093/jamia/ocaa161.

21.   Ignatenko E, Ribeiro M, Oliveira MD. Informing the design of data visualization tools to monitor the COVID-19 pandemic
in Portugal: A Web-Delphi participatory approach. Int
J Environ Res Public Health. 2022;19(17):11012. doi: 10.3390/ijerph191711012. [PubMed: 36078728].

22.   Setyawan MYH, Awangga RM, Afriyanti R. Dashboard settings design in SVARA using user-centred design method. J Telkomnika. 2019;17(2):615-9.

23.   Pal D, Vanijja V. Perceived usability evaluation of Microsoft Teams as an online learning platform during COVID-19 using system usability scale and technology acceptance model
in India. Child Youth Serv Rev. 2020;119:1-13. doi: 10.1016/j.childyouth.2020.105535. [PubMed: 33020676].

24.   Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: Adding an adjective rating scale. J Usability Stud. 2009;4(3):114-23.

25.   Nadj M, Maedche A, Schieder C. The effect of interactive analytical dashboard features on situation awareness and task performance. DSS. 2020;135:1-13. doi: 10.1016/j.dss.2020.113322.

26.   Ponce M, Sandhel A. covid19. analytics: An R Package to Obtain, Analyze and Visualize Data from the 2019 Coronavirus Disease Pandemic. J Open Source Softw. 2021;6(60):1-5. doi: 10.21105/joss.02995

27.   Vijay Anand V, Arunkumar Yogaraj G, Priya S, Priya Raj P, Brinda Priyadharshini C, Sridevi PN. A cross-sectional study on COVID19 mortality among people below 30 years of age in Tamilnadu-2020. Clin Epidemiol Glob Health. 2021;12:1-6. doi: 10.1016/j.cegh.2021.100827. [PubMed: 34230902].

28.   Florez H, Singh S. Online dashboard and data analysis approach for assessing COVID-19 case and death data. F1000Res. 2020;9:1-13. doi: 10.12688/f1000research.24164.1. [PubMed: 32884676].

29.   Dong E, Du H, Gardner LJTLid. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis. 2020;20(5):533-4. doi: 10.1016/S1473-3099(20)30120-1. [PubMed: 32087114].

30.   Campbell TC, Mistry ZS, Gorelick-Feldman GN, Hodanics CJ, Babin SM, Lewis SH. Development of the respiratory disease dashboard for the identification of new and emerging respiratory pathogens. Johns Hopkins APL Tech Dig. 2014;32(4):726-34.

31.   Yoo J, Jung KY, Kim T, Lee T, Hwang SY, Yoon H, et al. A real-time autonomous dashboard for the emergency department: 5-year case study. JMIR Mhealth Uhealth. 2018;6(11):e10666. doi: 10.2196/10666. [PubMed: 30467100].

32.   Buttigieg SC, Pace A, Rathert C. Hospital performance dashboards: a literature review. J Health Organ Manag. 2017;31(3):385-406. doi: 10.1108/JHOM-04-2017-0088. [PubMed: 28686130].

33.   Rouhani S, Zamenian S. An architectural framework for healthcare dashboards design. Journal of Healthcare Engineering. 2021;2021:1-12. doi: 10.1155/2021/1964054.

34.   Cornet VP, Toscos T, Bolchini D, Ghahari RR, Ahmed R, Daley C, et al. Untold stories in user-centered design of mobile health: Practical challenges and strategies learned from the design and evaluation of an app for older adults with heart failure. JMIR Mhealth Uhealth. 2020;8(7):e17703. doi: 10.2196/17703. [PubMed: 32706745].

35.   Fareed N, Swoboda CM, Jonnalagadda P, Griesenbrock T, Gureddygari HR, Aldrich A. Visualizing opportunity index data using a dashboard application: a tool to communicate infant mortality-based area deprivation index information. Appl Clin Inform. 2020;11(04):515-27. doi: 10.1055/s-0040-1714249. [PubMed: 32757202].

36.   Bersani K, Fuller TE, Garabedian P, Espares J, Mlaver E, Businger A, et al. Use, perceived usability, and barriers to implementation of a patient safety dashboard integrated within a vendor EHR. Appl Clin Inform. 2020;11(01):034-45. doi: 10.1055/s-0039-3402756.

37.   Anantham D, Chai-Lim C, Zhou JX, Phua GC. Operationalization of critical care triage during a pandemic surge using protocolized communication and integrated supportive care. J Intensive Care. 2020;8(1):1-9.

38.   Aristizábal-Torres D, Peñuela-Meneses CA, Barrera-Rodríguez AM. An interactive web-based dashboard to track COVID-19 in Colombia. Case study: five main cities. Rev Salud pública. 2020;22(2):214-9.