IRCMJ logo


Identification of the Key Performance Indicators for Designing an Emergency Department Dashboard in a Referral Cardiac Hospital

Mahnaz MayelAfshar1, Feridoun Noohi2, Leila Riahi3,* and Aniseh Nikravan3

  1. Ph.D. Candidate, Department of Health Services Administration, Sciences and Research Branch, Islamic Azad University, Tehran, Iran
  2. Professor of Cardiology, Rajaie Cardiovascular Medical and Research Center, Iran University of Medical Sciences, Tehran, Iran
  3. Associate Professor, Department of  Health Services Administration, Sciences and Research Branch, Islamic Azad University, Tehran, Iran

* Corresponding author: Leila Riahi, Department of Health Services Administration, Sciences and Research Branch, Islamic Azad University, Tehran, Iran. Tel: 09121131901; Email: [email protected]

 

Received 2021 January 31; Revised 2021 June 05; Accepted 2021 July 25.

 

Abstract

Background: Today, organization management in healthcare organizations needs to monitor and evaluate performance for better decision and policy making.

Objectives: This study aimed at determining the Key Performance Indicators (KPIs) using software and a management dashboard.

Methods: This study searched several articles discussing KPIs of emergency departments. A comprehensive list of indicators was obtained and presented to an expert panel with a wide range of experiences. The experts finalized the KPIs. A second round was performed to confirm the performance using Smart Pilates software. Based on the final panel’s rating, a list of KPIs was developed to be used. The extracted data was prepared to be entered into the computer system to design the dashboard using QlickView software. Subsequently, according to the type of indicator, the dashboard was designed with special software.

Results: The extracted 14 KPIs of emergency departments were determined in three dimensions of input, process, and output. Following that, the project team designed a dashboard with 14 KPIs.

Conclusion: To design and develop a dashboard, the management of information was essential for organizations. It is recommended that managers use KPIs for evaluating and monitoring emergency departments. Moreover, it can be used for planning and evaluating the performance in emergency departments.

 

Keywords: Designing dashboard, Emergency departments, Key performance indicators, Performance measurement


1. Background

Managers and decision-makers are always faced with the problem of choosing the best indicator for performance evaluation; therefore, a great deal of effort is required to collect large amounts of data for indicators. These efforts lead to administrative fatigue and mass information in hospitals (1). Furthermore, hospitals admit many critical patients through the emergency departments (2), and they are responsible for stabilizing patients' vital signs for admission to other wards (3).

Overcrowding in the emergency department and inefficient performance are the most concerning obstacles in getting emergency patients to access emergency services in a timely manner. The patients who visit the emergency department often have long waiting times to receive services (4). Emergency care service is an integrated platform for providing the patients with health services (time-sensitive) (5). In addition, many hospital emergency departments are affected by the organizational change (6), and their service requires emergency personnel, medical professionals, and resources (1). Assurance of rapid and accurate diagnosis, access to the necessary and required care, and organized emergency care system saves people and strengthen its impact on the other parts of the health system (4). The emergency department plays a key role in improving quality, reducing costs, and preventing rework. Therefore, it is highly valuable to review and evaluate its performance (7, 8). Evaluation and measurement of key performance indicators (KPIs) to improve and enhance performance are also vital and necessary (1, 9). The measurement of the emergency department indicators is very important, and the use of health information systems may be effective in increasing the effectiveness of the emergency department (10). It seems that the solutions to overcome these problems can be provided via technologies (11). In recent years, a variety of management tools for digital monitoring of essential data have been introduced under the name of the dashboard (12).

 


2.Objectives

Dashboard technology can help managers to make decisions by creating a link among different information systems (13). Moreover, it is a way to monitor performance indicators that immediately collect data from various resources (14). A management dashboard is used for monitoring the purposes of core operational processes in real-time. These dashboards trigger automatic alerts or tasks when predetermined thresholds are exceeded. The importance of real-time is to ensure that users are alerted to review the dashboard to oversee and fix problems as they occur (15, 16). It is undeniable that dashboards are one of the essential requirements for organizational performance as a tool for monitoring performance in order to diagnose and respond to problems. The present study aimed at identifying the KPIs for the emergency department and design an emergency department management dashboard.


3.Methods

3.1. Design and setting of the study

A descriptive cross-sectional study was carried out to identify the KPIs for the emergency department; furthermore, a management dashboard of the emergency department was designed in this regard. The first phase is related to the collection of KPI metrics for the emergency department; accordingly, a review of past studies, as well as referring to reputable databases, scientific and research articles, and Ph.D. dissertations were conducted in this study. The KPIs were developed and sorted into three Emergency Room (ER) patient Donabedian conceptual model, including input, process, and output (8, 9). Finally, the related articles were selected, and 30 key quality indicators were extracted (supplemental Table 1) by eliminating duplicate and unrelated indicators.

The study protocol was approved by the Ethics Committee of the Department of Health Services Administration, Sciences and Research Branch, Islamic Azad University, Tehran, Iran (Ethical code: IR.IAU.SRB.REC.1398. 025).

To measure the validity and reliability of the questionnaire, the expert opinions (including 30 faculty members of Shahid Rajaei Hospital, emergency physicians, supervisors, and head nurses) confirmed and determined the content validity of the KPIs. Regarding reliability, Cronbach's alpha coefficient was used (0.89) in SPSS software (version 20) indicating the acceptable reliability of the research questionnaire. Afterward, the content validity index (which was higher than 0.79) and content validity ratio (which was higher than 0.42) were measured, and the number of items in the questionnaire was determined. The final questionnaire included 14 KPIs. In this phase, 14 finalized questions from the first stage were rated on a 5-point Likert scale (supplemental Table 2) (17).

 

3.2. Participants

The statistical population included health professionals, process owners, managers, policy-makers, and nurses. Random sampling was used for sampling. Assuming the fact that the statistical method was used in data analysis for exploratory factor analysis, in order to obtain reliable results, the volume of the research sample was considered 5 to 10 times the number of questionnaire items. As a result, the number of samples was estimated at 70-140 cases. Accordingly, 150 questionnaires were distributed manually and electronically. After two weeks, 110 questionnaires were returned to the researcher, some of which were incomplete, and only 75 questionnaires were completed correctly.

 

Table 1.initial KPI Title

Age:                                          Work experience:

Education level:                         Organizational department:

Relationship 2

Importance1

KPI Title

 

 

 

 

 

 

 

 

 

Percentage of Leaving Before Screening

1

 

 

 

 

 

 

 

 

Average Bed to Doctor Time

2

 

 

 

 

 

 

 

 

Return to ED

3

 

 

 

 

 

 

 

 

Average ER Patients Acuity Level

4

 

 

 

 

 

 

 

 

Average ER Consultation Requests Turnaround Time (1-2level)

5

 

 

 

 

 

 

 

 

Average ER Consultation Requests Turnaround Time (3-4 level)

6

 

 

 

 

 

 

 

 

Average ER Radiology Requests Turnaround Time (min)

7

 

 

 

 

 

 

 

 

Average ER Lab Requests Turnaround Time (min)

8

 

 

 

 

 

 

 

 

Average ER Medications Requests Turnaround Time (min)

9

 

 

 

 

 

 

 

 

Patient’s Waiting Time to See a Physician First (5 level triage)

10

 

 

 

 

 

 

 

 

Length of Stay-All ER Patients (hours)

11

 

 

 

 

 

 

 

 

Percentage of ER Patients with LOS More than 6 hours

12

 

 

 

 

 

 

 

 

Percentage of Leaving Without Being Seen Mohan

13

 

 

 

 

 

 

 

 

Number of Medications Prescribed

14

 

 

 

 

 

 

 

 

Number of Tests Ordered

15

 

 

 

 

 

 

 

 

Number of Images Ordered

16

 

 

 

 

 

 

 

 

Patients Discharged from the Emergency Department within 12 Hours

17

 

 

 

 

 

 

 

 

Percentage of Discharge against Medical Advice

18

 

 

 

 

 

 

 

 

Patients Decided within 6 Hours

19

 

 

 

 

 

 

 

 

Percentage of Employee Satisfaction (in Emergency Room)

20

 

 

 

 

 

 

 

 

Percentage of Patient Satisfaction (in Emergency Room)

21

 

 

 

 

 

 

 

 

Bed Occupancy Rate

22

 

 

 

 

 

 

 

 

Percentage of Unsuccessful CPR

23

 

 

 

 

 

 

 

 

Total Number of ER Visits

24

 

 

 

 

 

 

 

 

Ratio of Daily ER Patients to ER Beds

25

 

 

 

 

 

 

 

 

Morbidity/Mortality

26

 

 

 

 

 

 

 

 

ED Mortality

27

 

Table 2. KPI Title

Impact*

key performance indicators

5

4

3

2

1

 

 

 

 

 

Average ER consultation Requests Turnaround Time

 

 

 

 

 

Average ER Radiology Requests Turnaround Time(min)

 

 

 

 

 

Average ER Lab Requests Turnaround Time(min)

 

 

 

 

 

Average ER Medications Requests Turnaround Time(min)

 

 

 

 

 

Average ER consultation Requests Turnaround Time

 

 

 

 

 

Length of Stay – All ER Patients(hours)

 

 

 

 

 

Percentage of ER Patients with LOS More than 6 hours

 

 

 

 

 

Patients discharged from the emergency department within 12 hours

 

 

 

 

 

Percentage of Discharge against Medical Advice

 

 

 

 

 

Patients decided within 6 hours

 

 

 

 

 

Employee satisfaction( in Emergency Room)

 

 

 

 

 

Patient satisfaction( in Emergency Room)

 

 

 

 

 

Total Number of ER Visits

 

 

 

 

 

Average ER Patients Acuity Level

* The highest impact of the item on the performance measurement was scored 5 and the lowest impact was scored 1.

 

Table 3. Kaiser-Meyer-Olkin Measure of sampling adequancy

Kaiser-Meyer-Olkin Measure of sampling adequancy

df

Approx. chi-square

sig

0.714

91

456.565

0.000

 

3.3. Statistical analysis

In order to investigate the existence of the necessary conditions for factor analysis, the Kaiser-Meyer-Olkin (KMO) sampling size adequacy variable was used. The test measures sampling adequacy for each variable in the model and the complete model (supplemental Table 3). The KMO value was obtained at 0.714 indicating that the sampling was adequate.

As the KMO index is greater than 0.6, the Bartlett test is significant at a 95% confidence level; as a result, factor analysis is allowed.

Initially, exploratory factor analysis (supplemental Table 4) and validation of Partial Least Squares Analysis (PLS) were used to obtain KPIs for emergency department management in SPSS software (version 20). The final model was extracted applying confirmatory factor analysis and approved using Smart PLS software (version 2) (18).

According to Table 4 (Supplemental Table 4), after conducting exploratory factor analysis, the research findings showed that 14 indicators were loaded with three factors, and significant paths were established based on the researcher's expectations and the content of the questions. Smart PLS software was utilized for practical confirmatory analysis. As shown in Figure 1. (Supplemental Figure 1) suitable factor loads are greater than 0.5. Subsequently, bootstrapping test was performed to evaluate the significance of factor loads and path coefficients. At a 95% confidence level, the relationship is significant if the value of the T statistic is greater than 1.96 (supplemental Figure 2).

 

Figure 1. Display factor loads on the model

 

Table 4. Factor loads of categorized questionnaire indicators

Main Components

key performance indicators

3

2

1

.673

 

 

Average ER consultation Requests Turnaround Time

.892

 

 

Average ER Radiology Requests Turnaround Time(min)

.773

 

 

Average ER Lab Requests Turnaround Time(min)

.886

 

 

Average ER Medications Requests Turnaround Time(min)

.763

 

 

Average ER consultation Requests Turnaround Time

.772

 

 

Length of Stay – All ER Patients(hours)

.782

 

 

Percentage of ER Patients with LOS More than 6 hours

 

.695

 

Patients discharged from the emergency department within 12 hours

 

.777

 

Percentage of Discharge against Medical Advice

 

.608

 

Patients decided within 6 hours

 

.559

 

Employee satisfaction( in Emergency Room)

 

.714

 

Patient satisfaction( in Emergency Room)

 

 

.715

Total Number of ER Visits

 

 

.598

Average ER Patients Acuity Level

 

3.4. Definition and determining color coding for indicators

Table 5 shows the definition and refinement of 14 indicators. Moreover, it indicates that critical and main indicators are altered by color coding (red, yellow, and green) to show the levels of threats. These alerts were defined based on performance target thresholds which are derived from the yearly goals and objectives. Thresholds were also set for each metric in the executive panel and defined target, warning, and trouble areas in green, yellow, and red, respectively. It should be noted that the updating information based on the users’ view, type of use, and importance of tasks were defined daily (17). The extracted data were prepared to enter the computer system to design the dashboard. Afterward, the dashboard was designed using QlikView software according to the type of indicator.

3.5. Design dashboard

The data were then extracted from the above systems in the form of Excel reports and then designed in QlikView software, which is an open-source software for data integration that can be implemented in hospital operating systems. To design the dashboard according to the type and function of the indicators, the information was extracted from the selected hospital information system in 2018 (Figure 3). At this stage, different levels of performance are identified with different colors. Accordingly, green, yellow, and red are considered for proper, warning, and problematic  

 

Figure 2. Bootstrapping test path coefficients

 

Table 5. Key performance indicators of emergency department management

Green

Yellow

Red

Key Performance Indicator Title

Categories

>90

70-90

0-70

Total number of ER* visits

Input

0-100

100-200

>200

Average ER patients acuity level

0-30

30-60

>60

(level 1) (s)

Average ER consultation request turnaround time

Process

0-60

60-120

>120

(level 2) (s)

0-30

30-60

>60

Average ER radiology request turnaround time (min)

0-75

75-150

>150

Average ER lab request turnaround time (min)

0-25

25-50

>50

Average ER medication request turnaround time (min)

0-30

30-60

>60

(level 1) (s)

Patient’s waiting time to see a physician first [5 level triage]

1-5

5-10

>10

(level 2) (min)

1-20

20-40

>40

(level 3) (min)

0-30

30-60

>60

(level 4) (min)

0-60

60-120

>120

(level 5) (min)

0-4

4-8

>8

Length of stay-all ER patients (hours)

0-35

35-70

70-100

Percentage of ER patients with a length of stay more than 6 hours

60-100

30-60

0-30

Patients discharged from the emergency department within 12 hours

Output

0-3

3-6

>6

Percentage of discharge against medical advice

70-100

40-70

40-0

Patients decided within 6 hours

70-100

30-70

0-30

Percentage of employee satisfaction (in Emergency Room)

70-100

30-70

0-30

Percentage of patient satisfaction (in Emergency Room)

*ER: Emergency Room

 

Figure3. Emergency department dashboard in a referral cardiac hospital

 

performance areas, respectively (14, 17). The prototype of the emergency department management dashboard was executed using data from the hospital's information system and paper documents. Dashboards can be defined based on the type of use and the level of user access, or the use can be on a monthly or weekly basis. The design of the management dashboard of the emergency department in the hospital contributes to the design of innovative solutions and improving the level of training and services through displaying hospital information in the form of a dashboard. All methods were carried out in accordance with relevant guidelines and regulations of Iran.


4.Results

4.1. Findings related to the first objective:

The KPIs of emergency department management of the hospital under study were recognized based on a review of studies of countries, articles, and models. Emergency performance indicators are one of the most important performance management indicators in a hospital. At this stage, KPIs of the emergency department were selected according to a review of previously conducted studies and preliminary studies (Table 5).

4.2. Findings related to the second objective:

The features of dashboard designing were determined for the emergency department management in a selected hospital in Tehran, Iran. In order to design the emergency management dashboard according to the pattern, items, such as the purpose of the design, time frame, observing security points, and access, as well as display quality were considered. Subsequently, the technical infrastructure was determined as follows:

 

4.3. Technical infrastructure means:

●The department manager and people involved in the management of the department were considered the target group.

●The purpose of designing a dashboard was determined as improvement in productivity, performance, and service quality of the emergency department.

●According to the available data, the below key characteristics were selected for displaying on the dashboard.

4.3.1. Other cases to consider:

●The source of the data needed to feed the dashboard was the hospital information system.

●The information was extracted from the above systems in the form of Excel reports.

●Since the dashboard prototype uses past information, it is monthly; however, daily reports can be extracted from the data.

●The database was designed using Excel software.

 


5.Discussion

According to the results, 14 indicators (input [n=2], process [n=7], and output [n=5]) were selected for the KPI of hospitals. To ensure high-quality care services, practical and effective measures are needed. Previous studies used various methodologies, such as DEMATEL Approach (19), RAND Delphi (20-23), mixed-method study (reporting of observation studies) (9, 24), and AHP technique (25) to identify the KPIs for the emergency department. However, this study used quantity tools to identify Emergency Department KPI and design a dashboard. What is certain is that it is important to choose the type of indicators that should be on the dashboard. Depending on its circumstances, each center needs to measure its own performance indicators, not all indicators. According to Donabedian conceptual model which provides a framework for evaluating healthcare services, the KPIs can be grouped by being related to one of the three components of the healthcare system, namely input, process, and output (26). Many researchers have investigated the three main groups of input, process, and output for categorizing the KPIs (9, 25, 27). Some components for input KPIs include a total number of patients (9, 28), return to ED (25, 29), and an average number of ER Staff (9, 30). On the other hand, other studies suggested a total number of ER visits (9, 28, 31) and an average ER patients' acuity level (9, 32), which is supported by the present study.

The majority of these KPIs are focused on the process. The indicators selected by some studies include average bed to doctor time (9), number of medications prescribed (33), number of tests ordered (28, 31, 34), number of images ordered (33, 34), bed occupancy rate (34, 35), a ratio of daily ED patients to ED beds (9), and average ED consultation requests turnaround time (3-4 level) (36). In addition, some other factors for the process were the length of stay-all ED patients (28, 29, 34, 37-40), the patient’s waiting time to see a physician first (24, 31, 34, 37, 38, 41), percentage of ED patients with Length Of Stay more than 6 hours (24, 37, 38), average ED consultation requests turnaround time (36-38), average ED radiology requests turnaround time (36-38), average ED lab requests turnaround time (36, 37), and average ED medications requests turnaround time (9). The present study confirms that the majority of previous researches consisted of process KPIs.

According to the studies reviewed, the third group is output KPIs. Some output KPIs components include the percentage of unsuccessful cardiopulmonary resuscitation (31, 37, 38), morbidity (29, 34), ED mortality (29, 34, 41), and percentage of leaving without being visited (10, 34, 42). Furthermore, some other suggestions by other studies for output KPIs included the percentage of employee satisfaction (in the emergency department) (24), percentage of patient satisfaction (in the emergency department) (24, 43, 44), patients decided within 6 hours (28, 37, 38), percentage of discharge against medical advice (30, 36, 37, 40), and patient discharged from the emergency department within 12 hours (31, 37, 38, 41). It is worth mentioning that the present study supports this group.

Selection and determination of the type of indicators that should be included in the dashboard are very important and depends on its situation (45). In addition, the key indicators must be meaningful, manageable, and measurable. Additionally, they should be categorized for manageability. Accordingly, the use of information technology to evaluate and improve the emergency department is a key tool. The dashboard is a business intellectual tool for controlling KPI which can extract key data from different systems and may be useful in real-time with easier reading for users (46). The dashboard can be applied in ED as used in other sections, such as operation rooms (47), hospital infection control (48), and MIDs (14). Therefore, the dashboard is a tool for quick, concise, and instantaneous display of data, and it must be organized in such a way to be easily read and interpreted at a glance.  


6.Conclusion

In determining the KPIs for the emergency department, it should be acknowledged that the emergency department is an area that is constantly growing; therefore, the emergency department needs to increase the volume and efficiency of its actions. The achievement of these goals requires regular monitoring of performance and better management of workflow processes in the department. In this regard, the use of performance indicators in the emergency department is a key tool to evaluate and improve processes since these indicators can identify department activities that have a negative impact on the quality of services provided, improve poor work processes and customer satisfaction, and formulate expenditure reduction strategies; as a result, the organization's revenue may increase.

Given the identification of the KPIs for the management performance of the emergency department in various input-process-output dimensions that have led to designing emergency department management dashboard, the information in the emergency department management dashboard can be a basis for making informed decisions to identify benefits, such as identifying best performance, making faster decisions, reducing errors, improving capacity management and workflow, allocating resources, and planning for growth and development. Since the initial model of the emergency management dashboard was designed and implemented in this study, only the first type of evaluation was performed (i.e., confirmation of the fact that the dashboard has the requirements to design and determine the indicators). However, the process of evaluating dashboards is dynamic and expanding. Considering that the environment of the emergency department is data-driven and technology-based along with having a variety of activities, dashboard technology is also evolving. Certainly, the results of this study will be subject to change in the future and will need to be improved and upgraded.


Acknowledgments

The authors acknowledge emergency department practitioners of this hospital for discussing their expertise and approving the findings of this study.


Footnotes

Authors’ contributions: Mahnaz Mayelafshar: Conception and design, analysis, interpretation of data, and drafting manuscript

Feridoun Noohi: Administrative, technical, or material support.

Leila Riahi: Acquisition of data, analysis and interpretation of data, and statistical analysis.

Aniseh Nikravan: Critical revision of the manuscript for important intellectual content, supervision.

Conflicts of Interest: The authors declare that there is no conflict of interest.

Ethics approval: The study protocol was approved by the Department of Health Services Administration, Sciences and Research Branch, Islamic Azad University, Tehran, Iran (Ethical code: IR.IAU. SRB.REC.1398.025)

Funding/Support: Nil.

Financial Disclosure: None declared.

Informed consent: All study respondents were provided with a detailed information regarding the purpose of the study. This form was sent via email as an attachment, or printed copies of the questionnaire and delivered to the respondents. 


References

  1. Mayelafshar M, Noohi F, Riahi L, Nikravan A. Key performance indicators of emergency department: a literature review. Iran J Cardiovascular Nurs. 2019;8(1):10-9.
  2. Danish Health and Medicines Authority. Islands Brygge: Styrket akutberedskab-planlægningsgrundlag for det regionale sundhedsvæsen; 2007.
  3. Mollaei S, Bahadori M, Ameryoun A. The factors affecting the refusal to report medical errors on medical errors among nurses of selected military hospitals in Tehran, Iran. J Mil Med. 2018;20(3):255-64.
  4. Higginson I. Emergency department crowding. Emerg Med J. 2012;29(6):437-43. doi: 10.1136/emermed-2011-200532. [PubMed: 22223713].
  5. World Health Organization. Emergency care systems for universal health coverage: ensuring timely care for the acutely ill and injured. Geneva: World Health Organization; 2019. P. 1-6.
  6. Castro GG, Gaviria SM, Berrio YE, Mendez JA. Efficacy of high fidelity clinical simulation in the learning of ITLS trauma assessment. International Trauma Conference, Quebec, Canada; 2017.
  7. Emamgholipour S, Jaafaripooyan E, Mohammadshahi M, Mohammadi Yazani E. The effect of Health Sector Evolution Plan on the performance indices of emergency department in hospitals of Tehran & Iran universities of medical sciences: interrupted time series analysis. Iran J Emerg Med. 2018;5:e9.
  8. Mirhoseinie M, Ziadloo D, Nasirie N, Sayerinia A. Knowledge and attitude of health care workers providing services in Kerman University of Telemedicine. Tehran, Iran: Bioresonanse and Modern Medical Diagnoses; 2014.
  9. Khalifa M, Zabani I. Developing emergency room key performance indicators: what to measure and why should we measure it? Stud Health Technol Inform. 2016;226:179-82. [PubMed: 27350498].
  10. Ajami S, Bagheri-Tadi T. Health information technology and quality of care. J Inform Tech Soft Engg. 2013;7:e003.
  11. Zhang J, Lu X, Nie H, Huang Z, van der Aalst WM. Radiology information system: a workflow-based approach. Int J Comput Assist Radiol Surg. 2009;4(5):509-16. doi: 10.1007/s11548-009-0362-6. [PubMed: 20033534].  
  12. Mansoori B, Novak RD, Sivit CJ, Ros PR. Utilization of dashboard technology in academic radiology departments: results of a national survey. J Am Coll Radiol. 2013;10(4):283-8. doi: 10.1016/j.jacr.2012.09.030. [PubMed: 23545086].
  13. Morgan MB, Branstetter BF 4th, Mates J, Chang PJ. Flying blind: using a digital dashboard to navigate a complex PACS environment. J Digit Imaging. 2006;19(1):69-75. doi: 10.1007/ s10278-005-8732-2. [PubMed: 16249835].
  14. Karami M, Safdari R. From information management to information visualization: development of radiology dashboards. Appl Clin Inform. 2016;7(2):308-29. doi: 10.4338/ ACI-2015-08-RA-0104. [PubMed: 27437043].
  15. Swoyer S. In praise of dashboard ubiquity. Woodland Hills: The Data Warehouse Institute; 2006.
  16. Eckerson W. Embedded analytics: Closing the loop between operational and analytical applications. Woodland Hills: The Data Warehouse Institute; 2006.
  17. Dargahi H, Torabi M, Safdari R, Goodarzi M, Bayat M. Designing dashboard: an innovation for effective management of medical laboratory. Lab Diagn. 2018;9(38):44-52.
  18. Safari R, Rajabzadeh A, Memarpour M. Statistics applications with SPSS, LISREL, SmartPLS software. Tehran: Negah Danesh Publication; 2014.
  19. Si SL, You XY, Liu HC, Huang J. Identifying key performance indicators for holistic hospital management with a modified DEMATEL approach. Int J Environ Res Public Health. 2017; 14(8):934. doi: 10.3390/ijerph14080934. [PubMed: 28825613].
  20. Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One. 2011; 6(6):e20476. doi: 10.1371/journal.pone.0020476. [PubMed: 21694759].
  21. Schull MJ, Guttmann A, Leaver CA, Vermeulen M, Hatcher CM, Rowe BH, et al. Prioritizing performance measurement for emergency department care: consensus on evidence-based quality of care indicators. CJEM. 2011;13(5):300-9. doi: 10.2310/8000.2011.110334. [PubMed: 21955411].
  22. Wakai A, O'Sullivan R, Staunton P, Walsh C, Hickey F, Plunkett PK. Development of key performance indicators for emergency departments in Ireland using an electronic modified-Delphi consensus approach. Eur J Emerg Med. 2013;20(2):109-14. doi: 10.1097/MEJ.0b013e328351e5d8. [PubMed: 22382650].  
  23. Madsen MM, Eiset AH, Mackenhauer J, Odby A, Christiansen CF, Kurland L, et al. Selection of quality indicators for hospital-based emergency care in Denmark, informed by a modified-Delphi process. Scand J Trauma Resusc Emerg Med. 2016;24:11. doi: 10.1186/s13049-016-0203-x. [PubMed: 26843014]
  24. Rahimi H, Kavosi Z, Shojaei P, Kharazmi E. Key performance indicators in hospital based on balanced scorecard model. J Health Manag Inform. 2017;4(1):17-24.
  25. Nikjoo RG, Beyrami HJ, Jannati A. Selecting hospital's key performance indicators, using analytic hierarchy process technique. J Community Health Res. 2013;2(1):30-8.
  26. Rademakers J, Delnoij D, de Boer D. Structure, process or outcome: which contributes most to patients' overall assessment of healthcare quality? BMJ Qual Saf. 2011; 20(4):326-31. doi: 10.1136/bmjqs.2010.042358. [PubMed: 21339310].
  27. Martinez DA, Kane EM, Jalalpour M, Scheulen J, Rupani H, Toteja R, et al. An electronic dashboard to monitor patient flow at the johns hopkins hospital: communication of key performance indicators using the Donabedian model. J Med Syst. 2018;42(8):133. doi: 10.1007/s10916-018-0988-4. [PubMed: 29915933].
  28. Stokes-Buzzelli S, Peltzer-Jones JM, Martin GB, Ford MM, Weise A. Use of health information technology to manage frequently presenting emergency department patients. West J Emerg Med. 2010;11(4):348-53. [PubMed: 21079707].
  29. Lindsay P, Schull M, Bronskill S, Anderson G. The development of indicators to measure the quality of clinical care in emergency departments following a modified-delphi approach. Acad Emerg Med. 2002;9(11):1131-9. doi: 10.1111/j.1553-2712.2002.tb01567.x. [PubMed: 12414461].
  30. Schneider SM, Gallery ME, Schafermeyer R, Zwemer FL. Emergency department crowding: a point in time. Ann Emerg Med. 2003;42(2):167-72. doi: 10.1067/mem.2003.258. [PubMed: 12883503].
  31. Parish B, Sherkati S, Hasanzade N, Omidi M, Alizade M. The indices provided by the ministry of health and medical education in Shahid Madani emergency department and strategies to improve it. Tabriz: Tabriz University of Medical Sciences; 2013.
    1. Horwitz LI, Green J, Bradley EH. US emergency department performance on wait time and length of visit. Ann Emerg Med. 2010;55(2):133-41. doi: 10.1016/j.annemergmed.2009.07.023. [PubMed: 19796844].
  32. Selck FW, Decker SL. Health information technology adoption in the emergency department. Health Serv Res. 2016;51(1):32-47. doi: 10.1111/1475-6773.12307. [PubMed: 25854423].
  33. Sørup CM, Jacobsen P, Forberg JL. Evaluation of emergency department performance - a systematic review on recommended performance and quality-in-care measures. Scand J Trauma Resusc Emerg Med. 2013;21:62. doi: 10.1186/1757-7241-21-62. [PubMed: 23938117].
  34. Forster AJ, Stiell I, Wells G, Lee AJ, van Walraven C. The effect of hospital occupancy on emergency department length of stay and patient disposition. Acad Emerg Med. 2003;10(2):127-33. doi: 10.1111/j.1553-2712.2003.tb00029.x. [PubMed: 12574009].
  35. Zohoor AR, Pilevar Zadeh M. Study of speed of offering services in emergency department at Kerman Bahonar hospital in 2000. Razi J Med Sci. 2003;10(35):413-9.
  36. Ministry of Health and Medical Education. Emergency department national indicators. Tehran: Ministry of Health and Medical Education; 2018.
  37. Baratloo A, Rahmati F, Forouzanfar MM, Hashemi B, Motamedi M, Safari S. Evaluation of performance indexes of emergency department. Iran J Emerg Med. 2015;2(1):33-8.
  38. Australian Capital Territory Auditor General Office. Health directorate. Performance audit report; emergency department performance information. Canberra, Australia: Australian Capital Territory Auditor General Office; 2012.
  39. Esteki R, Attafar A. Quality of nursing services (contemporary level of reality and level of expectation) from nurses' viewpoint on the basis of SERVQUAL Model in Al-Zahra Hospital in Isfahan (2010). Modern Care J. 2012;9(2):5206.
  40. McNatt Z, Linnander E, Endeshaw A, Tatek D, Conteh D, Bradley EH. A national system for monitoring the performance of hospitals in Ethiopia. Bull World Health Organ. 2015; 93(10):719-726. doi: 10.2471/BLT.14.151399. [PubMed: 26600614].
  41. Mohan MK, Bishop RO, Mallows JL. Effect of an electronic medical record information system on emergency department performance. Med J Aust. 2013;198(4):201-4. doi: 10.5694/ mja12.10499. [PubMed: 23451963].
  42. Diakogiannis A. Improving the performance of emergency departments: a survey from an operations management perspective. Ecole centrale Paris: Laboratoire Génie Industriel; 2014.
  43. Ajami S, Rezaee M, Taadi TB. Effect of Information Technology on ED’s Performance Indicators. J Inform Tech Softw Eng. 2016;6(181):2. doi: 10.4173/2165-7866.1000181.
  44. Ontario Hospital Association. Ontario hospital association 2013– 2016 strategic plan: a catalyst for change. Toronto, Canada: Ontario Hospital Association; 2016.
  45. Stevens A. Establishing quality indicators for medical imaging and the basic quality management toolbox. Durham, NC: eRADIMAGING; 2007
  46. Park KW, Smaltz D, McFadden D, Souba W. The operating room dashboard. J Surg Res. 2010;164(2):294-300. doi: 10.1016/j.jss.2009.09.011. [PubMed: 20006345].
  47. Blais R, Champagne F, Rousseau L. TOCSIN: a proposed dashboard of indicators to control healthcare-associated infections. Healthc Q. 2009;12:161-7. doi: 10.12927/hcq. 2009.20985. [PubMed: 19667795].