MCS Index Verification Statistics

            The performance of the MCS index is assessed using a dichotomous forecasting technique where a forecast is given (i.e., yes or no) based on a selected threshold value of the MCS index (-1.5, 0, & 1.5 chosen here) and whether or not a MCS is observed.  From the resulting 2x2 contingency table, numerous measures of forecast accuracy and skill can be calculated (see below).

            To verify the performance of the MCS index, IR satellite data are used to identify MCSs (see Jirak et al. 2003).  Any contiguous cloud cluster larger than 30,000 km² that persists for at least 3 h at the -52°C IR brightness temperature threshold is recorded as a MCS (i.e., a or c in the contingency table).  The value of the MCS index is examined for every MCS cloud pixel and is categorized as a correct forecast (i.e., a) if the value is larger than the selected threshold value and as an incorrect forecast (i.e., c) if the value is smaller than the threshold value.

            All of the remaining IR pixels are classified as being non-MCS (i.e., b or d).  Since the MCS index is defined as being contingent upon convective initiation, the non-MCS pixels are screened for convective areas, where convection is defined as any cloud pixel <-50°C.  Thus, any IR pixels warmer than -50°C (i.e., non-convective areas) are categorized as a correct “no” forecast (i.e., d).  The remaining pixels are convective, but are not MCSs.  They are categorized as an incorrect forecast (i.e., b) if the MCS index is larger than the threshold value and as a correct forecast (i.e., d) if the value is smaller than the threshold value.

2x2 Contingency Table

Heidke Skill Score (HSS) – :  <0-1 with 1 being best

Threat Score (TS) – :  0-1 with 1 being best

False Alarm Ratio (FAR) – :  0-1 with 0 being best

Hit Rate or Proportion Correct – :  0-1 with 1 being best

Probability of detection (POD) – :  0-1 with 1 being best 

Bias – :  <1 –  underforecasting, >1 – overforecasting