Performance Metrics
Contents
3.3. Performance Metrics#
In our encoder, the ECG signal is split into windows of \(n\) samples each which can be multiplied with a sensing matrix. Each window of \(n\) samples generates \(m\) measurements by the sensing equation \(\by = \Phi \bx\). Assume that we are encoding \(s\) ECG samples where \(s = n w\) and \(w\) is the number of signal windows being encoded. Let the ECG signal be sampled by the ADC device at a resolution of \(r\) bits per sample. For MIT-BIH Arrhythmia database, \(r=11\). Then the number of uncompressed bits is given by \(\bits_u = r s\).
3.3.1. Compression Ratio#
Let the total number of compressed bits corresponding to the \(s\) ECG samples be \(\bits_c\). This includes the overhead bits required for the stream header and frame headers to be explained later. Then the compression ratio (\(\compr\)) is defined as
Percentage space saving (\(\pss\)) is defined as
Note that often in literature, \(\pss\) is defined as compression ratio (e.g., [22]). Several papers ignore the bitstream formation aspect and report \(\frac{m}{n} \times 100\) (e.g., [37]) or \(\frac{n - m}{n} \times 100\) (e.g., [32]) as the compression ratio which measures the reduction in number of measurements compared to the number of samples in each window. We shall call this metric percentage measurement saving (\(\pms\)):
The ratio \(m/n\) will be called the measurement ratio:
The measurement ratio \(\frac{m}{n}\) is not a good indicator of compression ratio. If the sensing matrix \(\Phi\) is Gaussian, then the measurement values are real-valued. In literature using Gaussian sensing matrices (e.g., [37]), it is unclear how many bits are being used to represent each floating point measurement value for transmission. Under standard 32-bit IEEE floating point format, each value would require 32-bits. Then for MIT-BIH data, the compression ratio in bits would be \(\frac{11 \times n}{32 \times m}\). The only way the ratio \(\frac{m}{n}\) would make sense is if the measurements are also quantized at 11 bits resolution. However, the impact of such quantization is not considered in the simulations.
Now consider the case of a sparse binary sensing matrix. Since it consists of only zeros and ones, hence for integer inputs, it generates integer outputs. Thus, we can say that the output of a sparse binary sensor is quantized by design. However, the range of values changes. Assume that the sensing matrix has \(d\) ones per column. Then it has a total of \(n d\) ones. Thus, each row will have on average \(\frac{n d}{m}\) ones. Since the ones are randomly placed, hence we won’t have the same number of ones in each row. If we assume the input data to be in the range of \([-1024, 1023]\) (under 11-bit), then in the worst case, the range of output values may go up to\([-\frac{n d}{m} \times 1024, \frac{n d}{m} \times 1023]\). For a simple case where \(n = 2m\) and \(d=4\), we will require 14 bits to represent each measurement value. To achieve \(\frac{m}{n}\) as the compression ratio, we will have to quantize the measurements in 11 bits. If we do so, we shall need to provide some way to communicate the quantization parameters to the decoder as well as study the impact of quantization noise. This issue seems to be ignored in [35].
Another way of looking at the compressibility is how many bits per sample (\(\bps\)) are needed on average in the compressed bitstream. We define \(\bps\) as:
Since the entropy coder is coding the measurements rather than the samples directly, hence it is also useful to see how many bits are needed to code each measurement. We denote this as bits per measurement (\(\bpm\)):
3.3.2. Reconstruction Quality#
The normalized root mean square error is defined as
where \(\bx\) is the original ECG signal and \(\tilde{\bx}\) is the reconstructed signal. A popular metric to measure the quality of reconstruction of ECG signals is percentage root mean square difference (\(\prd\)):
The signal to noise ratio (\(\snr\)) is related to \(\prd\) as
As one desires higher compression ratios and lower \(\prd\), one can define a combined quality score (QS) as
Zigel et al. [38] established a link between the diagnostic distortion and the easy-to-measure \(\prd\) metric. Table 3.1 shows the classified quality and corresponding SNR (signal-to-noise ratio) and PRD ranges.
Quality |
PRD |
SNR |
|---|---|---|
Very good |
\(<\) 2% |
\(>\) 33 dB |
Good |
2-9% |
20-33 dB |
Undetermined |
\(\geq\) 9% |
\(\leq\) 20 dB |