3.3. Performance Metrics

In our encoder, the ECG signal is split into windows of $n$ samples each which can be multiplied with a sensing matrix. Each window of $n$ samples generates $m$ measurements by the sensing equation $\mathbf{y}=\mathrm{\Phi }\mathbf{x}$. Assume that we are encoding $s$ ECG samples where $s=nw$ and $w$ is the number of signal windows being encoded. Let the ECG signal be sampled by the ADC device at a resolution of $r$ bits per sample. For MIT-BIH Arrhythmia database, $r=11$. Then the number of uncompressed bits is given by ${\mathrm{bits}}_u=rs$.

3.3.1. Compression Ratio

Let the total number of compressed bits corresponding to the $s$ ECG samples be ${\mathrm{bits}}_c$. This includes the overhead bits required for the stream header and frame headers to be explained later. Then the compression ratio ($\mathrm{CR}$) is defined as

$$\mathrm{CR}\triangleq \frac{{\mathrm{bits}}_u}{{\mathrm{bits}}_c}.$$

Percentage space saving ($\mathrm{PSS}$) is defined as

$$\mathrm{PSS}\triangleq \frac{{\mathrm{bits}}_u-{\mathrm{bits}}_c}{{\mathrm{bits}}_u}\times 100.$$

Note that often in literature, $\mathrm{PSS}$ is defined as compression ratio (e.g., [22]). Several papers ignore the bitstream formation aspect and report $\frac{m}{n}\times 100$ (e.g., [37]) or $\frac{n-m}{n}\times 100$ (e.g., [32]) as the compression ratio which measures the reduction in number of measurements compared to the number of samples in each window. We shall call this metric percentage measurement saving ($\mathrm{PMS}$):

$$\mathrm{PMS}\triangleq \frac{n-m}{n}\times 100.$$

The ratio $m/n$ will be called the measurement ratio:

$$\mathrm{MR}=\frac{m}{n}.$$

The measurement ratio $\frac{m}{n}$ is not a good indicator of compression ratio. If the sensing matrix $\mathrm{\Phi }$ is Gaussian, then the measurement values are real-valued. In literature using Gaussian sensing matrices (e.g., [37]), it is unclear how many bits are being used to represent each floating point measurement value for transmission. Under standard 32-bit IEEE floating point format, each value would require 32-bits. Then for MIT-BIH data, the compression ratio in bits would be $\frac{11\times n}{32\times m}$. The only way the ratio $\frac{m}{n}$ would make sense is if the measurements are also quantized at 11 bits resolution. However, the impact of such quantization is not considered in the simulations.

Now consider the case of a sparse binary sensing matrix. Since it consists of only zeros and ones, hence for integer inputs, it generates integer outputs. Thus, we can say that the output of a sparse binary sensor is quantized by design. However, the range of values changes. Assume that the sensing matrix has $d$ ones per column. Then it has a total of $nd$ ones. Thus, each row will have on average $\frac{nd}{m}$ ones. Since the ones are randomly placed, hence we won’t have the same number of ones in each row. If we assume the input data to be in the range of $[-1024,1023]$ (under 11-bit), then in the worst case, the range of output values may go up to$[-\frac{nd}{m}\times 1024,\frac{nd}{m}\times 1023]$. For a simple case where $n=2m$ and $d=4$, we will require 14 bits to represent each measurement value. To achieve $\frac{m}{n}$ as the compression ratio, we will have to quantize the measurements in 11 bits. If we do so, we shall need to provide some way to communicate the quantization parameters to the decoder as well as study the impact of quantization noise. This issue seems to be ignored in [35].

Another way of looking at the compressibility is how many bits per sample ($\mathrm{bps}$) are needed on average in the compressed bitstream. We define $\mathrm{bps}$ as:

$$\mathrm{bps}\triangleq \frac{{\mathrm{bits}}_c}{s}.$$

Since the entropy coder is coding the measurements rather than the samples directly, hence it is also useful to see how many bits are needed to code each measurement. We denote this as bits per measurement ($\mathrm{bpm}$):

$$\mathrm{bpm}\triangleq \frac{{\mathrm{bits}}_c}{mw}.$$

3.3.2. Reconstruction Quality

The normalized root mean square error is defined as

$$\mathrm{N}\mathrm{RMSE}(\mathbf{x},\tilde{\mathbf{x}})\triangleq \frac{\Vert \mathbf{x}-\tilde{\mathbf{x}}{\Vert }_2}{\Vert \mathbf{x}{\Vert }_2}$$

where $\mathbf{x}$ is the original ECG signal and $\tilde{\mathbf{x}}$ is the reconstructed signal. A popular metric to measure the quality of reconstruction of ECG signals is percentage root mean square difference ($\mathrm{PRD}$):

$$\mathrm{PRD}(\mathbf{x},\tilde{\mathbf{x}})\triangleq \mathrm{N}\mathrm{RMSE}(\mathbf{x},\tilde{\mathbf{x}})\times 100$$

The signal to noise ratio ($\mathrm{SNR}$) is related to $\mathrm{PRD}$ as

$$\mathrm{SNR}\triangleq -20{\log }_{10}(0.01\mathrm{PRD}).$$

As one desires higher compression ratios and lower $\mathrm{PRD}$, one can define a combined quality score (QS) as

$$\text{QS}=\frac{\mathrm{CR}}{\mathrm{PRD}}\times 100.$$

Zigel et al. [38] established a link between the diagnostic distortion and the easy-to-measure $\mathrm{PRD}$ metric. Table 3.1 shows the classified quality and corresponding SNR (signal-to-noise ratio) and PRD ranges.

Table 3.1 Quality of Reconstruction

Quality	PRD	SNR
Very good	$<$ 2%	$>$ 33 dB
Good	2-9%	20-33 dB
Undetermined	$\ge$ 9%	$\le$ 20 dB