An efficient method for signal improvement in near-infrared spectroscopy measurements during calibration and validation processes

Standard spectroscopic practice involves the measurements of replicate spectra (designated as s) for each sample where these replicates are co-added in order to reduce random noise by a factor of s. However, when systematic or structured noise are present, due to instrument or sample upset conditions, or subject motion, this practice tends to degrade and distort spectral bands. When co-adding multiple replicate measurements for single samples, such distortion tends to cause biased calibration coefficients and larger prediction errors, resulting in loss of analytical accuracy. In this work a simple automated procedure is presented and aimed to circumvent the above mentioned concern. Multiple replicate spectra of a sample are correlated with the median of the entire set of replicate spectra and then ranked by similarity based on the correlation of each spectrum to this 'reference' median spectrum. A tunable 'binning size' parameter is chosen by dividing the set of ranked, correlated replicate spectra into sub-spectral groups. The highest correlation spectra then co-added with the median to yield what is termed here as a single 'ideal' spectrum. These steps are repeated for each set of sample measurements and performed for both calibration and validation data sets before modeling or prediction. Results from experiments show a substantial decrease in both standard errors of prediction and bias in comparison to the classical replicate spectra co-averaging approach highlights the effectiveness of the proposed method.