Advanced machine learning algorithms for blood pressure classification: Early detection or prevention could save lives

Data pre-processing

The dataset is split into a training set and a test set. We utilized sophisticated ML techniques with a supervised learning strategy to acquire knowledge from the data. This was achieved by training the models, cross-validating, and making predictions on the test dataset. We use the assumption that the BP conforms to a multinomial distribution with four distinct classes. The categorization of the BP levels obtained from the NHS England into four classes is based on the guidelines established by the American College of Cardiology and American Heart Association. Let $I_{BP}^{class}\left(Y|X\right)$ represent a binary indicator function for BP. This function is defined as a multiclass variable in the form:

where $I_{BP}^{class}\left(Y|X\right)$ follows multinomial distribution with probabilistic classifier and cutoff values defined by:

The data are split into a training set comprising 70% and a test set comprising 30%. In the first step, we apply a multinomial logistic regression model to the data. Thereafter, the feasibility of the fitted model is assessed, and we perform statistical tests to determine its significance. Subsequently, we utilize sophisticated ML classifiers to analyze the training data and make predictions for the test set. For this task, the supervised ML technique is suitable. The research employs advanced ML classifiers, namely Naïve Bayes, Adaptive Boosting (AdaBoost), feedforward neural network (FNN), and long short-term memory (LSTM). The effectiveness of the proposed algorithms was evaluated using a multiclass confusion matrix and various performance evaluation metrics, including prediction accuracy, classification accuracy, kappa statistics, F1-score, specificity, sensitivity, and precision. These metrics were compared to the results obtained from conventional techniques used in previous studies. Furthermore, we performed statistical tests of independence to further examine the potential association between hypertension and each of the categorical explanatory factors.

Naïve Bayes algorithm

The Naïve Bayes algorithm is a probabilistic classifier based on Bayes’ Theorem, which underpins a broad range of applications in ML. Fundamentally, Naïve Bayes assumes that the presence of a particular feature in a class is independent of the presence of any other feature.^[24] This assumption, often termed “conditional independence,” simplifies the computation, and while it may seem oversimplified, Naïve Bayes classifiers have proven remarkably effective in numerous practical applications, particularly in text classification tasks such as spam detection and sentiment analysis.^[25]

The algorithm leverages Bayes’ Theorem, which relates the conditional and marginal probabilities of stochastic events. Specifically, for a classification problem, it calculates the posterior probability of a class based on a set of predictors. The Naïve Bayes algorithm requires that attribute pairs are conditionally independent, given the class variable.^[26] In a nutshell, it is summarized as:

where, c_i is the class, P (c_i) is the class probability, and $\overrightarrow{b}$ is the attribute variable.

This model is computationally efficient and easy to implement, contributing to its widespread use in scenarios with large datasets and high-dimensional input spaces. The effectiveness of Naïve Bayes has been demonstrated, particularly when the conditional independence assumption holds reasonably well.^[27]

Multinomial logistic model

Multinomial logistic model extends a binary logistic to multiclass problems, where the dependent variable can take three or more categories. This statistical model estimates the probabilities of the different possible outcomes of a categorically distributed dependent variable, given a set of independent variables.^[28]

The model operates on the principle that each category’s log-odds are a linear combination of the independent variables. Mathematically, for K possible outcomes, the probability of the outcome k is modeled as:

where, β_k0 and β_k are parameters to be estimated, and x represents the vector of independent variables. The denominator ensures that the probabilities sum to one.

Multinomial logistic regression is widely used in areas such as medical research, market research, and social sciences, where the outcomes are nominal, and the independent variables are linearly related to the log-odds of the outcomes.^[29] Empirical studies have validated the utility and broad applicability of the multinomial logistic regression in interpreting complex relationships between categorical data. Researchers have postulated that it supports fewer complex models to produce accurate predictions.^[30]

The AdaBoost algorithm

The AdaBoost (Adaptive Boosting) algorithm is a powerful ensemble technique that combines multiple weak classifiers to form a robust classifier. The AdaBoost algorithm recursively merged linearly to form a more satisfying classifier.^[31,32] The AdaBoost is one of the most influential developments in the field of ML, specifically within the domain of classification.^[31]

AdaBoost works by training a sequence of classifiers, typically decision trees, on repeatedly modified versions of the data. Each successive classifier is trained on a dataset that has been altered in terms of the distribution of the training sets. Sets that were misclassified by previous classifiers are given increased weight, thereby focusing the learning of subsequent classifiers on the harder cases.

The output of the AdaBoost algorithm is a weighted sum of the classifiers’ predictions. Each classifier’s weight in the sum is assigned based on its accuracy, with more accurate classifiers having a higher influence on the final decision. Mathematically, the final model can be expressed as:

where. h_t (x) is the prediction of the t-th classifier, and ∝_t is its weight, calculated from its error rate on the training set.

AdaBoost’s effectiveness has been demonstrated to reduce both bias and variance, leading to improved prediction accuracy over individual classifiers and other ensemble methods.^[33] The algorithm’s simplicity and effectiveness have made it a standard benchmark in ML tasks.

FNNs

One of the methods we are considering in this study is the FNNs. The FNN is a pivotal class of artificial neural networks characterized by non-cyclical node connections. This noncyclical design ensures a seamless flow of data from input to output, enhancing their efficiency in tasks such as speech recognition and image classification.^[34] Unlike recurrent neural networks (RNNs), which utilize feedback loops, FNN are structured in distinct layers, input, hidden, and output, that facilitate parallel processing through interconnected computational units.^[35] The multilayer perceptron’s (MLPs) or multilayer neural network is an important special case of the FFN, whereby the links to the i^th layer come only from the immediately preceding layer (i^th–1).^[36,37] The input layer receives data and assigns each feature to a neuron; the hidden layers conduct complex computations to abstractly transform the input; and the output layer, often employing a SoftMax function, converts the outputs into a probability distribution for classification tasks.^[38] Each neuron processes a weighted sum of inputs through a non-linear activation function like sigmoid, tanh, or ReLU, with the choice of activation function critically affecting the network’s efficiency and training dynamics.^[35,38] The configuration of these layers, especially the number and size of the hidden layers, significantly influences the network’s performance.

Figure 1 shows a simple example of an FNN with a single hidden layer $\left(\overline{h_1}\right)$ and a single-out layer, where $h_{1n}=\varnothing \left({\displaystyle {\sum }_n^i{x}_i{w}_{im}}\right);$; ∅ is the activation symbol and w_im are the corresponding weight matrices, connecting neuron i from the input to a neuron m in the hidden layer.^[35] FNN utilizes forward propagation for data processing, where neurons sequentially transmit information up to the output layer, while the learning is driven by backpropagation, where the network updates its weights to minimize the error and improve its predictions. Backpropagation uses a mechanism where the loss function’s gradient is computed concerning each weight, guiding optimization algorithms such as SGD, Adam, or RMSprop to adjust weights and minimize errors.^[36]

Close

Figure 1:

A simple example of a feedback network with a single hidden layer and a single output layer.

Export to PPT

However, FNNs are prone to overfitting and can suffer from vanishing or exploding gradients during training, potentially leading to poor generalization and unstable updates.^[39] These challenges necessitate strategic network design and the implementation of regularization techniques to bolster the robustness and efficacy of FNN in practical settings.

LSTM

LSTM networks, a specialized subclass of RNNs, were developed by Hochreiter and Schmidhuber in 1997 to tackle the challenges associated with recognizing long-term dependencies in sequential data.^[40] These networks have revolutionized deep learning, significantly enhancing performance in tasks requiring comprehension of lengthy sequences, such as language translation, speech recognition, and time series analysis.^[41]

The distinctive architecture of LSTM is built around a structure called the cell, which consists of three gates: The input gate, the forget gate, and the output gate. These components are crucial for regulating the flow of information. The input gate controls the entry of new data into the cell, the forget gate decides what information to retain or remove from the cell state, and the output gate determines which parts of the cell state should be output at each step.^{[42, 43]} This configuration allows LSTMs to maintain or discard data over extended periods effectively.

Figure 2 is a simple example to demonstrate the LSTM method. Where X is the pointwise multiplication, + is the pointwise addition, i_m is the input gate (m = i), f_i is the forget gate, o_i is the output gate, tanh is the pointwise Tanh, tanh * is the Tan h activated and σ is the sigmoid activated (i = 1,2,…., n)[1]. A key feature of the LSTM is the cell state, which facilitates the smooth flow of information with minimal changes.^[35] This design is instrumental in mitigating the vanishing gradient problem common in traditional RNNs, where backpropagated gradients tend to decrease exponentially, hindering the learning of dependencies over longer sequences.

Close

Figure 2:

A simple example of the long short-term memory.

Export to PPT

In practical scenarios, LSTMs excel in fields where understanding contextual continuity is crucial.^[41] They outperform conventional RNNs and other models by adeptly managing data storage and retrieval, making them ideal for complex prediction tasks involving extensive historical data. Their capacity to learn sequence dependencies underscores their superiority and adaptability in the analysis of sequential data, demonstrating the significant impact of their innovative gating mechanisms on the efficacy of temporal neural models.

Data overview and pre-processing

The dataset used for this study consists of 15,000 observations, each containing eight variables: age, gender, weight, smoking status, alcohol consumption, fitness level, height, and BP level. The distribution of the BP levels was skewed, with most records (11,021) falling into the BP Level 1 category (normal), followed by 2,889 records in BP Level 2 (elevated), 702 in BP Level 3 (Stage 1 hypertension), and 388 in BP Level 4 (Stage 2 hypertension). Data pre-processing was performed to ensure that the dataset was free of missing values and errors. A Chi-square test was used to evaluate the relationships between categorical variables and BP levels. Significant dependencies were identified for gender and smoking status (P < 0.05), while no significant relationship was found for alcohol consumption and fitness level. The dataset was split into training set (70%) and test set (30%), with the original BP level distribution maintained. Synthetic Minority Oversampling Technique (SMOTE) was applied to the training data to address class imbalance.

Model training, testing, and evaluation

AdaBoost achieved a training accuracy of 83.43% and a test accuracy of 89.24%. It performed well on BP Levels 1 and 2, but struggled with the low-frequency categories, BP Levels 3 and 4. The F1-score was 0.70, with a kappa statistic 0.77, indicating substantial agreement. The model’s sensitivity was 0.81 and specificity was high at 0.97. The area under the curve (AUC) score was 0.91, reflecting strong discrimination between BP levels.

The LSTM model showed a training accuracy of 92.8% and a test accuracy of 89.24%. While the confusion matrix indicated strong performance across all categories, it had slightly lower sensitivity and F1-scores (0.66) than AdaBoost. The kappa statistic was 0.76, with a sensitivity of 0.74 and a specificity of 0.97. The AUC score was 0.85.

The FNN achieved the highest training accuracy of 93.33% and a test accuracy of 89.47%, indicating potential overfitting. It had an F1-score of 0.65, a kappa statistic of 0.77, and a sensitivity of 0.72, with high specificity (0.97). The AUC score was 0.84, indicating strong classification performance. As shown in Figure 3, the confusion matrix indicates a balanced performance with good sensitivity and specificity across all classes.

Close

Figure 3:

Graphs depicting different performance metrics for Adaboost algorithm in the classification of blood pressure levels. ROC: Receiver operating characteristic, AUC: Area under the curve.

Export to PPT

The Naïve Bayes model reached a training accuracy of 90% and a test accuracy of 89.07%. It excelled in BP Levels 1 and 2, but its performance was weaker for Levels 3 and 4. The F1-score was 0.69, with a kappa statistic of 0.79, reflecting the highest agreement among the models. Sensitivity was 0.78, while specificity remained high at 0.95, and the AUC score was 0.88.

The multinomial logistic model achieved a training accuracy of 83.21% and a test accuracy of 89.07%, making it a reliable baseline for comparison. The F1-score was 0.68, with a kappa statistic of 0.77. Sensitivity was 0.79 and specificity was 0.97, with an AUC score of 0.89, demonstrating its competence in distinguishing between BP levels.