Enhancing Triage Management in Healthcare: Harnessing the Hybrid Approach of Neurosymbolic AI for Efficient Disease Classification

Abstract

Accurate disease prediction and efficient triage management play a vital role in healthcare settings. This article introduces a hybrid approach that integrates neural networks and symbolic reasoning techniques to enhance disease classification and triage management. The neurosymbolic AI system harnesses the computational power of neural networks to extract patterns from data and analyze symptom severity while employing symbolic reasoning to leverage predefined rules and establish relationships between symptoms and diseases. By utilizing a knowledge graph constructed from disease-symptom relationships, the model provides comprehensive rule-based explanations for predictions and informed recommendations. The findings demonstrate the significant potential of this hybrid approach in significantly improving healthcare outcomes.

Introduction

In healthcare, disease prediction, and triage management are really important components for delivering timely and effective care to patients. However, accurately diagnosing and prioritizing patients can be challenging due to the complexity of diseases and the need for swift decision-making. Traditional methods often struggle to capture the intricate relationships between symptoms and diseases, resulting in suboptimal outcomes. Fortunately, neurosymbolic AI offers a promising solution by integrating the strengths of neural networks and symbolic reasoning. This fusion allows for improved disease classification and triage management, while also providing a transparent and interpretable framework. By combining neural networks’ ability to detect complex patterns in patient data with symbolic reasoning’s representation of medical knowledge, neurosymbolic AI enables healthcare professionals to understand the underlying rationale behind its predictions. This fosters trust, and collaboration, and ultimately leads to better patient outcomes as informed decisions can be made based on the explainable outputs of neurosymbolic AI.

Inspiration & Motivation

The inspiration behind this project comes from the growing need for accurate disease prediction and efficient triage management in healthcare. Traditional approaches to disease classification and triage often face challenges in accurately capturing complex relationships between symptoms and diseases, leading to suboptimal outcomes. The rapid advancements in artificial intelligence (AI) and machine learning have provided new avenues for addressing these challenges. In particular, the emergence of neurosymbolic AI, which combines the strengths of neural networks and symbolic reasoning, presents a unique opportunity to overcome the limitations of traditional approaches. By integrating the computational power of neural networks with the interpretability and logical reasoning capabilities of symbolic reasoning, neurosymbolic AI offers a promising framework to enhance disease prediction accuracy and facilitate effective triage management, at the same time providing an interpretable and explainable framework to help the user and healthcare professionals understand the predictions. This project seeks to leverage the potential of neurosymbolic AI to revolutionize disease classification and triage, ultimately improving healthcare outcomes for individuals worldwide.

Brief Explanation of the System:

The neurosymbolic AI system contains two main components: a neural part and a symbolic component.

In the system, neural networks are used to analyze large amounts of data and learn complex patterns, while symbolic reasoning relies on predefined rules and relationships between symptoms and diseases. By integrating these two approaches, the system aims to enhance the accuracy and interpretability of disease classification and triage decisions.

The hybrid model takes the patient symptoms as the input.

• The neural network component learns patterns from the training data and aanalyzes the severity of symptoms.
• The symbolic reasoning layer uses a knowledge graph from disease-symptom relationships to infer the most likely disease.

The model provides predictions for both disease classification and triage priority. It generates explanations (rule based and other explanations as well) based on the entered symptoms, helping both the users and the healthcare professionals understand the reasoning behind the predictions. The system also identifies the most important symptoms for the prediction and provides a neurosymbolic explanation of how the neural network and symbolic reasoning components contribute to the final prediction. The goal is to optimize patient care by accurately predicting diseases and prioritizing urgent cases for immediate attention, at the same time providing an explanation to explain how the disease is being predicted

About the Dataset:

To train the hybrid model, we will be using a comprehensive dataset consisting of disease-symptom relationships, symptom severity, disease description, symptom description, and disease precaution is collected. I have used the dataset from this website: https://www.kaggle.com/datasets/itachi9604/disease-symptom-description-dataset and another private dataset that I generated which has the explanations of each symptom.

For any Artificial Intelligence system, the data is one of the most important parts, and it is vital that this data is properly processed and cleaned. Cleaning, normalization, and encoding techniques are applied to prepare the data for training. Challenges in data collection, such as missing or inconsistent information, are addressed through validation and imputation methods. These preprocessing steps ensure consistent formatting ensure that the data is clean, organized, and ready for further analysis and modeling tasks.

Steps:

1. Data Collection & Preprocessing
2. Building the Hybrid Model
3. Obtaining Prediction, Explanation, Triage Priority, and Verdict
4. Displaying the output to the user

Step 1: Data collection & Preprocessing

To train the hybrid model, a comprehensive dataset consisting of disease-symptom relationships, symptom severity, disease description, symptom description, and disease precaution is collected. I have used the dataset from this website: https://www.kaggle.com/datasets/itachi9604/disease-symptom-description-datasetz

Code:

1. Installing & importing the necessary libraries:

The code imports several libraries to support the implementation of this neurosymbolic AI system.

# Input data files are available in the read-only "../input/" directory
# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory

import os
for dirname, _, filenames in os.walk('/kaggle/input'):
    for filename in filenames:
        print(os.path.join(dirname, filename))

import pandas as pd
import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers, Model
import networkx as nx

2. Importing the data

This code imports all the datasets available: symptom_severity_df, disease_symptom_df, disease_description_df , disease_precaution_df and symptom_explanation_df.

# Load the datasets
symptom_severity_df = pd.read_csv('/kaggle/input/disease-symptom-description-dataset/Symptom-severity.csv')
disease_symptom_df = pd.read_csv('/kaggle/input/disease-symptom-description-dataset/dataset.csv')
disease_description_df = pd.read_csv('/kaggle/input/disease-symptom-description-dataset/symptom_Description.csv')
disease_precaution_df = pd.read_csv('/kaggle/input/disease-symptom-description-dataset/symptom_precaution.csv')
symptom_explanation_df = pd.read_csv('/kaggle/input/symptom-explanation/Symptom explanation.csv')

• Symptom severity.csv: contains information about the symptoms and their given severity rating, ranging from 1 to 7
• dataset.csv: contains diseases and their symptoms, there are over 500 diseases and their associated symptoms existing
• symptom_Description.csv: contains the description of each of the diseases
• symptom_precaution.csv: contains all the precautions for each of the diseases
• Symptom explanation.csv: contains the explanation of each symptom

3. Data Preprocessing

These are the steps for processing and cleaning the data before converting it into input for the hybrid model.

# Incorporate symptom severity
symptom_severity_dict = dict(zip(symptom_severity_df['Symptom'], symptom_severity_df['weight']))

# Create a dictionary to map symptoms to their indices
symptom_to_index = {symptom: index for index, symptom in enumerate(symptom_severity_dict.keys())}

# Remove leading and trailing spaces from disease names in the disease_symptom_df dataset
disease_symptom_df['Disease'] = disease_symptom_df['Disease'].str.strip()
disease_symptom_df = disease_symptom_df.applymap(lambda x: x.strip() if isinstance(x, str) else x)

# Remove leading and trailing spaces from disease names in teh disease_precaution_df dataset
disease_precaution_df['Disease'] = disease_precaution_df['Disease'].str.strip()

# Remove leading and trailing spaces from disease names in the disease_description_df dataset
disease_description_df['Disease'] = disease_description_df['Disease'].str.strip()

disease_symptom_dup = disease_symptom_df.drop_duplicates()

# Function to capitalize words and remove underscores
capitalize_and_remove_underscore = lambda x: x.title().replace('_', '')
disease_precaution_df = disease_precaution_df.fillna('')

Create a dictionary for symptom severity: Creates a dictionary called symptom_severity_dict using the symptom severity dataset which maps each symptom to its corresponding severity weight

Create a dictionary for symptom indices: The code creates a dictionary called symptom_to_index to map each symptom to its index. It iterates over the keys of symptom_severity_dict and assigns each symptom to its corresponding index in the dictionary.

Remove leading and trailing spaces from disease names: The code removes any leading and trailing spaces from the disease names in the disease_symptom_df dataset using the str.strip() function. The same operation is performed on all columns of the data frame using the applymap function and a lambda function. The next line removes leading and trailing spaces from the disease names in the disease_precaution_df dataset. The next line removes leading and trailing spaces from the disease names in the disease_description_df dataset.

Remove duplicate entries from the disease-symptom dataset: The code removes duplicate rows from the disease_symptom_df data frame and assigns the result to disease_symptom_dup.

Function for capitalizing words and removing underscores: The code defines a lambda function called capitalize_and_remove_underscore.This function capitalizes the words in a string and replaces underscores with spaces.

Fill in missing values in the precaution dataset: The code fills in any missing values in the disease_precaution_df data frame with an empty string using fillna('').

Step 2: Building the Hybrid Model

Neural Network Component:

The neural network component forms a crucial part of the hybrid model. The architecture is designed to effectively learn patterns and relationships between symptoms and diseases. This includes selecting appropriate layers, activation functions, and regularization techniques. The model is trained using an optimizer and loss function customized to the specific requirements.

Symbolic Reasoning Component:

The symbolic reasoning layer uses a knowledge graph constructed from disease-symptom relationships. This component enables the model to reason about the relationships between symptoms and diseases using predefined rules (from the dataset). By leveraging this knowledge graph, the model can provide transparent and interpretable predictions and recommendations for triage management.

Hybrid Model and Predictions:

The hybrid model combines the outputs of both the neural network and symbolic reasoning components using a Multiply Layer to generate predictions and triage recommendations. The multiply layer is being used in this context to capture interactions and dependencies between the outputs of the neural network and symbolic reasoning components, allowing for a weighted combination that incorporates the strengths of both approaches in disease classification and prediction. Patient symptoms are inputted into the model, which produces a combined output that captures both the learned patterns from the neural network and the rule-based reasoning from the symbolic reasoning layer. The model’s predictions provide insights into the most likely disease and associated triage priority, enabling efficient resource allocation and patient care.

Code:

1. Creating a Hybrid Model

# Define symbolic rules
disease_symptom_dict = {}
symptom_disease_dict = {}

# Normalize symptom names
normalize_symptom = lambda symptom: symptom.strip().lower()

# Iterate over the rows of the dataset
for _, row in disease_symptom_dup.iterrows():
    disease = row['Disease']
    symptoms = [normalize_symptom(symptom) for symptom in row.drop('Disease').dropna().tolist()]

    disease_symptom_dict.setdefault(disease, []).extend(symptoms)

    for symptom in symptoms:
        symptom_disease_dict.setdefault(symptom, []).extend(disease)

# Define symbolic rules using a knowledge graph
knowledge_graph = nx.DiGraph()

# Add disease-symptom relationships to the knowledge graph
for _, row in disease_symptom_dup.iterrows():
    disease = row['Disease']
    symptoms = [normalize_symptom(symptom) for symptom in row.drop('Disease').dropna().tolist()]

    knowledge_graph.add_edges_from([(symptom, disease) for symptom in symptoms])

# Custom layer for symbolic reasoning
class SymbolicReasoningLayer(layers.Layer):
    def __init__(self, knowledge_graph, num_diseases, **kwargs):
        super(SymbolicReasoningLayer, self).__init__(**kwargs)
        self.knowledge_graph = knowledge_graph
        self.num_diseases = num_diseases
        self.symbolic_reasoning_output = layers.Dense(self.num_diseases, activation='softmax')

    def call(self, inputs):
        inferred_diseases = []
        for symptoms in inputs:
            inferred_disease = set()
            for symptom in symptoms:
                if symptom in self.knowledge_graph:
                    related_diseases = nx.descendants(self.knowledge_graph, symptom)
                    inferred_disease.update(related_diseases)
            inferred_diseases.append(list(inferred_disease))
        return self.symbolic_reasoning_output(tf.convert_to_tensor(inferred_diseases))

    def explain_prediction(self, patient_symptoms):
        inferred_diseases = []
        for symptoms in patient_symptoms:
            inferred_disease = set()
            for symptom in symptoms:
                if symptom in self.knowledge_graph:
                    related_diseases = nx.descendants(self.knowledge_graph, symptom)
                    inferred_disease.update(related_diseases)
            inferred_diseases.append(list(inferred_disease))
        return inferred_diseases


# Convert patient symptoms to a one-hot encoded vector
def encode_symptoms(symptoms):
    encoded_symptoms = np.zeros(len(symptom_severity_dict))
    for symptom in symptoms:
        if symptom in symptom_to_index:
            encoded_symptoms[symptom_to_index[symptom]] = 1
    return encoded_symptoms

# Create the hybrid model
symptom_input = layers.Input(shape=(len(symptom_severity_dict),), name='symptom_input')

# Neural network component with explainability
neural_network_output = layers.Dense(128, activation='relu')(symptom_input)
neural_network_output = layers.Dropout(0.5)(neural_network_output)
neural_network_output = layers.Dense(64, activation='relu')(neural_network_output)
neural_network_output = layers.Dropout(0.5)(neural_network_output)
neural_network_output = layers.Dense(len(disease_symptom_dict), activation='softmax', name='neural_network_output')(neural_network_output)

# Symbolic reasoning component with explainability
symbolic_reasoning_output = SymbolicReasoningLayer(knowledge_graph, len(disease_symptom_dict))(symptom_input)

# Combine neural network and symbolic reasoning outputs using a multiply layer
combined_output = layers.Multiply(name='combined_output')([neural_network_output, symbolic_reasoning_output])

# Create the hybrid model
model = Model(inputs=[symptom_input], outputs=[combined_output])

# Define optimizer with custom learning rate
learning_rate = 0.001
optimizer = keras.optimizers.Adam(learning_rate=learning_rate)

# Compile the model with the new optimizer and loss function
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])

# Convert disease-symptom dictionary to training data
X_train = []
y_train = []
for disease, symptoms in disease_symptom_dict.items():
    X_train.append(encode_symptoms(symptoms))
    y_train.append(np.array([1 if d == disease else 0 for d in disease_symptom_dict.keys()]))
X_train = np.array(X_train)
y_train = np.array(y_train)

# Train the model
model.fit(X_train, y_train, epochs=300, batch_size=32)

The code starts by defining empty dictionaries, disease_symptom_dict and symptom_disease_dict, which will store the relationships between diseases and symptoms.

The normalize_symptom function is defined to convert symptom names to lowercase and remove leading/trailing spaces.

The code iterates over the rows of the dataset (disease_symptom_dup) to populate the dictionaries. It extracts disease names and symptoms from each row, normalizes the symptoms, and adds them to the dictionaries. The disease_symptom_dict maps diseases to their associated symptoms, while symptom_disease_dict maps symptoms to the diseases they are related to.

A knowledge graph (knowledge_graph) is created using a directed graph (nx.DiGraph()) from the disease_symptom_dup dataset. The graph represents the relationships between symptoms and diseases. Then, the code defines a custom layer (SymbolicReasoningLayer) that performs symbolic reasoning based on the knowledge graph. It takes the knowledge graph and the number of diseases as inputs. The layer uses the knowledge graph to infer diseases based on patient symptoms and provides an explanation of the prediction.

The encode_symptoms function converts patient symptoms into a one-hot encoded vector.

The hybrid model is created using the functional API of Keras. It has an input layer (symptom_input) for the encoded symptoms.

The neural network component processes the input and generates predictions (neural_network_output). The neural component of the system utilizes a neural network to process patient symptoms and make predictions for disease classification. It consists of multiple dense layers (128, 64, and 41) with activation functions such as ReLU, followed by dropout layers for regularization.

Dense layer: Dense layers, also known as fully connected layers, are a fundamental building block in neural networks. These layers establish connections between every neuron in the current layer and the previous layer, allowing for the learning of complex patterns and relationships within the data.

Activation function (ReLu): ReLu sets any negative input values to zero, while leaving positive values unchanged. ReLU introduces non-linearity to the network, allowing it to capture complex relationships and patterns in the data. Its simplicity and computational efficiency make it an attractive choice, enabling faster training and inference. The sparsity-inducing property of ReLU helps prevent overfitting and promotes a more efficient representation of the data. f(x) = max(0, x)

Dropout Layer: A dropout layer is a regularization technique commonly used in neural networks during training to prevent overfitting. It randomly sets a fraction of input units to zero during each update, which helps prevent the network from relying too heavily on specific features or neurons.

The symbolic reasoning component (SymbolicReasoningLayer) performs reasoning based on the knowledge graph.

Knowledge graph: A knowledge graph is a structured representation of knowledge that captures relationships between entities or concepts. It consists of nodes (representing entities) and edges (representing relationships) connecting these nodes. Knowledge graphs are used to organize and model complex information, allowing for efficient storage, retrieval, and reasoning over interconnected knowledge.

The outputs of both components are combined using a Multiply layer (combined_output).

Multiply function: The multiply function refers to the element-wise multiplication operation performed by the multiply layer. The multiply layer takes the outputs of the neural network and symbolic reasoning components and performs element-wise multiplication between them.

The model is compiled with the Adam optimizer and categorical cross-entropy loss function. Adam adapts the learning rate for each parameter in the model based on the past gradients, allowing for faster convergence and better performance. It is well-suited for models with large datasets and complex architectures, making it a suitable choice for the hybrid model in this code. In this case, the model is trained to predict the probability distribution of diseases based on symptoms. Categorical cross-entropy encourages the model to assign high probabilities to the correct disease class and penalizes deviations from the true distribution. It is a suitable choice for training the hybrid model to accurately classify diseases based on symptoms.

The disease-symptom dictionary is converted to training data, where each disease’s associated symptoms are encoded as input (X_train) and the corresponding one-hot encoded disease labels are used as output (y_train). The model is trained using the training data (X_train and y_train) with 300 epochs and a batch size of 32. I experimented with a bunch of different epochs and batch sizes, and finally, I decided on these numbers as they provide the best output

Output:

The output represents the training progress of the model over multiple epochs. Each epoch refers to a complete iteration of the training data during the training process.

• Epoch X/Y: It indicates the current epoch number out of the total number of epochs.
• 2/2: It shows the progress within the current epoch, where 2 refers to the number of batches processed in the epoch.
• 0s 7ms/step: It displays the time taken to process each step (batch) within an epoch. In this case, it shows that each step took an average of 7 milliseconds to complete.
• loss: The loss value computed by the model during training for the current epoch. It quantifies the difference between the predicted output and the true output. A lower loss indicates better model performance.
• accuracy: The accuracy of the model on the training data for the current epoch. It represents the proportion of correctly predicted disease classifications compared to the total number of training samples. A higher accuracy value indicates better performance.

Looking at the specific values in the output, the loss decreases over epochs, while the accuracy generally improves. This indicates that the model is gradually learning to make more accurate predictions as the training progresses. In the last epoch (Epoch 300/300), the loss is 0.3214, and the accuracy is 100%.

Step 3: Obtaining Prediction, Explanation, Triage Priority, and Verdict

Explanations and Interpretability

The hybrid model generates explanations for its predictions and triage recommendations. These explanations provide insights into the reasoning behind the model’s decisions, which helps both users and healthcare professionals in understanding the factors influencing the outcomes. Feature analysis highlights critical symptoms and their importance in disease prediction and triage management, enhancing interpretability and trust in the model’s output.

Code:

1. Functions of Prediction, Explanation, Triage Priority, and Verdict

# Function to get feature importance based on model weights
def get_feature_importance(model, user_symptoms):
    feature_names = list(symptom_severity_dict.keys())
    user_symptom_indices = np.array([symptom_to_index[symptom] for symptom in user_symptoms if symptom in symptom_to_index])
    feature_importance = np.abs(model.get_weights()[0]).sum(axis=1)
    user_symptom_importance = feature_importance[user_symptom_indices]
    sorted_indices = np.argsort(user_symptom_importance)[::-1]
    sorted_features = [feature_names[i] for i in user_symptom_indices[sorted_indices]]
    sorted_importance = user_symptom_importance[sorted_indices]
    return sorted_features, sorted_importance


# Function to predict disease based on symptoms using the trained model
def predict_disease(symptoms):
    encoded_symptoms = encode_symptoms(symptoms)
    predicted = model.predict(np.array([encoded_symptoms]))
    disease_index = np.argmax(predicted)
    predicted_disease = list(disease_symptom_dict.keys())[disease_index]

    # Add explanation within the prediction process
    print("The model predicted the disease based on the following symptoms:")
    for symptom in symptoms:
        print("- " + symptom)
    print("The predicted disease is: " + predicted_disease)

    return predicted_disease


def explain_decision(symptoms, user_symptoms):
    encoded_symptoms = encode_symptoms(symptoms)
    predicted = model.predict(np.array([encoded_symptoms]))
    disease_index = np.argmax(predicted)
    predicted_disease = list(disease_symptom_dict.keys())[disease_index]
    top_diseases_indices = np.argsort(predicted)[0][-3:][::-1]
    top_diseases = [list(disease_symptom_dict.keys())[index] for index in top_diseases_indices]
    
    # Get disease description and precautions
    disease_description = get_disease_description(predicted_disease)
    disease_precautions = get_disease_precautions(predicted_disease)
    
    # Check for other diseases with the same symptoms
    potential_diseases = []
    for disease, symptoms in disease_symptom_dict.items():
        if set(user_symptoms).issubset(set(symptoms)):
            potential_diseases.append(disease)
    potential_diseases = set(potential_diseases)
    potential_diseases = list(potential_diseases)
    print("POTENTIAL", potential_diseases)
    
    diseases_same = []
    for disease in potential_diseases:
        dsymptoms = disease_symptom_dict[disease]
        check = all(item in dsymptoms for item in user_symptoms)
        if check is True:
            print()
            diseases_same.append(disease)
            print()
    
    print("TOP", top_diseases)
    for i in top_diseases:
        diseases_same.append(i)
    
    diseases_same = set(diseases_same)
    diseases_same = list(diseases_same)
    count = len(diseases_same)
    
    print()
    print("The predicted disease according to the model is ", predicted_disease)
    print(f"However based on your entered symptoms, there are {count-1} other diseases that can be checked, do you want to check them?")
    print("These are the diseases:")
    for i in diseases_same:
        print(i)
    print()
    
    check = input("Enter yes to check or else proceed with prediction")
    if check == 'yes':
        if(len(potential_diseases)>0):
            for i in potential_diseases:
                print("Checking for disease", i)
                dsd = set(disease_symptom_dict[i])
                us = set(user_symptoms)
                uncommon_symptoms = list(dsd.difference(us))
                for j in uncommon_symptoms[:2]:
                    print("What about this?", j)
                    ans = input("do u have this? (yes or no)")
                    if ans == 'yes':
                        user_symptoms.append(j)
    else:
        print("Okay, proceeding with models prediction without any additional symptoms")
        
    encoded_symptoms = encode_symptoms(user_symptoms)
    predicted = model.predict(np.array([encoded_symptoms]))
    disease_index = np.argmax(predicted)
    predicted_disease = list(disease_symptom_dict.keys())[disease_index]

    # Get disease description and precautions
    disease_description = get_disease_description(predicted_disease)
    disease_precautions = get_disease_precautions(predicted_disease)
    
    # Generate detailed explanation
    explanation = f"Based on the given symptoms, the model predicts the disease as '{predicted_disease}'.\n\n"

    # Feature Importance
    explanation += "Feature Importance:\n"
    features, importance = get_feature_importance(model, user_symptoms)
    explanation += "The most important symptoms for the prediction are:\n"
    for feature, score in zip(features, importance):
        explanation += f"- {feature}: Importance {score}\n"
    explanation += "\n"

    # Neurosymbolic Explanation
    explanation += "Neurosymbolic Explanation:\n"
    explanation += "The model combines neural network predictions and symbolic reasoning to arrive at the disease prediction. "
    explanation += "The neural network analyzes the severity of symptoms and learns patterns from the training data. "
    explanation += "The symbolic reasoning component uses predefined rules and relationships between symptoms and diseases to infer the most likely disease.\n\n"

    # Symptom Analysis
    explanation += "Symptom Analysis:\n"
    explanation += "The model considers the following symptoms and their severities:\n"
    for symptom, severity in symptom_severity_dict.items():
        if symptom in user_symptoms:
            symptom_description = symptom_dict.get(symptom, "Description not available")
            explanation += f"- {symptom}: {symptom_description}. Severity: {severity}\n"
    explanation += "\n"

    # Disease-Symptom Information
    explanation += "Disease-Symptom Information:\n"
    explanation += "The model incorporates information about diseases and their associated symptoms from the dataset.\n"
    explanation += "It learns the relationships between symptoms and diseases through the training process.\n"
    explanation += "\n"
    disease_symptoms = disease_symptom_dict[predicted_disease]


    # Disease Description
    explanation += "Description:\n"
    explanation += f"The predicted disease, '{predicted_disease}', is described as follows:\n"
    explanation += f"{disease_description}\n\n"

    # Precautions
    explanation += "Precautions:\n"
    for i, precaution in enumerate(disease_precautions):
        if (precaution!=""):
            explanation += f"{i + 1}. {precaution}\n"

    return explanation

def get_disease_description(disease):
    description = disease_description_df.loc[disease_description_df['Disease'] == disease, 'Description'].values[0]
    return description

# Function to get disease precautions
def get_disease_precautions(disease):
    precautions = disease_precaution_df.loc[disease_precaution_df['Disease'] == disease].drop('Disease', axis=1).values[0]
    # Apply the function to the numpy array
    precautions = np.vectorize(capitalize_and_remove_underscore)(precautions)
    
    return precautions

# Function to provide rule-based explanations
def get_explanations(symptoms):
    explanations = []
    for disease, symptoms_list in disease_symptom_dict.items():
        symptoms_list = set(symptoms_list)
        if set(symptoms).issubset(symptoms_list):
            explanations.append(f"If {', '.join(symptoms_list)}, then {disease}")
    return explanations

# Triage Categories and their weight ranges
NON_URGENT_CARE = (1, 20)
ROUTINE_CARE = (21, 40)
PRIORITY_CARE = (41, 60)
URGENT_CARE = (61, 80)
IMMEDIATE_ATTENTION = (81, float('inf'))

def calculate_triage_priority(symptoms):
    total_weight = sum(symptom_severity_dict[symptom] for symptom in symptoms)
    if IMMEDIATE_ATTENTION[0] <= total_weight <= IMMEDIATE_ATTENTION[1]:
        return "Immediate Attention"
    elif URGENT_CARE[0] <= total_weight <= URGENT_CARE[1]:
        return "Urgent Care"
    elif PRIORITY_CARE[0] <= total_weight <= PRIORITY_CARE[1]:
        return "Priority Care"
    elif ROUTINE_CARE[0] <= total_weight <= ROUTINE_CARE[1]:
        return "Routine Care"
    elif total_weight >= NON_URGENT_CARE[0]:
        return "Non-Urgent Care"
    else:
        return "Unknown Triage Priority"
    
urgent_care_diseases = [
    'Paralysis (brain hemorrhage)', 'Heart attack', 'Tuberculosis', 'Pneumonia', 'Malaria', 'Dengue', 'Typhoid', 'Hepatitis E'
]

prompt_medical_attention_diseases = [
    'Chronic cholestasis', 'Peptic ulcer disease', 'Jaundice', 'Alcoholic hepatitis', 'Common cold', 'Hypothyroidism',
    'Hyperthyroidism', 'Hypoglycemia', 'Urinary tract infection'
]

non_urgent_care_diseases = [
    'Fungal infection', 'Allergy', 'GERD', 'Drug Reaction', 'Gastroenteritis', 'Bronchial Asthma', 'Migraine', 
    'Cervical spondylosis', 'Varicose veins', 'Osteoarthristis', 'Arthritis', '(vertigo) Paroymsal  Positional Vertigo', 
    'Acne', 'Psoriasis', 'Impetigo'
]

def classify_disease(disease):
    if disease in urgent_care_diseases:
        return 'Urgent Care'
    elif disease in prompt_medical_attention_diseases:
        return 'Prompt Medical Attention'
    elif disease in non_urgent_care_diseases:
        return 'Non-Urgent Care'
    else:
        return 'Unclassified'
    
def provide_verdict(symptoms,predicted_disease):
    triage_priority = calculate_triage_priority(symptoms)
    disease_classification = classify_disease(predicted_disease)

    if triage_priority == "Immediate Attention" or disease_classification == "Urgent Care":
        importance = "High Importance"
    elif triage_priority == "Urgent Care" or disease_classification == "Prompt Medical Attention":
        importance = "Medium Importance"
    elif triage_priority == "Priority Care" or disease_classification == "Non-Urgent Care":
        importance = "Low Importance"
    else:
        importance = "Unknown Importance"

    return f"Verdict: Triage Priority - {triage_priority}, Disease Classification - {disease_classification}, Importance - {importance}"

1.get_feature_importance(model, user_symptoms):

This function calculates the feature importance based on the model weights.

• model: The trained model used for prediction.
• user_symptoms: A list of symptoms provided by the user.

It first retrieves the list of feature names (symptoms) from the symptom_severity_dict.Next, it maps the user symptoms to their corresponding indices in the feature list. The feature importance is obtained by summing the absolute weights of the model’s first layer. It extracts the feature importance values for the user symptoms and sorts them in descending order. Finally, it returns the sorted features (symptoms) and their corresponding importance values.

2.predict_disease(symptoms):

This function predicts the disease based on the given symptoms using the trained model.

• symptoms: A list of symptoms for which the disease is to be predicted.

It encodes the symptoms using the encode_symptoms function (not shown in the provided code). The encoded symptoms are passed to the trained model for prediction using model.predict. The index of the disease with the highest predicted probability is determined using np.argmax.The predicted disease name is obtained by mapping the disease index to the corresponding key in the disease_symptom_dict. The function then prints the given symptoms and the predicted disease. Finally, it returns the predicted disease.

3. explain_decision(symptoms, user_symptoms):

This function provides a detailed explanation for the predicted disease and additional information based on the given symptoms.

• symptoms: A list of symptoms used for prediction.
• user_symptoms: A list of symptoms provided by the user.

It performs similar operations to the predict_disease function to obtain the predicted disease. It identifies the top three diseases with the highest probabilities based on the model’s predictions. Next, it retrieves the description and precautions for the predicted disease using the respective functions (get_disease_description, get_disease_precautions). It checks for other diseases that have the same symptoms as the user’s input and stores them in the potential_diseases list. It then checks for diseases where all user symptoms are present and adds them to the diseases_same list. The function prints the predicted disease, the count of other potential diseases, and the list of potential and top diseases.

It prompts the user to enter whether they want to check the potential diseases or proceed with the prediction. Based on the user’s input, it may ask the user about additional symptoms related to potential diseases.

Finally, it encodes the user’s symptoms, predicts the disease again, and stores the description and precautions. It generates a detailed explanation by combining various information such as feature importance, neurosymbolic explanation, symptom analysis, disease-symptom information, description, and precautions. The explanation is returned as a string.

4.get_disease_description(disease):

This function retrieves the description of a given disease from the disease_description_df DataFrame.

• disease: The name of the disease for which the description is required.

It locates the row in the DataFrame where the disease name matches and extracts the corresponding description. The description is returned.

5.get_disease_precautions(disease):

This function retrieves the precautions associated with a given disease from the disease_precaution_df DataFrame.

• disease: The name of the disease for which the precautions are required.

It locates the row in the DataFrame where the disease name matches and removes the “Disease” column. The precautions are stored as a numpy array and then capitalized and underscores are removed using the capitalize_and_remove_underscore function (not shown in the provided code). The precautions are returned as a list.

6.get_explanations(symptoms):

This function provides rule-based explanations for the given symptoms.

• symptoms: A list of symptoms for which explanations are required.

It iterates over each disease in the disease_symptom_dict and checks if the given symptoms are a subset of the disease's symptoms. If the symptoms match, it generates an explanation using the disease and symptoms. The explanations are stored in a list and returned.

Triage Categories and their weight ranges:

Defines various triage categories along with their corresponding importance. These categories help prioritize the severity of symptoms based on the total weight calculated from the symptom_severity_dict. The categories range from “Non-Urgent Care” to “Immediate Attention,” with increasing levels of urgency.

7.calculate_triage_priority(symptoms):

This function calculates the triage priority based on the total weight of the symptoms.

• symptoms: A list of symptoms for which the triage priority is determined.

It sums up the severity weights of all the symptoms from the symptom_severity_dict. Depending on the total weight, it assigns the corresponding triage priority category. The triage priority is returned as a string.

8.classify_disease(disease):

This function classifies a disease into different categories based on predefined lists of diseases.

• disease: The name of the disease to be classified.

It checks if the disease name belongs to any of the predefined lists (urgent_care_diseases, prompt_medical_attention_diseases, non_urgent_care_diseases). If the disease is found in any list, it returns the corresponding category. Otherwise, it returns “Unclassified.”

9.provide_verdict(symptoms, predicted_disease):

This function provides a verdict based on the calculated triage priority, disease classification, and importance.

• symptoms: A list of symptoms used for prediction.
• predicted_disease: The predicted disease.

It calls the calculate_triage_priority function to determine the triage priority. It calls the classify_disease function to classify the disease. Based on the triage priority and disease classification, it assigns an importance level. The verdict, including the triage priority, disease classification, and importance, is returned as a string.

Step 4: Displaying the output to the user

The output displayed to the user provides comprehensive information regarding the predicted disease and additional details related to the symptoms entered.

First, it displays the predicted disease based on the model’s analysis. However, the user is also informed that there maybe other potential diseases that could be possible based on the symptoms provided. The user is given the choice to proceed with checking these alternative diseases or continue with the initial prediction. (If the user says yes, they would be asked about any other symptoms to predict the disease).

The model then generates a detailed analysis assuming the user proceeds with the prediction. It includes:

• rule-based explanations for the symptoms entered,
• the patient’s symptoms,
• the predicted disease, and
• a final verdict (about the disease classification and triage priority).

Additionally, feature importance is shown, which provides some insight on the symptoms that heavily influenced the prediction. The output also provides an explanation of how the neurosymbolic AI system works, offering insights into how the model combines neural network predictions and symbolic reasoning to make the disease prediction. Moreover, the symptom analysis provides a breakdown of the symptoms and their respective severities. Disease-symptom information is incorporated, showcasing the model’s understanding of the relationships between symptoms and diseases. A description of the predicted disease is given, along with precautions that should be taken.

Getting the symptoms and displaying the output

# Example usage
patient_symptoms = ['vomiting','breathlessness', 'sweating', 'chest_pain']
triage_priority = calculate_triage_priority(patient_symptoms)
predicted_disease = predict_disease(patient_symptoms)
explanation = explain_decision(patient_symptoms, patient_symptoms)
explanations = get_explanations(patient_symptoms)

# Print the explanations
if explanations:
    print("Rule based explanations based on entered symptoms:")
    for expln in explanations:
        print(f"- {expln}")
else:
    print("No rule based explanations found for the given symptoms.\n")

print('\nPatient Symptoms:')
for symptom in patient_symptoms:
    print(symptom)
print('\nPredicted Disease:\n', predicted_disease)

print("\n",provide_verdict(patient_symptoms,predicted_disease))
print('\nExplanation:\n', explanation)

The example usage shows how we can display the information to the user. It starts by providing a list of patient symptoms, which are then used to calculate a triage priority and predict the most probable disease. The explain_decision function generates an explanation for the prediction based on the entered symptoms. The get_explanations function retrieves multiple explanations for the symptoms. The code then proceeds to print the explanations if they exist, along with the patient's symptoms, the predicted disease, a verdict based on the symptoms and predicted disease and the explanation for the prediction. This example usage showcases the functionality of the system in providing explanations and insights based on patient symptoms.

Output:

The output displayed to the user provides information regarding the predicted disease and additional insights related to the symptoms entered. In this particular example, the model predicts the disease as ‘Heart attack’. The user is then informed that there are three other diseases that can be checked based on the symptoms entered: Tuberculosis, Malaria, and Bronchial Asthma. The user is given the option to proceed with checking these diseases or continue with the prediction. Since the user chooses to proceed with the prediction, the model displays rule-based explanations for the entered symptoms, patient symptoms, the predicted disease, and a verdict. The output also includes feature importance, neurosymbolic explanation, symptom analysis, disease-symptom information, a description of the predicted disease, and precautions to be taken, such as calling an ambulance, chewing or swallowing aspirin, and keeping calm.

Status Quo

Existing approaches for disease prediction and triage management have predominantly relied on statistical methods or rule-based systems. These approaches often face limitations in capturing complex patterns and adapting to new information. As a result, patient outcomes may be compromised. The introduction of advanced machine learning techniques, such as neurosymbolic AI, addresses these limitations by combining statistical learning with symbolic reasoning to enhance interpretability and accuracy.

1. Statistical Models: Statistical models use machine learning algorithms to analyze large datasets and identify patterns correlating with specific diseases. These models often employ techniques like logistic regression, decision trees, support vector machines, or random forests. While statistical models can be effective in predicting diseases based on patterns, they may struggle to capture complex relationships between symptoms and diseases. They heavily rely on the availability and quality of training data and may face challenges in adapting to new information or emerging diseases.
2. Rule-Based Systems: Rule-based systems rely on predefined rules and expert knowledge to classify diseases and prioritize patient care. These systems typically use if-then statements or decision trees to match symptoms with corresponding conditions. Rule-based systems can provide transparent and interpretable predictions. However, they may need more flexibility in adapting to variations in symptom presentations and often require frequent updates to accommodate new diseases or changes in medical guidelines.
3. Clinical Scoring Systems: Clinical scoring systems, such as the Acute Physiology and Chronic Health Evaluation (APACHE) or the Sequential Organ Failure Assessment (SOFA), use predefined scores and thresholds to assess the severity of a patient’s condition and guide triage decisions. These systems often rely on physiological parameters, laboratory values, and clinical observations. While clinical scoring systems provide standardized approaches to prioritize patients, they may overlook subtle symptom variations or fail to capture complex interactions between symptoms and diseases.
4. Decision Support Systems: Decision support systems integrate clinical data, patient history, and medical knowledge to provide recommendations for disease diagnosis and triage. These systems often utilize algorithms that combine statistical analysis with expert guidelines. While decision support systems can assist healthcare professionals in making informed decisions, they may still face challenges in accurately predicting complex diseases or handling large volumes of data in real-time.
5. Artificial Intelligence (AI) and Machine Learning: AI and machine learning approaches are increasingly being explored for disease prediction and triage management. These approaches leverage large-scale datasets and advanced algorithms, such as deep learning, to identify patterns and make predictions. However, they often require substantial amounts of labeled data for training and may face challenges in interpretability and explainability, which are critical in healthcare settings.

These existing approaches have made significant contributions to disease prediction and triage management. However, they do face limitations in capturing complex patterns, adapting to new diseases, handling uncertainty, and providing transparent explanations for their predictions. The integration of neurosymbolic AI, as a hybrid approach, aims to address some of these limitations by combining the strengths of neural networks and symbolic reasoning to enhance accuracy, interpretability, and adaptability in disease classification and triage management.

Strengths & Limitations

The neurosymbolic AI system for triage management and disease classification offers several strengths that contribute to its effectiveness in healthcare settings.

• Integration of neural networks and symbolic reasoning: allows for the analysis of symptom severity, learning patterns from data, and leveraging predefined rules and relationships between symptoms and diseases. This combination enables more accurate disease prediction and efficient triage management.
• Ability to provide rule-based explanations for predictions and recommendations: By using a knowledge graph constructed from disease-symptom relationships, the system can offer interpretable insights into its decision-making process. This feature is crucial in healthcare, as it enables both users and healthcare professionals to understand and trust the system’s recommendations, leading to improved collaboration between AI and human experts.
• Adapt to new information and update its knowledge base: As new research and data become available, the system can incorporate them into its symbolic rules, enhancing its accuracy and relevance over time. This adaptability ensures that the system remains up-to-date with the latest medical advancements and findings.

However, the neurosymbolic AI system also has certain limitations that need to be considered.

• Defining accurate and comprehensive symbolic rules: Creating and maintaining an extensive knowledge graph that encompasses the complexities of diseases and symptoms requires significant expertise and ongoing effort. Incomplete or erroneous rules could lead to incorrect predictions or recommendations, underscoring the importance of rigorous validation and continuous refinement.
• Availability and quality of data: Robust and diverse datasets are necessary to train the neural network component effectively. Insufficient or biased data could result in limited generalization and potential disparities in disease predictions across different populations or demographics. Addressing data biases and ensuring data representativeness are critical considerations in deploying and utilizing the system responsibly.
• Computational complexity of the neurosymbolic AI system: There may be some challenges in terms of scalability and real-time performance. The integration of neural networks and symbolic reasoning requires substantial computational resources, which could hinder its practical implementation in resource-constrained healthcare environments or in scenarios that demand real-time decision-making.

Implementation in the real world

The implementation of the hybrid approach, combining neural networks and symbolic reasoning in the neurosymbolic AI system, has several practical implications for integration with existing healthcare systems.

• As mentioned in the strengths of this system, the hybrid approach can enhance the accuracy and efficiency of disease classification and triage management. By leveraging the power of neural networks to analyze symptom severity and learn patterns from data, coupled with symbolic reasoning that incorporates predefined rules and relationships between symptoms and diseases, the system can provide more accurate predictions and recommendations.
• The integration of the hybrid approach with existing healthcare systems can lead to improved workflow and resource allocation. The system can aid in streamlining the triage process by automatically analyzing patient symptoms, predicting diseases, and assigning appropriate priority levels. This can optimize the utilization of healthcare resources, ensure timely interventions for critical cases, and reduce waiting times for patients.
• The neurosymbolic AI system’s ability to provide rule-based explanations for its predictions and recommendations has practical implications for healthcare professionals and patients. The system can generate understandable and interpretable explanations based on the underlying knowledge graph and symbolic rules. This can help healthcare professionals in understanding the reasoning behind the system’s decisions and building trust in its capabilities. Additionally, patients can benefit from transparent explanations that empower them to actively participate in their healthcare journey and make well-informed decisions.
• Integration with existing healthcare systems also entails considerations of interoperability and data sharing. In the future, the neurosymbolic AI system could be designed to seamlessly integrate with electronic health record (EHR) systems, enabling the exchange of patient data and facilitating the incorporation of real-time information for accurate predictions. Robust data security measures and compliance with privacy regulations are crucial to protect patient confidentiality and ensure the responsible handling of sensitive healthcare information.
• The implementation of the hybrid approach can contribute to the accumulation and utilization of medical knowledge. The system can continuously learn from new data, update the knowledge graph, and refine its predictions and recommendations over time. This iterative learning process can lead to the discovery of new patterns, insights, and potential correlations between symptoms and diseases, further improving the accuracy of the system.

Overall, integrating the neurosymbolic AI system with existing healthcare systems offers practical benefits such as enhanced accuracy, streamlined workflow, improved resource allocation, transparent explanations, and the potential for continuous learning. It holds promise for optimizing healthcare delivery, improving patient outcomes, and supporting healthcare professionals in making informed decisions.

Conclusion:

The proposed neurosymbolic AI system presents a hybrid approach that combines neural networks and symbolic reasoning for improved disease classification and triage management. The integration of statistical learning with rule-based reasoning enables accurate predictions and transparent explanations. The potential impact of the hybrid approach in optimizing healthcare outcomes and resource allocation is significant. By leveraging the strengths of both neural networks and symbolic reasoning, the model offers a promising solution to enhance triage management and disease classification in healthcare settings. There are several future research directions that can be pursued to further refine and expand the capabilities of the neurosymbolic AI model for triage management and disease classification:

1. Integration of Temporal Information: Incorporating temporal information, such as the sequence of symptom occurrence or changes over time, can provide valuable insights for disease diagnosis and monitoring. Future research can explore methods to integrate temporal data into the neurosymbolic AI model, enabling it to capture dynamic patterns and better adapt to evolving patient conditions.
2. Addressing Data Biases and Generalization: For now, the data only includes diseases and symptoms, however as it is integrated into real-life systems, there would be confidential patient data such as age, gender, etc. being integrated and there would be certain ethical considerations to be considered. These considerations in healthcare AI necessitate addressing potential biases in data and decision-making. Future research should focus on developing techniques to mitigate biases and ensure fairness in disease classification and triage management.
3. Explainability and Interpretability: Although the neurosymbolic AI model provides certain explanations to the user about the symptoms and the diseases, further research can be conducted to enhance the explainability and interpretability of its predictions. This includes developing methods to generate more detailed and context-specific explanations, as well as designing user-friendly interfaces that effectively communicate the model’s reasoning to healthcare professionals and patients.
4. Clinical Validation and Real-World Deployment: Conducting extensive clinical validation studies and evaluating the performance of the neurosymbolic AI model in real-world healthcare settings is crucial. Future research should focus on conducting rigorous evaluations to assess the model’s efficacy, reliability, and impact on healthcare outcomes. This includes validating its performance against existing clinical guidelines and benchmarking it against other established diagnostic methods.
5. Collaboration and Data Sharing: Collaboration among researchers, healthcare professionals, and institutions is essential for the further refinement and expansion of the neurosymbolic AI model. Future research should emphasize data-sharing initiatives, fostering collaborations to access larger and more diverse datasets. This can facilitate the development of more robust and generalizable models while addressing privacy and security concerns through appropriate data governance frameworks.

By exploring these future research directions, we can continue to advance the neurosymbolic AI model, refine its capabilities, and maximize its potential for improving triage management and disease classification in healthcare.