How to extract the decision rules from scikit-learn decision-tree?

Extracting Decision Rules from Scikit-learn Decision Trees

Decision trees are a popular machine learning algorithm for both classification and regression tasks. They offer interpretability by representing the decision-making process as a tree structure. In this article, we will delve into how to extract the decision rules from scikit-learn decision trees, enabling better understanding of the model’s predictions.

Understanding Decision Trees

A decision tree is a flowchart-like structure where each internal node represents a feature, each branch represents a possible value of that feature, and each leaf node represents a class label or a prediction value. The tree is constructed by recursively partitioning the data based on the feature that best splits the data into homogeneous subsets.

Example Decision Tree:

Consider a simple decision tree for classifying fruits based on their color and size:

In this tree, the first split is based on color. If the fruit is red, it is classified as an apple. If it is green, the next split is based on size. Large green fruits are classified as watermelons, while small green fruits are classified as limes.

Extracting Decision Rules

Scikit-learn provides methods to access the decision tree structure and extract decision rules:

1. Using the `tree.export_text()` Function:

The `export_text()` function in `tree` module returns a human-readable text representation of the decision tree. This text can be easily parsed to extract decision rules.

from sklearn.tree import DecisionTreeClassifier, export_text
# ... (code for training the decision tree)

text_representation = export_text(dtc)
print(text_representation)

2. Iterating through Tree Structure:

We can manually iterate through the decision tree structure using the `tree_.feature`, `tree_.threshold`, and `tree_.value` attributes. These attributes provide information about the features used, the thresholds for splitting, and the predicted values at each node, respectively.

from sklearn.tree import DecisionTreeClassifier
# ... (code for training the decision tree)

# Access the tree structure
tree = dtc.tree_

# Iterate through the nodes
for node in range(tree.node_count):
    # Get feature index, threshold, and predicted value
    feature_index = tree.feature[node]
    threshold = tree.threshold[node]
    value = tree.value[node]
    # Construct the decision rule
    if feature_index != -2:
        rule = f"Feature {feature_index} <= {threshold}"
    else:
        rule = "Leaf Node"
    print(f"{node}: {rule}, Value: {value}")

Interpreting Decision Rules

Once we have extracted the decision rules, we can interpret them to understand how the model makes predictions:

• Identify Key Features: The rules highlight the features used by the model for making decisions.
• Understand Decision Logic: The rules explain the decision process based on feature values and thresholds.
• Evaluate Model Accuracy: Analyzing the rules can reveal areas where the model may be biased or inaccurate.
• Gain Insights: Extracted rules can provide valuable insights into the underlying patterns and relationships in the data.

Conclusion

Extracting decision rules from scikit-learn decision trees is essential for model interpretability and understanding. By leveraging the `export_text()` function or manually iterating through the tree structure, we can gain insights into the model's decision-making process and improve its transparency.