Training a Model for XOR using scikit-learn
The XOR (Exclusive OR) problem is a classic example in machine learning that demonstrates the limitations of linear models. This article explains how to train a model for XOR using scikit-learn, a popular machine learning library in Python.
1. Understanding the XOR Problem
The XOR operation outputs a ‘True’ value only when one of the two inputs is ‘True’ and the other is ‘False’. Here’s a truth table representing XOR:
| Input 1 | Input 2 | XOR Output |
|---|---|---|
| False | False | False |
| False | True | True |
| True | False | True |
| True | True | False |
Linear models, like logistic regression, cannot directly learn this pattern due to the non-linear nature of XOR.
2. Creating the XOR Dataset
We’ll start by creating a dataset representing the XOR operation in Python:
import numpy as np X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]]) y = np.array([0, 1, 1, 0])
This code defines the input features (X) and the corresponding target values (y) for the XOR function.
3. Training a Multi-Layer Perceptron (MLP)
To solve the XOR problem, we’ll use a Multi-Layer Perceptron (MLP), a type of neural network capable of learning non-linear relationships.
from sklearn.neural_network import MLPClassifier mlp = MLPClassifier(hidden_layer_sizes=(2,), activation='relu', solver='adam', max_iter=1000) mlp.fit(X, y)
This code initializes an MLP classifier with a single hidden layer containing 2 neurons, uses the ReLU activation function, optimizes using the Adam algorithm, and sets the maximum number of iterations to 1000. Then, the model is trained on the provided data.
4. Making Predictions and Evaluating the Model
After training, we can make predictions on new data:
predictions = mlp.predict(X) print("Predictions:", predictions)
Predictions: [0 1 1 0]
The model accurately predicts the outputs for all the training data points. To evaluate the model’s performance, we can use metrics like accuracy:
from sklearn.metrics import accuracy_score accuracy = accuracy_score(y, predictions) print("Accuracy:", accuracy)
Accuracy: 1.0
The model achieves 100% accuracy on the training data, indicating it has successfully learned the XOR function.
5. Visualization (Optional)
We can visualize the decision boundary learned by the MLP using matplotlib:
import matplotlib.pyplot as plt from mlxtend.plotting import plot_decision_regions plot_decision_regions(X, y, clf=mlp, legend=2) plt.title("Decision Boundary for XOR") plt.xlabel("Input 1") plt.ylabel("Input 2") plt.show()
This code generates a plot showing the decision boundary separating the two classes, illustrating the non-linear separation achieved by the trained MLP.
Conclusion
Training a model for XOR using scikit-learn demonstrates the power of neural networks in solving non-linear problems. By utilizing an MLP, we were able to achieve high accuracy, effectively learning the XOR function and showcasing the ability of neural networks to model complex patterns.