Decision boundaries

A decision boundary is a boundary or a surface that separates the different classes in a classification problem. It is the dividing line that separates the points of different classes in a feature space. In other words, it is the boundary that separates the regions where different classes reside in the input space.

Here we trained the GB-DNN model on a generated dataset. The dataset is generated using the make_classification function from the scikit-learn library.

The experiment contains the following sections:

• Importing dataset: The dataset is generated using the make_classification function from scikit-learn. It generates a random n-class classification problem with n-informative features and n-redundant features.

X, y = make_classification(n_features=2,
                           n_redundant=0,
                           n_informative=2,
                           random_state=random_state,
                           n_clusters_per_class=1,
                           n_classes=3,
                           n_samples=500,
                           flip_y=0.15,
                           shuffle=True)

• Model defining: This section defines the plotModel function and model_training function.

def plotModel(x, y, clf, ax, title):

    x_min, x_max = X[:, 0].min() - 2, X[:, 0].max() + 2
    y_min, y_max = X[:, 1].min() - 2, X[:, 1].max() + 2

    xx, yy = np.meshgrid(np.arange(x_min, x_max, (x_max - x_min)/200.),
                         np.arange(y_min, y_max, (y_max - y_min)/200.))

    z = clf.predict_proba(np.c_[xx.ravel(), yy.ravel()])
    zz = z.copy()
    z.sort()
    z = z[:, -1]
    z = z.reshape(xx.shape)
    cm = plt.cm.viridis
    ax.contourf(xx, yy, z, levels=11, cmap=cm)


    #3D plot
    fig = plt.figure()
    ax3d = Axes3D(fig)
    ax3d.contour3D(xx, yy, z, 200)

    for i in range(zz.shape[1]):
        z = zz[:, i] - np.max(np.delete(zz, i, axis=1), axis=1)
        z = z.reshape(xx.shape)
        ax.contour(xx, yy, z, [0.0], linewidths=[1], colors=['k'])
        ax3d.contour3D(xx, yy, z, [200])

    ax.scatter(np.ravel(X[:, 0]), np.ravel(X[:, 1]),
               marker='o', c=np.ravel(y), edgecolors='k')

    plt.gca().set_xlim(xx.min(), xx.max())
    plt.gca().set_ylim(yy.min(), yy.max())
    ax.grid(True)
    ax.set_title(title)


    fig.suptitle("3D Probability with regard to data point coordinates")
    ax3d.set_title(title)

• Model training: This section trains the DeepClassifier.

params = {'config': Namespace(seed=111,
                              boosting_epoch=200,
                              boosting_eta=1e-1,
                              save_records=False,
                              additive_epoch=100,
                              batch=30,
                              units=100,
                              additive_eta=1e-3,
                              patience=10)}


model = GBDNNClassifier(config=get_config())
model.set_params(**params)
model.fit(x_train, y_train)

• Model evaluation: This section evaluates the trained models using the plotModel function. It generates the 3D and 2D plots that show the probability of each data point belonging to each class.

---