Model evaluation with RandomForest and AdaBoost

This notebook demonstrates how to train and evaluate machine learning models, specifically focusing on the Random Forest Classifier and AdaBoost Classifier. We will use synthetic data and visualize key metrics, including accuracy, precision, recall, and F1-score.

Parameters setup

# parameters model_type = 'sklearn.ensemble.RandomForestClassifier' n_estimators = 5 criterion = 'gini' max_depth= 10

Data generation

We are generating a synthetic classification dataset using make_classification. This dataset has 10,000 samples, 20 features, and 2 target classes.

X, y = datasets.make_classification(n_samples=10000, n_features=20, n_informative=5, n_classes=2, flip_y=0.1, class_sep=0.5, random_state=42) X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

Dynamic model import

Using the import_module function, it can be dynamically chosen ti import any classifier based on the model_type string.

module, _, attribute = model_type.rpartition('.') class_ = getattr(import_module(module), attribute) clf = class_(n_estimators=n_estimators, criterion=criterion, max_depth=max_depth) print("Fitting model...") _ = clf.fit(X_train, y_train)

Predictions and evaluations

y_pred = clf.predict(X_test) y_score = clf.predict_proba(X_test) cr = classification_report(y_test, y_pred, output_dict=True) accuracy = accuracy_score(y_test, y_pred) accuracy precision = cr['1']['precision'] precision recall = cr['1']['recall'] recall f1 = cr['1']['f1-score'] f1

Visualizing results

Confusion matrix

plot.confusion_matrix(y_test, y_pred)

Classification report

plot.classification_report(y_test, y_pred)

Precision-recall curve

plot.precision_recall(y_test, y_score)

Conclusion

In this notebook was demonstrated how to:

Use different machine learning classifiers dynamically by modifying the model_type string.

Evaluate the model performance using key metrics such as accuracy, precision, recall, and F1-score.

Visualize the results with confusion matrices, classification reports, and precision-recall curves.

This notebook structure provides a clear flow, starting from data generation, through model training, to evaluation and visualization, all while maintaining interactivity to foster exploration.