An Architecture Combining Convolutional Neural Networks (CNNs) and Truncated Regularized Kernel Logistic Regression (TR-KLR) for Image Classification

  • Mariam Salmeen

    Student thesis: Master's Thesis


    Nowadays, the field of Machine Learning (ML) continues to grow, where newer and improved models are being developed and employed. A prominent application of ML is in visual object recognition, which has gained popularity because it employs Deep Learning (DL) models, more specifically, Convolutional Neural Networks (CNNs). The strength of CNNs lies in their ability to automatically detect features in high-dimensional data via a hierarchy of layers. Standard CNNs employ the Softmax function at the final layer for classification. In recent literature, this architecture has been explored by replacing the Softmax with a different classifier, namely a Support Vector Machines (SVM) classifier. The resulting model showed a higher prediction accuracy. This research aims to provide further contribution, by exploring classifiers to replace the Softmax. This time, the classifier is a Truncated Regularized Kernel Logistic Regression (TR-KLR), which uses kernels and Newton methods for improving accuracy and computational speed. The study consists of both binary and multiclass classification, and the models were validated using cross validation and a final test. The results for the binary case shows a higher cross validation accuracy for the proposed model compared to the standard Softmax, and a similar final test accuracy. While for the multiclass case, the standard Softmax model performed better. When comparing the KLR and SVM approaches as the classifiers of a CNN, accuracies were similar and, while the time complexity suggests that KLR is more efficient. Future research can test different CNN architectures and different datasets to see the effect on the performance.
    Date of AwardJul 2019
    Original languageAmerican English


    • Deep learning
    • Convolutional Neural Networks
    • Kernel logistic regression
    • Truncated Newton methods
    • Softmax

    Cite this