Oil palm fresh fruit bunch ripeness classification on mobile devices using deep learning approaches

•The usage of 9 angle crop for data augmentation increased the accuracy of palm oil ripeness classification with CNN.•EfficientNetB0 is the most accurate lightweight CNN in palm oil ripeness classification.•Although being second in classification accuracy, MobileNetV1 is the fastest CNN in palm oil...

Full description

Saved in:
Bibliographic Details
Published in:Computers and electronics in agriculture 2021-09, Vol.188, p.106359, Article 106359
Main Authors: Suharjito, Elwirehardja, Gregorius Natanael, Prayoga, Jonathan Sebastian
Format: Article
Language:English
Subjects:
Accuracy
Applications programs
Artificial neural networks
Classification
Data augmentation
Deep learning
Electronic devices
Image classification
Lightweight
Machine learning
Measurement
Mobile computing
Mobile devices
Model accuracy
Oil palm
Training
Transfer learning
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•The usage of 9 angle crop for data augmentation increased the accuracy of palm oil ripeness classification with CNN.•EfficientNetB0 is the most accurate lightweight CNN in palm oil ripeness classification.•Although being second in classification accuracy, MobileNetV1 is the fastest CNN in palm oil ripeness classification. The implementations of deep learning combined with other methods such as transfer learning and data augmentation in oil palm fresh fruit bunch (FFB) ripeness classification have been researched throughout the years. However, most of the methods require devices with high computational resources which could not be implemented in mobile applications. To overcome this problem, this research would focus on creating a mobile application to classify the ripeness levels of oil palm FFB using lightweight Convolutional Neural Network (CNN). In this research, we implemented ImageNet transfer learning on 4 lightweight CNN models with a novel data augmentation method named “9-angle crop”, which would be further optimized using post-training quantization. Transfer learning with 3 unfrozen convolution blocks and 9 angle crop successfully increased the classification accuracy on MobileNetV1, and when it was compared to other lightweight models, EfficientNetB0 performed best with 0.898 test accuracy on Keras. Float16 quantization also proved to be the most suitable post-training quantization method for this model, halving the size of EfficientNetB0 with the least increase in image classification time and an accuracy drop of only 0.005 after the model was converted to TensorFlow Lite interpreter. In conclusion, the best model created in this research is EfficientNetB0, trained with the combination of transfer learning, 9 angle crop, and float16 quantization, which enabled it to achieve an overall test accuracy of 0.893 on TensorFlow Lite with 96 ms classification time per image, far surpassing the other 3 compared models with the second-best model being MobileNetV1 with 0.811 accuracy. The model itself was able to achieve similar results when it was implemented on an Android application to classify the ripeness levels of oil palm FFB images obtained through live camera input.
ISSN:0168-1699
1872-7107
DOI:10.1016/j.compag.2021.106359