Wood Species Classification Utilizing Ensembles of Convolutional Neural Networks Established by Near-Infrared Spectra and Images Acquired from Korean Softwood Lumber

Five Korean softwood species (cedar [Cryptomeria japonica], cypress [Chamaecyparis obtusa], Korean pine [Pinus koraiensis], Korean red pine [Pinus densiflora], and larch [Larix kaempferi]) were used for the wood species classification utilizing ensembles of CNN models established by different features (image and NIR). We collected 50 lumber samples of each species, measuring 50 mm × 100 mm × 600 mm (thickness × width × length), from several mills participating in the National Forestry Cooperative Federation of Korea. The lumber samples in each species were from different regions of Korea. The lumber samples were air-dried (10 – 15% MC) at 25°C and 65 ±10% relative humidity for three months. After air-drying, wood blocks measuring 20 mm × 2 mm × 40 mm (R × T × L) were prepared from each lumber (50 wood samples per species) and images of the radial surface were taken from the blocks. The same radial surface was used for acquiring the NIR spectrum.

We used a smartphone camera (Galaxy Note 9, Samsung, Korea) to obtain macroscopic images of the radial section of the wood samples. The smartphone was placed on a simple frame for stable support during image acquisition. The camera in the smartphone had an f/1.5 lens and produced a color picture of 4032 × 3024 pixels. Totally, we prepared 475 images from the samples. After the background removal of the images, the images were cropped to a size of approximately 2000 × 1000 pixels. According to Brunner et al. (1990), there was no distinct relationship between moisture content and discoloration of wood below fiber saturation point. In this study, sample discoloration was negligible because air-dried samples were employed for image acquisition.

The NIR spectra were taken from the same surface as that used by the smartphone to acquire images of the radial surfaces of woods (in all, 475 spectra). All the NIR spectra were acquired at 1-nm intervals of wavelength ranging from 780 nm to 2500 nm by a NIR spectrometer (SpectraStar 2600XT-R Unity Scientific, USA). The scans-to-average value was 12.

A random k-fold cross-validation (k = 5) was applied to the dataset. Table 1 lists the number of samples for each species, and the sample distributions of the cross-validation. Thus, an image and its corresponding NIR spectrum were paired in the k-fold cross-validation.

For training the CNN models and the ensemble model, we utilized a workstation with XEON CPU (14 cores) with 64 GB of memory as well as GPU with 24 GB (NVIDIA Quadro M6000). The operating system was Ubuntu 16.04 LTS with CUDA 8.0, Python 3.5, Tensorflow 1.2, and Keras 2.0.

The structures of CNN models are shown in Table 2. LeNet3 is a 2D CNN model for the species classification model using image data, as already discussed in our previous study (Kwon et al., 2017). The background-removed images were reduced to 128 × 128 × 3 as input images for training purposes. The pixel values of the input images were normalized by 255. Image augmentation was performed before model training with the following parameters: rotation range = ±30°, shift range in width and height = ±10%, shear range = ±20%, zoom range = ±20%, and horizontal flip. We utilized the adaptive moment estimation (Adam) algorithm (Kingma and Ba, 2014) as an optimizer with learning rate = 0.0001. The number of epochs was 500.

NIRNet is a 1D CNN model using NIR spectrum data. The spectra having a dimension of 1721 × 1 were used as the input for training. The absorbance values of the input spectra were standard normalized by subtracting the mean and then scaled to unit variance. Standard normalization was applied to each wavelength, not sample. We utilized the Adam algorithm as an optimizer with learning rate = 0.0001. The number of epochs was 1500.

The classification performances of the species classification for each CNN model and their ensembles were evaluated by constructing a confusion matrix. The confusion matrix is a table that consists of the classification results. It shows a matrix with the actual and the predicted classes (Fig. 1). From the confusion matrix, some indices including precision, recall, and F1 score could be evaluated to describe the accuracy of a model (Goutte and Gaussier, 2005). The details of reliability indices are described in our previous paper (Yang et al., 2019). In this study, F1 score is used for the evaluation of the classification performance of the models. The F1 score, which is the harmonic mean of precision and recall, can be calculated as described in Fig. 1.

Fig. 1. Confusion matrix (Yang et al., 2019).

Download Original Figure

An ensemble of CNN models requires several operational classification models such as LeNet (Lecun et al., 1998) and VGGNet (Simonyan and Zisserman, 2014). We utilized two CNN models to construct ensemble models for the classification of two types of information from wood surfaces: (1) images of the radial surface and (2) NIR spectra for the same area in the radial surface. On the basis of the CNN classification results, we applied two types of ensemble models for prediction results: (1) averaging and (2) max voting.

Table 3 shows the classification performance indices of the LeNet3 model based on the image data. The classification performance of the LeNet3 model was obtained by averaging the classification results of each fold in the k-fold classification (k=5). The average precision, recall, and F1 score of the LeNet3 model were 93.28%, 90.96%, and 91.98%, respectively. The LeNet3 model based on images from the radial surface had poorer classification performance than that from the transverse surface in our previous study (Kwon et al., 2017). From the classification results, we inferred that the radial surface of wood has less information than the transverse surface of wood. The precision ranged from 88.27% (Korean red pine) to 100.00% (cypress); however, the recall ranged from 85.23% (cypress) to 94.74% (Korean pine). F1 score, which is the harmonic average of precision and recall, ranged from 89.88% (larch) to 93.83% (cedar). Thus, the LeNet3 model based on images from the radial surface had low classification performance for larch.

Table 4 shows the classification performance indices of the NIRNet model based on the NIR spectral data. The k-fold cross-validation (k=5) was also applied. The classification performance of the NIRNet model was obtained by the same averaging in the case of the LeNet3 model. The average precision, recall, and F1 score of the NIRNet model were 86.22%, 85.69%, and 85.94%, respectively. The NIRNet model had lower classification performance than the LeNet3 model in this study. The precision ranged from 76.21% (Korean red pine) to 96.22% (cedar), and the recall ranged from 72.19% (Korean red pine) to 96.31% (cedar). F1 score ranged from 74.15% (Korean red pine) to 96.27% (cedar). Thus, the NIRNet model based on the NIR spectra faced difficulties in classifying the Korean red pine.

Tables 5 and 6 show the classification performance indices of the LeNet3-NIRNet ensemble models by using the averaging and the max-voting method, respectively. The k-fold cross-validation (k=5) was also applied. There were slight differences in the precision and the F1 score for cedar between the two methods. Both ensemble methods improved the classification indices compared with the individual LeNet3 model and the NIRNet model. The average precision, recall, and F1 score of the averaging ensemble model were 96.28%, 94.58%, and 95.31%, respectively. For the max-voting ensemble model, they were 96.04%, 94.42%, and 95.12%, respectively. For both the ensemble models, the F1 scores of cedar and larch were considerably improved; however, those of Korean red pine and cypress were similar to those calculated by the LeNet3 model. In our results, the LeNet3-NIRNet ensemble model using the averaging method was more accurate for the softwood species classification than individual CNN model.

The improvement of the classification performance of the ensemble model was interesting because two independent models with lower indices enhanced the classification result. It can be inferred that the ensemble method using different features (smartphone images and NIR spectra) compensated the classification performance. However, note that this study was a preliminary application, so it was carried out with a small dataset (475 samples). The results should be clearly confirmed with database expansion in a further study.