Development of a Novel Autoinspection System for Paddy Seed Early-germination Performance

We present a novel vision machine for autoinspecting paddy seed early germination. The system comprises an inlet–outlet mechanism, machine vision hardware and software, and a control system for inspecting paddy seed early germination. Differences in color are used to segment the structure of germinating seedlings. A thinning operator is employed to extract the skeleton of the root system, and the skeleton pruning method is used to remove branched roots and extract taproots. Features such as the length and width of the taproot and the length and curvature of the seed axis are provided as the input neurons of neural networks to classify seed germination as “normal” or “abnormal”. The inspection accuracy was found to be 86.09%. The experimental results indicated that early-germinating paddy seeds can be inspected efficiently by using the developed system.


Introduction
In Asia, paddy is a crucial crop for the economy. Planting high-quality seeds is an effective technique for reducing production risk and increasing grain yield to achieve overall increases in value and profit. Healthy seedlings from a seedling propagation station (nursery) are used to cultivate fields; therefore, seed quality is a crucial factor for growing seedlings. Seed testing is essential for maintaining quality. The International Seed Testing Association (ISTA) has provided rules for testing. (1,2) One inspection item involved in the germination test is used to determine the maximum germination potential of seeds. The purpose of this study is to develop a novel machine for studying the germination potential of paddy seeds.
Image processing is a powerful and widely used method for inspecting agricultural products. Huang and Cheng (3) presented an auto-sorting system with machine vision for Chinese cabbage seeds. Shape, color, and texture were the features used to sort seeds according to quality by using a neural network. Tu et al. (4) studied the selection of high-quality pepper seeds and presented an effective model based on a multilayer perceptron neural network consisting of 15 physical traits, including color and size. Belin et al. (5) used thermal imaging for monitoring the growth of seedling organs and segmented the organs of seedlings by using the thermal contrast in the images. Tajima and Kato (6) compared 16 algorithms for processing rice root images and estimated the root length for various morphologies. Chung et al. (7) proposed a method to distinguish infected and healthy seedlings at the age of 3 weeks by using machine vision. Sako et al. (8) used an inverted flatbed scanner to capture digital images of germinating seedlings and then used analysis software to compute the growth, uniformity, and vigor index value of lettuce seedlings grown for 3 days. Lurstwut and Pornpanomchai (9)(10)(11) proposed a machine vision application to evaluate paddy seed germination according to the features of paddy seeds. Color (red, green, and blue), size (eccentricity, area, major and minor axis lengths), shape (roundness and aspect ratio), and texture features of seeds were used as input neurons with an artificial neural network (ANN) to analyze the paddy seed germination. The features of paddy seedlings were not employed to inspect germination in their study. Therefore, here, we propose methods to segment the root system and extract the taproot for classifying early-germinating seedlings. Nguyen et al. (12) employed U-Net, a convolutional neural network, to segment rice seeds, and ResNet was used to classify rice seeds into two groups: germinated and nongerminated seeds. Lee et al. (13) established machine vision hardware and software for vigor testing of soybean seeds. Features, including area, average diameter, and length of the primary root, were measured and taken into account. Iqbal et al. (14) classified fruits, such as oranges, sweet limes, and lemons, using a machine vision system. Their algorithm mainly utilized hue as an indicator to measure the level of maturity and classify fruits into different levels.
According to data obtained from the Taiwan Seed Improvement and Propagation Station (TSIPS), the labor cost for inspecting paddy seed germination in Taiwan has recently increased considerably. Consequently, operators have been facing the problem of high cost, necessitating the development of a novel automatic inspection system for improving inspection processes. Therefore, the aim of this study was to design a machine vision system for inspecting earlygerminating seedlings. The technical goal was to develop an algorithm for extracting the geometric features of early-germinating seedlings and subsequently classifying the seedlings into different quality levels by using the aforementioned features.

Experimental samples
In this study, paddy seeds were obtained from the TSIPS. The structure of a germinating seedling, including the root system (i.e., taproot and branched roots), seedling axis, and cotyledon (seed), is presented in Fig. 1.

Machine vision system
In this study, a novel container was designed for paddy seed germination, as illustrated in Fig. 2. The container included the following: • Bottom plate: a white, 2-mm-thick, A4-size acrylic sheet. • Blue filter paper: A4 size. • Blue grille: enables the growth of seedlings. • Cover plate: a transparent acrylic sheet.
• Clasp: to fasten container. The machine vision system used in this study consisted of a flatbed scanner (Epson Perfection V30), inlet-outlet system, platform, Arduino UNO R3 microcontroller, and programmable controller (PC), as illustrated in Fig. 3. The flatbed scanner with a film scanning area of 216 × 297 mm 2 can capture a film image with a spatial resolution of 300 dpi. Images were processed using Visual Studio 2016-C#. The inverted scanner was placed above the inlet-outlet system. The machine vision system is controlled using a push button. An image ( Fig. 4) was captured when the platform with the seedling container was below the scanner.

2.3
Region segmentation and feature extraction

Region segmentation
Segmenting seeds, cotyledons, and the root system region for the paddy seedling image is an essential procedure and was performed using a back-propagation neural network (BPNN) classifier. The red, green, and blue bands of color in the image were considered as input neurons, and the seeds, cotyledons, root system region, and background region were considered as output neurons. These seven hidden neurons were used to train and test the BPNN. The BPNN mechanism included binary, opening, closing, and hole-filling operations (15) for removing noise and filling the holes in seedlings to obtain a complete binary image. The segmentation steps followed, and the results obtained in this study are presented in Fig. 5.

Cotyledons and root system extraction
Branches had to be removed before extracting the features of cotyledons and taproots. Therefore, the skeleton pruning method was proposed for removing branches after thinning. (16) The skeleton pruning method is based on 4-adjacency (n 1 , n 3 , n 5 , n 7 ), 8-adjacency (n 0 -n 7 ), and m-adjacency matrices (Fig. 6). (15) The three defined points of the branches are as follows ( Fig. 7): (1) endpoint: only one pixel belongs to the 8-adjacency matrix, (2) branch point: three or four pixels belong to the 8-adjacency matrix rather than to the 4-adjacency matrix, (3) normal point: only two pixels belong to the 8-adjacency matrix. The skeleton pruning method was used for removing redundant branches and obtaining representative skeletons. This method includes two procedures, which are detailed as follows: Length-computing algorithm: 1. Search all points that are defined as endpoints.
2. Start searching for branch points from each endpoint along the skeleton.
3. If the length from the endpoint to the branch point is <10 pixels, then remove the branch. 4. Repeat steps 1-3 until no branches with a length of less than 10 pixels remain. The results are presented in Fig. 8. Red, green, and black points are endpoints, branch points, and normal points, respectively. The number of black points represents the length from an endpoint to a branch point. Branches A (11 pixels) and D (13 pixels) were retained, whereas branches B and C were removed, as illustrated in Fig. 8(b).   For example, the branch (green) point was found by connecting three branches. The branch with the smallest width was removed after three width points were identified and their diameters were computed, as shown in Fig. 9.
The exact taproot was not obtained because a few roots were removed by following the aforementioned procedures; thus, the skeleton had to be connected through the following steps:   1. The center and radius (the latter requires setting a threshold value) of a circle were the seed centroid and d f , respectively, as illustrated in Fig. 9. 2. When one of the endpoints was inside the circle, the skeleton was defined as the first skeleton and the other skeleton was defined as the second skeleton.

Connecting conditions:
(a) The connected endpoint is on the second skeleton. (b) The distance between the first and second endpoints is less than d c (threshold value of connection). (c) The angle between the two skeletons is less than 45°. 4. If the second skeleton is connected to the first skeleton, then the second skeleton is merged with the first skeleton. 5. Repeat steps (2)-(4) until no skeleton needs to be connected.

Feature extraction
Geometric feature analysis was employed extensively for classification. In this study, the length (L tr ) and width (W tr ) of taproots and the length (L c ) and curvature (C) of the seedling axis were used to classify paddy seedlings as either normal or abnormal. The mathematical formula used for obtaining the curvature of cotyledons is as follows: where d e is the distance between two endpoints of a cotyledon. Taproot width is defined as the mean diameter of three circles 10 pixels apart on the middle position, as illustrated in Fig. 10.

Neural network classifier
According to ISTA rules, paddy seedlings are classified as normal, abnormal, or ungerminated seeds. In this study, the seedlings were classified into two grades, normal and abnormal (i.e., abnormal and ungerminated), for simplification using the BPNN classifier. The BPNN classifier consisted of three layers: an input layer, a hidden layer, and an output layer. The input features [i.e., the length (L tr ) and width (W tr ) of the taproot and the length (L c ) and curvature (C) of the seedling axis] were normalized to between 0 and 1. The output layer was composed of nodes related to the two grades (normal and abnormal). The numbers of nodes in the hidden and output layers were seven and one, respectively. The output value (V out ) was between 0 and 1. The conditional expressions were as follows: The structure of the BPNN classifier is presented in Fig. 11, wherein W ij and b ij are the weight and bias of the input layer in the hidden layer, and W jk and b jk are the weight and bias of the hidden layer in the output layer, respectively. The purpose of BPNN training was to construct a standard to classify each seedling into normal and abnormal seeding by the pattern-composed features. During training, the BPNN classifier was used to analyze training samples at a given learning rate, and its weights and biases were adjusted until the mean squared error was less than the tolerance error, which indicated that the BPNN classifier had completed training and its weights and biases were stable. In this study, training samples were analyzed at a learning rate of 0.01 before training completion, which was defined as achieving a tolerance error of 0.001.

Overall descriptions
The steps involved in autoinspection are presented in Fig. 12. The complete autoinspection procedure involved placing the container on the platform, raising the platform to the scanner after pushing the button, capturing an image with the scanner, performing inspection, displaying results on the user interface, and establishing a database; these steps are described in the previous four subsections. The inspection algorithms employed region segmentation with a BPNN, the skeleton pruning method, and a BPNN classifier.

Results
In this study, a device consisting of a platform, inlet-outlet system, container, flatbed scanner, PC, and Arduino UNO R3 microcontroller was constructed for the autoinspection of paddy seed early germination. The inspection software for paddy seed early germination was developed using Visual Studio 2016-C# and EmguCv × 86 3.3. The software functions included file operations (acquiring, loading, and saving images), image analysis (i.e., segmentation and feature extraction), BPNN operations, and image database creation. The quality of paddy seedlings could be determined accurately and rapidly through software computation. The skeleton of the taproot was extracted after BPNN segmentation and skeleton pruning, as illustrated in Fig. 13.
Samples germinated for 7 days were used for paddy seed early-germination testing based on our methods. The developed system was separately trained and tested with the BPNN using randomly sampled seeds (3070 normal and 2531 abnormal), as listed in Table 1. Three professional inspectors inspected the samples. Overfitting often occurred when the training set contained some incorrect samples in the BPNN. Because the grades of paddy seedlings in the training samples were known prior to training, overfitting was unlikely. The inspection  accuracies for normal and abnormal seedlings were 91.64 and 86.09%, respectively; the results are presented in Table 2. Two types of failure were observed during feature extraction, namely, those with complicated branches and overlapping roots; thus, the taproot could not be extracted, as illustrated in Fig. 14.
The proposed system can inspect and classify normal and abnormal seedlings accurately and efficiently. In the future, we expect the inspection system to be used for the germination of other seeds.

Conclusions
In this study, we developed an autoinspection system for paddy seed early germination with an inlet-outlet system, machine vision hardware and software, and a control system. A novel container was designed for growing seedlings. A BPNN classifier was employed to segment the cotyledon (seed), seedling axis, and root system. The skeleton pruning method was used for removing redundant branches and obtaining a representative skeleton. The length and width of the taproot and the length and curvature of the seedling axis were obtained as features to develop the BPNN classifier. The inspection accuracies for normal and abnormal seedlings were 91.64 and 86.09%, respectively. The test results indicated that paddy seedlings can be inspected efficiently by using this inspection system. The system has been provided to the TSIPS for inspecting paddy seed early germination.