Fishing Position Detection by Wearable Sensing to Achieve Fishery Support

Many types of research focus on supporting fisheries by utilizing information and communication technology (ICT) to realize sustainable and efficient fishing industries. However, most of the previous studies focused on monitoring environmental data and fishing vessels. Our study focused on identifying the timing of when a fisherman catches an octopus in pot drift fishing for North Pacific giant octopus, in order to support fishery industries. A support vector machine (SVM) detects the timing at which a fisherman catches an octopus, using the feature values derived from acceleration data obtained with a wearable device. In this study, the author adopted three types of feature value sets for the SVM: peak frequency of fast Fourier transform (FFT), statistics, and the combination of peak frequency and statistics. From the evaluation results, the SVM was found to achieve a recall of over 90%. Future works include collecting a large amount of data and developing an application to detect the timing of catch.


Introduction
Capture fishery and aquaculture industries play vital roles in supporting the food demand of the world as the population continues to increase. According to the state of world fisheries and aquaculture published by the Food and Agriculture Organization (FAO) of the United Nations, the average annual increase rate in global fish consumption between 1961 and 2016 is higher than the population growth of the world in the same period. The production amount of world capture fisheries and aquaculture exceeded 170 million tons in 2016. (1) In Japan, however, the trend is different from the world trend. The total production volume of capture fishery and aquaculture peaked in 1984 at 12.82 million tons, but the amount decreased to approximately 4.36 million tons in 2016. (2) The production volume of aquaculture has remained almost the same since the 1980s, but the volume of capture fishery has been decreasing since the 1980s. The reasons for the decrease in production volume are the decline of pelagic fishery through the introduction of exclusive economic zones and the marked decline in sardine resources. Moreover, the trend in the species of fish caught is changing owing to global warming. According to the Japan Meteorological Agency, the sea surface temperature around Japan increased by 1.11 ℃ in about 100 years. (3) To protect the Japanese fishery industry, it is necessary to achieve a sustainable fishery industry adapted to the environment. It is also essential to obtain fishermen's expertise and skills to educate new fishermen.
In recent years, many studies utilizing information and communication technology to support the fishery industry have been conducted. In representative studies, marine environment (water temperature, salinity, dissolved oxygen, and so forth) observation systems have been developed using internet of things (IoT), sensor networks, and cellular networks. (4)(5)(6) Other studies focus on collecting and utilizing the positional information of fishing vessels. (7)(8)(9) In our previous studies, we developed the fishing position recording system for pot drift fishing for the North Pacific giant octopus to create a catching distribution map. In this system, there are only three buttons to minimize the burden to the fisherman. When the fisherman catches an octopus, he pushes the button corresponding to weight classification. (10) Previous studies, however, focused on the ocean environment and fishing vessels. Few studies focus on the behavior of fishermen during fishing. The data on their actions may reveal a variety of information about their activities, e.g., positions where fishermen catch fish, that may enable the visualization of the experience and intuition of veteran fishermen. There is a possibility that such information will be useful in realizing sustainable fisheries.
The author concentrates on the behavior recognition of fisherman. In this paper, the author describes the detection of the timing of when fishermen engage in pot drift fishing of the North Pacific giant octopus, by utilizing machine learning. In this study, the fishermen wore wearable devices, and the author utilized the acceleration data obtained via the devices. Figure 1 shows the mechanism of octopus pot drift fishing. In this type of fishing, pots and lures are used as fishing gear. A pot and a lure are tied with string. A fisherman throws the pots and lures into the sea, whereby the lure is dragged along the bottom of the sea. When the lure passes near the territory of an octopus, the octopus clings to the lure because it mistakes the lure for its food or enemy. If the octopus holds the lure, the movement of the pot to which it was tied with a string changes. The fisherman checks the movement of the pot and carefully pulls up the pot and lure to catch the octopus. When the fisherman pulls up the fishing gear, the action of lifting the string tends to be periodic. In the past, the author embedded a global positioning system (GPS) watch in the fishing gear to obtain positional information of the fishing gear. However, the watch mounted on the fishing gear disturbs the work efficiency and it takes much time to change/recharge the battery of the watch because fishermen usually use about 15 sets of fishing gear during one fishing run. Therefore, in this study, one wearable device was used to obtain the acceleration data of hand movement of fishermen to estimate the timing of the catch.

Obtaining data
To monitor the behavior of fishermen, the author adopted a smartwatch (SmartWatch 3 SWR 50, Sony Mobile Communications Inc.). In this study, the smartwatch only collects acceleration data. Unfortunately, the battery capacity was insufficient to acquire data during fishing operation. Therefore, the author also used an acceleration logger (ActiGraph GT9X Link, ActiGraph LLC), as shown in Fig. 2.
The fishermen wore these device on the nondominant arm, and 3-axis acceleration data of their actions were obtained. The sampling rate was set to 50 Hz. In order to correlate the acceleration data with the work of the fishermen, the author recorded the fishing operation with a video camera. The author used video to tag the acceleration data with "catching (positive sample)" and "other work (negative sample)". During this time, the author collected acceleration data with the cooperation of two fishermen belonging to the Shinsei Marine Fishery Cooperative Association (Rumoi City, Hokkaido, Japan).

Classification method
In order to judge whether a fisherman is working on "catching", the support vector machine (SVM) is used. The input to the support vector is feature values calculated from the acceleration data obtained via the smartwatch or acceleration logger. The SVM performs two-class classification ("catching" or "other work"). In this study, the following feature value sets are used. The data of the latest 10 s were used to calculate the feature values. Because of the shortage of acceleration data, a window, which is a range in which to calculate the feature value, was adopted to increase the amount of data. The window size was 10 s and the amount of shift of the window was 1 s. As a result, 3966 positive samples and 63540 negative samples were obtained. Imbalance between the positive and negative samples degrades the performance of machine learning. To solve this problem, the author undersampled the negative samples by RandomUnderSampling and oversampled the positive samples by the synthetic minority oversampling technique (SMOTE). (11) Normalization was performed as preprocessing of learning. The formula of the normalization is where z is normalized data, x is raw data, u is the mean value of training data, and σ is the standard deviation. Table 1 shows the summary of the collected data. The table includes the data about the fishermen, date, data-obtaining devices, duration, and the number of octopuses that the fishermen caught. There were days when the fishermen could not depart from the port due to bad weather and bad marine conditions, and there was no catch. However, the author obtained three days of acceleration data (approximately 19 hours, 55 catches in total).  Figure 3 shows the acceleration data of hand movement of a fisherman when he engaged in octopus pot drift fishing. Figure 3(a) shows the acceleration data when the fisherman was catching an octopus. The data seem to be periodic because the fisherman reeled the string of the fishing gear by hand. Figure 3(b) demonstrates the data when the fisherman was controlling the fishing vessel. There is little change in the data since the fisherman only operates the controller of the fishing vessel. The fisherman can maneuver the vessel by small movements since the controller of the vessel is a dial type. Figure 3(c) shows the data when the fisherman was throwing fishing gear into the sea. The data have periodicity but the period is faster than that shown in Fig. 3(a). The fisherman needs to pull up the line carefully to prevent an octopus from escaping when an octopus clings to the lure. However, if the octopus does not cling, the fisherman can pull up the line speedily because there is no worry that an octopus will escape.  Table 2 shows the precision, recall, and F-measure in each feature value set. From the table, we can see that the peak frequency is insufficient to distinguish positive and negative samples since the number of dimensions of the feature values is small (only four dimensions). "Statistics" and "Combination of 1 and 2" set show the better performance than "Peak Frequency". Statistics dominates the performance of the SVM. Table 3 shows the confusion matrix of the "Combination of 1 and 2" set. From this table, it is clear that the SVM achieved an accuracy rate of over 90% for both positive and negative samples. However, in our other evaluations, the precision, recall, and F-measure deteriorated when there was no resampling. The author considered the amount of raw data to be insufficient. Figure 4 shows the identification result of daily fishing using the "Combination of 1 and 2" set. To derive the result, the first three-quarters of fishing data values were used  as training data and the remaining one-quarter as test data. However, the result for June 15, 2018 was omitted because the number of catches was too small to evaluate the result. There were two cases of overlooking on May 6, 2017 and three misdetections on May 7, 2018. The definition of overlooking is that the SVM could not detect "catching." The definition of misdetection is that the SVM incorrectly detected a catch when the fisherman was not catching an octopus. From the recorded videos, one case of overlooking and all misdetections occurred during preparations for returning to the fishing port (the fisherman was collecting the fishing gear). The fishing vessel returning to port can be easily judged from the trajectory of the vessel. Therefore, there is a possibility that the trajectory of the vessel mitigates overlooking and misdetection because fishermen return to port after the preparation is completed. The other case of overlooking was of standard catching work. It is considered that the acceleration data differ depending on, for example, water depth, tide speed, and the weight of the octopus. As mentioned previously, more acceleration data will be required to reduce the number of cases of overlooking and misdetection.

Conclusions
In this paper, the author focused on the behavior of fishermen and identified the time at which the octopus was caught. To obtain the acceleration data, fishermen wore a wearable device, the SVM, which distinguished the fishermen's activities. From the evaluation results, the SVM achieved a recall of over 90%. However, as mentioned in Sect. 3.1, the author obtained only three days of data. It is necessary to obtain more data. In addition, there are also many machine learning techniques. In future work, the author will evaluate the performance of other techniques such as Randomforest and neural network.
Improvement of the identification provides the possibility of developing an application for fishery resource management and of visualizing the expert skills of veteran fishermen. A smartwatch embedded with a learning model realizes the identification of the timing of catching in real time. It is also possible to determine the location where octopuses are caught by combining the identification result and the positional information obtained via the smartwatch. Moreover, we may derive efficient techniques to catch octopus by comparing acceleration data and additional data, such as angular acceleration, between veteran and rookie fishermen.