A Study of Bottom-sediment Classification System Using Seabed Images

1Graduate School of Science and Technology, Niigata University, 8050 Ikarashi 2 no-cho, Nishi-ku, Niigata City, Niigata 950-2181, Japan 2School of Engineering, University of Shiga Prefecture, 2500 Hassaka-cho, Hikone City, Shiga 522-8533, Japan 3Graduate School of Instructional Systems, Kumamoto University, 2-40-1 Kurokami, Kumamoto City, Kumamoto 860-8555, Japan 4Abashiri Fisheries Research Institute, 1-1-1 Masuura, Abashiri-shi, Hokkaido 099-3119, Japan


Introduction
Hokkaido Prefecture has the highest production in Japan's fishing industry.Among the Hokkaido fisheries, the scallop is a key product, accounting for 28.3% of the fish catch. (1)In the fishing industry, it is necessary to understand the fishing-ground conditions and estimate the amount of available resources in order to stabilize the production volume, strengthen our international competitiveness, and enable a quick recovery when the fishing ground is damaged.
The improved performance and lower price of digital video (DV) cameras in recent years have enabled us to acquire underwater images.Unlike conventional sampling surveys, which directly collect target resources, fishery-resource surveys that use digital images can minimize the impact on the resource subjects and fishing grounds.We took seabed videos using a DV camera for a fishery-resource survey in the scatter scallop fishing grounds in the Sea of Okhotsk, Hokkaido.Off the coast of Monbetsu in 2015, we acquired about 0.02 km 2 of seabed videos.The length of the seabed videos that we acquired was about sixteen hours.We estimated the amount of scallops using seabed images.In addition, if we can grasp the environment in which the scallops inhabit, we can estimate the amount of resources with higher accuracy.However, since the seabed images are enormous, automatic measurement technology is required.With this survey, we can estimate the scallop population, understand the fishingground environment, and promptly grasp the situation in times of damage.However, it is very time consuming because experts are doing those work.
Currently, hanging and sowing cultures are mainstream growing methods in scallop fisheries. (2)nlike hanging cultures, sowing cultures are restricted by the depth of the shore profile, bottom currents, and seabed composition. (3)Particularly in bottom-sediment areas, scallops are known to have high habitat densities in ballast fields and low habitat densities in sand-based sediments.Such differences in environmental conditions not only affect the growth of the scallops but also their survival rate after release.However, in sowing culture, the fishing ground is located at a depth of 30-50 m.Therefore, we found it difficult to grasp the situation of the fishing ground.
Kamoshita et al. proposed a method of investigating the water-bottom surface using a sidescan sonar. (4)When using a side-scan sonar, we can acquire a wide range of data at once; however, the equipment is expensive and expert knowledge is required to analyze the acquired images.In order to grasp the environment in which the scallop is found, the spatial resolution is insufficient with the side-scan sonar.Furthermore, since the undersea environment changes with high frequency, an inexpensive investigation method is necessary.On the other hand, we have developed an inexpensive and efficient method of recording seabed images using a DV camera.We will describe this method in Sect. 2. In a survey using a DV camera, we can obtain images with a higher resolution than a side-scan sonar.In 2015 in Monbetsu, the whole fishing ground is 200 km 2 , and the area of the sea floor recorded is 0.02 km 2 .We recorded 0.01% of the whole fishing ground; however, this is too much data for the manual classification of the bottom sediment.
Deep learning is an effective method of extracting features from large amounts of data. (5)ai et al. proposed a deep convolutional network for zooplankton classification. (6)Deep learning is also used in the marine field.It enables automatic and highly accurate classification.In this study, we use a convolutional neural network (CNN) for bottom-sediment classification from the seabed images and propose a bottom-sediment classification system using CNN.This system enables us to quickly grasp the fishing-ground situation, which was not previously known.Moreover, we created a fishing-ground map based on the bottom sediment for visualizing the seabed environment.By creating maps, for example, we can release young shells to a place with good growth of scallops.

Bottom sediment
In this study, we classified the seabed environment of the scallop fishery off the coast of Monbetsu, Hokkaido.We classified four types of sediment: sand, ballast, gravel, and shell bank.Figure 1 shows images and Table 1 provides the features of each bottom-sediment type.Because the scatter scallop fishing grounds in Hokkaido are maintained, we estimated that the four environments related to the growth of scallops are dominant in the fishing grounds.The bottom sediment is related to the strength of the tide flow; generally, strong tide-flow areas produce coarse sediment, and weak tide-flow areas produce fine sediment.Scallops inhabit sand, ballast, and gravel environments.A shell bank is the accumulation of scallop carcasses, generally caused by stormy weather.The difference in bottom sediment affects the scallop growth and the survival rate of young shells.On the basis of expert opinion, we set the target value of classification accuracy to 90%.The scallops are covered with sand and only the edges of the shell are visible.

Ballast
Environment with broken shells scattered on the sand.Generally, scallops have higher habitat densities in ballast.

Gravel
Loose aggregation of rock fragments (greater than 2 mm).

Shell bank
Accumulation of scallop carcasses caused by stormy weather.

Recording apparatus
The apparatus used to acquire the images in Fig. 1 is shown in Fig. 2. In Fig. 2, the red part plays the role of the sleigh.The weights of the recording apparatus are about 110 kg in air and about 105 kg in water.The device has a width of 1000 mm and a length of 1850 mm.The cabinet is made of a 32 mm stainless steel square pipe, and the sleigh part is made of steel.Three lights, one DV camera, and a global positioning system (GPS) logger were attached to the apparatus.The camera is made by Sony (model number: HDR-CX550).In addition, we attached a 100 m waterproof housing to the camera.Three steps were needed to acquire the seabed data.First, the apparatus was submerged in the sea.The camera was about 75 cm from the seabed.Second, the apparatus was attached to a fishing boat, using a rope, and then towed.The towing speed was about 1-4 m/s.Finally, seabed videos were recorded.The video frame rate was 29.97 frames per second.
We have been acquiring over 100 km of seabed videos every year from the sea of Okhotsk, Hokkaido.Since the scallop fishing ground off the coast of Monbetsu is more than 30-50 m deep, almost no sunlight reaches the recorded videos.The recorded video is 1920 × 1080 pixels, and the bottom surface shown in the image has a horizontal width of 75 cm.The camera's shutter speed is set to 1/2000; therefore, there is little motion-blur influence.In this study, we continued recording videos of the seabed for about 80 min until the apparatus was submerged in the sea and pulled up.

CNN
The use of CNN, one of the classical deep-learning methods, can extract features from a large amount of learning data. (7,8)Dai et al. proposed a deep convolutional network for zooplankton classification. (4)To classify the four types in Fig. 1, we had difficulty in designing optimal feature quantities because similar features were mixed.On the other hand, if we had sufficient data, we could construct optimal feature quantities and classifiers using CNN.In this study, we used CNN for bottom-sediment classification from seabed images.The framework is shown in Fig. 3.The use of CNN enabled automatic high-speed classification.

Seabed image dataset
We used the seabed images taken in Hokkaido in August 2015 for the dataset.We captured one frame per second from a video clip.In addition, we divided the 1920 × 1080-pixel seabed images into arbitrary sizes and classified them according to expert opinion.In this study, we divided the seabed images into 256 × 256-pixel images and downsized these images into 64 × 64-pixel images for CNN.We used 36000 images from each class as the dataset.Ninety percent of the dataset was used for training data and 10% was used for test data.Figure 4 shows examples of each class used for the dataset.Color images were converted to grayscale images and used in the dataset.For the experiment, we used an Intel Xeon Processor, E5-2695 v4, as the CPU, a GeForce GTX TITAN as the GPU, 128 GB of memory, and an Ubuntu 16.04 LTS as the operating system.

Classification accuracy
Table 2 shows the accuracy of the bottom-sediment classification.The classification accuracies of sand, ballast, and shell bank were over 95%.However, the classification accuracy of gravel was only 76.50%.From the results in Table 2, we presume that color information is important for classifying gravel.Once we create the bottom-sediment classification model, we can process huge volumes of data using that model.
We classified the image blocks using CNN. Figure 5 shows the classification results for each block.In Fig. 5, the red, blue, green, and yellow frames indicate blocks classified as sand, ballast, gravel, and shell bank, respectively.

Seabed map
We created a bottom-sediment map based on the classification results.When creating the map, we identified the image's bottom-sediment type as the most represented type.For example, in Fig. 5(a), we can confirm eight blocks of sand and 20 blocks of ballast in the image.Thus, we defined the image's bottom-sediment type as ballast.In Fig. 5(b), we confirmed no blocks of sand, 12 blocks of ballast, nine blocks of gravel, and seven blocks of shell bank.We again identified the bottom-sediment type as ballast.In this study, we classified one image per 30 s from the seabed video and displayed the result on the map.We made three bottom-sediment maps, Line 1, Line 2, and Line 3, shown in Fig. 6.In addition, we visualized the proportions of the bottom sediment within a certain range of each line.In the scatter scallop fishery in Hokkaido, the fishing ground is divided into 500 m squares and managed per section.On the basis of this method, we visualized the proportions of the bottom sediment at every 500 m and the result is shown as a pie chart in Figs.7-9; again, red, blue, green, and yellow are classified as sand, ballast, gravel, and shell bank, respectively.While we acquired the latitude and longitude information, we cannot publish the exact location where we acquired the seabed videos.In this study, we created a map using the distance of the investigation.Line 1 is located about 10 km from the coast, Line 2 is located about 7 km from the coast, and Line 3 is located about 4 km from the coast.All lines are located parallel to the coastline.
Figure 7 shows that there is no gravel in Line 1.However, in Figs. 8 and 9, we can confirm the existence of gravel.From these results, we can observe that, in this area, gravel tends to be close to the coast.Figures 8 and 9 show that shell banks occur near gravel.On the basis of this result, we can estimate that shell banks occur in places where the tide flow is strong, e.g., a gravel field.

Conclusions
We proposed a bottom-sediment classification system using seabed images to visualize the seabed environment.We used CNN for the bottom-sediment classification and achieved an average accuracy of about 95% for three of the four types, and the gravel was 76.5%.In the three sediments excluding gravel, we achieved an accuracy of 90%.However, the misclassification rate of gravel as ballast was 20.90%.Since gravel and ballast have similar properties, we presume that this rate is not a big problem in practical use.Actually, it is expected that there are environments other than the four targeted in this paper on the ocean floor.As future work, we would like to add environments other than the four sediments to the classification targets.We obtained a wide range of high-resolution seabed information and classified it with high accuracy.Furthermore, we created a seabed map on the basis of the CNN classification results.With this bottom-sediment classification system, it became possible to quickly grasp the fishing-ground situation.In future, we would also like to discuss the bottomsediment estimation method of the entire fishing ground.
In this study, we used seabed images taken in 2015 for CNN and the seabed map.As future work, we would like to use seabed images taken in 2016 or 2017 and create a new map.In addition, we would like to combine this system with scallop inhabitation and thereby identify good fishing environments.We can then enable an efficient and stable fishing operation.

Fig. 5 .
Fig. 5. (Color) Classification results for each block.A red frame indicates sand, blue is ballast, green is gravel, and yellow is shell bank.(a)(b)

Table 1
Features of each sediment type in scallop fishery.

Table 2
Accuracy of bottom-sediment classification using CNN.