Biomimetic Hybrid Tactile Sensor with Ridged Structure That Mimics Human Fingerprints to Acquire Surface Texture Information

For robots manipulating objects, more sophisticated manipulations would be possible if surface information about the object were available. This information becomes even more critical when the manipulation is to be carried out in an unstructured environment. Tactile sensing has been the norm for collecting such surface information. In this paper, a sensor for recognizing surface texture was developed, in which a ridged structure on its surface, which generates vibrations when moved along the surface of the object of interest, is used. The pattern used for the ridged structure combines two distinct patterns that are found in different parts of a human fingerprint. In this work, sensors with two distinct fingerprint-like structures were first developed and tested to acquire surface texture information. Texture features were also extracted and identified with these sensors. Then a hybrid pattern was created by combining the two previously tested distinct structures. Finally, a series of experiments was conducted to compare the performance of sensors with a ridged structure having a single fingerprint pattern and a hybrid sensor whose ridged structure consists of two fingerprint patterns. In the experiments, the sensors collected surface information from nine different textured samples. Through these experiments, it was confirmed that the sensor using the hybrid pattern has better surface identification capabilities than the sensors using a single pattern.

For robots manipulating objects, more sophisticated manipulations would be possible if surface information about the object were available. This information becomes even more critical when the manipulation is to be carried out in an unstructured environment. Tactile sensing has been the norm for collecting such surface information. In this paper, a sensor for recognizing surface texture was developed, in which a ridged structure on its surface, which generates vibrations when moved along the surface of the object of interest, is used. The pattern used for the ridged structure combines two distinct patterns that are found in different parts of a human fingerprint. In this work, sensors with two distinct fingerprint-like structures were first developed and tested to acquire surface texture information. Texture features were also extracted and identified with these sensors. Then a hybrid pattern was created by combining the two previously tested distinct structures. Finally, a series of experiments was conducted to compare the performance of sensors with a ridged structure having a single fingerprint pattern and a hybrid sensor whose ridged structure consists of two fingerprint patterns. In the experiments, the sensors collected surface information from nine different textured samples. Through these experiments, it was confirmed that the sensor using the hybrid pattern has better surface identification capabilities than the sensors using a single pattern.

Introduction
Tactile sensors have many benefits in various robotics applications; in particular, they have proved useful for challenging manipulation tasks. (1) They provide information about the external environment, such as the contact area, temperature, vibration, and pressure, as well as providing sliding detection. (2) Typically, tactile sensors are more sensitive than other sensors, such as visual or acoustic sensors. In recent years, tactile technology has been recognized as one of the fundamental technologies in robotics that will help achieve higher functionality and improved safety. (3) Recently there have been many successful applications of traditional tactile sensors. They are particularly suitable in industrial applications where the working environment is carefully predefined. (4) They can detect the existence of an object, the presence of contact, and the intensity of the contact forces. However, as the areas of robotic application evolve, the environment in which robots operate changes. Although robots have been successfully working in a repetitive and predictable manner under a structured environment, in the near future, it is hoped they will be able to carry out much more complicated tasks while gathering information about the unstructured environment around them. (5) As a result, it is becoming more important to recognize and manipulate unknown objects rather than just perform specific tasks defined in advance. Handling an object without any information about it is a very difficult task. If an operating system does not have enough information about an object, it is impossible to determine its shape and color. (6) Also, the vertical and shear forces along with the strain applied to an object would be a good combination to know for detecting the hardness of the object. (7) Various types of sensors have been developed to obtain information on the surface of an object. These sensors are mainly divided into a visual-based method (8,9) and a tactile-based method. (10,11) Vision-based sensors take advantage of the superior resolution of the image sensors. (12) This sensor requires a sufficient distance between the point of contact and the image sensor to obtain an image. It causes the disadvantage of increasing the overall size. A tactilebased sensor, where it is difficult to integrate a large number of tactile cells, has a limitation in resolution even if it is made in the form of an array. However, it can be implemented in various ways such as film form (13) or fabric from. (14) In addition, different approaches can be used depending on the purpose. (15) In particular, it is effective when obtaining information about depth among surface information. (16) Using microvibration, it is possible to detect subtle differences in depth, which are difficult to understand with a vision-based sensor.
In order to measure the texture of an object using a tactile sensor, the examination of the principles of the human tactile function should be the first step. (17) We obtain tactile information from a number of mechanoreceptors located in the dermis layer of our skin. There are four receptors: the Meissner corpuscle, Merkel corpuscle, Pacinian corpuscle, and Ruffini cylinder. They are known to react in contrasting ways to stimulation. The Meissner and Merkel corpuscles react to pressure and touch. The Pacinian corpuscle reacts to vibration and deep pressure, whereas the Ruffini cylinder reacts to the elongation of the skin. The Merkel corpuscle and Ruffini cylinder are slowly adapting (SA) mechanoreceptors, and the Pacinian corpuscle and Meissner corpuscle are rapidly adapting (RA) mechanoreceptors.
There are two ways to recognize the surface of an object in contact with our skin. (18,19) First, there is the static spatial recognition of the force needed to manipulate it, for which information on the weight and hardness of the object as well as the friction coefficient of the contact surface are required. Therefore, enhanced identification of an object is needed for a robot to work in an unstructured environment. Objects can be identified by their shape, color, hardness, and texture. Visual information can be used to develop a method that collects texture information using spatially evenly distributed sensors. Second, there is transient time-dependent recognition, which is sensitive to time-dependent variations of the skin-object interaction, especially their relative movements. SA receptors detect spatial deformation of the skin, whereas RA receptors sense temporal deformation. SA receptors recognize the shape of an object by measuring the cutaneous pressure, for example, simply by touching the surface of an object with a human fingertip. On the other hand, RA receptors detect variations in the skin-surface interaction that arises with relative movements such as rubbing. (20) Vibration signals detected by RA receptors can be processed into texture information about an object. The RA receptors, particularly the Pacinian corpuscle, play a prominent part in the recognition of the roughness of the texture. For example, if someone wants to know the texture of two different objects, they must rub their fingers along each surface; simple contact or a pressing motion is not enough. Furthermore, SA receptors are not very effective for recognizing the characteristics of extremely fine surfaces, because these receptors would have to be very densely packed for successful identification of a fine surface. As a result, it is insufficient to try to mimic the function of only SA receptors in a tactile sensor for recognizing texture. If the sensor uses both RA and SA receptors, it will have excellent performance. (21) However, this entails structural complexity. RA receptors show better performance if only one receptor is used for structural simplicity because they mainly use vibration for the identification of a surface, which allows a low spatial distribution density.
To create a texture sensor that replicates RA receptors, a scanning process involving rubbing the surface of an object is necessary. This process is in contrast to the fact that a normal force can only be measured by touching. Instead, the generated vibration signal is a dynamic signal. These dynamic signals have various frequency components that can be used to extract texture features through appropriate signal processing. (22) Haptic texture perception proceeds through a predictive model or an artificial neural network. (23) There has been research on the data processing approach to achieve reliable screening results. (24) If a tactile sensor is able to record vibration signals more accurately, its performance will be improved. To better detect vibration signals, humans have fingerprints. The primary role of a fingerprint is not to increase the frictional force when gripping an object but to enhance the vibrations felt during the skin-surface interaction. (25) To achieve this, fingerprints convert horizontal motion between the skin and an object's surface into a vertical movement between the skin and mechanoreceptors. A sensor structure that simulates human fingerprints has been developed by Salehi et al., (26) who demonstrated how the fingerprint structure affects vibration induction. Furthermore, the effects of the geometry and the sliding direction of the fingerprint were studied by Oddo et al., (27) who showed that fingerprints made of straight lines have different characteristics from those made of curved lines in terms of roughness encoding.
We conducted a study into how to improve the performance of distinguishing the textures of the surfaces of various objects. Fingerprint structures with two distinct geometries were adopted to accomplish this. Each fingerprint structure was created by simulating human fingerprint structures. The vibration signals generated from these fingerprint structures when they are in contact with an object were measured, and a series of signal processing steps extracted the texture's features. When each fingerprint pattern was used for measurement independently, the signal profiles produced by each one were different. Moreover, when the two patterns were combined side by side and used together, we observed increased performance at distinguishing the textures of surfaces.

Tactile Sensor System
This section describes the two parts of the tactile sensor system. One part is the hardware that measures vibrations. This vibration measurement simulates the human RA receptors by using an artificial fingerprint structure. The other part distinguishes the surface of the object using the signal received from the sensor. We will describe how to post-process the vibration signal in the following. An algorithm for classifying surfaces using the extracted signal is also presented.

Sensor design
The developed sensor was fabricated by simulating the fingerprint structure and RA receptor. It consists of an artificial fingerprint structure that mimics a human fingerprint structure and a piezoelectric element that mimics an RA receptor.
The artificial fingerprint structure amplifies the vibration generated when the sensor rubs the target surface. When the fingerprint structure touches the surface and slides horizontally, vibration occurs owing to the stick-slip phenomenon. (28) Oddo et al. reported that the vibration pattern caused by the stick-slip phenomenon changes according to the shape and direction of the fingerprint structure. (27) They found that a curved fingerprint structure was more effective when scanning in multiple directions, but no details were provided on which fingerprint pattern would be more advantageous overall. In this paper, sensors have two curved fingerprint structures as follows.
As shown in the rectangular shaded area in Fig. 1(a), the front edge of a human fingertip has sinusoidal wave patterns. Note that we often first touch a new surface with our fingertips. On the other hand, in the circular shaded area in Fig. 1(a), the center of the human fingertip has a concentric circular pattern. This part is used when the finger is in contact with a large surface area. Observing human fingerprint patterns and their contact situations, we selected the sinusoidal wave pattern and the concentric circular pattern to determine the dependence of the sensor efficiency on the pattern.  The ridge pattern is designed with multiple intervals in order not to generate a single resonance frequency. This is essential because time domain data are also used in the classification instead of using only the frequency domain data. If the ridge spacings are uniform, the signal at a particular frequency can resonate, and regions of other frequencies are blurred. (25) That means the loss of information in the time domain data. Then the classification of the surface may not be easy. Hence, the ridges of the patterns were formed with uneven distances of 0.9, 1, and 1.1 mm to produce different vibratory signals from one input.
To facilitate the role of a fingerprint structure, a silicone material was adopted. Firm silicone is preferred for a structure to detect a ridged texture effectively. Also, it must be resistant to external chemical damage or contamination. Most importantly, it should sustain a constant response to horizontal forces in a situation where the sensor is rubbed against a surface. Polydimethylsiloxane (PDMS) was considered the most suitable silicone material to satisfy our requirements. (29) Thus, the fingerprint structure of the developed sensor was made using PDMS (Sylgard 184, Dow Corning). A fingerprint mold was fabricated using a 3D printer (Object24, Stratasys) to form the PDMS into a fingerprint shape. VeroWhitePlus material was used for the mold, as shown in Fig. 2(a). The PDMS was poured into the mold then cured at room temperature for 24 h.
The sensing element installed under the fingerprint structure is made of a piezoelectric material. Piezoelectric materials convert the vertical vibration signal received from the fingerprint structure into a voltage signal. Hidaka et al. placed a fingerprint structure on the outside and a sensing element inside to fabricate a sensor. (30) However, they observed the lowfrequency band with a strain gauge. A sensing mechanism that measures high-frequency signals, such as one made of piezoelectric material, is advantageous to simulate the RA receptor. A PVDF film was used as the piezoelectric material. PVDF film is freely molded and has flexible properties at small thickness. The PVDF film was polarized in the d33 direction for high sensitivity along with vertical vibration. The whole surface of the PVDF film was coated with graphite to form an electrode. The PVDF film was 100 µm thick.
A sensor was fabricated by combining the PDMS fingerprint structure and PVDF sensing elements. The molded PDMS was cut along the dashed line shown in Fig. 2(b). The cut PDMS structure was attached to PVDF film with silicone tape. The sensor attached to the curved tip is shown in Fig. 2(c). The voltage signal output from the sensor was passed through an analog filter and then measured through a data acquisition (DAQ) instrument at a 1 kHz sampling frequency (PXI 4472B, NI). The differential amplifier had a gain of 10, and a voltage follower was installed to reduce noise. A low-pass filter with a cutoff of 50 Hz was set at the rear end. The signal was collected with LabVIEW.

Texture identification
The developed sensor rubs the target surface and receives a vibration signal in the form of a voltage. Vibration signals are time-series data, and signal processing is essential to distinguish surfaces through them. The determination of the critical parameter that represents the texture information is important. Hence many groups have extracted features from sensor output data by various methods. Fishel and Loeb adopted an active classification approach to the problem of surface identification using artificial fingers sliding over surfaces. (31) They selected three distinct properties, traction, roughness, and fineness, that could be used for surface identification. Also, they developed a feature extraction method to map the measured dragging forces in their linear stage and the vibrations of the finger. Chathuranga and co-workers used the covariance of values obtained from two accelerometers as the feature and conducted classification work using signal processing. (32,33) Dallaire et al. used seven features from the signal found through a statistical approach. (24) In the present study, four features are selected using only the vibration signal from surface contact. When distinguishing textures, it is advantageous to use features related to not only the frequency domain but also the time domain. (34) Observing the signal in the frequency domain reveals the periodicity of the surface. Conversely, by observing the signal in the time domain, aperiodic information on the surface can be obtained. In the frequency domain, the spectral centroid is used; then, the variance, skewness (third moment), and kurtosis (fourth moment) are also used to represent a surface's characteristics in the time domain.
The spectral centroid and principal frequency are mainly used to generalize signals in the frequency domain. The spectral centroid characterizes a signal in the frequency domain, providing a broader representation of the overall frequency signal than the principal frequency. The principal frequency does not represent all the components of the texture; since only the most characteristic frequency is used, the other details are ignored. The spectral centroid, however, does not ignore other details of signals, and most of the characteristic frequencies have a significant impact. For this reason, the spectral centroid is used to identify the overall composition of the surface texture.
The variance, skewness, and kurtosis are obtained from a signal in the time domain. The variance represents the roughness of the surface. Since a piezoelectric signal is a pair of positive and negative signals, the average value of the signals does not represent the strength of the signal. However, the variance represents the distance from the average, regardless of the sign. Therefore, it can represent the intensity of the signal, in other words, the roughness of the surface. The skewness represents the symmetry of the signal. For example, a large absolute value of skewness appears on a surface that becomes rougher in one direction. It is possible to know whether there is a trend in the change on a surface through the value of skewness. The kurtosis is a measure of how outlier-prone a distribution is. A surface with irregularities can be distinguished by using the kurtosis. For example, sparsely populated surfaces with large bumps have large values of kurtosis.

Experiments
We performed two experiments. The first experiment was to confirm that the wave and circle patterns give different results. A wave-pattern sensor and a circle-pattern sensor were fabricated, and the test samples were then measured by each sensor. The results were compared, and it was examined whether the two patterns created signals with distinct characteristics. In the second experiment, unlike the first experiment in which each pattern was used independently, a sensor in which the two patterns were used together was tested. If two fingerprint patterns are used in one sensor, they are bound to be affected by each other. We checked whether using the two patterns together can help to distinguish the surface despite interference.
The experimental apparatus, shown in Fig. 3, was configured to set the contact force when sliding begins. The linear guide ensures that the sensor maintains a stable constant velocity. The contact force was set to 1.5 ± 0.2 N, a value that has been used in many studies to allow stable sliding. (35,36) The constant velocity of the linear stage was set to 72 mm/s. Each sample was tested ten times.
Surfaces with various characteristics are needed to check whether the developed sensor can effectively distinguish surfaces. Hoelscher et al. set the following conditions on an object to evaluate an algorithm they developed. (37) The object should be made of one material, be large enough to be scannable, have a flat shape, and not be so sharp as to damage the silicone. We selected nine surfaces with repetitive and nonrepetitive patterns that satisfied these conditions, as shown in Fig. 4. Sample #1 is an acrylic plate that has a flat, defect-free surface. Sample #2 (felt) has a soft surface. Sample #3 is a Korean traditional paper (Hanji), which is soft but has a complex surface texture owing to the wood tissue used. Samples #4, #5, and #6 are materials with rough surfaces: sample #4 is a towel, sample #5 is a loofah, and sample #6 is a synthetic sponge. Sample #7 is a wood sheet. Sample #8 is corrugated cardboard and sample #9 is silver dots, with both having a homogeneous geometry.

Experiment 1: distinct data from different patterns
The nine selected surfaces were identified by the sensors with wave and circle ridged patterns. The fabricated patterns are shown in Figs. 5(a) and 5(b). Through this experiment, it was confirmed that the obtained signals were different for the different patterns. This is because the amplified vibration signal depends on how the different ridged patterns interact with the same material surfaces. The results for the two patterns are shown in Table 1. Each  pattern was analyzed to obtain the spectral centroid, variance, skewness, and kurtosis. The experimental data are average values of ten measurements from each surface. This result shows how the wave fingerprint pattern and the circular fingerprint pattern show different results. The correlation coefficients between the two patterns are as follows: spectral centroid 0.7115; variance 0.7443; skewness 0.4013; kurtosis 0.9067. The closer to 1 the correlation coefficient is, the more direct the proportionality, and the closer to 0, the weaker the relationship. (38) There is a strong correlation since the two fingerprint patterns measure the same contact surface in the sensor of the same structure. However, the two patterns do not give the same result.
Looking at the results for the wave-pattern sensor, it can be seen that the nine samples generally have different values and can be easily distinguished clearly. The exceptions are samples #5 and #6, which are difficult to distinguish except by the skewness. The circle-pattern sensor is also able to distinguish the nine samples in general. However, samples #2 and #3 have similar feature values, except for the skewness. These results are reasonable because samples #2 and #3 and samples #5 and #6 have physically similar textures. However, to improve the performance of the tactile sensor, more accurate distinction is needed. The wave-pattern sensor distinguishes better between samples #2 and #3 because it shows a larger difference in skewness. The circle-pattern sensor has different spectral centroid and skewness values for samples #5 and #6, making this sensor better for their classification.
These results confirm that the vibration patterns depend on the geometry of the fingerprint structure. The geometry may be favorable or adverse depending on the object surface. If the results of the two fingerprint patterns are used, better classification performance can be obtained. However, this requires two sensors and two active touches.

Experiment 2: application of hybrid sensor
A better way to distinguish the surface is to install two fingerprint patterns on one sensor; then the surface can be well distinguished with one active touch. As shown in Fig. 5(c), a hybrid pattern sensor was manufactured to check the degree of interference that may occur. The substrate of the wave pattern and circle pattern of the hybrid sensor have the same size, and the vibrations from the surface are acquired at the same time by both parts of the sensor. The left blue rectangle is the part of the sensor with a wave pattern (referred to as sensor 1 for convenience) and the right red rectangle is the part with the circle pattern (sensor 2). Figure 6 presents surface information measured with the hybrid sensor. The x-axis corresponds to data from sensor 1 and the y-axis corresponds to data from sensor 2. The basic signal detection of the hybrid sensor unit is less sensitive than that when the wave-and circlepattern sensors are used individually. This is because the interference between the patterns may not be eliminated completely. However, the hybrid sensor is advantageous when interpreting the measured signal to identify textures. For example, in Fig. 6(a), we can see that according to sensor 1, samples #1 and #3 have similar textures since the sensor outputs are in the range from 15 to 18, whereas according to sensor 2, they have considerably different textures. Consequently, the proposed hybrid sensor offers better texture identification capability owing to the enhanced interpretation of combined signals. As demonstrated with the example, the proposed sensor can differentiate surface texture through a characterization process as follows. Data sets are first sorted on the basis of the spectral centroid to categorize the surfaces, as shown in Fig. 6(a). However, in this process, there are overlapping classes: samples #4 and #7 are not clearly differentiated, and neither are samples #5, #6, and #8. Next, the variance can help distinguish between samples #4 and #7. With sensor 1, sample #4 (marked with inverted triangles) has a variance of about 0.14, whereas sample #7 (right triangles) has a variance of about 0.07. Therefore, the two sets can be clearly separated from each other. Skewness and kurtosis can also be used to improve accuracy. A similar approach is applied to classify samples #5, #6, and #8. Sensor 2 gives variance values of 0.29, 0.13, and 0.06, respectively. In addition, inspecting the kurtosis of each sample increases the decision accuracy. With sensor 1, samples #5, #6, and #8 can be distinguished from their kurtosis values of 16.8, 11.3, and 8.7, respectively. If either sensor 1 or sensor 2 were used alone, it would be much more difficult to distinguish the textures. Therefore, we can see that the proposed hybrid sensor enhances the surface texture identification performance despite its small form factor.
In this work, the Davies-Bouldin index, (39) Dunn index, (40) and silhouette index (41) were used to evaluate the sensors. Clustering evaluation methods have been used for quantitative evaluation of the discrimination performance of sensors to determine how well different groups are distinguished from each other. If the measurement with the hybrid sensor is better for identification than that with the individual sensors, the texture discrimination performance will also be better. Table 2 shows the results of evaluating each sensor. The lower the Davies-Bouldin index, the better the performance, whereas the higher the Dunn and silhouette indices, the better the performance. As shown in the table, the hybrid sensor shows better results than the individual sensors in terms of the Davies-Bouldin and silhouette indices, whereas the circle-pattern sensor shows better overall results than the hybrid sensor in terms of the Dunn index. However, the overall results of the three sensors do not show a significant difference. Unlike other metrics, the Dunn index uses only the largest intracluster distance and the smallest distance between the two clusters. As a result, the degree of similarity of an extreme group is used instead of the similarity of an average group. Thus, the accuracy tends to decrease as the number of groups increases. Since there are nine groups in this experiment, some results are evaluated poorly by the Dunn index. In contrast, the Davies-Bouldin and silhouette indices show that the hybrid sensor is superior to the single-pattern sensors. The above two evaluation methods use the average of the dissimilarity calculated from each group. Hence, they show results that reflect more data. From these results, it can be seen that a hybrid sensor is more effective at distinguishing each texture than individual single-pattern sensors.

Conclusions
In this paper, we proposed a hybrid sensor with two distinct patterns of its ridged structure; these patterns mimic different parts of a human fingerprint. The developed tactile sensor can successfully distinguish various surface textures. It uses the vibration signal that is generated when the sensor rubs against the surface of an object. Each fingerprint-patterned structure generates different vibration signatures when rubbed. Using two different patterns in the ridged structure of the sensor improves its discrimination accuracy.
Human sensory receptors were simulated with a biomimetic approach when fabricating these sensors. PDMS was used to simulate skin and its fingerprint structure, which converts horizontal vibrations into vertical vibrations. Molds with different fingerprint patterns were printed, with the designed fingerprint structure being formed in PDMS. A piezoelectric sensing element was used to simulate the RA mechanoreceptors beneath human skin. Vibrations were measured using the piezoelectric properties of the PVDF film attached under the PDMS ridged fingerprint structure to mimic the human sensory system.
Experiments were conducted to confirm the operation of the developed sensors. Two sensors with different single-pattern fingerprint structures were fabricated to measure the surfaces of nine objects. It was confirmed that the different fingerprint structures generate different vibration signals. We then confirmed that a hybrid sensor with both types of fingerprint structure is better than the single-pattern sensors in distinguishing the textures. Davies-Bouldin, Dunn, and silhouette indices were used for the evaluation. The hybrid sensor had slightly inferior performance to the circle-pattern sensor in terms of the Dunn index but superior performance in terms of the Davies-Bouldin and silhouette indices. It was thus confirmed that the hybrid sensor is better at distinguishing textures than the sensors with a single pattern type.