Development of a Marketing System for Recognizing Customer Attribute Using Pressure Sensors

In this paper, we propose an automated marketing information system for recognizing customer attributes using pressure sensors and artificial intelligence technology. In this case, the customer attribute refers to the gender of customers in retail outlets such as supermarkets, shopping malls and shopping centers. In Japan, many retailers are now facing problems of low productivity per unit selling area. The ability to recognize customer attributes enables retailers to target a specific customer class, which can improve responses to special sales promotions and lead to improved sales. At the place of measurement, such as the entrance or shopping aisles, the system can recognize the attributes of various customers in detail. The apparatus used in this study is a pressure sensor that has the advantage of not invading customer privacy. Customer gender is estimated by measuring foot size, foot shape, step and weight. From the obtained experimental results, we verified that the proposed system can recognize customer attributes. Our research can contribute to raising the productivity of a retail area using information technology.


Introduction
The tertiary sector is an important sector accounting for nearly 70% of the Japanese economy (in terms of GDP and employment). Within this tertiary sector, the proportion that service industries (in a narrow sense) contributed to Japan's GDP increased from 41% in 1990 to 44% in 2000. While the role of service industries is increasing, its rate of productivity growth is relatively low compared with that of the domestic manufacturing industry or service industries in other countries. Thus, achieving innovation and productivity improvement in service industries is important for the development of the Japanese economy, which is suffering from drastic changes in the demographic structure due to a dwindling birthrate and an aging population.
The government of Japan declared that the growth rate of productivity per manhour should be increased by 50% over the next five years in order to realize growth under conditions of a declining population as well as to maintain a high quality of life. A strategy for service innovation that improves productivity by IT utilization was developed. (1) The Ministry of Economy, Trade and Industry (METI) made a proposal for improving the efficiency of investment in IT and raising the standard of service industries. (2) Moreover, METI promoted some survey programs to support service industry productivity improvements. (3,4) The Third Science and Technology Basic Plan, (5) which was established by the Japanese Ministry of Education, Culture, Sports, Science and Technology, pointed out that continuous improvement of labor productivity is the key for sustaining Japan's economic growth and overcoming population decline in an aging society with fewer children. Moreover, it is considered that in service areas for which productivity is low from an international perspective, innovation in science and technology may greatly contribute towards improving international competitiveness. (5) In order to nurture core human resources that greatly contribute towards improving service industry productivity and international competitiveness, MEXT developed human resource development programs for service innovation. (6) The above developments indicate that it is very important to achieve productivity increases in the service industry, and utilizing IT is the most effective method to encourage productivity improvements. In order to respond to this trend, an academic society, the Japanese Society for Artificial Intelligence, published a journal special issue "Service Innovation and Artificial Intelligence." (7) The aim of our research is to improve the productivity of the retail industry in the service sector. Specifically, we develop an automatic marketing information system for recognizing customer attributes in retail areas of outlets such as supermarkets, shopping malls and shopping centers. The ability to recognize customer attributes enables retailers to develop strategic sales promotions that are well suited to their customers and can boost sales. Now, there is a marketing information system using silhouette, footfall and weight. (8) This system can identify human gender and count customers with high accuracy. This system uses accelerometer, tactile sensor and camera. Generically, identification ratio can augment using many types of sensors. However, using many types of sensors increases the possibility of usage restrictions and restriction of usage environment. In addition, the large-scale system increases the cost. Moreover, the camera invades customer privacy. Thus, in this study, we propose a marketing information system that uses one type of sensor, which is not a camera with high identification ratio.
In other words, our research is a scientific and engineering approach for the service sector to produce productivity improvements.

Present situation of Japanese retail industry
The Japanese distribution industry (wholesale and retail industry) accounts for nearly 14% of the Japanese economy (in terms of GDP). The distribution industry is the second largest industry after manufacturing, and is the largest service industry. However, competition in the distribution industry is expected to become even more intense since the Japanese domestic market will become smaller with the declining population. The productivity of the retail industry is relatively low compared with the domestic manufacturing industry or other nonmanufacturing industries. (9) Thus, achieving productivity improvement in service industries including the retail industry is important for the development of the Japanese economy. Under these circumstances, the government of Japan suggests that information technology can play a central role in improving productivity.

Marketing information systems
For market managers, the acquisition of information on the attributes of customers and analyses of their behaviors play an important role in determining management strategies. In particular, in self-service shops where customers walk freely to check and choose goods, such as supermarkets, their behaviors are variable and much latent information can be obtained by investigating their behaviors, e.g., interest and hesitation due to goods and new behaviors when new goods appear. The marketing information system in this paper is a system for recognizing customer attributes. There are many systems for recognizing human attributes, such as gender and age. Most of these systems use video cameras and image processing techniques. However, recognizable facial images obtained by video camera invade customer privacy. Therefore, these systems avoid this problem by deleting images from data storage. However, customers' impressions of a retail outlet are important, so video recording of customers is preferably avoided to prevent unpleasant feelings and the invasion of customer privacy. Thus, to obtain attributes on customers, other systems are required.

Function of recognizing customer attribute
The customer attribute is the customer gender in this paper. For the sake of designing a system for determining attributes of market customers, we present a method for discriminating genders of customers, including visitors in addition to purchasers, using pressure sensors. Since the pressure sensors extract features on human characteristics without recording humans directly, in contrast to video checks, customer privacy is not invaded. Combination of the proposed system and the POS system would enable market managers to obtain valuable information for management strategies, e.g., the number of male and female purchasers among all the customers at an area in a supermarket.
In this paper, we focus on the gender of customers in a supermarket. Generally, supermarkets periodically change goods and the location of goods, and customers do not always visit a supermarket having decided what to buy. On the other hand, most customers who visit a convenience store already know the locations of most goods.
Therefore, we have assumed that supermarkets are more suitable than convenience stores for investigating market strategies using data on the attributes of customers.

Pressure sensors used in the proposed system
The proposed system uses pressure sensors (Xiroku, Inc., LL sensor). This sensor is composed of a rectangular panel (length: 80 cm, width: 60 cm).
The merits of utilizing this sensor are easy installation and easy use. The sensor functions when the lighting condition is poor. Additionally, it never acquires customer image information directly, which is the most important feature when an apparatus is selected for monitoring customers.
Although systems for monitoring people with video cameras are sometimes applied into retail areas, camera use is not always preferable. For example, if the retail area stocks expensive goods like jewelry or is open 24 h a day, like some convenience stores, customers will accept cameras for security. However, if the selling area stocks regular goods and foods, like a supermarket, in most cases, because store managers regard customer impressions of the selling area as important, video checks of customers are avoided because of the unpleasant feeling that is generated and the invasion of customer privacy that occurs. Thus, to obtain attributes on customers, these retailers require other systems instead of a video check. A pressure sensor enables retailers, such as supermarkets, shopping malls and shopping centers to avoid invading customer privacy, which is of special importance to retailers.
The outline specification of an LL sensor is shown in Table 1 and the structure of the LL sensor is shown in Fig. 1. This sensor has 4,800 pressure-sensitive points. Each pressure-sensitive point has a simple structure that is composed of a cell part with an orthogonal coil and a component consisting of a metal piece and an owing elastic body. When the metal piece approaches the cell part owing to pressure, electromagnetic coupling increases. Therefore, change in of electromagnetic coupling indicates pressure level.

Method of recognizing customer gender
In order to recognize customer gender, we use a Support Vector Machine (SVM). (11)(12)(13)(14) An SVM was originally designed for binary classification. In addition, SVM has exceptional generalization capability, does not lapse into local solution, and SVM is useful for identifications of postural change. A SVM performs classification by constructing an N-dimensional hyperplane that optimally separates the data into two categories. In addition, we use a linear kernel in SVM. This is based on the result of a preliminary experiment.

Installation of pressure sensors
We use 4 of the aforementioned sensors that are identical. Figure 2 shows the arrangement of the pressure sensors. These sensors act as a single-panel sensor with an area of 320 cm × 60 cm. Figure 3 shows a sample output of footprints from the pressure sensor. The red part shows the high-pressure area, and the green and blue parts show the low-pressure area.

Obtained data
We obtained data of footprints using these sensors. Thirty people participated in the experiment (male: 15, female: 15). Data obtained from the sensors is structured by coordinates of pressure-sensitive points and values of pressure. The experimental process was as follows. The participants walk on the sensors as usual (with shoes). The sensors continually obtain data during walking. We extract 1 frame data from a data aggregate, which is a pair of footprints in a walking step. This data is extracted by one sensor of 80 cm × 60 cm, because the length of stride is within 80 cm. Therefore, data comprise 4,800 pressure values and 1 bit of gender information (0: male, 1: female). Table 2 shows an example of the obtained data. The labels x and y are the coordinates of sensitive points. In addition, the maximum x is 60; the maximum y is 80. We conduct the experiment 10 times per participant, and the total number of experiments is 300.  Table 2 Example of obtained data. No. x:

Experimental results
The analytical algorithm uses SVMs to discriminate obtained data for recognizing customer gender.
In order to obtain the discrimination rate, we use 10-fold cross-validation. Firstly, 10 percent of the obtained data is chosen as test data and the other 90 percent is regarded as learning data. Next, the ratio of correct discriminations of test data is calculated. After the above processes are repeated ten times by changing the set of test data and learning data, the mean of the discrimination rate is finally obtained. Table 3 shows the discrimination ratio of SVM. Table 4 shows the discrimination ratio of Linear Discrimination Analysis (LDA). Bold numbers in Table 3 and Table 4 are ratios of successful discrimination. The average successful discrimination ratio of SVM is 89.3%, and the average successful discrimination ratio of LDA is 82.3%. Moreover, Table 5 shows both average ratios.

Consideration
The average successful discrimination ratio of SVM is 7.0% higher than the discrimination ratio of LDA. In particular, there is a significant difference in the female successful ratio between SVM and LDA. Therefore, we consider that SVM is a more effective method than LDA for this discrimination. The experimental data includes weight, foot size and length of stride. We consider that the difference between the genders is mainly due to weight. The successful ratio of female is 40% in one of the experiments. This experiment showed a lower pressure than another experiment. At this pressure, recognizing gender could not be difficult. The cause of this phenomenon derives from extracted data that shows the condition before the participant's foot leaves the pressure sensor. To resolve this problem, we need to have a threshold for selecting pressure. This is the goal of future research.

Conclusions
In this paper, we proposed a system for recognizing customer gender using pressure sensors and SVM. From the obtained experimental results, the ratio of successful discrimination is 89.3% on average. In addition, we compared the ratio of SVM with the ratio of LDA, then showed the effectivity of SVM.
The pressure sensor has the advantage of not being affected by changes in lighting conditions. Moreover, invasion of privacy can be avoided since the sensors do not record humans directly. This feature makes the system especially useful because the sensor does not have a legal status with regard to the invasion of the rights of individuals or the protection of personal information. In addition, it does not give customers an unpleasant feeling.
As future objectives, we are considering the estimation of the character of groups, e.g., a young couple, a mother and two children, and an elderly group by combining the proposed gender discrimination system and existing marketing systems that are able to detect groups and ages. Moreover, we will develop a system function that obtains the trajectory of buying behavior. This function will anticipate busy or nonretail areas. We think that we can make a system with these functions using a large-area pressure sensor. (15)