Biologically Inspired Reasoning Scheme for Semantic Sensor Network Ontology in Efficient Disaster Surveillance

Semantic sensor web (SSW) technologies from the sensor web enablement (SWE) standard of open geospatial consortium (OGC) are useful for surveillance in disaster situations. In SSW, the characteristics of sensors are represented as a semantic sensor network (SSN) ontology, which enables semantic interpretation and situation learning. For efficient disaster surveillance, various sensors are deployed over a large-scale geographic area. Furthermore, mobile devices carried by citizens can be recruited during emergencies. However, the heterogeneity of the recruited devices results in the need for additional processing of data attributes. To overcome the shortage of resources during an emergency, a biologically inspired learning scheme can be adopted. The scheme is based on the spike rates of each sensor, thus ignoring much of the information by calculating the relative timing between individual signals shared, and integrated semantic ontologies help deduce information from temporal and spatial contexts. Our approach focuses on the asynchronous and spiking nature of sensors and extracts relevant temporal features in spatial dynamics. We propose a scheme utilizing spike-timing-dependent plasticity (STDP) to process the vast number of signals sent from newly recruited sensors, which factors in the relative timing of signals. To achieve higher reasoning efficiency, mechanisms behind brain synaptic plasticity, specifically, latent inhibition, long-term depression, and long-term potentiation observed in the STDP learning rule are applied. These mechanisms enable a more suitable response inference under time-critical circumstances.


Introduction
For cooperation between surveillance organizations, semantic sensor web (SSW) technologies from the sensor web enablement (SWE) (1) standard have been proposed, which is the general standard enacted at the open geospatial consortium (OGC). (2) In SSW, the characteristics of sensors are represented as a semantic sensor network (SSN) (3) ontology. This enables semantic interpretation and situation learning. The works in Refs. 4-6 utilized the SSN ontology in monitoring or surveillance. For efficient disaster surveillance, various sensors are deployed over a large-scale geographic area. (7,8) Furthermore, mobile devices carried by citizens can be recruited during an emergency. (9,10) Such mobile devices are augmented to compensate the blind regions where the sensors are broken, and rectify their estimation accordingly. However, as the number of sensors increases, event signals generated from the sensors can contain redundant information. Also, the heterogeneity of the recruited devices means that the additional processing of diverse formats is required. Such processing requires time, energy, and high computational power in addition to reasoning cost. Although the integration of sensor data reinforces information, unrestricted random reception of sensor data can cause harm in time-critical circumstances. To efficiently use limited resources during an emergency, a biologically inspired learning scheme can be adopted. The scheme is based on the spike rates of each sensor, thus ignoring much of the information by calculating the relative timing between individual signals.
Shared and integrated semantic ontologies help deduce information from temporal and spatial contexts. Our approach focuses on the asynchronous spiking nature of sensors and extracts relevant temporal features as seen in the processing of a neuron. We propose a scheme using spike-timing-dependent plasticity (STDP) (11)(12)(13) to process the large number of signals sent from newly recruited sensors, which factors in the relative timing of signals. The previous work utilizing citizen sensors did not filter or factor the signals using a biological plasticity mechanism. Here, we introduce a new approach that can extract and weigh ontology from a specified data set using the STDP criterion.
To achieve higher reasoning efficiency, mechanisms behind brain synaptic plasticity are adopted. Latent inhibition (LI), long-term depression (LTD), and long-term potentiation (LTP) observed in the STDP learning rule are applied in the reasoning process of ontologies, generating a faster and better response.
The rest of the paper is organized as follows: In Sect. 2, we introduce SWE and STDP in relation to the proposed strategy. Then the problem caused by human sensor recruitment is defined. We continue with the adoption of our approach in the reasoning domain. In Sect. 3, we describe the experiments and give analysis results. In Sect. 4, we present our conclusions.

SSN web enablement
Semantic technologies for sensor networks improve semantic interoperability and integration in surveillance. (14) This allows sensors and the resulting data to be organized and managed through high-level interfaces. The sensor ontologies provide a framework for describing sensors and reasoning agents. The framework conforms to the OGC standards. The World Wide Web Consortium (W3C) initiated the Semantic Sensor Networks Incubator Group (SSN-XG) to develop the SSN ontology. SSN ontology models sensor devices, systems, processes, and observations. This incubator group was later transitioned into the Semantic Sensor Networks Community Group (SSN-CG). The SSN ontology enables the expressive representation of sensors, sensor observations, and knowledge of the environment.
Furthermore, SSW has been proposed for SSN ontology application in the SWE of OGC. A sensor web can be seen as a sensor network based on the Internet web service. In SSW, SSN ontology describes the abilities and properties of the sensors. It enables the semantic compatibility necessary for the syntactic interoperation of sensors and analytic inference of agents. Figure 1 shows an overview of the SSN classes and properties from an observation perspective.
(3) For better response inference, produced signals should be applicable in data analyzers and should be combined with others in real time. With the rapid spread of events, the coordination of rights, restrictions, and responsibilities between people and data are also constantly subjected to changes. For the real-time adaptation, rapid application of SSN ontology is needed.
The objective of semantic operations is to provide an interpretation of the information. The use of semantic measures of similarity is related to reasoning. The level of semantic similarity between concepts should be measured to help reasoning.
Several approaches evaluating the semantic similarity among concepts are surveyed in Ref. 1. Some approaches are based on measuring the shortest path length between concept nodes. One limitation of these approaches is that they only consider the relationship between concept nodes without giving them a different weight based on their importance.
In this paper, we evaluate semantic similarity in weighted ontology introducing the weight of each node, which is based on the timing of each event and defined in Sect. 2.2.
Although the proposed approach can be applied to any metric of similarity, the Wu and Palmer (WP) metric (15) is adopted in the study. The WP metric calculates the conceptual similarity by considering the depths of two terms along with the depth of the least common subsumer (LCS). Assuming that the LCS of two concepts x and y is the most specific concept that is an ancestor of both x and y, ConSim WP calculates the semantic similarity between the concept nodes C 1 and C 2 as follows: where N 1 , N 2 , and N 3 are distances from the LCS. We define a modified WP metric that considers the weighted ontology based on STDP. We named it the WPSTDP. The weighted ontology accommodates a weighted graph structure as shown in Fig. 2.
WPSTDP takes into account the edge weight from the importance weight of the concept node. The new metric is expressed by Eq. (2). A weight w that expresses the importance of each event of the ontology is assigned, with To define each node mapped with a weighted edge, importance is calculated following an STDP criterion. It gives a higher weight in LTP and a lower weight in LTD as shown in Sect. 2.2.

Citizen sensor reasoning
Refined citizen sensor data enable a disaster surveillance system to cover blind regions where the sensors are broken, and cover the malfunctioning sensors. (16)(17)(18) However, the increased amount of data and the heterogeneity of the recruited devices result in an additional computational cost.
With this in mind, the STDP conceptual model is adopted. From the asynchronous and spiking nature of these sensors, temporally correlated features are extracted. Dynamic eventbased sensor signals are factored in using STDP. Sequences of individual signals construct moving traces. Learning rules temporally correlate patterns from a sensor's signals. Signals are generated from the sensors in response to relative changes in a disaster. However, event signals that did not recently contribute to the post-signal activation are depressed.
To prevent sensory overload, a generic human brain uses LI to filter out repetitive unimportant events. LI is a mental process in which an individual does not create any meaning with a given stimulus. A common example of LI is when a person ignores a constantly repeating stimulus and automatically deems it insignificant or irrelevant. Without LI, an individual loses concentration directed towards a specific task. The application of LI to surveillance helps filter out a number of unimportant SNS signals. This is similar to the way the human brain achieves new information.
Furthermore, the STDP learning rule is implemented and follows the processes of human learning. This rule was found in biological neurons and is widely used in both computational neuroscience and machine learning. The process in the learning algorithm can be classified into self-organization and reinforcement learning. Self-organization is the gradual convergence of the neuron from a given stimulus to a specific pattern. As an advanced example of selforganization, in STDP, a synapse is strengthened or weakened. In STDP, the presynaptic spikes occurring immediately before the current postsynaptic spike strengthen the interconnecting synapses (LTP); otherwise, the synapses are weakened (LTD).
The mechanisms of LTP and LTD observed in the brain are applied in the reasoning process of ontologies. The mechanisms generate information for strengthening or weakening features acquired by dynamically controlling the weight for context recognition.
The weight change Δw j of a concept node j depends on the relative timing between presignal arrivals and post-signal responses. Let us name the pre-signal arrival times of node j as where F(Δt) denotes the STDP functions given in Eqs. (4) and (5) and illustrated in Fig. 3.
The width of the learning window is specified by τ, which is known to be 10-20 ms, where τ + and τ − determine the ranges of pre-to post-signal interspike intervals over which strengthening and weakening occur. A + and A − determine the maximum amounts of modification, which occur when Δt is close to zero. The parameters A + and A − may depend on the current value of the weight w j . Only the ratio A + /A − is relevant. For stability, A + /A − must be balanced. This ratio is generally assumed to be slightly smaller than unity to lessen the potentiation. This is because if LTD is stronger than LTP, A + < A − , and it would result in more rapidly converged reasoning.

Simulation Results
In this section, we will demonstrate how the learning scheme we propose can be used to extract overlapping features. To evaluate the efficiency of our system, we developed a simulated environment using MATLAB. The STDP rule was implemented as the learning algorithm, which is widely used in machine learning. Only signals followed by a pre-signal were activated, while others were depressed. These activated signals were strongly potentiated according to Eq. (1).
In this event-driven simulator, signal sequences were created by a Poisson process at various rates, in which the inter-arrival time follows an exponential distribution of rate λ. Each signal sequence represents the event trajectory of synapses, which were shown as a function of the firing rate. Plasticity typically requires 60 to 100 pre-post spike pairs. We compared more than 300 spike pairs that had the shortest time interval between the pre-post spikes. In STDP, LTP occurs when presynaptic spikes precede postsynaptic spikes by 0-20 ms defined as positive Δt, while LTD is induced when postsynaptic spikes lead the presynaptic spike by 20-100 ms defined as negative Δt. (12) The specific characteristics of the experiment were modified according to the suggested environment in Ref. 12. When deciding the intercepts for x and y in Fig. 3, A + was decided as 20 ms and the A − /A + ratio was decided as 1.25 to induce a higher effect. This made the synaptic weakening through STDP more apparent than the synaptic strengthening. Moreover, for the calculation of F(Δt) in Eqs. (4) and (5), τ + = 20 ms and τ − = 20 ms were defined according to the general assumption.
The semantic similarities were measured according to both the WP metric and the WPSTDP metric. On the basis of the previous WP metric, ConSim WP drawn from Eq. (1) is calculated. Likewise, the newly proposed ConSim WPSTDP drawn from Eq. (2) is calculated on the basis of the WPSTDP metric. To compare each calculated similarity, N 3 (shown in Fig. 2) was given as a constant. In the experimental result shown in Fig. 4, the constant designated for N 3 was 5 (N 3 = 5). Simulation resulted in producing a set of keywords for the reasoning system, and these were sorted according to their similarities. The output of reasoning was a set of ontology items with higher similarity values. Figure 4 shows the trend of calculated variations of similarities. Following the STDP rule, when LTP was applied, the similarity increased by 20-30% in general. In other cases, LTD was applied resulting in decreased similarity. Increased or decreased similarity values elaborate the ontology search result and improve the inference procedure efficiency.

Conclusion
Semantic technologies with SSN ontology can improve interoperability between sensors.
In this paper, we present a semantic web-based framework to facilitate disaster management using sensors. The framework provides the collection, sharing, access, usage, and management of data organized spatially. The result of this project shows that effective and efficient disaster management can be achieved, particularly regarding the inference of responses. The ConSim WPSTDP processing conducted prior to the inference procedure enhances it, resulting in better outcomes.
Also, by using a citizen sensor with proper filtering by STDP, the surveillance network can control the situation more easily and deduce better responses in emergency situations. Disaster response through better coordination of the involved agents improves the reasoning process, having all of the information available, accessible, and interoperable for use. The SSN ontologies act as a useful assistive framework for the development of a web-based inference system.
In this paper, we introduced the unsupervised learning scheme capable of extracting temporally overlapping features, using the STDP learning rule. This scheme extracts features directly from data and is also practical. The size of the reasoning-related data can be markedly reduced by using LI and LTD. The results of this paper showed high efficiency of dynamic citizen sensors. It could be used as a pre-processing layer for an intelligent inference agent, where the extracted features could be automatically labeled and higher-level object tracking can be performed.
The methodology and the model developed through this research can also be used in daily security other than disastrous events. Spatial data and related timing have been proved to be crucial for collaborative decision-making. Analysis based on their relative timing elaborates the inference procedure and produces better security responses.