هستی شناسی فازی برای مدل سازی معنایی و شناخت رفتار انسان

We propose a fuzzy ontology for human activity representation, which allows us to model and reason about vague, incomplete, and uncertain knowledge. Some relevant subdomains found to be missing in previous proposed ontologies for this domain were modelled as well. The resulting fuzzy OWL 2 ontology is able to model uncertain knowledge and represent temporal relationships between activities using an underlying fuzzy state machine representation. We provide a proof of concept of the approach in work scenarios such as the office domain, and also make experiments to emphasize the benefits of our approach with respect to crisp ontologies. As a result, we demonstrate that the inclusion of fuzzy concepts and relations in the ontology provide benefits during the recognition process with respect to crisp approaches.

Human activity study is a complex but key aspect in the development of Ambient Intelligence (AmI) systems. Different techniques have been developed for activity and user modelling, and they may be classified as data-driven [1] and knowledge-based [2] approaches. The methods in the first category are aimed at providing robust models for handling human behaviour specific features using statistical and machine learning techniques. The best strengths of these models are their ability to handle noise, uncertainty, or incomplete sensor data [3], and they have proven to be accurate in different domains where semantics are not key. However, the need for training data and the time and performance required for these models are limitations in dynamic environments and situations where context-aware data prevail. Furthermore, data-driven algorithms do not offer abstract reasoning mechanisms that allow inferring the meaning of the actions according to their semantics [4]. On the other hand, knowledge-based techniques have been applied in pervasive computing environments to improve interoperability and adaptation to different context situations. Usually, context data sources are dynamic, continuously changing depending on the environment, not always mobile, known, nor taken into account in advance. For this reason, these methods show advantages with respect to data-driven models due to the inclusion of context management tools. Further features of knowledge-based techniques that are interesting for human activity representation are the possibility of providing both the environment and the user with semantics to aid in the context definition process, facilitate the definition and comprehension of human behaviours (e.g. machine readable and easier to interpret), and consequently, ease the development of new learning and recognition models able to better understand the meaning of human actions and execute logic reasoning about future needs, situations, or actions. In addition, all this can occur considering the context information where the activity is performed. Examples of knowledge-based techniques contain logic-based approaches [5] and [6], rule-based systems [7], and ontological models [8]. Despite the fact that most of the approaches about human activity recognition are focused in Ambient Assisted Living and Smart Home assistance [2], [9] and [10], another emerging scenario is the office/work domain and public buildings environments. In this case, the goals are aimed at improving energy efficiency and work assistance [11], [12], [13] and [14]. For instance, MOSES [15] localizes work staff and identifies the tasks they are doing at any moment, being able in this way to give advice on the remaining tasks to be carried out and warn about potential oversights or forgotten actions. Another example is iShopFloor [16], a multi-agent architecture to plan and control industry processes. However, semantic technologies have not been generally included into these models, although there are exceptions such as in intelligent meeting rooms [17] or maintenance of large buildings [16] and [18]. AmI scenarios in offices or work environments focus on easing the work in groups and optimizing the office space. For example, EasyMeeting [19] is an intelligent meeting room system that builds on the design of CoBrA [20]. RFID sensors embedded in the walls and furniture detect the presence of the users’ devices and clothing. On receiving information about the user’s context and intention, the broker sharing platform allows the activation of the projector, slide downloading, and lighting control. In [18], ontology-based interoperability is applied to Smart Spaces for a context-aware maintenance of large buildings, monitoring environmental variables, automatically detecting building-related faults, and executing multi-modal interventions. An architecture for an ubiquitous group decision support system, WebMeeting [21], is able to consider the emotional factors of participants and their associated argumentation processes. The system shows available information to the participants, analyses the meeting trends, and suggests arguments to be exchanged with others. Further interesting projects regarding activity recognition in the office domain are the AIRE project [12], the SmartOffice or Monica project [14], the Interactive Room (iRoom) [22], or the NIST Smart Space and Meeting Recognition projects, which develop tools for assistance in meeting rooms [13]. The most widely used tool to integrate semantics into activity recognition systems are ontologies [4]. However, there are current limitations of ontology-based activity recognition techniques that must be tackled: they require good knowledge engineering skills to model the domain, OWL DL does not allow interval (i.e. overlapping) temporal reasoning, ontological reasoning can be computationally expensive [23], and they cannot deal with uncertainty [4]. In this work, we provide advances to solve this last limitation and propose a fuzzy ontology to give support for imprecision and uncertainty, typical of everyday life situations. For instance, a sensor can give readings with a certain degree of reliability, or work only at specific times or in certain conditions; users may perform subtle changes in the way they perform their activities, the execution of an activity may be detected with a certainty or satisfiability degree, and all this information should be taken into account into the reasoning process. Unfortunately, classic crisp ontology proposals cannot handle this type of information. In our approach, different levels of granularity are designed so that incremental context acquisition allows behaviour abstraction and a more accurate, i.e., low-level recognition. By setting a behaviour specification structure, a set of rules can define how to recognize a human behaviour out of a sequence of observations. And, since fuzzy ontologies can handle uncertainty, our approach is able to solve this limitation with respect to crisp approaches. Fuzzy logic was already proposed as an argument to “reject the maximality rule, according to which only altogether true sentences are true, and embracing instead the rule of endorsement, which means that whatever is more or less true is true” [24]. As argued in [24], positing fuzzy predicates usually simplifies theories in most scientific fields; fuzzy predicates are much more plausible and give a more cohesive world view than their crisp counterpart. In this way, classical ontologies are not suitable to deal with imprecise and vague knowledge, which is inherent to real world domains [25]. On the other hand, fuzzy ontologies have the advantage of extending information queries, allowing the search to also cover related results. This makes the decisions about relatedness based on modelled domain knowledge, i.e., instead of just offering exact matches, the search can be extended to cover also related concepts, so that precise wording is not needed to get a useful hit (as the context of a document does not have to be exactly the same one for the user to benefit from it) [26]. This results on more effective retrieval. Likewise, another advantage of fuzzy ontologies is the fuzzy semantics, as they are more flexible towards mapping between different ontologies [26]. Let us put an example to show the benefits of fuzzy ontologies versus crisp ones. Because in a fuzzy ontology we can define that the CoffeeBreak activity is recognized accounting for different weights on the actions that compose it (e.g. 0.3 TakeMug, 0.3 TakeCoffeePan, 0.4 TakeMilk), thus, when one action has been skipped due to an exception (e.g. milk run out) or a missing sensor reading, the activity can still be recognized to a lower degree. In contrast, the same activity formalized in a crisp ontology could not be recognized if any of the exclusive elements that compose it is missing. The rest of the paper is organized as follows: the following section describes related work on ontologies for human activity recognition and introduces fuzzy ontologies as the main tool for the rest of the manuscript. After that, in Section 3, we present a novel ontology for human activity modelling and its extension to Fuzzy OWL 2 with support for the fuzzyDL reasoner. We detail concepts and relationships in the fuzzy ontology as well as Section 4.1 presents the use case on domain specific entities for the office environment. Section 4.2 describes an evaluation of the approach with respect to the crisp case, and finally, conclusions and future work are shown in Section 5.

Knowledge-based techniques, such as ontology-based activity modelling, add a set of advantages for incremental and context-aware recognition. It is a suitable approach to achieve interoperability, abstraction, and modularity in an easier way. However, some expressiveness limitations in OWL DL are also found related with the lack of support for temporal reasoning [35] and often, an external rule engine is used to express more complex modelling. A set of ontologies for human activity modelling has been created in the recent past [4] which is able to deal with different context information. The proposal in this work shows a set of advantages in respect to the existing ones. The main contribution is the support to model and treat uncertain, incomplete, vague, or imprecise information, therefore easing the modelling of more flexible models as well as allowing incomplete but real-life queries. For instance, we may want to get notifications the days when a certain office has “very high” temperatures, or we may want to know at what time window our work’s restaurant is least crowded. When exact numbers are not known or they are not relevant to get a fast answer, imprecise knowledge eases the task of information retrieval. In addition, we demonstrate that fuzzy ontologies may be more realistic and provide better accuracy for human behaviour recognition than crisp ontologies. In the experimental section, we show that if facts are not completely true, crisp rules cannot fire, while a fuzzy approach fires them with a satisfiability or certainty degree. Solving this situation in crisp solutions would require a continuous threshold management that would make the problem more complex, while fuzzy systems deal with this type of situations in a natural way. Apart from being more accurate, we also showed that the fuzzy approach is scalable for larger sizes of KBs. Furthermore, varied (from fine to coarse-grained) levels of abstraction are provided to identify atomic actions, activities composed by actions, and behaviours comprised by an aggregation of the latter two. Recursion, for more flexible and scalable modelling, is also allowed at activity and behaviour levels. A behaviour can be customized and associated to a unique user, user group, or certain type of action, activity, or context dependence. Our ontology is applied to human behaviour recognition in the office and public buildings domain, but is easily expandable to other domains. Since we provide different granularity levels of activities, incremental context acquisition – typical of ontology-based reasoning – is supported and made easier to abstract behaviours. Likewise, this results on a precise and accurate definition of activities. We provide not only knowledge engineering for dealing with uncertainty, but we also allow for modelling activities where its actors or locations where they take place are unknown. These conditions allow, in this way, more realistic settings and also, more abstract and different group activities. After experimenting with a real life and complex enough ontology, we can affirm that there is a need for more complete fuzzy reasoners that can handle real-time notifications (such as a subscription mechanism as in e.g. M3 RDF store [56]) to avoid bottlenecks with constant querying. This does not occlude fuzzyDL’s potential and the fact that it has shown to be successful in diverse domains. In our presented case study, modelling rules in fuzzyDL also showed some challenges. As fuzzyDL does not allow yet to express implication rules where the subject in a triple (s, p, o) is a concrete individual, our experiment focused on general rules for individuals of a given class C that acts as the subject of the query. In the future, we expect to have more efficient ways of concreting the rule so that it can specifically apply to unique instances/individuals so as to achieve a proper rule personalization. At the moment, three workarounds solve this situation. a) An extra class (e.g. NataliaClass concept) can be created for each rule we want it to affect solely to a concrete individual (e.g. individual Natalia from class User). This makes explicit, by naming a class with an individual’s name, that rule should only apply to the given individual. b) The firing of the rule for a given individual can be detected by first instantiating the individual of interest (in this case Natalia) as an instance of that rule (e.g., Rule 4): ((instance Natalia Rule4). This step is required to give a correct answer in the second step. Secondly, a rule firing can be detected by querying the degree of satisfiability of individual Natalia satisfying Rule 4. This is done by querying for Natalia being an instance of the rule’s consequent (this applies to a given moment, i.e. state of the KB). E.g.: (min-instance? Natalia (some performsActivity DoStretching)). c) It is also possible to find all individuals satisfying a given rule by querying: (all-instances? Rule4) after having instantiated all individuals that we want the rule to apply to (as in step b). E.g.: (instance Natalia Rule4). Although the novelty of our ontology is to overcome the problems of existing proposals which are unable to model uncertainty, the limitations are those common problems of ontology-based modelling on complex behaviours. This is because ontology-based modelling may not overcome performance as its most characteristic feature and definitions can seem unnatural at times [35]. Therefore, future works should consider these aspects more specifically and improve fuzzy ontology modelling real human interaction, as well as interfaces for end-users. One possible solution would be to create hybrid approaches involving data-driven techniques and fuzzy ontologies for human behaviour recognition as a next step in our research. Another future direction to explore is modelling and detecting human behaviour changes. Using learning instance matching, i.e., (data level, non-schema) ontology mapping for new data integration [57], could be an approach towards automating the evolution or learning of behaviour changes.