طبقه بندی و مقایسه نوآوری جامعه در سیستم های مدیریت ایده

The Idea Management Systems are a tool for collecting ideas for innovation from large communities. One of the problems of those systems is the difficulty to accurately depict the distinctive features of ideas in a rapid manner and use them for judgement of proposed innovations. Our research aims to solve this problem by introducing annotation of ideas with a domain independent taxonomy that describes various characteristics of ideas. The findings of our study show that such annotations can be successfully transformed into new metrics that allow the comparison of ideas with similar successfulness as the metrics already used in Idea Management Systems but in greater detail. The presented results are based on experiments with over 50,000 ideas gathered from case studies of four different organisations: Dell, Starbucks, Cisco and Canonical.

In the era of globalization the markets become more competitive and the organisations seek new ways of innovating. Among those attempts, are Idea Management Systems that employ Information Technology and crowd-sourcing principals to support innovation processes in the organisations. In particular, the notion behind those systems originates from simple suggestion boxes but is transformed into a more sophisticated process [48]. During the last decade of their evolution IdeaManagement Systems have extended their coverage from collecting ideas from large communities via computer networks to collaborative improvement of those ideas, the assessment of ideas and idea management in synergy with other enterprise processes [53]. Currently, Idea Management Systems are considered a very promising branch of computer software market [23] and various analyses of the vendor landscape [42] and [13] show rapid adoption growth in many enterprises in recent years. Nevertheless, the current state of the art Idea Management Systems still face key problems related to the large amount of human effort needed during the idea management process. Based on the testimonials of Idea Management Systems vendors [8] and case studies of various companies [30] and [9], the main origins of those problems are: large volume of submitted ideas, sudden peaks of submissions, redundancy of ideas, and large quantities of trivial ideas. In our research we relate the above issues to the idea assessment phase and focus on challenges that arise when trying to quantify the value of information contained in ideas and its impact on innovation in the enterprise. According to the study of contemporary solutions by Hrastinski et al. [27], the problems of idea assessment are approached by: 1) the use of a handful of automatically generated yet very simple community statistics; and 2) expert reviews that require a considerable amount of knowledge and impose serious time constraints thus increase the costs of the entire idea management process. In this article we present a solution for idea assessment that combines the advantages of those two cases mentioned by Hrastinski: rapid generation of metrics that require little expert knowledge yet offer more diversity and versatility than the current community metrics. In particular, we deliver a methodology for obtaining the metrics via analysis of idea annotations made with a domain independent taxonomy that expresses idea characteristics. The focus of the following article is to show that the proposed set of metrics can be applied to Idea Management Systems in a meaningful way that would allow to capture the distinctive features of ideas and to compare entire idea datasets. The article is structured as follows: firstly we summarise the past research achievements in terms of metric generation for Idea Management Systems as well as other kinds of computer-supported cooperative work systems (see Section 2). Additionally, in the same section, we discuss research on capturing the meaning of innovation in general and show how it influenced our work. Afterwards, we introduce our contribution in a form of a taxonomy for describing idea characteristics and present in more detail the theoretical grounding by referring to particular innovation models (see Section 3). Finally, we show how to utilise the proposed taxonomy in practice of Idea Management Systems by transforming the idea annotations into metrics that characterise the entire systems (see Section 4). At the end, we present the results of our experiments that test the usage of the taxonomy for annotation (see Section 5.1) as well as verify the performance of metrics generated from those annotations in relation to the contemporary parameters of Idea Management Systems (see Section 5.2).

We have proposed a set of new automatically generated metrics to aid the decision making process during the assessment of ideas in Idea Management Systems. Our hypothesis was that these metrics could be derived from annotations made with a specially crafted taxonomy, and used to characterise community generated innovation in a sufficient way to compare the gathered data. This hypothesis has been confirmed with a number of experiments that used the taxonomy as a tool to discover differences and similarities of various case studies. Furthermore, we presented an evaluation of all the steps underlying the generation of metrics and obtained valuable insight into conditions under which our methodology performs best. We determined that out of the four proposed taxonomy sub-trees substantial parts of two (Trigger and Object) can be applied automatically with satisfactory results, while the characteristics represented in the two remaining sub-trees (Innovation and Proposal Type) should be analysed and applied by a human. Furthermore, we have shown that the manual annotation delivers significantly better results when done by a single annotator rather than a group (regardless of the level of expertise with innovation theory). Finally, we evaluated the use of metrics not only for comparison of entire datasets but also for decision making process of selecting the individual ideas for implementation. We determined that the borderline cases of community activity that are currently used for filtering ideas (vote count, comment count etc.) do not influence the values of metrics proposed by us (e.g. more original ideas are not more commented or voted on). In addition, the obtained results have shown that our metrics deliver slightly better results to predict winning ideas in comparison with the contemporary used community metrics. Most notably, our results show best performance for Idea Originality and Object Dependability as best measures of idea adoption, standing out in comparison to any other metric. In terms of future work we envision to peruse a fully automated approach by putting more impact on analysis of different automatic annotation methods and attempting to simplify the taxonomy without much sacrifice on the level of knowledge that it carries. Furthermore, the introduction of the taxonomy opens a range of new possibilities for clustering and ranking ideas that could be a significant step toward brining better organisation to Idea Management.