اندازه گیری ادراک اعضای جامعه از اثرات مشارکت پژوهشی در خدمات اجتماعی و بهداشتی

Currently, there are no psychometrically sound outcome measures by which to assess the impacts of research partnerships. This article describes the development of a 33-item, survey questionnaire measuring community members’ perceptions of the impact of research partnerships addressing health or social issues. The Community Impacts of Research Oriented Partnerships (CIROP) was developed using information from the literatures on health promotion, community development, research utilization, and community-based participatory research, and from focus groups involving 29 key informants. Data from 174 community members were used to determine the factor structure, internal consistency, and test–retest reliability of the four CIROP scales, and to provide evidence of construct validity. The CIROP informs research partnerships about the extent of their impact in the areas of Personal Knowledge Development, Personal Research Skill Development, Organizational/Group Access To and Use of Information, and Community and Organizational Development, allowing them to demonstrate accountability to funding bodies. As well, the CIROP can be used as a research tool to assess the effectiveness of knowledge sharing approaches, determine the most influential activities of research partnerships, and determine structural characteristics of partnerships associated with various types of impact. The CIROP provides a better understanding of community members’ perspectives and expectations of research partnerships, with important implications for knowledge transfer and uptake.

Community–university research partnerships provide an infrastructure from which knowledge is generated, findings are shared, and research skills are developed. These partnerships consist of groups of people who have come together to address a particular topic in a concerted way. In the field of science/technology, these research entities are referred to as knowledge value alliances. These alliances involve knowledge producers and users pursuing a unifying knowledge goal but with diverse ends in mind, including curiosity, skill development, and application (Rogers & Bozeman, 2001). Community–university research partnerships in the health and social service fields vary with respect to the number of universities and community organizations involved, the formality of their organizational structure, and their ways of operating (King, Servais, et al., 2008). Despite this variation, community–university research partnerships have three common functions: (a) knowledge generation, (b) knowledge sharing to improve the functioning of community organizations and the well-being of communities, and (c) research education/training to improve the research skills both of university students and community service providers (Currie et al., 2005). Researchers in diverse fields, including health and science/technology, refer to knowledge generation, knowledge sharing, and research education/training as the core missions, functions, or processes of research centers or partnerships (Bozeman & Boardman, 2004; Israel, Schulz, Parker, & Becker, 1998; King, Currie, Smith, Servais, & McDougall, 2008; Youtie, Libaers, & Bozeman, 2006).

Overall, the CIROP displayed a clear factor structure, accounting for an appreciable 76.2% of the variance in responses. The CIROP scales displayed excellent internal consistency, but only acceptable test–retest reliability (due to insufficient sample size). Content validity of the CIROP was assured through the development process, which used information from the literature and from key informants with diverse perspectives, and through pilot testing with members of partnerships. Construct validity was assessed through hypotheses involving respondents’ roles with the partnership and their organizations of employment, and a series of variables assessing aspects of the research transfer, utilization, and uptake process (i.e., involvement, fit, relevance, and extent of use). These analyses indicated that the CIROP discriminated between groups of respondents in expected ways. For example, respondents who were more actively involved with a partnership reported higher impact on their personal research skill development than did respondents who were simply recipients of information. For the most part, the CIROP scales also discriminated meaningfully between people's reports of research transfer and uptake on the personal versus organizational levels. 8.1. Limitations of the CIROP As with any measurement tool, providing evidence of validity is an ongoing process. Further test–retest reliability information is required, and future research should examine the responsiveness of the CIROP to change over time. The CIROP provides a measure of impacts in a reasonable and not too limited time frame (a 1-year period), but this requires that administrations be separated by 1–2 years for there to be an opportunity to detect changes in levels of impact. The CIROP was designed to be a measure of the mid-term impact of research, rather than a measure of the quality of research partnerships or end-user engagement. The CIROP can, however, be used in research studies to examine the influence of respondents’ expectations, roles, or degree of engagement on perceptions of research impact. Different expectations about what it means to be a partner and about hoped-for gains may affect respondents’ scores on the CIROP scales. The CIROP provides quantifiable information about the perceived impacts of research partnerships in the eyes of community members. Knowledge is, however, a complex process of “meaning generation” that is inherently contextual in nature (Lave and Wenger, 1991, Schwandt, 2005 and Stacey, 2001). We therefore recommend that the CIROP be used in comprehensive examinations of the impact and utility of research partnerships, involving the assessment of structure, processes, and outcomes (Israel et al., 1998 and Tash and Sacks, 2004), and the use of tools tailored to target audiences and their environments. 8.2. Implications of the CIROP scales for understanding research impact The CIROP scales, determined using factor analysis, indicate that community members focus on the benefits of research partnerships with respect to personal development; tangible resources, materials, and opportunities; and useful tools and ideas that contribute to organizational and community outcomes and capacities. Using the impact model (Currie et al., 2005) as our starting point led us to generate items in modules referring to “enhancements” of knowledge and research skills (the researcher's perspective). The groupings of the items in the factors led us, however, to label scales in terms of “development” (the utilizer's perspective). A visual portrayal of this two-way perspective—researchers “looking out” at the community and community members “looking in” at partnerships—is available at www.impactmeasure.org/newmodelb.htm. This figure portrays the relationship of the CIROP scales, which capture the major types of benefits in the eyes of community members, to the three main functions of research partnerships. The figure shows what researchers do (the three key functions) and what recipients of their outputs see, appreciate, or value. Recipients/community members see research outputs as tools they can potentially utilize. They focus on the things they gain from the research education/training and knowledge sharing functions of partnerships. Understanding community members’ worldviews, priorities, and expectations has important implications for knowledge transfer and uptake. Because community members see themselves as active utilizers of supplied information, they may not fully recognize or value the intensive knowledge generation and synthesis work done by members of research partnerships. The knowledge created and shared by researchers is simply one part of the broader package of knowledge, information, beliefs, and values that community members use to create what is important to them—policy documents, new programs, revisions to existing services, and changes to ways of operating. 8.3. Use and utility of the CIROP Several features of the CIROP contribute to its usefulness as a measure by which to study the research impact process, and as an assessment and planning tool for research partnerships. First, the CIROP captures several types of impact, which gives it broad applicability. Second, because the CIROP is based on the three basic functions of research partnerships, it provides information that is relevant to any research collaboration. Third, the CIROP captures impact attributed to a particular partnership. By nature, impact is diffuse and evolving, and it is hard to trace the use of a specific product, innovation, or idea to a partnership's specific influence. The CIROP deals with this issue by directing attention to the notion of a partnership rather than a particular product. 8.3.1. Use in research studies on the nature of impact The CIROP provides a needed outcome measure for research examining the nature of research impact and how to improve it. The CIROP can be used to determine the structural or operational characteristics of partnerships most highly associated with impacts of different types (Hays et al., 2000 and Paine-Andrews et al., 1997) and to examine the relative effectiveness of different strategies involved in knowledge mobilization efforts, such as the use of written materials, website dissemination, or face-to-face methods of sharing information. As well, the CIROP can be used to examine relationships between characteristics of community members and their perceptions of impact. It can also be used to ascertain the relative effectiveness of different types of approaches to doing research involving community members, including a participatory action approach. These various potential uses of the CIROP reflect the current emphasis on knowledge utilization, which considers the dynamic and interdependent influence of various structural, process, and user-related factors in the utilization of research information (Landry et al., 2001a). 8.3.2. Use by research partnerships Partnerships can use the CIROP to examine the real-world usefulness of their activities and the information they share. This would involve defining the community of interest and then selecting a group of respondents to complete the CIROP. For example, there may be interest in knowing whether service providers feel their personal knowledge has grown as a result of receiving materials from a partnership. To date, information about impact has largely been based on conjecture, anecdote, simple counts of outputs, or the viewpoints of the internal partnership team. In contrast, the CIROP allows partnerships to capture the different types of impacts they have helped to create, from the perspective of their targeted audiences or communities. This is considered to be more valid than assessments that rely on the opinions of those involved with the partnership itself (Florin, Mitchell, Stevenson, & Klein, 2000), as there is sparse evidence that these perceptions are related to community-based outcomes (Hayward, DeMarco, & Lynch, 2000; Kegler, Steckler, McLeroy, & Malek, 1998). The CIROP can be used to evaluate the success of research partnerships, determine where to make refinements to strategic directions or operations, and demonstrate accountability to community stakeholders, advisory boards, and funding bodies. By including plans to use the CIROP in grant proposals, researchers can assure funding bodies of their commitment to being accountable, and will be able to provide evidence of the value of their work to the community. In conclusion, the CIROP promises to have utility in assisting researchers to understand more fully the complex phenomenon of impact. The CIROP also has practical utility for community–university research partnerships addressing real-world issues concerning health and social services. The measure provides quantified information that can be used to celebrate successes, provide evidence of accountability, and improve the effectiveness of operations and knowledge sharing efforts.