اندازه گیری تاثیر برنامه بزرگ آموزشی مدیریت پروژه : برنامه مدیریت پروژه در فمنیکا
|کد مقاله||سال انتشار||مقاله انگلیسی||ترجمه فارسی||تعداد کلمات|
|3329||2013||14 صفحه PDF||سفارش دهید||محاسبه نشده|
Publisher : Elsevier - Science Direct (الزویر - ساینس دایرکت)
Journal : International Journal of Project Management, Volume 31, Issue 2, February 2013, Pages 285–298
This work aims at providing evidence of the impact of project management training programs. The research focuses on the impact of training effectiveness (higher learning) on the project managers' competencies (applied behaviors). Data have been collected within PMP-Project Management Program: a corporate training program developed and delivered through an academic–industrial collaboration between Politecnico di Milano University and the Education and Human Resources Development Department of Finmeccanica SpA. The program started in 2007 and has involved more than 2300 participants worldwide so far. The empirical analyses are based on an extensive survey conducted one year (rolling) after the training activities. The analysis of the relationships among the different variables provides evidence of the positive impact of training effectiveness on project management competencies. Moreover, the results show that two other variables, role-training matching and environment factors, have a strong conjoint effect, significantly increasing the impact of training on the project management competencies.
Organizations are spending more on professional development through corporate development programs targeted to specific communities or professional societies. According to the 2010 ASTD State of the Industry report, U.S. organizations spent $125.88 billion on employee learning and development in 2009.1 Additionally, in the project management environment, training and knowledge-transfer programs are gaining more attention, leading researchers to focus on how to design, implement and measure them more effectively (Alam et al., 2008, Ashleigh et al., 2012 and Lee-kelley and Blackman, 2012). How to measure the impact of major training and development initiatives to justify training investments is still an open debate (Hashim, 2001, Lien et al., 2007 and Preskill, 1997). The importance of evaluating the training activity objectively and quantitatively is continually stressed in the training literature (Bober and Bartlett, 2004, Noe, 2000 and Swanson and Holton, 1999). However, training evaluation is very difficult, and to date, a single best practice has not been identified (McLean, 2005). Different models have been proposed (Garvin, 1995 and Swanson and Holton, 1999), but Kirkpatrick's hierarchical model (Kirkpatrick, 1976 and Kirkpatrick, 1994) is the most studied, used and criticized. The model encompasses 4 levels, each investigating different issues in a training and development program (see Fig. 1): - Level 1: Reaction. The degree to which participants react favorably to the training. Every program should be evaluated at this level, at least, to answer questions regarding the learners' perceptions and to improve training. This level yields knowledge about whether the participants liked the training and whether they thought that it was relevant to their work. - Level 2: Learning. The degree to which participants acquire the intended knowledge, skills, attitudes, confidence, and commitment based on their participation in a training event. Level 2 evaluations are conducted before training (pre-test) and after training (post-test) to assess the amount of learning that has occurred. - Level 3: Behavior. The degree to which participants apply what they learned during training when they are back at work. Evaluations at this level attempt to assess whether the training has been transferred back to the job. This evaluation should be performed at least 3 to 6 months after training. - Level 4: Results. The degree to which targeted outcomes occur as a result of the training event and subsequent reinforcement. This evaluation measures the success of the training program in terms that executives and managers can understand, such as increased production, increased sales, decreased costs, improved quality, etc.Many criticisms of the causal hierarchical mode have been made (Bates, 2004); the main ones are listed below: Causal linkage assumption The first criticism addresses the assumption of causal relations between the levels of evaluation. Various empirical analyses (Alliger and Janak, 1989 and Alliger et al., 1997) have highlighted a lack of correlation among the measures identified at different levels of the model. Moreover, Bates (2004) notes that the causal linkage assumption may even lead to an over-reliance on reaction measures (level I), diverting trainers' attention from efforts to make training truly effective toward a focus on developing entertaining, amusing, and easygoing training that participants enjoy (Michalski, 2000). Environment factors The second main criticism raised against the model concerns the very narrow set of variables considered. Even assuming an increase in learning, it is clear that environmental factors such as the organization, the characteristics of the individual, the commitment of the management, etc., may either support or inhibit behavioral changes (application of learning to the job) (Holton et al., 2000 and Mathieu et al., 1992). Several studies have established that environmental factors affect the linkage between level II and level III (Bates et al., 2000, Cannon-Bowers et al., 1995, Ford and Kraiger, 1995, Holton et al., 2000, Kontoghiorghes, 2001 and Salas and Cannon-Bowers, 2001), showing that environmental factors can affect the transformation of learning into new behaviors. Focus on higher levels Assuming that level IV is the most important and that affecting results is the aim of every training activity can be wrong for many reasons (Bates, 2004). First, most training efforts have little ability to directly affect a company's results. Many training activities are of short or modest duration (2–3 days) and are meant to have only a limited effect on the participants. Second, the lack of impact on level IV could be due to the ineffectiveness of the training, as well as due to having poorly designed the training activity for the results that are sought. In other words, a perfect training program delivered to the wrong participants will lead to poor results (e.g., if participants are trained on practices not included in their roles), and focusing only on the economic measure would not allow the discovery of ways to improve the effectiveness of the training. Economic Estimation Finally, many researchers note the need to estimate the economic impact of training activities (Geber, 1995 and Wang, 2003). However, the usage of models based on indicators such as return on investment is limited, mainly because it is difficult to give a quantitative estimation of the costs and benefits of training activities (Alliger and Janak, 1989, McLean, 2005 and McLinden, 2008). Starting from the work of Phillips (2003), Alam et al. (2008) propose a model based on five different levels, adding return on investment as level five. However, many authors argue that it is not possible to make measurements that enable ROI to be evaluated with respect to intangible benefits (Rowe, 1994).
نتیجه گیری انگلیسی
Previous studies provide strong evidence of the important effect that training can have in improving project manager behaviors. Unlike other previous studies that suggested a lack of correlation among learning and changes in behavior (Alliger and Janak, 1989 and Alliger et al., 1997), our results provide evidence of a clear relationship between them. Moreover, this research highlights the conjoint effect that both Role-Training Matching and Environmental Factors have on the impact of training on behaviors. Our results provide evidence that the relationship between Training Effectiveness and the Behavior Frequency Increase is influenced both by the compatibility between the training activity and the role of trainees and by the context in which trainees operate. Our results allow us to draw some relevant conclusions that companies should consider when designing a training program on project management. 6.1. Increase the role-training matching Close attention should be paid to designing activities that are consistent with what trainees' roles are. It is important to emphasize that a competency model allows us to measure the extent to which the real roles match it and to what extent the training activities do the same. Thus, the definition of the right competency model is crucial to managing the role-training matching that our results show has a major role in increasing the effect of learning on behavioral changes. Dainty et al., 2003 and Dainty et al., 2004, clearly showed that a generic, competency-based approach cannot fit organizations managing complex projects. In the same vein, Cicmil et al. (2006) focus on the distance between the “traditional project management” and the “actuality” of project-based working and management. In this aspect, the PMP can be considered a best practice. The target population of the training program was composed of project managers operating in more than 18 companies within the Finmeccanica group. The homogeneity in their roles in terms of responsibilities, methodologies, tools, etc., was low. With such a starting point, it is suggested that companies not implement a common and standard model of competencies, but instead develop a new one specific to the group. Eight main technical competencies were investigated: bidding, scope management, resource management, time management, cost management, performance control, risk management and value management. For some of these competencies, it was possible to build on existing internal directives (e.g., value management based on EVA or operating control based on earned value), while for others, best practices were sought among the companies involved. The definition of the competencies and the identification of the practices were reached with the participation of practitioners and academics in the process, as suggested by Walker et al. (2008). Every company contributed to these activities, increasing the final matching between the training and the actual roles of PMs in the companies. Of course, it was not possible to create a model that fit all the different implementations of project management in the companies, but this approach allowed the most matching between training and PM roles. After the definition of the competency model, the collaboration between practitioners and academics continued with the design of the training activities. As suggested by Walker et al. (2008) advanced academic degrees, in particular those completed by PM practitioners, can be beneficial because the companies can use project managers themselves as sources of knowledge. Similar observations on the effectiveness of knowledge transfer were reported by Bakker et al. (2011), who studied knowledge transfer between temporary inter-organizational projects and permanent parent organizations. In the PMP case, large sections of training were based on role-play games derived from PMs' real experiences. Group project managers were involved as trainers during these sections, and the alignment between the competency model and the delivered training benefits from this involvement. 6.2. Increase the contextual effects Previous studies stated that the environmental conditions in which trainees operate have a strong and relevant impact on the effectiveness of training programs (transfer of training). Holton et al., 2000 and Holton et al., 2003, and more recently Velada et al. (2009), analyzed the phenomenon by using the LTSI (Learning Transfer System Inventory) that starts from 68 items and identifies 16 main constructs ranging from trainee characteristics (e.g., learner readiness) to work environment characteristics (e.g., supervisor support). Similarly, Kontoghiorghes (2004) noted the importance of transfer climate factors (e.g., peer support) and work environment factors (e.g., sociotechnical system design). For this aspect, the PMP generated a context-specific list of factors based on the feedback on the transferability of training provided by each participant at the end of each course. It is important to note that each item of the PMP list was already included in previous works. The contextual variables tested in these studies were as follows: i) management commitment, ii) IT infrastructures, iii) degree of collaboration of the functions involved, iv) corporate requests and v) customer requests. Previous research shows contradictory results about the real moderating effect of environment factors on the transformation of training into behavior. Richman-Hirsch (2001) found a positive effect of supportive work environments, while Chiaburu and Marinova (2005) found a positive effect for peer support, but not an effect for supervisory one. Similar findings were reported by Van der Klink et al. (2001). Our data showed that all the analyzed factors had significant effects in moderating the transformation of training into behavior. One of the main reasons for the success of the PMP is that some of these factors have been explicitly managed. Unfortunately, factors such as customer requests or IT infrastructure are completely exogenous to the program or very difficult and slow to change, and no specific effort has been made to modify them. However, other factors were accurately managed in the program. A multilevel training architecture was designed to increase the management commitment and the degree of collaboration of the functions involved in projects. The training architecture encompassed three variants of the same basic course: fundamentals, systemic and strategic. The systemic edition was the longest and most specific and was delivered to project managers. The fundamental edition targeted the team members and was aimed at giving them the same knowledge base given to project managers. Finally, the strategic edition targeted the senior project managers and the program managers, or the organizational supervisors of project managers. One of the main aims of these courses was to increase the managerial alignment; including the senior managers was aimed at increasing the management commitment, while including the team members was aimed at increasing the degree of collaboration of the functions involved in projects. Finally, the corporation increased the contextual pressure to change behaviors, acting on the variable corporate requests by increasing the degree of control on the application of the existing group directives regarding earned value, risk management, value management, bid management, etc. We would like also to highlight the limitations of this work. First of all, we should consider that data refers to a single training activity. Even if the specific training program has involved several project managers in different operating companies, evidence from different contexts would allow to replicate results and to confirm the extent of these results. Second, only eight competences – i.e. those relevant for the analyzed training courses – have been investigated. Results confirm the mentioned results for the different competences, however including also other competences would confirm the overall implications of this work.