CURRENT EVENTS

B. Kenneth Estabrook, Jr.
Supervisor, Training and Development
Johns Hopkins University Applied Physics Laboratory
Robert W. Ware, PMP, Project Masters, Inc.
April J. O’Koren, PMP, Project Masters, Inc.

Abstract

As instructional approaches continue to mature and modern technologies effect change in the educational arena, those who design and deliver project management training are faced with the challenge of incorporating new approaches and technologies into an effective training strategy. This paper seeks to provide a case study on how one organization—the Johns Hopkins University Applied Physics Laboratory—integrated several complementary methodologies into its PM training strategy. The strategy attempts to achieve an optimal blend that capitalizes on the relative strengths of each approach and technology to overcome the relative weaknesses of others, with due consideration to maintenance of the training and learning resources, as well as future developments that are likely to occur.

Introduction

Founded in 1942, the Johns Hopkins University Applied Physics Laboratory (APL) is a not-for-profit applied research and development organization dedicated to solving a wide range of complex problems that present critical challenges to the United States. Using advanced technologies, world-class facilities, hands-on operational knowledge, and a systems engineering approach, APL’s 3800 employees achieve affordable, effective solutions for its customers on more than 400 projects of various sizes and duration (approximately $600 million total annually).

Given the variety and complexity of APL’s projects, sound project management (PM) processes are essential to achieving the goals of APL’s customers. One of the factors that play a critical role in ensuring the development and promulgation of sound PM processes throughout the organization is the effectiveness of the training and performance support provided to the Laboratory’s project managers.

While APL has a generally excellent reputation for developing its project managers, the ever-increasing pace of change—particularly over the past ten years or so—has increased the variety and complexity of APL’s projects, while also demanding that project managers build and maintain their knowledge and skills more quickly. Mentoring and informal on-the-job training is no longer sufficient, and generic, off-the-shelf courses don’t provide the necessary knowledge and skills in organization-specific processes.

To provide improved learning and performance resources to the laboratory’s project managers, APL’s Training and Development (T&D) Office and Project Masters (Ellicott City, MD), have collaborated during the past two years to develop an efficient and effective PM training strategy, taking advantage of lessons learned from previous training efforts at APL, insights gained through an organization-wide training needs assessment, and a solid foundation of Project Masters’ educational resources.

The resultant APL Project/Program Managers (P/PM) Training Program is designed to provide an optimal blend of off-the-shelf and customized content in a flexible, multi-modal methodology aimed at meeting the learning needs of a wide variety of program participants. This paper examines the design process from the initial assessment stages to the evaluation process, providing a road map for other organizations desiring to integrate optimally the latest instructional approaches and technologies into their PM training strategies.

Survey of Past and Current PM Training Approaches

In considering whether a better way of teaching PM exists, it is useful to first examine past and current PM training approaches and their relative strengths and weaknesses. This can help explain why these approaches—particularly when used alone—often prove less effective than expected in building the knowledge and skills that project managers need to be successful. Most project managers learn the knowledge and skills they need through one (or more) of the following means:

• Self-study resources (e.g., websites, textbooks, professional journals, policy and process documentation, etc.)
• Project assignments
• Mentoring by experienced managers and colleagues (either within one’s organization or through one’s professional contacts)
• Generic, off-the-shelf PM training courses—either instructor-led or online—designed to meet the learning needs of relatively broad audiences
• Customized PM training programs developed by organizations (often, with the assistance of outside vendors) to meet their specific organizational needs

None of these methods are inherently better or worse than the others; they all possess particular strengths and weaknesses that should be—but all too often are not—accounted for in the design process.

• Self-study resources can be vague and difficult to understand without mentoring or training.
• Knowledge gained through one’s project assignments may cost less initially than formal training, but when mistakes are made, experience can also prove an expensive and unforgiving teacher.
• Mentoring and peer interactions can provide “real-time” information that more structured methods often cannot. However, such interactions can prove inadequate, particularly when “experts” are asked to mentor in areas where they lack expertise. Also, project managers may learn about shortcuts and unorthodox ways of getting things done, which may be undesirable when such methods conflict with what the organization wants or needs.
• Off-the-shelf PM training provides some degree of structure that mentoring does not and is usually less expensive than customized training. However, the process knowledge and skills provided by generic training often does not reflect the organization’s specific processes accurately and completely.
• Training customized to meet the specific requirements of the organization may provide the ideal content, but can also be expensive to create and maintain.

Accordingly, multiple methods should be—and often are—employed, especially over the course of a project manager’s career. An organization, for example, may rely on project assignments and mentoring to provide managers with certain knowledge and skills early in their career. Then, as the managers’ learning needs evolve, generic courses may serve to provide a more structure framework for general PM knowledge and skills. Finally, customized training may be necessary meet the need of more experienced managers for organization-specific process knowledge and skills.

APL P/PM Training Program Design Considerations

Before addressing the various elements of the APL P/PM Training Program, it is useful to detail some of the theoretical and practical design considerations that influenced the program’s content and format.

Theoretical Design Considerations

On a theoretical level, the design of the APL P/PM Training Program was shaped by two primary influences: the constructivist theories of Lev S. Vygotsky and others, and Ronald N. Giere’s models-based approach to scientific theory.

Much has written about the important role of constructivist architectures in learning, with Ernst von Glasersfeld perhaps articulating it most clearly and succinctly when he postulated that “knowledge is not passively received either through the senses or by way of communication; rather knowledge is actively built up by the cognizing subject” (von Glasersfeld 1995, 51). Certainly, behaviorism did have some influence on the design of the APL P/PM Training Program (most particularly in methods of teaching and drilling basic PM terminology and concepts). However, because participants possessed different levels of PM knowledge, skills, and experience coming into the course, and because they played diverse roles within their projects, they came into the course with different needs, and hence, constructivist theories had a much greater influence on the program’s content and format. Lev S. Vygotsky took constructivism one step further by proposing that knowledge is not constructed alone, but is “social in nature and is constructed through a process of collaboration, interaction, and communication among learners in social settings” (Vygotsky 1978, 35). This suggested that participants would be likely to learn more collectively than they would on their own.

The design of the program was also influenced by Ronald N. Giere’s models-based approach to scientific theory (Giere, 2004), and, in particular, his use of the idea of a theoretical model’s “fitness” to reality as being similar to the “fitness” of a map to its intended purposes (Giere, 1999, 25-26, 81-82, 214-215). Giere’s emphasis on the use of “maps” was adopted in discussing models of projects (e.g., real-world examples and fictional case studies) and their “fitness” to the real-world projects to be carried out. The level of concreteness that these approaches provide make learning PM tools and methods—and transferring what is discussed and learned back to the workplace—much easier.

Practical Design Considerations

In addition to the theoretical design considerations discussed above, there were a number of practical considerations that needed to be taken into account during the design process. These practical considerations came to light through (a) lessons learned from previous PM training and (b) a formal assessment conducted to determine the specific needs of the APL audience.

The APL P/PM Training Program is not APL’s first attempt to develop project managers, and the lessons learned from these previous attempts had a significant influence on both the current program’s content and format. Incorporating these lessons learned helped to ensure that past mistakes were not repeated and encourage feedback from participants as to whether “fixes” successfully addressed the issues.

APL has been developing project managers since its founding in 1942, generally relying on project assignments, mentoring, university education, and ad hoc short courses conducted by outside and in-house experts. As a division of the Johns Hopkins University, APL has easy access to the university’s engineering programs, which has offered a Project Management concentration in its Technical Management Program for many years.

Starting in the early 1990s, in recognition that a more structured program was essential to meet the needs of APL project managers, APL contracted outside vendors to teach multi-day PM courses. Course materials underwent minor revisions to reflect APL terminology and processes, and instructors taught the course multiple times, which meant that they were able to gain an understanding of APL terminology and processes and provide consistent instruction to each set of participants. In the end, these two experiences demonstrated that generic PM training did not generally fulfill the need for APL-specific content and that it was highly desirable to have the same person or persons teach the course each time.

In the mid-1990s, APL also developed a highly customized PM training course, designed to meet the needs of more senior project managers. In spite of a great deal of effort being put into the needs assessment and instructional design processes, this course suffered from a number of fatal flaws and was terminated after the first half of the course was piloted. First, because of APL’s highly decentralized structure at the time, it was nearly impossible to provide enough detail on APL-specific PM processes to be useful, yet remain general enough to be taught to a group of managers from across the Laboratory. Also, the course was taught in two-hour sessions, one session each week over fourteen weeks. This proved too long an interval for most participants to remain focused on the learning process. Finally, by design, the discussion of PM processes tended to be descriptive, rather than prescriptive in nature, leading to frequent discussions about how processes ought to work, all too often without any standardized answer being offered.

In addition to these lessons learned, practical design consideration were also discerned through an organization-wide training needs assessment conducted by the T&D Office using a focus-group methodology. Two separate focus groups were held, one to address the needs of new and prospective project managers, and a second to address the needs of more senior project managers. While it was initially thought that these two groups had divergent learning needs, the focus groups seemed to indicate that this was not the case.

In sum, the needs assessment and lessons learned provided input on or validation of a number of practical design considerations.

• The program should address seamlessly the needs of project managers at all levels of responsibility and managing all types of projects (i.e., large or small, long or short term, products or services, hardware or software, etc.). The reasons for this included (a) the need to generate adequate “throughput” to support holding the program several times each year, and (b) because managers could be—and often were—promoted on a moment’s notice to different kinds of projects or to positions with greater responsibility. It was understood, however, that certain audiences (e.g., very senior managers, those managing certain types of projects, etc.) had particular learning needs that, at least for the time being, would remain outside the scope of the program being developed.
• The program should promote and encourage standardized processes and tools with an optimal blend of description and prescription, as well as best practices from other organizations. For this reason, designers started to focus on the Project Management Institute (PMI) and it's Guide to the Project Management Body of Knowledge (PMBOK).
• The program should provide detailed coverage of assessing and managing risk, as well as how to communicate risk considerations to our customers, due to the nature of much of APL’s work (i.e., applied research and development, critical outcomes, ambiguous scope descriptions, and short timelines).
• The program should incorporate APL-specific content, particularly on organizational-specific business systems and processes.
• The program should modularize the instruction to provide sufficient flexibility as to when participants attend a topic, thereby better fitting their busy schedules.
• The program should cultivate a wider PM community to support a continuous learning process.
• The program should have a consistent instruction team comprising outside experts, particularly for general topics, and in-house experts, particularly for APL-specific content.

Program Elements

In its current configuration, the APL P/PM Training Program comprises six elements, designed to provide a comprehensive solution to meet the needs of APL project managers. Before starting the program, participants attend a brief orientation that describes the content, format, and purpose of the program elements. They also are asked to reflect on and discuss their own learning needs, so that the course can be adjusted to accommodate those needs to the extent possible.

Online PMBOK-aligned PM courses (SkillSoft Corp.)

APL licenses online technical and business courses from SkillSoft Corporation (Nashua, NH), including their Third Edition PMBOK-aligned curriculum. These online PM courses support the APL P/PM Training Program in several ways. First, they are used to help prospective participants to assess their readiness for the program. Secondly, they are used as pre-work to ensure that all participants begin the program with a basic understanding of PM concepts, processes, and tools, since some basic material is covered only briefly in the instructor-led portions of the program to preserve time in the classroom for more advanced content. Finally, the online courses contain a substantial body of content that is outside the scope of the instructor-led courses and, therefore, play an important role in follow-on training.

Instructor-led PMBOK-aligned PM course (Project Masters)

The program’s foundation is the four-day APL Project Management Fundamentals course, which was developed from Project Masters’ PMBOK-aligned instructional materials and customized over time to meet the needs of the APL audience (e.g., APL-specific processes and terminology). While all nine PMBOK knowledge areas are covered to some degree in the course, the primary focus is on managing project scope, time, cost, and risk. This design choice incorporates the traditional primary constraints of scope, time, and cost management, but also an organizational need—uncovered during the needs assessment process—to provide substantial coverage of risk management.

The Fundamentals course is taught over the course of two weeks by one of Project Masters’ highly experienced instructors. Teaching it over two weeks provides a compromise solution that balances the needs of those who believe that the course would be most effectively taught in four consecutive days—thereby reducing the opportunity to forget material between sessions—and those who would prefer for it to be taught one day each week over four consecutive weeks—thereby reducing the impact of the course on their work schedules. It also allows the participants some time between sessions to reflect on how what they’re learning in class applies to their work, as well as to confer with their mentors.

During the course, an imaginary case study provides a framework for applying new knowledge and skills, as well as for demonstrating how MS Project can be used to manage projects effectively. Those who want to delve more deeply into MS Project and its features can take a supplemental three-day MS Project course, participation in which is restricted to those who have completed the Fundamentals course.

APL-Specific Content Sessions

As discussed above, APL’s previous experience with PM training demonstrated the critical role APL-specific content plays in bringing about a successful learning outcome. Therefore, after attending the APL PM Fundamentals course, participants are expected to attend several APL-specific content sessions taught by APL subject matter experts in half-day sessions. Some of these sessions elaborate on PMBOK knowledge areas:

• Project Human Resource Management
• Project Procurement Management
• Project Risk/Quality Management

Other sessions cover PM-related processes and tools:

• APL Program Financial Management, which covers the financial management of project at APL
• Security (including classified material handling)
• Environmental Health and Safety (taught in conjunction with Project Risk/Quality Management)
• Business Development and Contracts with the Federal Government
• APL Resource and Management Information System (RMIS) for Project Managers
• Export Control

This modularized approach allows participants to attend the APL-specific content sessions with the other members of their Fundamentals course, as is preferred, or during a future offering, if necessary due to scheduling conflicts. Participants can even attend the sessions more than once, if they so desire.

Online APL PM Community (Ensemble Collaboration, Inc.)

A theme that emerged during the needs assessment process was that project managers generally recognized that their success was due to a large degree to how extensive their professional network was and how effectively they tapped into it. In addition, they understood that their learning process was a continuous one, and not one that could deal with in a single course.

To support these requirements, APL licensed Collaboration on Demand from Ensemble Collaboration (Fredericton, NB) to help project managers connect online to ask questions and share resources within the context of the training program or their work. While there are a growing number of applications available to support collaborative learning, one of the benefits of Ensemble’s approach is that it allows proprietary or other sensitive information to reside inside APL’s firewall, while the hosted collaboration space resides outside the firewall. This gives APL added flexibility by allowing project managers to access the full set of resources—both open and proprietary—when operating inside APL’s firewall, and yet still access many of the resources—albeit not those that are sensitive—when operating outside the firewall (e.g., while on travel).

While APL has a large community of project managers used to using technology to communicate, it is also a decentralized community. Therefore, starting an online community has taken longer than expected, and the community is still in its infancy. Etienne Wenger, who has done a lot of research on establishing and growing “communities of practice,” proposes seven principles for cultivating strong communities that continue to guide efforts to nurture the PM community (Wenger, 2002, 49-64):

• Design for evolution
• Open a dialogue between inside and outside perspectives
• Invite different levels of participation
• Develop both public and private community spaces
• Focus on value
• Combine familiarity and excitement
• Create a rhythm for the community

Mentoring

Due to the number of different customers and projects that APL has, a critical part of any PM training approach is mentoring, which helps participants apply what they are learning to their particular projects and customers. For example, in learning to manage project communications, project managers consider the specific steps to be taken to establish sound project communication plans with their particular customers.

Because of the important role that mentoring plays, participants are asked to assess what areas they need mentoring in and then determine who can provide mentoring. In some cases, they are able to do this on their own; in others, they request the assistance of a supervisor or peer. The mentoring process is expected to start during the Fundamentals course, but may continue through the APL-specific content sessions and beyond.

Textbooks

Each participant receives a copy of the Guide to the PMBOK, Third Edition, which serves three purposes:

• A “read-ahead” resource to assess readiness and prepare for the four-day course
• An instructional resource during the class
• A reference resource after the class

In addition, some participants have learning needs that fall outside the program’s design learning objectives. When these needs can be effectively addressed through textbook learning (and perhaps some mentoring), a participant may request that the T&D Office procure an appropriate textbook. In some instances, the participant retains the textbook, while in others they are kept in the Laboratory’s Staff Development Resource Library (thus made available to other employees). In all instances, though, the participant is asked to provide a review of the textbook, which is posted on the online PM Community for future participants to use in determining the textbook’s appropriateness as a resource for them.

Program Evaluation

The T&D Office conducts numerous evaluations of the APL P/PM Training Program and its elements. The evaluation strategy draws on Donald Kirkpatrick’s classical model for evaluating training, which suggests four progressive levels of evaluation: reaction, learning, behavior, and results (Kirkpatrick, 1987, 302). The use of multiple evaluations is designed to obtain immediate feedback about the program (i.e., reaction), while also seeking information about the program’s long-term effectiveness, both in meeting the program’s learning objectives (i.e., learning), as well as improving the performance of APL’s project managers over the long-term (i.e., behavior). Participant feedback has also resulted in changes to the program’s content and format. For example, it was participant feedback that led to a re-design of the course to incorporate small group exercises using MS Project.

During the Fundamentals course, each participant receives an evaluation form asking for feedback on the content and format of the course (including the instructor and training materials), as well as on what learning needs have—and have not—been met. In addition to the information gathered through the evaluation forms, short verbal quizzes and practical exercises (including the case study) are conducted during the course to confirm that learning objectives are being met. Participants are asked to complete similar evaluation forms for each APL-specific content session they attend.

One year after they begin the program, participants are asked to complete an overall evaluation, including feedback on each element of the program. By that time, they are supposed to have completed all of the program elements, so this feedback is designed to provide a longer-term perspective on the program’s effectiveness.

Evaluations for the first eight sessions have been encouraging. Most participants say that they have learned a lot—both on conceptual and practical levels—from the program. Several program elements and features have received special note:

• The prescriptive nature of a PMBOK-aligned Fundamentals course has proven helpful in outlining how projects ought to be organized and managed. In addition, the incorporation of MS Project in the practical exercises has been viewed positively.
• The concern about participants having different levels of knowledge, skills, and experience coming into the program has turned out to be less of a problem than anticipated. Even senior managers with many years of experience appear to have benefited from the more formal and structured program of learning.
• Participants have not attended the APL-specific content sessions as widely as expected. The flexible schedule may be proving to be too flexible.
• The Online APL PM Community remains a little used resource. This is not unexpected, however, and some comments have provided insights that may prove useful for increasing participation.

Summary

Has APL’s P/PM Training Program been a success? Perhaps the most accurate gauge of its effectiveness is how quickly each offering fills up, how many new participants relate that a previous participant recommended the program to them, and how quickly the PMBOK is starting to provide a common language and structure for discussing sound PM practices.

What does the future have in store? Certainly, the program is not a finished product—and probably will never be. Certain program elements—such as the online PM Community and mentoring—will require more work to help them attain their appropriate level of effectiveness. But, as the program continues to mature course content—particularly in the APL-specific topics—continue to grow and evolve. And, project managers are beginning to create and share resources, as well as develop training in topics in which they have expertise.

Future developments in technology and organizational development promise to keep change a primary element of the program. Mobile business technologies—such as cell phones, wireless networking, and tablet notebook computers—will certain affect how project managers choose to obtain their learning and performance support. And, trends in knowledge management and enterprise applications will influence how organizations maintain their enterprise knowledge base.

References

Giere, R. N. (2004, December) How Models are Used to Represent Reality. Philosophy of Science 71(5), 742-752.

Giere, R. N. (1999) Science without laws. Chicago: University of Chicago Press.

Kirkpatrick, D. L. (1987) Evaluation. Training and development handbook: A guide to human resource development, R. L. Craig, ed., 301-319. New York: McGraw Hill Book Company.

von Glasersfeld, E. (1995) Radical constructivism: A way of knowing and learning. Washington, DC: The Falmer Press.

Vygotsky, L. S. (1978) Mind in society: The development of higher psychological processes. Cambridge (Mass.): Harvard University Press.

Wenger, E., McDermott, R. & Snyder, W. M. (2002) Cultivating Communities of Practice: A guide to managing knowledge. Boston: Harvard University Press.

Back to top