Analyzing Needs – Quick but not Dirty

Analysis of learning or training needs is a covert mental operation that sets the cognitive wheels flying in all directions looking for scattered dots to join, unjoin and rejoin till a new, coherent picture builds up inside. The trouble with such covert processes is that they are difficult to articulate (exactly what it is you are doing?), difficult to explain (why did do this?) and most difficult to teach (how did you do this?).

The fuzziness of this mental circus presents many difficulties.  For one, it becomes difficult to estimate how much time and what resources will be required to analyze the need. Another pointed difficulty of a largely abstract activity like analysis is the difficulty in collaborating with the business, the line, sponsor or with the learner for that matter.

How do you analyze a learning need and then talk about it or write it out it so everyone (non-ID and sometimes non-learning professionals) will understand? In this post we bring you the quick, but not dirty way of conducting a learning/training needs analysis based on Thomas Gilbert’s Behavioural Engineering Model or BEM. Using this model as a framework for analyzing learning or training needs makes:

  1. An abstract activity like analysis more tangible and concrete
  2. Possible the collaboration of other stakeholders in the system
  3. It easy to teach learning analysis to novice Instructional Designers

Gilbert introduced the BEM in his path-breaking book ‘Human Competence: Engineering Worthy Performance’ published in 1978. Gilbert posits that performance is a product of the organizational or external environment and an employee’s internal resources.

To do their jobs well, employees need support from their external environment. They need toknow what their job requires, they must have the necessary resources and they must be adequately incentivized. A good job performance also depends on the internal environment which includes the employees’ inner drive to perform, their capacity and their knowledge and skills.

Since the time the BEM was introduced in 1978, the model has been updated several times to suit the changing structure of organizations and industries. The figure below shows the ‘updated BEM’ as suggested by Roger Chevalier.

The Updated BEM by Roger Chevalier (info in a few cells modified by iDesign Skills)

The Updated BEM by Roger Chevalier (info in a few cells modified by iDesign Skills)

What this model says is when you have a performance problem; a training program may not be the only solution. The model urges you to analyze both external and internal factors. The model asserts that the performance equation is a sum of both internal and external factors. And this is how the BEM serves as the perfect diagnostic tool for analyzing a performance problem in an organization – for an individual or for a team. And it also concretizes the otherwise abstract mental operation of needs analysis. Here’s how.

We are pre-disposed to concluding that a performance shortfall is the result of ‘lack of knowledge’. Like in the Prima Worth scenario the Delivery Head finds that Project Managers (PMs) do not provide informal feedback. He assumes two things here – one that the need is correct and two – this need can only be fulfilled by training. After a Needs Identification process, if indeed it is found that PMs do not provide day-to-day informal feedback, it is very likely that a half-day training program on informal feedback will be organized. The training program will hit the nail on the head only if the reason for PMs’ performance shortfall is ‘lack of knowledge’ about informal feedback. If the shortfall is due to a lack of motive or lack of information, the training program is not likely to do any good.

The BEM has the potential to save the day by putting itself as a diagnostic tool between verifying a need and proposing  a (most often a training) solution. After verifying the need (using the Questions Grid and expressing it in the 4W format) this need must undergo cause analysis to determine the ‘why’ PMs do not provide informal feedback.

It is possible that PMs’ performance shortfall is a lack of motives (internal factor 4), incentives (external factor 3) or simply are not aware that informal feedback is a part of their role (external factor 1)? Having them undergo training would be an off-the-mark solution.

To enable this analysis, Gilbert’s BEM comes with a questionnaire called the PROBE or Profiling Behaviour Questionnaire. This questionnaire provides a set of generic questions for each factor which can be customized according to context.

The process of identifying and analyzing learning needs then can be summarized as:

Step 1: Decide the information you want to dig up through the NI exercise. Do your best to put it down as a NI goal statement.

Step 2: Your goal statement of step 1 will help you determine the roles you need to interview. This is a good time and place to decide the sample size of your respondents.

Step 3: For each role you will interview, use the Questions Grid to plan and phrase your questions appropriately.

Step 4: Execute your plan. Record responses. Make sure you do not record an interpretation of the responses.

Step 5: Publish your findings. Express your findings in clear unambiguous terms. A format we find useful in expressing a need is the 4W Format. W – who, W – what, W – when, W – why.

Step 6: Use the updated BEM to determine ‘Why’; the cause of the need.

The BEM is simple enough to understand intuitively and deep enough for a meaningful analysis. This quality of the BEM lends it a great deal of credibility among clients who otherwise are skeptical of an analysis on a project. It has helped us to have conversations about needs with stakeholders in projects. The BEM is really a boon that has made analysis a front-end, tangible activity for us at iDesign Skills.

This series on foundations of ID continues with focus on another type of analysis. That’s for next week. Until then…

 

Source:

1. Handbook of Human Performance Technology, 3rd Edition, Pfeiffer

 

 

 

 

 

 

Identifying Needs: Quick but not Dirty

Prima Worth is a large IT organization based in India with a global presence. Each IT project is managed by a team of 5 to 6 Project Managers who report to a Program Manager. The duration of each Project is around 18 to 36 months. The Program Managers report to Sharad Gupte Delivery Head, Projects.

Sharad has requested for a meeting with Prima Worth’s L&D Head, Pradeep Mandke. Sharad is having grave concerns over how his Project Managers manage and handle day-to-day, informal feedback with their project teams.

 “Pradeep, I need you to design a ½ day module on feedback for my PMs” begins Sharad after the initial pleasantries are over. “Sure Sharad, We can. Tell me more. What is the problem?” Pradeep responds. Sharad offers a quick description of what he believes to be the problem. He explains to Pradeep how PMs are not well versed with the idea of day-to-day informal feedback. Sharad emphasizes to Pradeep that while the quarterly review is important and has its own place, it is the day-to-day, informal feedback that saves the day. It helps keeping a check on minor discipline issues, tracking deadlines and quality related issues.

“Sharad, I’ll have a team member take this up immediately. Meanwhile…” Pradeep explains that he will need time to plan a quick study before the program is actually put together. Sharad is tempted to raise an objection to the ‘study’. Deep down he believes that the study will be a waste of time but holds himself back. Instead he promises that he and his team of PMs will give whatever time is required for the study. After discussing possible launch dates for the program, the meeting between them ends.

(This scenario is fictional and sketchy, written only for driving the main idea of this post.)

Most L&D professionals might relate to this scenario. A request for a training program from the line or from a business, where the problem, its solution and the training need are already drawn up clearly before L&D is brought in. The line manager or the business is all too often confident about the need. And I am sure in many cases their conclusion may be spot on. But I feel that is no reason to skip or avoid or assume that a Needs Identification (NI) exercise is not required. If nothing, a NI exercise provides validation of a training need.

In the scenario described above, a NI exercise will determine the ‘real’ need that leads to minor disciplinary and quality issues.  Pradeep’s team must conduct a NI exercise that may either validate Sharad’s conviction or provide new information about the reasons for daily discipline issues amongst team members.

The most practical way forward to gather information about the real issues around daily discipline would be to talk to Project Managers and other stakeholders like team members, Program Managers and Sharad. Of course, this ‘talk’ would need to be planned as a series of interviews with members from each role. The focus of this post is on the importance of interviews as a quick, practical way forward to a good NI exercise. And interviews are made up of questions. At the heart of good interviews are good questions. Nothing deep, not even remotely insightful can come from a question like “What challenges do you face while giving informal feedback to your team members?”

Crafting good questions needs time and the Questions Grid – a framework of different question types you can ask get to the heart of the matter. Just like you need different types of tools to do different tasks, you need different type of questions to ‘dig’ out relevant information from your interviewees. The Questions Grid does just that.

The Questions Grid

The Questions Grid

The question types on the X axis are obvious. But here’s a quick definition of the question types on the Y axis.

Role-specific Questions: Questions that reveal more about the role

Technical Questions: Questions that reveal technical aspects of a role

Problem-based Questions: Questions that reveal specifics of a problem in the role that the employee maybe encountering at that time

Situation Specific Questions: Questions that reveal more about a situation in which the problem occurs and in some cases, a situation- specific question maybe similar to a problem-based question

The Questions Grid is a tool for generating questions and needs to be as a part of a quick, but not dirty 5-step NI process.

Step 1: Decide the information you want to dig up through the NI exercise. Do your best to put it down as a NI goal statement.

Step 2: Your goal statement of step 1 will help you determine the roles you need to interview. This is a good time and place to decide the sample size of your respondents.

Step 3: For each role you will interview, use the Questions Grid to plan and phrase your questions appropriately.

What you need to do is pick and choose from any question combinations that you think will best elicit the information you seek from your respondents. For example you may need a couple of situation-specific probing and clarifying questions for the Delivery Head or a few situation-specific open-ended questions for the Project Manager.

With your NI goal, roles and questions in place, you have a simple but robust NI plan in place. You may also now fine-tune details about the mode of the interview, the time to be allocated to each interview and how you will record the answers you will receive for each question.

A real, jargon-free NI plan can quell apprehensions that managers like Sharad may have about the ‘study’. Most cynical stakeholders may see a value of asking well prepared questions and may actually encourage a NI study the next time around.

Step 4: Execute your plan. Record responses. Make sure you do not record an interpretation of the responses.

Step 5: Publish your findings. Express your findings in clear unambiguous terms. A format we find useful in expressing a need is the 4W Format. Who – describes your audience, What describes the skill or behavior the audience lacks, When describes the specific conditions under which the skill or behaviour is absent and Why describes the reasons for the lack of that skill or behaviour or both. Here is what the format looks like:

The 4W Format

The 4W Format

The result of a NI exercise becomes concrete and specific when expressed in the 4W Format. Stakeholders involved in the project find it easy to agree or disagree on something stated with specificity. A clearly articulated statement like this can take a needs related conversation to another level altogether.  Once agreed upon, this needs statement can serve as a starting point to define a clear specific learning goal or outcome.

The use of constructs like the Questions Grid and the 4W Format make NI a quick but not necessarily dirty process. In fact both constructs add a great deal of value to NI by virtue of making NI simple, intuitive and a transparent process, devoid of technical or ID jargon that all stakeholders understand.  And when stakeholders understand NI, they are seldom resistant to it.

Continuing our series on quick and practical methods of NI, in the next post we dive into the 4th W called “Why”. How do we determine the cause of a need? Until then…