Qualitative Analysis: During risk identification, the focus was on quantity – identifying as many as possible.  But not all the risks identified merit action.  The purpose of qualitative assessment is to subjectively declare which ones are the most important ones deserving focused attention.  The term subjective is an important distinction from methods used later during quantitative analysis because the assessment is not measured in any precise way.  During qualitative analysis judgment is used to assess evaluations.  This is done by working through the following steps:

1. For each risk, the impact is assessed according to a standard project scale (declared during project risk planning)
2. For each risk the probability or likelihood of occurrence is assessed between (0.0 and .99)
3. The impact rating is multiplied by the probability to obtain a risk score
4. The risk scores are rank ordered to determine the relative priorities
5. According to predetermined thresholds determined during project risk planning, the risks with scores above those thresholds or the top so many in rank order move on to either quantitative analysis (if done for the project) or risk response planning.

Quantitative Analysis:  Not all projects will warrant quantitative analysis.  This is usually only performed for complex projects, long projects, important or high profile projects.  You should use your best judgment as to whether this is warranted or whether it makes more sense to move directly to risk response planning.   Many organizations that perform this process invest in software tools that perform Monte Carlo simulation when there are continuous probability distribution iterations necessary to calculate the possible impact on project objectives.  However, even without such software, it is still possible to perform quantitative analysis by simply shifting emphasis to numerically and objectively analyzing impact and probability for those risks that advance to this phase, rather than the subjective measurement done in qualitative assessment.  Impacts are declared in precise terms of time and cost such as $10000.00 or 10 days rather than a scale value of .8 for example.  Rather than calculating risk scores, expected monetary value (EMV) is used to calculate the exact cost in both time and money for each risk.  This leads to a contingency reserve, which is a budget set aside for funding risk contingency and fallback plans.  The following are the steps:

1. For each risk, the impact in exact time units is assessed (i.e. days, etc.)
2. For each risk, the impact in cost is assessed (i.e. $)
3. If not already available, for each risk numeric probability (0.0 – .99) declared
4. EMV (Time) = probability X Impact time
5. EMV (Cost) = probability X Impact Cost
6. Contingency Reserve (Time) = SUM ( #4)
7. Contingency Reserve (Cost) = SUM (#5)

This allows a more precise calculation of exactly how much the risks and their management will cost.  In my next article I will discuss risk response planning where we will take a look at how to plan strategies to control risks in order to prevent threats and leverage opportunities.

The discipline of risk has six sub component processes:
• Risk management planning
• Risk identification
• Qualitative assessment
• Quantitative assessment
• Risk response planning
• Risk monitoring and controlling
In this article we talk about the first two – planning and identification. In the remaining articles of the series we will discuss the other processes.

Risk Management Planning:  A risk management plan is a document that declares how the remaining steps of the risk process will be performed. It can be a very simple document prepared for each project from an organizational template that states methods and standards that will be utilized for the project. Important things to include in the risk management plan are:
• Methods/techniques to be used for risk identification. Methods/techniques/tools to be used for quantitative analysis (if this step is done)
• Whether quantitative analysis will even be done for the project.
• Historical records and risk categories to use for risk identification – which project archives to explore.
• Roles and responsibilities – who will be part of the team assigned to identify and manage risks.
• Timing – when risk activities will occur – what meetings, etc.
• Definitions of probability and impact – define a common scale and meaning for qualitative ratings.
• Thresholds and tolerances – define the thresholds for advancement of a risk score to risk response planning and where there are special sensitivities (such as cost or time).
• Reporting formats – risk management reports that will be issued and the format.
• Tracking – how the risk process will be reviewed and audited and how activities will be documented. (e.g. where the risk register lives, etc.)
Risk Identification:  After you’ve put some thought into how you are going to do all your risk activities for the entire project, decided who needs to be involved, and written it down into the risk management plan, it’s time to begin identifying risks and writing them down into a risk register. The first thing to be aware of is that a risk is an uncertain event. A fact is NOT a risk. If you know that a certain key team member will not be available for a critical task, that is not a risk to be dealt with in risk management activities. That is a fact to be dealt with in project scope/time/cost/resource activities. So be sure you understand the difference when you begin identifying risks. Also, although not intuitive from the name, a risk can be either positive or negative – either an opportunity or a threat, so be sure to focus on identifying both. Don’t just focus on preventing threats, but also think about how to take advantage of opportunities like sourcing from a supplier that has a lower cost, or finishing a task sooner than planned, etc. Think about who needs to be included to help you with the risk identification activities. These should be subject matter experts who have expertise in the various disciplines involved. This might require that you even go outside the project team and even include customers and stakeholders up and down the organization and in a wide variety of functions. The goal is for quantity over quality at this point so you don’t want to be too restrictive. If someone says “We might get struck by lightning”, go ahead and capture it. You can assess the probability and prioritize it in the next step of the process. Capture all risks in the risk register. Some techniques for risk identification include:
• Review list if risk categories – either a generic commercially available category list or one tailored for your organization. Better still would be one created from past projects. This should not be done as the first risk id step. This should be done after one or two other steps as a “catch-all” to see if anything was missed. The danger in using this first is that people get a false sense of comfort that the list is a comprehensive list of everything possible and they forget to think about what is not on the list.
• Review historical records from previous projects (if available) that are similar to current project – look at risks that were relevant for those projects that may plague your project.
• Review project documentation – There is a reason risk identification comes after all the other project planning activities. You need the output of those exercises to see what the project entails so you know where there is risk. Review the charter, scope statement, WBS, schedule, budget, requirements, specs, etc. for risk.
• Expert Interviews – talk to key stakeholders, customers, sponsors, or members of senior management as necessary to obtain opinions about risk.
• Brainstorming – perhaps the most popular technique, if done properly this method can generate a lot of risks. Best practice is to have 2 scribes in addition to the facilitator. Try to solicit risks in multiple ways: by the project as a whole, by each activity, and by category. Don’t judge – just collect as many as possible! Group them, then clarify.
In the next article we will discuss how to assess and prioritize the risks once you’ve identified them with the qualitative and quantitative assessment steps.

Processes:   The primary scrum processes allow for grooming of the product backlog or clarifying the complexity by specifying effort factor (story points), grouping work from the product backlog into work iterations, defining and estimating technical work tasks for the work iterations, and packaging completed outputs of the work iterations into a working release product.  Below is a breakdown of the different scrum processes along with their descriptions and analogous waterfall counterpart.

Meetings:    Since frequent feedback is the hallmark of agile, the scrum meeting are the settings that facilitate some of the above processes, as well as allow for frequent status during the work intervals or sprints, and provide the vehicle for the customer acceptance and process improvement.  Below is a table that describes the major scrum meetings, descriptions and their waterfall counterparts.

 Artifacts:  Artifacts are sparse compared to a typical waterfall methodology. All work “to do” is tracked on the product backlog.  Work “left to go”  is tracked on one of 2 types of burn down charts and are meant to show progress to the organization.  Business requirements will seem sketchy in agile as compared to waterfall since preference is given to conversations amongst members of the team, and verification through acceptance criteria and demonstrations.  Below is a description of scrum artifacts.

 In scrum, relative estimation keeps the estimation process intuitive and non specific.  It’s done in group session with the entire team where everyone “votes” on their number all at once to keep voting independent.  The relative story points assist the product manager in backlog prioritization.  Sprint planning makes up the rest of the planning for a 30 day period.  This is where the detailed task decomposition and hourly estimation occurs.

Once a sprint is planned, work is executed and monitored in the daily standup meetings.  Issues are tracked in the Sprint impediment log.  Progress is displayed to the organization in artifacts such as the sprint and product burn down charts.

At the end of the sprint, the work is verified and accepted or rejected in the sprint demonstration meeting.  Then the sprint processes are reviewed and suggestions for improvement are made in the sprint retrospective, before reprioritizing the product backlog if necessary and beginning a new sprint.

• Interviews – a one on one or one on several (2-3) of stakeholders question and answer session guided by a pre scripted list of tailored questions.  The questions should be comprised of a set of context free inquiries to promote bias free discussion.  Once the context free answers have been documented, some solution specific questions can be used to explore stakeholder leanings related to specific solutions.
• Brainstorming workshops – a facilitated multi hour, perhaps all day session with all key stakeholders present.  There are usually ice breaking and warm up exercises to set ground rules and promote free thinking.  Then there is grounding in basic project material, followed by group brainstorming to elicit multiple ideas.  Usual rules are to generate as many ideas as possible, piggy back ideas off each other, and suspend judgment on any idea. Towards the end of the workshop the ideas must be grouped, clarified, reduced and prioritized.  These sessions are notorious for sometimes being difficult to manage and requiring the skills of a trained facilitator – preferable one from outside the organization with no political ties to any of the stakeholders.  Typical challenges include:
  • Time Management
  • Domineering
  • Lackluster participation
  • Negative, petty comments
  • Turf wars
  • Post lunch energy sag
• Role Playing – basically this is like walking a mile in the stakeholder’s shoes.  When possible the analyst carries out a day in the job of the stakeholder he/she is trying to gather requirements from – i.e. using the computer application to enter an order, welding a part, etc.
• Scripted walkthroughs – similar to role playing, when an actual role play is not possible, this method allows a simulated experience via a script.
• Prototypes – typically used in the context of a software development effort, a prototype is a partial implementation – a mock up of the external interface or what the stakeholder experiences when he interacts with the system.  This is a good way to let the stakeholder play with the model and discover what feels to be missing.  This sometimes uncovers entire feature sets that until then were unexpressed.
• Storyboarding – Similar, but less elegant than prototypes, story boarding is used to gain early reactions from stakeholders on concepts proposed for deliverables.  Story boards can exist in 3 different styles:
  • Passive, containing screen shots, written business rules, process flow diagrams, output reports, etc. all laid out in a sequence so as to paint a picture or tell a “story” about how things will work.
  • Active, containing slideshows, animation, or simulations to try to make stakeholders see a movie that hasn’t been created yet.
  • Interactive, requiring participation in order to allow the stakeholder to experience the system in as realistic a manner as possible.

From my experience, I like to employ a combination of the above methods.  Personally, I never like to convene a large group to discuss anything with a blank sheet of paper.  I believe beginning a large group session with no content already established at least at a high level – i.e. a blank canvass is an invitation for chaos.  My preference is to outline the areas of requirements and determine who the primary stakeholders are for each area first, from analysis of project objectives and discussion with sponsors.  Then create a targeted questionnaire around each focused area and interview 1 – 3 primary stakeholders who are most directly affected / involved with the topic to get the core facts.  Sometimes a role play or scripted walk through can be combined with an interview to allow the stakeholder to demonstrate their world to the business analyst.  The larger “big tent” sessions can then be a refinement or validation session of those core facts by secondary stakeholders who can add their concerns and needs.  These secondary stakeholders may be primary stakeholders of another area. 

Once an initial set of requirements have been gathered story boards and prototypes allow analysts and implementation teams to demonstrate back to the stakeholders whether they “got it” right and seek verification before they commit to final implementation.

Requirements elicitation is hard work requiring a mind meld of those participating in the process on whatever level of detail is being discussed. The more you limit the number to those who have expertise on a focused topic, the more likely you are to reach consensus.

What are your thoughts?

References:

Leffingwell,D., Widrig, D. (2000) Managing Software Requirements, A Unified Approach, Upper Saddle River, NJ: Addison-Wesley.

The basic premise is quite simple: each participant gets to cast multiple votes on a list of options.  If the list is static, one not generated by the group, this can even be done before a brainstorming meeting. 

The Card Sort: if you have a pre-defined list of options and want to have a clear idea of priorities before the meeting starts, use the Card Sort.  Provide each meeting attendee with a list of options and ask them to sort the options in order of their preference.  This can be done literally, by providing each participant with a stack of index cards (one option written on each card) and ask them to rank the cards.  This can also be done virtually, by sending out a spreadsheet and asking participants to identify the options as their first choice, second choice, etc.  On online survey tool will allow participants to submit their rankings anonymously, if that is important to you group.  Regardless of how you get the information, you will benefit by understanding what your group is thinking before your meeting starts.  If you come in to a brainstorming session with your priorities already identified, you can spend your time thinking about those options instead of coming to agreement on what they are. 

The Poker Game: multi-voting in its truest sense allows participants to cast more than one vote on a list of options the group has created.  There are generally two rules: each participant gets the same number of votes, usually 3 or 5, and can only vote for a certain idea so many times.  Casting votes can be as simple as each participant putting a dot or check mark by the options of their choice, or can involve each participant getting certain number of relics (for example, poker chips) to spend on the options they like best. 

Target Exercise: this technique can help your team identify not only what the priorities are, but what options should be put on the backburner.  For each option under consideration, draw a target on a whiteboard or large piece of butcher paper, with each circle marked as follows:

• Center circle – CRITICAL
• 2^nd circle – Somewhat Important
• 3^rd circle – Marginally Important
• Outer circle – Unimportant

Give each participant a large marker, a stack of post-it notes or a sheet of sticky dots.  For every option, each participant puts a dot in the circle that best represents their feelings about that option.  This allows the group to quickly see which options the group is in agreement on (whether they feel the option is important or unimportant) and which options might need more consideration or discussion because they group lacks consensus.

 This concludes my team creativity series.  I hope you have enjoyed reading it as much as I have enjoyed writing it.

Again, I will reference Team Troubleshooter:  How to Find and Fix Team Problems by Dr. Robert Barner for a description of Attribution Analysis:  “With this technique, your team can attack a problem creatively by breaking it down into its components and then brainstorming selectively around each part.  The last section of the technique rearranges these components in unique combinations that generate unexpected solutions”.   

For a simple example of how Attribution Analysis might work, let’s imagine that our team is responsible for ordering a birthday cake for our CEO’s upcoming birthday.  We know that the cake will be made up of the following components: cake flavor, filing, frosting and decoration.  We create a chart for our team to work with during our brainstorming session.

Now your team can move on to brainstorming all the possibilities for each of these component parts.

Once all the different alternatives have been identified for each component, then a selection can be made.  In the scenario above, let’s say the team selects the following cake for the CEO’s birthday:

Now, this is a very simple example.  But imagine if your team was facing a tough decision about how to handle a daily service outage for a system your entire organization relies on.  You work for a global company, so someone is always using this system – timing the outage “in the middle of the night” is not an option – it is always someone’s business day, somewhere in your global operation. 

Start by getting your team to brainstorm to identify all the possible attributes or categories of exploration related to this particular problem.  Then come up with all the possible options for each category, either as a team, or divided into subgroups with each group responsible for a different category. 

Finally, use all the ideas that have been generated to come up with a solution or a series of partial solutions that might be combined creatively to solve your problem. 

In my next blog, I’ll discuss several different applications of multi-voting in a team problem-solving situation.

There are many benefits to this exercise:

• Participants have time to organize their thoughts and are not influenced by ideas from others
• Participants are less emotional when writing down their ideas
• Less assertive team members are heard equally with more dominant members
• Ideas are de-personalized because they are submitted anonymously

 Today I will give you some additional methods to use with the Nominal Group Technique (NGT), to maximize the many benefits of using this simple tool. 

Affinity Diagramming is a useful technique for organizing ideas that are generated using NGT.  After all of the ideas the team has generated have been read aloud, put the Post It notes or note cards on a table and ask the team to categorize them into related groups.  Some practitioners believe this exercise should be done silently, with team members arranging and rearranging note cards until everyone is satisfied that each idea is sorted into correct groupings.  Others believe it is acceptable for the team to discuss what they are doing.  You can make the best choice for your group based on your history with them and your understanding of their group dynamics. 

However you choose to do it, this allows ideas to be grouped into categories and then sub-categories, and gives your team a broad framework for their detailed discussion of the ideas they have generated.   

The Plus/Delta Technique is an approach for guiding a discussion about ideas that have been generated using NGT.  As the brainstorming session progresses, ideas will be analyzed and evaluated, to determine which ones will become part of an action plan, or left on a parking lot list, or discarded entirely.  A short list of ideas will be selected for detailed discussion.

During this part of the meeting, divergent opinions emerge and participants may become entrenched in the belief that an idea is either “good” or “bad”.  The Plus/Delta Technique is a straightforward means to make sure your team open-minded about the relative benefits and drawbacks of a particular idea. Simply go around the room and ask each participant to name a benefit (Plus) and a drawback (Delta) of the idea under discussion.  This forces each person to acknowledge both the merits and the flaws of an idea, and can move your group past a conversational deadlock.  

Brain Writing is a variation of NGT.   Instead of having each participant write down all the ideas they can think of in the allotted time, ask each participant to spend 1 or 2 minutes and write down three ideas. 

At the end of the time, each person passes their three ideas to the person on their right.  That person must contribute three ideas by either adding to the ideas that have been passed to them, or to create new ideas.  At the end of the time, each person passes all the cards in front of them, up to six cards, to the person on their right.   Post It Notes are really effective for this technique, because ideas can be stuck together and passed in a chain.  Continue passing the notes and either building on ideas or creating new ones for a few rounds, until the chain of ideas becomes unwieldy.  

When the exercise ends, have your team use Affinity Diagramming to organize the ideas.  This is a great tool to have the team collaborate without initial discussion, and can be effective at overcoming a variety of challenges with how a group interacts with each other.  Participants that don’t know each other well or have a history of competitive behavior may be able to collaborate more successfully using this technique. 

In my next blog I will talk about using Attribution Analysis as a team brainstorming technique.