Quality Assurance Guidelines For Projects in

Quality Assurance Guidelines For Projects in “Process for Analyzing and Managing Project Risk”

Purpose of the Process

This risk management process is used by project teams to identify and handle the risks on their project. It covers the needs of the project team to proactively manage their project. The process may be used to provide information to the risk management work of the overall organization. The process may be used to supply information to Quality Assurance Review activities being done for the State of Texas Quality Assurance Team, but the effort of the Quality Assurance Team has its own process separate from this.

Scope of the Process

Activities Tailoring

The following table describes how risk management activities may be tailored for different types of projects

Activity	Low Focus	Medium Focus	High Focus
Identify Risks	Use a list of categories of risks, or use a list of the key risks often encountered in the organization, to decide whether or not risks need to be handled	Follow the process described in this guideline at the start of the project	Follow the process described in this guideline at the start of the project and at each new phase
Analyze Risks	Review identified risks with a small team and determine how threatening they are to the project	Follow the process described in this guideline at the start of the project	Follow the process described in this guideline at the start of the project and at each new phase
Plan Risk Handling Actions	Include plans for the top 1 or 2 risks in the project work	Follow the process described in this guideline at the start of the project	Follow the process described in this guideline at the start of the project and at each new phase
Track and Control Risks	Monitor risk mitigation activities like other project actions	Monitor risk mitigation activities like other project actions; watch for additional risks to add to those handled	Monitor risk mitigation activities like other project actions; watch for additional risks to add to those handled

Roles Tailoring

The following table describes how roles may be tailored for different types of projects.

Role	Low Focus	Medium Focus	High Focus
Risk Identification Team	Only the project manager and team members	Project team and other stakeholders	Project team, representatives of all stakeholders and other organizations involved in the project
Risk Mitigation Team	Only the project manager and team members	Project team and members of management	Project team and any stakeholder or other organization which is well-equipped to help handle a given risk

Deliverables Tailoring

The following table describes how the deliverables may be tailored for different types of projects.

Activity Deliverable	Low Focus	Medium Focus	High Focus
Top N Risk Chart *	Usually fewer than five top risks	May have 6 to 10 top risks	May have more than 10 top risks. It is recommended those projects with more than 10 top risks are examined for restructuring to reduce the number of top risks. Alternatively, consider terminating the project
Mitigation Action Plan	One sentence action statements	Include actions in WBS	Include actions in WBS
Contingency Plan	Usually not needed	Short, narrative descriptions with rough cost and schedule estimates	Plan size and detail are related to investment levels.
Risk Status Report*	Informal, as part of status updates	Use one report for all risks being handled	Use one report for all risks being handled, with detailed item tracking for most threatening risks

*See Appendix C for supporting templates

Roles in the Process

Role Names	Role Definitions
Project Manager	· drives the risk management process at the start of a project · participates in risk identification, mitigation, and tracking progress throughout the project · accepts or rejects the level of risk for the project
Project Team	· performs the risk management process for this project
Risk Identification Team	· provides input to the process for identifying risks · includes representatives of all affected groups involved in the project (including the Project Team, Steering Committee, organization management, etc.), as well as any others expected to have insight into risks for this project (such as Internal Audit, and/or external reviewers)
Risk Mitigation Team	· performs actions to reduce the exposure from this risk, focused on either or both of probability and consequence of the risk · may be members of the project team, other affected groups, user, customer, management, and others, depending on the risk item

The following sections provide details on each activity – a description of the purpose, entry and exit criteria, deliverables, and the sequence of tasks to be done. Tasks are shown along with the roles generally responsible and/or involved in those tasks. Refer to the “Process for Project Planning” to see how the activities here tie in with those.

When considering the risk management work of the project, there are three areas to consider:

1. Risk management – identifying, analyzing, planning, and monitoring – should be a small part of the project infrastructure.

2. Risk mitigation – the work required to handle the risks. It may be small or it may be significant. In either case, it’s a part of the work breakdown structure of the project, and it gets scheduled like any other work item.

3. Contingency management – work defined in contingency plans. This is generally not included in the project work breakdown structure, but is additional work to be budgeted and done if the contingency condition indicates it is time for the contingency plan. The risk management plan should make such contingency estimates clear, and the plan should identify the method for getting approval for the funds/effort/other resources to conduct the contingency plan, if the need should arise.

As the organization considers a prospective project, it evaluates potential risks to the opportunity, to be able to build a project plan that maximizes the probability of project success. Risk identification is generally done as part of a feasibility study, at the beginning of the active project work, and at each new phase of a large project. The process of identification is assisted by use of risk factor tables that capture indicators of commonly encountered risks. (See the Example Risk Factor Table in the appendices, to understand the content of such a table.)

Roles	Tasks
Project Manager	Select risk identification team. Potential participants include · project team · support groups (SQA, CM, test, documentation, training, etc.) · representatives from other elements of the program, if the project is part of a larger program · partner or supplier representative · customer and user representatives Identify sets of risk factors that may apply. Examples include · general risk factor chart (or one tailored to an organization) · specific risk factor chart for this type of project · lessons learned on previous projects in this organization
Risk Identification Team	Identify which risk factors are relevant to the project, and rate their potential for indicating risk to the project (high, medium, low)
Risk Identification Team	For each high factor, identify the specific risk(s) to the project, citing the condition that could arise and the consequence to the project if it does arise.
Risk Identification Team	Organize the specific risks into sets that support analysis of impact and developing countermeasures. (These vary by project.)
Project Manager	Determine which risks to analyze further.

The identified risks are analyzed to establish the project exposure for each risk and to determine which risk items are the most important ones to address. Risk exposure is defined as the product of the likelihood that the risk will occur and the magnitude of the consequences of its occurrence. In rare cases, the overall project risk exposure will be so high that the opportunities represented by the project really cannot be attained at a reasonable expense. In most cases, though, attacking the most significant of the risk items will maximize the project opportunity.

While the initial risk analysis deals with those risks identified early in the project, more analysis may be needed as the project proceeds. In cases where a new risk is identified, that new risk is analyzed and its exposure compared to that risks already being handled. That new risk may or may not be addressed with a mitigation action, depending on the cost of that action and the ranking of this new risk against others already being handled.

Roles	Tasks
Risk Identification Team	Review each risk item and estimate · probability of occurrence of this risk item · loss if the risk occurs (financial loss or simply a scalar value from 1 to 10) Calculate the risk exposure for each risk item. Rank the identified risks in order of exposure. Review ranked results and ensure team agreement with the ordering.
Project Manager, Senior Management	Consider level of risk represented by the project overall and work decide whether or not to proceed with project as now planned. Assuming the project is to proceed, identify those risks that merit mitigation efforts.

· Accept the risk, with no investment of effort or cost. This is appropriate when the cost of mitigating exceeds the exposure, and the exposure is acceptable.

· Transfer the risk to someone else, or agree to share the risk. If a customer or partner is better able to handle the risk, this is probably the most effective approach.

· Fund and staff the efforts to reduce the probability that the risk will become a problem. Such mitigation tasks might include providing additional staff to help develop the product, getting special training for members of the team, or following a dual development path for the whole project.

· Fund and staff the efforts to reduce the loss associated with the risk should it become a problem. Examples might include keeping a backup LAN operational during the deployment of a new network.

For significant risks that cannot be mitigated, or where countermeasures are unreliable, contingency plans may be established and then executed if the risk becomes a problem. Contingency plans are normally budgeted and approved apart from the plans for project deliverables.

Roles	Tasks
Project Team and Project Manager	For each key risk, identify an approach to handle the risk, and estimate the effort or cost required for that action. For risks that require them, identify contingency plans.
Project Manager	Incorporate the risk handling actions into the project plan. Document required contingency plans and their anticipated cost and effort. Establish agreement with funding source about how to decide when and if to authorize using a contingency plan.

Throughout the project, the project team tracks progress handling the risks, to ensure that:

· when risks for which there is no possible mitigation action have reached a trigger point for the contingency plan, that contingency plan is performed

In addition, the team watches for additional risks that need to be addressed, as well as changes in impact or probability to previously identified risks.

Measures that can be used to determine the effectiveness of risk management include the following:

· Project Risk Exposure Removed - For each of the risks that was managed, determine the monetary exposure the project faced before performing risk mitigation (the probability the risk would occur multiplied by the monetary impact if the risk should become a problem.) Compare this to the amount of exposure that remained after the mitigation activities were done (mitigation should have reduced either the probability and/or the monetary impact of occurrence.) This measure can be used only if the initial analysis of a risk was quantified with financial loss, such as the amount needed for extra memory to add to systems deployed across the state.

· Cost Effectiveness of Risk Mitigation – For each individual risk (or for the total across all risks), compute the monetary value of the risk exposure, the cost of the mitigation action, and ratio of the two to determine the cost effectiveness of the action taken.

Measures that can be used to track the risk management activities include the following:

· Completion of Risk Mitigation Actions - For each mitigation action identified, track attributes such as the following:

· Effort required – compare the amount of effort used for mitigation against what was planned

While risk management is being done, the following verification activities are appropriate for management:

· When reviewing project progress, include risk management status among the items being reviewed. Ensure that the project manager and project team are performing their planned risk management activities and their planned risk mitigation activities.

· When risk management actions require the assistance of management, ensure that the management tasks have been accomplished according to the plan that was set.

The following verification activities are appropriate for Quality Assurance personnel:

· When the project plan is complete, review the approach to risk management to ensure that it is compliant with this process.

· At intervals that match the project’s plan (weekly, monthly, or other), review the activities being done to regularly identify risks, review status in handling the risks, and change the plans as needed to keep the risks in control.

· Upon completion of the project, ensure that a Lessons Learned session has been held, and that risk management activities were included in the items reviewed.

Boehm, Barry. Software Risk Management (an IEEE Tutorial). Washington, DC: IEEE Computer Society Press, 1989.

Charette, Robert. Software Engineering Risk Analysis and Management. NY: McGraw-Hill, 1989.

Department of Information Resources, State of Texas. Guidelines for Quality Assurance Review. February, 1994.

Dorofee, Audrey J., Julie A. Walker, Christopher J. Alberts, Ronald P. Higuera, Richard L. Murphy, and Ray C. Williams. Continuous Risk Management Guidebook. Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 1996.

Down, Alex, Michael Coleman, and Peter Absolon. Risk Management for Software Projects. London: McGraw-Hill, 1994.

Hall, Elaine. Managing Risk, Methods for Software Systems Development, Reading, Mass.: Addison Wesley, 1998.

Jones, Capers. Assessment and Control of Software Risk. Englewood Cliffs, NJ: Prentice-Hall, 1994.

Karolak, Dale Walter. Software Engineering Risk Management. Los Alamitos, CA.: Computer Society Press, 1996.

Project Management Institute Standards Committee. A Guide to the Project Management Body of Knowledge. Upper Darby, PA: Project Management Institute, 1996.

Statz, Joyce. “Whose Turn Is It to Walk the Rhino? Or How Can We Use Risk Management Effectively?” Cutter IT Journal, Vol. 11, No. 6, June 1998, pp. 30-37.

Statz, Joyce and Don Oxley. “From Project Risks to Organizational Learning – A Path to Practical Knowledge Management,” Application Development Strategies, Vol. X, No. 6, June 1998.

Statz, Joyce, Don Oxley, and Patrick O’Toole. “Identifying and Managing Risks for Software Process Improvement,” Crosstalk, Vol. 10, No. 4, April 1997.

Statz, Joyce and Susan Tennison. “Getting Started with Software Risk Management,” American Programmer, Vol. 8, No. 3, March, 1995, p. 23-30.

This table shows an example of what might be found in a risk factor table. It is used for identifying potential risks to a project and for recording lessons learned about risks to projects. As an organization encounters new areas of risk, it extends the table with new factors and/or categories. As processes and methods improve to remove specific risks, factors related to those risks may be removed from the organization version of a table like this.

ID	*Risk Factors*	*Low Risk Cues*	*Medium Risk Cues*	*High Risk Cues*

	Mission and Goals Category
1	Project Fit to Customer Organization	directly supports customer organization mission and/or goals	indirectly impacts one or more goals of customer	does not support or relate to customer organization mission or goals
2	Work Flow	little or no change to work flow	will change some aspect or have small affect on work flow	significantly changes the work flow or method of organization
	Project Characteristics Category
3	Requirements Stability	little or no change expected to approved baseline	some change expected against approved set	rapidly changing or no agreed-upon baseline
4	Customer Expectations	expectations aligned with those of project team	expectations not explicitly reviewed; appear to be in synch	expectations of user differ from those of project team
5	Completion Criteria	completion criteria are agreed on with customer	completion criteria have not been explicitly addressed	completion criteria are in dispute between project team and customer
	Project Deployment Category
6	User Training Needs	user training needs considered; training in progress or plan in place	user training needs considered; no training yet or training plan is in development	requirements not identified or not addressed
7	Customer Service Impact	requires little change to customer service	requires minor changes to customer service	requires major changes to customer service approach or offerings

		Total Categories	3
		Total Factors	7

· Risk factor tables – collections of risk factor tables for different kinds of projects, organized in categories, with cues to help identify risks to a project. These tables are useful for the following kinds of projects

· Top N Risk Chart – form to use to track and manage the top risks in a project

· Risk Item Description – form used to describe a specific risk in detail

· Risk Status Report – form used to report on management of risks at regular project reviews

· Risk Initiation Checklist – items to consider when checking that a project team has begun its project with appropriate risk management

· Risk Progress Checklist – items to consider on a regular basis (perhaps monthly) to ensure that a project stays focused on risk management, identifying new risks and tracking its efforts on handling risks

· Risk Completion Checklist – items to consider when a project finishes, or when a major phase finishes, to ensure the process works well, that lessons learned have been captured, and that the risk management effort was worth the investment it took

Purpose:	Name and describe the specific risks faced in this project, to be able to analyze those risks and decide on an approach to handle them.
Entry Criteria:	Project has been approved for work and team has been named

Purpose:	Determine the projected risk exposure for each identified risk item
Entry Criteria:	Identification process is complete and there is a team that can review the risks to estimate their impact

Purpose:	Establish an affordable set of actions to minimize the exposure from the key risks identified and ensure project success.
Entry Criteria:	List of key risks has been prioritized.

Purpose:	Ensure that mitigation actions are keeping the risks under control, and monitor indicators to know when to invoke contingency plans.
Entry Criteria:	Risk handling actions are staffed and funded. Contingency plans are defined where appropriate.

Revision	Date	Description
0.1	11/10/99	First draft; for review by internal review team
0.2	11/17/99	Internal review updates
0.3	11/23/99	Additions for tailoring of roles and work products; moved process improvement activity to post-project review process
0.4	12/6/99	Incorporated request for example risk factor table and other small changes
0.5	12/7/99	Incorporated changes to tailoring elements of Scope section
1.0	2/1/00	Incorporate Advisory Group revisions

Risk Factor Table	Recommended Project Use
Generic Software Acquisition Management (SAM) Project Risk Factors	Outsourcing of some or all of the project’s software development
Generic Project Risk Factors	Any kind of project, including non-software projects, e.g. infrastructure implementation
Generic Software Project Risk Factors	Software development and maintenance projects
Packaged Systems Risk Factors	Acquisition of third party software packages