PROCESS EFFECTIVENESS ASSESSMENT POLICY INTERPRETATION

National Center for Advanced Technologies
1250 Eye Street, N.W., Suite 1100, Washington, D.C. 20005

April 19, 1996

The Process Effectiveness Assessment Policy statement and The Policy Interpretation describe a fundamental change in systems acquisition practice. The Policy is intended to foster disciplined integrated design and manufacturing processes during the design, development and production of weapon systems. It also promotes the use of the advanced quality systems being applied successfully in today's highly-competitive global economy.

The intent of the Process Effectiveness Assessment Policy is to provide a capability and quality profile of companies competing for, or implementing, DoD contracts. This profile, along with other assessments such as those relating to past performance and facilities can be used as an acquisition management tool for risk assessment/management and for motivating a capable, responsive industrial capability, both commercial and organic. It is not the intent of the Policy that procurement agencies specify or require specific processes or process metrics ("Degree of Maturity"). It is the intent that essential processes be identified by the company involved, and that specially trained DoD teams perform the relevant maturity assessments. Such assessments are not required for every procurement or RFP but should be conducted on a reasonable, periodic basis (determined by major program specifics and process change dynamics) at the facilities of contractors involved or about to be involved in a DoD procurement. This activity must be integrated with the single plant/common process facility initiatives.

As indicated in the Policy, the effectiveness, or maturity, of an enterprise's processes --(e.g. management, engineering, manufacturing, quality, procurement/subcontracting, and logistics support) -- is a significant measure of the productive health and quality of that enterprise. Further, the evidence of continual improvement of those processes is a strong indicator of future health and quality.

Companies that are leaders in their field use the process assessment mechanism to measure competitive strengths and weaknesses and to drive efforts that will enhance their success in the marketplace. Smart buyers of major systems, products, or services use assessments of suppliers' processes to help calibrate and mitigate (or manage) the risk associated with any procurement. In this light, it is the purpose of this Policy to assist in making DoD a "smart buyer". When critical processes are truly effective and under control, the value benefits to the buyer (in terms of performance, delivery, schedule, cost, and quality/reliability) are greatly enhanced.

Included in these critical processes are those associated with interfunctional relationships (engineering/manufacturing/support), intercompany relationships (teaming, supplier management), and customer relationships. These are often embedded in IPPD and IPT activities, and should be included in any process assessment initiative.

In support of the Defense Manufacturing Council, a root cause analysis of "problem" procurements was accomplished by a multi-discipline team from industry and government. The results indicated that a major cause of DoD acquisition problems is the imbalance between product/system goals and the maturity of engineering/manufacturing processes used to reach those goals. (e.g. stable and capable processes are needed to support design, production, operational use, and logistic support for DoD products and systems).

The old paradigm of the defense industry/acquisition environment seemed to favor rapid, low-cost prototyping and development without a balanced emphasis on production. The Process Effectiveness Assessment Policy mandates earlier planning and development of relevant manufacturing processes during the development phase. As a consequence of doing the right technical tasks at the right time in the program life cycle, the traditional program funding profiles must likewise shift, pulling some of the normal production-oriented funding into earlier phases of the acquisition life cycle, [Engineering and Manufacturing Development (EMD) or even Demonstration and Validation (DV).] The Process Effectiveness Assessment Policy is intended to enable a new overall acquisition architecture and procurement approach by more completely addressing the production article along with its design and manufacturing process requirements. In addition, the Process Effectiveness Assessment Policy also requires the acquiring agency to establish source selection criteria which will aid in selecting contractors truly capable of implementing the intent of the technical and programmatic effort through effective processes.

The Policy clearly states that industry -- or individual companies -- "own(s)" the processes that it implements to conduct its business and to address the issues outlined above. As such, each company will necessarily fully characterize each critical process and will define the metrics that measure process effectiveness. Only in this way can a company ensure that a process performs to requirements, is kept under control, and is amenable to continuous improvement.

The DoD assessment of a company's processes will utilize that company's current process performance metrics (along with historical process data) as one of the measures of process effectiveness. Additionally, those metrics that demonstrate process "institutionalization" should also be considered in the assessment. These include:

To fill out the "metrics" suite, a description of general process performance metrics is included in the JACG Report on Key Supplier Processes for Acquisition Programs. This report covers the dimensions of major processes in a very clear and definitive manner.

To assist in the evaluation of the metrics considered in an assessment, process maturity models such as those developed by the Software Engineering Institute (SEI) (e.g. Software Development, System Engineering, IPPD), along with Quality Models such as ISO-9000 or ANSI/ASQC Q90, and manufacturing "models" such as the Air Force Manufacturing Development Guide and Maturity Matrix, can be helpful. It is important that these models not be used to specify requirements or "how to's", but be used as evaluation aids. Process Assessment activities must avoid any temptation to use "guidelines" or "maturity models" as specifications or contract requirements.

As clearly stated in the Policy (and defined in more detail in the JACG Report), Process Assessments should include those related to supplier/subcontractor management, and the related critical process effectiveness in second and third tier suppliers.

To maintain Process Assessment consistency and quality across all acquisition activities, OSD will assemble and train joint service assessment teams. These teams can help enable the implementation of "common processes" within a contractor's facility, as well as assessing the effectiveness of the individual processes.

The viewgraph copy showing a process maturation guide is meant to portray a path to leadership" or a way to view process maturation.

A list of those processes that should be considered in an assessment is attached to this supplement. It is intended that the list be updated as required to ensure relevance to all major procurements.

TYPICAL PROCESSES TO BE CONSIDERED FOR ASSESSMENT

The Process Effectiveness working group endorses the general top level processes as defined by the NGS-IPT/JACG report. These are:

However, in order to show a more detailed view, the following typical processes are those a major company might use. These are provided for information purposes.

Systems Engineering
Product/Sub-system Design
Manufacturing Process Definition/Characterization

Quality

Supply Base Management

Cost Management Processes

Program/Functional (Engineering, Manufacturing, etc.) Management Processes

Information Management/Architecture Processes

Sustainment/Logistics

Go to TOP of page


Home - About NCAT - Affordability - IPPD - Contacts - Links - Archives
This site is under continuous change, last updated on March 21,1999
Please email all comments, errors, or additions to the webmaster.