The objective of this Grant Opportunity for Academic Liaison with Industry (GOALI) collaborative research project is to create and validate a utility-based decision methodology for large design organizations using a quantitative model based on risk indicators derived from data about prior design projects. A risk indicator is a state of affairs that has predictive power regarding risk in a design project. The new methodology will provide designers with a formalized means to go from a qualitative understanding of what indicates risk to a quantitative model of the likely outcomes of risk-mitigating actions. The methodology will include novel methods for risk indicator elicitation, model quantification, and decision problem formulation. These will be validated using mathematical analysis and empirical studies. Initial empirical evaluations will be conducted using classroom design projects. Detailed studies will be conducted with engineers from Boeing using engineering project data. Their participation is key because it enables an industrial-scale evaluation of the methodology.
If successful, this research will have a significant impact on large industrial and government organizations that design complex systems under risk. The main outcome of this research will be a validated and cohesive methodology that provides such organizations with new operational capabilities. Prototypes of the computational tools needed to execute the methodology will be made available for use and can provide a roadmap to the development of new commercial tools. It is anticipated that the industry-scale evaluation performed in partnership with Boeing will lead to faster and more widespread adoption of the methodology elsewhere. Workshops are planned to disseminate results to a wide community of design practitioners. Research methods and study results will be incorporated into courses in system design (OSU and TAMU) and engineering management (OSU). Underrepresented students from Oregon middle schools will be engaged through an established program at OSU, where compelling examples of design from Boeing will be highlighted.
Richard Malak, Principal Investigator The goal of this project was to investigate a decision making methodology for large design organizations using a quantitative model based on risk indicators derived from data about prior design projects. The approach provides designers with a formalized means to go from a qualitative understanding of what indicates risk to a quantitative model of the likely outcomes of potential risk-mitigating actions. There main goals of the research include: 1) Elicitation: Identification and codification of risk indicators, risk-mitigating actions, and project outcomes based on both quantitative and qualitative information about past projects; 2) Quantification: Generation and validation of a probabilistic model for project outcomes given observed risk indicators and choice of risk-mitigating action. The research team has found that design project archives can be much less structured and complete than engineering organizations believe them to be. This seems to be a consequence of several things, including the tool used to create the archival records, the culture among those recording the information (i.e., do they see it as a worthwhile use of their time?), and whether people are budgeted sufficient time to complete the work. Common problems included ambiguous entries, missing information, and limited context for the information. This suggests that there may exist an opportunity for research into how large engineering organizations can archive design information effectively. Although there has been much research on information modeling, there is comparatively little research that looks at how engineers can be effective at recording the desired information. This relates to organizational issues, cognition, incentives, and information representations. We expect that our method could have a significant impact to engineering managers and even more broadly to the field of project management. Some of the findings and methodology from this research have been submitted to two premier project management journals and have been well-received. Other findings have appeared in premier engineering design journals, focusing on risk attitudes and data collection in large engineering organizations. The specific deliverables from this project were as follows: (1) a new method to discover taxonomic structure in poorly-structured design archives, (2) a technique for formalizing a decision maker’s risk attitude based on a survey instrument for cases of constant risk aversion, (3) a general methodology for modeling the evolution of engineering technology over time under uncertainty and making design decisions based on this information, (4) a taxonomy of risk indicators for design projects.
