The objective of this award will be to conduct a one day workshop with the aim of investigating whether the design of large complex systems, such as aircraft, spacecraft, or automobiles, should employ a process that allocates requirements down to each component design team, or a process that flows down objective functions. A relevant example of requirements allocation would be an automobile transmission that is required to cost less than $700 to manufacture, last more than 200,000 miles, and transmit power to the wheels with an efficiency greater than 99%. In contrast, an objective function would combine cost, life and efficiency into a scoring function and ask designers to make a transmission with the highest possible score. The objective function fits into an optimization approach which is favored by many theories of engineering design. However, requirements allocation is the standard presented in systems engineering manuals and is the process used throughout industry. The two approaches are very different, and should lead to very different outcomes. Which is better? This award will fund a workshop which will bring together world class experts in systems engineering, engineering design, and engineering optimization to discuss and compare the advantages and disadvantages of each method for the first time. Resulting from the workshop will be a research agenda aimed at significantly advancing the state of design of large-scale complex systems.
Today, designs of large complex systems tend to overrun schedules, often by years, exceeding development and manufacturing budgets (by an average of about 50% in the Department of Defense and NASA). Systems are often cancelled after years of development because cost and schedule are out of control. Preliminary research suggests that most of these delays and overruns, amounting to tens of billions of dollars in losses per year, could be prevented by flowing down objective functions to enable better use of optimization in engineering. A National Science Foundation workshop in February 2010 on fundamental issues in systems engineering identified requirements versus objective functions as the most important research question in the most critical area in systems engineering today. Therefore, this workshop will critically compare the two approaches and recommend an agenda of research to develop the best process and bring it into practice. The results will be disseminated through presentations, conference papers, and journal articles.
On September 12 and 13, 2010, the National Science Foundation sponsored a workshop in Ft. Worth, Texas, to explore the question, "Which provides better guidance for design teams: requirements or objective functions?" with respect to systems engineering on very large complex systems like ships or space launch rockets. Most formal approaches to design are built on optimization, a process that uses objective functions to guide the results. An objective function is a kind of scoring system that evaluates whether one design is better than another. On the other hand, all systems engineering manuals mandate the use of requirements, such as, "the mass of the satellite shall be less than 325 kilograms," and all large design projects use requirements. Instead of comparing designs, requirements determine whether a design is good or bad. Thus, there is a large gap between design theory and design practice regarding the use of requirements. The two methods are quite different and ought to produce very different results. The workshop was structured to explore the advantages and shortfalls of each approach in an informal debate format. Workshop participants included academic researchers and practicing systems engineers from industry and government. At the end of the debate, everyone on the objective functions team, as well as almost all the participants who were asked to argue for requirements, concluded that objective function flowdown is the better way to manage large projects. There were many reasons supporting this outcome, but the two strongest were that (1) objective functions provide clearer guidance to engineers making design decisions, whereas (2) requirements limit the design space and probably, in complex designs, rule out the very best designs. On the other hand, many details about how objective functions can be flowed down in large engineering projects have not been worked out. Considerable methodology remains to be developed. The workshop advanced the fundamental theory of systems engineering by placing a rigorous footing under guidance technologies used for coordinating and directing detailed design activities for organization of thousands of design engineers. Much of the structure of systems engineering, including waterfall and spiral development processes, are anchored in the recursive process by which components at some level in the design hierarchy are decomposed into subcomponents at a lower level. This decomposition encompasses two actions: partitioning the component into subcomponents, and providing guidance to the subcomponent design. This workshop focused on how guidance is provided to design, leading to the surprising result that today's primary guidance technology, requirements flowdown, needs to be replaced. Previous research has suggested that, for large engineering design projects like aircraft and spacecraft, flowing down objective functions rather than allocating requirements could correct massive cost overruns (averaging 50% per program) and years of delays that are responsible for tens of billions of dollars of annual losses in the aerospace industry. The workshop outlined a program of rigorous research to provide a solid base for achieving the goal of eliminating such losses. The workshop was strengthened by inviting a culturally diverse group of researchers, including women and under-represented minorities.
