This Small Business Innovation Research (SBIR) Phase II project focuses on the development of an assessment tool that will enable users to profile a decision-maker's cognitive agility and expertise in high-level business situations. It is appropriate for evaluating decision makers in organizations and students who aspire to leadership roles. This version of the product can also be self-administered. It is based on results from recent basic research conducted by Workplace Technologies Research Inc. (WTRI) that revealed the cognitive mechanisms involved in the thinking of highly accomplished experts in business. It uses knowledge elicitation technology that WTRI has developed over several years to support research on the identification of intuitive expertise (in the sense of Dreyfus 1997). The proposal outlines a plan to develop an on-line Internet based version that is self-scoring and tested among well-known experts. The product will be field-tested for its ability to predict general vs. industry specific expertise. The expected outcome is an easy to use tool for professional evaluators, professors, students or individuals, which will assist in staff development and education. The profiles generated by the product will identify hidden strengths, areas of weakness, and suggestions for further development. The long-term goal is distribution by recruiters, coaches, universities and consultancies.
In the current climate of rapid workplace change, decision-makers need to continually evaluate their ability to adapt to changes and re-invent their organization's value and competitive future. Few assessment tools address the cognitive underpinnings of the skill set involved. Rather, they evaluate personal traits or sub-skills that have some correlation with leadership, broadly defined. Using an empirically verified model of expertise in business strategy development and performance prediction, the research team at WTRI has built an assessment tool that locates an individual with regard to this model; much like chess players are evaluated against a notion of a Chess Grand Master. When applied to individual client situations, this tool has been shown to have powerful predictive capability and thus has successfully informed staff development efforts. Its distinctive feature is assessment of the ability to analyze disparate sources information in order to make strategy level decisions and supporting tactical plans. Making the tool more widely available and usable by non-scientists could importantly contribute to efforts to increase the performance of both organizations and decision makers. Organizations, distributors and several institutions of higher learning have expressed interest in this technology, which they consider to be addressing an area of unmet need.
The Cognitive Agility Assessment instrument was developed as a way to measure business leaders’ business acumen and cognitive agility when making decision in an increasing uncertain and volatile business world. It is based on a technique for measuring and mapping the mental models of highly skilled experts called "knowledge elicitation". Knowledge elicitation methods are used by cognitive scientists and cognitive psychologists as a way of understanding intuitive expertise. It does not rely on self report. Rather, performance on specially designed tasks reveal the thinking of the decision maker. The participant does not talk about his or her skill, but rather shows what they know and how they know it. WTRI had been conducting knowledge elicitation interviews using such tasks for many years in order to conduct its own research on expertise and deploy its other products to the best benefit of business customers. Customers began asking for the assessment as a product in itself when the data revealed helpful to succession planning, hiring and team composition at the senior level. Finding significant blind spots in the thinking of senior staff also helped companies avoid risk. WTRI developed a set of online tools with automated scoring that was intended to accomplish the same thing as the structured interviews, but in fact, may perform better due to the elimination of interviewer skill or bias. When used with other assessment tools – which measure personality, leadership, communication style or general intelligence – the profiler adds to a complete picture of an executive’s fitness for a role. The product is sold two ways: First, highly customized instruments measure the participant against a model of a world class expert in (a) business in general, (b) within a specific industry, and (c) in the context of specific strategy that the company is pursuing. These highly tailored instruments have performed very well for executive committee assignments, professional development and succession planning. The second type is much lower in cost and measures the participant against the model of performance of world class expert in a specific sector, such as biotech or durable goods manufacturing. WTRI currently addresses about 20 industries with its set of these OTS versions. Although sector specific, these are not company or strategy type specific. Both types of instruments have online access and automated scoring. Detailed coach-ready reports are available. The software that writes the reports is separate from the instrument itself, so nearly limitless options are available for reporting. In some cases, customers have also requested statistical analyses of trends of groups of managers within a company. The intellectual merit of this product is that with every use, our understanding of true "business expertise" is deepened. At the same time, ongoing research using the data contributes to cognitive science models of expertise in general and the science of knowledge elicitation. The broader impact of the tool is that it has filled an important gap in the information used to plan employee development, succession planning, hiring and the recruitment of executives. Due to its objective and business-skill specific nature, it has enjoyed increased interest and sales since the market downturn of 2007-2008, especially among larger clients. Companies are telling us that they need to feel more confident and are looking for more objective measures of an executive’s ability to do a specific job. For further reading about the Cognitive Agility Assessment Tool (rebranded as the FutureView Profiler) please refer to the company’s website: www.wtri.com. In addition to product descriptions and case studies, there are downloaded published articles about ways the instrument has been used.
