Putting the Ghost in the Machine: Emulating Cognitive Style in Large Language Models

The performance of design teams is dictated not only by the abilities of individual team members but also by their problem-solving styles. Effective management of the design process is essential for navigating complex challenges. It is known that recognizing and understanding diverse cognitive styles enhances collaboration and aligns team members for more effective solutions [1]. Furthermore, embracing the diversity of cognitive styles within a team fosters resilient problem-solving tailored to the team’s needs and strengths [1–3].

One of the theoretical lenses that has been adopted to understand differences in problem-solving cognitive style is Kirton's Adaption–Innovation Theory (A–I theory), which introduces a continuum of cognitive styles ranging from more adaptive individuals to more innovative individuals [4]. According to A–I Theory, adaptors prefers solving problems in a prevailing structure whereas innovators prefer to solve problems with groundbreaking ideas, often navigating through a more flexible structure [5]. This theory has been used to understand the way individuals manage change [6], and it explains the way they display creativity [7] and the way that they perceive and solve problems [8]. This information can be used to allow teams to strategically assemble a blend of individuals with complementary cognitive styles. As organizations increasingly seek innovative solutions to complex problems, the integration of such approaches in management is imperative for staying ahead in a rapidly evolving landscape [2–7].

Artificial Intelligence (AI) has already started to reshape the problem-solving methodologies and decision-making processes that underpin engineering and design [9]. Among them, Large Language Models (LLMs) signify a significant innovation in the field of AI, providing a paradigm shift on how teams solve problems [6,9]. The broad capabilities of LLMs position them as potential contributors within design teams, providing a valuable asset in the complex landscape of design challenges [10,11]. The integration of LLM agents into design teams provides an opportunity to bridge the cognitive gap between more adaptive individuals and more innovative individuals, enabling teams to more easily harness the unique strengths of both cognitive styles [9,11]. This convergence has the potential to design and enable customized design of teams to satisfy specific compositional requirements, therefore presenting a unique opportunity to explore the intersection of machine-generated solutions and human thought processes [12]. However, the degree to which LLMs can emulate important aspects of cognition, such as cognitive style, is currently unknown.

The integration of LLM agents into design teams provides an opportunity to bridge the cognitive gap between more adaptive individuals and more innovative individuals, enabling teams to provide more feasible and better design problem solutions [5]. This convergence has the potential to revolutionize design and enable the customized design of teams to satisfy specific compositional requirements. However, the extent to which LLMs can replicate cognitive styles as described by A–I Theory, and thereby align with a team's problem-solving approach, remains unclear.

This study explores the relationship between LLMs and Kirton's Adaption-Innovation Theory in the context of design-based problem-solving. Specifically, we prompted an off-the-shelf LLM (GPT-3.5) to emulate specific cognitive styles while generating solutions to several canonical design problems. We specifically use a basic zero-shot prompting approach in order to examine the fundamental capability of the LLM. We then evaluated the solutions with respect to feasibility and paradigm relatedness and indicated ways in which the observed values align with patterns known to be displayed by human designers. The remainder of this paper is organized as follows. Section 2 reviews the background that is relevant to this work, including research on cognitive style and LLMs in design. Next, Sec. 3 details the methodology that we utilized in conducting our comparative analysis. Section 4 describes the results of the analysis, and Sec. 5 concludes the paper with a summary of key results, limitations, and future work.

This study explores the relationship between LLMs and Kirton's Adaption-Innovation Theory in the context of design-based problem-solving. We dissect this connection into two key areas: the A–I continuum, and LLMs in design. Our goal is to contribute insights to both the artificial intelligence field and the practical use of LLMs in addressing real-world design challenges shaped by individual cognitive nuances.

Cognitive style refers to the characteristic ways in which individuals perceive and process information, influencing their problem-solving and decision-making approaches [10,11,13]. Moreover, the cognitive style, as assessed by the Kirton Adaption–Innovation Inventory (KAI), has been observed to predict an individual's ability to generate innovative and unique ideas during an engineering design problem [9]. This is often employed to understand how individuals' structure and organize information during problem-solving [14,15]. This highlights the practical application of the A–I continuum in predicting and understanding individual performance in creative problem-solving tasks.

Kirton's Adaption–Innovation theory focuses on the Adaption–Innovation continuum, extensively applied to assess individuals' cognitive styles in problem-solving, design, and innovation contexts [9,10]. Kirton proposed the Adaption–Innovation theory in 1976 which focused on how people fall on the range of adaptors and innovators [6,10]. Rather than strictly categorizing individuals as either adaptors or innovators, the theory depicts a continuous range, acknowledging that people exhibit varying cognitive styles between these two extremes [5]. The terms “more adaptive” and “more innovative” are used to more accurately describe people who fall more to one end of the spectrum than the other, emphasizing the nuanced nature of cognitive styles in problem-solving and innovation. Moreover, it is important to note that one cognitive style is not necessarily better than another—rather, they are all simply different ways of being creative and solving problems.

More adaptive individuals prefer incremental changes in existing frameworks [7]. Their inclination is toward generating solutions that enhance the current paradigm, essentially aiming to improve existing approaches [16]. They also prefer their solutions to be more structured and are more rule-based and comfortable generating solutions that are under constraints [17]. As they are associated with more structures, they tend to tighten definitions [7]. Collaborative work within formal structures is a common preference for more adaptive individuals [15,16]. These individuals are often more comfortable solving problems than finding them [18].

In contrast, individuals who are more innovative prefer transformative solutions and may rarely take existing frameworks into consideration [6]. Their inclination is toward generating solutions that are paradigm-modifying or even paradigm-breaking [12]. In simpler terms, they prefer approaching problems in unconventional ways, seeking distinct solutions [19]. More adaptive individuals typically operate outside traditional rule structures and focus on the broader perspective of a problem, disregarding specific rules and regulations, and preferring to solve a problem with a looser structure [10,16]. Due to their inclination toward more flexible approaches, more innovative individuals may prefer working alone, as group dynamics may lead to conflicts stemming from differing ideas [16,18]. Their problem-solving approach is centered on discovering new problems and finding novel solutions, contrasting with more adaptive individuals who prioritize solving known problems [15,18].

A mixture of adaptors and innovators is known to lead to dynamic and resilient innovation ecosystems [20]. Adaption–Innovation theory acknowledges that both adaptive and innovative individuals are crucial and contribute significantly to the overall progress of teams [5]. For instance, a predominantly adaptive thinker may excel in optimizing processes and maintaining stability, while a predominantly innovative thinker may thrive in generating groundbreaking ideas and driving transformative change [16–19]. Thus, it is often necessary to form teams that have a balanced mix of adaptive and innovative team members, enhancing problem-solving and overall performance. This approach fosters adaptability and continuous innovation in organizations, highlighting the crucial importance of diverse cognitive styles for holistic problem-solving.

Large Language Models (LLMs) are increasingly widespread in both academic and industrial domains [20,21]. These models, such as OpenAI's GPT-3.5, are known for their ability to generate human-like text [22]. Essentially, these models are deep learning models trained to produce text that closely resembles human language, achieved through computing probability distributions over possible combinations of pre-embedded words (tokens) [20,23,24]. A key feature of LLMs is their in-context learning capability, where the model is trained to generate text based on a given prompt, contributing to their contextual understanding and coherent text generation [24,25]. This provides an advantage over other AI models such as Bidirectional Encoder Representations from Transformers (BERT) which are less focused on contextual adaptation [26,27]. Furthermore, LLMs are versatile and have better generalization capabilities as compared to BERT which makes them more suitable for solving engineering design problems [27].

Transformers, the underlying architecture of LLMs, are deep learning models capable of sequence-to-sequence predictions, including tasks like text generation [25,28]. GPT-3.5, with its transformer-based design, exhibits the capability to focus on specific parts of the information it receives, enhancing its adaptability and precision in generating contextually relevant text [9,29]. LLMs have proven to be versatile tools with applications in various domains, including text completion, language translation, and even creative writing [29]. Their proficiency in contextualizing information and generating coherent responses has ignited interest and innovation in the fields of machine learning and AI.

Within the context of design, it may be possible to use LLMs to enhance design efficiency by automating tasks like generating specifications and prototypes [30]. This potentially allows human designers to focus on creative aspects, boosting productivity which can in turn help in building a strong economic future [29,31]. This collaboration between LLMs and design teams facilitates quicker iterations and improvements; for instance, an LLM can generate innovative ideas for team members with a high innovation orientation while providing detailed, incremental enhancements for those who focus on refining existing designs [32,33]. This tailored approach streamlines the design process, leverages the cognitive strengths of all team members, and enhances creativity. Additionally, LLMs enable more dynamic and iterative design processes by rapidly generating and evaluating multiple options, which is especially valuable in fast-paced or complex environments [32,33]. Another benefit of incorporating LLMs into design teams is that these models can help in providing inspiration for different design problems, thereby enhancing the efficiency of problem-solving approaches [34].

The interplay between cognitive styles, LLMs, and engineering design represents a pivotal advancement that reshapes the landscape of design methodology. Within the realm of engineering design, where complexity and innovation conger, understanding and accommodating diverse cognitive styles become crucial. Design problems demand flexible approaches [35]. Engineers exhibit a broad range of cognitive styles—from high adaptors who excel at refining existing solutions to high innovators who thrive on creating novel approaches [34,36]. This variability complicates traditional design methodologies, often leading to suboptimal outcomes or friction within design teams. Design problems, by their nature, require flexible and adaptive approaches [37,38]. Traditional design processes, which may not fully account for this variability, can lead to inefficiencies and reduced collaboration effectiveness [30,39]. This is where the integration of AI, specifically LLMs, can revolutionize the design process. By leveraging their in-context learning capabilities, LLMs can assimilate prompts and generate solutions that resonate with the cognitive preferences of design team members [40]. For example, when designing a new wearable device, an LLM could assist an innovative designer in brainstorming cutting-edge features such as advanced biometric sensors, while providing an adaptive designer with detailed suggestions for optimizing battery life and improving ergonomics based on existing data. By integrating human ingenuity with machine intelligence, this paradigm shift promises to unlock new frontiers of design innovation while addressing the inherent complexities of problem-solving in the modern era. The synergy between LLMs and cognitive styles fosters a more inclusive and adaptive design process, where diverse perspectives and problem-solving preferences are harmonized [33]. This holistic approach promises to enhance design efficiency, foster creativity, and lead to groundbreaking solutions that push the boundaries of engineering design [41].

This paper investigates the integration of LLMs within design problem-solving frameworks, specifically through the lens of Kirton's Adaption–Innovation (A–I) theory. The research addresses the gap between the theoretical potential of LLMs and their practical application in engineering design. While existing literature has explored the use of design prompts with LLMs [40,42], this study offers a novel perspective by focusing on how these models can emulate and adapt to different cognitive styles within design teams.

Kirton's theory posits that individuals possess varying cognitive styles, from high adaptors to high innovators. By examining the degree to which GPT-3.5, a common LLM, can emulate these cognitive styles, this paper aims to enhance the understanding of how AI can complement human creativity in complex design scenarios. The paper critically examines the accuracy with which LLMs, specifically GPT-3.5, can emulate cognitive styles, particularly those outlined in the A–I theory, offering a deeper insight into their role in facilitating more effective and harmonious design teamwork.

This work explores the potential for emulating cognitive style in LLMs. To do so, we conducted an experiment in which GPT-3.5 was prompted to generate solutions either from an adaptive perspective or an innovative perspective. This section details the methodology used in that experiment. Section 3.1 provides a thorough examination of the design prompts utilized to generate solutions for design problems. Section 3.2 then details the A–I Theory prompt that we generated as part of this work. Next, Sec. 3.3 sheds light on the evaluation metrics employed to analyze the solutions, aligning with Kirton's Adaption–Innovation theory. These metrics play a crucial role in objectively assessing the efficacy and adaptability of the proposed solutions.

For the research, we employed GPT-3.5 to generate solutions for three design problems. The adaptive prompts for these problems are provided in Table 1, while the innovative prompts are provided in Table 2. These design problems were extracted from the previous research paper written by Goucher–Lambert et al. [11,19,29,31,43]. It should be noted that periods were eliminated from the end of each prompt based on insights gained from prior research [18]. This adjustment was made to enhance the effectiveness and conciseness of generated ideas. It was observed that adding a period to the prompts resulted in shorter and less descriptive solutions, while the omission of periods led to more comprehensive and insightful descriptions, thus providing a better understanding of the viability of each solution.

In order to provide more context to GPT-3.5, the prompts provided in Tables 1 and 2 were prepended with a paragraph describing key features of Adaption–Innovation Theory. This provided context for the LLM to apply while generating prompts specifically for the A–I continuum. The prompt used was inspired by Kirton's book on KAI Theory [5]. The prompt given in full was:

It should be noted that our approach conforms to a zero-shot prompting paradigm [18]. Zero-shot prompting refers to scenarios in which the LLM is presented with a prompt or task without any specific demonstrations or examples [44]. A zero-shot prompting approach was chosen to understand if LLMs can reflect the cognitive styles without nuanced prompt engineering techniques. Techniques such as few-shot prompting and chain-of-thought should be examined in future work. In this context, GPT-3.5 was tasked with generating solutions for design problems (given in Tables 1 and 2) with the only additional context being the cognitive style prompt.

Every solution generated by the LLM was evaluated based on feasibility and paradigm relatedness. These metrics were chosen as they align closely with KAI theory. Specifically, we expect that more adaptive individuals should provide paradigm-conforming solutions with higher feasibility, while more innovative individuals should provide paradigm-breaking solutions as seen in previous research work [45]. The rubrics used for evaluation were inspired by prior research work [20,25,32].

Feasibility is a metric that explores whether an idea can be effectively executed and aligns with established constraints [39,40]. The evaluation of feasibility adopted an approach inspired by previous research papers [1,22], utilizing context-specific guidelines as the scoring requirements. To systematically evaluate the feasibility of the design solutions, we formulated a set of specific requirements related to both technical and physical aspects of each solution, drawing from established guidelines [1]. Each problem was subjected to four feasibility requirements, with two being consistent across all problems. These requirements were that the principles utilized in the design problems were scientifically sound and the solutions takes size and weight into consideration, whereas the other two requirements were specific to the problem type. This evaluation approach, previously utilized by Henderson et al. [46], Silk et al. [45] and others [47–49], was employed to evaluate the overarching question: Does the design concept function effectively both technically and physically? This rubric was chosen to assess both technical and physical feasibility, as well as their alignment with the design prompts [1]. The requirement for scientific soundness guarantees that the design principles are grounded in credible theories and practices, while the consideration of weight and size ensures practical applicability. By including context-specific guidelines, we address unique challenges to each design prompt, hence enhancing the relevance of the feasibility assessment. To quantify feasibility for each design solution, the values corresponding to each requirement were summed, yielding a feasibility score. The overall feasibility score for each design problem was determined by calculating the average feasibility scores, offering a comprehensive view with scores ranging from 0 to 4. Statistical tests were conducted using the Kruskal–Wallis test, a nonparametric method used to assess differences among groups based on ranks [50]. The method was chosen due to its robustness in handling nonparametric data and its suitability for comparing multiple groups which aligns with the nature of the feasibility scores obtained for each design solution.

Paradigm relatedness is important for assessing solution outcomes within the context of A–I theory [1]. Here, a paradigm refers to the available ways of perceiving and acting in a given situation or problem, while relatedness measures the extent to which an idea operates within that paradigm [24,25]. For the evaluation, three distinct levels of paradigm relatedness were taken into consideration. The first one is paradigm-preserving (PP) considered as 0, the second one is paradigm-modifying (PM) considered as 1, and lastly, strongly paradigm-modifying (PM+) is considered as 2 [45]. A paradigm-preserving solution is one that clearly functions within the established paradigm. On the other hand, strongly paradigm-modifying solutions fall outside of the established solution paradigm in more than one way [24,25]. To evaluate paradigm relatedness for each design solution, scores were assigned based on these three levels. The overall Paradigm relatedness score for each design problem was determined by calculating the average PR scores, offering a comprehensive view with scores ranging from 0 to 2. Additionally, the standard error was computed for paradigm relatedness, contributing to the reliability and precision of the evaluation process. Along with that, the statistical tests were conducted to evaluate statistical differences in paradigm relatedness scores between prompts and design problems.

Solutions for three design problems were generated using GPT-3.5, and these solutions were then assessed based on their feasibility and paradigm relatedness. The results are discussed in sections corresponding to these metrics.

When scoring feasibility, the overall Cohen's Kappa score was found to be 0.78. The level of agreement indicates that there was substantial agreement in the evaluation of design problems, affirming the reliability of the feasibility assessments. Higher feasibility scores are typically associated with solutions produced by more adaptive individuals, while relatively lower feasibility scores are sometimes associated with more innovative individuals. Figure 1 illustrates the distribution of feasibility scores for the solutions to all three design prompts when prompted adaptively. Similarly, Fig. 2 displays the feasibility scores for the solutions to all three design problems when prompted innovatively.

From these plots, it is evident that the LLM produced solutions with higher feasibility when the adaptive prompt was used. The median score for solutions generated using the adaptive prompt across all three design problems was calculated to be 4, whereas for the innovative prompt, the median score was calculated to be 2.5 (see Fig. 3). A Kruskal–Wallis test revealed that there are significant differences in feasibility scores between the two prompt styles across all design problems (H = 15.39, p = 8.75 × 10⁻⁵). Specifically, for the personal belongings design problem, the tests yielded substantial H statistics and low p-values, indicating significant discrepancies in feasibility scores (H = 5.5, p = 0.018). Additionally, for the food container design problem, the statistical tests showed a significant difference in feasibility scores (H = 7.03, p = 0.008). For the portable exercise machine design problem, the results trended toward significance (H = 2.618, p = 0.106). These findings validate the efficacy of both adaptive and innovative prompt styles emulating cognitive style.

The observed patterns in the feasibility scores, evident from the plots, clearly indicate that the adaptive approach consistently demonstrated higher feasibility compared to the innovative approach across all three design problems. The robustness of the adaptive approach is emphasized by the consistently higher feasibility scores observed across all three design problems, suggesting that the GPT-3.5 implemented adaptive approach inherently excels in generating solutions aligned with the adaptive nature emphasized by Kirton's Adaption–Innovation theory. Kirton's theory emphasizes the delicate balance between introducing novelty and maintaining practicality. The adaptive approach, by its very nature, produces solutions that are exceedingly feasible within the established framework.

In contrast, the innovative approach showed lower feasibility scores. The divergence introduced by innovative ideas may have led to solutions perceived as less feasible which shows that the solutions were less adaptable. Innovative ideas by their nature, often introduce complexity and deviation from established norms. In the context of Kirton's theory, where adaptation is crucial, the innovative approach might have struggled to align its ideas therefore leading to lower feasibility. GPT-3.5 generated solutions that, while creative, posed practical challenges in terms of implementation. Thus, efficiently satisfying the norms of the more innovative individual shown by Kirton's theory. One limitation observed is that even though the score is low for an innovative approach, it remains relatively high. For the innovative approach, the feasibility is generally estimated to be 1 or 0 which shows that there might be some discrepancies in GPT-3.5. As GPT-3.5 is trained on text databases that are found on the internet [28], the solutions are closely formed to what the knowledge GPT-3.5 has from before therefore not able to generate completely revolutionary solutions for the innovative approach.

Additionally, the calculated standard errors provide insights into the precision of the feasibility measurements. The standard error of 0.03 for the adaptive approach indicates a consistent and reliable perception of feasibility, suggesting a high level of agreement in how its solutions were perceived across the three design problems. This consistency adds credibility to the conclusion that the adaptive approach consistently showed higher feasibility therefore satisfying Kirton's Adaption–Innovation theory. On the other hand, the slightly higher standard error of 0.05 for the innovative approach suggests greater variability in the perceived feasibility of its solutions. This shows a broader range of perceptions regarding the feasibility of solutions. This variability may reflect the diverse nature of innovative ideas and shows that it moderately agrees with the KAI theory.

When evaluating paradigm relatedness, the overall Cohen's Kappa was found to be 0.71. The level of agreement indicates that there was substantial agreement in the evaluation of design problems, affirming the reliability of the paradigm relatedness.

Much like the results for feasibility, the results for paradigm relatedness largely align with expectations. Individuals who are more adaptive are more likely to produce paradigm-preserving solutions, due to a greater sense of rule/group conformity and lower sufficiency of originality. In contrast, more innovative individuals are more likely to produce paradigm-modifying or strongly paradigm-modifying solutions. Figure 4 shows the paradigm relatedness across all three design problems, generated using the adaptive prompt. Similarly, Fig. 5 displays the paradigm-relatedness scores for all three design problems generated using the innovative prompt. A value of 0 indicates a paradigm-preserving solution, while a score of 2.0 indicates a strongly paradigm-modifying solution.

From these plots, it is evident that the adaptive approach exhibited paradigm-preserving solutions, while the innovative approach had paradigm-modifying solutions. The median score for the adaptive approach across all three design problems was calculated to be 0.0 (paradigm-preserving), whereas for the innovative approach, the median score was calculated to be 1.0 (paradigm-modifying). This difference is illustrated in Fig. 6. Moreover, Kruskal–Wallis tests revealed a significant difference in the paradigm relatedness of solutions generated using adaptive and innovative prompts (H = 18.65, p = 1.57 × 10⁻⁵). These differences were observed for the personal belongings design problem (H = 4.7, p = 0.029), the food container design problem (H = 5.6, p = 0.018), and the exercise machine design problem (H = 8.02, p = 0.005). These results highlight the potential of LLMs to emulate diverse cognitive styles.

Looking at the plots, the detailed analysis shows that the adaptive approach yielded solutions closely adhering to and preserving existing paradigms, as is evident from the median score of 0.0, which is indicative of being close to paradigm-preserving solutions. This suggests that GPT-3.5 was able to generate solutions that maintain continuity within established norms, therefore satisfying KAI theory. Whereas, the innovative approach demonstrated paradigm-modifying characteristics, as the overall median score was 1.0. This indicated that the solutions were transformative and redefined established design norms. GPT-3.5's capability to produce solutions closely aligned with the creativity and paradigm-shifting tendencies of high innovators is evident in this result, hence satisfying the KAI theory. The detailed examination of the scores suggests that GPT-3.5 possesses the versatility to generate solutions that align with both adaptive and innovative approaches.

The solutions generated by GPT-3.5 for the adaptive approach are consistent as the standard error was calculated to be 0.04. This low standard error suggests a high level of reliability in the adaptive approach's performance, aligning with Kirton's Adaption–Innovation theory. Whereas, the innovative approach's higher overall standard error of 0.07 suggests greater viability in performance. While the approach excels in generating paradigm modifying, it is expected to have strongly paradigm-modifying solutions for a more innovative approach. There are some solutions generated by GPT-3.5 as paradigm-preserving which showed that GPT-3.5 was unable to create all 10 solutions as paradigm-modifying. Examining the plots for the adaptive approach reveals a predominant trend of paradigm preservation, further confirming the consistency observed. However, in the case of the innovative approach, the solutions appear to be a mix of paradigm-preserving, paradigm-adapting, and paradigm-modifying. This variability suggests a potential limitation in GPT-3.5's ability to consistently generate highly revolutionary solutions. By refining zero-shot prompting techniques, LLMs can generate solutions that align with specific cognitive styles, enhancing their performance. This improvement suggests a promising avenue for more effective human–AI collaboration, with LLMs delivering tailored and relevant solutions.

The research presented promising results by exploring the ability of LLMs to generate solutions based on Kirton's Adaption–Innovation Theory. GPT-3.5 demonstrated proficiency in generating solutions for the adaptive approach across all design problems. However, when it came to the innovative approach, the LLMs provided solutions that were not as paradigm-modifying as expected. This could be attributed to some bias toward prior knowledge, as GPT-3.5 is trained on data sourced from the internet. This limitation suggests that the model might lack the information needed to generate entirely distinct solutions aligned with highly innovative individuals. Acknowledging and addressing these biases is crucial for improving the model's capacity to generate more purely innovative solutions.

A significant limitation observed in the study is also the focus on a specific set of design problems. Consequently, the generalizability of the approach used here to more complex or nuanced design challenges remains uncertain. Future research could expand the scope to encompass a broader range of design scenarios, providing a more comprehensive understanding of the model's capabilities.

Fine-tuning strategies specific to Kirton's Adaption–Innovation theory is essential. This involves tailoring the model to better understand and generate truly revolutionary solutions. Incorporating additional training data or refining the fine-tuning process for both adaptive and innovative approaches could enhance the model's performance. In addition, employing prompting engineering strategies like chain-of-thought or few-shot prompting could result in a more accurate emulation of cognitive style. Few-shot methods involve including examples in GPT-3.5 to provide a better context and understanding of the topic. Additionally, the research focused solely on GPT-3.5, which may introduce bias into the results. Future studies could explore alternative LLMs or consider fine-tuning existing models to yield better design responses. This would contribute to a more comprehensive understanding of the capabilities of different language models in the context of design innovation.

The motivation for this research is to create AI-assisted design teams to provide efficient problem-solving methodologies. The study evaluates the potential of LLMs to emulate cognitive styles, specifically Kirton's Adaptive-Innovation Theory. The results indicate that the LLMs are able to emulate some attributes of more adaptive and more innovative individuals, when evaluated in terms of feasibility and paradigm relatedness. The results demonstrate that with base line zero-shot prompting the LLM's generates highly feasible solutions when prompted adaptively, while generation solutions with lower feasibility when prompted innovatively. Along with that, the evaluation of paradigm relatedness revealed the LLM's ability to provide paradigm-preserving solutions for the adaptive approach, aligning with structured rules and innovation within established frameworks. However, for the innovative approach, the solutions lean toward paradigm-modifying but lack strong paradigmatic shifts.

Building upon these insights, future work could involve fine-tuning LLMs or exploring alternative models for enhanced performance. Additionally, implementing few-shot prompting or chain-of-thoughts prompting methods may prove instrumental in refining the generation of solutions for complex design problems. Furthermore, the paper only covers the initial design solution of the design problems which can be discussed more in the future as design problems are an iterative process and a deeper study can be done on using LLMs for iterative processes to support human–AI collaboration. This research lays the groundwork for advancing AI-driven problem-solving in design teams, with avenues for further optimization and exploration of novel methodologies.

The authors would like to thank Yashraj Shankar, Sriya Nallani, and Rayhan Ishank, graduate students of Carnegie Mellon University who helped in evaluating the design solutions for both paradigm relatedness and feasibility.

There are no conflicts of interest.

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.

The appendix shows the evaluation metrics and the solutions generated by LLMs. Paradigm relatedness was common for all of them.

Problem 1: Design a way for someone solutions to secure people's belongings in public to prevent theft quickly without disrupting the space.

F1. The principles utilized in the design concept are scientifically sound.

F2. The design concept successfully secures items.

F3. The design concept successfully maintains the space.

F4. The design concept considers technical feasibility requirements such as weight and size.

Problem 2: Design a way for individuals who have limited or no use of one upper extremity to open a lidded food container with one hand.

F1. The principles utilized in the design concept are scientifically sound.

F2. The design concept successfully offers a way for people to open a container with one hand.

F3. The design concept considers technical feasibility requirements such as weight and size.

F4. The design concept can be operated solely with human power.

Problem 3: Design a lightweight exercise device that can be used while traveling.

F1. The principles utilized in the design concept are scientifically sound.

F2. The design concept successfully provides a solution to the problem.

F3. The design concept takes size and weight into consideration.

F4. The design concept is portable and easy to take around.

Problem 1: Design a way for someone solutions to secure people's belongings in public to prevent theft quickly without disrupting the space.

Problem 2: Design a way for individuals who have limited or no use of one upper extremity to open a lidded food container with one hand

Problem 3: Design a lightweight exercise device that can be used while traveling