Background: This is a project in collaboration with the Economics Department at the LMU and a start-up called Naven Labs. Naven Labs is developing an AI-based solution for firms to diagnose potential roadblocks in their strategy implementation and to identify effective solutions. First, AI-led interviews are conducted with decision makers in the firm, in which they are asked about barriers to strategy execution. The interviews are then analyzed and aggregated, and patterns and problem areas are identified. The AI then suggests corresponding solutions. All results are visually represented in an interactive dashboard.

As part of the analysis, the AI interprets the interviewee's answers and identifies underlying issues and barriers, aggregating them across all interviews to identify patterns and pin down the key problems. It also deduces a numerical score from 1-5 to rank a set of pre-defined indicators. For example, the quality of leadership or clarity of the strategy are scored on a scale from 1-5 by the algorithm based on the interviewee's answers. In order to provide consistent, transparent and high-quality data, the analysis needs to be tested and quality-assured and the optimal strategy to aggregate the data across interviewees needs to be developed.

Tasks: In cooperation with the Economics Department and Naven Labs, your task will be to help improve the data analysis process and scoring system of the AI-led interview and design an optimal strategy for aggregation. This includes

• Conducting a literature research on existing resources about optimal interview data analysis, interpretation, and aggregation by an AI algorithm
• Designing an optimal scoring system that provides high-quality, consistent ratings
• Setting up and running an A/B-test comparing the output generated by human evaluators, an autonomous LLM and the model-augmented LLM
• Supporting the start-up in the programming of the prototype
• Documenting findings and insights in a BA or MA thesis and presenting them in the Disputationsseminar
• Supporting the creation of a report on the findings and insights to be used by the start-up in sales pitches and their own documentation

Skills needed:

• German language skills
• Strong communication skills, esp. towards non-experts in the field
• Good knowledge of Python and TS