“It is our job to create computing technology such that nobody has to program. And that the programming language is human.”,

Jensen Huang

Software development has always been a dynamic field. The advent of Large Language Models (LLMs), such as OpenAI's GPT series, has introduced a new paradigm of AI-assisted software engineering, significantly enhancing the speed of code creation.

Nothing new comes without challenges, and developers, along with organizations, must be mindful of the potential complexities of code generation. Let's consider differences in concerns of ‘Rapid prototyping’ and ‘Traditional development’ and how we can overcome them.

Rapid prototyping

Key concern:

• Speed – Prioritizing fast development cycles to quickly validate ideas, iterate on solutions, and adapt to changing requirements.

We already have AI-assisted tools that revolutionize rapid prototyping of ideas. Tools like bolt.new, v0.dev, screenshottocode.com can generate code for entire full-stack web and mobile applications.

The results are impressive and good enough to get customer feedback and quickly iterate, but they are not production-ready. Given the emphasis on speed in rapid prototyping, it is a great use-case for AI-assisted engineering.

Traditional development

Key concern:

• Quality – Ensuring the generated code adheres to non-functional requirements (architectural characteristics) like reliability, maintenance, security, etc.

Let us find a balance between the speed that AI can bring and demands of a production environment.

Feature Level Quality - TDD with a twist

To address quality concerns, I propose to build on top of the concept of Test-Driven Development (TDD), which was introduced by Kent Beck in the late 1990s while working on Extreme Programming (XP).

The idea of TDD is simple: you write tests for your functions before writing the actual code to ensure that the code behaves as expected. If you need a quick refresher, I recommend Martin Fowler’s article on TDD.

3 key reasons it is better that way:

1. higher solution quality - well-structured, modular, and maintainable code

2. higher test quality - tests are not biased towards generated code

3. higher generated code quality - LLM will use tests to ensure that generated code meets requirements (self-verification & correction)

The TDD twist is that you need not only tests for generating code. You will need 3 things:

1. function signature with typified input and output parameters

2. detailed definition of function behavior

3. test-scenarios that cover all the aspects of expected function behavior

Code example in Python:

Code generation flow chart:

System Level Quality - Fitness Functions

You still need to cover non-functional requirements (NFRs), including security. To address this, I recommend leveraging the concept of the "architectural fitness function”, an idea introduced by ThoughtWorks architects and explored in the book Building Evolutionary Architectures.

“During test-driven development, we write tests to verify that features conform to desired business outcomes; with fitness function-driven development we can also write tests that measure a system’s alignment to architectural goals.”, ThoughtWorks’ FFDD (Fitness Function Drivem Developmemt)

What makes fitness functions effective? They’re automated, quantifiable, and continuous—integrated into your CI/CD pipeline. By embedding these system-level tests into your development process, you ensure that NFRs are consistently validated. For example, you might define a fitness function to measure response times under load (performance), enforce encryption standards (security), or verify system uptime (reliability).

Fitness functions act as guardrails, detecting deviations before they escalate into problems.

Summary

Rapid prototyping

There are already available tools for rapid prototyping that can generate code for entire full-stack web and mobile applications to quickly validate ideas, iterate on solutions, and adapt to changing requirements. Given the emphasis on speed during rapid prototyping, no additional considerations are typically required.

Traditional development

As LLMs can generate invalid, insecure, slow, etc. code, you need to maximize probability of valid code generation and protect other architectural characteristics (non-functional requirements) of a solution.

To ensure code quality, you should define before code generation:

1. function signature with typified input and output parameters

2. detailed definition of function behavior (what function should do)

3. test-scenarios that cover all the aspects of expected function behavior

To ensure solution quality, you should create a set of automated, quantifiable, and continuous—integrated into your CI/CD pipeline tests.

By embedding these system-level tests into your development process, you ensure that architectural characteristics are consistently validated. For example, you might define a test to measure response times under load (performance), enforce encryption standards (security), or verify system uptime (reliability).

To avoid legal issues, many LLMs creators focus on usage of permissively licensed code (e.g., MIT, Apache 2.0) or code explicitly opted into training. However, to protect themself they still can ask you to accept liability for LLM usage.

IDE plugins developers that use LLM for code generation can further push liability to users of their plugins. Right now based on my analysis of popular IDE plugins there are 3 options:

1. IP indemnity - plugin developer agrees to protect users against claims, losses, or damages arising from intellectual property disputes.

2. Source code reference tracker - plugin provides license details for generated code and a user should make sure that source code license limitations are acceptable.

3. Users accept all liabilities - for any plugin usage users accept all the liabilities.

See my IDEs plugin comparison (“Protection from legal risk”) for details on options provided to you by plugin developers.

Plugin developers can also use your code to train their models. I added another row to my IDE plugin comparison (“Disallow to store - code / history / telemetry”) so you can opt-out if possible.

Parameters that are often disregarded but can be important are:

1. protection from legal risk

2. local codebase awareness

3. ability to disallow the collection of your code and telemetry (usage data, error reports, user interactions, and performance metrics)

The updated version with working links is in my GitHub repository - Code Generators Comparison.md.

Please, let me know if you believe additional parameters should be considered for evaluation.

Skill/Will Matrix

History

The Skill/Will Matrix is conceptually based on the Situational Leadership theory developed by Paul Hersey and Ken Blanchard in the late 1960s. Situational Leadership underscores the need for leaders to adapt their management style according to their team members' competence and commitment.

It was popularized by Max Landsberg, a British author and management consultant. He introduced this framework in his book "The Tao of Coaching" published in 1996.

The matrix

The matrix suggests that you can only delegate to people with ‘high motivation & high competence’ and you need to work hard to bring others to that state.

I added 2 things to the matrix to simplify its usage:

1. summary of actions that I believe leaders should take for the situation presented by the quadrant.

2. distribution of power during decision-making.

My experience tells me that newcomers:

• start in the ‘high motivation - low competence’ quadrant as they do not understand the culture and processes of a new company/group, or lack some skills.

• move to the ‘high motivation - high competence’ quadrant.

• tend to move to ‘low motivation - high competence’ as their understanding of tasks that they need to do in the role improves and it brings less of a challenge and the leader's task is to find that optimal balance. ‘The Goldilocks Rule’ of optimal challenge explains it well.

• Assessment of Leadership Potential: Identification of individuals who have the potential to be effective leaders based on measuring traits that are believed to be critical for successful leadership.

• Leadership Development: Identification of strengths and weaknesses to create development plans that target weaknesses as areas for improvement and strengths as a foundation to build on.

I made a review of the most popular psychometric tools used nowadays. If you just want to see the summary in a neat table, click here.

Myers-Briggs Type Indicator (MBTI)

Intended Purpose

To identify an individual's (and team members’) behavioral style and preferences to learn to communicate more effectively, resolve conflicts, and work more efficiently in teams.

History

MBTI was created by Katharine Cook Briggs and her daughter Isabel Briggs Myers. Neither Myers nor Briggs had formal education in the discipline of psychology, and both were self-taught in the field of psychometric testing.

After studying “Psychological Types” by Swiss psychiatrist Carl Jung Isabel Myers was particularly fascinated by the concept of introversion but felt the book was too complex for the general public, and therefore she tried to organize the Jungian cognitive functions to make it more accessible.

They began creating their indicator during World War II in the belief that a knowledge of personality preferences would help women entering the industrial workforce for the first time to identify the sorts of war-time jobs that would be the "most comfortable and effective" for them. The “Briggs Myers Type Indicator Handbook”, published in 1944, was re-published as "Myers–Briggs Type Indicator" in 1956.

Validity

Despite its popularity, MBTI has been widely regarded as pseudoscience by the scientific community as the indicator exhibits significant scientific (psychometric) deficiencies.

Psychometric specialist Robert Hogan wrote:

"Most personality psychologists regard the MBTI as little more than an elaborate Chinese fortune cookie...".

Researchers also confirm that MBTI has poor predictive validity of employees' job performance ratings. After all the MBTI measures natural preferences for certain ways of thinking, feeling, and behaving, rather than the actual ability to perform specific tasks or skills.

CliftonStrengths

Intended Purpose

To identify an individual's personality characteristics with a focus on strengths to put them to maximum use to achieve success and fulfillment, rather than trying to fix weaknesses.

History

CliftonStrengths (former StrengthsFinder) was developed by Don Clifton. He taught and researched educational psychology at the University of Nebraska–Lincoln from 1950 to 1969. He researched what distinguished talented people from others and identified that successful people had certain personal attributes that benefited them in their work. When Clifton left the university, he founded Selection Research Inc. (SRI) helping companies with employee selection.

Clifton's firm SRI acquired Gallup Inc. in 1988 and Clifton became chairman. Under Clifton, Gallup expanded beyond public opinion polls, entering the management consulting business. Gallup consulted companies on ways to improve their businesses by homing in on their employees' strengths.

In 1999, Clifton created the online assessment tool StrengthsFinder which focuses on 34 themes that make up a person's personality. The results of the test are intended to help individuals understand their unique strengths and how they can use them to achieve their goals and improve their performance.

In 2006, Gallup researchers completed a review of the Clifton StrengthsFinder psychometrics and revised it to bring a new version StrengthsFinder 2.0 that brings a "personalized" explanation of a person's Top talents.

Validity

While Gallup Inc. is the primary source of research on CliftonStrengths, there have been independent studies conducted as well. The results showed that people who used CliftonStrengths had higher levels of self-awareness and self-efficacy, as well as greater career decision-making self-efficacy, than those who did not use the tool. The studies also found that the benefits of using CliftonStrengths were consistent across different demographic groups.

DISC

Intended Purpose

To identify an individual's behavioral style and preferences to communicate more effectively, resolve conflicts, and build stronger relationships.

History

The groundwork for the DiSC assessment was created by Dr. William Moulton Marston, Harvard scholar, and psychologist. He published a book in 1928 called “Emotions of Normal People” where he suggested that normal emotions led to behavioral differences and presented four behavioral styles:

• Dominance (Drive): results-orientation, decisiveness, and focus on achieving goals

• Inducement (Influence): openness, sociability, enthusiasm, and enjoyment of working with others

• Submission (Steadiness): patience, cooperation, and preference for a stable and predictable environment

• Compliance (Caution, Conscientiousness): an analytical approach, detail orientation, and focus on accuracy and precision

These four styles became the building blocks for the DISC model of human behavior and later led to the modern-day DISC assessment. Since then, many researchers and practitioners have contributed to the development and refinement of the DiSC model.

Although Marston built the groundwork for DISC, it wasn’t until the 1950′s that an actual DISC personality profile test was created by Dr. Walter Clark, an industrial psychologist.

Validity

Research suggests that while DiSC assessments may be moderately valid and reliable for measuring certain aspects of behavioral style and preferences, their accuracy and usefulness may be influenced by a range of factors. In relation to other outcomes, such as leadership effectiveness and team performance results are mixed with some studies showing moderate levels of validity and others showing lower levels.

There are also several versions of DiSC assessments available now making it even harder to reliably use DiSC assessment.

Hogan assessment

Intended Purpose

To identify an individual's personality characteristics, including traits related to interpersonal interactions, work style, and values particularly as they relate to job performance.

The assessment consists of several components:

• Hogan Personality Inventory (HPI), which measures broad personality traits such as agreeableness, conscientiousness, and openness to experience

• Hogan Development Survey (HDS), which identifies potential "derailers" or negative tendencies that may impede an individual's success in a leadership role

• Motives, Values, Preferences Inventory (MVPI), which assesses an individual's underlying values, motivations, and preferences.

History

Hogan's assessment was created by Robert Hogan, an American organizational psychologist known for developing socioanalytic theory that fuses Sigmund Freud’s psychoanalytic theory and role theory with evolutionary theory. Whereas Freud believed that how we think about ourselves determines how we interact with others, role theory argues that our interactions with others determine how we think about ourselves. Both theories predict that many of us will have trouble in these areas—and we will not know why. Hogan’s socioanalytic theory concerns three questions:

1. in what meaningful ways are people alike—human universals;

2. in what meaningful ways are people different—individual differences;

3. how to explain anomalous or self-defeating behavior.

Job performance

Building on his distinction between identity and reputation, Hogan developed a socioanalytic perspective on job performance. Performance evaluation, he observed, depends on workplace interactions, after which each person will assess how rewarding the other person was to deal with. According to Hogan, rewardingness—being interesting and agreeable, meeting the other person’s expectations, satisfying their desires, and promoting their agenda—forms the basis for performance appraisal.

Expanding on the five-factor model of personality, Hogan identified seven dimensions as an initial taxonomy of personality variables needed to predict job performance, which formed the foundation of the Hogan Personality Inventory (HPI). Job performance is ultimately a function of the perceptions of others.

Validity

It has been validated over several decades, and it has been shown good predictive validity of :

• personality characteristics, including traits related to interpersonal interactions, work style, and values;

• job performance, job satisfaction, and turnover.

The assessment measures a range of personality characteristics, including traits related to interpersonal interactions, work style, and values. Studies have shown that the assessment accurately measures these traits and that the results are consistent over time.

Summary table

There are only 2 psychometric tools confirmed by research: Hogan Assessment and Cliffton Strength. They can be useful even if you test only yourself or a few team members as you will be able to better understand how to more efficiently interact with people.

As I am a believer in Gallup's approach to people management (in detail explained in Gallop’s ‘First break all the rules’ book) where the focus is set on strengths to put them to maximum use instead of trying to fix weaknesses I would recommend ClifftonStrengh.

I would appreciate it if you would let me know what worked well for your teams if you have a moment.