Written by Matt Van Itallie, Sema Software

As the leader of a software company focused on making tech explainable to non-technical audiences, like investors and non-technical CEOs, I frequently remind them that code is a craft, not a competition. I even wrote about this in 2022.

Over the last ten months, we have been researching and working on ways to help companies adopt GenAI tools in the SDLC (like GitHub Copilot, CHatGPT, etc.) the right way. We’ve talked with hundreds of technology leaders and developers to try to get this right.

Since GenAI is on everyone’s minds these days, here are four points that could be helpful in your journey.

1. Using GenAI tools in the SDLC is a win-win for developers and organisations

No doubt your CEOs have been asking about the possible productivity gains for devs achievable through GenAI… and perhaps even being a bit too enthusiastic…

We have seen a range of productivity impacts reported from 55% to negative impact, i.e. the tools make developers less effective.

Our research—including a meta-analysis of all of the other studies, to be published shortly as a Working Paper—suggests that, if implemented correctly (a big if!), GenAI tools can deliver at least a 10%  productivity improvement.


This is a blend of GenAI tools’ impacts on different SDLC stages: GenAI can decrease the time it takes to build prototypes by 90% and has a similar impact on writing test cases.

However, the productivity impact is much lower for core engineering challenges that require deep thought, experience, and context.

Nonetheless, even a 10% blended rate is quite impactful—helping shops deliver products to users 10% faster is a big advantage for organisations.

All of the above concerns the potential benefits for organisations from GenAI coding tools. Our research also indicates a big positive impact for developers. These tools can automate repetitive or less satisfying tasks like documentation or understanding spaghetti code.

2. The risks of using GenAI tools for coding can be mitigated with developers’ active involvement

If CEOs are asking about productivity gains, then no doubt many of your Counsel’s offices are asking about the risks.

Our research indicates that GenAI is fundamentally safe to use in almost all situations, again, if implemented correctly. This implementation requires significant developer involvement.

First, it is crucial for developers to have access to enterprise-grade GenAI tools to prevent data leakage.

Second, developers must actively review the GenAI results for accuracy, maintainability, and security. GenAI coding tools work like tireless interns—they do produce results, but they must be checked via code reviews, the quality and security toolchains, etc.

We say that good GenAI usage leads to Blended GenAI code, i.e. where developers have modified the results. Too much Pure or unmodified, GenAI code is a red flag.

Third, while there are some risks related to the code’s Intellectual Property protection, these risks are mitigable. We’ve put together a series of Working Papers to help guide your legal teams on this.

3. Teams need to take the right way to implement GenAI tools seriously

Just like any tool, it is not enough to give developers access to GenAI tools and voila! expect significant and correct adoption.

Instead, we highly recommend a structured approach to helping developers get the most out of the tools.

One reason is that many developers have an ‘identity question’ about using GenAI: ‘Am I really a developer if I use GenAI tools?’

My answer to that is simple: ‘Do you think real developers use Open Source?’

In many ways, GenAI code and Open-Source code are similar—code that the developer didn’t write on their own, and it helps them significantly, but it also comes with risks that should be managed. Developers use the best tools available so they can focus on what they do best in the software development process.

To help developers overcome concerns about adoption, we recommend setting up a developer council, having anonymous forums to share feedback, and collecting and sharing usage metrics—no secrets about team-level AI adoption, please.

4. GenAI composition in the code will be a part of future diligence

For CTO Craft Community members who work at companies backed by investors, you know that technical due diligence (TDD) is in your future. TDDs look at the quality, security, risks and consistency of the codebase, as well as how development activity has changed over time.

Historically, TDDs have included Open-Source composition analysis to ensure that the Open-Source code passes legal and security standards.

What is coming—and in fact, is already here for major software investors’ TDD—is a GenAI composition analysis.

Investors are looking for a ‘just right’ GenAI usage: too little GenAI usage, and the company may not be taking advantage of the modern tech stack. Too much GenAI—specifically, too much Pure rather than Blended GenAI—and the company’s Intellectual Property may not be defensible.

For investor-backed tech companies, the best way to avoid this risk is to start measuring AI usage now and ensure that the organisation’s use is within the desired ranges.

Further resources

We’ve put together a series of resources and tools to help on this journey.

Our blog has many articles, including Working Papers on the ROI and risks of GenAI in the SDLC, a comparison of tiers of GenAI tools, and a guide to effective implementation.

CTO Craft Community members have access to a free proof of concept for a dashboard to track GenAI usage (for example, Pure vs. Blended, by product, compared to company standards). You can read the release notes here.

***

We’re proud to announce the launch of CTO Craft Con: London 2024 at the prestigious QEII venue. Grab your tickets now.

Join now to become a member of the free CTO Craft Community, where you’ll get exclusive access to Slack channels, conference insights and other valuable content. Subscribe to Tech Manager Weekly for a free weekly dose of tech culture, hiring, development, process and more.