Amazon Q Developer – Assisted Proof Of Concept

What is Amazon Q Developer? 

Amazon Q Developer is a generative artificial intelligence (AI) powered conversational assistant. It can help you understand, build, extend, and operate AWS applications. You can ask questions about AWS architecture, best practices, documentation, support, and more. Amazon Q is constantly updating its capabilities so your questions get the most contextually relevant and actionable answers. This includes the ability to connect you to a human if needed.

When used in an integrated development environment (IDE), Amazon Q additionally provides software development assistance. This includes code explanation, code generation, and code improvements such as debugging and optimisation.

Amazon Q is powered by Amazon Bedrock, a fully managed service that makes foundation models (FMs) available through an API. AWS augmented the model that powers Amazon Q with high quality AWS content to get you more complete, actionable, and referenced answers. This will help you to accelerate your building on AWS.

Benefits, Risks & Challenges of Amazon Q Developer

Amazon Q Developer brings many potential benefits including;

• Significant increases in developer productivity
• Faster modernisation and more efficient upgrades 
• Faster time to market 
• Improved collaboration and developer experience
• Integration with key developer IDEs (Jetbrains, VS Code)

There are of course potential risks and concerns to be aware of and avoided

• Quality of the code and adherence to company standards
• Bottlenecks  – just writing code faster will create queues elsewhere if you don’t address the whole value stream
• Understanding the impact of GenAI requires metrics and transparency
• Ethical and legal considerations 

Curious to se the benefits, risks and challenges that we have observed in greater detail? We discuss them in the blog “Leveraging GenAI:A Game-Changer in Software Engineering”

Why Do An Assisted Proof Of Concept?

Being an early adopter of AI technologies allows organisations to gain a significant competitive advantage. However, as discussed in the previous section, while there are great benefits linked to AWS Q Developer, there are also risks and challenges. Like all GenAI tools these risks are mainly the adoption, especially at scale.

Selecting an experienced and qualified partner will de-risk and accelerate the evaluation and adoption of this potentially transformative technology. As one of a small number of partners certified by AWS with their GenAI competency we have a proven track record of expertise and experience helping organisations to do just that.

Devoteam’s Assisted Proof Of Concept, aided by experienced DevOps Consultants experienced in the technology and best practices will guide you through a thorough evaluation process and provide guidance on how to adopt this powerful toolset in the optimum way.

The outcomes you can expect include;

• a clear understanding of what your organisation can achieve with Amazon Q developer, along with a roadmap for implementation
• detailed metrics pertaining to the productivity and quality gains to support the extent of the business case for further investment to adopt Amazon Q more widely
• training and support for developers in the optimum use of the technology 
• guidance on the integration of the tool within the existing toolchain
• management of change and communication – reassuring, generating positive engagement and support for change

Approach to build an Amazon Q Developer – Assisted Proof Of Concept

Step 1 – Identify A POC Team and a Control Group

The most successful POC’s have participants and control groups. Idenitify one or more developer teams to participate in the POC. Give these engineers Amazon Q Developer training and support from the start and throughout the POC. 

Then identify a second cohort of development teams of similar size and who work on similar stories. They can act as a control group. This group will not be given Amazon Q Developer but will provide a comparative set of metrics with which to compare to those teams who do.

Step 2 – Benchmark, Metrics & Survey

It is important to gather some clear metrics with which to benchmark both of the groups of developers’ current performance across a variety of metrics.

A qualitative and quantitative assessment of the current maturity of the software engineering function will first be taken along with some specific metrics such as;

• the average number of lines of code produced per developer per day
• adoption and engagement 
• review efficiency  

As an output of the assessment, a report will be compiled detailing the current capability and any recommendations for improvement.

An often recommended option is a simple dashboard with which to monitor developer performance on an ongoing basis.

Below is an example of such a dashboard that Devoteam have delivered for many of our customers based on DevOps Research and Assessments (DORA)

In addition to these metrics, developers would receive a short survey through which they would respond to a series of questions designed to capture how they perceive themselves. The questions will provide a level of insight into how the developers perceive their performance, confidence, quality of work, value as an employee, contribution to business goals and so on.

Step 3  – Training & Enablement

Supported by Devoteam’s experienced DevOps Consultants, the developers participating in the POC (not the control group) will be equipped with Amazon Q Developer and provided with training on its use. This training will include techniques such as Prompt Engineering and how to use the tool with confidence. 

Step 4 – Proof Of Concept Begins

The POC should run for approximately 12 weeks. Throughout the POC, the developers using Amazon Q Developer will have ongoing support and guidance from a Devoteam DevOps engineer.

Step 5 – Post Proof Of Concept Benchmark & Results

At the end of the proof of concept, the qualitative survey will be repeated for both the POC and control group. In addition, the same metrics gathered as a benchmark at the beginning of the POC will be gathered.

A report will be generated providing;

• A detailed quantitative comparison of the developer teams performance both before Amazon Q Developer was introduced and after across a variety of lenses.
• A set of qualitative observations relating to the developers own perception of Amazon Q Developer contribution to their performance, confidence, experience, engagement etc.
• A detailed set of recommendations and a business case for wider adoption of Amazon Q Developer.

If you would like to discuss how Devoteam can assist your organisation to evaluate Amazon Q Developer please get in touch here.