What is Amazon Q Developer ?
Amazon Q Developer is a generative artificial intelligence (AI) powered conversational assistant that can help you understand, build, extend, and operate AWS applications. You can ask questions about AWS architecture, best practices, documentation, support, and more. Amazon Q is constantly updating its capabilities so your questions get the most contextually relevant and actionable answers, including the ability to connect you to a human if needed.
When used in an integrated development environment (IDE), Amazon Q additionally provides software development assistance, including code explanation, code generation, and code improvements such as debugging and optimization.
Amazon Q is powered by Amazon Bedrock, a fully managed service that makes foundation models (FMs) available through an API. The model that powers Amazon Q has been augmented with high quality AWS content to get you more complete, actionable, and referenced answers to accelerate your building on AWS.
Benefits, Risks & Challenges
Amazon Q Developer brings many potential benefits including;
- Significant increases in developer productivity
- Faster modernisation and more efficient upgrades
- Faster time to market
- Improved collaboration and developer experience
- Integration with key developer IDEs (Jetbrains, VS Code)
There are of course potential risks and concerns to be aware of and avoided
- Quality of the code and adherence to company standards
- Bottlenecks – just writing code faster will create queues elsewhere if the whole value stream is not addressed
- Understanding the impact of GenAI requires metrics and transparency
- Ethical and legal considerations
We discuss the benefits, risks and challenges that we have observed in greater detail in the blog “Leveraging GenAI:A Game-Changer in Software Engineering”
Why Do An Assisted Proof Of Concept ?
Being an early adopter of AI technologies allows organisations to gain a significant competitive advantage. However, as discussed in the previous section, while there are great benefits to be had with AWS Q Developer, like all GenAI tools there are risks and challenges associated with their adoption, especially at scale. 
Selecting an experienced and qualified partner will de-risk and accelerate the evaluation and adoption of this potentially transformative technology. As one of a small number of partners certified by AWS with their GenAI competency we have a proven track record of expertise and experience helping organisations to do just that.
Devoteam’s Assisted Proof Of Concept, aided by experienced DevOps Consultants experienced in the technology and best practices will guide you through a thorough evaluation process and provide guidance on how to adopt this powerful toolset in the optimum way.
The outcomes you can expect include;
- a clear understanding of what your organisation can achieve with Amazon Q developer, along with a roadmap for implementation
- detailed metrics pertaining to the productivity and quality gains to support the extent of the business case for further investment to adopt Amazon Q more widely
- training and support for developers in the optimum use of the technology
- guidance on the integration of the tool within the existing toolchain
- management of change and communication – reassuring, generating positive engagement and support for change
Approach
Step 1 – Identify A POC Team and a Control Group
The most successful POC’s have participants and control groups. One or more developer teams should be identified to participate in the POC. These engineers will be given Amazon Q Developer training and support from the start and throughout the POC.
A second cohort of development teams of similar size and who work on similar stories should also be identified to act as a control group. This group will not be given Amazon Q Developer but will provide a comparative set of metrics with which to compare to those teams who do.
Step 2 – Benchmark, Metrics & Survey
It is important to gather some clear metrics with which to benchmark both of the groups of developers’ current performance across a variety of metrics.
A qualitative and quantitative assessment of the current maturity of the software engineering function will first be taken along with some specific metrics such as;
- the average number of lines of code produced per developer per day
- adoption and engagement
- review efficiency
As an output of the assessment, a report will be compiled detailing the current capability and any recommendations for improvement.
An often recommended option is a simple dashboard with which to monitor developer performance on an ongoing basis.
Below is an example of such a dashboard that Devoteam have delivered for many of our customers based on DevOps Research and Assessments (DORA)
In addition to these metrics, developers would receive a short survey through which they would respond to a series of questions designed to capture how they perceive themselves. The questions will provide a level of insight into how the developers perceive their performance, confidence, quality of work, value as an employee, contribution to business goals and so on.
Step 3 – Training & Enablement
Supported by Devoteam’s experienced DevOps Consultants, the developers participating in the POC (not the control group) will be equipped with Amazon Q Developer and provided with training on its use. This training will include techniques such as Prompt Engineering and how to use the tool with confidence.
Step 4 – Proof Of Concept Begins
The POC should run for approximately 12 weeks. Throughout the POC, the developers using Amazon Q Developer will have ongoing support and guidance from a Devoteam DevOps engineer.
Step 5 – Post Proof Of Concept Benchmark & Results
At the end of the proof of concept, the qualitative survey will be repeated for both the POC and control group. In addition, the same metrics gathered as a benchmark at the beginning of the POC will be gathered.
A report will be generated providing;
- A detailed quantitative comparison of the developer teams performance both before Amazon Q Developer was introduced and after across a variety of lenses.
- A set of qualitative observations relating to the developers own perception of Amazon Q Developer contribution to their performance, confidence, experience, engagement etc.
- A detailed set of recommendations and a business case for wider adoption of Amazon Q Developer.
If you would like to discuss how Devoteam can assist your organisation to evaluate Amazon Q Developer please get in touch here.