Following on from part 1 of our series on Enterprise Knowledge and GenAI, in this part 2 we look at strategies and best practices for getting an Enterprise Knowledge AI assistant up and running.
Starting out
Amazon Q Business offers an exceptionally user-friendly way to integrate an AI chatbot into your Enterprise Knowledge Strategy. This platform boasts a vast array of connectors, seamlessly aggregating data from various enterprise applications into a single, cohesive interface.
As recognized AWS GenAI partners, we’ve had the unique opportunity to extensively work with Q Business. We are excited to share valuable insights and lessons learned from our experience in developing a comprehensive Enterprise Knowledge Solution using this innovative tool.
Below are our key findings and some good practice we developed when building our first Q Business application.
Key Strategies and Best Practices
1. Define clear personas
Start by defining clear personas and specific use cases to build something of tangible value. While it’s appealing to explore the capabilities of Q Business and experiment with the art of the possible, the key to ensuring user adoption is to create solutions that people genuinely want to use. For instance, using Q’s plugins to create tickets or incidents is impressive, but it may not deliver the most significant value or address your most pressing pain points.
Focus on these critical areas to build trust that your Q solution can be a valuable business asset, then gradually add more functionality. For example, in our organisation, we identified knowledge management as a common challenge. We tailored our initial application to consult our internal file storage system, enabling it to quickly retrieve information and reusable assets. This functionality significantly reduces the time spent searching for files and consulting team members for file locations, streamlining our workflows and enhancing productivity.
An example Persona we created for our Proof of Concept
2. Create user stories
As you define personas, it’s crucial to identify the specific data sources they will need to effectively perform their roles within the designated use cases. Focus on integrating these essential data sources first, as they are directly relevant to your users’ workflows.
While it might be tempting to experiment by adding additional data sources and functionalities through plugins, be cautious. These can divert attention from the core use cases—those user journeys that, when executed well, provide your business with a tool that not only simplifies workflows but also fosters trust in the solution you’re developing.
By focusing on delivering a robust core experience first, you lay a solid foundation for trust and utility. This approach not only enhances the initial acceptance of your Q solution but also sets the stage for successful adoption of more advanced features and enhancements in the future.
Some example user stories we created as part of our Proof of Concept
3. Architect what you want to build
Use diagrams to visually explain what you are building. This helps in conveying complex information more effectively and aligns stakeholders with the project vision.
Below is an example diagram used in our Proof of Concept:
An edited view of our Amazon Q Business architecture
4. Choose your data sources
- Understand your Enterprise Knowledge data sources, starting with the key data sources. In our case we focused on Google Drive, Confluence and other shared literature in our S3 buckets. Additional data sources can be incorporated after testing and making sure these key data sources are returning useful responses to your prompts.
- Obtain access to the data sources – a non-trivial task requiring credentials with elevated privileges to ingest all the data. System owners are understandably cautious about granting access due to security concerns, and so deciding what data you want to ingest and for what purpose is key before approaching these individuals to produce a solid business case.
- Determine the volume of data to assess cost implications. Consider limiting the types of data you ingest.
- Identify which data sources are necessary for particular personas, conduct some user research and ask your people how they’d use a AI assistant dedicated to your organisation and which tools they typically use to complete their job. This may involve assessing whether a single app suffices or if multiple applications are needed.
Our Data Sources in Amazon Q Business
5. Ingest the data
Ensure that the data and files you plan to incorporate into your Q solution are well-organized and clean beforehand. The effectiveness of your Q implementation hinges on the quality of the data it processes. Implementing a robust data management policy is vital for maximising the benefits and managing costs of any AI investment. (More on this in part 3.)
6. Test data access and security
Conduct regular tests to ensure that Q Business provides reliable and secure outputs. This includes:
- testing different user roles
- testing different data sources
- ensuring relevance tuning is accurate (is the answer coming from the data source you expect?)
Amazon’s ACLs manage access control, but it’s essential to verify that these controls are properly configured and effective.
7. Keep an eye on costs
Monitor data ingestion closely to understand the associated costs. Costs can quickly escalate as you integrate more data sources and synchronise them, which is why we advocate starting with a few straightforward use cases and personas. This initial phase allows you to monitor how much data you are ingesting and understand the associated costs.
We have found it crucial to set up cost alerting thresholds and maintain visualizations of our Q-related expenditures. These measures are key in managing costs effectively while experimenting with Q. Additionally, we recommend initially avoiding the ingestion of very large file types until you have a more established handle on the system’s operation and cost implications. This cautious approach helps in keeping costs manageable as you scale up your use of Q.
Conclusion: A Call to Action
Q Business offers an exciting opportunity to leverage Enterprise Knowledge for a significant productivity boost. However, implementing a complete solution is a different challenge altogether.
At Devoteam, we’ve experimented with Q Business and identified potential pitfalls in Q application development, so you don’t have to. We’re here to guide you through your AI integration journey and are eager to discuss how we can tailor a solution to meet your specific needs.
Feedback
We welcome your thoughts and questions on this topic. Please feel free to reach out via graham.zabel@devoteam.com or adam.james@devoteam.com.