A.I Copilot Ideation

Creating use cases for Teradata's AI powered copilot within a SQL editor.
Client: Teradata
Year: Summer 2024
Role: UX Design Intern

The Project

I had a summer internship with Teradata, a software company specializing in data analytics. VantageCloud, Teradata's cloud-based analytics software provides users with a new way of working with existing data by navigating within a console and SQL editor. I was tasked to come up with ideas and use cases for an AI copilot that would assist users in creating SQL code and working within VantageCloud's console.

The Challenge

While SQL is often used by data scientists, data analysis and other technically proficient users to manipulate data, not every user has knowledge of SQL. There are professionals in non-technical roles that want to gather insights without having to refer to someone who is more experienced. Teradata users find that the time it takes new users to adapt to VantageCloud is too long and the learning process is very complicated compared to similar products. An opportunity to implement an AI copilot has emerged, especially with the early adoption of AI in the data analytics and warehousing space. Along with the challenges of having to work virtually for the first time on an independent UX project, I also faced the challenge of finding ways to tackle the aforementioned issues while figuring out the ins and outs of data analytics and AI.

Getting Familiar with AI Copilots

A big part of my project involved getting comfortable with competitor products for a competitive analysis. I analyzed the capabilities of Snowflake, IBM Cognos Analytics, Google Bigquery with Gemini AI, Github Copilot and Amazon Redshift. Their AI copilots as well as their companies had many strengths and weaknesses. I had to use each product's AI copilot along with their SQL editor, which ended up being a very technical task as someone who knows no SQL. However, this would perfectly put me in the shoes of, let's say a CFO, who wanted to analyze data but didn't know the technical aspects of coding.

I spent two weeks testing out each competitor's AI copilot and its ability to complete tasks in the SQL editor. I was also tasked with researching how some copilots use Intelligent Search in their functionality. These are some notes I took for each AI copilot.

Google Bigquery+ Gemini AI

  • I was able to get around the console easily and even if there were technical issues, a lot of that was my general unfamiliarity with the UI.
  • It got my queries wrong a lot of times, but when they were right, it was because it used placeholder variables. The AI didn’t make up variables and it gave me the chance to input my own information.
  • I could see how it could improve code quality. I created a SQL query using ChatGPT, it took the code snippet and suggested I add a new column and a WITH clause to reduce the time calculating functions and to make my code more readable and efficient.
  • Intelligent Search: They use something called profile scan data to ground (base information on an established dataset) queries generated using data insights. This data provides essential information about the dataset, including details about data distribution, data types, and statistical summaries.

Amazon Redshift

  • I really struggled with using this. I didn’t mind doing a little work to use this but it was so cumbersome to load data onto the database.
  • The actual copilot was hard to find. I expected to see it when I clicked ‘query data’, but I had to look around for a bit.
  • It can simplify and regenerate code to make it more efficient. It’ll still suffer from the inconsistent query results but it works well enough.
  • Intelligent Search: Amazon Q Generative SQL simplifies queries by analyzing user intent, query patters, and scheme metadata. Natural Language Processing algorithms parse user queries to extract key information and combines it with the schema metadata to create SQL code.

Github Copilot

  • I watched a 5 minute tutorial that covered everything I needed to know. I’ve only spent maybe 30 minutes at most downloading VS Code and the Github Copilot at most.
  • I asked it a basic question and I got my answer back quickly in a way I can understand. I thought I had to code to get insights, but the copilot chat helps me, someone who has a surface level understanding of coding at best gain valuable insights.
  • It’s super easy to access from the side panel. Users have the ability to attach documents and the AI can fix, explain code and offer alternative suggestions.

Snowflake

  • This is actually the easiest ai copilot I’ve tested. I was able to get two successful queries with correct SQL code that read directly from the file I uploaded. It was a hell of a lot less complicated than the dumpster fire I had to deal with from AWS Redshift.
  • I can type in a prompt and know if I'm going to get a meaningful result. There’s an indicator at the bottom of a prompt response saying if the prompt is able to be executed or not. This gives me the ability to edit my prompt or edit the code given to make sure I have a valid query.
  • It still gave me SQL code to work with even if some queries wouldn’t execute. I feel like query optimization quality depends on how you write the prompt. I put in some really simple queries that could work with one dataset and I got good results and the queries were fast.
  • Intelligent Search: Snowflake partnered with Neeva to create Universal Search, and it allows users to search through documentation, valuable datasets in their marketplace, and locate tables and views stored in instances.

IBM Cognos Analytics

  • I didn't need to think about the numbers. The AI tells users the top insights along with a graph so users get text and visual feedback.
  • This is more of a no code solution, which I’m including because this caters towards business analysts who might also use Teradata’s product. I like that it can read natural language and SQL code, allowing users to choose their preferred way of querying.
  • It’s strong point is in business analytics. It’s a very practical and simple ai copilot that can create dashboards and visualizations easily within the chat.

Competitve Analysis Summary

In order to summarize my research, I created an Extended SWOT (Strengths, Weaknesses, Opportunities, Threats + Primary Users) Analysis. I tried to reference my own experiences as well as existing user reviews on sites like Trustpilot. Since I am not a regular user of these products, it was important to gain perspective into who the typical users are and what issues they come across when using competitor products.

View Extended SWOT Analysis

Subject Matter Expert Interviews

At this point, I had spent a month learning about VantageCloud Lake, it's competitors, and what technology powers the capabilities of AI copilots, particularly Intelligent Search. I had a lot of knowledge under my belt but I wanted to bridge that gap between research and creating ideas for the AI copilot. I consulted a mentor about this and they suggested I looked at the product from the perspective of the users, such as data analysts, data scientists, data engineers, database administrators and so forth. I instantly became inspired from that meeting and decided to quickly gather people who worked in the field of data analytics to ask them about their thoughts on AI technologies and their use of them in their workflow. I interviewed two data scientists to add further insights to my research.

A large part of my project included figuring how to utilize Intelligent Search technology within the AI copilot to improve the productivity of users. Intelligent Search is an AI powered technology that goes beyond keyword matching and uses the context clues in a search query in order to find relevant results as quickly as possible. This is done by using semantic mapping, where the AI is trained to understand the meaning behind words and includes looking at certain words to decipher the underlying connotation of sentences. Using intelligent search results in more accurate and more meaningful, contextually accurate answers. Two of the data scientists I interviewed stated that Intelligent search helps them in their tasks and allows them to save time.

Use of SQL and AI copilots

Since I mainly focused on the SQL editor of VantageCloud's Console, I interviewed the two data scientists about their use of SQL code within their daily tasks. Both said that they regularly use SQL as well as Python to manage and manipulate customer data so that they can look for overarching patterns to use in projects. They also experimented with AI copilots, one used the trial version of Github Copilot and the other regularly used ChatGPT to do more general tasks. They also state that integrating an AI copilot into a SQL editor can help them improve their code as well as help them with other tasks too. For example, one of the interviewees works on a project where they need to go through a large amount of papers. He expressed his enthusiasm for having a copilot that could summarize texts without breaking his workflow within the console.

I asked these questions during the interview:

  • Can you describe some of the most common projects or tasks you work on as a data scientist?
  • What are your opinions and experiences with existing AI copilots that translate natural languals to SQL?
  • What is your definition of "intelligent search" and what are your expectations of an "intelligent search" or "cognitive search" feature?
  • Would implementing an intelligent search feature be helpful for you? Explain why.
  • In what other areas of the VantageCloud Console, besides the SQL editor, do you envision a copilot could assist you?
  • What is your opinion on querying data directly within a conversational bot and recieving answers, instead of seeing the SQL code?
View Interview Notes

Journey Map and Persona

In order to help extract potential use cases for Teradata's AI copilot, I had to further synthesize my research findings by creating a journey map and a persona (example user) to design for. This gives me a stable foundation built on research to create meaningful solutions to potential frustrations.

Journey Map

For the journey map below, I put myself in the typical data scientist's shoes, since I learned about their perspective the most. I also used my previous experience of trying out each SQL editor on my own to inform what the three overarching tasks a potential user would complete.

Persona

I created a fictional scenario where Samantha Sanders, a new data scientist, has to present a report to her company's IT department. The company she works for sells smart home devices and wants to find a way to raise efficiency while lowering the potential for device errors. Samantha needs to complete three major actions in order to finish the report: Access her company's database for smart devices, use SQL to query the database and analyze any patterns found, and export relevant tables/charts.

She expects to create and access the database easily, use the SQL editor to explore, optimize, and analyze data, and export relevant tables and charts to include in her report. I will show off ways the copilot could possibly be used by following Samantha's workflow with creating the report

Ideation and Mockups

Tasks

To create idea design solutions, I had to be empathetic and put myself into the typical user's shoes. My persona, Samantha, is taking on that role. She needs to complete three major actions in order to successfully finish the report for the IT department.

  1. Explore the console and create an environment, this will let her access her company's database. Learning how the console works will help her get started.
  2. Use SQL queries to find patterns in the data and make sense of the information gathered. This will be the biggest task, but also the most complex because she has a lot of data to sort through.
  3. Create charts and graphs that take the complex data patterns she found and condense them into easy to read visuals. Having well made visuals helps ensure smooth communication between teams.

Context-Aware Console Expert

Samantha is a newer user of VantageCloud and she depends on the copilot to get around. The copilot is by her side, offering her contextually relevant topics for her to start chats with depending on what page she's on. This makes the copilot aware of the context of her tasks and helps her focus on what she really needs. Just like the console, it changes depending on the role and need of the user. Samantha can ask the AI for help with creating environments other analytica functions.

Helpful Tips within the Copilot

Samantha wants to know how she can be more productive within the console and how the copilot can help her with additional tasks. Whenever a user sends out multiple queries to the copilot, the copilot can provide tips and suggestions on what to ask it based on the last few messages the user sent.

For example, a user clicks on the copilot icon on the lower right, contextually relevant topics pop up on the menu and once the chat activates, the user asks the AI about configuring alerts for storage consumption. The copilot gives them a helpful tip for how to configure alerts, and if they are still confused, the copilot can guide the user by taking them to the relevant page to give a series of short video tutorials helping them step by step at their own pace. Samantha can use this feature anytime on any important page in the console.

Editing Code

Her second task involves using SQL to create queries. Samantha is in the middle of writing SQL queries in the editor to explore the database her company has and wants extra help with improving code. The copilot works both in the chat and the editor to regenerate, optimize, and explain code snippets. This allows her to work in tandem with the copilot.

The copilot is enabled to do multiple functions within the editor including regenerating, optimizing, and explaining code. When she right-clicks, Samantha is presented with a list of these options and when selected, the copilot generates an answer in the chat and changes the highlighted code within the editor.

Intelligent Search

Intelligent search goes beyond keyword matching. It understands the meaning behind your words and tries to find the most relevant information for you.

Samantha has over 30 tables she needs to go through, all of them containing at least 20 columns of data. She doesn’t have time to sort through everything. She can ask the copilot to search for tables related to a certain topic and copilot will use its intelligent search technology to find and highlight the columns needed. She asks the AI to find relevant tables related to customer satisfaction and the copilot returns a list of tables while also highlighting the corresponding tables in the object browser.

Attaching files

Samantha wants to summarize and analyze some documents related to her tasks but doesn’t want to waste time going to another copilot to do so. Teradata’s copilot allows her to attach documents that can be read, summarized, and analyzed with in the same chat. She enjoys the extra convenience and the extra time she saves.

Samantha uploads her local files, these files can be word documents, spreadsheets, PDFs, etc. and are only stored in the current chat. In the bottom video, Samantha types in "Give me the results of the customer satisfaction survey." and "Which pricing strategy is expected to maximize profits?" and the copilot responds with an answer gathered from analyzing the documents.

Data Visualization Assistance

Samantha is almost done with her report and wants to have visuals along with it but isn’t sure how to best represent her data. The copilot gives her advice on the optimal way to visually show her data. It also helps Samantha create visualizations by giving her previews of charts within the chat.

In the chat, she asks the copilot which type of chart is best for representing her data and it suggests using a pie chart. She also wants to know where the ideal location of the chart labels should be. The copilot gives her a tip saying it can provide a preview of what a chart would look like if a property was modified. She requests to see a version of the chart with the labels on the inside of the pie chart and a preview thumbnail is shown. This helps her quickly iterate on which styles best convey her findings.

Reflection

This was my first experience with employing UX strategies in the real world. I'm happy to know that my ideas could potentially be included in a tangible product and the work I did was appreciated by the UX team of Teradata. I believe that I've grown as a designer from this experience. I feel like I did a lot of things right, like consulting with my mentors when I couldn't figure out how to conceptualize an idea or I wasn't sure about the direction I needed to take. I also felt like I had some more emotional challenges to deal with, mainly imposter syndrome. I knew I was a capable student based on my grades, but I wasn't sure how well I'd do on a real solo UX project, surrounded by career professionals in a virtual environment. These factors made my internship difficult at first, but I felt my confidence as a designer grow towards the end. Overall, believe I delivered quality results and this project was completed successfully with the help of the talented professionals around me.