Back

Back

Summary

Problem area

My Role

What I learned

Back

Summary

Problem area

My Role

What I learned

Gemini Workspace

2024

Evaluated Gemini AI Integration in Google Workspace for SMBs

Google needed to understand whether Gemini's integration into Workspace would deliver real value for small and medium businesses before broader rollout. Would SMB owners find it helpful enough to adopt? Which features would work best for their workflows?

Understanding if AI features deliver real value to small business owners

Problem area

Could Gemini's AI features deliver real value to small business owners before broader rollout?

Google was preparing to roll out Gemini AI across Workspace tools: Docs, Sheets, Slides, Gmail, and Drive. The team at Google needed to know if small and medium business owners would actually find it useful. Before launching broadly, they needed evidence: which features worked, which fell short, and whether SMB owners would adopt them into their everyday workflows.

When product friction became a retention problem

Problem 1

Small business owners juggle multiple tools and workflows, leaving little room for technology that adds friction instead of reducing it

Problem 1

Small business owners juggle multiple tools and workflows, leaving little room for technology that adds friction instead of reducing it

Problem 2

SMBs lack dedicated IT support, meaning complex or unreliable AI features get abandoned rather than troubleshot

Problem 3

Time is their scarcest resource if a feature doesn't work immediately, they won't invest in learning it

My role

Ensuring data quality and participant management

I supported the multi-session usability study by assisting with data collection, participant coordination and qualitative coding.

My operational role

Qualitative coding

I worked alongside senior researchers to code open-ended responses. I helped categorize feedback themes and flagged hallucinations and "prompt phrasing challenges"

Qualitative coding

I worked alongside senior researchers to code open-ended responses. I helped categorize feedback themes and flagged hallucinations and "prompt phrasing challenges"

Data Collection

After each of the 12 tasks, I collected survey responses measuring: Completeness, Understandability, Groundedness and Truthfulness.

Participant Coordination

I managed logistics for 23 participants across multiple sessions, including reminders, troubleshooting technical issues, and quality checks.

Research Approach

How the study was structured

We conducted a mixed-methods study with 23 SMB owners across 3 sessions, testing 12 Gemini tasks through completion rates, surveys, and qualitative feedback.

Demographic details

23 SMB owners across 15 states

23 SMB owners across 15 states

3 sessions testing 12 tasks total

Mixed-methods: task completion + surveys + qualitative feedback

Key Findings

Reducing errors with smarter confirmations


The Clear Winner

Email drafting was the standout performer. 83% task completion, 82% rated it "great," and 79% said they'd use it regularly. Participants valued how well Gemini captured their tone in first drafts.

The Clear Winner

Email drafting was the standout performer. 83% task completion, 82% rated it "great," and 79% said they'd use it regularly. Participants valued how well Gemini captured their tone in first drafts.

Where it fell short

Sheets formula creation was the biggest failure 57% couldn't complete the task and 57% rated it "bad," yet it was the most wanted feature with 52% saying they'd use it regularly.

Pattern across all tasks

Simple tasks hit 70-83% completion, but complex multi-file tasks dropped to 35-43%, with over half rating summarization "bad" and 28% flagging truthfulness issues.

Email Drafting

Participants loved getting strong first drafts that captured their tone.

“This email is succinct and a great starting basis for editing an email. I probably wouldn't feel good about sending 100% AI written email like this, but getting a starting template like this is super useful and saves a bunch of time. I'm impressed with how it was able to accurately describe my company as well, with little prompting.”

Jake

Female, GenX

“Gemini did a GREAT job. I would use this. I feel like I always have a million emails and I am NOT good about cleaning it up so I am missing things often.”

Claire

Female GenZ

“Impressed with this function and the wording offered in the email starter.”

Kia

Female, MIllenial

“This email is succinct and a great starting basis for editing an email. I probably wouldn't feel good about sending 100% AI written email like this, but getting a starting template like this is super useful and saves a bunch of time. I'm impressed with how it was able to accurately describe my company as well, with little prompting.”

Jake

Female, GenX

“Gemini did a GREAT job. I would use this. I feel like I always have a million emails and I am NOT good about cleaning it up so I am missing things often.”

Claire

Female GenZ

“Impressed with this function and the wording offered in the email starter.”

Kia

Female, MIllenial

Document Summarization

Bullet-point format made quick comprehension easy.

“It was OK as a quick summary. It told me what the document was about, accurately, but did not pick out the major findings or points in the document. For instance, for comparing locations, mainly data on Austin, Texas, vs San Diego, California, it could have listed some key data points from the document.”

Jake

Female, GenX

“Responded with bullet point format. That helped with reading the response summary. Good way to organize.”

Max

Female GenZ

“Although, Gemini did an amazing job, I chose occasionally just because I wouldn't want to rely on this JUST in the case that something may have been missed. I just wouldn't feel good if I didn't read the document first myself.”

Claire

Female, GenZ

“It was OK as a quick summary. It told me what the document was about, accurately, but did not pick out the major findings or points in the document. For instance, for comparing locations, mainly data on Austin, Texas, vs San Diego, California, it could have listed some key data points from the document.”

Jake

Female, GenX

“Responded with bullet point format. That helped with reading the response summary. Good way to organize.”

Max

Female GenZ

“Although, Gemini did an amazing job, I chose occasionally just because I wouldn't want to rely on this JUST in the case that something may have been missed. I just wouldn't feel good if I didn't read the document first myself.”

Claire

Female, GenZ

Sheets Formula

The feature users wanted most didn't work reliably.

“I could not get this to work. I tried 15 different times rewording my prompt and using predetermined suggested prompts.”

June

Female, GenX

“I could not figure out how to frame the request to receive a response”

Rita

Female, Millenial

“It has a lot of potential. I only know most basic Excel formulas, so if it could create more advanced ones in response to simple instructions, that would be excellent. It could save me a lot of time and frustration.”

Max

Female, GenZ

“I could not get this to work. I tried 15 different times rewording my prompt and using predetermined suggested prompts.”

June

Female, GenX

“I could not figure out how to frame the request to receive a response”

Rita

Female, Millenial

“It has a lot of potential. I only know most basic Excel formulas, so if it could create more advanced ones in response to simple instructions, that would be excellent. It could save me a lot of time and frustration.”

Max

Female, GenZ

What I learned

About research operations

Quality data requires operational rigor. Before this project, I didn't realize how much participant retention impacts research quality. When someone drops out mid-study, you lose the ability to track their experience over time. Maintaining 100% completion meant:

  • Being responsive to technical issues

  • Making sessions convenient and well-scheduled

  • Building rapport so participants stayed engaged

About data quality

Raw data is messy. Survey responses had:

  • Inconsistent formats (some wrote paragraphs, others single words)

  • Missing data that needed flagging

  • Responses that didn't match the questions asked

I developed a systematic cleaning process that balanced preserving participant voice with making data analyzable.

Qualitative and quantitative tell different stories. The numbers showed Sheets formulas had 57% failure rate. But the quotes I pulled revealed why: "I couldn't figure out how to frame the request," "I don't trust the math to be correct." Numbers said "it failed." Words said "users felt confused and lost trust."

About supporting analysis

Good quotes do heavy lifting. The research team's final presentation relied heavily on the quotes I pulled. I learned to select quotes that were:

  • Representative of common experiences

  • Specific enough to be credible

  • Concise enough to be impactful

Context matters when coding. Working with senior researchers to code responses taught me that the same words can mean different things in context. "It's fine" after a successful task means something different than "it's fine" after multiple failed attempts

What I'd do differently

Automate quality checks earlier

I caught some data issues during cleaning that could have been prevented with upfront validation rules in the survey tool.

Automate quality checks earlier

I caught some data issues during cleaning that could have been prevented with upfront validation rules in the survey tool.

Create a quote database as we go

I pulled quotes after data collection, but organizing them in real-time by theme would have saved time and caught patterns faster.

Build better participant feedback loops

Some technical issues could have been caught earlier with a quick check-in system between sessions.

Other projects

Identified Critical Gaps in YouTube Music's Automotive Experience

Check the process →

Automotive Research

Competitive Positioning

Researcher

Identified Critical Gaps in YouTube Music's Automotive Experience

Automotive Research

Competitive Positioning

Researcher

Create a free website with Framer, the website builder loved by startups, designers and agencies.