scrollTop
TOP





        

Smart Library

Voice recognition, Accessibility
Speech-to-Text: Automatic Speech Recognition app to help students easily find what they need

Client, Year
Personal/Google Android Pixel 3, 2023

Role
UX Research/UI Design,
Mobile app development

Duration
7 Weeks (March-May)

Skills
UX Design, Visual Design, Coding

Tool
Figma, Javascript

Team
Joann Myung, Sophie Lee


Background
How can I enhance the quality of my daily life at school to align with the overall quality of my daily life by utilizing a library app? Bridging the gap between students' daily experiences and library resources.

Challenge
Designing the JavaScript code to convert audio into written text presented the most significant challenge in envisioning the future of the NYU library. This task required me to immerse myself in an unfamiliar environment, step outside my comfort zone, and explore innovative prototyping methods. These adaptations were necessary to ensure the successful execution of voice commands from the web and their seamless transfer to mobile Android device.



How might we simplify and invent user experience?





PROBLEM

The current library booking system requires too many steps.





RESEARCH

My research first centered around the competition space in order to understand how other apps are addressing similar issues. More than 30 user survey responses would help quantify what students are looking for, and what they’re hoping to resolve. And lastly, one-on-one interviews would help me understand users’ journeys and specific issues the app would need to address.



INTERVIEW FINDINGS
The interviews indicate a lack of clarity and awareness regarding the library's booking system, with one participant finding it confusing and another expressing surprise at its existence.

“ I'll take a chance and go to the library to check, as the library booking system is incredibly confusing."

“ I didn't know there was a booking system for library app."



JOURNEY MAP
GOLDEN PATH




INSIGHT

Students are looking for a simpler and more user- friendly library app that includes only the “essential features.






SOLUTION
Voice Trigger Library App

Provide a fast & precise conversion of audio into written text, while also displaying the information to students who face difficulties in using a touch screen.(e.g. due to paralysis, tremor, or temporary injury) use their Android device by voice.

Voice Access provides commands for:
  • Navigation (e.g. "I want to book a study room")
  • Controlling the current screen (e.g. "tap next", "scroll down")


EARLY DESIGN 
Option A : Barcode Scan


Option B : Voice Trigger
︎︎︎





LO-FI WIREFRAME


BEHIND THE SCENE (Speech-to-Text-API)


AS A USER
  • Voice control for your device
  • Navigate your device using sentences
  • Dictation capabilities
  • Voice Access assists individuals facing challenges in touchscreen manipulation (e.g., due to paralysis, tremors, or temporary injuries) in using their Android device through voice commands



  • FINAL WIREFRAMES + DEMOS 






    REFLECTION
    I was thrilled to embark on this project, as it allowed me to delve into the fascinating realm of speech-to-text technology. Recognizing it as an excellent opportunity to gain insights into both the backend functionality and user interface design, I eagerly embraced the challenge. Transitioning between roles as both an engineer and a designer pushed me out of my comfort zone, presenting significant challenges along the way. However, overcoming these hurdles has been rewarding, and am proud of the results we achieved.

    Moving forward, I am eager to continue exploring this intersection of technology & design, and I am enthusiastic about any opportunities to further refine and expand upon this project. I welcome any feedback you may have and appreciate your time in considering my journey throughout this process.




    jm8785@nyu.edu