[Access Lunch] Monday Dec 4: Oscar Romero Lopez talk

[Aaron Steinfeld]

Hi everyone,

Oscar will be presenting some work from our lab on Monday:

"Independent Navigation and Interaction for Visually Impaired Users via
AR, Computer Vision, and LLMs"

Abstract

Navigating unfamiliar indoor environments and interacting with objects are
challenging for people who are blind or low-vision (pBLV). In this talk, I
describe a functional prototype for smartphone-based, indoor navigation via
a conversational assistant for pBLV. The prototype utilizes a cognitive
architecture to integrate three different technologies: real-time
object/people detection for obstacle avoidance

and discovery; augmented-reality spatial anchors for high-precision
localization and virtual content interaction; and large language models
(LLMs) for information extraction, conversational interaction, and
turn-by-turn navigation. We assessed the individual and collective impact
of different technologies through a human performance analysis where task
performance time for a user is measured. We conclude that conversation
helps holistically integrate disparate technologies to deliver a better,
more context-aware user experience while remarkably reducing task
completion time.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/accessibility-lunch/attachments/20231201/f84408e7/attachment.html>