arXiv · 2611.13640
EyeQ: Exploring Corneal Reflection for Mobile Context Sensing
Abstract
What if a single face-facing camera could sense your surroundings---through your eyes? We introduce EyeQ, which explores corneal reflection as a modality for context-aware mobile interaction. Traditional vision-based context sensing relies on world-facing cameras that demand explicit aiming or dual-camera setups; EyeQ instead shows that an off-the-shelf VLM can read scene context from corneal reflections captured with a smartphone camera. To recover usable detail from mobile corneal crops, EyeQ pairs on-device iris-guided capture with server-side AI enhancement, then leverages VLMs to extract scene types, names, and objects from warped, unfamiliar eye-image sequences. In an N=10 study, we characterize how capture conditions shape reflection fidelity, evaluate what VLMs detect, misclassify, and hallucinate, and demonstrate interaction opportunities, including detecting a steering wheel to distinguish drivers from passengers. This work reframes the eye in HCI---from a target for gaze estimation to a passive mirror of one's surroundings---and examines the privacy tensions it raises.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Jaewook Lee, Judith Amores, Donghoon Shin, Bo-Eun Choi, Jason Kim, Davin Win Kyi, Seok-Young Kim, Jon E. Froehlich, Andrew D Wilson. 2026-11-12. EyeQ: Exploring Corneal Reflection for Mobile Context Sensing. https://arxiv.science/abs/2611.13640
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.