About
I am currently the Staff Tech Lead (ICT5) for the Vision Applications team at Apple.
PATENTS
Patent US12067909
Electronic Devices with Dynamic Brightness Ranges for Passthrough Display Content
Patent US12099653
User Interface Responses Based on Gaze-Holding Event Assessment
Patent Pending US20230418372
Gaze Behavior Detection
Patent US10890968
Electronic Device with Foveated Display and Gaze Prediction
PhD Research
Prior to working at Apple, I completed my Ph.D in Neural Science at NYU, studying in the lab of Eero Simoncelli.
In my PhD, I became obsessed with how to model the human ability to discern differences between pairs of images. This problem fascinates me because it poses interesting scientific questions, connects the structure of the human visual system to the process of perception, and opens tremendous potential engineering applications.
I've approached the problem by constructing models from known computational building blocks found in the physiology of the human visual system. Working with two former postdocs in the lab, Valero Laparra and Johannes Balle, we've used these models to predict human sensitivity to changes, or distortions, to images. We've also used these models as as a way to build a framework for optimizing the display of images on any screen or rendering device.
While studying models of this type, I created a novel method for evaluating models of human perception by forcing them to make predictions that we could test on real human subjects. I used this method to show that deep neural networks trained to perform object recognition tasks at or beyond human ability are not necessarily good models of human perception. Additionally, I showed that representations within simple models of the early human visual physiology actually outperform current state-of-the-art deep networks as perceptual loss functions. This work also suggests that building some of the computational building blocks from visual physiology into deep networks trained on object recognition may improve their ability to generalize to other visual task beyond that which they were trained for. This work was awarded an oral presentation at NIPS 2017.
My thesis work in toto was awarded the Samuel J. and Joan B. Williamson Dissertation Fellowship at NYU for the 2017-2018 academic year.
External Talks
2017 Dec NIPS 2017 Selected Oral Presentation 2017 Oct Stanford NeuroAI Lab 2017 Oct Google Machine Perception