About
I am a postdoctoral researcher at Tokyo College, The University of Tokyo, working with Takeo Igarashi. I am also a visiting researcher at Google and a visiting researcher at RIKEN AIP, working with Tatsuya Harada. I received my Ph.D. at UTokyo, supervised by Koji Yatani, and my B.S. from Shanghai Jiao Tong University.
I conduct Human-Computer Interaction (HCI) research. I am interested in utilizing foundational models to create new interactive experiences. During my Ph.D. program, I specialized in Interactive Machine Teaching, developing interactive systems that enhance user experience when prototyping AI models. My past research is related to visual programming, low-level CV, LLMs, pose/gesture detection, and participatory system design.
Be free to email me if you want to collaborate or discuss with me about those research!
News
- Mar. 2025 InstructPipe received an Honorable Mention Award at CHI 2025.
- Jan. 2025 Two papers are conditionally accepted to CHI 2025.
- Dec. 2024 I joined UTokyo as a postdoc, working with Takeo Igarashi.
- Oct. 2024 DIPA 2 received the IMWUT Distinguished Paper Award.
- May 2024 I started my position as a visiting researcher at Google.
- Apr. 2024 I started my postdoc at RIKEN AIP, working with Tatsuya Harada.
- Mar. 2024 I gave a talk at Google, host by Ruofei Du.
- Mar. 2024 I received my Ph.D. from UTokyo.
- Jan. 2024 I will serve as a poster presentation chair at CHI 2025.
Publications
Papers
InstructPipe: Generating Visual Blocks Pipelines with Human Instructions and LLMs
Vision-Based Multimodal Interfaces: A Survey and Taxonomy for Enhanced Context-Aware System Design
DIPA2: An Image Dataset with Cross-cultural Privacy Perception Annotations
SoundTraveller: Exploring Abstraction and Entanglement in Timbre Creation Interfaces for Synthesizers
Gesture-aware Interactive Machine Teaching with In-situ Object Annotations
Bringing Rolling Shutter Images Alive with Dual Reversed Distortion
SyncUp: Vision-based Practice Support for Synchronized Dancing
An Image-based Approach for Defect Detection on Decorative Sheets
Workshop Papers (posters, demos, etc.)
Experiencing InstructPipe: Building Multi-modal AI Pipelines via Prompting LLMs and Visual Programming
Experiencing Rapid Visual Programming in Visual Blocks for ML
DIPA: An Image Dataset with Cross-cultural Privacy Concern Annotations
Exploiting and Guiding User Interaction in Interactive Machine Teaching
Enhancing Model Assessment in Vision-based Interactive Machine Teaching through Real-time Saliency Map Visualization
Vision-based Scene Analysis toward Dangerous Cycling Behavior Detection Using Smartphones
