Shengxin Li    李  盛忻
Inclusive Design Lab,
Logo Kyoto University

When we design for disability, we all benefit. | Elise Roy

I'm 李 盛忻 (Shengxin Li, リ    セイ キン), a Master's student at Inclusive Design Laboratory, in Social Informatics Course, Graduate School of Informatics, Kyoto University, Japan, under the supervision of Prof. Kentaro Takahira and Prof. Naomi Yamashita. Prior to that, I obtained my Bachelor of Engineering's degree at ShanghaiTech University, majoring in Computer Science while minoring in Interaction Design, where I was supervised by Prof. Quan Li.

My research interests lie in the field of Human-Computer Interaction (HCI). I aim to convert real-world problems and user needs into design solutions, and examine the effectiveness of these solutions through user studies. My current research focuses on improving commuters' experience and awareness of the surrounding environments on their daily routine.

🙌 I'm actively seeking for technical job opportunities in Japan and around the world! Please feel free to reach out to me if you have any questions or opportunities!

During my free time, I enjoy (railway) travelling and photography. I'm specifically interested in collecting visa stickers and entry/exit passport stamps. I'm also a big fan of サカナクション. I deeply resonate with their songs. Don't miss their hottest song 怪獣!

Curriculum Vitae
GitHub LinkedIn ORCID
Education
  • Kyoto University
    Kyoto University
    Master's Student in Informatics
    Oct. 2025 - Sept. 2027 (Expected)
  • ShanghaiTech University
    ShanghaiTech University
    B.Eng Student in:
    • Computer Science (Major)
    • Interaction Design (Minor)
    Sep. 2021 - Jul. 2025
Experience
  • Inclusive Design Lab
    Inclusive Design Lab
    Master's Student
    Oct. 2025 - Present
  • ViSeer Lab
    ViSeer Lab
    Undergraduate Research Intern
    May. 2023 - Jul. 2025
Skills
  • Design
    Design
    Human-Centered Design, Interaction Design & Prototyping, Graphic Design, Video Editing;
    Figma, Adobe Illustrator, Adobe PS, Adobe Pr.
  • Research
    Research
    Quantitative & Qualitative Research, User Study, Interview, Iterative Design.
  • Programming
    Programming
    Python (Basics, DS Libraries & PyTorch), Frontend (JavaScript, Vue, HTML/CSS), C/C++,
    Arduino C++, SQL, Kotlin(Android).
  • Languages
    Languages
    Mandarin Chinese (native), English (proficient, TOEFL 105/120, TOEIC 965/990),
    Japanese (intermediate, JLPT N2).
Projects (view all )
AI-Based System for Facilitating Awareness in Everyday Environments
AI-Based System for Facilitating Awareness in Everyday Environments

Shengxin Li

Lead Researcher & Full-Stack Developer · Master's Research Project · 2026 - Present

My research project proposes a multilingual web system that helps people reflect on overlooked details along familiar routes. I designed and built a staged experience from unaided recall and video-aided reseeing to AI-supported discovery and revisable co-creation using Next.js and TypeScript. Its multimodal pipeline combines OpenAI frame-level vision analysis with Gemini full-video reasoning to surface environmental and seasonal observations grounded in video evidence. I also implemented participant-isolated authentication, data storage, and interaction logging with Supabase and private Cloudflare R2 storage, and independently led a two-week longitudinal field study with 15 participants.

[Demo] [Slides (progress ongoing)]

AI-Based System for Facilitating Awareness in Everyday Environments
AI-Based System for Facilitating Awareness in Everyday Environments

Shengxin Li

Lead Researcher & Full-Stack Developer · Master's Research Project · 2026 - Present

My research project proposes a multilingual web system that helps people reflect on overlooked details along familiar routes. I designed and built a staged experience from unaided recall and video-aided reseeing to AI-supported discovery and revisable co-creation using Next.js and TypeScript. Its multimodal pipeline combines OpenAI frame-level vision analysis with Gemini full-video reasoning to surface environmental and seasonal observations grounded in video evidence. I also implemented participant-isolated authentication, data storage, and interaction logging with Supabase and private Cloudflare R2 storage, and independently led a two-week longitudinal field study with 15 participants.

[Demo] [Slides (progress ongoing)]

Go Touch Grass: AI-Assisted Low-Barrier Bilingual Community Volunteer Platform
Go Touch Grass: AI-Assisted Low-Barrier Bilingual Community Volunteer Platform

Shengxin Li*, Claudio González Alanís*, Zixi Zhu*, Wenkai Jia* (* equal contribution)

Full-Stack Developer & Project Manager · Practice Information Systems · Spring 2026

Go Touch Grass connects volunteers and community organizers through a Japanese and English platform. I built the end-to-end volunteer experience with Next.js and TypeScript, including role-based registration, opportunity discovery, task-level applications, status tracking, and pre-event communication. For organizers, I developed a bilingual guided form with Gemini-assisted drafting, using Prisma and PostgreSQL to store multi-task event listings and application states. As project manager, I led sprint planning, user-story authoring, and backlog management while coordinating reviews and contributing to debugging and testing across features.

[Slides]

Go Touch Grass: AI-Assisted Low-Barrier Bilingual Community Volunteer Platform
Go Touch Grass: AI-Assisted Low-Barrier Bilingual Community Volunteer Platform

Shengxin Li*, Claudio González Alanís*, Zixi Zhu*, Wenkai Jia* (* equal contribution)

Full-Stack Developer & Project Manager · Practice Information Systems · Spring 2026

Go Touch Grass connects volunteers and community organizers through a Japanese and English platform. I built the end-to-end volunteer experience with Next.js and TypeScript, including role-based registration, opportunity discovery, task-level applications, status tracking, and pre-event communication. For organizers, I developed a bilingual guided form with Gemini-assisted drafting, using Prisma and PostgreSQL to store multi-task event listings and application states. As project manager, I led sprint planning, user-story authoring, and backlog management while coordinating reviews and contributing to debugging and testing across features.

[Slides]

VR Japanese Daily-Life Training: Situated Japanese Practice in VRChat
VR Japanese Daily-Life Training: Situated Japanese Practice in VRChat

Shengxin Li*, Yiyan Deng*, Siyi Wang*, Zixi Zhu*, Wenkai Jia* (* equal contribution)

Developer & Project Manager · Practice Information Systems · Spring 2026

This VRChat convenience-store prototype connects Japanese conversation with an everyday shopping procedure. I implemented proximity-triggered interfaces, voice- or click-based choices, and synchronized Guided/Free practice states using Unity and UdonSharp. A Python/FastAPI pipeline with faster-Whisper advances guided dialogue from live speech, using step-specific prompts, confidence checks, duplicate suppression, and TTS-echo filtering. For open-ended practice, I integrated OpenAI structured outputs and speech synthesis to separate concise language feedback from the clerk's next reply and route Japanese speech through VRChat's microphone controls for repeated role-play.

[Slides]

VR Japanese Daily-Life Training: Situated Japanese Practice in VRChat
VR Japanese Daily-Life Training: Situated Japanese Practice in VRChat

Shengxin Li*, Yiyan Deng*, Siyi Wang*, Zixi Zhu*, Wenkai Jia* (* equal contribution)

Developer & Project Manager · Practice Information Systems · Spring 2026

This VRChat convenience-store prototype connects Japanese conversation with an everyday shopping procedure. I implemented proximity-triggered interfaces, voice- or click-based choices, and synchronized Guided/Free practice states using Unity and UdonSharp. A Python/FastAPI pipeline with faster-Whisper advances guided dialogue from live speech, using step-specific prompts, confidence checks, duplicate suppression, and TTS-echo filtering. For open-ended practice, I integrated OpenAI structured outputs and speech synthesis to separate concise language feedback from the clerk's next reply and route Japanese speech through VRChat's microphone controls for repeated role-play.

[Slides]

All projects
Selected Publications (view all )
UPinch: Enabling Unaligned Gaze-Hand Coordination forSelection in 3D Environments
UPinch: Enabling Unaligned Gaze-Hand Coordination forSelection in 3D Environments

Yuchen Wu,, Shizhen Zhang,, Shengxin Li,, Yang Ouyang,, Qian Zhu,, Quan Li

Accepted by Proceedings of the 2025 International Conference on Human-Engaged Computing (ICEHC 2025)

Interaction techniques in virtual environments (such as Mixed Reality) necessitate Gaze-Hand Alignment, which requires gaze fixation and hand selection on the same target at the same time. However, people's gaze-hand behaviour in real world is often unaligned. We proposed UPinch, a gaze-hand based selection technique that adapts the inherent gaze-hand coordination observed in human reach-to-grasp process to 3D environments. We conducted a series of cross-reality experiments comparing UPinch to Gaze + Pinch, Gaze + Handray and Reality, identifying their gaze-hand characteristics in diverse tasks.

[ACM DL]

UPinch: Enabling Unaligned Gaze-Hand Coordination forSelection in 3D Environments
UPinch: Enabling Unaligned Gaze-Hand Coordination forSelection in 3D Environments

Yuchen Wu,, Shizhen Zhang,, Shengxin Li,, Yang Ouyang,, Qian Zhu,, Quan Li

Accepted by Proceedings of the 2025 International Conference on Human-Engaged Computing (ICEHC 2025)

Interaction techniques in virtual environments (such as Mixed Reality) necessitate Gaze-Hand Alignment, which requires gaze fixation and hand selection on the same target at the same time. However, people's gaze-hand behaviour in real world is often unaligned. We proposed UPinch, a gaze-hand based selection technique that adapts the inherent gaze-hand coordination observed in human reach-to-grasp process to 3D environments. We conducted a series of cross-reality experiments comparing UPinch to Gaze + Pinch, Gaze + Handray and Reality, identifying their gaze-hand characteristics in diverse tasks.

[ACM DL]

<em>Understood</em>: Real-Time Communication Support for Adults with ADHD Using Mixed Reality
Understood: Real-Time Communication Support for Adults with ADHD Using Mixed Reality

Shizhen Zhang*, Shengxin Li*, Quan Li (* equal contribution)

Accepted by The 38th Annual ACM Symposium on User Interface Software and Technology (UIST 25)

Adults with ADHD face persistent communication difficulties stemming from executive dysfunction and emotional dysregulation, yet existing interventions mainly focus on children and lack everyday support for adults. Through formative interviews and a design workshop, we developed Understood, a Mixed Reality system on HoloLens 2, designed to assist adults with ADHD in real-world communication. Understood integrates real-time conversation summarization, context-aware word suggestions, and topic-shift detection to reduce cognitive load and disfluency. A within-subject user study and expert interviews show that Understood offers effective and usable support, serving as a valuable complement to therapist-led interventions.

[ACM DL] [Arxiv Preprint]

<em>Understood</em>: Real-Time Communication Support for Adults with ADHD Using Mixed Reality
Understood: Real-Time Communication Support for Adults with ADHD Using Mixed Reality

Shizhen Zhang*, Shengxin Li*, Quan Li (* equal contribution)

Accepted by The 38th Annual ACM Symposium on User Interface Software and Technology (UIST 25)

Adults with ADHD face persistent communication difficulties stemming from executive dysfunction and emotional dysregulation, yet existing interventions mainly focus on children and lack everyday support for adults. Through formative interviews and a design workshop, we developed Understood, a Mixed Reality system on HoloLens 2, designed to assist adults with ADHD in real-world communication. Understood integrates real-time conversation summarization, context-aware word suggestions, and topic-shift detection to reduce cognitive load and disfluency. A within-subject user study and expert interviews show that Understood offers effective and usable support, serving as a valuable complement to therapist-led interventions.

[ACM DL] [Arxiv Preprint]

<em>DancingBoard</em>: Streamlining the Creation of Motion Comics to Enhance Narratives
DancingBoard: Streamlining the Creation of Motion Comics to Enhance Narratives

Longfei Chen*, Shengxin Li*, Ziang Li, Quan Li (* equal contribution)

Accepted by The 30th Annual ACM Conference on Intelligent User Interfaces (IUI 25)

Motion Comic, a form of animation that appropriates an existing comic book into a screen-based animated narrative, proposes challenges for amateur creators as they lack proficiency with professional creation tools.
We conducted (1) a formative study to identify challenges faced by amateurs and (2) a review of the Motion Comics design space. Based on these results, we developed DancingBoard, an integrated authoring tool streamlines motion comic creation for amateur creators. We evaluated DancingBoard’s usability and the outcome’s efficiency in conveying the story through two user studies and semi-structured interviews.

[ACM DL] [Temporary]

<em>DancingBoard</em>: Streamlining the Creation of Motion Comics to Enhance Narratives
DancingBoard: Streamlining the Creation of Motion Comics to Enhance Narratives

Longfei Chen*, Shengxin Li*, Ziang Li, Quan Li (* equal contribution)

Accepted by The 30th Annual ACM Conference on Intelligent User Interfaces (IUI 25)

Motion Comic, a form of animation that appropriates an existing comic book into a screen-based animated narrative, proposes challenges for amateur creators as they lack proficiency with professional creation tools.
We conducted (1) a formative study to identify challenges faced by amateurs and (2) a review of the Motion Comics design space. Based on these results, we developed DancingBoard, an integrated authoring tool streamlines motion comic creation for amateur creators. We evaluated DancingBoard’s usability and the outcome’s efficiency in conveying the story through two user studies and semi-structured interviews.

[ACM DL] [Temporary]

<em>Trinity</em>: Synchronizing Verbal, Nonverbal, and Visual Channels to Support Academic Oral Presentation Delivery
Trinity: Synchronizing Verbal, Nonverbal, and Visual Channels to Support Academic Oral Presentation Delivery

Yuchen Wu, Shengxin Li, Shizhen Zhang, Xingbo Wang, Quan Li

Accepted by International Symposium of Chinese CHI 2024; Best Paper Award.

Academic Oral Presentation allows students to express ideas and present research findings. However, students often face the challenge of integrating verbal, nonverbal and visual elements into the presentation.
Based on a need-finding survey, a design study and an expert interview, we proposed Trinity, a hybrid delivery support system that provides guidance for multichannel delivery on-the-fly.
We conducted a controlled between-subject user study to investigate the usability, effectiveness, interaction, influence, trust and collaboration of Trinity.

[ACM DL] [Temporary]

<em>Trinity</em>: Synchronizing Verbal, Nonverbal, and Visual Channels to Support Academic Oral Presentation Delivery
Trinity: Synchronizing Verbal, Nonverbal, and Visual Channels to Support Academic Oral Presentation Delivery

Yuchen Wu, Shengxin Li, Shizhen Zhang, Xingbo Wang, Quan Li

Accepted by International Symposium of Chinese CHI 2024; Best Paper Award.

Academic Oral Presentation allows students to express ideas and present research findings. However, students often face the challenge of integrating verbal, nonverbal and visual elements into the presentation.
Based on a need-finding survey, a design study and an expert interview, we proposed Trinity, a hybrid delivery support system that provides guidance for multichannel delivery on-the-fly.
We conducted a controlled between-subject user study to investigate the usability, effectiveness, interaction, influence, trust and collaboration of Trinity.

[ACM DL] [Temporary]

All publications