VR · Robotics · Embodied AI

Building immersive systems for human perception and robot learning.

I design and build VR, eye-tracking, and robotics-simulation systems that explore how humans perceive virtual worlds — and how human behavior can teach embodied AI.

VR perception Eye tracking Humanoid teleoperation Real-time rendering Embodied AI

Move your cursor or touch the visual to simulate gaze-aware rendering.

Research signature

Perception-aware immersive intelligence.

My work connects perceptual science, real-time graphics, and robot learning: building immersive interfaces that sense attention, adapt visual information, and turn human behavior into training signal for embodied AI.

01

Perception

Controlled VR studies of distance judgment, attention, and embodied calibration.

02

Immersive Systems

Unity, shaders, eye tracking, and real-time interaction pipelines for VR/AR prototypes.

03

Robot Learning

VR teleoperation and gaze-labeled data collection for humanoid robots in Isaac Sim, feeding world-model and imitation-learning pipelines.

Selected impact

Research depth backed by practical engineering outcomes.

Ph.D. research
130+

VR study participants

Designed and managed controlled distance-perception experiments with high-frequency HMD tracking.

Perception result
80% to 97%+

Distance-judgment accuracy

Quantified how pre-experiment walking can substantially improve VR distance perception accuracy.

Embodied AI
100 Hz

Gaze-labeled robot datasets

Built a VR teleoperation lab for a Unitree H1 humanoid in Isaac Sim that records synchronized multimodal sessions with zero manual annotation.

HCI algorithms
70 Hz

Behavioral data analysis

Built reusable Python modules for path integration, turning-point detection, and step detection.

Visteon automation
3-4 daysto <5 min

Workflow reduction

Built V-PEDAT, a Python desktop tool that automated parsing, plotting, and report generation.

Product validation
26,500+

Plots automated

Replaced manual analysis of large test datasets with repeatable, analysis-ready outputs.

Applied AI
60,000+

Training images prepared

Created reproducible ML data pipelines and Dockerized environments for depth-estimation research.

Teaching scale
150-200

Students per semester

Teach and coordinate large-enrollment undergraduate CS courses across multiple sections.

CS curriculum
7

Courses taught

Cover introductory programming, data structures, C, discrete math, software engineering, theory, and languages.

About

I work at the intersection of perceptual science and real-time rendering. My background is in VR distance perception, controlled experiments, and the software pipelines required to run studies reliably at interactive frame rates.

Recently, I have been extending this foundation in two directions: gaze-aware rendering (eye-tracked focus and blur control in FocusWeave), and embodied robotics — a VR teleoperation and gaze-driven data-collection lab for the Unitree H1 humanoid built on NVIDIA Isaac Sim, where human attention becomes annotation-free supervision for robot learning.

Core strengths
  • Unity VR prototyping with robust experiment logic
  • Robotics simulation and teleoperation (Isaac Sim, OpenXR, PhysX)
  • Graphics systems and shader work (OpenGL / GLSL, Unity shaders)
  • End-to-end experiment pipelines (instrumentation, logging, analysis)
  • Applied ML and analytics experience from industry internships
Tooling
  • Unity, C#, shaders
  • Isaac Sim / Omniverse, OpenXR
  • Python (NumPy, SciPy, Pandas)
  • OpenGL / GLSL
  • Data logging and analysis workflows

Featured projects

Featured project

Isaac Sim Humanoid Behavior Lab

VR teleoperation × robot learning

A VR-based teleoperation and data-collection system for the Unitree H1 humanoid, built on NVIDIA Isaac Sim 6.0. I embody the robot from its eye-level, first-person perspective through a Meta Quest Pro, and every session produces a synchronized, gaze-labeled multimodal dataset for embodied-AI research.

  • Gaze-driven object selection: calibrated binocular gaze is raycast into the PhysX scene (with self-hit filtering), highlighting attended objects and logging 3D gaze-collision points — visual-attention supervision with zero manual annotation.
  • Embodied teleoperation: OpenXR head and hand tracking drives the H1’s arms while the robot keeps balance, with physics-based grabbing and experimental step-in-place locomotion.
  • Crash-safe recording: ~100 Hz synchronized streams (HMD pose, hand poses, gaze, object states, full joint angles) plus first-person video, time-aligned automatically per session.
  • Learning pipeline: a phased roadmap from synchronized datasets and CSV baselines to frozen V-JEPA 2 embeddings, action-conditioned latent world models, and MPC planning.
Embodied VR control Gaze-labeled attention 100 Hz multimodal logs World-model learning
Isaac Sim 6.0 OpenXR Quest Pro eye tracking Unitree H1 PhysX Python & C++
Live demos Captured in-headset, real time
Through the robot’s eyes. The calibrated gaze raycast locks onto a cube — yellow highlight, red gaze-collision marker — and the H1’s arms reach in to grab it. Stereo OpenXR view, running live on a Quest Pro.
Embodied teleoperation. OpenXR hand poses drive the H1’s arms while the red gaze cursor tracks live eye movement across the scene.
Gaze raycast, visualized. In minimal render mode the ray from the robot’s eyes tags the attended cube as the humanoid reaches toward it.

View project on GitHub

Featured project

FocusWeave

Gaze-aware rendering

Eye-tracked focus-aware rendering and gaze-contingent blur control in Unity, targeting perceptual evaluation in VR distance judgment tasks.

  • Meta Quest Pro eye tracking integration
  • Stable smoothing and parameterization for experimental conditions
  • Data logging and analysis-ready outputs
Meta Quest Pro Unity Gaze-contingent blur Experiment logging
Demo sequence 9 clips
Demo 1
Demo 2
Demo 3
Demo 4
Demo 5
Demo 6
Demo 7
Demo 8
Demo 9

View project on GitHub

Experience

Lecturer in Computer Science, University of Massachusetts Boston

2024 to present
  • Teach and coordinate large-enrollment undergraduate CS courses, typically 150-200 students per semester across multiple sections
  • Design curricula, programming assignments, exams, and projects across introductory and advanced CS courses
  • Build scalable assessment workflows using Gradescope and custom autograding pipelines
  • Mentor student work spanning theory, systems, software engineering, and VR/AR prototypes

Ph.D. Researcher, Michigan Technological University

2019 to 2023
  • Thesis: the impact of pre-experiment walking on distance perception in VR
  • Designed controlled VR studies and built experiment software and analysis pipelines
  • Collected and analyzed high-frequency HMD tracking data (about 70 Hz) across 130+ participants
  • Quantified pre-experiment blind walking effects, improving distance judgment accuracy from about 80% to 97%+
Methods and algorithms (selected)
  • Cumulative distance integration: computed true walked distance by integrating path deviations
  • Turning point detection: identified meaningful turns using angular velocity signals
  • Step detection: filtering plus scipy.signal.find_peaks on vertical HMD motion
  • Reusable tooling: Python modules for data processing and visualization

Visteon Corporation (3 co-op terms)

2020 to 2022
  • Applied ML, data engineering, and system integration to automotive workflows
  • Progressed from foundational AI research to technical lead responsibilities
  • Delivered production-minded tools that reduced multi-day analysis cycles to minutes
Co-op III (Summer 2022): Product Design Technical Lead Intern
  • Built V-PEDAT, a Python desktop application (Tkinter, Pandas, Matplotlib) to automate analysis
  • Automated processing of 26,500+ data points and graphs previously handled manually
  • Impact: reduced analysis time from multiple days to under 5 minutes
Co-op II (Summer 2021): AI and System Integration Engineering Intern
  • Optimized a DMS-related algorithm using sin(2x) = 2sin(x)cos(x) for efficient arm-angle detection
  • Worked with Leica 3D Disto lidar hardware and conducted testing on Mahindra test benches
  • Skills: Python, C, Bash, Docker, embedded testing, computer vision
Co-op I (Summer 2020): ADAS Engineering Intern
  • Built monocular depth estimation pipelines, converting 60,000+ images from .npz into CNN-ready datasets
  • Trained PackNet models using Keras and TensorFlow, managed reproducible runs with Docker

Publications

Selected work. Full list on Google Scholar.

  1. VR Distance Judgements (ACM SAP 2022)
    ACM DL
  2. Brightness vs. Distance Judgements (ACM SAP 2020)
    ACM DL
  3. Sorting Algorithm Comparison (ACM ACAI 2019)
    ACM DL

Google Scholar

Service

Program committee

  • ACM Symposium on Applied Perception (SAP) Program Committee (2025, 2026)

SAP 2025

Reviewer

  • IEEE VR 2023, IEEE VR 2024
  • ACM VRST 2023, ACM VRST 2024
  • PeerJ (paper reviewer)

Volunteering

  • ACM SIGGRAPH 2024, Student Volunteer
  • AWE USA 2025, Volunteer

Teaching

Lecturer, University of Massachusetts Boston

2024 to present

Teach and coordinate large-enrollment undergraduate computer science courses, typically serving approximately 150-200 students per semester across multiple sections.

CS110: Introduction to Computing CS210: Data Structures CS220: Applied Discrete Mathematics CS240: Programming in C CS410: Software Engineering CS420: Theory of Computation CS450: Higher-Level Programming Languages
  • Design structured curricula, programming assignments, exams, and projects with emphasis on conceptual clarity, problem-solving, and real-world applicability.
  • Implement scalable assessment systems using Gradescope and custom autograding pipelines to improve grading efficiency and consistency across large cohorts.
  • Coordinate course logistics across multiple sections to keep instruction, grading, and student experience consistent.
  • Foster an inclusive learning environment through clear communication, organized course structure, and responsive student support.
  • Mentor student work spanning theory, systems, software engineering, and VR/AR prototypes.

Honors

  • Finishing Fellowship (Ph.D.), 2023
  • Business Opportunity Recognition (VRSPACE), 2022

Contact

Email is best: sepahyarsoheil@gmail.com You can also reach me via LinkedIn.