▶ Experience design and research for next-generation MR head-mounted display Jan. 2024 – Mar. 2024
Contributed to designing and developing VR/MR prototyping HMDs with innovative features for next generation devices. Collaborated with software engineers to ensure PC SDK compatibility and tested various application scenarios and system integration.
▶ AR-based human collaborative demonstration for robotics assembly tasks Jan. 2023 – Sep. 2023
Built a system to demonstrate the task to the robot by virtually performing it on the AR interfaces. The system identifies the user's attention and integrates it into dual-level task planning. This approach improves the efficiency and success rate of long-horizon assembly tasks, which are evaluated on the RAMP benchmark.
▶ A mobile manipulator for telepresence Sep. 2022 – Present
Designed a mobile manipulator system capable of performing remote tasks through human interaction in a hazardous environment. This system enables the perception of the remote environment and control of the robot arm through an Augmented Reality system with promising telepresence and intuitiveness.
▶ Cybersecurity risks in Mixed Reality-based smart manufacturing applications Sep. 2022 – Nov. 2022
Led an exploration of potential cybersecurity risk in MR-based smart manufacturing applications where digital passwords can be deciphered through mid-air gestures by an RGB camera. The study develops a machine learning-based hand gesture recognition method with high accuracy (up to 97.03%) to estimate password input.
▶ A telerobotic system with Mixed-Reality interaction for fitness instruction Jun. 2022 – Sep. 2022
Engineered the development of a tele-robot control system that utilizes a Mixed Reality headset (HoloLens 2), to remotely control Automated Guided Vehicles (Rover Pro). The system features a user-friendly interface that offers three control methods: motion control, physical button control, and virtual button control.
▶ A global calibration to solve the misregistration problems in the VST AR system Oct. 2021 – Feb.2023
Proposed a calibration method for the registration problem in a Video See-Through Augmented Reality
(VSTAR) system. This study investigates 4 error sources and presents an efficient calibration procedure to reduce the misalignment accuracy via HMD-to-Camera transformation.
▶ Augmented reality assisted smart factory management system development Dec. 2021 – Feb. 2022
Developed an AR headset-based prototyping system to offer a clear and intuitive digital visualization, aiming to explore the potential applications in smart factories. This project presents a case study of a maintenance application in a CNC machine shop for technicians.
▶ Augmented reality assisted maintenance and repair system development Dec. 2021 – Feb. 2022
Created an Augmented Reality interface for maintenance procedures by utilizing digital twin technology. By presenting a demo of a repair process (PC memory card replacement), this work aims to highlight the benefits of using AR for instruction, resulting in improved accuracy and ease of operation for technicians.
▶ An application of depth map completion and super-resolution in AR rendering Sep. 2021 – Nov. 2021
Mediated in an experiment on an AR-based human-robot interaction scenario, aimed at demonstrating the effectiveness of a proposed method for addressing occlusion problems in the rendering of virtual objects. The method involves the use of depth completion and super-resolution techniques to enhance the visual quality.
▶ Visualization error analysis for augmented reality stereo VST system May. 2021 – Nov. 2021
In this research study, the sources of registration errors were identified in stereo video see-through systems and a mathematical model was established to describe the propagation of these errors in the display process.
▶ System design of a human-centered AR HRI with cognition awareness Nov. 2019 – Oct. 2021
Spearheaded and proposed a novel system, HAR2bot, for path-planning tasks in complex environments. The proposed system effectively balances and minimizes the cognitive burden associated with the design loop, leading to improved task efficiency and accuracy, which is evaluated by a 20-participant user study.
▶ Augmented reality-based human-robot interaction system Aug. 2019 – Nov. 2020
Launched an intuitive AR system framework and developed an immersive AR-based interface for cobot (AUBO i5) programming in complex environments. Users repeat the process of waypoints definition and editing, collision checking, and path feasibility verification, to obtain a satisfactory path in the proposed system.
▶ Mathematic modeling for robotic polishing of complex models Aug. 2018 – May. 2019
Drafted a mathematical model for robotic polishing and the proposal of a trajectory generation method. The proposed method was rigorously evaluated through motion simulations, which confirmed its effectiveness in producing optimal polishing trajectories.
▶ Structure Design of Micro Multi-legged Robot Based on Piezoelectric Actuators Mar. 2018 – Jun. 2018
Designed a piezoelectric micro multi-legged robot and improved the corresponding control programs to enable the robot to follow a planned path with open-loop control).