Director of the XR Collaboratory Harald Haraldsson, PhD student Shuo Feng, and Cornell Tech Assistant Professor Thijs Roumen posing with a custom wire-bent Cornell Tech Twisted T.
By Sarah Marquart
The XR Collaboratory (XRC) at Cornell Tech has selected two winning proposals for its inaugural XR Collaboratory Prototyping Grant, which provides technical guidance and monetary support for developing augmented and virtual reality (AR/VR) applications. Thijs Roumen, assistant professor of information science at Cornell Tech, who is supporting PhD student Shuo Feng, and François Guimbretière, professor of information science at Cornell University, who is supporting PhD student Mose Sakashita, are the recipients of the new grant award.
The XR Collaboratory at Cornell Tech accelerates activities in augmented and virtual reality through course offerings, projects, and cross-campus collaborations. The team’s primary interests are in 3D user interfaces and interaction design for head-mounted displays. They build high-fidelity prototypes with students through projects and coursework, and collaborate with Cornell faculty on exploratory AR/VR-related research across disciplines such as computer vision, computer graphics, and human-computer interaction.
“Conducting research involving the development of 3D user interfaces for AR/VR can be both challenging and time consuming,” says Harald Haraldsson, Director of the XR Collaboratory. “Our aim with this grant is to provide resources to help with applying best practices to AR/VR prototyping for research purposes. This includes how to develop an inventory of modular, reusable components, being consistent across the research team in the use of design patterns and code style, all with the objective of speeding up the prototyping process and ensuring continuity from project to project.”
Roumen and Feng’s XR project focuses on wire bending, a crucial aspect of industrial manufacturing in which items like springs, paperclips, and hooks are mass-produced. Recently, there’s been a growing interest in custom wire bending for medical applications, entertainment, soft robotics, and fabrication support. To enhance the design process and meet this need, the team has proposed building an AR interface to design wire-bent structures.
Guimbretière and Sakashita’s ongoing work explores the potential of integrating VR with physical robots to enhance remote collaboration. It’s based on a prior initiative by the pair called the VRoxy system, a mobile robot with a monitor for a head that can be present on a remote user’s behalf. While VRoxy can mirror the user’s non-verbal cues — such as pointing or face tilting — in real-time, the bot can’t physically interact with objects, limiting certain types of collaboration. Guimbretière and Sakashita’s proposal includes addressing these limitations by implementing a 3D user interface that enables the remote VR user to navigate larger environments and manipulate physical objects with robotic arms.