Visit



A “ghostdriver” car seat costume, developed to better assess pedestrians’ reactions to autonomous cars on the road.

By Melanie Lefkowitz

In Mexico City, some pedestrians reacted playfully to apparently autonomous vehicles, walking in front of them or nudging friends toward them. In rural Colima, Mexico, around 450 miles away, pedestrians were more likely to wait for the cars to pass before crossing the street.

And in the Netherlands, some pedestrians were surprised to encounter driverless cars, believing the decision to introduce them should have been discussed at town hall meetings ahead of time.

In a series of studies conducted in three countries over more than five years, a Cornell Tech-led team has pioneered the use of “ghostdrivers” – cars with drivers disguised under a car seat-like hood, to make the car appear driverless – in order to assess and compare how pedestrians across cultures might actually behave when encountering these cars on the roads.

“Researchers were running experiments, but it was really hard to get naturalistic behavior because people knew they were coming to a special place to see a special car,” said Wendy Ju, assistant professor of information science at the Jacobs Technion-Cornell Institute at Cornell Tech.

“Our study was the first to pioneer having fake autonomous cars on the roads, and it’s really useful in eliciting the way people are going to interact with them,” she said. “This kind of research has to happen before autonomous systems are in place.”

Ju is senior author of “On-Road and Online Studies to Investigate Beliefs and Behaviors of Netherlands, U.S. and Mexico Pedestrians Encountering Hidden-Driver Vehicles,” which won an honorable mention at the 2020 Association for Computing Machinery/Institute of Electrical and Electronics Engineers International Conference on Human-Robot Interaction. (The conference was canceled because of the coronavirus pandemic but some talks were held online.)

The paper’s first author is Jamy Li of the University of Twente, Netherlands. The study was co-authored by David Goedicke, a Cornell Tech doctoral student; Hamish Tennent, formerly a researcher at Cornell Tech; Rebecca Currano, Aaron Levine and David Sirkin of Stanford University; and Vanessa Evers of the University of Twente.

After developing the ghostdriver technique, Ju’s team experimented with it in California’s Bay Area, but wanted to try different locations because autonomous cars are frequently tested in the Bay Area, and residents are relatively used to seeing them.

“Over the course of this research, we discovered that there are many different localized rules and norms for how people interact with autonomous cars and with each other,” she said.

For example, in a 2018 study, they found pedestrians in Mexico City were more likely to cross in front of a ghostdriven vehicle, while residents of Colima would wait for it to pass; and more Mexico City residents took a curved path across the street than Colima residents.

In the most recent study, the team found that pedestrians in Europe, like their North American counterparts, took longer to cross when the vehicle appeared to be autonomous. They also found that pedestrians in groups spent less time looking at normal cars than did single pedestrians, but more time looking at the ghostdriven cars than did single pedestrians.

The researchers also conducted experiments with people watching autonomous cars on video, both in the Netherlands and California. They found that in both locations, people were less likely to believe a car was autonomous when watching a video than when they encountered it in real life.

While testing the ghostdriven cars on the road, the researchers found pedestrians hardly looked at cars before crossing.

“Sometimes people wouldn’t remember that a car was at the intersection at all, because crossing is such an automatic behavior and people are preoccupied,” she said. Researchers also discovered people tended to look at the car’s front bumper and front wheel, seeking eye contact with the driver only if something seemed unusual.

“So what we think is the normal behavior,” Ju said, “is actually the breakdown behavior – what we do when things are uncertain.”

In interviews conducted after the encounter, pedestrians in the Netherlands expressed fear of the seemingly driverless car. “We were trying to run away from it … . It’s creepy,” one said; “We don’t know how advanced it is or how it works. … we’re afraid, because we don’t know,” another said.

Understanding cultural norms for pedestrians and drivers is important not only for safety but for successfully integrating autonomous vehicles into cities and towns, Ju said.

“If you don’t have any idea what people are going to do, there are going to be more incidents,” she said. “The other thing is that people have really strong opinions about these cultural norms. It’s not enough that the cars are not running people over; people have a feeling about whether cars behave in the right way or the wrong way, and they get really angry if the car doesn’t do the right thing.”

Very slow driving, for example, might be safer but it also might enrage the other drivers on the road. “People expect technology to adapt to the way they drive,” she said. “It’s really important that we teach these autonomous cars to learn the lingua franca of on-road interactions.”

This story originally appeared in the Cornell Chronicle.


Serge Belongie, member of the Computer Science department and Associate Dean at Cornell Tech, has been named Andrew H. and Ann R. Tisch Professor. In response to his investiture as an endowed chair, which began on April 1st, Belongie says “I wish I had the words to express my gratitude for this remarkable honor.”

In his capacity as Associate Dean at Cornell Tech, Belongie is “busy with coronavirus pandemic-related planning for Fall semester course offerings.” As professor, he is working on “growing our cross-campus research efforts in Mixed Reality.” The latter initiative “gathers efforts from across Cornell’s campuses that relate to augmented and virtual reality, and their core disciplines of computer vision, computer graphics, and human-computer interaction.” At present Belongie and others are co-organizing the 4th Workshop for AR/VR at CVPR 2020. Workshop sponsors include Facebook, Magic Leap, Google, and Cornell.

This story originally appeared in the Cornell CIS News.


By Melanie Lefkowitz

In mid-March, Rebecca Brachman, a postdoctoral fellow in the Jacobs Technion-Cornell Institute at Cornell Tech’s Runway Startup Postdoc Program, was thinking about big-picture ways to tackle the COVID-19 pandemic. She called another fellow in the program, Server Ertem, Ph.D. ’09, to brainstorm.

In their 15-minute conversation, the two realized that a cancer detection tool under development by Ertem’s company could be adapted to the new coronavirus. Now they’re working with hospitals in New York City and companies around the world to develop a rapid, inexpensive and accurate test for coronavirus immunity that could help individuals and communities determine who might safely rejoin the workforce or care for the sick.

“We realized that what we’ve already been developing on cancer actually had a direct correlation to how the COVID-19 virus protects itself, and that’s how we’ve realized that we already have the tools and why we can deploy so fast,” said Ertem, CEO of Katena Oncology, a biotech startup. “We have extensive clinical partners, and we’re working with them to validate our test as fast as possible.”

Ertem and Brachman, co-founder and director of the Social Outcomes Initiative (SoGo), were among the presenters at an April 1 virtual demonstration of Runway program projects aimed at helping society cope with the global coronavirus pandemic. An online audience of around 50 alumni, entrepreneurs and others also heard from Runway Startup Postdoc Param Kulkarni, CEO of AwareHealth, an AI-based mental health app to help medical professionals who are treating coronavirus patients; and Neuralpositive, which delivers hyper-personalized [music] playlists designed to have therapeutic effects.

“They saw the challenge of COVID-19 and they just dove into it,” said Fernando Gómez-Baquero, director of Runway. “So they put their minds, their thoughts and their companies to work on it.”

The Runway program provides one or two years of academic and business mentorship, and funding for recent Ph.D.s with promising business ideas that need time and guidance to develop. Since 2014, Runway participants have created 28 startups – 22 of which are still in operation and two exits – employing 189 people and raising nearly $100 million in funding. “We bring the best postdocs in the world [here] and help them create companies and make an impact beyond themselves and academia,” Gómez-Baquero said.

Runway fellow Yasmine Van Wilt, co-founder and co-CEO of Neuralpositive, gave her presentation with long pauses for coughs and breaths because she herself is battling COVID-19.

“I acquired mine probably from patients while at the hospital doing this research,” said Van Wilt, who works on therapeutics combining insights from neuroscience, neurobiology, epidemiology and music with patients in pulmonary distress. “We’ve also lost family members, in our team, over the last few days. So we’re coming together and this fight is personal for us.”

While numerous research teams are racing to develop quick tests to detect who has COVID-19, as of March 31 no one had approached the U.S. Food and Drug Administration about a test for immunity, Server and Brachman said during their presentation.

Such a test – especially if it could be cheaply made and easily used – could have important implications for reopening society and boosting a struggling economy. It could also help determine which health care professionals could safely administer to the sick, or when it’s safe for a family member to take care of a relative with the virus, they said.

“We’re looking at numbers in the millions here. So that’s extremely important, especially in a city like New York, if you can get one out of eight people back in the workforce,” said Brachman, who has a doctorate in neuroscience. “We need to know who has individual immunity, so they are protected if they are on the front lines. And then we also need to know at some point when we have herd immunity, which will allow us to more generally open up these cities and states because we can be more confident that we won’t see another big spike in infections.”

One of the Runway program’s strengths is bringing together thinkers from different backgrounds and with different expertise to brainstorm solutions together, said Ron Brachman, director of the Jacobs Institute and professor of computer science.

“Because of this pandemic situation in which we find ourselves, we’re innovating in some really interesting ways,” he told the virtual audience. “Jacobs and the Runway program really push the envelope around radical experimentation at the intersection of education, research and entrepreneurship.”

This story originally appeared in the Cornell Chronicle.


Wendy Ju, Assistant Professor at the Jacobs Technion-Cornell Institute at Cornell Tech and in the Information Science field at Cornell University, recently received an Amazon Research Award (ARA) for her proposal, “Enabling Machines to Recognize and Repair Errors in Interaction.” The award gives $70,000 in funding to Ju’s team, as well as an additional $20,000 in Amazon Web Services (AWS) promotional credits to help them to pursue their proposed research plan.

As a part of Amazon’s goal to advance scientific research, the company collaborates with academic institutions worldwide to support multi-year initiatives through its Amazon Research Awards program. The ARA program funds academic research and related contributions to open-source projects by top academic researchers around the world. ARA aims to fund projects leading toward a PhD degree or conducted as a part of post-doctoral work.