Visit

Approximately 466 million people worldwide have “disabling hearing loss,” according to the World Health Organization. That’s more than five percent of the world’s population.

Christopher Caulfield and Devon Bain, both candidates for the Technion-Cornell Dual Master’s Degrees in Connective Media ‘19, have personal connections with some of the communication barriers facing people who are deaf or hard of hearing. Caulfield was born profoundly deaf and Bain’s mother has hearing loss.

For their specialization project (a requirement of all Jacobs Technion-Cornell Institute master’s students), Caulfield and Bain are creating technology to make one-on-one conversations easier for people who are deaf or hard of hearing.

Using Research to Develop Technology

“Connective Media is really unique because it doesn’t just teach us about software development, it also teaches about user research and design,” said Bain.

The pair is working with Assistant Professor Shiri Azenkot, whose research focuses on accessible technology. Azenkot is an assistant professor at the Jacobs Technion-Cornell Institute at Cornell Tech who also specializes in research and technology for people who are blind or, like herself, have low vision. Azenkot has provided valuable feedback on the project based on her accessibility research. The fact that she is not deaf or hard of hearing has given them an additional perspective, said Caulfield.

Caulfield and Bain interviewed seven deaf or hard of hearing individuals of all backgrounds to learn about the technology they currently used, the challenges they faced with one-on-one conversations, and the technology that would be most helpful. The people ranged from students and senior citizens to working professionals. The primary difficulties most participants faced included difficulties hearing and participating in conversations in noisy locations like restaurants, subways, and public places. Some of the individuals felt frustrated, angry, and less motivated to socialize even though they wanted to participate in conversations. People also expressed worry about how others would interpret their hearing loss and accessible technology use. For example, one woman was nervous to disclose her hearing loss at work because she was nervous about how it would be received.

Most of the people they interviewed wanted a wearable augmented reality captioning system that could be used for lengthier conversations and ones in loud places. The insight motivated Caulfield and Bain to show captions in augmented reality glasses because they found that most of the current technology shows captions on computers or handheld devices. “Our research focuses on how this is a barrier on one-on-one conversations,” said Caulfield, “We want to make conversations more seamless using augmented reality.”

Building Commercial Projects

Students meet with company advisors throughout two semesters to create a presentation and demo which they pitch to company stakeholders at their mentoring company. Caulfield and Bain are working closely with a Verizon executive who is interested in augmented reality. They have people test their prototypes to provide feedback for iterations. “Hopefully that will lead to a fully-functional commercial project that people will use,” said Bain.

Caulfield and Bain built a prototype that uses augmented reality headsets. They used ARKit, the iOS platform for augmented reality, and a text box to populate captions for the user, who inserts their smartphone into a MERGE augmented reality headset. Right now they are using a script to show the overall experience, but later they plan to integrate automated speech recognition. They are using computer vision and face recognition to locate a person’s face and to position the captions directly under the speaker’s chin and the text size and color changes based on the speaker’s tone and emphasis.

Building More Accessibility Products at Cornell Tech

“Our project has really inspired a lot of people to think about accessibility at Cornell Tech,” said Bain. Now a group of Cornell Tech students meets regularly to discuss accessibility projects and Caulfield and Bain think it is likely that one of the ideas will be refined in Startup Studio.

“We were surprised about how each of them [was] interested in accessibility because they were related to someone with a disability or good friends with someone with a disability,” said Caulfield. And Bain said that even people without a personal connection to someone with a disability cared because they are focused on social good.

When Caulfield decided to attend Cornell Tech, his mission was to complete a specialization project related to a painful point he experienced in his life — hearing loss. Bain had a shared vision because he wanted to create technology that makes it easier for his mother and other people with hearing loss to have more seamless discussions. Their dedication has motivated other Cornell Tech students to use their skills to build accessibility technology to help people with a range of needs.


By Diane Levitt
@diane_levitt

I joined Cornell Tech five years ago with a master’s degree in early childhood education and a career that mostly centered on education and philanthropy. I knew very little about computer science. I thought Java was coffee, Scratch was for itching, Python was a snake, and Basic was, well, basic. I came to Cornell Tech to explore what was possible in K-12 computing. Through this exploration, I’ve developed a set of values—a creed of sorts—that guides our work here at Cornell Tech.

Students thrive when we teach at the intersection of rigor and joy. In computer science, it’s fun to play with the real thing. But sometimes we water it down until it’s too easy—and kids know it. Struggle itself will not turn kids away from computer science. They want relevant learning experiences that lead to building things that matter to them. “I can do hard things!” is one of the most powerful thoughts a student can have.

Teachers matter. We can’t prepare students without preparing teachers. There is no online platform as effective as a skilled, caring human being in the room.

Great computer science teachers take many different paths to the classroom. On my team, we have four gifted master teachers from four backgrounds: special ed, social studies, tech, and design. There’s no traditional route to teaching K-12 computer science today. This is a shift from the recruitment of computer science teachers in the past and has an impact on how we recruit and prepare teachers. It also adds a wonderful dimension of diversity to our community.

Getting teachers prepared to teach computer science takes time and consistency. A few days of professional development here and there is not enough to get teachers ready for computer science. They need training and support over time. This was the thinking behind Cornell Tech’s Teacher in Residence program: we’re investigating whether putting a highly skilled computer science coach in a school for a year or longer helps teachers deliver instruction more competently and confidently.

The biggest lever we have is the one we aren’t using enough yet: preservice education for new teachers. The sooner we start teaching computer science education alongside the teaching of math and reading, during teachers’ professional preparation programs, the sooner we get to scale. It’s expensive and time-consuming to continually retool our workforce. Eventually, if every teacher enters the classroom prepared to include computer science, every student will be prepared for the digital world in which they live. This is what we mean by equity: equal access for every student, regardless of geography, gender, income, ability, or, frankly, interest.

We need to know more about how and what to teach. We have a little research and some survey data. We’ve transferred some research from other subjects. Some assumptions have been set in stone. This is an imperfect and difficult-to-navigate set of guidelines for educators. Because our curriculum comes from many sources, each with its own set of questions and reasons to research, we have a very fractured picture. We would learn a lot if, as a community, we agreed on a set of metrics for the next five years and were transparent about our findings.

Integrating computer science into other subjects is hard. There’s only so much time in the school day, and in response, there’s a serious effort to embed computer science into other disciplines as our best hope to reach every student. But can we bring computer science into a math lesson and do both subjects justice? I’ve seen some great examples that argue yes—and more that show one subject losing ground to the other. Better understanding the benefits and costs of integrating computer science is a high priority for our field.

It’s not (only) about jobs. When we focus on teaching computer science solely to fill open jobs in technology it changes what and how we teach. We sacrifice deep learning for short-term gains. We teach material that may be obsolete before our students enter the workforce. Our job is to prepare students for the world so they can seize all opportunities, personal or professional, that come their way. We need to let pedagogy — the practice of teaching based on the science of learning — lead the way.

We have to start planning for success.  We can’t continue to offer introductory lessons to students who are on their way to mastery or we will bore them out of computing. We need to be ready for them with fresh curriculum that builds on their skills.

I’m more confident than ever that we can take a highly complex subject and translate it for every student. We can teach rigorous, joyful computer science to kids of all abilities from all backgrounds. We can prepare teachers from diverse backgrounds, grades, and subjects to understand and teach computer science. But we will have to do it intentionally, and commit time and money. There is still so much to learn. We need to know more to do better. But I’ve seen students with special needs, emerging bilingual students, students of color—all underrepresented in tech— navigate computing with dexterity and purpose. So I know it’s possible because it’s already happening for some. And now that we know it’s possible, we must make it happen for all.

This blog is the first in a series of posts called Ground Truth, which is a term from multiple fields describing information provided by direct observation as opposed to inference. Over the course of the year, I’m going to share conversations with some of the people whose leadership I’ve had the opportunity to observe and learn from. Observe with me and share your ground truth on Twitter (@diane_levitt).




On secondhand marketplaces like eBay, people trust online sellers who post their own high-quality photos of items for sale more than they trust those who use stock images or poor-quality photos, a Cornell Tech study has found.

The findings could help online marketplaces improve trust in their sites by offering guidelines on how to take better photos or introducing augmented reality features that instruct users to change lighting or camera angles, the researchers said.

This is particularly important for new or growing sites that are working to establish trust, said Xiao Ma, lead author of “Understanding Image Quality and Trust in Peer-to-Peer Marketplaces,” to be presented at WACV 2019, Jan. 7-11 in Waikoloa Village, Hawaii.  

“The high-quality product images selected by our model automatically outperform stock images in generating perceptions of trustworthiness,” said Ma, a doctoral student in the field of information science at Cornell Tech. “People believe the user-generated images represent the actual condition of the product better. Stock images present more uncertainty and raise questions such as whether they are too good to be true.”

The study, co-authored with Mor Naaman, associate professor of information science at the Jacobs Technion-Cornell Institute at Cornell Tech, and Serge Belongie, professor of computer science at Cornell Tech, as well as colleagues at École Polytechnique, Google Research and eBay, grew out of previous work on trust in Cornell Tech’s Connected Experiences Lab. Trust is essential for society to function, but research shows levels of trust have been declining in recent years. Establishing and building trust in digital environments is even more complex.

“Without face-to-face interactions there is a lot of uncertainty,” Ma said. “You don’t know what’s going on on the other side of the screen. People could lie; they could post different images. In the beginning of e-commerce there were a lot of studies on trust, but that was limited because our ability to understand images and language computationally has been limited. Now, because of computer vision and natural language processing, we’re able to understand a lot of these online interactions better, and there’s an opportunity to revisit these questions of online trust.”

For this study, the researchers used publicly available data from the mobile classifieds app Letgo.com and private data from eBay. They focused on shoes and handbags because they’re among the most popular goods found on secondhand marketplaces, and because they are visually distinctive enough to pose an interesting computer vision challenge.

Using the images, they developed a deep learning algorithm – a kind of artificial intelligence frequently used for classification tasks – to predict image quality. They found the algorithm to be around 87 percent accurate, but because of the way deep learning works, researchers could not tell how the model arrived at its decisions. To learn more about which elements improve an image, they also analyzed images using classic computer vision methods and linear regression, a kind of statistical modeling.

Among their findings: Images are more likely to be labeled high quality if they are brighter, and less likely if they have a high foreground to background ratio. A good-quality image should have high contrast for the product and low contrast for the background, they found.

Once they had established methods of predicting quality, the researchers investigated its impact on sales and consumer trust. Shoes with higher-quality images were found to be 1.17 times more likely to be sold than those with lower-quality images, and handbags with better photos were 1.25 times more likely to sell. But because sales are also subject to factors such as price, further research is needed, Ma said.

To test trustworthiness, the researchers designed three hypothetical marketplaces and populated one of them with high-quality images, one with low-quality images and one with stock photos. They recruited 300 people to rate the marketplaces from one to five on a series of statements gauging trust.

The site with the better images scored the highest, with participants rating it around 3.8 for the statement “I believe that the products from these sellers will meet my expectations when delivered,” compared with around 3.7 for the site using stock imagery and 3.4 for the site with low-quality images.

The results – which surprised researchers, who did not expect high-quality personal images to perform better than stock imagery – could be especially helpful to new sites, which might introduce features to improve the quality of users’ photos. For example, instead of automatically using the first uploaded image as a thumbnail, apps could use an algorithm to choose the best-quality image. The research could also be applicable to other kinds of sites, such as real estate or dating.

“Digital environments create new challenges and opportunities for different types of trust,” Ma said. “A lot of the challenge in starting online platforms is in gaining the users’ trust to have people adopt it.”

The study was partly funded by a Facebook equipment donation and by Oath, which is part of Verizon, and Yahoo Research through the Connected Experiences Lab.

This article originally appeared in the Cornell Chronicle.

Concertio, a Runway Startup at the Jacobs Technion-Cornell Institute, aims to optimize hardware- and software-system-tuning using AI-powered tools that work in concert with currently running workloads.

Learn more about Concertio in this Q&A with co-founder and CEO Tomer Morad.

What does your company do?

Concertio provides AI-powered performance optimization tools that boost the performance of a computing system by tailoring the many system settings (in processors, firmware, OS, and applications) to the running workloads. Our tools are used by the likes of Intel, Marvell, and Mellanox to boost the performance of a multitude of devices and applications.

How has the Jacobs Institute’s Runway program helped you to develop your company?

The Runway program, like its name, provided us with the “runway” we needed to develop deep and risky technology and apply it to real-world applications. Thanks to the guidance, mentoring, and a pre-seed investment provided by the program, we now have a working deep-tech product and paying customers.

What impact do you hope your company will have in the industry/world?

We’re in an era where there’s an ever-growing thirst for computing capacity, driven by new and exciting applications such as AI. The pace of improvements in process technology, however, can barely keep up with this trend, so general-purpose computing systems will need to become more efficient. This will be our impact: our technology will allow these systems to self-tune and adapt to their current workloads in real time, thereby providing performance that is closer to special-purpose systems but achieved on general-purpose systems.

Where did you earn your Ph.D. and what was your research focus?

I earned my Ph.D. at the Technion – Israel Institute of Technology. My research focused on computer architecture, and I’ve researched how to provide higher performance and higher energy efficiency in resource-constrained computing systems.  

Why did you want to commercialize that research? What was the inspiration behind your company?

My Ph.D. research is not directly commercialized, but rather it served as the inspiration for Concertio. In my career as a hardware and software engineer, as well as in my research, I’ve come to notice that tuning systems is a very difficult task. In the industry, system tuning is important for achieving higher performance and energy efficiency. In academia, system tuning is imperative for evaluating whether new ideas have merit or whether their benefits will disappear in tuned systems. Since machine learning algorithms were already efficient and accurate enough to run in parallel to workloads, I thought it would be technically possible to build a dynamic tuning software product.

Why did you apply to the Runway program?

Applying to the Runway program was a natural choice for me, as I already had startup experience and was looking for the next big thing after finishing my Ph.D. Knowing how difficult it is to start and run a startup, I viewed the benefits of the Runway program as too great to ignore.

What has been the biggest challenge switching your mindset from a researcher/academic to an entrepreneur?

For me, the challenge has been the other way around: how can I possibly reach the finish line with my Ph.D. with all these ideas running in my head?!


In his research on college students’ productivity, Cornell Tech graduate student Fabian Okeke heard many accounts of time lost to social media, beginning with a click over to Facebook or YouTube for a quick distraction.

But the distraction was not always so quick.

“You have cases where a few minutes becomes an hour, because these programs have been designed to keep pulling people back in,” said Okeke, a doctoral student in the field of information science at Cornell Tech. “So we started thinking about ways we could design different kinds of interventions that could help people focus back on work.”

Okeke and his colleagues didn’t want to block Facebook or other potentially addictive apps – blocking tools already exist, these sites can be helpful or entertaining, and people might avoid signing up for a tool that bars them from certain apps completely. Instead, they looked to the theories of behavioral economics and psychology and developed an app that uses negative reinforcement, in the form of persistent smartphone vibrations, to remind users they’d exceeded their predetermined time limit.

Once they exceeded that limit, study participants’ phones vibrated every five seconds until they navigated away from the targeted app. Over the duration of a study focused on Facebook usage, participants reduced their time on the Facebook app by an average of 20 percent.

“The fact that something as simple as a repeated vibration could help people reduce their usage was pretty powerful,” said Okeke, lead author of “Good Vibrations: Can a Digital Nudge Reduce Digital Overload,” presented at the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services in September in Barcelona.

Co-author Michael Sobolev, a postdoctoral associate at Cornell Tech, said the researchers were inspired by the way cars beep until drivers and passengers put on their seatbelts.

“They tried to increase fines, but that didn’t work; the only technology that actually solved the problem is the beeping noise,” Sobolev said. “You want to give people the freedom to do what they want, but also to nudge them in the right direction. The vibration doesn’t prevent you from doing anything, but it functions as a reminder and a negative reinforcement.”

Because smartphones already have a vibration feature, the method doesn’t require any additional hardware, making it not only simple but cost-effective, the researchers said.

The researchers chose Facebook for their study because it is the world’s most dominant mobile app, and because prior research showed that 90 percent of people who wanted to limit digital consumption specifically sought to reduce time spent on Facebook.

For the study, the researchers recruited 50 participants who already had the Facebook app installed on their phones. They divided them into a group which received interventions, and a control group, which did not.

During the first week, all participants used Facebook without intervention. In the second week, the phones of users in the intervention group vibrated gently every five seconds when they exceeded their limits until they stopped using the app. In the third week, participants’ Facebook usage continued to be monitored but they no longer received vibration “nudges.”

The researchers found that users who received vibrations reduced their time spent on Facebook by more than 20 percent during that week, but the vibrations did not reduce the number of times they opened the app. Participants returned to their regular use levels when the vibration intervention stopped in the third week.

In the future, the researchers said they plan to look into larger-scale studies and they also hope to develop this tool as a publicly available app.

“We’re looking for ways to actually help people achieve whatever goals they set for themselves,” Okeke said. “We’re not looking to say, ‘completely stop using Facebook’; we want to give people control to exercise how much they consume digital content.”

The paper was co-authored with Nicola Dell, assistant professor of information and computer science at the Jacobs Technion-Cornell Institute at Cornell Tech, and Deborah Estrin, the Robert V. Tishman ’37 Professor of Computer Science and associate dean at Cornell Tech and a professor of health care policy and research at Weill Cornell Medicine. The work was supported by the National Science Foundation and Oath, which is part of Verizon, through the Connected Experiences Lab at Cornell Tech.

This article originally appeared in the Cornell Chronicle.


Tatch, a Runway Startup at the Jacobs Technion-Cornell Institute, transforms the cumbersome sleep-labs into smart wireless patches at home. It creates a first-of-its-kind solution that combines advances in printed electronics and machine learning to make sleep medicine more seamless and accurate than ever before.

Learn more about Tatch in a Q&A with co-founder and CEO Amir Reuveny.

What is Tatch?
Tatch makes smart patches for sleep-disorder diagnostics and outcome management. We developed the first, fully diagnostic wearable patch, designed for patient comfort and professional accuracy. Tatch brings together advances in flexible electronics, machine learning and wearable technology to make sleep medicine more seamless and accurate than ever before.  



What impact do you hope your company will have in the industry/world?
Hundreds of millions of people around the world are suffering from chronic sleep disorders, and cannot have a good night sleep on a daily basis. The vast majority of those people, almost 90%, remain undiagnosed and untreated and have higher risk for heart-attacks, diabetes, and high blood pressure. By making sleep diagnostic more accessible and affordable, Tatch aims to change this reality and allow more people to be diagnosed and get the right treatment for their illness, making their life better and longer.

How has the Jacobs Institute’s Runway Startup Postdoc Program helped you to develop your company?
Thanks to the Runway Startup program, Tatch can make something big. We are bringing innovation to a huge market to impact the lives of many. This market can be sometimes conservative and reluctant to change. We need time, funds, and professional network that will help us leverage our expertise. This is exactly what the Runway program provides. With Runway we can get the depth and stability of the academic environment combined with the drive and spirit of the entrepreneurial world. We have enough time to transition from the academic state of mind, to the business one, study different markets and needs, build our prototype, and get the initial funds for a sustainable growth. In my view, Runway is an amazing opportunity for people who have the skills and expertise, and want to do something great in practical business domains. You can’t find it anywhere else.
 
Where did you earn your PhD and what was your research focus?
I completed my PhD at the University of Tokyo, Japan, specializing in the field of ultraflexible organic electronics.

Why did you want to commercialize that research? What was the inspiration behind your company?
Electronic devices and sensors become an essential part of our lives, supporting us in almost every action we do. However they remain rigid, heavy and bulky, affecting their ability to effectively track our health and performance. With the advancements in flexible electronics we can make circuits and sensors soft, thin and almost imperceptible, dramatically improving the experience and reliability of the data collected. This has an immense importance especially in sleep medicine where collecting data should not affect your normal sleeping patterns. This is what we change in Tatch.

Why did you apply to the Runway program?
It was a perfect match. I had just graduated from my PhD and decided to go after entrepreneurial endeavors. The Runway program provides an amazing platform to cultivate and grow great ideas and technologies into viable products.

What has been the biggest challenge switching your mindset from a researcher/academic to an entrepreneur?
One of the major changes is to start thinking about the user experience when encountering your technology and product. You never think about it as a researcher.


2018 was another great year at Cornell Tech.

After opening the campus in September 2017, we finally started to feel settled in to our new space this year.

We welcomed more than 7,000 visitors to the campus through events, tours and seminars. After graduating our largest class of master’s students in May, our alumni community is now well over 500.

Over 50 startups have spun out of the campus — most from the Studio program or the Jacobs Institute’s Runway Startup Postdoc Program creating over 200 jobs. This year also saw our first Cornell Tech startup acquisitions: Trigger, Uru and Gitlinks.

We also broadened our work in the broader New York City community through campus initiatives. More than 3,000 New York City public school students have been impacted by our K-12 Education Initiative. Through WiTNY, a partnership with CUNY, more than 1,200 young women have participated in one or more program designed to support and encourage more female participation in the technology field.

In case you missed any of it, here are some of the best and most popular stories of the year.


Remaking the City: Masters Students Build Products for Roosevelt Island Community Organizations

In the fall 2017 semester, teams of masters students were paired with Roosevelt Island organizations like the Senior Center or the Roosevelt Island Operating Corporation (RIOC) to understand their needs and challenges and develop technological and design solutions for them.

""


Winternship Recap 2018

In January, 177 CUNY women interested in technology careers took part in Winternships at 46 companies. A Winternship is a two- or three-week mini-internship during the winter academic break for freshmen and sophomore CUNY women.

Winternships give these young women an immersive experience in different tech businesses/industries as well as a resume credential that will make them more competitive when applying for a summer tech internship.


Could This Smart Patch Help People Finally Get a Good Night’s Sleep?

Sleep should be a time to recharge, be comfortable, and dream, yet nearly one in five Americans suffer from a chronic sleep disorder that prevents them from restful slumber. It is a problem that TATCH, a Runway Startup Postdoc Program company at the Jacobs Technion-Cornell Institute, is keen to tackle and their innovative solution could bring relief to millions of troubled sleepers.


Can Humans and Robots Interact Naturally?

This question fascinated Dr. Wendy Ju, who joined the Jacobs Technion-Cornell Institute at Cornell Tech as Assistant Professor of Information Science last year. Ju is setting up a robotics research lab for human-and-robot interaction, where she will seek how to design machines that can easily and naturally interact with humans.

""


Cornell Tech Announces Winners of 2018 Startup Awards

In May, we awarded four student startup companies co-working space and pre-seed funding worth up to $100,000 in the fourth annual Startup Awards competition — the first held on the new Roosevelt Island campus.

Winners of the awards were:

  • litOS: 800 million illiterate people worldwide struggle to use smartphones. litOS is a text-free, voice-assisted mobile OS solution to solve this.
  • ReverCare: ReverCare connects families to curated resources and coaches to ease the burden of caring for elderly loved ones.
  • Kipit: Kipit builds devices that automatically track your personal items and alert you before you leave anything behind.
  • Crater: Crater is a revolution in local TV led by a new generation of creators. Armed with AI-powered video tools, anyone can create their own self-produced local TV shows and episodes through their mobile phone.

""

 

 


Video: Transforming Computer Science Education in Public Schools

Diane Levitt, senior director of K-12 education at Cornell Tech, discusses the mission of our K-12 initiative and how we aim to prepare every child for full citizenship in the digital age.


Cornell Tech Announces Inaugural Chief Practice Officer

""

In August, we announced Josh Hartmann joined the campus as its first Chief Practice Officer. In this position, Hartmann is responsible for leading Cornell Tech academic activities that bridge graduate education and practical implementation. Previously, Hartmann held interdisciplinary leadership positions serving as former Chief Technology Officer and Chief Operating Officer at several tech companies including Amplify and Travelocity.


Health tech pioneer Deborah Estrin named MacArthur fellow

Deborah Estrin, the Robert V. Tishman ’37 professor of computer science at Cornell Tech and of healthcare policy and research at Weill Cornell Medicine, has been awarded a 2018 MacArthur Foundation fellowship for her innovative work using mobile devices and data to address social challenges.

Estrin, who also serves as an associate dean at Cornell Tech, will receive a no-strings-attached award of $625,000 over five years – widely known as the “genius grant.”

""


How Cornell Tech Aims to Foster ‘the right kind of entrepreneurship’

New technologies are now faster, cheaper, and easier to develop than ever before. But such rapid and often unchecked innovation can sometimes lead to harmful outcomes.

For Frederic Rubinstein ’52, LLB ’55, “The right kind of entrepreneurship and technology should make a significant positive contribution to improving life on our planet.”

""


Video: Tech with a Purpose

Cornell Tech is building a diverse environment of academics and practitioners who excel at imagining, researching and building digitally-enabled products and services to directly address societal and commercial needs, particularly in areas that both draw on and contribute to the vibrancy of New York City.

 


People explore less when they get recommendations from voice-based platforms such as Amazon’s Alexa or Apple’s Siri, making it more likely that they’ll hear options chosen by an algorithm than those they might actually prefer.

A study by Cornell researchers, exploring the broader implications of how content will be discovered as smart speakers grow more widespread, found that people who read choices online consumed information nine times faster and explored at least three times as much as those who heard them listed.

“We found that this problem is quite significant,” said Longqi Yang, a computer science doctoral student at Cornell Tech and first author of the paper, “Understanding User Interactions with Podcast Recommendations Delivered via Voice,” which was presented at the ACM Conference on Recommender Systems in October. “With these devices becoming more popular and more people adopting them, this kind of interface becomes very important, because it’s one of the major channels for people to be exposed to information.”

Smart speakers and virtual assistants could be designed differently to address this challenge, Yang said. The researchers recommended that smart speakers offer top-ranked choices that are diverse, personalized and frequently changed, so users have access to a wider range of information even if they choose from the first few items.

“We don’t want people to be offered an overly narrow set of content and opinions or be exposed only to what is most popular,” Yang said. “That might be acceptable when recommending shoes, but not when recommending information and cultural content.”

According to consumer research, 16 percent of Americans own a smart speaker – around 40 million people – and 65 percent of those say they would not go back to life without one.

In this experiment, the researchers asked 100 people to choose a podcast they would commit to listening to for five minutes. Half the participants saw the list of podcast titles and half of them heard the same list spoken out loud. They were then asked questions about whether they liked the podcast they’d chosen.

The researchers found listeners were far more likely to choose one of the first choices offered, while people who read the choices explored six times more deeply into the list of recommendations. People reading their choices also did more skimming and browsing.

Recommendation algorithms generally prioritize popular content, potentially creating an echo-chamber effect, Yang said. In the study, people who read their recommendations were less likely to choose the most popular or top-rated options.

There was no statistical difference in how much people from either group enjoyed the podcasts they chose.

“One important problem with these kinds of recommendation systems is that they selectively share information with users, so your information exposure is determined by what the system explicitly offers you,” Yang said. “In the web interface, you have the ability to browse, you can scroll and skim. You get a very broad and wide exposure to different kinds of information that’s out there. With voice, people don’t really have the patience or won’t really wait for so many items to decide what they want to consume.”

The paper was co-authored with senior author Deborah Estrin, associate dean and Robert V. Tishman ’37 Professor of Computer Science at Cornell Tech, Cornell Tech postdoctoral associate Michael Sobolev and Christina Tsangouri of the City University of New York. The experiment was part of the team’s broader research into the relationship between computer recommendations and people’s intentions when it comes to making choices.

The research was funded by the National Science Foundation and Oath, which is part of Verizon; and supported by Cornell Tech’s Connected Experiences and Small Data labs.

This article originally appeared in the Cornell Chronicle.