Over one billion people—15 percent of the population—live with some kind of disability, and this number will continue to rise as people get older and live longer. At Google I/O this week, we shared a few new ways that we’re helping people with disabilities. Here’s a bit more about these new products, as well as a behind-the-scenes look at how we designed I/O to make it more accessible and enjoyable for everyone:

Lookout:

Lookout is a new Android app designed to help people who are blind or visually impaired gain more independence by giving auditory cues about objects, text and people around them. People simply wear a Pixel device on a lanyard around their neck, with the camera pointing away from their body, and the app shares relevant information about the things around them, as they move through a space. Lookout is a big step in an effort to use technology to make the ever-changing and evolving world around us more tangible to people. It uses AI technology to bridge the virtual world with the physical world, making day to day tasks and interactions a little easier.

Now, people who communicate using Morse code can do so on Gboard. To do this, we collaborated closely with Tania Finlayson, who was born with cerebral palsy and is an expert in Morse code assistive technology. Tania has been using Morse code to communicate since the 1980s, and she’s also the designer and co-developer of the TandemMaster. Her insights into the nuances of Morse code as an alternative assistive technology were invaluable throughout the design process, and by bringing Morse code to Gboard, we hope that more people might also be able to use Morse to communicate more freely. To get Morse for Gboard beta and to learn how to type Morse code, go to g.co/morse. This feature is currently available in the public beta version of Gboard, and will roll out more widely on Gboard for Android in the coming weeks.

YouTube Live Automatic Captions

In February, we announced that YouTube is bringing English automatic captions to live streams, and have been slowly rolling it out. With our new live automatic captions, creators have a quick and inexpensive way to make live streams more accessible to more people. With our speech recognition (LASR) technology, you’ll get captions with error rates and latency approaching industry standards.

Also at I/O, we introduced more features that developers can use to create more accessible app experiences for users with disabilities, including new accessibility testing, best practices and APIs for Android P.

Time and time again, we’ve seen the benefits of not just designing for one person or one community, but with them. By working together, we can truly make technology more available and useful to everyone.

There are over 253 million blind or visually impaired people in the world. To make the world more accessible to them, we need to build tools that can work with the ever-changing environment around us. Our new Android app Lookout, coming to the Play Store in the U.S this year, helps people who are blind or visually impaired become more independent by giving auditory cues as they encounter objects, text and people around them.

We recommend wearing your Pixel device in a lanyard around your neck, or in your shirt pocket, with the camera pointing away from your body. After opening the app, and selecting a mode, Lookout processes items of importance in your environment and shares information it believes to be relevant—text from a recipe book, or the location of a bathroom, an exit sign, a chair or a person nearby. Lookout delivers spoken notifications, designed to be used with minimal interaction allowing people to stay engaged with their activity.

There are four modes to choose from within the app: Home, Work & Play, Scan or Experimental (this allows you to test out features we’re working on). When you select a specific mode, Lookout will deliver information that’s relevant to the selected activity. If you’re getting ready to do your daily chores you’d select “Home” and you’ll hear notifications that tell you where the couch, table or dishwasher is. It gives you an idea of where those objects are in relation to you, for example “couch 3 o’clock” means the couch is on your right. If you select “Work & Play” when heading into the office, it may tell you when you’re next to an elevator, or stairwell.  As more people use the app, Lookout will use machine learning to learn what people are interested in hearing about, and will deliver these results more often.

The core experience is processed on the device, which means the app can be used without an internet connection. Accessibility will be an ongoing priority for us, and Lookout is one step in helping blind or visually impaired people gain more independence by understanding their physical surroundings.