Pixel

A focus on portrait mode: behind the scenes with Pixel 2’s camera features

This week the Pixel 2 and Pixel 2 XL, Google’s newest smartphones, arrive in stores. Both devices come with features like Now Playing, the Google Assistant, and the best-rated smartphone camera ever, according to DXO.

We designed Pixel 2’s camera by asking how we can make the camera in your Pixel 2 act like SLRs and other big cameras, and we’ve talked before about the tech we use to do that (such as HDR+). Today we’re highlighting a new feature for Pixel 2’s camera: portrait mode.

With portrait mode, you can take pictures of your friends and family (that includes your pets too!) that keep what’s important sharp and in focus, but softly blur out the background. This helps draw your attention to the people in your pictures and keeps you from getting distracted by what’s in the background. This works on both Pixel 2 and Pixel 2 XL, on both the rear- and front-facing cameras.

girl-with-the-orange-hat-s.jpg

Pictures without (left) and with (right) portrait mode. Photo by Matt Jones

Technically, blurring out the background like this is an effect called “shallow depth of field.” The big lenses on SLRs can be configured to do this by changing their aperture, but smartphone cameras have fixed, small apertures that produce images where everything is more or less sharp. To create this effect with a smartphone camera, we need to know which parts of the image are far away in order to blur them artificially.

Normally, to determine what’s far away with a smartphone camera you’d need to use two cameras close to each other, then triangulate the depth of various parts of the scene—just like your eyes work. But on Pixel 2 we’re able to combine computational photography and machine learning to do the same with just one camera.

How portrait mode works on the Pixel 2

Portrait mode starts with an HDR+ picture where everything is sharp and high-quality.

Next, our technology needs to decide which pixels belong to the foreground of the image (a person, or your dog) and which belong to the background. This is called a “segmentation mask” and it’s where machine learning comes in. We trained a neural network to look at a picture and understand which pixels are people and which aren’t. Because photos of people may also include things like hats, sunglasses, and ice cream cones, we trained our network on close to a million pictures—including pictures with things like those!

Just creating two layers—foreground and background, with a hard edge in between them—isn’t quite enough for all pictures you’d want to take; SLRs produce blur that gets stronger with each fraction of an inch further from the thing that’s in sharp focus. To recreate that look with Pixel 2’s rear camera, we use the new Dual Pixel sensor to look through the left and right sides of the camera’s tiny lens at the same time—effectively giving us two cameras for the price of one. Using these two views we compute a depth map: the distance from the camera to each point in the scene. Then we blur the image based on the combination of the depth map and the segmentation mask.

The result? Portrait mode.

girl-with-coffee.jpg

Pictures without (left) and with (right) portrait mode. Photo by Sam Kweskin

Portrait mode works a little differently on the front-facing camera, where we aren’t able to produce a depth map the same way we do with the more powerful rear-facing camera. For selfies, we just use our segmentation mask, which works particularly well for selfies since they have simpler compositions.

bikeride-selfie-comp-s.jpg

Selfie without (left) and with (right) portrait mode. The front-facing camera identifies which background pixels to blur using only machine learning—no depth map. Photo by Marc Levoy

When and how to use portrait mode

Portrait mode on the Pixel 2 is automatic and easy to use—just choose it from your camera menu then take your picture. You can use it for pictures of your friends, family, and even pets. You can also use it for all kinds of “close-up” shots of objects such flowers, food, or bumblebees (just don’t get stung!) with background blur.

flower-comp-s.jpg

Close-up picture without (left) and with (right) portrait mode. Photo by Marc Levoy

Here are some tips for how to take great portraits using any camera (and Pixel 2 as well!):

  • Stand close enough to your subjects that their head (or head and shoulders) fill the frame.
  • For a group shot where you want everyone sharp, place them at the same distance from the camera.
  • Put some distance between your subjects and the background.
  • For close-up shots, tap to focus to get more control over what’s sharp and what’s blurred. Also, the camera can’t focus on things closer than several inches, so stay at least that far away.

To learn more about portrait mode on the Pixel 2, watch this video by by Nat & Friends, or geek out with our our in-depth, technical post over on the Research blog.

Adobe MAX

Adobe Enhances Partner Ecosystem at MAX

Today we’re excited to announce new advancements across our global partner ecosystem including an extended partnership with Tata Consultancy Services and the introduction of the new Adobe Exchange marketplace.

Machine Learning

Fighting phishing with smarter protections

Editor’s note: October is Cybersecurity Awareness Month, and we’re celebrating with a series of security announcements this week. This is the third post; read the first and second ones.

Online security is top of mind for everyone these days, and we’re more focused than ever on protecting you and your data on Google, in the cloud, on your devices, and across the web.

One of our biggest focuses is phishing, attacks that trick people into revealing personal information like their usernames and passwords. You may remember phishing scams as spammy emails from “princes” asking for money via wire-transfer. But things have changed a lot since then. Today’s attacks are often very targeted—this is called “spear-phishing”—more sophisticated, and may even seem to be from someone you know.

Even for savvy users, today’s phishing attacks can be hard to spot. That’s why we’ve invested in automated security systems that can analyze an internet’s-worth of phishing attacks, detect subtle clues to uncover them, and help us protect our users in Gmail, as well as in other Google products, and across the web.

Our investments have enables us to significantly decrease the volume of phishing emails that users and customers ever see. With our automated protections, account security (like security keys) and warnings, Gmail is the most secure email service today.

Here is a look at some of the systems that have helped us secure users over time, and enabled us to add brand new protections in the last year.

More data helps protect your data

The best protections against large-scale phishing operations are even larger-scale defenses. Safe Browsing and Gmail spam filters are effective because they have such broad visibility across the web. By automatically scanning billions of emails, webpages, and apps for threats, they enable us to see the clearest, most up-to-date picture of the phishing landscape.

We’ve trained our security systems to block known issues for years. But, new, sophisticated phishing emails may come from people’s actual contacts (yes, attackers are able to do this), or include familiar company logos or sign-in pages. Here’s one example:

Screenshot 2017-10-11 at 2.45.09 PM.png

Attacks like this can be really difficult for people to spot. But new insights from our automated defenses have enabled us to immediately detect, thwart and protect Gmail users from subtler threats like these as well.

Smarter protections for Gmail users, and beyond

Since the beginning of the year, we’ve added brand new protections that have reduced the volume of spam in people’s inboxes even further.

  • We now show a warning within Gmail’s Android and iOS apps if a user clicks a link to a phishing site that’s been flagged by Safe Browsing. These supplement the warnings we’ve shown on the web since last year.
safelinks.png
  • We’ve built new systems that detect suspicious email attachments and submit them for further inspection by Safe Browsing. This protects all Gmail users, including G Suite customers, from malware that may be hidden in attachments.
  • We’ve also updated our machine learning models to specifically identify pages that look like common log-in pages and messages that contain spear-phishing signals.

Safe Browsing helps protect more than 3 billion devices from phishing, across Google and beyond. It hunts and flags malicious extensions in the Chrome Web Store, helps block malicious ads, helps power Google Play Protect, and more. And of course, Safe Browsing continues to show millions of red warnings about websites it considers dangerous or insecure in multiple browsers—Chrome, Firefox, Safari—and across many different platforms, including iOS and Android.

pastedImage0 (5).png

Layers of phishing protection

Phishing is a complex problem, and there isn’t a single, silver-bullet solution. That’s why we’ve provided additional protections for users for many years.

pasted image 0 (5).png
  • Since 2012, we’ve warned our users if their accounts are being targeted by government-backed attackers. We send thousands of these warnings each year, and we’ve continued to improve them so they are helpful to people. The warnings look like this.
  • This summer, we began to warn people before they linked their Google account to an unverified third-party app.
  • We first offered two-step verification in 2011, and later strengthened it in 2014 with Security Key, the most secure version of this type of protection. These features add extra protection to your account because attackers need more than just your username and password to sign in.

We’ll never stop working to keep your account secure with industry-leading protections. More are coming soon, so stay tuned.

Adobe After Effects CC

Redefining Modern Creativity with the Next Generation of Creative Cloud

By introducing a range of new applications in Creative Cloud, along with significant innovations across our flagship tools, we’re enabling creative professionals and enthusiasts alike to express themselves with apps and services that connect across devices, platforms and geographies.

Adobe XD

Welcome Adobe XD CC

Today we announce the 1.0 release of Adobe XD. Since the first beta release, we have been laser-focused on building a solution that enables fast-paced, iterative design.

Digital Imaging

Adobe and Coca-Cola Go for Gold to Support Special Olympics

Adobe and Coca-Cola have invited the global creative community to create a work of art inspired by Coca-Cola and their iconic branding. By providing special branded assets and a unique creative brief, participants can build a piece of art and share it with the world.

Adobe XD Product Updates

What’s Next for Adobe XD CC?

Upcoming XD roadmap for the near future fo Adobe XD, covering four categories of features: Design, prototype, share, and 3rd-party integrations.

Adobe MAX

Adobe Delivers Modern, Intelligent Design Solutions for Global Brands and Agencies

The path to digital transformation is both exhilarating and daunting. The opportunity for brands has never been greater through digital and mobile customer relationships, but successful digital transformation requires brands to work differently. Adobe is addressing this with several exciting announcements at the Adobe MAX conference today.

Adobe MAX

Introducing the Master Artists Motion Graphics Template Collection

We are proud to announce the addition of Motion Graphics Templates to Adobe Stock. Hear from our contributors Andrew Kramer, Nik Hill, and Valentina Vee about why motion graphics are important for all video editors and film makers.

8nm LPP

Samsung Completes Qualification of 8nm LPP Process

Samsung Electronics, a world leader in advanced semiconductor technology, announced today that 8-nanometer (nm) FinFET process technology, 8LPP (Low Power

Education

Making computer science accessible to more students in Africa

Computer Science (CS) fosters innovation, critical thinking and empowers students with the skills to create powerful tools to solve major challenges. Yet, many students, especially in their the early years, do not have access to opportunities to develop their technical skills.

At Google, we believe that all students deserve these opportunities. That is why,  in line with our commitment to prepare 10 million people in Africa for jobs of the future, we are funding 60 community organisations to hold training workshops during Africa Code Week 2017.These workshops will give over 50,000 students a chance to engage with CS and learn programming and computational-thinking skills.

Africa Code Week is a grassroots movement that encourages programming by showing how to bring ideas to life with code, demystifying these skills and bringing motivated students together to learn. Google has been involved in this campaign as a primary partner to SAP since 2015, providing sponsorships to organizations running initiatives to introduce students to CS.

This year, we received more than 300 applications from community organizations across Africa. We worked with the Cape Town Science Centre to select and fund 60 of these organizations that will deliver CS workshops to children and teens (ages 8 to 18) from October 18-25 in 10 African countries (Botswana, Cameroon, Ethiopia, Ghana, Kenya, Lesotho, Nigeria, South Africa, Gambia and Togo).

Some of the initiatives we are supporting include:

Google is delighted to support these great efforts. Congratulations to the recipient organizations. Step into the world of Google in Computer Science Education at edu.google.com/cs.

Adobe Illustrator CC

Thousands of Apps for Any Task

Designed to complement your Creative Cloud workflow and fill in gaps with specialized features for the task at hand, there’s little you can’t do on the Creative Cloud platform with 3rd party apps.

Adobe After Effects CC

Live Stream Series | Make Good Videos GREAT with Motion Graphics

Jason Levine launches the first of three, 3-week interactive live stream series to show how to take your video creation skills to the next level. First up: Motion Graphics – starting on October 27, 2017.

comcast

Our Thoughts On John Chapman Being Inducted Into the Cable Pioneers This Week

John will be inducted into the Cable Pioneers at a dinner event on Tuesday, October 18, at the Brown Palace Hotel. Our deepest congratulations, John!

cisco industrial ethernet

Join Us for a Meeting of the Minds and Machines

Next week Cisco will be exhibiting at the 2017 GE Minds + Machines conference in San Francisco (October 25 and 26). Cisco is a platinum sponsor, and if you’ve never attended this conference, it’s one of the premier Industrial IoT (IIoT) events. Topics of discussion will include: IIoT architecture IIoT data management and analytics Industrial control […]

Scroll Up