By Sean Kelly, Product Management Director, Messenger Over the holidays we all want to take time to celebrate, reflect and stay in touch with the people we love. It’s been 25 years since the first text message was sent, sparking a revolution in how we keep in touch with each other, and we’ve come a […]
Over the past few years, many people have experienced virtual reality with headsets like Cardboard, Daydream View, and higher-end PC units like Oculus Rift and HTC Vive. Now, augmented reality has the potential to reach people right on their mobile devices. AR can bring information to you, and that digital information can enhance the experience you have with their physical space. However, AR is new, so creators need to think carefully when it comes to designing intuitive user interactions.
From our own explorations, we’ve learned a few things about design patterns that may be useful for creators as they consider mobile AR platforms. For this post, we revisited our learnings from designing for head-mounted displays, mobile virtual reality experiences, and depth-sensing augmented reality applications. First-party apps such as Google Earth VR and Tilt Brush allow users to explore and create with two positionally-tracked controllers. Daydream helped us understand the opportunities and constraints for designing immersive experiences for mobile. Mobile AR introduces a new set of interaction challenges. Our explorations show how we’ve attempted to adapt emerging patterns to address different physical environments and the need to hold the phone throughout an entire application session.
Key design considerations
Mobile constraints. Achieving immersive interactions is possible through a combination of the device’s camera, real-world coordinates for digital objects, and input methods of screen-touch and proximity. Since mobile AR experiences typically require at least one hand to hold the phone at all times, it’s important for interactions to be discoverable, intuitive, and easy to achieve with one or no hands. The mobile device is the user’s window into the augmented world, so creators must also consider ways to make their mobile AR experiences enjoyable and usable for varying screen sizes and orientations.
Mobile mental models and dimension-shifts. Content creators should keep in mind existing mental models of mobile AR users. 2D UI patterns, when locked to the user’s mobile screen, tend to lead to a more sedentary application experience; however, developers and designers can get creative with world-locked UI or other interaction patterns that encourage movement throughout the physical space in order to guide users toward a deeper and richer experience. The latter approach tends to be a more natural way to get users to learn and adapt to the 3D nature of their application session and more quickly begin to appreciate the value a mobile AR experience has to offer — such as observing augmented objects from many different angles.
Environmental considerations. Each application has a dedicated “experience space,” which is a combination of the physical space and range of motion the experience requires. Combined with ARCore’s ability to detect varying plane sizes or overlapping planes at different elevations, this opens the door to unique volumetric responsive design opportunities that allow creators to determine how digital objects should react or scale to the constraints of the user’s mobile play space. Visual cues like instructional text or character animations can direct users to move around their physical spaces in order to reinforce the context switch to AR and encourage proper environment scanning.
Visual affordances. Advanced screen display and lighting technology makes it possible for digitally rendered objects to appear naturally in the user’s environment. Volumetric UI patterns can complement a 3D mobile AR experience, but it’s still important that they stand out as interactive components so users get a sense of selection state and functionality. In addition to helping users interact with virtual objects in their environment, it’s important to communicate the planes that the mobile device detects in order to manage the users’ expectations for where digital items can be placed.
Mobile AR 2D interactions. With mobile AR, we’ve seen applications of a 2D screen-locked UI which gives users a “magic-hand” pattern to engage with the virtual world via touch inputs. The ability to interact with objects from a distance can be very empowering for users. However, because of 2D UI patterns’ previous association with movement-agnostic experiences, users are less likely to move around. If physical movement is a desired form of interaction, mobile AR creators can consider ways to more immediately use plane detection, digital object depth, and phone-position to motivate exploration of a volumetric space. But be wary of too much 2D UI, as it can break immersion and disconnect the user from the AR experience.
Mobile AR immersive interactions. To achieve immersion, we focused on core mobile AR interaction mechanics ranging from object interaction, browsing, information display, and visual guidance. It’s possible to optimize for readability, usability, and scale by considering ways to use a fixed position or dynamic scaling for digital objects. Using a reticle or raycast from the device is one way to understand intent and focus, and designers and developers may find it appropriate to have digital elements scale or react based on where the camera is pointing. Having characters react with an awareness to how close the user is, or revealing more information about an object as a user approaches, are a couple great examples of how creators can use proximity cues to reward exploration and encourage interaction via movement.
These are some early considerations for designers. Our team will be publishing guidelines for mobile AR design soon. There are so many unique problems that mobile AR can solve and so many delightful experiences it can unlock. We’re looking forward to seeing what users find compelling and sharing what we learn along the way, too. In the meantime, continue making and breaking things!
Images in this post by Chris Chamberlain
In October 2015, as part of our Digital News Initiative (DNI)—a partnership between Google and news publishers in Europe to support high-quality journalism through technology and innovation—we launched the €150 million DNI Innovation Fund. Today, we’re announcing the recipients of the fourth round of funding, with 102 projects in 26 European countries being offered €20,428,091 to support news innovation projects. This brings the total funding offered so far to €94 million.
In this fourth round, we received 685 project submissions from 29 countries. Of the 102 projects funded today, 47 are prototypes (early stage projects requiring up to €50,000 of funding), 33 are medium-sized projects (requiring up to €300,000 of funding) and 22 are large projects (requiring up to €1 million of funding).
In the last round, back in July, we saw a significant uptick in interest in fact checking projects. That trend continues in this round, especially in the prototype project category. In the medium and large categories, we encouraged applicants to focus on monetization, which led to a rise in medium and large projects seeking to use machine learning to improve content delivery and transform more readers into subscribers. Overall, 21 percent of the selected projects focus on the creation of new business models, 13 percent are about improving content discovery by using personalisation at scale. Around 37 percent of selected projects are collaborations between organizations with similar goals. Other projects include work on analytics measurement, audience development and new advertising opportunities. Here’s a sample of some of the projects funded in this round:
[Prototype] Stop Propaghate – Portugal
With €49,804 of funding from the DNI Fund, Stop Propaghate is developing an API supported by machine learning techniques that could help news media organizations 1) automatically identify if a portion of news reporting contains hate speech, and 2) predict the likelihood of a news piece to generate comments containing hate speech. The project is being developed by the Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), a research & development institute located at University of Porto in Portugal.
[Medium] SPOT – France
Spot is an Artificial Intelligence-powered marketplace for curating, translating and syndicating valuable articles among independent media organizations, and is being developed by VoxEurop, a European news and debate website. With €281,291 of funding from the DNI Innovation Fund, Spot will allow publishers to easily access, buy and republish top editorial from European news organizations in their own languages, using AI data-mining technologies, summarization techniques and automatic translation technologies, alongside human content curation.
[Large] ML-based journalistic content recommendation system – Finland
Digital news media companies produce much more content than ever reaches their readers, because existing content delivery mechanisms tend to serve customers en masse, instead of individually. With €490,000 of funding from the DNI Innovation Fund, Helsingin Sanomat will develop a content recommendation system, using machine learning technologies to learn and adapt according to individual user behavior, and taking into account editorial directives.
The recipients of fourth round funding were announced at a DNI event in London, which brought together people from across the news industry to celebrate the impact of the DNI and Innovation Fund. Project teams that received funding in Rounds 1, 2 or 3 shared details of their work and demonstrated their successes in areas like local news, fact checking and monetization.
Since February 2016, we’ve evaluated more than 3,700 applications, carried out 935 interviews with project leaders, and offered 461 recipients in 29 countries a total of €94 million. It’s clear that these projects are helping to shape the future of high-quality journalism—and some of them are already directly benefiting the European public. The next application window will open in the spring. Watch out for details on the digitalnewsinitiative.com website and check out all DNI funded projects!
As 2017 draws to a close, it’s time to look back on the year that was with our annual Year in Search. As we do every year, we analyzed Google Trends data to see what the world was searching for.
2017 was the year we asked “how…?” How do wildfires start? How to calm a dog during a storm? How to make a protest sign? In fact, all of the “how” searches you see in the video were searched at least 10 times more this year than ever before. These questions show our shared desire to understand our experiences, to come to each other’s aid, and, ultimately, to move our world forward.
Many of our trending questions centered around the tragedies and disasters that touched every corner of the world. Hurricanes devastated the Caribbean, Houston and Florida. An earthquake struck Mexico City. Famine struck Somalia, and Rohingya refugees fled for safety. In these moments and others, our collective humanity shined as we asked “how to help” more than ever before.
We also searched for ways to serve our communities. People asked Google how to become police officers, paramedics, firefighters, social workers, activists, and other kinds of civil servants. Because we didn’t just want to help once, we wanted to give back year round.
Searches weren’t only related to current events—they were also a window into the things that delighted the world. “Despacito” had us dancing—and searching for its meaning. When it came to cyberslang like “tfw” and “ofc,” we were all ¯\_(ツ)_/¯. And, finally, there was slime. We searched how to make fluffy, stretchy, jiggly, sticky, and so many more kinds of slime….then we searched for how to clean slime out of carpet, and hair, and clothes.
From “how to watch the eclipse” and “how to shoot like Curry,” to “how to move forward” and “how to make a difference,” here’s to this Year in Search. To see the top trending lists from around the world, visit google.com/2017.
Since becoming a professor 12 years ago and joining Google a year ago, I’ve had the good fortune to work with many talented Chinese engineers, researchers and technologists. China is home to many of the world’s top experts in artificial intelligence (AI) and machine learning. All three winning teams of the ImageNet Challenge in the past three years have been largely composed of Chinese researchers. Chinese authors contributed 43 percent of all content in the top 100 AI journals in 2015—and when the Association for the Advancement of AI discovered that their annual meeting overlapped with Chinese New Year this year, they rescheduled.
I believe AI and its benefits have no borders. Whether a breakthrough occurs in Silicon Valley, Beijing or anywhere else, it has the potential to make everyone’s life better for the entire world. As an AI first company, this is an important part of our collective mission. And we want to work with the best AI talent, wherever that talent is, to achieve it.
That’s why I am excited to launch the Google AI China Center, our first such center in Asia, at our Google Developer Days event in Shanghai today. This Center joins other AI research groups we have all over the world, including in New York, Toronto, London and Zurich, all contributing towards the same goal of finding ways to make AI work better for everyone.
Focused on basic AI research, the Center will consist of a team of AI researchers in Beijing, supported by Google China’s strong engineering teams. We’ve already hired some top experts, and will be working to build the team in the months ahead (check our jobs site for open roles!). Along with Dr. Jia Li, Head of Research and Development at Google Cloud AI, I’ll be leading and coordinating the research. Besides publishing its own work, the Google AI China Center will also support the AI research community by funding and sponsoring AI conferences and workshops, and working closely with the vibrant Chinese AI research community.
Humanity is going through a huge transformation thanks to the phenomenal growth of computing and digitization. In just a few years, automatic image classification in photo apps has become a standard feature. And we’re seeing rapid adoption of natural language as an interface with voice assistants like Google Home. At Cloud, we see our enterprise partners using AI to transform their businesses in fascinating ways at an astounding pace. As technology starts to shape human life in more profound ways, we will need to work together to ensure that the AI of tomorrow benefits all of us.
The Google AI China Center is a small contribution to this goal. We look forward to working with the brightest AI researchers in China to help find solutions to the world’s problems.
Once again, the science of AI has no borders, neither do its benefits.
Americans know the importance of Dec. 7, 1941, when a Japanese attack on Pearl Harbor resulted in the deaths of 2,400 Americans. Another Dec. 7 event is worth remembering, one that isn’t known by nearly as many people, yet is “connected in an interesting way to the events that were unleashed on Dec. 7, 1941,”…
The post Why Dec. 7 represents the forces that both divide and bring us together appeared first on The Official Microsoft Blog.
Craig Tranter is a former educator, and now serves as a technology presenter for Cisco. This blog is part of a series on advancements and opportunities in education. All views are his own. One thing that all teachers will be fairly familiar with by now is the use of Interactive White Boards (IWBs). However, the […]
Can you say “shake”? This week we’re making the introduction to some of #teampixel’s furry friends who always make the day a little brighter. From fabulous felines to a French Bulldog in PJs, scroll through this week’s “pawsome” picks and get to know the pets of #teampixel—13/10 would portrait mode again.
Want to get your Pixel photos featured on The Keyword? Make sure to tag your photos with #teampixel for the opportunity to see your photos here next!
Today, Talos is publishing a glimpse into the most prevalent threats we’ve observed between December 01 and December 08. As with previous round-ups, this post isn’t meant to be an in-depth analysis. Instead, this post will summarize the threats we’ve observed by highlighting key behavior characteristics, indicators of compromise, and how our customers are automatically […]
In a previous blog, we discussed the role of new product introduction processes (NPI) and the part that collaboration tools are playing in helping drive better ideation while speeding time to production. Similarly, collaboration tools are rapidly changing maintenance, repair, and operations (MRO) procedures on the factory floor. Downtime is always the antagonist of manufacturing. […]