Last week took place annual developer conference Google I/O held online by Google. New updates to the existing products as well as “soon to be available” software and tools were announced during the 3-day event.

Here is the list of most important updates:

• Smart Canvas on Google Docs
• Google Meet immersive experience
• Development of LaMDA (Language Model for Dialogue Applications)
• Multitask Unified Model (MUM) for a more advanced search
• Google Lens
• AR objects in Google Search
• “About this result” feature
• Google Maps
• Google Shopping
• TPU v4 - new generation of Tensor Processing Units
• Little Patterns and Cinematic Moments on Google Photos
• More accurate and inclusive cameras
• Android
• Safety & Security
• Wear OS
• Google Health
• Project Starline

Smart Canvas on Google Docs

Till the end of 2021 Google promises to launch Smart Canvas to deliver the next evolution of collaboration for Google Workspace. It is going to enhance, already familiar to everybody, Docs, Sheets and Slides to make them more flexible and interactive. 

What will change:

• New interactive building blocks (smart chips, templates, and checklists);
• New smart chips for recommended files and meetings. Type "@" to see a list of recommended people, files, and meetings and to insert smart chips into work. The feature is already available for Docs and in the upcoming months will be added also to Sheets.
• The new pageless format in Docs. Thanks to this users will be able to remove boundaries from documents and adjust the content to the device or screen they are using.
• New views in Sheets to better manage and interact with data. The first launch will be of Timeline view. This feature will allow  easily track different tasks, marketing campaigns, project milestones, schedules, and cross-team collaborations.

Google I/O 2021: Timeline view in Sheet

• Presenting content to a Google Meet directly from the Doc, Sheet, or Slide where the user is already working with the team (already released). And starting from autumn 2021, Google promises to bring Meet directly to Docs, Sheets, and Slides on the web, so people can actually see and hear each other while they’re collaborating.
• Creating and Editing Slides from Google Chat rooms as it is already available in Docs and Sheets.
• Introduction of emoji reactions in Docs (coming in the next few months).
• Introduction of additional assisted writing features in Docs. This includes stylistic suggestions, warnings about offensive language, etc.
• More assisted analysis functionality in Sheets to make it easier for everyone to derive insights from data.

Google I/O 2021: More assisted analysis functionality view in Sheets

• Creation of additional APIs in order to bring the information and actions from third-party tools directly into smart canvas elements like smart chips, checklists, and table templates.

Click here to read more about Smart Canvas.

Google Meet immersive experience

Besides the implementation of Google Meet directly into Docs, Sheets and Slides later this year, there are a few more important updates in Google Meet.

• Companion mode gives each person in the office their own video tile so they can stay connected to the remote colleagues and everyone can participate in polls, chat and Q&A in real-time.
• Noise cancellation during online meetings.
• Machine learning in Google Meet to automatically adjust camera zoom and lightning.
• Customized views - a presenter can easily arrange people’s faces to follow their reactions while staying focused on the content.

Google I/O 2021: Customized views in Google Meet

• Live captions and translations. Already now we are able to use captions in 5 languages but soon, starting with English-language, Google is going to introduce live captions translated into Spanish, Portuguese, French, or German.

Development of LaMDA (Language Model for Dialogue Applications)

LaMDA is a conversational language model for more natural conversations with computers.

Google I/O 2021: LaMDA answers the questions as if it is a paper plane

Google I/O 2021: LaMDA answers the questions as if it is a planet Pluto

So far, it is trained only on the text so it is important to create a Multimodal model that would allow people to naturally ask questions across different types of information.

Google I/O 2021: Multimodal model of LaMDA

Sundar Pichai, CEO of Google, noted that LaMDA is still under development and may give meaningless responses during a conversation but still it has great potential. 

For example, Google is looking forward to introducing better conversational features into Google Assistant, Search and Workspace.

Multitask Unified Model (MUM) for a more advanced search

Google would like to provide helpful experience across Google Search, Shopping and Photos.

For this reason, Google started the development of a new mechanism called Multitask Unified Model (MUM). 

MUM is built on the transformer architecture like BERT. However, it is 1000x more powerful than BERT.

Google I/O 2021: MUM's characteristics

It would allow users to do more complex queries just with one search.

Here is an example of the query MUM would be able to process:

“I’ve hiked Mt. Adams and now want to hike Mt. Fuji next fall, what should I do differently to prepare”.

With its ability to multitask, deep knowledge of the world, possibility to analyze and process different types of information, knowledge of different languages, MUM will be able to find and translate the information in order to give a comprehensive answer even for such specific questions.

Moreover, with MUM users will be able to take a photo of the boots and ask if they are appropriate shoes for hiking on Mount Fuji.

Google I/O 2021: Example of a Search request in MUM

Google Lens

Google is introducing a new capability that combines visual translation with educational content which is available in more than 100 languages. 

For example, now students can take a photo of a science problem and Lens will provide learning resources in the preferred language.

AR objects in Google Search

In 2019 Google introduced Augmented Reality into Search with 3D AR animals. This year Google adds animated athletes to the Search. 

Now we would be able just by pointing our phones on the flat surface to see demonstrations of athletes and their techniques in Search.

Google I/O 2021: Augmented Reality of animated athletes in Search

“About this result” feature

Earlier this year Google launched the “About this result” feature in Search to evaluate the credibility of the source of the information. 

By clicking on three dots next to the search result, we can see details about the site, when it was first indexed in Google and if the connection to the site is secure.

Google I/O 2021: “About this result” feature in Search

Already in May 2021, Google promised to launch “About this result” to all English searches globally with more languages to follow. 

Later this year more details will be added like:

• how the site describes itself;
• what other sources are saying about it;
• related articles to check out.

Google Maps

Live View expansion

Users will be able to access Live View right from the map and instantly see details about the shops and the cafes around including the information on how crowded a place is, reviews, photos of the most popular dishes, working hours, etc. This became possible because Google matches what the user's camera sees with rich information provided by the businesses on Google Maps. 

Here is what comes new to the Live View later this year:

• prominent virtual street signs

Google I/O 2021

• pointers towards key landmarks and important places like the direction to the hotel;

Google I/O 2021

• Live View indoors to help to navigate around such places as airports, transit stations or shopping centres

Google I/O 2021

Indoor Live View is already available in Zurich top stations and airports and coming next to Tokyo. 

Detailed street maps

Now Google Maps will include the information about where crosswalks, pedestrian islands, sidewalks are. Detailed street maps will be launched in 50 cities by the end of 2021. 

Tailored maps

Google Maps will be able to highlight only the most relevant information for the user at the moment. For example, if it is 8 am, Maps will display coffee shops and bakeries. While at 5 pm dinner restaurants will be displayed.

Google I/O 2021: Tailored Map in Google Maps for the search made at 8 am

Moreover, users will be able to see what places they have been to and receive suggestions of similar spots. 

Tailored maps will be launched in the coming weeks.

Area busyness

The feature that displays the busyness of the specific places like shops, restaurants or airports was very helpful for people during the last year. That’s why Google expands it to neighbourhoods with a feature called “Area busyness”. Now users will be able to see how crowded a specific area of the city is. 

The feature will be rolled out worldwide during the next few months. 

Google Shopping

More than a billion times per day people are shopping across Google. 

Shopping Graph is a comprehensive data set for billions of products and the merchants that sell them. 

Shopping Graph brings together information from websites, prices, reviews, videos and product data Google receives from the merchants directly.

Google I/O 2021: Number of listings in Google Shopping

The shopping Graph spans widely across Google to make it easier and faster to make a purchase.

Now, when users do the screenshot of some product they like there would be a suggestion to search the photo with Lens and they will see organic results for such item along with similar products. 

Also earlier this year Google made it possible to view products directly from YouTube videos. 

Google I/O 2021: "View Products" feature on YouTube

The feature is in pilot now, so more updates may come soon.

Another interesting update coming soon is the possibility to see opened carts during the last few weeks when a user opens a new tab. 

Google I/O 2021: "Opened carts" feature

Moreover, it will also be possible to find promotions and discounts if the person chooses to opt-in.

Google I/O 2021: Display of promotions and discounts in the "Open carts" feature

Soon Google will be using favorite loyalty programs for retailers like Sephora or Target to show the best purchase options. 

For example, if the person qualifies for the promotion but is not ready to buy he or she can enable Price Tracking to receive price drop notifications. 

Google I/O 2021: "Price Tracking" feature in Google Shopping

Google Shopping launches a seamless integration with Shopify. Now, just with a few clicks retailers on Shopify can appear in Google’s Shopping journey (Search, Maps, Images, Lens, YouTube).

TPU v4 - new generation of Tensor Processing Units

TPUs (Tensor Processing Units) are hardware accelerators specialized in deep learning tasks. TPUs are connected together into supercomputers called pods. A single V4 pod contains 4096 V4 chips. And each pod has 10x the interconnect bandwidth per chip at scale compared to any other networking technology. As a result, TPU v4 is more than 2x faster than TPU v3.

Google I/O 2021: TPU v3 VS TPU v4

It is the fastest system that was ever deployed in Google. TPU infrastructure is used for Google’s own AI and also is rented by Cloud customers. And TPU v4 pods will be available for Google Cloud customers till the end of the year. 

Such achievement was a great accomplishment but it is just a beginning. The long-term goal is to create an error-corrected quantum computer. The success would mean an increased battery efficiency, more sustainable energy, improved drug discovery, etc.

Little Patterns and Cinematic Moments on Google Photos

Little Patterns is a new feature that will find the pattern in your Google Photos and present it as a set of photos to give a new meaning to those moments. 

The feature uses machine learning to translate photos into a series of numbers and then compares how visually or conceptually similar are those pictures. When it finds three or more photos that have color or shape in common they will be surfaced as a pattern. 

Google I/O 2021: "Little Patterns" feature in Google Photos

Machine Learning is used also for Cinematic Moments. This feature allows Google Photo users to create a vivid moving image using a few similar photos of the same shot.

Cinematic Moments will take these almost identical pictures and use neural networks to synthesize the moment between image A and B. The technology interpolates the photos and fills in the gaps by creating new frames.

Google I/O 2021: "Cinematic Moments" feature in Google Photos

This can work on any image no matter on which device or how they were taken. 

Machine learning enables regular users to do an effect in a few seconds that would take a professional animator many hours.

Also, Google Photo users would be able to see the collection of memories in the photo Greed based on the occasion they were celebrating. 

Google I/O 2021: Collection of memories in the photo Greed based on the occasion - Google Photo

Soon Google promises to give more controls over photos and memories to the user. 

More accurate and inclusive cameras

In order to render a better true colour of the darker skin tones, enhanced cameras will be implemented. Auto white balance adjustments and algorithmically reducing stray light will bring out natural brown tones and prevent the over brightening and desaturation of darker skin tones.  

Google I/O 2021: Auto white balance effect

Android

Google I/O 2021

Material You

Material You is a new design that includes You (end-user) as a co-creator, that allows users to change the look of apps by generating personal material palettes and styles. 

Material You comes first to Google Pixel already this autumn, including all of Google Apps. And during the coming years, Material You concept will be introduced on Chrome OS, web, smart displays, wearables, etc. 

Android 12 Beta 1

The new version of Android was characterized as the most personal OS ever. The first public beta is being rolled out and from now available on Google Pixel, Xiaomi and OnePlus.

New UI for Android 

Inspired by Material You, Google overhauled everything starting from the lock screen to the settings. 

For example, if you change the wallpaper photo the system will create a palette based on the photo's colours and apply it to the elements of the screen.

Google I/O 2021: Android 12 - new UI

Design is more interactive now - dynamic lighting, customized Quick Settings, more intuitive notifications, Google Assistant, etc.

Privacy & Security

• Privacy Dashboard was created to show what type of data was accessed and when. 

Google I/O 2021

Dashboard reports on all apps on the phone including Google apps. Now users can manage permissions directly in the dashboard.

Google I/O 2021

• Also, an indicator was added to make it clear when the app is using your camera and microphone.

Google I/O 2021

• Two new toggles appear in the Quick Settings so you can completely disable the camera and mic access to every app.

Google I/O 2021

• Private Compute Core enables things like Now Playing (tells what song is playing in the background) and Smart Reply (suggests replies based on user’s reply patterns).

• Processing of all sensitive audio and video is isolated from the network and happens exclusively on the user's phone.

Integration with Chrome OS and Google TV

Users can connect their Androids with a Chromebook with one tap. Incoming chat notifications from the phone will be displayed in Chrome OS. 

In order to share a picture, users can access recent photos on their phones from their computers with one click.

Google I/O 2021

Google is also building a TV remote feature directly into the phone. 

The goal is to have all user’s devices work together and manage easily from the phone.

Expansion of Android Auto

Google I/O 2021: Number of Android Auto compatible cars

The most popular Car Manufactures will now support Android Auto wireless. 

Besides, Digital Car Key support was introduced. The feature will allow users to lock, unlock and start the car using a mobile phone. Moreover, they can remotely share the digital key of their cars with friends. Car Key will be launched this autumn with Google Pixels and Samsung Galaxy smartphones.

Safety & Security

Every day mail blocks more than 100 million fishing items.

Google Photos encrypts 4 billion photos.

Google Play Protect runs security scans on 100 million apps installed around the world.

And yet, it is sometimes not enough. One of the big problems is bad passwords.

Statistics show that  ⅔ of people use the same password across accounts. 

That’s why Google is on the way to creating a Password-free future.

Here are 4 new upgrades in the Password manager:

• Easier to get started with a simple tool that imports passwords saved in other password managers.
• Deeper integration between Chrome and Android so user’s secure passwords go with them from sites to apps.
• Automatic password alert if any of the saved passwords have become compromised in the third-party breach.
• Quick feature in Chrome where the Assistant will help to navigate to compromised accounts and change passwords.

Moreover, during the conference, it was also mentioned that users will get better control over their data. For example, now by tapping into profile picture, the user will be able to immediately delete recent search history:

Google I/O 2021

Google maps will also give more control to the users - they will be able to turn off and turn on location history in their timelines.

Google I/O 2021

Locked folder and photos - photos in this folder are private and won’t show up in the Google Photos or other apps on the device of the user. First will be released on Google Pixels and coming to more Android devices.

Wear OS

Merging with Samsung Tizen

Google I/O 2021

Integration of Wear OS and Samsung Tizen is focused on faster performance, longer battery life and thriving developer community.

This unified platform will be available for all device manufacturers. Developers can build apps using API and reach customers through the Google Play store.

UI redesign

It includes:

• New navigation.
• Expanding the collection of Tiles. With Tiles API any developer can create one. 
• Material design update and expanded capabilities (includes turn-by-turn navigation, Google Pay available in 37 countries and more than 200 public transit systems, downloading music in the YouTube music to listen offline without a phone nearby).

Integration with Fitbit

Some of the most popular features of the Fitbit will be now available also on Wear watches including health tracking and on-wrist celebrations. In the long term, it is planned to build premium smartwatches based on Wear that would combine Fitbit best features and Google's ambient computing capabilities.

Google Health

AI-powered dermatology assist tool uses machine learning to help find answers to the most common dermatological problems using a smartphone or a computer. Just by taking pictures of the skin problem from different angles and answering a few questions about symptoms, the user will receive in a few seconds a list of possible diagnosis. 

Google I/O 2021: AI-powered dermatology assist tool in Google Health

Building such technology requires the capability to analyze different skin types and tones. The feature will be available in the EU by the end of 2021 for Chrome OS users and will cover 288 skin conditions.

Project Starline

Project Starline is an experiment communication model developed by Google that will allow users to have a near in-person conversation. 

Thanks to high-resolution cameras and custom-built depth sensors, the technology captures a persons shape and appearance to create a detailed 3D model. Specially developed a light field display shows the realistic representation of the person like he or she sits in front. 

At the moment the technology is available in Google’s offices but it is planned to expand access to partners in healthcare and media.