Immersive Marketing Insights

Google trains AI to recognize your hand movements

Researchers at Google are training algorithms for hyper-accurate hand and finger tracking, so our AR wearables of the future can ditch the handheld controller. We know that machine learning of facial gestures has advanced dramatically in recent years. Google has progressed on this far more challenging computer vision task, as hands lack the same contrast to distinguish features. The AI team revealed a demo for mobile devices that can instantly map up to 21 points of the hand and finger… and interpret your gestures for meaning. Their real-time performance on mobile phones, built in the MediaPipe framework, achieved an average rate of precision of nearly 96%. Its compatibility with iOS and Android is a critical accomplishment to build these algorithms into future AR wearables.

TG Takeaway: Microsoft’s HoloLens 2 is the leading wearable with hand tracking, enabling users to simply tap the digital objects in air to interact with them. Hand tracking is clearly an emerging user interface — the thought is that computer vision of hands and fingers will come to succeed our familiar tapping on the smartphone screen.

Read more: Google blog, Next Reality

DreamWorld AR dives back into wearables with 2D private theater

Rumors swirl and hype grows for big tech AR glasses, but plenty of smaller players are innovating and bringing this technology to market today. DreamWorld AR revealed its second wearable, the DreamGlass Air, which is a portable private theater for viewers to watch 2D content. Effectively it’s a head-mounted projector, allowing you to mirror your mobile phone content (wireless or with tether) onto an 100 inch augmented display with 90 degrees field of view. Its consumer viability is limited at this point, at a starting point of $489. Unclear why consumers would look to a 2.5K 2D display at a higher tag than an Oculus Quest. The company has well exceeded its Kickstarter goals for the product, which is scheduled to ship to backers in December.

TG Takeaway: We’ve written before that the supply of AR glasses is growing far faster than any consumer demand. But we flag technology like this as the start-ups building these products are collecting data and insights from consumers, while big tech companies hold their breath. The DreamGlass AR is a preview of how 2D content will be re-imagined in a spatial computing world — amazing, full-resolution video content needs to carve out its place in AR. Consider how wearables like the DreamGlass Air may enable second screen augmented experiences… or how projected displays might impact television hardware.

Read more: Next Reality, Kickstarter

Color in your Kellogg’s box and interact in AR

Kellogg’s and Crayon collaborated on a new line of black-and-white cereal boxes for customers to “color and win.” The package redesign is also equipped with a web-based AR activation. Here’s how it works:

1. Purchase any specially marked Kellogg’s product before 8/31/2019
2. Colour in the back of the pack
3. Upload a photo of your picture & tell us in 25 words or less what inspired your creation to win
4. Bring your picture to life with Augmented Reality on a mobile device

The boxes were rolled out in Australia in July, and the sweepstakes end on August 31. Australian goats: act fast!

TG Takeaway: “Color and Win” is a super clever use case of AR for CPG brands. We really believe in the potential of AR packaging to make grocery shopping more fun and engaging. Whether it’s in-store or at home, any box can be augmented with more advanced image tracking. In addition, we love to see WebAR as the platform of choice here — it has the most potential distribution and requires the least activation energy. CPG companies: contact us and let’s make this happen.

Read more: Campaign site, Mumbrella, Dieline

This week in branded AR filters (that we can find)

The latest additions to Snapchat featured filters are branded experiences from Cheetos and Coca Cola. Cheetos filter is a simple “Open your Mouth” interactive, whereas the Coke project has a more advanced UI menu to choose which team to represent for College Football kickoff. Swipe left on the main camera to see which filters Snapchat targets at you. If you find filters that we haven’t seen, send us a Snap! Haven’t yet seen targeted Spark AR filters, only a rapidly growing repository of user-generated content.

TG Takeaway: You can make these too! Contact us to build out a Social AR strategy. Reach the over 200 million daily active users that use the platform, 70% of which interact with lenses. Seriously: “Snapchat creators earn serious money from making AR filters,” CNN.

Snoop Dogg is still making records: his latest has an AR album cover

Favorite story of the week is Snoop Dogg’s foray into augmented reality. His whopping 20th album is here and equipped with an augmented cover. Open his Snapchat filter, point it at the album cover and see a 3D model of the rapper deliver his Walk of Fame speech “I Wanna Thank Me.” Scan the snap code below and hover over the album cover to try for yourself.

TG Takeaway: This project is both a super fun, simple activation and a flex of a Snapchat comparative advantage over Spark AR from Instagram. Snap has the best image marking technology of these Social AR services, allowing users to hover over only a corner of the image to recognize it and track the plane without glitches.

Read more: Next Reality

Mark Ronson’s latest music video is built for mobile, shot with Instagram AR Filters

Mark Ronson has been getting plenty of buzz lately after winning an Oscar for co-writing “Shallow” in A Star is Born. He partnered with Sony Music to lean into the mobile entertainment trend by posting his latest song, “Pieces of Us” featuring King Princess, with a 3+ minute music video on Instagram TV. It also is one of the best and most public uses of Spark AR filters on the platform in the less than one week of their public beta status. Watch the music video here. You’ll be one of over 90K to see it.

TG Takeaway: Next Reality writes that “The record label is planning to incorporate fan-generated content into the story, a move aimed to make the music video feel less like a static piece of art and more like a living breathing thing.” We love both the subtle use of AR filters for 2D entertainment and the user-generated aspect to this artwork — filter generation is open to everybody now and we can’t wait to see what people make. Brands need to capitalize on the growth of both IGTV and Spark AR filters, and Two Goats is here to help.

Read more: Next Reality

USA Today publishes 1619 augmented story, publishes AR tab in app

To observe 400 years since the beginning of the American slave trade, USA Today debuted a multimedia series featuring AR. The augmented story, accessible via the newly minted AR section of their app, opens on a portal to a 17th-century slave ship. Once inside, users walk around the virtual ship environment and trigger spatial audio stories by tapping on digital artifacts scattered throughout the ship. The cumulative effect of the audio stories is a sense of the dangerous journey in the Middle Passage aboard the San Juan Bautista boat. “The AR interactive was written by best-selling author Dolen Perkins-Valdez, narrated by storyteller Tamara Winfrey-Harris and showcases a painting by noted Hampton artist Richard Press Sr.” The project is the USA Today team’s ninth emerging tech story of 2019.

TG Takeaway: “1619: Searching for Answers” is a hugely important project and USA Today‘s multimedia approach is exactly the right one to convey the scope and shed light on the awful legacy of the slave trade. The immersive component of this series plays a part in giving users a more memorable and impactful glimpse into these old ships… something that archival paintings or later illustrations can’t quite rival. From a purely AR perspective, the use of portals and sophisticated spatial audio distinguish the story. Like the slew of AR projects made for the Apollo 11 anniversary, publications (the NYT too) had plenty of time to prepare and design these feature multimedia pieces: the jury is still out on how AR technology can be deployed fast and in the field for breaking news.

Read more: 1619 project, USA Today announcement

Hasbro buys Entertainment One, doubles down on multimedia content

Toymaker Hasbro made waves this week with its $4 billion purchase of eOne, a production company specializing in multimedia and children’s programming. “Hasbro will leverage eOne’s immersive entertainment capabilities to bring our portfolio of brands that have appeal to gamers, fans and families to all screens globally…” said Hasbro CEO Brian Goldner. eOne has invested in AR/VR entertainment, including a 2016 purchase of Secret Location. The move signals the benefits brands can unleash with investment in content based around their IP.

TG Takeaway: A toy company buying a content studio is massive news — the lines between content and experiences continue to blur. We hope to see AR activation come out of this combination, as playful and educational AR finds its stride. We covered products like the Shifu Orboot that bring toys to life with amazing 3D content.

Read more: Variety,  ANB Media (photo)