Top

Connectedness: Virtual and Local

Smartware

The evolution of computing

December 18, 2017

As recently as 25 years ago, the physical reality in which we lived was an analog world that was becoming increasingly global. While globalization is still very much a factor today, our world is now decidedly connected and is becoming increasingly virtual. However, thanks to a combination of enabling technology and the possible impacts of global warming, some aspects of globalization are shifting back to being local. This connectedness—both virtual and local—is contributing to the emerging world of smartware.

As we detailed in “The Smartware Transformation,” smartware is a convergence of emerging technologies and science. Artificial intelligence (AI) is fueling its rise. The technologies that are enabling smartware include the Internet of Things (IoT), mixed-reality environments, and additive fabrication, or 3D printing, as are incredible advances in sciences such as genomics and neuroscience. Some or all of these advances are core to the emergence of incredible new products that are just over the horizon—products such as self-driving vehicles and neighborhood parts manufacturing. In “Smartware, AI, and Magical Products,” we took a look at the current darling of technology and entertainment media: artificial intelligence. We’ll continue that analysis in this installment, as we look at some other core smartware technologies, before covering the key sciences underlying smartware in our next column.

Champion Advertisement
Continue Reading…

The IoT and Data Collection

For those of us working in technology, the Internet of Things is old hat. Dirk was on the advisory board of an IoT company as early as 2006. For years, the IoT was mostly hype and promise, with the exception of the niche activity trackers that power nerds adopted. Once activity-tracking capabilities became mainstream, products such as the Nike+ FuelBand, shown in Figure 1, legitimized the category for a broader market beyond early adopters and galvanized the exercise community.

Figure 1—The Nike+ FuelBand
The Nike+ FuelBand

Image source: Nike

Smart home products such as the Nest from Google or Echo from Amazon have invaded our living spaces. However, today, the popular narrative about the IoT has seemingly stalled. Instead, the conversation about emerging technology has moved on to AI, with the IoT, if not forgotten, at least relegated to second-tier status. But, ironically, the primary reason AI has the potential to be world changing is precisely because of the Internet of Things.

What makes the IoT amazing is its ability to collect and provide real-time reporting of data in analog contexts. Pedometers have been around for more than 50 years. But they didn’t become interesting to anyone except the most hardcore exercise nuts until they became connected. While we’d been able to collect this type of data for a long time, the big advances were connecting these IoT products, using software to collect this data in more nuanced ways, and using the Internet to report this data in real time.

However, the most interesting parts of the IoT surface where data collection is as novel as data reporting. On the consumer side, security systems that use facial-recognition technology and interact with our phones provide examples of IoT products that are replacing dumb, analog security products. IoT security systems are making the most of the currently available technology and provide real-time data. Such solutions are now everywhere. For lovers of BBQ, there’s even a sensor-laden, IoT-enabled smoker that can monitor cooking progress in real time!

Still, many of the really exciting IoT innovations are happening quietly in business-to-business (B2B) markets, with what is being called the Industrial Internet of Things (IIoT). Machine suppliers are driving much of this. For example, in Germany, Siemens is working toward creating machines for their customers that support fully automated manufacturing. While achieving that aspiration requires the IoT, the way they already collect and report data is essential to approaching such a goal. Human involvement in such manufacturing systems is largely limited to monitoring IoT-style reporting. Robots replace humans on the assembly line, and a vast data-collection infrastructure oversees all of the processes. Similarly, in the United States, Caterpillar is proactively helping their customers not just to monitor their Caterpillar machines, but also to manage their fleets in a more strategic way. The companies that are manufacturing Industrial IoT equipment have installed systems that are generating massive troves of relevant data that, even within the last decade, we could not measure, much less collect, report, and organize.

Where does all of this tie back to AI? The current trend in machine learning feeds off of data. Without oceans of data, there is no machine learning. Thanks to the Internet of Things, we have at least lakes, perhaps even seas, of data. Oceans of data are just over the horizon. So, while the Internet of Things is important in and of itself, for its ability to generate useful data that we can instrument and act on, it is in the IoT’s enabling of machine-learning technology that things become synergistic and really interesting. While the current state of AI is certainly overhyped, the availability of more data will maximize its promise. As the IoT matures, we will have orders of magnitude more data than ever before. (This may be a frightening thought for people who, 20 years ago, thought we were already experiencing information overload and were in the midst of a data tsunami!)

Living in a Mixed Reality

We’re now making interesting strides forward from a connected world to a virtual-reality (VR) or mixed-reality (MR) world, in the wake of the early 2010s hype cycle. The promise of Oculus Rift as a virtual-reality bombshell over five years ago recently became more nuanced. The 2015 emergence of Pokemon Go as an augmented-reality craze fed a hype cycle around VC funding of mixed-reality technologies. Channeling excitement about the possibilities of MR in the game-design world culminated in a coterie of thoughtful, innovative game-design leaders—including Nicole Lazzaro, Jesse Schell, and Chelsea Howe—designing games for MR platforms. Despite these luminaries shifting their professional focus to work on MR and the novelty that these new experiences offer, so far, they have not realized wow experiences. Or, at least the experiences, while novel, are not yet encouraging people to buy the technology en masse or devote their free time to MR or VR systems. The hardware is still either too clumsy or too low fidelity. These factors constrain these software systems, generally resulting in an uncanny valley that is just short of mixed-reality immersion. Similar to mobile computing in the years before the iPhone, MR hasn’t quite yet arrived.

Nevertheless, these constraints have little bearing on the future. Even today, you no longer need to strap a big box around your head—as with the Oculus Rift—to experience virtual reality, as was necessary just a year or two ago. The integration of feature-film, computer-generated imagery (CGI) into MR—imagery that has only recently gotten to the point where it is indistinguishable from reality when skillfully executed—will play a crucial role in creating software for a more humane, useful, effective hardware platform. We’ll still encounter barriers in creating these experiences. At a minimum, we’ll see the same things that made the Nintendo Wii a toy that required users to rearrange their furniture, to say nothing of the environmentally antisocial aspects of users becoming rhapsodized by a world that is separate from the actual people around them. But viable technology that is no longer novel will drive a platform shift in the way we consume entertainment at home. As our ability to immerse ourselves in physical environments remotely enables us to rethink everything that previously required physical presence, we’ll even remake markets and business models. While we probably won’t see this progress until the 2020s, you can almost touch it—if only virtually.

3D Printing: From Novelty to Industry

Now, let’s consider additive fabrication and the local aspect of our narrative. 3D printing, another technology that, while decades old, has developed substantially in recent years, may seem almost like science fiction to newcomers. But the ability to print out physical components is, indeed, real—and has evolved from plastics to metals, as shown in Figure 2, as well as a variety of other materials. The consumer uses of 3D printing are still largely reserved for the serious hobbyist or novelty seeker. But its business applications—at least among very progressive companies—are already creating new forms of manufacturing and construction. We’re seeing the 3D printing of homes, dental appliances, and replacement parts—even though the technology is still somewhat new.

Figure 2—MarkForged metal 3D printing system
MarkForged metal 3D printing system

Image source: MarkForged

It is not too great a leap to imagine the 3D printing of large home appliances, automobiles, and computers in the not-distant future. While, ultimately, there must be limits to what these additive fabrication technologies can create, those boundaries are not yet crisp. Like mixed reality—and possibly in an ecosystem where 3D printing is closely integrated with mixed reality—additive fabrication could change the face of manufacturing. Will manufacturers fabricate consumer products hyperlocally, eschewing the cliché of the Chinese factory for an efficient maker lab in your local town? Will you choose your next car using an AR headset, then have your highly customized car fabricated at a shop around the corner and delivered to your door the same day? Microsoft is already imagining the first part of this equation, as depicted in Figure 3. Needless to say, if and when these technologies reach this level, the world will be meaningfully different from the one to which we are accustomed.

Figure 3—Microsoft’s HoloLens and Volvo cars
Microsoft's HoloLens and Volvo cars

Image source: Microsoft

Closing Thoughts

This is the thrill of these smartware technologies that are emerging at roughly the same time and happen to be incredibly synergistic. Leveraging all of these technologies—from AI and the IoT, to MR and 3D printing, and the various connections between them—will deliver meaningful impact to the world, in ways that the applications of the early 2000s never did. This is a heady time, as we’re beginning to see a delicious near-term future. But despite the excitement around all of this new technology, we haven’t even gotten into the other game-changing part of our narrative: the sciences. In the midst of the age of digital, cutting-edge science has never been closer to commercialization. We’ll soon see the integration of great scientific advances into the product pipeline. In our next installment, we’ll explore this most exciting aspect of the smartware revolution. 

Managing Director, SciStories LLC

Co-owner of Genius Games LLC

Boston, Massachusetts, USA

Dirk KnemeyerAs a social futurist, Dirk envisions solutions to system-level problems at the intersection of humanity, technology, and society. He is currently the managing director of SciStories LLC, a design agency working with biotech startups and research scientists. In addition to leading SciStories, Dirk is a co-owner of Genius Games LLC, a publisher of science and history games. He also cohosts and produces Creative Next, a podcast and research project exploring the future of creative work. Dirk has been a design entrepreneur for over 15 years, has raised institutional venture funding, and has enjoyed two successful exits. He earned a Master of Arts from the prestigious Popular Culture program at Bowling Green.  Read More

Principal at GoInvo

Boston, Massachusetts, USA

Jonathan FollettAt GoInvo, a healthcare design and innovation firm, Jon leads the company’s emerging technologies practice, working with clients such as Partners HealthCare, the Personal Genome Project, and Walgreens. Articles in The Atlantic, Forbes, The Huffington Post, and WIRED have featured his work. Jon has written or contributed to half a dozen non-fiction books on design, technology, and popular culture. He was the editor for O’Reilly Media’s Designing for Emerging Technologies, which came out in 2014. One of the first UX books of its kind, the work offers a glimpse into what future interactions and user experiences may be for rapidly developing technologies such as genomics, nano printers, or workforce robotics. Jon’s articles on UX and information design have been translated into Russian, Chinese, Spanish, Polish, and Portuguese. Jon has also coauthored a series of alt-culture books on UFOs and millennial madness and coauthored a science-fiction novel for young readers with New York Times bestselling author Matthew Holm, Marvin and the Moths, which Scholastic published in 2016. Jon holds a Bachelor’s degree in Advertising, with an English Minor, from Boston University.  Read More

Other Columns by Dirk Knemeyer

Other Columns by Jonathan Follett

Other Articles on Experience Trends

New on UXmatters