TMC PULSE

T M C » P U L S E | J U LY 2 0 1 8 32 Solutions: TMC Innovations accomplish a wide range of activities, such as walking down the street, nav- igating an airport or even reading a bedtime story. Members of the AT&T Foundry for Connected Health team, while devel- oping an artificial intelligence (AI) and machine learn- ing system to read the labels on medication bottles, partnered with Aira about a year ago to provide network connectivity to Aira's smart glasses. The result is "Hey Chloe," a recognition solution with built-in, voice-activated technology that debuted in March 2018. Aira's new AI platform identifies prescriptions and over-the-counter medications. Nadia Morris, former director of the AT&T Foundry, explained the process. "First, [the computer] has to determine if it is a medication bottle or not," she said. "It's similar to the TxTag, where a photo is taken. Their systems are trained to know what a license plate looks like." The TxTag system, which allows drivers to pre-pay tolls, works off an AI system that recognizes license plates. Morris's team applied the same process to medication bottles; team members brought in their own bottles and trained the computer to read them. The team even set up a secure system for other AT&T colleagues to donate images of their bottles to help train the computer. "We crowdsourced it," Morris said. "A lot of employees run the spectrum of age, gender and ethnic background, so it was a good cross section." Although major pharmacies have offered "talking" pill bottles for several years—typically, a health professional records instructions on a device that attaches to the bot- tle—"Hey Chloe" accesses instruc- tions in a different way. "Hey Chloe" users can activate the AI assistant by asking, "Hey Chloe, what medication is this?" The AI assistant will scan the field around the user and find the bottle of prescription medication. The glasses will read the label and turn that information into an audio file that is read into the user's ear, Morris said. The system also works for over-the-counter medication. The AT&T Foundry team learned a few things during this project. One challenge with machine learning is providing a varied data set from which the computer can learn. In addition, because most pill bottles are cylinders, the user often must rotate the bottles for the glasses to read the prescriptions correctly. Team members also discovered that while it seems like there is a CVS or Walgreens on every corner, a large number of prescriptions come from independent pharma- cies, so they had to train the computer to recognize different types of labels, Morris said. Can't Read the Label on Your Meds? Ask Chloe. "Hey Chloe" is 'like having a sighted person in your pocket' B y C h r i s t i n e H a l l N early 70 percent of Americans take prescription drugs, accord- ing to the Mayo Clinic. Reading the tiny print on medi- cation bottles—where dose amounts and critical instructions are found— can be tricky. For those with low or no vision, it's next to impossible. But a recent collaboration between the AT&T Foundry for Connected Health, located in the Texas Medical Center Innovation Institute, and La Jolla, California- based Aira, which makes smart glasses for people who are blind or have low vision, offers a solution. Aira's remote assistive technol- ogy connects smart glasses users— dubbed "explorers"—with a network of certified agents via an augmented reality dashboard. The agents serve as visual interpreters, helping users Members of the AT&T Foundry for Connected Health team partnered with Aira to provide network connectivity to Aira's smart glasses. Credit: Courtesy of Aira

• Cover