This is one in a series on careers in science, technology, engineering and mathematics made possible with generous support from Alcoa Foundation.
A sword and shield hang on the wall in James Anhalt’s office. Soon, he’ll add a crown to this collection. He is a software engineer at Blizzard Entertainment in Irvine, Calif. This company has created many hit video games, from World of Warcraft to the StarCraft and Diablo series. Swords, shields and crowns abound in these fantasy-themed games. So the company gives out real ones when employees reach work anniversaries. When Anhalt gets the crown, it will represent 20 years with the company.
Anhalt’s job is fun but also demanding. “Kids who play a lot of games might not realize how much work it takes,” Anhalt points out. One major game, 2015’s StarCraft II: Legacy of the Void, took more than 2.5 years to develop. And that was with the help of more than 80 engineers, programmers, designers, artists, writers and others working long hours. Every little detail needed time and attention.
Anhalt worked on the game engine. This software is the framework that developers start with when they create a new game. It includes ways to add artwork, sound, special effects, characters, enemies and more. But programmers often need to modify the game engine. They may fix errors or add new features for artists and designers. That’s where Anhalt came in. He made sure all the parts of the game fit together and ran well.
Clearly, the team succeeded. The game was a big hit, just like the previous two games in the Starcraft II trilogy, which began in 2010. Every year, thousands of elite players from around the world compete in StarCraft II tournaments for prizes of up to $100,000.
But video games are about much more than winning or losing. In this story, we’ll meet two engineers and a neuroscientist who take games very seriously. Anhalt and his colleagues work hard to make their games as fun as possible. An audio engineer is working on a new kind of interactive technology. It’s called mixed reality, and it could transform how people interact with all digital media, not just games. And a neuroscientist builds games to fix people’s brains. He believes that doctors may one day prescribe video games to treat a variety of conditions.
Making the most of memory
StarCraft II is a game of strategy. Players make split-second decisions about which structures to build, where to explore and when to fight or retreat. All these tasks take up space in a computer’s memory. Memory is the space available on a physical chip or disk for storing information. A computer (or tablet or mobile device) only has so much memory. To keep a game running smoothly, developers must pay close attention to how that game is using a device’s memory. If it uses up too much memory, the game will run slowly or even freeze or crash.
Top-of-the-line computers pack in more and more memory every year. But game developers have to make sure that their product will work on everyone’s computers, not just the most powerful ones, Anhalt says. So those developers want to make the game run as efficiently as possible. And that means getting creative with code. Computer code is the set of step-by-step instructions a computer program follows.
Sometimes, following those instructions takes too long. As an example, Anhalt points to graphics (a game’s visual elements). At any moment in the game, each pixel on a display screen needs to be a certain color. Many factors affect that color. A pixel in the sky might be blue, but one on a character’s armor might be gray. Nearby light and shadows or special effects (such as snowfall or an explosion) can affect the color too. “If we have, say, 16 different options that can all be on or off, that generates 65,000 different programs,” Anhalt explains.
While working on StarCraft II: Legacy of the Void, his team found that running through all the color-choosing programs for every pixel on the screen was using too much memory. They managed to compress all those programs down to just 5 percent of their original size. They combined some similar programs together and cut out others completely. They also rewrote sections of the code to make it simpler.
Making the color-choosing process more efficient gave game artists more memory to work with. That meant they could add even more special effects to the game.
As computers get more and more powerful, graphics and other game features will get more realistic, Anhalt says. More memory means more lifelike detail.
Soon, games will be so lifelike that they jump off the screen. Anhalt and many other experts agree that the next big things in gaming — and in technology in general — are virtual and augmented realities.
Virtual reality is already here. The Oculus Rift headset came out in March 2016 and the Sony PlayStation VR will come out soon. These headsets both resemble something a deep-sea diver might wear. But when you put one on, you find yourself in another world. Everything you see, hear and touch is a digital illusion.
Augmented reality, in contrast, places digital illusions into the real world. Pokémon Go is a recent augmented reality sensation. The game uses a smart phone’s camera and global positioning system (GPS) to reveal Pokémon monsters sitting on sidewalks or in parks.
But Pokémon Go still uses a screen. “I think screens are going away,” says Gene Rozenberg. He’s an engineer at a company called Magic Leap in Dania Beach, Fla. The company is developing a new kind of augmented reality technology that it calls “mixed reality.”
“We make you believe that a computer image is really in the world around you,” Rozenberg says. In a demonstration for the magazine Wired, this new technology made text messages float above a person’s desk. It also made three-dimensional jellyfish swim toward the ceiling.
Rozenberg’s role at Magic Leap is to design sound effects. Throughout his career, he has worked on many games, including the Madden NFL series. No matter where he has worked, he’s spent his days composing music and recording sound effects or voiceovers. He also has made sure that these sounds play at just the right moments in a game. That means programming.
“I myself am not a programmer,” he says. But he’s taking a class now to learn more about reading and writing computer code.
Why does a sound guy need to know about programming? Think about footsteps, Rozenberg says. The sound has to change every time a character steps on a different surface, like grass, wood or stone. Characters might be gigantic or tiny, or wearing different types of shoes. To make footsteps sound good, he says, “I have to think like a computer would.” He has to give very precise instructions to make sure the right sounds play at the right time.
In both virtual and augmented reality, sound gets even more complicated. Developers in these new mediums face a big problem when it comes to sound, explains Rozenberg. A mixed-reality device, for example, plays sounds straight into someone’s ears. But those sounds need to seem as if they are coming from above, below, behind or in front of that person — not from the headphones they are wearing. “This hasn’t been done before,” he notes.
He can’t talk about the steps he might be taking to solve this problem, though. Since Magic Leap’s technology hasn’t come out yet, the company is still secretive about exactly how it works and what it does.
One thing Rozenberg can say is that the human brain is a very important part of the system.
“Your brain is a computer. It processes information and decides how that information is [delivered] to you, the person,” he says. To fool the brain into thinking there really is a jellyfish on the ceiling, developers need to understand how the brain processes information. If they give the brain the right visual and sound signals, the jellyfish will seem like reality.
Fixing brains with games
At his lab in California, neuroscientist Adam Gazzaley is delving even more deeply into the mysteries of the human brain. And it’s not only for the sake of fun. He wants to turn video games into medicine.
An iPad game nicknamed Project: EVO got its start at Gazzaley’s lab at the University of California, San Francisco. Its players drive through frozen and fiery worlds at top speed. They have to navigate around icebergs and fire pits. At the same time, they perform tasks such as collecting all the red creatures that pop up, while ignoring green and blue ones.
Right now, different teams of researchers are studying whether the game can treat a variety of disorders, including attention deficit hyperactivity disorder (ADHD), autism and depression. If it works, then the U.S. Food and Drug Administration (FDA) might approve the game as a medical treatment.
Imagine going to the doctor and getting a prescription for a video game! This could happen within just a year or so, says Gazzaley. He co-founded a company, Akili Interactive Labs, to turn Project: EVO into a product that people can buy.
And Gazzaley has good reason to think this game could become a success. Before EVO, there was Neuroracer. This early version of Gazzaley’s game also challenged players to drive while collecting objects. In a 2013 study, the scientist and his colleagues showed that the multi-tasking in Neuroracer could give older adults a brain boost.
Attention and memory get worse as people age. So Gazzaley recruited volunteers ages 60 to 85 to practice the game for a month. They got much better at playing the game. More importantly, they performed better in tests of their attention and working memory. (Working memory is the brain’s ability to hold onto information for a short period of time.) Control groups — people who were not being treated — did not show the same benefits.
One control group didn’t play the game. Another control group played the racing part of the game without the collecting task. Six months later, people who had played the full game still had sharper brains compared with the other two groups.
Kids with ADHD also struggle with attention and working memory. EVO might give these kids a brain boost just as Neuroracer did for older adults. In 2015, one of the first studies of EVO got positive results. Kids with ADHD who played the game regularly for four weeks became better at paying attention. They also were less impulsive, meaning they were less likely to act without thinking.
Gazzaley sees Neuroracer and EVO as personal trainers for the brain. “The game is constantly recording how you’re doing and then adjusting the challenge level to be appropriate to your ability,” he says. The better you play, the harder it gets.
Right now, these games use a player’s mistakes or successes to set the level of difficulty. But eventually, Gazzaley hopes to connect games directly to a player’s brain.
In the lab, he’s testing a technology he calls “glass brain.” It lets people see their own brain activity in real time. To use it, a person wears an electroencephalography (Ee-LEK-troh-en-seff-uh-LAH-grah-fee) cap. This cap contains sensors. They record a person’s brainwaves and feed those data to the glass-brain computer system. The system displays a three-dimensional image of the brain. Active brain areas flash with colored light.
A video game could use this real-time brain activity to figure out why a player made a certain mistake, Gazzaley explains. Maybe the person’s reaction time was too slow, for example. The game could then create challenges to train — and then treat — reaction times.
Why would a neuroscientist turn to games to fix people’s brains? Simply put, because games are fun.
Typically, Gazzaley says, when researchers want to study the brain, “We use such boring tests. People don’t engage in them deeply.” People are more likely to learn and build skills if they enjoy the training process.
Anhalt, Rozenberg and other game creators also work hard to create exciting, engaging experiences. When they succeed, you won’t think about how hard they worked behind the scenes. You’ll be having too much fun playing your favorite game.
(for more about Power Words, click here)
attention deficit hyperactivity disorder (ADHD) This is a disorder characterized by not being able to focus or pay attention, being physically overactive, not being able to control behavior, or a combination of these.
audio Having to do with sound.
augmented reality A technology similar to virtual reality that places digital illusions into the real world.
autism (also known as autism spectrum disorders ) A set of developmental disorders that interfere with how certain parts of the brain develop. Affected regions of the brain control how people behave, interact and communicate with others and the world around them. Autism disorders can range from very mild to very severe. And even a fairly mild form can limit an individual’s ability to interact socially or communicate effectively.
brainwave An electrical signal produced through the coordinated activity of billions of neurons in the brain of an animal. When charted, the signal typically looks wavy or spiky.
code (in computing) To use special language to write or revise a program that makes a computer do something.
colleague Someone who works with another; a co-worker or team member.
computer program A set of instructions that a computer uses to perform some analysis or computation. The writing of these instructions is known as computer programming.
control A part of an experiment where there is no change from normal conditions. The control is essential to scientific experiments. It shows that any new effect is likely due only to the part of the test that a researcher has altered. For example, if scientists were testing different types of fertilizer in a garden, they would want one section of it to remain unfertilized, as the control. Its area would show how plants in this garden grow under normal conditions. And that give scientists something against which they can compare their experimental data.
depression A mental illness characterized by persistent sadness and apathy. Although these feelings can be triggered by events, such as the death of a loved one or the move to a new city, that isn’t typically considered an “illness” — unless the symptoms are prolonged and harm an individual’s ability to perform normal daily tasks (such as working, sleeping or interacting with others). People suffering from depression often feel they lack the energy needed to get anything done. They may have difficulty concentrating on things or showing an interest in normal events. Many times, these feelings seem to be triggered by nothing; they can appear out of nowhere.
digital (in computer science and engineering) An adjective indicating that something has been developed numerically on a computer or on some other electronic device, based on a binary system (where all numbers are displayed using a series of only zeros and ones).
disorder (in medicine) A condition where the body does not work appropriately, leading to what might be viewed as an illness. This term can sometimes be used interchangeably with disease.
engineer A person who uses science to solve problems. As a verb, to engineer means to design a device, material or process that will solve some problem or unmet need.
Food and Drug Administration (or FDA) A part of the U.S. Department of Health and Human Services, FDA is charged with overseeing the safety of many products. For instance, it is responsible for making sure drugs are properly labeled, safe and effective; that cosmetics and food supplements are safe and properly labeled; and that tobacco products are regulated.
game engine Specialized software used to create video games.
gene (adj. genetic) A segment of DNA that codes, or holds instructions, for producing a protein. Offspring inherit genes from their parents. Genes influence how an organism looks and behaves.
global positioning system Best known by its acronym GPS, this system uses a device to calculate the position of individuals or things (in terms of latitude, longitude and elevation — or altitude) from any place on the ground or in the air. The device does this by comparing how long it takes signals from different satellites to reach it.
graphics (in computing) Images or video displayed on a computer screen or other digital media.
illusion A thing that is or is likely to be wrongly perceived or interpreted by the senses.
impulsive (n. impulsivity) Quick to act; not willing to wait. Not waiting for deliberation or a weighing of consequences.
information (as opposed to data) Facts provided or trends learned about something or someone, often as a result of studying data.
media Singular: medium. (in the social sciences) A term for the ways information is delivered and shared within a society. It encompasses not only the traditional media — newspapers, magazines, radio and television — but also Internet- and smartphone-based outlets, such as blogs, Twitter, Facebook and more. The newer, digital media are sometimes referred to as social media.
neuroscience The field of science that deals with the structure or function of the brain and other parts of the nervous system. Researchers in this field are known as neuroscientists.
pixel Short for picture element . A tiny area of illumination on a computer screen, or a dot on a printed page, usually placed in an array to form a digital image. Photographs are made of thousands of pixels, each of different brightness and color, and each too small to be seen unless the image is magnified.
programming (in computing) To use a computer language to write or revise a set of instructions that makes a computer do something. The set of instructions that does this is known as a computer program. The term is also used, in an analogy to computer programming, to describe the way something — such as cells — have been designed to function. A person who writes or revises software is a known as a programmer.
sensor A device that picks up information on physical or chemical conditions — such as temperature, barometric pressure, salinity, humidity, pH, light intensity or radiation — and stores or broadcasts that information. Scientists and engineers often rely on sensors to inform them of conditions that may change over time or that exist far from where a researcher can measure them directly. (in biology) The structure that an organism uses to sense attributes of its environment, such as heat, winds, chemicals, moisture, trauma or an attack by predators.
software The mathematical instructions that direct a computer’s hardware, including its processor, to perform certain operations.
software engineer An engineer who designs and works with computer programs.
technology The application of scientific knowledge for practical purposes, especially in industry — or the devices, processes and systems that result from those efforts.
virtual Being almost like something. An object or concept that is virtually real would be almost true or real — but not quite. The term often is used to refer to something that has been modeled — by or accomplished by — a computer using numbers, not by using real-world parts. So a virtual motor would be one that could be seen on a computer screen and tested by computer programming (but it wouldn’t be a three-dimensional device made from metal).
virtual reality A three-dimensional simulation of the real world that seems very realistic and allows people to interact with it. To do so, people usually wear a special helmet or glasses with sensors.
voiceover (in TV and moviemaking) The recording of a human voice over film or animation.
working memory The ability to hold something in the mind for a short period of time, such as a mental grocery list or a phone number.
K. Kelly. “Hyper Vision: The Untold Story of Magic Leap, the World’s Most Secretive Startup.” Wired. May 2016.
E. Sohn. “What video games can teach us.” Science News for Students. January 19, 2004.
E. Sohn. “The violent side of video games.” Science News for Students. January 12, 2004.
E. Sohn. “A classroom of the mind.” Science News for Students. October 20, 2003.