Lying with Pixels
brian | 11.06.2005 01:13
Lying with Pixels by Ivan Amato
Seeing is no longer believing. The image you see on the evening news could well be a fake—a fabrication of fast new video-manipulation technology.
Last year, Steven Livingston, professor of political communication at George Washington University, astonished attendees at a conference on the geopolitical pros and cons of satellite imagery. He didn’t produce evidence of new military mobilizations or global pandemics. Instead, he showed a video of figure skater Katarina Witt during a 1998 skating competition.
In the clip, Witt gracefully plies the ice for about 20 seconds. Then came what is perhaps one of the most unusual sports replays ever seen. The background was the same, the camera movements were the same. In fact, the image was identical to the original in all ways except for a rather important one: Witt had disappeared, along with all signs of her, such as shadows or plumes of ice flying from her skates. In their place was exactly what you would expect if Witt had never been there to begin with—the ice, the walls of the rink and the crowd.
So what’s the big deal, you ask. After all, Stalin’s staff routinely airbrushed persona non grata out of photos more than a half-century ago. And Woody Allen ushered a variation on reality morphing into the movies 17 years ago with Zelig, in which he inserted himself next to Adolf Hitler and Babe Ruth. In films such as Forrest Gump and Wag the Dog, reality twisting has become commonplace.
What sets the Witt demo apart—way apart—is that the technology used to “virtually delete” the skater can now be applied in real time, live, even as a camera records a scene and instantly broadcasts it to viewers. In the fraction of a second between video frames, any person or object moving in the foreground can be edited out, and objects that aren’t there can be edited in and made to look real. “Pixel plasticity,” Livingston calls it. The implication for those at the satellite imagery conference was sobering: Pictures from orbit may not necessarily be what the satellite’s electronic camera actually recorded.
But the ramifications of this new technology reach beyond satellite imagery. As live electronic manipulation becomes practical, the credibility of all video will become just as suspect as Soviet Cold War photos. The problem stems from the nature of modern video. Live or not, it is made of pixels, and as Livingston says, pixels can be changed.
The best-known examples of real-time video manipulation so far are “virtual insertions” in professional sports broadcasts. Last January 30, for instance, nearly one-sixth of humankind in more than 180 countries repeatedly saw an orange first-down line stretched across the gridiron as they watched the Super Bowl. New York-based Sportvision created that line and inserted it into the live feed of the broadcast. To help determine where to insert the orange pixels, several game cameras were fitted with sensors that tracked the cameras spatial positions and zoom levels. Adding to the illusion of reality was the ability of the Sportvision system to make sure that players and referees occlude the virtual line when their bodies traverse it.
Last spring and summer, as Sportvision and rivals such as Princeton Video Imaging (PVI) in Lawrenceville, N.J,, were airing virtual insertion products, including simulated billboards on walls behind major league batters, a team of engineers from Sarnoff Corp. in Princeton, N.J., flew to the Coalition Allied Operations Center of NATO’s Operation Allied Force in Vicenza, Italy. Their mission: transform their experimental video processing technology into an operational tool for rapidly locating and targeting Serbian military vehicles in Kosovo. The project was dubbed TIGER, for “targeting by image georegistration.” “Just dial in the coordinates and the thing goes,” explains Michael Hansen, a young, caffeinated Sarnoff gadgeteer who can hardly believe he was helping fight a war last year.
Compared to PVI’s job, the military’s technical task was more difficult—and the stakes were much higher. Instead of altering a football broadcast, the TIGER team manipulated a live video feed from a Predator, an unmanned reconnaissance craft flying some 450 meters above Kosovo battlefields. Rather than superimposing virtual lines or ads into sports settings, the task was to overlay, in real time, “georegistered” images of Kosovo onto the corresponding scenes streaming in live from the Predator’s video camera. The terrain images had been previously captured with aerial photography and digitally stored. The TIGER system, which automatically detected moving objects against the background, could almost instantly feed to the targeting officers the coordinates for any piece of Serbian hardware in the Predator’s view. This was quite a technical feat, since the Predator was moving and its angle of view was constantly changing, yet those views had to be electronically aligned and registered with the stored imagery in less than one-thirtieth of a second (to match the frame rate of video recording).
In principle, the targeting step could have been hotwired to precision guided weapons. “We weren’t actually doing that in Allied Force,” Hansen notes. “We were just telling targeting officers exactly where Serbian targets were and then they would vector in planes to go strike the targets.” That way the human decision makers could pre-empt flawed machine-made decisions. According to the Defense Advanced Research Projects Agency, TIGER technology was used extensively in the final three weeks of the Kosovo operation, during which “80 to 90 percent of the mobile targets were hit.”
So far, real-time video manipulation has been within the grasp only of technologically sophisticated organizations such as TV networks and the military. But developers of the technology say it’s becoming simple and cheap enough to spread everywhere. And that has some observers wondering whether real-time video manipulation will erode public confidence in live television images, even when aired by news outlets. “Seeing may no longer be believing,” says Norman Winarsky, corporate vice president for information technology at Sarnoff. “You may not know what to trust.”
etc
http://www.nodeception.com/articles/pixel.jsp
Seeing is no longer believing. The image you see on the evening news could well be a fake—a fabrication of fast new video-manipulation technology.
Last year, Steven Livingston, professor of political communication at George Washington University, astonished attendees at a conference on the geopolitical pros and cons of satellite imagery. He didn’t produce evidence of new military mobilizations or global pandemics. Instead, he showed a video of figure skater Katarina Witt during a 1998 skating competition.
In the clip, Witt gracefully plies the ice for about 20 seconds. Then came what is perhaps one of the most unusual sports replays ever seen. The background was the same, the camera movements were the same. In fact, the image was identical to the original in all ways except for a rather important one: Witt had disappeared, along with all signs of her, such as shadows or plumes of ice flying from her skates. In their place was exactly what you would expect if Witt had never been there to begin with—the ice, the walls of the rink and the crowd.
So what’s the big deal, you ask. After all, Stalin’s staff routinely airbrushed persona non grata out of photos more than a half-century ago. And Woody Allen ushered a variation on reality morphing into the movies 17 years ago with Zelig, in which he inserted himself next to Adolf Hitler and Babe Ruth. In films such as Forrest Gump and Wag the Dog, reality twisting has become commonplace.
What sets the Witt demo apart—way apart—is that the technology used to “virtually delete” the skater can now be applied in real time, live, even as a camera records a scene and instantly broadcasts it to viewers. In the fraction of a second between video frames, any person or object moving in the foreground can be edited out, and objects that aren’t there can be edited in and made to look real. “Pixel plasticity,” Livingston calls it. The implication for those at the satellite imagery conference was sobering: Pictures from orbit may not necessarily be what the satellite’s electronic camera actually recorded.
But the ramifications of this new technology reach beyond satellite imagery. As live electronic manipulation becomes practical, the credibility of all video will become just as suspect as Soviet Cold War photos. The problem stems from the nature of modern video. Live or not, it is made of pixels, and as Livingston says, pixels can be changed.
The best-known examples of real-time video manipulation so far are “virtual insertions” in professional sports broadcasts. Last January 30, for instance, nearly one-sixth of humankind in more than 180 countries repeatedly saw an orange first-down line stretched across the gridiron as they watched the Super Bowl. New York-based Sportvision created that line and inserted it into the live feed of the broadcast. To help determine where to insert the orange pixels, several game cameras were fitted with sensors that tracked the cameras spatial positions and zoom levels. Adding to the illusion of reality was the ability of the Sportvision system to make sure that players and referees occlude the virtual line when their bodies traverse it.
Last spring and summer, as Sportvision and rivals such as Princeton Video Imaging (PVI) in Lawrenceville, N.J,, were airing virtual insertion products, including simulated billboards on walls behind major league batters, a team of engineers from Sarnoff Corp. in Princeton, N.J., flew to the Coalition Allied Operations Center of NATO’s Operation Allied Force in Vicenza, Italy. Their mission: transform their experimental video processing technology into an operational tool for rapidly locating and targeting Serbian military vehicles in Kosovo. The project was dubbed TIGER, for “targeting by image georegistration.” “Just dial in the coordinates and the thing goes,” explains Michael Hansen, a young, caffeinated Sarnoff gadgeteer who can hardly believe he was helping fight a war last year.
Compared to PVI’s job, the military’s technical task was more difficult—and the stakes were much higher. Instead of altering a football broadcast, the TIGER team manipulated a live video feed from a Predator, an unmanned reconnaissance craft flying some 450 meters above Kosovo battlefields. Rather than superimposing virtual lines or ads into sports settings, the task was to overlay, in real time, “georegistered” images of Kosovo onto the corresponding scenes streaming in live from the Predator’s video camera. The terrain images had been previously captured with aerial photography and digitally stored. The TIGER system, which automatically detected moving objects against the background, could almost instantly feed to the targeting officers the coordinates for any piece of Serbian hardware in the Predator’s view. This was quite a technical feat, since the Predator was moving and its angle of view was constantly changing, yet those views had to be electronically aligned and registered with the stored imagery in less than one-thirtieth of a second (to match the frame rate of video recording).
In principle, the targeting step could have been hotwired to precision guided weapons. “We weren’t actually doing that in Allied Force,” Hansen notes. “We were just telling targeting officers exactly where Serbian targets were and then they would vector in planes to go strike the targets.” That way the human decision makers could pre-empt flawed machine-made decisions. According to the Defense Advanced Research Projects Agency, TIGER technology was used extensively in the final three weeks of the Kosovo operation, during which “80 to 90 percent of the mobile targets were hit.”
So far, real-time video manipulation has been within the grasp only of technologically sophisticated organizations such as TV networks and the military. But developers of the technology say it’s becoming simple and cheap enough to spread everywhere. And that has some observers wondering whether real-time video manipulation will erode public confidence in live television images, even when aired by news outlets. “Seeing may no longer be believing,” says Norman Winarsky, corporate vice president for information technology at Sarnoff. “You may not know what to trust.”
etc
http://www.nodeception.com/articles/pixel.jsp
brian