Lock in Your Premium Membership Discount!

Your content will be up shortly. Please allow up to 5 seconds
News & Insight

Microsoft’s First-Person Hyperlapse Algorithm Makes Shaky GoPros Videos Watchable

By Justin Heyes on August 11th 2014

Researchers at Microsoft have changed the way we look at the videos taken from the first person perspective. Using a newly developed algorithm, they can transform long, boring and sometimes nauseating footage taken from helmet cams or Google glasses into a silky smooth hyperlapse that are ultimately more enjoyable to watch.

If the source video was a walk down the street it would be jittery and have monotonous shots waiting at the cross walk for the light to turn or looking down at your feet. The algorithm would help render footage that would have steadicam like smoothness.

[REWIND: ‘ENTER PYONGYANG’ | WALK INTO NORTH KOREA THROUGH THIS HYPERLAPSE OF UNPRECENTED ACCESS]

How the process works could be described as computer wizardry. The process begins by removing redundant frames from the source video such as shots waiting at a cross walk. Then the algorithm reconstructs the the trajectory of the camera in a in a 3D depth map using structure from motion. microsoft-hyperlapse_1 - Reconstuction

Each frame is then analyzed for similar geometry to help create a path for the camera to follow. The path is chosen on a per pixel basis to remain smooth and have have a consistent render. The shots that interrupt the flow are identified and removed.

microsoft-hyperlapse_2 - Path Planning

microsoft-hyperlapse_per pixel analyzation

microsoft-hyperlapse_identifying stationary frames

The frames for the final rendering are made up of several neighboring frames from the source video and are then stitched together and blend to match white balance and exposure. When comparing the resulting render to a stabilized time-lapse of the source, there is no contest. 

microsoft-hyperlapse_3 - Rendering

microsoft-hyperlapse_identify geometry

microsoft-hyperlapse_proxy geometry

microsoft-hyperlapse_no blending of frames

Here technical break down behind the algorithms and what it does to achieve these results:

Think of all the GoPro videos out in the world that could be condensed into something more watchable. Let’s just hope this project doesn’t fall to the same fate as Photosynth. 

There is no timeline to when this hyperlapse algorithm will be available. Creators Johannes Kopf, Michael Cohen and Richard Szeliski hope to make it into a Windows app soon.

(via Microsoft/Youtube – Images via screencaps)

About

Justin Heyes wants to live in a world where we have near misses and absolute hits; great love and small disasters. Starting his career as a gaffer, he has done work for QVC and The Rachel Ray Show, but quickly fell in love with photography. When he’s not building arcade machines, you can find him at local flea markets or attending car shows.

Explore his photographic endeavors here.

Website: Justin Heyes
Instagram: @jheyesphoto

Q&A Discussions

Please or register to post a comment.

  1. Michael Moe

    this algorithm will improve a lot of hyperlapses! great

    | |
  2. Greg Faulkner

    That looks really cool. I hope they actually do something with this research

    | |