jLuger.de - Private theater with google cardboard

How about watching all your films in a private movie theater? Would be nice, wouldn't it? Well, with Google cardboard, the app VR Theater for Cardboard and a compatible smartphone it is possible. For about 15 minutes. Then the film stopped and the battery went low really fast. On another occasion I've plugged my smartphone (a Nexus 5 2013) in the charger and restarted the movie every it stopped. After 30 minutes the smartphone rebooted because of heating problems. It seems that the conversion of the film has to get of the phone.

At a first step I've took some screenshots from the app while playing a film. Comparing this screenshots with the compare tool of imagemagick gave me this:


This shows some distortion as it seems to be typical with VR.
Wikipedia explains that this is barrel distortion and is needed to fight the pincushion distortion caused by the lens of the cardboard. There is a nice youtube video that explains this for the Oculus Rift. The lenses itself are needed so that you can focus on the screen. Well at least I couldn't without them.

In order to get a better view on the overall distortion I've made two screenshots of another app: Cardboard Camera. The comparing of them is this:


In order to get a hint on how to calculate this distortion I've marked the corner points of the overall distortion and the virtual theater screen on a page:

Giving me an error margin of 3 pixel, you clearly see here that the points are symmetric to the y axis but not to a single point. All the barrel distortion algorithms I've found require to be point symmetric.

While looking at the screenshots to get a solution, I've wondered if a normal rectangle will work for the film area. As you can see the distortion isn't that intense here. I've decided to use 545x300 as the film area and reduced a photo to that size. Then I've created a black image with the landscape size of my smartphone (1920x1080) and added the small photos at 130,375 and 1243,375. The commands for convert are this:

convert crop.png -resize 545x300 small.png
convert -size 1920x1080 xc:black black.png
composite -geometry +130+375 small.png black.png vr.png
composite -geometry +1243+375 small.png vr.png vr.png

When viewing this on a normal photo viewer with my cardboard I've needed some seconds to adjust but finally I've got one big image with slight pincushion distortion. Slight enough to ignore it and give a film a try.

Before converting a film I went over the film size again and found out that 544 would be a better width as 544 is a multiply of 16. The height should be 306 then.
To redo the steps for the image I've created a black film that is one second longer than the original film I wanted to convert for viewing with cardboard.

ffmpeg -framerate 1/7014 -i black.png -c:v libx264 -r 25 black_film.mp4

Yes, the film has 7013 seconds.
It isn't necessary crop the film before as ffmepg can do this while adding it to another film. The resulting command is this:

ffmpeg -i black_film.mp4 -vf "movie=cool_movie.mp4, scale=544:-1 [inner1]; \
movie=cool_movie.mp4, scale=544:-1 [inner2]; [in][inner1] overlay=131:375 [step1]; \
[step1][inner2] overlay=1243:375 [out]" vr_film.mp4

This will create a film without sound. That needs to be extracted from the original film and than readded to this film.

ffmpeg -i cool_movie.mp4 -vn -acodec copy cool_movie.aac
ffmpeg -i vr_film.mp4 -i cool_movie.aac -c:v copy -c:a aac -strict experimental vr_film_sound.mp4

At first the film looked ok but had some undefinable issues. After viewing it for some time I've found out that there were some artificial 3d effects in the movie. While it was cool it made watching the film very tiring. I haven't done any further experiments and leave the information here for others to investigate.

Update 17.01.2016: Please read The power of ffmpeg to do all the calls to ffmpeg with one call.