VMD VR Movie Rendering for YouTube, Google Cardboard, GearVR, and Oculus Rift

VMD Example VR movies produced with panoramic and
omnidirectional stereoscopic ray tracing techniques, described
in DOI: 10.1016/j.parco.2015.10.015

One of the advanced features provided by VMD versions 1.9.3 and later is the ability to render omnidirectional stereoscopic 3-D images and movies, useful to create so-called "VR" movies on YouTube and for VR HMD movie players on devices such as GearVR, Oculus Rift, and others. The process of developing VR (omnidirectional stereoscopic) movies using VMD follows a series of steps:

  • Set VMD renderer and rendering settings for VR/360 movies
  • Render the movie
  • Encode the raw frames into a YouTube-compatible MP4
  • Tag the MP4 movie with special YouTube-compatible metadata to indicate that it is a VR movie
  • Upload the movie to YouTube, and mark it as 3-D/stereoscopic

VMD Renderers and Settings

At present, the special omnidirectional stereoscopic projection (OSP) and associated image formats are most easily used via the TachyonL-OptiX (aka "TachyonLOptiXInternal" in text commands) GPU-accelerated ray tracer in VMD. It can be used both for batch mode movies and for live interactive previewing.

By default, the TachyonL-OptiX ray tracing engine uses the same camera projection mode that is selected in the main VMD graphical interface. Since the VMD OpenGL renderer is not yet capable of rendering the special OSPs and lighting modes needed for VR movies, the graphical interface in VMD doesn't yet provide a projection mode for OSP rendering, and it must be enabled by setting some environment variables, as described below. Variables can be set and unset either in the unix shell or within VMD itself, but they will not take effect until the next time the TachyonL-OptiX renderer runs to completion.

    Setting/unsetting an environment variable
  • bash: export VARIABLE=1
  • bash: export VARIABLE=
  • tcsh: setenv VARIABLE=1
  • tcsh: unsetenv VARIABLE
  • vmd/tcl: set env(VARIABLE) 1
  • vmd/tcl: unset env(VARIABLE)
    Environment variable and associated rendering results
  • VMDOPTIXAOMAXDIST: change AO shadows from "infinit" to a maximum occlusion distance formulation. Units are specified in camera coordinates. This is needed within virus capsids or other dark areas that would otherwise be completely dark, and which would otherwise look "2-D" if only the VR headlight was used. A good starting value would be in the range 0.5 to 1.0, increasing to collect more shadow, or decreasing to collect less shadow.
  • VMDOPTIXHEADLIGHT: set to enable a VR-camera-centered "headlight". This light works just like the standard VMD directional lights, but it is always positioned at the camera location.
  • VMDOPTIXEQUIRECTANGULAR: set to enable the omnidirection projection
  • VMDOPTIXSTEREO: set to enable stereoscopic rendering

    Other more specialized environment variables and associated rendering results
  • VMDOPTIXNODESTROYCONTEXT: set to speed up batch-mode movie rendering -- when set all subsequent environment variable changes will be ignored until unset again

Rendering the Movie

For early testing purposes, the movie can be rendered at low or moderate resolutions, e.g. 512x512, 1024x1024, or 2048x2048, which will both speed up rendering and also speed up YouTube uploading and associated post-processing. Final completed movies should be rendered at either 3840x2160 or 4096x4096 resolution, to ensure good quality playback on the latest generation smartphones that have 2560x1440 displays.

Video encoding settings

Medium quality video encoding parameters used for testing.

Loop a few input frames until a certain length movie is reached. 

ffmpeg -framerate 24 -loop 1 -t 01:00 -i frame.%05d.tga -c:v libx264 -profile:v baseline -level 3.0 -pix_fmt yuv420p  looped.mp4
Standard movie sequence. 
ffmpeg -framerate 24 -i frame.%05d.tga -c:v libx264 -profile:v baseline -level 3.0 -pix_fmt yuv420p testing.mp4

High quality, high-bitrate, full-resolution chrominance (4:4:4) video encoding suitable for final upload to YouTube etc. (standard movie sequence) 

ffmpeg -framerate 24 -i frame.%05d.tga -c:v libx264 -profile:v high444 -level 4.0 -crf 8  /ssd/movie_4Kx4K.mp4

Video tagging for YouTube

Before a video can be uploaded to YouTube, it must first be tagged as a "VR" 360/3-D movie. This is accomplished using Google-provided Python scripts for manipulating spatial movie metadata. For the VR movies produced by VMD, the tags are added as follows: 
360VideosMetadata.py -i -s top-bottom movie-orig.mp4 movie-tag-injected.mp4

Uploading the movie into YouTube

The process of uploading the movie to YouTube is the same as for most traditional 2-D movies, except that you must ensure that the movie is also marked as 3-D stereoscopic, and that you want it to play at maximum quality by default.

To tag the movie so that it uses high-quality playback by default, add this tag to the "Tags" section in the YouTube video manager: yt:quality=high

Setting the movie as "3-D" can be somewhat tricky as it won't be possible to do this until YouTube has completely processed the movie to produce all of the lower and intermediate resolution movie versions needed for smooth video streaming. Once YouTube has processed the movie, the 3-D/stereoscopic movie format checkbox should be selected in the "Advanced" tab of the YouTube video manager tools. You can tell if it is updated or not because the "top-bottom" format choice will not appear until YouTube has processed the movie completely (do not check the box when it says "side-by-side").