One of my major reasons I decided to take this course was to learn how VFX artist in many movies, videos, and other media successfully placed CG elements among live action. Movies like transformers, The Terminator and many more showed CG characters full of life interacting with live action elements in a very realistic way.
It is to answer this question that my final project idea came into fruition. Of course I did not set out to do a production quality 3D action film. I knew from the very beginning that I wanted to do something extremely simple that showcased my ability composite the elements as such
After initial research it turned out that the method or technique artists use (at least one of them) was called match-moving. Match-moving is when a live action video is shot and certain points are tracked by a tracking program. The program then calculates the various tracking information from the points and uses that information to solve for a 3D camera. In other words the program creates a 3D camera that animates almost precisely the same (of course it is a guess and only as accurate as the quality of tracks) as the real world camera. Then the software creates a 3D scene (in maya for this case) with the virtual camera imported in already keyed. It is then up to the user to import all objects/scenes they want to be included in the final shot.
I was able to successfully do this for a rested smart phone on top of a live action table.
Production
The program used for matchmoving is called Matchmover. Matchmover was introduced as part of Maya software package from version 2010. I learned the basics of this program using the included help section under the menu bar.
-Filming base footage.
The first thing needed here is to film the actual shot (live action) where you want to composeting to take place. Matchmover provides very good high quality tracking but a lot of this quality is based on the quality of the original footage. Therefore it is very important to have good quality footage with proper lighting and set up.
When filming my phone, I had a lot of problems finding good track points because my phone is pretty uniform by design, thus it had no natural design properties that allowed for good tracking points (plain simple design). You can see here that I added green construction paper to serve as points I wanted to track, similar to green screening.
-Bringing it into Matchmover
Matchmover only work with image sequences. I had to import this shot into after effect and exported it as a tiff image sequence. Once I was done with this step I then opened it in matchmover via File>load image sequence. Now in order to actually matchmove the 3D object, Matchmover allowed me to import a .obj file or create a primitive myself to serve as the object (i chose the former option). At this point I was not actually working with my main model just yet, it is only necessary to import or create an object just so that the tracking points can be set. (Note: it is very important to try to get the object used in this stage to be the relatively the same size and shape as the real model being used. For example: a cylinder object should be used if you were trying to matchmove a model of a can of soda)
This is done by first changing Matchmover into 3D mode (clicking on the "3D" icon towards the top left corner) and then right clicking anywhere on the workspace and selecting "Import Scene."
-Setting up tracking points
When my object was imported I rotated it to a similar angle based on the live action object being replaced (in this case my phone). Then I switched from light to full mode. To make the long story short, I connected the tracking points on each of the green construction paper cut outs to the corresponding point (more or less) on the 3D object. There was a total of 7. Once the points were set up I used the analyze command (similar to what we learned in combustion) to run through the track of each point. (press FN+F3 to do this). I also fixed the track over the time line by changing the pixel it was tracking for a better/sharper one. Matchmover does an amazing job providing you with smooth tracks so you don't have to fine tune much.
-Where the magic took place - solving for 3D camera
Once all of my tracks have been analyzed and I was happy with them. It was then time to solve for the 3D camera. I made sure I saved the file before doing this in case it didn't work properly. I selected 3D tracking>solve for camera up from the menu bar. If everything went smoothly the camera would compute the rotation, transformation and scaling of the object over the course of the shot, just like it should if it was really there (in live action). To see this clearly I made sure to select view>lock on camera to see from the camera's perspective. The process did go smoothly for me and I was done with matchmoving.
-Putting them all together
Finally Matchmover allowed me to export the project as a Maya scene. To my surprise, the scene generated by Match mover contained a key-framed camera with an image plane attached to it that played the image sequence imported earlier. In addition Match mover also placed empty placeholders that indicated where the model should be placed in order to look convincing (these were the 7 tracking points I have created earlier). Finally all I had to do was import my real model and line it up with the placeholders. I also had to scale among different axis to properly cover my real phone (in the image plane) with my virtual phone (positioned in front of it) from the camera's perspective. This is why I mentioned the importance of getting a live object that is very similar in size.
And there you have it, my report on how this was made. If you would like to do this yourself, I highly recommend you open Matchmover and open it's help>contents menu. It is very simple to follow and it is also short.
