3D classic vs 3D beta

• Classic still incorporate FBX or Alembic. Nodes are green.
• Beta adapt to USD workflow. Nodes are red.
• You can’t mix nodes from the classic 3D system and the new 3D system. There are a few notable exceptions to this rule, like CameraTracker and DepthGenerator. Classic 3D system nodes are green by default and new 3D nodes are red.
• The red new version is still quite unstable, we should use the classic version for now.

3D navigation

Scene Node: is used to manage and assemble 3D geometry, lights, and cameras. It acts as a container/collector for all your 3D elements. When you connect multiple 3D objects to a Scene node, you can manipulate them as a single unit, which simplifies the process of working with complex 3D composites.

Camera Node: In Nuke, the Camera node is a virtual representation of a real or imaginary camera. It’s used in 3D compositing to define the perspective and projection of a scene. You can adjust parameters like focal length, aperture, and field of view. This node is crucial for match-moving and integrating 3D elements into 2D footage, as it helps in replicating the original camera movements and settings.

ScanlineRender Node: This is a rendering node used to render 3D geometry in a scene. It works by converting the 3D data into a 2D image, using the settings from the Camera node

Lens Distortion

Resources:
Working with Lens Distortion: https://learn.foundry.com/nuke/content/comp_environment/lens_distortion/adding_removing_lens_distortion.html

Lens distortion refers to the way a camera lens can warp or distort the image it captures. This isn’t necessarily a flaw; in fact, it’s often a characteristic of the lens design. There are mainly two types of lens distortions: barrel distortion and pincushion distortion.

1. Barrel Distortion: This occurs mostly in wide-angle lenses. It makes the image appear bulged outwards from the center. Think of how a fisheye lens makes things look curved and expansive. It’s often used for artistic effect or to create a sense of immersion.
2. Pincushion Distortion: The opposite of barrel distortion, this happens more in telephoto lenses. The image appears pinched at the center, making the edges seem to bow inwards. It’s less common in everyday photography and filmmaking.

LensDistortion Node: This node is used for correcting or applying lens distortion. It allows you to analyze an image for any barrel or pincushion distortion and correct it, or you can intentionally add distortion to match CG elements with live-action footage

It’s important to use LensDistortion (Undistort) before doing many match-move or tracking in order to work with correct information then use LensDistortion (Redistort) afterward to return the plate to correct format.

LensDistortion > Analysis > Detect or Draw (Straight Lines) > Solve
Draw horizontal and vertical lines to help giving as much information to Undistort as possible, especially around the edges. Then click Solve.

LensDistortion (StMap) introduce “forward” and “backward” as an option to view and is lighter to use & faster to view.

1. Denoise your plate then treat it (color grade etc)
2. Roto out area that you want to avoid tracking (things that move/not static, be mindful of reflective objects)

CameraTracker analyses the motion in a 2D footage and extrapolates this movement into a 3D camera path. It tracks various points in the footage (usually high-contrast or distinctive features) across frames to determine how the camera was moving when the footage was shot.

Source: sequence or still

Autotrack: to improve tracking

Click on error-max. click on graph and press F

reduce Max error to 6, click delete unresolved and delete rejected

Usually solved error is anywhere around 1 or below is good

To check tracking:

• Select some point > Create > Cube/Plane
• Plug the object Card into Scene > move it to match the ground plane

In today’s class, we learn about stages in Production for film in compositing,
– Temps/Postviz? – Rough version/mock up?
– Trailer – Key shots
– Finals
– Quality Control

Project Management Softwares:
– Google Doc/Spreadsheet/Notion
– Ftrack
– Shotgun

Production roles:

• Line Producer: The person who manages and keeps track of the whole team in terms of production.
• VFX Producer: ensure studio projects completion on time. Strive to complete project by the set deadline and within budget & available resources.

Tech Check before publishing a version of your work:

• Check if you did all the notes of the shot/ follow briefs
• Compare new version with previous version
• Check editorial? Any retime in the shot?
• Has your shot had the latest CG and FX?
• Does your shot have the latest camera match move?
• Write in personal notes
• Do you have different alternatives for one shot?
• Quality Control

1. Roto Paint (P)

The RotoPaint node gives you a broader scale of tools to use than Roto, though many of the controls are shared across both nodes. It is used to clean up and clone out unwanted elements from the plate. Cons: Heavier than original Roto node

• Clone: Ctrl + click = choose clone source. Shift + drag = change brush size
• Reveal: Reveals the original background after cloning (change paint source from ‘fg’ to ‘bg’ to take effects).
• Paint: Paint/draw on the shot
• Blur, sharpen, smear, dodge, burn: Adjusts the area painted on the shot (similar to Photoshop).
• Dodge for highlight, Burn for shadow

2 ways to separate rotopaint from the plate

• Film grain is the texture in photographic film, caused by small metallic silver particles developed from light-sensitive materials.
• Unlike digital noise, which is electronic interference in digital cameras, film grain is a physical attribute of analog film.
• Grain and noise both impart a comparable appearance, sensation, and texture to filmic images. Techniques utilized for footage from both digital and film sources, such as adding grain, adjusting perspective, and applying denoising, help in creating a scene that feels more natural and organic.

• When working on grainy footages, we will typically start with denoising procedure first. This step is essential to help the tracking of elements that we want to replace in the scene later on.
• Once the editing is completed, grain is re-added to the video to restore its original texture and maintain visual continuity throughout the shot
  
  

• Denoise Node: Link to your source footage the set a bounding box to analyze the existing noise in the footage. After this, you can fine-tune settings like the amount of Denoise, Smoothness, and other parameters.
• Grain Node: Can be quite difficult to match the grain to source footage as you’ll need to go through different RGB channels one by one and adjust settings like grain size, irregularity, and intensity to achieve a close match.
• F_ReGrain Node: Offers a more precise grain matching compared to the Grain node. You connect the grain to the original footage and then link the Src (source) from the shot you are trying to replicate. Note that this is available only in NukeX and is much heavier than Grain node.

DasGrain node

• normalised grain – average denoised, clean plate and cleaned up plate
• common_key = look for the difference between clean plate and cleaned up plate

Homework

I tried to match grain using Grain node by going through RGB channel. I wouldn’t say it looks perfect, especially for Blue channel. But the result seems to be pretty good at the end.

Transform node deals with translation, rotation and scale as well as tracking, warping and motion blur. Sometimes you want to animate these values just by using one transform node, but sometimes it’s better to use rotation or scale node separately to understand the process better.

2D tracker

Tracker node: container of pixel in x and y
– Allows you to extract animation data from the position, rotation, and size of an image.
– Using expressions, you can apply the data directly to transform and match-move another element.
– To stabilize the image you can invert the values of the data and apply them to the original element
– We can also make several tracking nodes from the main Tracker node to automatically make the scene stable, match the movement, and either reduce or add shakiness.

General process for tracking an image:

1. Connect a Tracker node to the image you want to track.
2. Use auto-tracking for simple tracks or place tracking anchors on features at keyframes in the image.
3. Calculate the tracking data.
4. Choose the tracking operation you want to perform: stabilize, match-move, etc.

– 2D track: x & y
– 2.5: still x & y but in 4 points to mimic a sense of perspective. Use planartracker for this.
– 3D: XYZ

– inner tracking square: track the main shape/point
– outer tracking square: find movement around the inner square to track it

Sometimes we should treat the original plate to obtain better tracks, if the scene is too noisy or grainy. In this case, we use a ‘Denoise’ node to make the noise or grain less, helping the tracker read the changes between frames better. We can also use tools like ‘Laplacian’, ‘Median’, or ‘Grade Contrast’ to fix the grain issue.

1. Denoise plate (denoise node – median node).
2. Increase contrast with grade node.
3. Lapalachian node can help in certain case to lock better tracks.
   

It’s always important to use a Quality Control (QC) backdrop to make sure the tracking and any added rotoscoping is done right.

I attempted to do this assignment twice, as the first time I was really confused of the process and messed up the nodes. Doing it the second time made me realize that I first need to track the 4 points on the phone to create a Transform_stabilize node. This comes before the first Merge operation, following with a Transform_matchmove from the same tracking. Doing this ensures that the phone mockup is merged correctly with the tracked points.

I was not satisfied with the roto of the fingers at first because of the green spill. In this particular node setup as well haven’t touched on green spill operation, yet I managed to compensate for it by using Filter_Erode node with a slight blur to the edges to make the roto not too obvious.

I also used Erode node for the phone mockup as it doesn’t fully cover the green screen at the top of the phone, no matter how accurately I tried to adjust the CornerPin2D.

Homework feedback:
– My final work is good but personally I was not satisfied with the roto of the finger as it is still woobly, I want to learn how to roto better in the future.
– I learned that I could have used Curve Editor to smooth out my animation by using curve editor to control animation from linear to flow
– x → f → press h on in & out points (for easy in & out)
– y → f → press h on in & out points to smooth animation or move curve