On my first roto attempt, I decided to divide the roto of the wall into 4 section with bezier curve. This worked out pretty well, yet I think it can be improved after being reviewed in class.
So I came back and try again, this time with smaller section, using B-spline and Open spline to really go into the details.
Passes, often part of the AOVs in a broader sense, are specifically categorized render outputs that represent different elements or effects within a rendered scene. While AOVs provide the technical variables, passes focus on the compositional elements that make up the beauty shot or contribute to visual effects, such as:
Merge (Plus Lights): Diffuse / Indirect / Specular / Reflections
Merge (Multiply Shadows): AO / Shadows
LayerContactSheet is used to view all the passes contained in the EXR
Tips: Ctrl+Shift Drag node onto another to swap/replace node
ModelBuilder (Geometry) node:
Purpose:
Project3D is used to project a 2D image onto a 3D object. It’s like shining a slide projector onto a physical model; the image “wraps” around the 3D shape according to the geometry and camera position.
Project on a Match-move Geometry
Framehold (Choose a frame that is the closest to the camera and appear the clearest) MergeMat (Shader): Similar to the Merge node, this is specifically designed for 3D space operations.
In this setup above, we use 2 framehold nodes, one the closest and one furthest from the camera. Then merge 2 Project nodes together using Mergemat. This approach ensures a more natural result by projecting the patch at different distance.
ModelBuilder (only in Nuke X) – For building geometry. right click and choose mode. right click and change selection mode (like 3D softwares)
Resources:
https://learn.foundry.com/nuke/content/reference_guide/3d_nodes/project3d.html
CameraTracker analyses the motion in a 2D footage and extrapolates this movement into a 3D camera path. It tracks various points in the footage (usually high-contrast or distinctive features) across frames to determine how the camera was moving when the footage was shot.
Several properties in this tab can help achieve a better track:
After tracking process is done, click ‘Solve’
Export by choosing ‘Scene’ or ‘Scene+’
Make sure link output is enabled
To check tracking:
Resources:
https://lesterbanks.com/2020/07/getting-started-with-3d-camera-tracking-in-nuke/
https://lesterbanks.com/2013/09/nuke-3d-tracker-fundamentals/
Reflection:
Overall, I am happy of how this project turned out, as well as appreciating the knowledge that I have earned through the process of making it. One of the most challenging aspect for me at the beginning that has affected my ability to start was being unsure of what I wanted to make, which got me really anxious and definitely did not help my creativity. I figured I was being too ambitious and got overwhelmed by all the ideas I wanted to make and all the softwares to learn. I eventually had some reflection and reminded myself that the main aim of this project is for me to learn new softwares and practice VFX fundamentals, so I need to lower my expectation for myself and just experiment. I did not have a plan or storyboard at first, and the idea only started coming together as I was creating my Unreal Scene. I have had the vision in mind of the theme that my art usually follows, so every idea after that was by all mean a trust in my own process and experimentation.
Unreal Engine:
In terms of my scene in Unreal Engine, this was the first time that I have used the software and was definitely amazed by how powerful it is in handling such a heavy scene full of foliage (in real time as well). That being said, it took me experimenting on a dozen of projects before I was finally satisfied with what I created, which eventually helped me get more fluent with the software itself. If I have had more time or had a clearer vision in mind, I would have modelled more of my own objects and learned how to texture them realisticly to put into the scene. However, I optimized by playing around with every single material in the scene, from static meshes to foliage to landscape material. This has definitely made me feel more confident on working with models and materials in Unreal. Moreover, I have learned the workflow of importing Alembic into Unreal for animation. The most time consuming part of this was setting up and applying the material for every single parts of the model. In the future, I want to learn how to animate properly and using FBX or USD format to understand the workflow more intensively, also to give myself more freedom in posing and animating the characters.
Zbrush/Substance Painter/Daz
Throughout this project, I have strengthen my skill and love for 3D art when I got to learn my 2 favourite softwares Zbrush and Substance Painter. The leaf boat in particular was something so simple yet I put much effort to make it my own creation. I definitely have spent way more time than I needed to on the sculpting of the boat. If I had to do it again, I would have painted the veins of the leaf with Substance Painter. Yet the workflow that I took luckily taught me valuable lessons in how to work with high & low poly in Zbrush, as well as the baking process before moving to texturing. Considering what I have learned for this, I consider it as a win 🙂
Other used softwares:
Blender: Modelling, Particle system, Animation
Nuke: Compositing the bubbles
Photoshop: Texture creation and editting
Premiere Pro: Final Video editing and rendering
What I want to improve in the future projects:
Apart from strengthening my skill in animation, modelling/sculpting/texturing, I definitely want to be more mindful about the filmmaking/cinematic aspect in the future. I think the final video turned out beautifully, yet it lacks storytelling. I believe if I had solidify a vision earlier on, I would have spent more time on planning, writing a script and making a storyboard. This is typically a workflow known in the industry as well, so eventhough I have always been working purely based on intuition and experimentation, I need to improve on this so I can create more impactful visuals and for anyone who wants to understand my creative process better.
Furthermore, I want to use Nuke more in my future projects since I think it’s a very powerful software. However I have considered its uses in this project and thought it would be better if I know how to to 3D tracking and compositing due to the light and shadow of the scene (which we haven’t fully learned on yet). Luckily the bubble worked out perfectly as it has a complex shader that I have set up, which would be hard to transfer from Blender to Unreal, therefore the use of Nuke in this case is justified and helped blending the bubbles into the scene nicely.
After all, thank you so much for all the help from my tutors to make this project happened!
Working with sequences and rendering process was pretty easy once I got a hang of how to do it. My decision was to work with a 35mm film format as I was inspired by Wes Anderson’s movie style. I believe it added to the fantasy film look, like a fairytale coming to life. I also chose a prime lens and heavily dependent on camera movement to film my final edits.
How to work with sequences:
I actually was testing my render, expecting some problems to come up that needs troubleshooting. To my surprise, not only my render came out beautifully, the character hair appeared much nicer compairing to the viewport, which was a problem that I was really trying to fix through shaders. After knowing that the rendering process will be good, I came back and spent some time fixing the character’s materials and makeup, particularly the eyes , which kept being black at first. I resolved this by using a different texture.
I decided to render my scene in EXR sequences format as it ensures high quality render and due to the fact that I will later import the sequences into Nuke for further compositing.
Since I wanted to add some bubbles to the scene, I thought it would be better to try using Nuke for this. So I created some bubble animation using Blender’s particle system, then rendered it with a transparent background.
Resources
As time was restricted, I decided to use my favourite software Daz for human model creation. I have morphed my own characters so it was very convenient to pose them exactly how I wanted. The model also comes with its own textures that I can use later in Unreal Engine. My idea behind the pose is to later situated the model inside the leaf boat, then add some butterflies into the scene. I also wanted some movement so the model doesn’t look like a static mesh, so I animated the eye blink movement, tried my best to make them dynamic and as lively as possible. If I had more time I would have animated the whole body as well, but I feel good about the result nevertheless.
I was not particularly happy with the fact that Alembic import didn’t name or sort out the material, so it took me quite a bit of time to assemble and set up all the materials one by one. Luckily, I have worked with Daz model before in Blender and there is a particular order to each body part, so it did help me in guessing which one is which. Initially I got frustrated with this so I tried out different technique to import the animation in, from using Daz to Unreal plugin (which didn’t work), to FBX, etc… I decided that nothing looks better than the Alembic route so I went back and tried to finish.
After setting up all the materials manually, there were two problems arised.
I created two simple butterflies animation in Blender, using pictures I found online. Then I put them into Photoshop to create their normal maps and opacity maps. I would say this is one of my favourite technique to learn and can be used in so many cases in the future.
Resources:
3D classic vs 3D beta
Tab in viewer – switch between 2D and 3D
Middle mouse – Pan
Alt + right click – Orbit
Scroll – Zoom
Scene Node: is used to manage and assemble 3D geometry, lights, and cameras. It acts as a container/collector for all your 3D elements. When you connect multiple 3D objects to a Scene node, you can manipulate them as a single unit, which simplifies the process of working with complex 3D composites.
Camera Node: In Nuke, the Camera node is a virtual representation of a real or imaginary camera. It’s used in 3D compositing to define the perspective and projection of a scene. You can adjust parameters like focal length, aperture, and field of view. This node is crucial for match-moving and integrating 3D elements into 2D footage, as it helps in replicating the original camera movements and settings.
ScanlineRender Node: This is a rendering node used to render 3D geometry in a scene. It works by converting the 3D data into a 2D image, using the settings from the Camera node
Lens Distortion
Resources:
Working with Lens Distortion: https://learn.foundry.com/nuke/content/comp_environment/lens_distortion/adding_removing_lens_distortion.html
Lens distortion refers to the way a camera lens can warp or distort the image it captures. This isn’t necessarily a flaw; in fact, it’s often a characteristic of the lens design. There are mainly two types of lens distortions: barrel distortion and pincushion distortion.
LensDistortion Node: This node is used for correcting or applying lens distortion. It allows you to analyze an image for any barrel or pincushion distortion and correct it, or you can intentionally add distortion to match CG elements with live-action footage
It’s important to use LensDistortion (Undistort) before doing many match-move or tracking in order to work with correct information then use LensDistortion (Redistort) afterward to return the plate to correct format.
LensDistortion > Analysis > Detect or Draw (Straight Lines) > Solve
Draw horizontal and vertical lines to help giving as much information to Undistort as possible, especially around the edges. Then click Solve.
LensDistortion (StMap) introduce “forward” and “backward” as an option to view and is lighter to use & faster to view.
CameraTracker analyses the motion in a 2D footage and extrapolates this movement into a 3D camera path. It tracks various points in the footage (usually high-contrast or distinctive features) across frames to determine how the camera was moving when the footage was shot.
Source: sequence or still
Autotrack: to improve tracking
Click on error-max. click on graph and press F
reduce Max error to 6, click delete unresolved and delete rejected
Usually solved error is anywhere around 1 or below is good
To check tracking: