Category: VFX Fundamentals
Collaborative Unit – Character Animation
With this idea, there would be 2 animations that I had to work on. It’s going to be the first time I work intensively on animating realistic character, so despite my lack of experience, this is something I’m excited to learn and try out. I already have made a character of my own in Daz, opting for using the software for animation as well as I know animating on Daz is a bit more intuitive than traditional animation, which can help me save some time.
I decided to use 30fps (since 25fps is too realistic, and 60fps – which can be perfect for slow motions, yet might be too heavy and taking too long to render later on), aiming to make the final video around 1 minute long. That would be around 2000 frames to work with. 1000 frames for the running animation, and 1000 frames for the giant girl.
Running Girl Animation
This animation was thought to be an easy one, as I found a few running presets in Daz, ready to use. However, those animations are pretty fast, and I wanted the sequence to be slow so that we can showcase more of the environment and the movement of the character.
I spent a lot of time trying to slow down the running girl animation, opting for manually moving each and every keyframe just so I have more control over the flow of the sequence. I wanted the first few seconds to be significantly slow, where the camera would zoom into her face, her slowly opening her eyes then start running. Overall, I reduced the speed to 〜4 times for the animation to fit into 〜1000 frames range, adding some eye blink animation and expression as well.
I intentionally left 200 frames at the beginning, in case we need to do any cloth/hair simulation later on. The screenshot shows the overall keyframes of how this animation plays out.
Nuke: CG (Part 2)
ReadGeo node
- Used to Read/Import Alembic (.abc) / .obj / .usd files
- Can be rendered through ScanlineRender
StMap node
Functionality:
- An STMap is used for image distortion based on a UV map (STMap).
- It works by using the red and green channels of an image to define the new X and Y coordinates for pixel remapping
- The STMap node takes an input image and an ST map and warps the input image according to the coordinates defined by the ST map.
PositionToPoints node
- Converts a position pass (an image where RGB values represent 3D coordinates) into a 3D point cloud.
- Useful for visualizing the spatial layout of a scene rendered from 3D applications within Nuke.
Import 3D geometry and texture:
- Use the ReadGeo node to import 3D model into Nuke. Connect the ReadGeo node to a Scene node to include the model into 3D scene.
- Apply the STMap node to warp or adjust a 2D texture based on the UV mapping specified in the STMap image, using UV Expression . Connect your texture to the STMap node as the source.
Relative Path: Reconnect or keep plates’ directory by copy the project directory (found in project setting)
Example: [python {nuke.script_directory()}]
Paste this before the root of the folder
Example: C:/Users/23037923/OneDrive – University of the Arts London/Nuke/Week_13_CG Nuke/Images/LegoCar_No_PPlane_V3.exr
to [python {nuke.script_directory()}]/Images/LegoCar_No_PPlane_V3.exr
-> This will ensure the link of directory even when you move the folder around
Tip: you can Copy/Paste a node or a setup as python code to share
Garage Project: Initial Idea & Modelling
For this garage project, I have come up with this idea of intergrating a plant communication machine into the scene. I used AI to visualize and come up with some interesting shapes in mind of how this machine is going to look like. I want the look to be sci-fi but not with heavy machinery, rather more organic modelling. If I have more time at the end I want to make some complex wire connection with some human intergrated into the scene as well.
Through this project, I want to have a better understanding of how to work with CG in Nuke, combining 2D and 3D elements together seamlessly, as well as improving my modelling and texturing skills. I am very happy with the idea and looking forward to bring it to life 
Variation 1:
At the first blocking out and testing textures stages, I figured I didn’t like the shape of this variation very much. Imagining placing it in the Garage plate, I opt for a design that is wider horizontally, also to look for some hard surface machinery details to make the model more complex and realistic.
Machinery Research
After looking at a few references on Pinterest, I proceeded to try sketching some ideas on top of the plate.
Final Design Visualization:
Eventually, I decided to use AI to visualize some more designs that are tailored to my vision.
I ended up choosing design 6 & 7, I think they look pretty and not too complex, yet can still convey the overall look that I have been looking for.
Most challenging parts:
- Pipes, I thought they would be easy to make but turns out there are so many different techniques that could be used depending on your needs. I eventually found one that works best for me, so I used it throughout the whole design.
- Importing Quixel Bridge Assets into Blender: Textures error/no Texture (Resolved)
Test render:
Final Project: Final video & reflection
Reflection:
Overall, I am happy of how this project turned out, as well as appreciating the knowledge that I have earned through the process of making it. One of the most challenging aspect for me at the beginning that has affected my ability to start was being unsure of what I wanted to make, which got me really anxious and definitely did not help my creativity. I figured I was being too ambitious and got overwhelmed by all the ideas I wanted to make and all the softwares to learn. I eventually had some reflection and reminded myself that the main aim of this project is for me to learn new softwares and practice VFX fundamentals, so I need to lower my expectation for myself and just experiment. I did not have a plan or storyboard at first, and the idea only started coming together as I was creating my Unreal Scene. I have had the vision in mind of the theme that my art usually follows, so every idea after that was by all mean a trust in my own process and experimentation.
Unreal Engine:
In terms of my scene in Unreal Engine, this was the first time that I have used the software and was definitely amazed by how powerful it is in handling such a heavy scene full of foliage (in real time as well). That being said, it took me experimenting on a dozen of projects before I was finally satisfied with what I created, which eventually helped me get more fluent with the software itself. If I have had more time or had a clearer vision in mind, I would have modelled more of my own objects and learned how to texture them realisticly to put into the scene. However, I optimized by playing around with every single material in the scene, from static meshes to foliage to landscape material. This has definitely made me feel more confident on working with models and materials in Unreal. Moreover, I have learned the workflow of importing Alembic into Unreal for animation. The most time consuming part of this was setting up and applying the material for every single parts of the model. In the future, I want to learn how to animate properly and using FBX or USD format to understand the workflow more intensively, also to give myself more freedom in posing and animating the characters.
Zbrush/Substance Painter/Daz
Throughout this project, I have strengthen my skill and love for 3D art when I got to learn my 2 favourite softwares Zbrush and Substance Painter. The leaf boat in particular was something so simple yet I put much effort to make it my own creation. I definitely have spent way more time than I needed to on the sculpting of the boat. If I had to do it again, I would have painted the veins of the leaf with Substance Painter. Yet the workflow that I took luckily taught me valuable lessons in how to work with high & low poly in Zbrush, as well as the baking process before moving to texturing. Considering what I have learned for this, I consider it as a win
Other used softwares:
Blender: Modelling, Particle system, Animation
Nuke: Compositing the bubbles
Photoshop: Texture creation and editting
Premiere Pro: Final Video editing and rendering
What I want to improve in the future projects:
Apart from strengthening my skill in animation, modelling/sculpting/texturing, I definitely want to be more mindful about the filmmaking/cinematic aspect in the future. I think the final video turned out beautifully, yet it lacks storytelling. I believe if I had solidify a vision earlier on, I would have spent more time on planning, writing a script and making a storyboard. This is typically a workflow known in the industry as well, so eventhough I have always been working purely based on intuition and experimentation, I need to improve on this so I can create more impactful visuals and for anyone who wants to understand my creative process better.
Furthermore, I want to use Nuke more in my future projects since I think it’s a very powerful software. However I have considered its uses in this project and thought it would be better if I know how to to 3D tracking and compositing due to the light and shadow of the scene (which we haven’t fully learned on yet). Luckily the bubble worked out perfectly as it has a complex shader that I have set up, which would be hard to transfer from Blender to Unreal, therefore the use of Nuke in this case is justified and helped blending the bubbles into the scene nicely.
After all, thank you so much for all the help from my tutors to make this project happened!
Final Project P3: Camera Sequence & rendering
Working with sequences and rendering process was pretty easy once I got a hang of how to do it. My decision was to work with a 35mm film format as I was inspired by Wes Anderson’s movie style. I believe it added to the fantasy film look, like a fairytale coming to life. I also chose a prime lens and heavily dependent on camera movement to film my final edits.
How to work with sequences:
- Add level sequence > Save it into a Sequence folder
- Drag camera into timeline
- Animate camera using keyframes
I actually was testing my render, expecting some problems to come up that needs troubleshooting. To my surprise, not only my render came out beautifully, the character hair appeared much nicer compairing to the viewport, which was a problem that I was really trying to fix through shaders. After knowing that the rendering process will be good, I came back and spent some time fixing the character’s materials and makeup, particularly the eyes , which kept being black at first. I resolved this by using a different texture.
I decided to render my scene in EXR sequences format as it ensures high quality render and due to the fact that I will later import the sequences into Nuke for further compositing.
Since I wanted to add some bubbles to the scene, I thought it would be better to try using Nuke for this. So I created some bubble animation using Blender’s particle system, then rendered it with a transparent background.
Resources
Final Project P2: Character and Butterflies Creation
As time was restricted, I decided to use my favourite software Daz for human model creation. I have morphed my own characters so it was very convenient to pose them exactly how I wanted. The model also comes with its own textures that I can use later in Unreal Engine. My idea behind the pose is to later situated the model inside the leaf boat, then add some butterflies into the scene. I also wanted some movement so the model doesn’t look like a static mesh, so I animated the eye blink movement, tried my best to make them dynamic and as lively as possible. If I had more time I would have animated the whole body as well, but I feel good about the result nevertheless.
I was not particularly happy with the fact that Alembic import didn’t name or sort out the material, so it took me quite a bit of time to assemble and set up all the materials one by one. Luckily, I have worked with Daz model before in Blender and there is a particular order to each body part, so it did help me in guessing which one is which. Initially I got frustrated with this so I tried out different technique to import the animation in, from using Daz to Unreal plugin (which didn’t work), to FBX, etc… I decided that nothing looks better than the Alembic route so I went back and tried to finish.
After setting up all the materials manually, there were two problems arised.
- Firstly, the hair was really patchy and not looking great at all, especially at the root and area that has heavier shadow. As I imported the whole model in as a single alembic, there was no way I could separated the hair from the body for modification. I eventually took the time and imported a separated model without the hair first. Then I imported the hair into Blender and converted it to a particle system, exported as Alembic and imported again into Unreal Engine. The hair is now read with UE5 Grooming system, so i have more options to adjust the hair roots and tips until it looks better
- Secondly, the eyes textures turns black after importing into UE5. I resolved this by finding a few different textures to test until it worked out. I also had to set the Cornea material to transparent in order for the Irises to show.
I created two simple butterflies animation in Blender, using pictures I found online. Then I put them into Photoshop to create their normal maps and opacity maps. I would say this is one of my favourite technique to learn and can be used in so many cases in the future.
Resources:
Nuke W10 – 3D tracking
3D classic vs 3D beta
- Classic still incorporate FBX or Alembic. Nodes are green.
- Beta adapt to USD workflow. Nodes are red.
- You can’t mix nodes from the classic 3D system and the new 3D system. There are a few notable exceptions to this rule, like CameraTracker and DepthGenerator. Classic 3D system nodes are green by default and new 3D nodes are red.
- The red new version is still quite unstable, we should use the classic version for now.
3D navigation
Tab in viewer – switch between 2D and 3D
Middle mouse – Pan
Alt + right click – Orbit
Scroll – Zoom
Scene Node: is used to manage and assemble 3D geometry, lights, and cameras. It acts as a container/collector for all your 3D elements. When you connect multiple 3D objects to a Scene node, you can manipulate them as a single unit, which simplifies the process of working with complex 3D composites.
Camera Node: In Nuke, the Camera node is a virtual representation of a real or imaginary camera. It’s used in 3D compositing to define the perspective and projection of a scene. You can adjust parameters like focal length, aperture, and field of view. This node is crucial for match-moving and integrating 3D elements into 2D footage, as it helps in replicating the original camera movements and settings.
ScanlineRender Node: This is a rendering node used to render 3D geometry in a scene. It works by converting the 3D data into a 2D image, using the settings from the Camera node
Lens Distortion
Resources:
Working with Lens Distortion: https://learn.foundry.com/nuke/content/comp_environment/lens_distortion/adding_removing_lens_distortion.html
Lens distortion refers to the way a camera lens can warp or distort the image it captures. This isn’t necessarily a flaw; in fact, it’s often a characteristic of the lens design. There are mainly two types of lens distortions: barrel distortion and pincushion distortion.
- Barrel Distortion: This occurs mostly in wide-angle lenses. It makes the image appear bulged outwards from the center. Think of how a fisheye lens makes things look curved and expansive. It’s often used for artistic effect or to create a sense of immersion.
- Pincushion Distortion: The opposite of barrel distortion, this happens more in telephoto lenses. The image appears pinched at the center, making the edges seem to bow inwards. It’s less common in everyday photography and filmmaking.
LensDistortion Node: This node is used for correcting or applying lens distortion. It allows you to analyze an image for any barrel or pincushion distortion and correct it, or you can intentionally add distortion to match CG elements with live-action footage
It’s important to use LensDistortion (Undistort) before doing many match-move or tracking in order to work with correct information then use LensDistortion (Redistort) afterward to return the plate to correct format.
LensDistortion > Analysis > Detect or Draw (Straight Lines) > Solve
Draw horizontal and vertical lines to help giving as much information to Undistort as possible, especially around the edges. Then click Solve.
LensDistortion (StMap) introduce “forward” and “backward” as an option to view and is lighter to use & faster to view.
3D tracking
- Denoise your plate then treat it (color grade etc)
- Roto out area that you want to avoid tracking (things that move/not static, be mindful of reflective objects)
CameraTracker analyses the motion in a 2D footage and extrapolates this movement into a 3D camera path. It tracks various points in the footage (usually high-contrast or distinctive features) across frames to determine how the camera was moving when the footage was shot.
Source: sequence or still
Autotrack: to improve tracking
Click on error-max. click on graph and press F
reduce Max error to 6, click delete unresolved and delete rejected
Usually solved error is anywhere around 1 or below is good
To check tracking:
- Select some point > Create > Cube/Plane
- Plug the object Card into Scene > move it to match the ground plane
3D: UV Unwrap, Baking & Texturing
UV Unwrap
After sculpting the leaf boat, I imported the low poly model into blender and tried to UV unwrapped it. I found this tutorial that showed how to UV unwrap complex object using a tool called “Seam from island” that helped me create seams from separated parts of the model that I have previously unwrapped individually. Since this is my first time trying out UV unwrapping, I don’t think this was the best result, but it’s good enough for me to import into Substance Painter and start texturing the model.
Texturing using Substance Painter
At this point I was wondering why the holes and cuts on the leaf didn’t show up in the viewport, eventhough I saw that it baked out an opacity map. Tempted by the question, I found out that the solution is to add an opacity channel and also to change the Shader Setting into PBR-metal-rough-with-alpha in order for the opacity map to appear properly.
Leaf texture references
I have decided to go with variation 2 as it matches the overall look of my UE5 scene more.
3D: Modelling & Sculpting Leaf Boat
While searching for inspiration for a boat design to add into my scene, I found this illustration of a Leaf Boat that perfectly matches my vision in terms of simplicity and nature-theme related. I have always wanted to practice with organic shapes and texture, so this can be a fun idea to try out.
Some other illustation that inspired the vibe I want to go for
Keywords:
- Pastel colour
- Nature
- Light Fantasy
- Flow
- Dreamlike
- Fairycore
- Mushroom
For me the most challenging part in the modelling process was figuring out how to make this end bit of the leaf that looks like a tree trunk.
I try to keep this as a low poly version as I will sculpt another detailed high poly version before moving to texturing.
After modelling for awhile, I have an idea of animating the leaf to open up and revealing something inside. So I created a second leaf and put it on top. I might make a character and put her inside the boat.
<– I really like the flow curves of the leaves
I then imported the rough model into Zbrush for further modification before sculpting in the detailed version
At this point, I notice that the edges of the cut-off mesh has these jagged problems. So I have found a solution by using
Deformation > Polish by features (with a closed O)
This has helped cleaned up the holes and cuts.
When I’m happy with the overall mesh, I proceeded to researching on leaf textures. I don’t want to overdo it as it’s only a stylized leaf boat, but adding some textures on it would help it matches with my realistic scene in Unreal better.
After a few trial and errors, I have successful sculpted the details onto the leaf. I used Damstandard for the veins as it resulted in more smoothed lines than Standard or Inflate brushes. Then I used Blob to further texturing the rest of the leaf.
Overall this process took me a lot of time, yet I have learned so much through experimenting with different technique. I learned about how to use Zremesher to retpologize my mesh, tried out projecting details from high to low poly (which I ended up didn’t use), how to sculpt with mask and regular brushes in Zbrush, etc…
This was probably the most fun and satisfying part, sculpting the veins on the leaves is pretty abstract yet I figured that they do follow a sort of pattern.
I am overall satisfied with the result so far, I will then move on to Substance Painter to bake the details and paint on this model.
Reference: