3rd Edition

The VES Handbook of Visual Effects Industry Standard VFX Practices and Procedures

Edited By Jeffrey Okun, VES, Susan Zwerman, VES Copyright 2020
    908 Pages
    by Routledge

    908 Pages
    by Routledge

    The award-winning VES Handbook of Visual Effects remains the most complete guide to visual effects techniques and best practices available today. This new edition has been updated to include the latest, industry-standard techniques, technologies, and workflows for the ever-evolving fast paced world of visual effects. The Visual Effects Society (VES) tasked the original authors to update their areas of expertise, such as AR/VR Moviemaking, Color Management, Cameras, VFX Editorial, Stereoscopic and the Digital Intermediate, as well as provide detailed chapters on interactive games and full animation. Additionally, 56 contributors share their best methods, tips, tricks, and shortcuts developed through decades of trial and error and real-world, hands-on experience.

    This third edition has been expanded to feature lessons on 2.5D/3D Compositing; 3D Scanning; Digital Cinematography; Editorial Workflow in Animated and Visual Effects Features; Gaming updates; General Geometry Instancing; Lens Mapping for VFX; Native Stereo; Real-Time VFX and Camera Tracking; Shot/Element Pulls and Delivery to VFX; Techvis; VFX Elements and Stereo; Virtual Production; and VR/AR (Virtual Reality / Augmented Reality).

    A must-have for anyone working in or aspiring to work in visual effects, The VES Handbook of Visual Effects, Third Edition covers essential techniques and solutions for all VFX artists, producers, and supervisors, from pre-production to digital character creation, compositing of both live-action and CG elements, photorealistic techniques, and much more. With subjects and techniques clearly and definitively presented in beautiful four-color, this handbook is a vital resource for any serious VFX artist.

    Coming Soon …

    Chapter 1: Introduction

    First, Some Ground Rules

    Now, the Introduction

    Why Use Visual Effects

    Chapter 2: Pre-Production /Preparation

    Overview

    Breaking Down a Script – Budgeting

    Ballpark Budget

    More Detailed Budgets

    Bidding

    Plate Photography

    Temp Screenings

    Reviewing Bids

    Contracts

    Rebidding during Shooting

    Rebidding in Post

    Monitoring the Budget and Schedule

    Keeping the Budget Down

    Working with the Director and Producer

    Demo Reel

    The Meeting

    Moving Forward

    Production Departments

    Production Design

    Camera

    Working with the Cinematographer

    Special Effects

    Stunts

    Wardrobe

    Makeup

    Production

    Visual Effects

    Editorial

    Locations

    Production Meeting

    Designing Visual Effects Shots

    Guidelines for Directors

    Storyboards

    Previs

    Objective of the Shot

    Concept Art

    Continuity

    Photorealism

    Original Concepts

    Budget

    Reality and Magic

    Camera Angles

    Framing

    Scale

    Detail

    Speed

    Scaled Images

    Depth of Field

    Sequence of Shots

    Camera Motion

    Less Is More

    Action Pacing

    CG Characters

    Creatures and Character Design

    Powers of 10 Shots

    Visual Effects Techniques

    Technique Considerations

    Additional Suggestions for Determining Techniques

    Development of Previs Techniques

    History and Background

    What is Previs and Other Forms of Visualization?

    What is Previs?

    Different Types of Visualization and When to Use Them

    Visualization: The New Essential

    The Application of Previs: Who Benefits and How?

    The Benefits of Previs

    Project Types

    Post-Visualization

    What is Post-Visualization?

    Why Use Postvis?

    Who Does Postvis?

    Cautions and Suggestions for Good Practice

    The Perils of Previs!

    Passing the Work On

    The Role of the VFX Supervisor in Previs

    Previs: Advanced Techniques

    Visualization Usefulness

    VR as a Tool

    Visualization in Engine

    Render in Engine

    Visualization in Real Time

    AR as a Tool

    Camera Angle Projection

    Introduction

    Drawing What the Lens Sees

    Techvis

    What Is Techvis?

    Who is Techvis for?

    Virtual Production

    What is Virtual Production?

    How is Virtual Production Used?

    Chapter 3: Acquisition / Shooting

    Working on Set

    Common Types of Special Effects

    What are Special Effects?

    A Brief History of Special Effects

    The Special Effects Coordinator

    Working with the Visual Effects

    Visual Effects in Service to SFX

    Special Effects Design and Planning

    Storyboards and Previs

    The Elements: Rain, Wind, and Snow and Ice

    Smoke, Fire, and Pyrotechnics

    Mechanical Effects

    Flying Wire Rigs and Stunts

    Safety

    Greenscreen and Bluescreen Photograph

    Best Practices and Otherwise

    Overview

    Function of the Backing

    Negative Scanning and Digital Conversion

    Backing Uniformity and Screen Correction

    The Alpha Channel

    The Processed Foreground

    The Composite

    Recommended Specifications and Practices

    How to Expose a Green Screen Shot, and Why

    Setting Screen Brightness

    Choosing the Backing Color

    Floor Shots, Virtual Sets

    Foreground Lighting

    Controlling Spill Light

    Lighting Virtual Sets

    Tracking Markers

    On-Set Preview

    Cameras for Blue Screen or Green Screen Photography

    Underwater Photography

    On-Set Data Acquisition

    Camera Report

    Tracking Markers

    Props for the Actors

    Cyberscanning

    Digital Photos

    Lidar/Laser Scanning

    Lens Distortion Charts

    HDRI and Chrome Balls

    Lidar Scanning and Acquisition

    On-Set 3D Scanning Systems

    Types of Technology

    Lidar

    Photogrammetry

    Prop Scanners

    Lighting Data

    Gathering Lighting Data

    Beware of False Savings!

    Using Conventional Still Cameras

    Reference Shooting Considerations

    Clean Plates

    Shooting the Clean Plate

    Locked-Off Camera

    Moving Camera

    Other Issues

    Post-Process

    Alternates without Clean Plates

    Other Uses for Clean Plates

    Monster Sticks

    On-Set Animation Capture: Witness Cam (IBMC)

    Wireless Non-Video Motion Capture

    Factors Affecting Witness Cameras

    Dealing with the Data in Post-Production

    Camera Tracking for Real-Time Visualization

    Camera Tracking Pre-Production

    The Camera Department

    Prior to Shooting

    Current Tracking Systems in Use

    Triangulation As a Method of Recording Camera Data

    Camera/Subject Positional Information

    Basics: The Toolkit

    Basics: Nodal Point

    Photographic Reference

    How to Proceed

    Shooting Video as a Reference

    Rules, Setup, and Testing

    Do a Complete Test Shot!

    Why Run Through Example or Test Shots?

    Digital Cinematography

    Digital Definitions

    High Dynamic Range (HDR)

    Lens Metadata

    Look Management

    The Recording System

    Lens Mapping for VFX

    VFX Photography

    The Camera Array

    Designing an Array Shot

    Technicians

    Shoot Day

    Special Techniques

    Post

    The Future

    Filming Live-Action Plates to be Used in VFX

    Camera Position (Station Point)

    Angle of View

    Lighting Considerations

    Camera Tilt

    Background Quality

    Moving Plates

    Scouting the Camera Positions

    A Case Study

    Camera Cars

    Camera Car Safety Issues

    Purpose-Built Crane Cars

    Vibration and Camera Stabilization

    Road Speed

    Precautions

    Panoramic Rigs

    On the Water

    Air to Air

    Cable Systems

    Shooting Elements for Compositing

    What Is an Element?

    Stock Footage

    Types of Elements

    Generic versus Shot-Specific Elements

    Determining Element Needs

    Cheating

    Backgrounds

    Black Backgrounds

    Line-Up

    Camera Format Considerations

    Assorted Methods for Shooting Elements

    High-Speed Photography and Filming Elements

    Cameras

    Technicians

    Director of Photography

    Lighting

    Application

    Locking Down the Camera

    Video Assist

    Post

    Supervising Motion Control

    What is Motion Control?

    Performance Choreography

    Multiple-Pass Photography

    Scaling

    Import and Export of Camera Move Data

    The Data

    Types of Motion Control Systems

    Motion Control Software

    Camera Types

    Sync and Phase

    Dealing with Production

    Acquisition of Motion / Still Photographic Textures for Mapping onto CG

    Panoramic Backgrounds

    Tiled Stills

    Motion Tiling and Synchronous Plates

    Practical Considerations

    Stills for Textures and Lighting

    Stop-Motion

    Evolution of Stop-Motion Photography

    The Time Required to Shoot in Stop-Motion

    Preparation before Shooting

    Setting up a Shooting Space for Stop-Motion

    Use of Motion Control in Stop-Motion

    Useful Caveats

    Evolution of a Shot

    Use of Stop-Motion in Visual Effects

    Chapter 4: Performance and Motion Capture

    What is Motion Capture?

    Is Motion Capture Right for a Project?

    The Mocap Look

    Technical Specifications

    Entry Point

    Budget

    Which Technology is Right for a Project?

    Gauging a Project’s Needs and Constraints

    Passive Retroreflective Optical

    Active Optical

    Inertial

    Structured Light

    Dense Stereo Reconstruction

    Bend Sensors

    Preparing for Motion Capture

    Actors

    Motion Capture Suits

    Marker Placement – Body

    Marker Placement – Face

    Rigging for Motion Capture

    Shot List

    Technology Considerations

    Hardware

    The Strobe

    Markers

    Lenses

    Filter

    Image Sensor

    Onboard Processor

    Inputs/Outputs

    Setup

    Software

    Acquisition

    Calibration

    Post-Processing

    Reconstruction

    Labeling

    Gap Filling

    Cleaning

    Solving Motion Capture

    Facial Motion Capture

    Facial Actor Survey

    Actor Survey – Hardware

    Reference Data

    Statistical Data

    Facial Rigging

    Facial Acquisition

    Audio

    Facial Motion Capture Solving

    Real-Time Motion Capture

    Real-Time Uses

    Real-Time Limitations

    Alternate Technologies

    Motion Capture Resources

    Virtual Production

    World Building

    Previsualization

    On-Set Visualization

    Virtual Cinematography

    Chapter 5: Stereoscopic 3D

    How 3D Works

    Accommodation and Convergence

    Interaxial Separation

    Toe-in Versus Horizontal Image Translation

    Parallax or Depth Budget

    Positive and Negative Parallax

    Floating Windows

    Fix It in Post

    Stereoscopic Design

    The Emerging Grammar of 3D

    Creative Use of Depth

    Previsualization

    Avoiding Painful 3D

    The Aesthetic of Scale

    Cutting for 3D

    Designing for Multiple Release Formats

    Immersion-Based versus Convergence-Based Stereo

    Native Stereo

    Pre-Production

    On-Set

    Stereography in the Visual Effects Process

    Stereography for Finishing

    HFR as a Solution for Better 3D Movies

    VFX Elements and Stereo

    Introduction – How VFX Elements are Used

    Native Stereo Content

    Mono Capture – Packaged Script and Element Deliveries

    Mono Capture – Hybrid Approach for Stereo Delivery

    Mono Capture – Full CG Approach for Stereo Delivery

    Creating Depth – Layout and Stereography

    Stereo Camera – Depth Wedges

    Stereo Compositing

    Requesting a Full CG Stereo Render Mid-Production

    VFX Production Tasks

    2D to 3D Conversion

    Depth Creation Preparation

    Visual Analysis of 2D Depth Cues

    Pre-Production and Previs for Conversion

    Source and Target Perspective

    Shared Shots/Shared Workflows

    Main Stages of 2D-to-3D Conversion

    Major 2D-to-3D Conversion Workflows

    Special Cases

    Re-projection Mapping Workflow

    Pixel Displacement or Pixel Shifting

    Other 2D-to-3D Conversion Workflows

    Is "Real" Always Right?

    2D-to-3D Conversion Management

    Stereoscopic Visual Effects

    Prepping for the Third Dimension

    Shooting the Third Dimension

    Visual Effects in the Third Dimension

    Photographed Elements

    Accuracy and Attention to Detail

    Artistic Skill Level

    Data Management

    Stereoscopic Digital Intermediate Workflow

    Stereoscopic 3D Process Milestone

    Viewing 3D Dailies

    Projection Screens for Stereoscopic Viewing

    3D Editorial Processes

    Data Workflow

    Applying the 3D Grade

    3D Stereo Deliverables

    3D Home Video Deliverables

    Stereoscopic Window

    The Stereoscopic Window

    Placement of the Window in Relation to the 3D Scene

    Window Violations

    Window Placement Logic

    How to Create a Stereoscopic Window

    Producing Movies in Three Dimensions

    Development – Getting the Green Light

    Production – What to Look Out For

    Chapter 6: Post-Production / Image Manipulation

    Resolution and Image Format Considerations

    Formats

    Transport

    Resolution

    Academy Color Encoding System (ACES)

    ACES Components

    ACES Benefits

    ACES Color Space Encoding

    Viewing ACES

    Preparations for Using ACES

    Image Compression/File Formats for Post Production

    Image Encoding

    Still Image Compression

    Other Lossless Compression Methods

    File Formats

    Color Management

    The Three Guidelines

    Digital Color Image Encodings and Digital Cameras

    Color Management at the Desktop

    Bringing Color Management to Film Workflows

    Digital Intermediate

    Shot Element Pulls and Delivery to VFX

    Introduction

    The Lab

    "Production" Is to "Lab" as …

    The Merge

    The Handoff

    VFX Editorial

    The Select

    The Pull

    VFX Editorial

    Editing within a Shot: The Art of Precompositing (Precomp)

    How It Came to Be

    Modern Day Tracking and Disseminating of Information

    As the Shot Changes

    Wrapping It Up

    Editorial Workflow in Feature Animation

    Introduction

    Editorial Crew Staffing and Structure

    Editorial Involvement with Feature Animation Production Stages

    Communication with Artists

    Starting

    Working with Teams

    Working Globally

    Reference and Perspective

    Shot Production

    Communicating with Artists in Other Departments

    Completion

    Compositing of Live-Action Elements

    Modern Digital Compositing

    Scene Tracking

    Rotoscoping and Paint

    Rotoscoping

    Digital Painting and Plate Reconstruction

    Matte Paintings/Creative Environments

    Matte Paintings: Art of the Digital Realm

    What Is a Matte Painting?

    Matte Painting Pioneers and History

    Visualizing the Matte Painting Shot in Pre-Production

    On-Set Supervision for Matte Painting Shots

    Basic Skills and Tricks of the Trade

    Miniatures and Computer-Generated Sets

    Finding the Best Frame

    Re-Projected Photo Survey

    The Need for Creative Compositing

    3D Matte Painting

    Chapter 7: Digital Element Creation

    Digital Modeling

    Overview: The Importance of Modeling

    Types of Modeling

    Model Data Types

    Development of Models

    Modeling for a Production Pipeline

    Engineering Aspects for Polygons

    Engineering Aspects for NURBS

    Rigging and Animation Rigging

    Rigging: What is It?

    Animation Rigging

    Deformation Rigging

    Texturing And Surfacing

    The Importance of Texture Painting

    Hard Surface Models

    Creature Models

    Types of Geometry: Their Problems and Benefits

    Prepping the Model to Be Painted

    Texture Creation

    Various Other Map-Driven Effects

    Texture Painting in Production

    Model Editing

    Digital Hair / Fur

    Hair Generation Process

    General Issues and Solutions

    Digital Feathers

    Morphology of Real Feathers

    Modeling Digital Feathers

    Similarities between Hair and Feathers

    Differences between Hair and Feathers

    General Geometry Instancing

    Asset Creation

    World Building

    Shot Considerations

    Dynamics and Simulation

    How is a Simulation Created?

    When is Simulation Appropriate?

    Tricks and Cheats

    Important Considerations

    Planning and Preparation

    Software Solutions: A Broad Overview of Current Options

    Particles

    What are Particle Systems?

    The Next Step

    The Birth of Particles

    Creating Effects

    Rigid-Body Dynamics

    How Rigid-Body Dynamics are Created

    Potential Problems

    Other Issues

    Tricks for Getting It Done

    Digital Lighting

    Light in Reality and in Computer Graphics

    Case Study of Reality Compared with Simple CG Simulation

    Visual Sophistication through Texture Mapping

    Physically Derived Shading Models

    Beneath the Surface

    Goals of Lighting in Visual Effects

    Work Flow for Successful Creative Digital Lighting

    The Technologies of Lights in Computer Graphics

    Direct Lighting: Source to Surface to Camera

    Reflections

    Photographed Reflections

    Shadows

    Image-Based Lighting

    Rendering Occlusion

    Ambient Occlusion

    Reflection Occlusion

    Creating Light Sources from Environment Maps

    Physically Based Rendering

    Physically Plausible Rendering

    Volumetric Lighting Effects

    Shader Basics

    What are Shaders?

    Shading Models

    Bump and Displacement

    Map-Based Shaders

    Procedural Shaders

    Shader Design

    Anti-aliasing Considerations

    2D Compositing

    2D File Formats

    Image Quality: Color Bit Depth and Concatenation

    Log vs. Linear

    Low Dynamic Range and High Dynamic Range Images

    Mattes and Pre-Multiplied Alpha

    Working with Rendered CG Elements

    Integration Techniques

    2D Compositing

    Z-Depth Compositing

    Adding Depth of Field

    Adding Motion Blur

    Relighting

    3D Compositing

    Working with 3D Data in a Compositor

    Pan and Tile

    Camera Projections

    Set Extensions

    Coverage Mapping

    3D Mattes

    3D Retouch and Cleanup

    Adjusting Camera Moves

    Particles

    Deep Compositing

    Crowd Generation and Simulation Techniques

    Live-Action Replication

    Sprites

    Computer-Generated Crowds

    Modeling for Replication

    Variation

    Mesh Density

    Animation Cycles for Replication

    Motion Capture

    Keyframe Animation

    Dynamic Motion Synthesis

    Behaviors and Crowd Control

    CG Prosthetics and Actor Enhancements

    On-Set Tracking and Capture Considerations

    Eye Enhancements

    3D Techniques

    2D Techniques

    2D Techniques

    Silhouette Changes

    Re-Projection

    3D Products, Systems, and Software

    Digital Element Creation Process

    3D Graphics Software

    3D Tracking

    Special Effects

    Rendering

    Texturing

    Chapter 8: Interactive Games

    Overview

    How the Gaming Industry and Film/TV Industries are different

    Game Engines and Real-Time Rendering

    Runtime Component

    Disciplines and Job Titles

    Game Design

    Engineering

    Production

    Test

    Art

    The Art Director

    Concept Art

    Environment Artists

    Texture Artist

    Characters

    Hard Modeling

    Props

    Lighting

    Baked vs. Dynamic Lighting

    Shadows

    Effects

    System Effects

    Environmental FX

    Breakables

    Destruction

    Tech Artist

    Animation

    UI

    Real Time Shaders and Materials

    Pre-Rendered Cinematics vs. Real Time Visuals

    Optimization and Runtime Budgets

    Performance Analysis and Profiling

    CPU vs. GPU bound

    Technical Terminology

    User Calibration

    Latency

    "Game Mode" on Televisions

    HDR10

    PBR – Physically Based Rendering

    FBX

    Mip Mapping

    Filtering

    Texel

    Screen Space Ambient Occlusion

    Level of Detail

    Vertex Shaders and Fragment Shaders

    AR/VR

    Future of Gaming

    On-Demand Rendering, Cloud Distribution and Ray Tracing

    Chapter 9: Complete Animation

    What Is An Animation Project?

    Full Animation versus Visual Effects

    Difference Between Visual Effects and Animation

    Production Pipelines

    Production

    A Survey and History of Animation Techniques

    Traditional Animation

    Stop-Motion

    Computer Graphic Technology

    Considerations for a Full CG-Animated Feature Pipeline

    CG Feature Animation Pipeline

    Managing an Animated Film

    Film Management and Personal Style

    Building Brain Trusts

    Building the Core Creative Team

    Writing and Visual Development

    Working with a Studio

    Facilities and Environment

    Managing the Event

    The Production Process: An Animator’s Perspective

    Working on CG-Animated Content in Live-Action Features

    Planning the Process

    Production

    Character and Environment Interaction

    Chapter 10: General Workflow Considerations

    Virtual Studio Technology

    Analysis of A Production Workflow

    From Workflow to Pipeline

    Service Bureau versus In-House Requirements

    Design of a Production Workflow

    From Analysis to Design

    Deploying a Production Workflow

    From Design to Implementation

    Infrastructure

    Tracking Assets

    What is Task and Asset Tracking?

    Commercial Task and Asset Tracking Systems

    Building Task and Asset Tracking Systems

    Scene Assembly

    3D Scene Assembly

    2D Scene Assembly (Compositing)

    Working Across Multiple Facilities

    Images

    Models

    Texturing

    Animation

    Compositing

    R&D

    Chapter 11: VR / AR (Virtual / Augmented Reality)

    A Note from the Editors

    Prelude to Virtual Reality & Augmented Reality

    Pre-Production for VR/AR

    Production for VR/AR

    Post-Production for VR/AR

    Editorial Post-Production for VR/AR

    How to Direct the Viewer?

    The Post-Process

    Nonlinear Editorial, Timelines and Edits

    World Lock and Forced Perspective

    Types of Head-Mounted and Handheld Displays

    What are the HMD and the "VR Presence"?

    Electronical Designs of HMDs and their Respective Performances

    Optical Designs of HMDs and their Respective Usages

    Image Quality Factors in VR HMD Displays

    Hemispheres and Domes

    Game Mechanics Are What it is All About

    Overcoming Doubt and Preconceived Notions

    Dome Projections

    The Future of Domes

    VR/AR Tracking Displays

    Uses of VR

    Enterprise

    Narrative Storytelling

    Future of VR and AR

    This is Just the Beginning

    Acknowledgments

    Appendix A: Charts and Formulas

    Appendix B: Credits / Titles to Be Submitted in Accordance with VES Guidelines

    Appendix C: Glossary

    Index

    Biography

    Jeffrey A. Okun, VES, is an award winning Visual Effects Supervisor, having worked on a large number of feature films, commercials and television shows. He started in the inudstry in 1976, and he is currently a member and Fellow of the VES and The Academy of Motion Pictures, Arts and Sciences as well as the American Society of Cinematographers (ASC), the Television Academy and the Editors Guild. He was the chair of the VES for 9 years and 1st vice chair for 16 years, and chair of the L.A. Section for 2 years. He created the VES awards along with Tim McGovern and Kim Lavery, VES.

    Okun created a visual effects tracking and bidding software in 1992 that is still in wide use within the industry today, as well as the revolutionary visual effects techniques dubbed the "PeriWinkle Effect" (an underwater blue screen technique) and the "Pencil Effect" (accurately predicts the final visual effects count and budget).

    Susan Zwerman, VES has been a member of the VES since 1998. She is a highly respected Visual Effects Producer who has been producing visual effects for more than 25 years. Zwerman is also a well-known seminar leader and author. As Chair of the DGA’s UPM/AD VFX Digital Technology Committee, Susan emphasizes the importance of the visual effects teams to DGA members at visual effects seminars organized under her guidance.

    In 2013, Susan received the Frank Capra Achievement Award in recognition of career achievement and service to the industry and the Directors Guild of America. She is also a member of the Academy of Motion Picture Arts and Sciences, the Producers Guild of America and a Fellow of the VES.