Creation Platform Rigging Paradigm

August 27, 2012, Posted by Phil Taylor

Introduction


Scenes in graphics applications are stored as directed acyclic graphs. The graph represents the dependencies between nodes, and the direction in which data is allowed to be propagated.

This graph structures the work to be done during scene evaluation, and the behavior of the graph is defined by the combination of nodes, and connections that make up the graph.

Traditional Rigging Tools.

Transform Hierarchies

In a traditional scene graph based rigging paradigm, the hierarchy of the skeleton defines the way transforms are accumulated. The transform of any given node, is computed by inheriting the parent node’s transform, and applying a local offset transform value. By simply parenting a node to an existing node the hierarchy, the node’s global transform is computed based on the new parent node’ transfrom. The global transform of a given node is implicit in the structure of the the graph.

To generate basic forward kinematic (FK) animation, the local transform values are changed over time. In the simplest case, the local position, orientation and scale values are driven using function curves. These function curves are the tools that the animator works with to construct sequences. By clicking and dragging on nodes in the viewport, the local transform values for the selected nodes are modified, possibly creating keyframes. The graph is then re-evaluated, and the new values are used to compute the updated pose of the character.

Constraints

Constraints are used to modify, or override the transform computed by FK hierarchy. In many cases, the global transform of a node, once it is computed, is overridden by a constraint, such as a position, orientation, or aim constraint. The constraint generates a new transform value for the node which is used to replace the existing transform value. To override the global transform value computed as part of the hierarchy’s evaluation, constraints, IK solvers, and scripted operators are assigned to a node. In your average character rig, you will find many ‘Constraints’, and ‘Inverse Kinematic Solvers’. These types of operators serve to override the inherited transform values with values computed from dependencies.

Manipulation

The graph of the rig is usually set up to define the coordinate space for a manipulated object, or in other words, much of the logic of a rig is computing the parent transform for a given manipulator so that it appears in the correct place on screen, and when manipulated the local transform function curves produce useful results.


During manipulation of objects in the scene, the manipulation system is generating a new local transform value. The manipulation system is used to create or modify the function curves that are driving the local transform value. In most systems, this part of the application is closed, and an implicit part of the architecture. In Fabric, this part is open, and defined entirely in Python, and is discussed in detail below.

Graph Evaluation

The transform value of the manipulator is then read by another node that is a child of the manipulator in the dependency graph. This node could be a constraint, scriped operator, or IK solver. Once this node has computed its result, the next node evaluates reading the values of its dependencies to computes its own value, and so on and so forth.

Manipulating the manipulator nodes cause the graph to become dirty, or invalid, which causes it to be re-evaluated. As the size of this graph grows, the performance of the graph starts to drop, and we see the performance of the character become a problem.

Character Rigging Toolbox

Character Rigging is the art of assembling a set of low level nodes into a large graph that, when evaluated, produces a collection of transforms that define the pose of the character. Tools such as scripted operators and C++ plugins enabled the TD to extend the rigging toolbox beyond what the scene graph provides out of the box.

In Creation Platform, rigging is performed exclusively through the application of solvers to a character rig node.

Characters in Creation Platform

It would be possible to re-implement a classic rigging paradigm in Creation Platform using graphs constructed from a predefined list of low level nodes. We have chosen however to build a character rigging system based on higher level nodes with higher level KL operators for defining behaviors.

The components that make up a character rig are isolated and stored in high level nodes. The computation of the pose of the character is performed by a stack of solvers, which implement high level operators that define the behavior of the character.

Character Skeleton

The hierarchical skeleton definiton of the character is isolated, and stored in a node called the `Character Skeleton`. This node stores all the static values that define the initial, or reference pose of the character, the hierarchy of bones, and any other skeleton data. Hierarchies such as character skeletons are tree structures, and as such can be flattened and stored as arrays, rather than a graph. This enables a character skeleton to be stored in a value in the skeleton node.

The hierarchy of bones that make up a character can be stored as an array value in a single node, using integer index values to specify the parent relationships. A parent index of -1 means that the bone is a root bone, and therefore inherits the transform off the ‘CharacterController’ node.

The bone tables are either built manually, or generated from resource files such as Fbx or Collada. The table is generated or constructed in a sorted order, such that any given bone’s parent appears higher in the table than the bone. This simplifies the computation of global Xfo values as the hierarchy can be walked by simply iterating down the table, as is illustrated in the code sample below.
The table of bones becomes a portable representation of the skeleton. Adding and removing bones is managed in python, and the indices are updated.

Character Rig

The CharacterRig is a node that stores the animation state of the character. The CharacterRig node stores the pose of the character that is computed each frame, and any other data that is used to compute the pose, such as animation values, and simulation values .The pose of the character is stored as an array of transforms with the same size as the skeleton array. Each evaluation, the character rig evaluates its assigned solvers to compute the new pose of the character.

All computation of the character pose is performed through solvers that update the pose of the character stored in the characterRig node.

In Place Rigging

By using high-level solvers to compute the pose of the character, rather than a large graph, the pose of the character is computed in place. There is no need for a `shadow rig` or simplified hierarchy of any kind, because the rig is driving the shadow rig directly. All of the extra nodes that would normaly polut e arigs hierarchy are encapsulated into the sovlers, meaning only the deformer bones are represented in the skeleton. 

Solvers

The pose of the character is computed by a series of solver functions. Each solver binds an operator to the rig that computes the pose of a set of specified bones.

operator solveFKHierarchy(  Xfo controllerXfo,  io Xfo pose[],  Bone bones[], ){  
  for (Integer i = 0; i < bones.size; i++) {
    if (bones[i].parent == -1) {
      pose[i] = controllerXfo *  bones[i].localXfo;
    }
    else {
       // We can assuming that the parent value has already
       // been computed, because it is higher in the bone list.
       // We use it’s value to compute the current bone’s transform.
       pose[i] = pose[bones[i].parent] *  bones[i].localXfo;   
    }  
  }
}

The code above is psuedo-code for an operator that computes the pose of the character based on a static local transform value. Computing the new global pose of the bone based on the bone’s parent, and the localXfo value stored in the bone data structure. Within the Creation Platform framework, all transform values must be computed by an operator. Without an operator driving a given bone’s transform, it will remain stationary at the position defined when the rig was constructed.

There is no implicit computation, and all computation must be defined through the applied solvers

Elimination of Nulls and Helpers

The Solvers are written to encapsulate all the computation required to drive the transforms of the bones over time. Often in a character rig, nulls or helper nodes are attached to the hierarchy simply to provide a transform value with which to drive some other constraint or script.

An example would be attaching a null to a the hip, so that a curve can be constrained to the null, which in turn is used to dive a bezier curve onto which the spine bones are constrained. Building a bezier spline based solver using Creation would involve computing the null transforms, and the bezier spline within the solvers KL code. The elimination of the nulls and curve remove the possibility of these pieces being broken, or deleted by accident during production. All of the computation is encapsulated into a single operator.

Solver Layering

Solvers are applied to the character as a stack, which are evaluated in order of application. Each solver operates on its assigned bones, and subsequent solvers in the stack can read the values of previously solved bones. The enables a layering of solvers. Typically the solvers in the stack are applied such that the primary bones of a character are computed first. The primary bones in a skeleton, such as those that make up the torso and limbs are computed, followed by secondary bones, such as twist bones, muscle simulation, and fat jiggle bones. Dependencies between solvers are satisfied by ensuring that the dependent solvers appear first in the operator stack.

Manipulation in Creation Platform


Manipulation in Creation is managed through the interpretation of mouse events. `Gizmos` are the geometries displayed onscreen which are configured to respond to mouse events, and modify the source animation data of the character.

The solvers specify, in KL code, the gizmos that will be drawn on screen as an interface for animation. Each solver can generate whatever gizmos are applicable for the current state of the character, the the solver can interpret the Python mouse events in whatever way the developer chose. This means for example, that a solver can display gizmos based on external parameters, such as selection, or can expose different sets of manipulators based on configuration settings on the solver. Multiple manipulators can affect the same parameter, and a single gizmo can also affect multiple parameters.

There is no `built-in`manipulation behavior, but instead the workflow for a given character rig is explicitly defined in Python. Pre-defined event handler functions are provided that cover all the basic interactions, such as `click and drag to move, rotate, or scale`. If a TD were to build a rig where paint strokes were interpreted and used to pose a character, this is entirely feasible in Creation Platform by simply defining custom gizmos, and event handler functions in Python.

PsuedoCode

 // draw gizmo  
 gizmoGeometry.drawCircle(....)  
 // specify which callback should be fired   
 gizmoGeometry.setCallback(“footGizmo”)
 

During manipulation, the gizmo geometry is raycasted and if hit, the assigned event handler functions are invoked.

 def footGizmo(event):
   hitPos = event[‘hitPos’]
 

In Python, the footGizmo function defines how the mouse events on that gizmo geometry are converted to changes in the characters animation parameters. These changes are in turn used to modify the keyframe animation tracks driving the character.

From Mouse Events to Keyframes

During interaction, the mouse events describe the actions that the animator is performing as the mouse is moved, clicked, and dragged across the screen. The coordinates of the mouse pointer is used to compute a ray that is fired into the scene from the camera. The rays fired intersect with the gizmos drawn on screen, and the returned ray intersection data structure also lists the name of the event handler for the hit gizmo. The name of the event handler is used to determine which event handler function should be used to interpret the mouse events.

Animation Data

The keyframe animation data for a character rig is stored in a data structure called a `KeyframeTrackSet`. The KeyframeTrackSet encapsulates all of the keyframe tracks that drive a given character rig.

During evaluation, all of the keyframe tracks in the track set are evaluated in parallel, and the resulting values are used to update the `PoseVariables`data structure. The PoseVariables is the set of parameters that the solvers need to be able to generate the pose of the character. The `PoseVariables` encapsulates the source animation state data for the solvers.

The pose variables structure can be generated from a variety of sources, and provides an abstraction layer between the animation data source(e.g. keyframe tracks), and the solvers themselves. The pose variables can be generated from multiple sets of pose variables, enabling systems such as animation trees, procedural animation systems, and or non-linear animation systems to drive the animation of a character.

Integrating with deformers

The character rig outputs a set of bone transforms, which are used to compute the skinning matrices for the skinning deformer. Skin deformation can either be computed using a multi-threaded KL operator, or offloaded to the GPU for computation as part of a shader. Some custom skin deformations, such as joint angle based blend shapes may require the rig to generate custom data, along with the bone transforms. This is easily accomplished in the solver code by adding custom member values to the CharacterRig node which can then be access by the deformation components.

Procedural Animation

By isolating animation source data into the `PoseVariables` structure, animation can be generated procedurally, and used to drive a character`s solver system. We have provided examples of procedural animation which generate pose variables structures dynamically.

Code as a Graph

As part of LLVM’s optimization process, source code is converted into a graph representation for analysis and optimization. LLVM IR is the CPU architecture-neutral ‘Intermediate Representation’ (IR) of the source code. By using KL to define solver behaviours, a graph is in fact generated from the code, and this graph is optimized by LLVM. A high level solver operator represents a large LLVM IR graph which is then optimized by LLVM.

By encapsulating high-level behaviors in KL code, we leverage the LLVM graph optimization passes that optimize and reduce the graph.

Visual Programming

Initially, with Fabric Engine, we built a visual programming interface on top of LLVM, enabling users to interactively build the LLVM IR graph.

Each node in the graph equates to an instruction, and with the concept of nested subgraphs, complex graphs can be built quickly. The result of compiling graphs is very compact and efficient piece of machine code that can be executed directly on the CPU or in future the GPU.

Visual Programing is an interesting topic and will be implemented at some point in the future.

Benefits of the Creation Platform Rigging Paradigm

Portability

Solvers define the behavior of a set of bones, but not the bones themselves as the hierarchy definition is stored in the CharacterSkeleton node. This means that the same solver can be applied to different skeletons, giving them the same behaviour.  The set of dependencies for any given solver are well defined, and specified in the solver code.

High Level Solvers

Solvers can represent higher level concepts that encapsulate the behaviors of an entire limb, digit, spine or even the entire body of a character. Higher level solvers such as full body solvers, or ragdoll are easily implemented as they simply control a greater proportion of the bones of a rig.

For example, a ragdoll solver may drive a collection of primary bones, such as the limbs and torso. This solver can then be layered with the secondary solvers such as those that compute muscle simulation and fat jiggle.

Robustness

Part of defining a solver, is defining its animation interface. By this, I mean the set of gizmos exposed to the animator that will respond to mouse and keyboard events. Only parameters exposed in this way are modifiable at runtime by an artist.
It is impossible for an artist to break the rig, because all the fragile parts that are normally available for… are hidden away in KL operators. The ‘null’ does not exist in the graph, only the transform that would have been computed by the null exists, and only as a stack variable in a KL operator.
It has been argued that often it’s useful to be able to modify the hierarchy of a character in a scene for special purpose, and that abstracting the inner workings of a solver to a black box solver function will limit the ability of TDs to fix rigs on the fly.

The equivalent to this is the evolution of software architecture. The central premise of object oriented programming, is that of interfaces. A class exposes an interface with which other classes interact with. Only the public functions are the ones that other classes can call, and all data access is moderated via functions. Similiarly, a solver in Creation exposes a Python API for TDs to interact with then configuring the solver, and an animation interface via Gizmos, and parameters for the artists to work with.

Performance

Rather than evaluating large graphs, solver operators are defined as functions in KL which get compiled to machine code. The stack of solver operators are evaluated in sequence and the rig pose is modified in place. The overhead to the solver system are minimal, and so the performance of the rigs is exceptional.

Slicing

Rigs are nodes that are designed to support slicing, which enables a single rig node to represent multiple characters in the scene. All of the animation state data for a charcter is stored in the rig node, and when sliced, each slice of the node represents the state of a character instance. Each of the operators assigned to the character rig are evaluated in SIMD to process all of the rigs in parallel.
As set of rigs that share the same set of solvers can be stored in the same rig node. The Skeleton node can also be sliced, which means that each of the rigs in the sliced node can reference a different skeleton, enabling a rich array of characters.

GPU Evaluation

GPUs are especially good at performing SIMD execution, and as the KL GPU compiler improves, character solvers will be able to be compiled to execute on the GPU, meaning hundreds of characters can be evaluated in parallel. This will mean large crowds can be defined using the same rigging components used in single characters for hero shots.

Flexibility

By handing the transformation of mouse events into animation parameter values using Python, there is no graph determining how a given set of function curves are manipulated. Therefore multiple manipulators can be configured to modify the same function curves, and manipulators can also modify any number of animation parameters. This makes it possible to set up manipulators that would normally be impossible in a graph based rigging paradigm. For example, symmetrical manipulators that each can manipulate symmetrical pose values, so when the left manipulator is manipulated, the Left and Right parameters are modified, and when the right manipulator is manipulated, the right, and also the left values are modified.

Ease of Use

Initially, rigging entirely using code sounds daunting, because we lose the iterative, trial and error methods that many of us used to learn rigging, however, there are a few reasons why rigging using Creation is actually a lot easier.

ReUsable Components

The rigging system is designed to make it possible to build re-usable solvers that can then be applied to a character, without necessarily having to understand how the solver works. Although a strong understanding of math is required to design and write solvers, these solvers are easily handled by less experienced TDs building rigs. In a studio context, solvers can be designed and tested in isolation, before being applied to production rigs.

Simple rigs can be built by simply assigning the solvers provided with Creation Platform, or by the community. Based on a common basic framework, solvers are more easily shared and reused.

Debuggability

Every Fabric application is actually running a full build system, that re-compiles the KL source code whenever changes are made. Even on an animators workstation where often bugs are discovered, the source code can be analyzed, and debugged by simply editing the source code of the running application. Solvers in Creation can be debugged interactively, and once bugs are found, the fixed source code is committed to version control. The fixed code can then automatically propagate to all characters in production.
The character rigs are embedded with debug geometry, which enables arbitrary geometry to be drawn onscreen from any solver operator. This is immensely useful, as simply drawing a vector onscreen can provide a vital clue as to why a solver is behaving strangely. Code can be commented out, modified, print statements generated, and debug geometry generated to help a TD understand the state of the running system. These tools make debugging rigs in Creation Platform far easier than debugging scripted or C++ operators.

Building either simple rigs using existing components, or building very sophisticated rigs with custom behaviour are both made a lot easier by the Creation Platform character framework.

Summary

The character framework presented in Creation Platform provides a powerful and novel approach to building complex characters that perform exceptionally well. The system we have defined can be used as a basis for a custom character pipeline. All Python and KL source code is available for the example solvers and we will continue to polish and fine tune the system.

Comments

  • Juhani Karlsson

    Thanks for this! I`m mostly interested in character rigging in Fabric and it seems to have very solid foundation.
    However because Fabrics character tools works bit different than to say Softimage`s documentation like this is very good to have! : )

  • http://charleslooker.wordpress.com/ charles

    Excellent article, i like the use of encapsulation and layering of solvers. Can a system like this work with dynamic transforms? say keyable pivots, space switching etc?

    • Philip Taylor

      Hi Charles, It certainly can. Keyable pivots (like what you find in Softimage) are computed using a more complex set of transforms. The Maya pivot transform math is documented here:
      http://download.autodesk.com/us/fbx/20112/FBX_SDK_HELP/index.html?url=WS1a9193826455f5ff1f92379812724681e696651.htm,topicNumber=d0e7429

      You can see the difference between Max and Maya.
      If you wanted keyable pivots on the foot for example, you could embed the same Math in the solver that controls foot, and get the same behaviour.
      One alternative to keyable pivots, is something called ‘pivot manipulation’, where during manipulation, keyframes are created on an object that make it rotate around a given pivot point. This approach isn’t possible in most apps, but its quite easy to setup with Creation.

      Space switching is usually performed by animating a constraint driving the parent transform of the target node, and storing the delta in an offset value. This approach could also be achieved with Fabrics solver system. A workflow would have to be setup to enable the dynamic assignment of dependencies in the graph, but its certainly possible.

      Soon we’ll have to show an example of space switching, because we’ve had other people asking about that too.

  • Sébastien Courtois

    Really promising !
    About “Skin deformation … offloaded to the GPU for computation as part of a shader”, i’ve implemented a node to test this feature in maya’s viewport 2.0 and performance gain on dense mesh was astonishing even on my “old” gtx 275 !