Animation retargeting is the process of transferring an animation from one character (the source) to another character (the target) that may have different proportions, joint structures, or skeletal configurations. This technique is widely used in computer graphics, particularly in video games and film production, to reuse animations across different characters, saving time and resources.
For animation, and specifically, for retargeting is very important the space used when computing bone's transformations so the results can be very different:
- World space or Global Space: Object transformation based upon its place in the world view, that is, relative to the (0,0,0) of the world.
- Local space: Object transformation relative to its parent.
- Source Character: Original character that has the animation data
- Target Character: Character that will receive the animation
The skeleton of an avatar is defined by a heriarchy of joints (defined by positions) or bones (defined by an initial position, direction and length), depending of the software. Each joint has a scale, a rotation and a position, the latter being an offset with respect to its parent. The position with respect to its parent and children, determine in which axis a limb needs to be rotated to achieve a particular pose. This implies that depending on how the skeleton is modeled, the local transforms of a skeleton might differ from others, even if they are successfully applied to the same mesh. The retargeting algorithm attacks this issue by working in world space with an auxiliary pose that looks the same for both skeletons.
When working with skeletons, it is important to consider the following questions about bone transformations:
- The scale has to be homogeneous and with positive values. Otherwise, it can give unwanted problems.
- It is better to calculate rotations using quaternions. This avoids gimbal lock.
To perform a retargeting, a correspondence between the joints of the source skeleton and the target skeleton has to be established. Not only for the names, also for the missing joints if it’s the case. This often requires manual setup or the use of automated algorithms.
A pose is a particular configuration of the transformation (position, rotation and scale) of the bones/joints of a skeleton. The most common poses are the following:
- Bind Pose: Default initial pose for the skeleton before it is animated. Used as a reference for attaching/binding the mesh to the bones (skinning). Usually, the pose shapes a T (T-pose) or an A (A-pose)
- Rest Pose: Default or neutral pose of the skeleton when no transformations or animations are applied. It is likely that this pose will be the same as the bind pose.
- Base Pose: Pose of the skeleton without rotations applied to the bones.
The algorithm used derives from @sketchpunk. Given two skeletons, an animation can be approximately retargeted using an auxiliary pose shared by both skeletons. As long as the bone heriarchy and the auxiliary pose of both skeletons are similar, the retargeting can be successfully performed. However, some issues can appear as bone proportions might differ. This might result in missed bone contacts. This algorithm is particularly useful to retarget vague animations such as running or walking.
The algorithm starts mapping each joint of the source skeleton to a corresponding joint in the target skeleton by name. In this approach, the same skeleton structure is assumed, so the mapping is one-to-on. But some joints may need to be interpolated if the skeleton has a different structure.
Once the mapping is done, the next step is posing the source and target avatars into the same pose, each with their respective local transforms. This auxiliary pose can be the bind pose as long as the configuration is exactly the same for both skeletons. This ensures each bone to be retargeted has the same direction in world space for each of the avatars.
When an animation is applied to the source avatar, each local rotation can be transfered from one avatar to the other by computing the offset with respect to the auxiliary pose. Since both skeleton share the same auxiliary pose, the offset in world space should be the same. Then it is only a matter of changing between local and world spaces.
The rotation (quaternion) computations look as follows (where bind means the auxiliary pose):
trgLocal = invBindTrgWorldParent * bindSrcWorldParent * srcLocal * invBindSrcWorld * bindTrgWorld
These arbitrary multiplications can be explained as follows:
- Each bone's new rotation will be transformed isolated from the rest of the other bone's new rotations. Instead, the auxiliary pose will be used for the rest of the bones.
- srcWorldRot =
bindSrcWorldParent*srcLocal: compute world rotation of the avatar with this bone's new rotation and the auxiliary pose for the rest of the bones. - offsetWorldRot = srcWorldRot *
invBindSrcWorld: by multiplying on the right by the inverse of the auxiliary pose (including the current bone), the auxiliary pose is removed and results in the world offset rotation.
Note
Multiplying by the left instead, would result in the local offset rotation.
- trgWorldRot = offsetWorldRot *
bindTrgWorld: add the offset to the target skeleton's auxiliary pose. Since both source and target poses are the same, the same movement should be expected. trgLocalRot=invBindTrgWorldParent* trgWorldRot: by multplying on the left by the inverse of the auxiliary pose (excluding the current bone), the parent's auxiliary pose is removed and results in the local retargeted bone rotation. The current bone needs to be excluded from the inverse because the complete local retargeted rotation is desired, not just the offset with respect to the auxiliary pose.
The implemented algorithm uses a more sophisticated approach which applies some extra world rotation as, sometimes, it is easier to modify the container of the skeleton, rather than the actual skeleton in order to build the auxiliary pose.
trgLocal = invBindTrgWorldParent * invTrgEmbedded * srcEmbedded * bindSrcWorldParent * srcLocal * invBindSrcWorld * invSrcEmbedded * trgEmbedded * bindTrgWorld