In this article, we will be going through how to stream the data from Rokoko Studio into Houdini and onto a character. The process is similar if you want to retarget your animations to your character.
Enable streaming in Rokoko Studio
Go to settings and click on Studio Live in the dropdown menu
Turn on streaming to Houdini by enabling the plugin and enabling the default IP and port.
Updating SideFX Labs
When opening Houdini make sure that you have the latest version of SideFX Labs on your computer. To update, go to Windows ->SideFX Labs. Then click on the "Update" button
Setting up the nodes
Importing your character
To create a node click TAB on your keyboard a search for "geometry" and hit Enter.
Rename it to something appropriate and double click the node to access the "Geometry" node
Inside the "Geometry" network view create the "FBX Character Import" node by clicking tab and searching for it
In the "Parameters" panel go to the "Import" tab and locate the .fbx file for your character in the "FBX File" parameter.
The character is now visible in the viewport.
Connect the data and retarget
Create the "Bone Deform" SOP node to deform the geometry using live-streamed data that we are going to feed the node.
Connecting the data from Rokoko Studio
To begin streaming the Rokoko Studio data into Houdini we will create the node "Labs Rokoko Mocap Stream". Simply click TAB and search for "rokoko". When placing the node go to the "Parameter" panel and make sure that the "IP" and "Port" parameter matches the settings that we configured in the "Enable streaming in Rokoko Studio" section of this article
Preparing the skeleton
To begin retargeting animation data onto the character create the node "Rig Match Pose" and connect "output3" from "fbxcharacterimport1" to "input1" of "rigmatchpose1". The take the output from "rokoko_mocap_stream1" and connect it to the last available input of "rigmatchpose1".
Click on the newly created node and enable the display flag.
Go to the "Parameter" panel and enable "Enable Target" in the "Interaction" section. Disable "Enable Source"
Click inside of the viewport and hit "Enter". This will show the rest pose of the "Source" and "Target" skeletons. The two skeletons rest pose needs to match. Because we enabled the interaction of the "Target" skeleton we will manipulate that skeleton to match the rest pose of the "Source" skeleton.
If something doesn't match the rest pose of the "Source" skeleton we will have to manipulate the "Target" to match. This is done by rotating and translating the specific joints. In the gif below we are straightening out the fingers to match the pose of the "Source" fingers. If you have a character rigged in A-Pose you will have to put it into a T-Pose using this method.
Mapping the bones
To map the matching bones we will create the node "Map Points". Place it so that it connects to the "rigmatchpose1" node. Then change the display flag to "mappoints1"
Select the newly created node, go to the viewport, and hit "Enter". Then connect the matching joints from the "Target" skeleton onto the "Source" skeleton by clicking and dragging between the matching joints.
Once you have mapped all the joints create the node "Full Body IK" and connect the input to the output of "mappoints1"
Then connect "output1" of "fullbodyik1" to "input3" of "bonedeform1". When connected change the active display flag to the "bonedeform1" node.
Character with animation
Using this setup will stream the data into Houdini and onto the specified character.