Rokoko Studio Subscription: Rokoko Studio Plus, Pro or Enterprise
In order to stream to Unreal Engine, you need to have installed the Unreal Engine plugin. Instructions for doing this can be found here.
Video Tutorial: Live Streaming to Unreal Engine 4.23 and above
These videos guides you on how to setup real-time streaming of full performance capture, Face and Body, from Rokoko Studio on to your custom characters in the Unreal Engine 2.23 and above. As we have changed the way of live streaming face capture to Unreal Engine to a more simple way, please follow the second video for that part!
More details on configuring Rokoko Studio and Unreal 4.23 and above can be found here in our blog.
For guidance on how to set up live streaming of face tracking data with live link, please see this article or watch this tutorial:
Rokoko Studio settings
Enabling Rokoko Studio Live
Go to settings and click on Studio Live in the dropdown menu
Configure the data Stream
Before you can begin streaming data you need to configure the receiver information. Start by clicking the cogwheel in the top right corner, which will flip the box around.
Add one receiver by clicking the plus button, which applies one receiver to the list. At this stage you can modify the IP address and Port number.
Click the arrow down button to configure VP (Virtual Production) Port.
Enable and disable receiver
Each individual receiver can be enabled and disabled by toggling the switch
Enable data stream
Enable the data stream by clicking on box. The green outline indicates that the data stream is enabled, click again to disable. (In order to enable a data stream, at least one receiver must be enabled.
Unreal Engine settings
If you haven't installed the plugin or set up a scene before, please follow this article!
Installing the Rokoko Live plugin in Unreal Engine
Setting Up Scene
Use the search bar under the ‘Modes’ tab on the left and type “Smartsuit Controller”
Click and drag the “Smartsuit Controller” inside of the viewport
Use the search bar again and type “Smartsuit Receiver”
Click and drag the "Smartsuit Receiver" inside of the viewport.
Click and drag the "Smartsuit Controller" inside of the viewport.
Change "Streaming Data Port" and "VPListen Port" to the corresponding values used in Rokoko Studio.
"Streaming Data Port" Default is "14043" In Rokoko Studio
"VPListen Port" Default is "14045" In Rokoko Studio
Click the “SmartsuitController” object inside of the World Outliner.
Within the Details tab, make sure that ‘Auto Connect’ is checked.
If this is not checked, you will then have to input the suit ID inside of the ‘Suitname’ field to get it to connect.
You find the smartsuit ID inside of Rokoko Studio in the top right corner.
Setting Up Character
You will need to import your character:
Right-Click within the Content Browser and select “Import to /Game…”
Find the FBX of your character and select the Open button.
If your character isn’t using an existing skeleton, leave Skeleton set to ‘None’.
Once your character is imported, right-click on the skeletal mesh of your character, highlight ‘Create’ and click ‘Anim Blueprint’
Double-click on the newly created animation blueprint
Create the following set of nodes:
Right-click and type “Get Smartsuit Controller” and select that node
Left-click and drag from ‘Return Value’, type “Smartsuit Body Pose” and select that node
Left-click and drag from ‘Animation Pose (Component Space)’, type “Component to Local” and select that node
Left-click and drag from ‘Animation Pose’ and plug it into ‘Result’ on the ‘Output Pose’ node
Left-click the ‘Smartsuit Body Pose’ node and expand ‘Bone Map’
If you are using a Mixamo rigged character, match the following character bones to the bone map:
Rokoko Bone Map Custom Character/Mixamo Character Hip Hips Stomach Spine Chest Spine2 Neck Neck Head Head Head Top HeadTop_End Left Shoulder LeftShoulder Left Arm LeftArm Left Forearm LeftForearm Left Hand LeftHand Left Finger Tip LeftHandMiddle4 Right Shoulder RightShoulder Right Arm RightArm Right Forearm RightForearm Right Hand RightHand Right Finger Tip RightHandMiddle4 Left Upleg LeftUpLeg Left Leg LeftLeg Left Foot LeftFoot Left Toe LeftToe_End Right Upleg RightUpLeg Right Leg RightLeg Right Foot RightFoot Right Toe RightTo_End
‘Relative to Start’ checkbox:
When this is True (checked), the animation will keep the starting position on the XY-plane of the character when the game starts
When this is False (unchecked), the animation will jump to the absolute position of your character inside of Rokoko Studio when the game starts
‘Scale Bones’ checkbox:
When this is True (checked), this will scale the bones to match the model’s body dimensions. This can make the animation look more natural on the character, but may cause unusual effects on the mesh
This is usually left set to False, unless you have a character specifically created to take advantage of this feature
Once you’re done assigning bones, click the ‘Compile’ button at the top left
A warning may appear, however, this will not affect performance so you can ignore it
Close the Animation Blueprint window
Drag and drop the character’s animation blueprint from the Content Browser into the scene
It is important to note that the XYZ axis rotations for your character MUST be set to 0. Otherwise, this could cause some issues with the bones being rotated improperly when the game starts.
Press the Play button and watch as your character moves along with the Smartsuit Pro’s movement