Create ticket
Welcome
Login  Sign up

Getting Started: Streaming to Unreal Engine

Requirements

Rokoko Studio Subscription: Rokoko Studio Plus, Pro or Enterprise

In order to stream to Unreal Engine, you need to have installed the Unreal Engine plugin. Instructions for doing this can be found here. 

Video Tutorial: Live Streaming to Unreal Engine 4.23 and above

These videos guides you on how to setup real-time streaming of full performance capture, Face and Body, from Rokoko Studio on to your custom characters in the Unreal Engine 2.23 and above. As we have changed the way of live streaming face capture to Unreal Engine to a more simple way, please follow the second video for that part!

Body Mocap


More details on configuring Rokoko Studio and Unreal 4.23 and above can be found here in our blog.

Face Capture

For guidance on how to set up live streaming of face tracking data with live link, please see this article or watch this tutorial:

Rokoko Studio settings

Enabling Rokoko Studio Live

Go to settings and click on Studio Live in the dropdown menu


Configure the data Stream

Before you can begin streaming data you need to configure the receiver information. Start by clicking the cogwheel  in the top right corner, which will flip the box around.

Add receiver

Add one receiver by clicking the plus button, which applies one receiver to the list. At this stage you can modify the IP address and Port number.

 

Click the arrow down  button to configure VP (Virtual Production) Port.

Enable and disable receiver

Each individual receiver can be enabled and disabled by toggling the switch  

 

Enable data stream

Enable the data stream by clicking on box. The green outline indicates that the data stream is enabled, click again to disable. (In order to enable a data stream, at least one receiver must be enabled.

 

Unreal Engine settings

If you haven't installed the plugin or set up a scene before, please follow this article!
Installing the Rokoko Live plugin in Unreal Engine

Setting Up Scene

  • Use the search bar under the ‘Modes’ tab on the left and type “Smartsuit Controller”

    • Click and drag the “Smartsuit Controller” inside of the viewport

  • Use the search bar again and type “Smartsuit Receiver”

    • Click and drag the "Smartsuit Receiver" inside of the viewport.

    • Click and drag the "Smartsuit Controller" inside of the viewport.

  • Change "Streaming Data Port" and "VPListen Port" to the corresponding values used in Rokoko Studio.

    • "Streaming Data Port" Default is "14043" In Rokoko Studio

    • "VPListen Port" Default is "14045" In Rokoko Studio

  • Click the “SmartsuitController” object inside of the World Outliner.

    • Within the Details tab, make sure that ‘Auto Connect’ is checked.

      • If this is not checked, you will then have to input the suit ID inside of the ‘Suitname’ field to get it to connect.

      • You find the smartsuit ID inside of Rokoko Studio in the top right corner.

Setting Up Character

You will need to import your character:

  • Right-Click within the Content Browser and select “Import to /Game…”

  • Find the FBX of your character and select the Open button.

  • If your character isn’t using an existing skeleton, leave Skeleton set to ‘None’.

  • Click “Import”.

  • Once your character is imported, right-click on the skeletal mesh of your character, highlight ‘Create’ and click ‘Anim Blueprint’

  • Double-click on the newly created animation blueprint

  • Create the following set of nodes:

    • Right-click and type “Get Smartsuit Controller” and select that node

    • Left-click and drag from ‘Return Value’, type “Smartsuit Body Pose” and select that node

    • Left-click and drag from ‘Animation Pose (Component Space)’, type “Component to Local” and select that node

    • Left-click and drag from ‘Animation Pose’ and plug it into ‘Result’ on the ‘Output Pose’ node

  • Left-click the ‘Smartsuit Body Pose’ node and expand ‘Bone Map’

    • If you are using a Mixamo rigged character, match the following character bones to the bone map:

      Rokoko Bone Map
      Custom Character/Mixamo Character
      Hip
      Hips
      Stomach
      Spine
      Chest
      Spine2
      Neck
      Neck
      Head
      Head
      Head Top
      HeadTop_End
      Left Shoulder
      LeftShoulder
      Left Arm
      LeftArm
      Left Forearm
      LeftForearm
      Left Hand
      LeftHand
      Left Finger Tip
      LeftHandMiddle4
      Right Shoulder
      RightShoulder
      Right Arm
      RightArm
      Right Forearm
      RightForearm
      Right Hand
      RightHand
      Right Finger Tip
      RightHandMiddle4
      Left Upleg
      LeftUpLeg
      Left Leg
      LeftLeg
      Left Foot
      LeftFoot
      Left Toe
      LeftToe_End
      Right Upleg
      RightUpLeg
      Right Leg
      RightLeg
      Right Foot
      RightFoot
      Right Toe
      RightTo_End


  • ‘Relative to Start’ checkbox:

    • When this is True (checked), the animation will keep the starting position on the XY-plane of the character when the game starts

    • When this is False (unchecked), the animation will jump to the absolute position of your character inside of Rokoko Studio when the game starts

  • ‘Scale Bones’ checkbox:

    • When this is True (checked), this will scale the bones to match the model’s body dimensions. This can make the animation look more natural on the character, but may cause unusual effects on the mesh

      • This is usually left set to False, unless you have a character specifically created to take advantage of this feature

  • Once you’re done assigning bones, click the ‘Compile’ button at the top left

    • A warning may appear, however, this will not affect performance so you can ignore it

  • Close the Animation Blueprint window

  • Drag and drop the character’s animation blueprint from the Content Browser into the scene

    • It is important to note that the XYZ axis rotations for your character MUST be set to 0. Otherwise, this could cause some issues with the bones being rotated improperly when the game starts.

  • Press the Play button and watch as your character moves along with the Smartsuit Pro’s movement



Did you find it helpful? Yes No

Send feedback
Sorry we couldn't be helpful. Help us improve this article with your feedback.