Create ticket
Welcome
Login  Sign up

Getting Started: Streaming Face Capture to Unreal Engine

Requirements

Rokoko Studio Subscription: Rokoko Studio Plus, Pro or Enterprise

Add-Ons: Face Tracking

Required Hardware:

iPhone X-Series or 11-series with TrueDepth Camera.

Rokoko Phone Mount (Optional)

Required Software (App): Rokoko Remote for IOS, download here.

In order to stream to Unreal Engine, you need to have installed the Unreal Engine plugin. Instructions for doing this can be found here

Video Tutorial: Live Streaming Face Capture to Unreal Engine 4.24 and above




Rokoko Studio settings

Connecting Rokoko Remote to Rokoko Studio

For connecting Rokok Remote to Rokoko Studio, please follow this article:
Getting Started Guide: Face Capture
Enabling Rokoko Studio Live

Go to settings and click on Studio Live in the dropdown menu


Configure the data Stream

Before you can begin streaming data you need to configure the receiver information. Start by clicking the cogwheel  in the top right corner, which will flip the box around.

Add receiver

Add one receiver by clicking the plus button, which applies one receiver to the list. At this stage you can modify the IP address and Port number.

 

Click the arrow down  button to configure VP (Virtual Production) Port.

Enable and disable receiver

Each individual receiver can be enabled and disabled by toggling the switch  

 

Enable data stream

Enable the data stream by clicking on box. The green outline indicates that the data stream is enabled, click again to disable. (In order to enable a data stream, at least one receiver must be enabled.

 

Unreal Engine settings

If you haven't installed the plugin or set up a scene before, please follow this article!
Installing the Rokoko Live plugin in Unreal Engine

Setting Up Scene

  • Use the search bar under the ‘Modes’ tab on the left and type “Smartsuit”

    • Click and drag the "Smartsuit Receiver" inside of the viewport.

  • Change "Streaming Data Port" and "VPListen Port" to the corresponding values used in Rokoko Studio.

    • "Streaming Data Port" Default is "14043" In Rokoko Studio

    • "VPListen Port" Default is "14045" In Rokoko Studio

Setting Up Character

You will need to import your character:

  • Right-Click within the Content Browser and select “Import to /Game…”

  • Find the FBX of your character and select the Open button.

  • If your character isn’t using an existing skeleton, leave Skeleton set to ‘None’.

  • Remember to turn on "Morph Targets", else the character won't be imported with blendshapes.

  • Click “Import”.

  • Once the character is imported, right-click the in the content browser.

  • Select "Blueprint Class", in "All Classes" search for "RokokoFaceRemapAsset" and select it.

  • Double-click the "FaceRemap" asset and you will be greeted with a list of

    Blendshapes/Morph-Targets:

    • The setup is based on Apples ARKits naming convention. If your character follow that naming convention you don't have to change the shape names.

    • If you're using your own or different naming convention. You will have to go through the list and map the shape name to the corresponding morph-target on your rig.

    • You can open the "Skeleton" for your character to get the list of morph-targets available.

    • For more information and reference to Apples ARKit Documentation, please follow here.

  • Once they are all mapped, right-click on the skeletal mesh of your character, highlight ‘Create’ and click ‘Anim Blueprint’

  • Double-click on the newly created animation blueprint

  • Create the following set of nodes:

    • Right-click and type “Make LiveLinkSubjectName”, change the "name" to the Face ID found in Rokoko Studio and select that node.

    • Left-click and drag from ‘Live Link Subject Name’, type “Live LinkPose” and select that node.

    • In the 'Live LinkPose' node in "Details" change the "Retarget Asset" in the drop-down menu to the "FaceRemap" asset we created.

    • Connect the "Animation Pose" from 'Live LinkPose' to "Result" of the 'Output Pose' Node.

    • Once you're connected these nodes, click the ‘Compile’ button at the top left

    • If you already have body mocap set up for the character. The setup should look like this.

    • Follow this article about the Body setup:
      Getting Started: Streaming to Unreal Engine

  • Close the Animation Blueprint window

  • Drag and drop the character’s animation blueprint from the Content Browser into the scene

  • Now we need to turn on Unreal Engines native live link plugin! Go to Window->Live Link->+Source->Rokoko Studio Source and select "Studio"

  • Press the Play button and watch as your character animate with the streamed data!


Did you find it helpful? Yes No

Send feedback
Sorry we couldn't be helpful. Help us improve this article with your feedback.