Create ticket
Welcome
Login  Sign up
The Cinema 4D Plugin for Studio has landed!

Say hello to our new Cinema 4D plugin. Get the full scoop here

Getting Started: Streaming Face Capture to Unreal Engine

Requirements

Rokoko Studio Subscription: Rokoko Studio Plus, Pro or Enterprise

Add-Ons: Face Tracking

Required Hardware:

iPhone X-Series or 11-series with TrueDepth Camera.

Rokoko Phone Mount (Optional)

Required Software (App): Rokoko Remote for IOS, download here.

In order to stream to Unreal Engine, you need to have installed the Unreal Engine plugin. Instructions for doing this can be found here


Rokoko Studio settings

Connecting Rokoko Remote to Rokoko Studio

For connecting Rokok Remote to Rokoko Studio, please follow this article:
Getting Started Guide: Face Capture
Enabling Rokoko Studio Live

Go to settings and click on Studio Live in the dropdown menu


Turn on streaming to Unreal Engine by enabling the plugin and enabling the default IP and port.

Unreal Engine settings

If you haven't installed the plugin or set up a scene before, please follow this article!
Installing the Rokoko Live plugin in Unreal Engine

Setting Up Scene

  • Use the search bar again and type “Rokoko Receiver”

    • Click and drag the "Rokoko Receiver" inside of the viewport.

    • Configure the Rokoko Receiver Port to match the streaming port in Rokoko Studio (14043)

Setting Up Character

You will need to import your character:

  • Right-Click within the Content Browser and select “Import to /Game…”

  • Find the FBX of your character and select the Open button.

  • If your character isn’t using an existing skeleton, leave Skeleton set to ‘None’.

  • Remember to turn on "Morph Targets", else the character won't be imported with blendshapes.

  • Click “Import”.

  • Once the character is imported, right-click the in the content browser.

  • Right-click on the skeletal mesh of your character, highlight ‘Create’ and click ‘Anim Blueprint’

    • Double-click on the newly created animation blueprint

    • Create the following set of nodes:

      • Inside the anim blueprint create a"Rokoko Face Pose" node and connect it to the Output Pose.

        Now click on the plus next to variables and change the variable type to Name and the variable name to Rokoko Actor Name. Then drag it out into the animGraph and chose Get Rokoko Actor Name. Connect it to Rokoko Body Pose and the Rokoko Face Pose. Then click on the Rokoko Actor Name and type in the Actor Profile Name from Studio. Hit apply and compile the blueprint.

         

  • Close the Animation Blueprint window

  • Drag and drop the character’s animation blueprint from the Content Browser into the scene

  • Now we need to turn on Unreal Engines native live link plugin! Go to Window->Live Link->+Source->Rokoko Studio Source and select "Studio"

  • Press the Play button and watch as your character animate with the streamed data!


Custom blendshape naming

If you have different naming than the standard ARKIT naming please follow the steps below:

  • Select "Blueprint Class", in "All Classes" search for "RokokoFaceRemapAsset" and select it.

  • Double-click the "FaceRemap" asset and you will be greeted with a list of

    Blendshapes/Morph-Targets:

    • The setup is based on Apples ARKits naming convention. If your character follow that naming convention you don't have to change the shape names.

    • If you're using your own or different naming convention. You will have to go through the list and map the shape name to the corresponding morph-target on your rig.

    • You can open the "Skeleton" for your character to get the list of morph-targets available.

    • For more information and reference to Apples ARKit Documentation, please follow here.

    • The double-click your animation blueprint

    • Click on "Rokoko Face Pose" and in the retarget section select the name of your "FaceRemap" asset.

Did you find it helpful? Yes No

Send feedback
Sorry we couldn't be helpful. Help us improve this article with your feedback.