Release 8 - new features, cleanup and documentation

What's new in Release 8

• The default is ONE camera. If you need TWO cameras read this to see how to get the old setup back.
• The default is VIVE trackers using LiveLink. If you need to get in a different kind of tracking read this document.
• The telemetry viewer for debugging tracker problems has it's own document, read it here
• There is a new tool for measuring your studio read the document here
• ActorTransformTelemetry replaces ALL the other telemetry sending blueprints. See this document for datails on how to use it.
• ActorTransformTelemetry runs "post physics tick" so all actors are updated before it sends their positions.
• LiveLinkTracker supports trackers that send relative data. Select tracker type "relative tracker" and supply "Relative Tracker Offset" to align tracking to the coordinate system of the Talent Mark.
• Autorig no longer displays a vive puck at the top. This is unnessary because the LiveLinkTracker draws the tracker mesh.
• Autorig sizes it's rods to the exact rig measurements you enter so they don't stick out.
• Autorig's rods are now colored to match unreal conventions for x, y, z
• Added ThinAxis actor, makes it easier to see the orign of smaller objects, just attach it.
• Vive controller buttons work when using livelink. They still need to be mapped in the Vive control panel and the Unreal 3d window has to have focus.
• Added RecordMeasurements a way to measure your studio using the VIVE controllers/trackers
• Added MeasurementMarker actor, displays an axis marker and current position, used by RecordMeasurements
• The default map is now SINGLE CAMERA.
• Flying logo is now self contained in one actor. Epic fixed a bug that prevented this in earlier versions
• Flying logo can be started by pressing 9
• Added categories for some actor variables to organize them, also added some tooltips for them and set limits on variables that needed them.

Bug Fixes

• Fixed desk, floor and wall matte actors so the size parameters work. Note there is nothing special about these, they are just simple shapes with some blueprint code to make resizing them easier. They must be added to Window->Layers->GarbageMatte to be recognized as part of the matte.
• Fixed measuring grid so the display/don't display menu works in editor as well as play mode

Changes you won't notice unless you have customized my blueprints

These are mostly changes to make the whole setup work better, simplify the internals and make everything more consistant.

• Renamed media player assets that go with the Flying Screen actor
• Renamed all the parts of the AJA media bundles in a consistant way.
• Renamed Autorig "entrance pupil" transform to "output" for consistancy with other rigs.
• Removed "Telemetry Order" in all tracker actors, no longer needed.
• MotionControllerTracker, DelayedOutput has been renamed to "Output" for consistancy with LiveLinkTracker
• All tracker actors use a more efficient way of setting the correct mesh for the tracker.

Release 7 - Unreal 4.26 and LiveLink support bug fix

Bug Fix Release

• The LiveLinkMeasuringGadget was not working, returning measurements of zero. Should be fixed now.

• The AxisGuideWorld and PlumbLine actors were only working after pressing play. Now they will work when using LiveLink in the editor.

For More Information

Take a look at VPStudio Release 7 on my YouTube channel

Release 6 - Unreal 4.26 and LiveLink support

Unreal 4.26 support with Live Link tracking

• If you are NOT ready to update to Unreal 4.26 yet don't download this, it won't work with earlier Unreal versions.

• Some internal changes to make sure everything works on Unreal 4.26

• Added support for trackers using OpenXR and LiveLink.

• The "LiveLinkTracker" actor works with anything that sends tracking over LiveLink, this should include any device that works with Steam as well as other LiveLink trackers like Optitrack. Right now it has settings for vive pucks and controllers, support for other LiveLink devices can be easily added to the blueprint.

• Live Link Trackers and everything connected to them (rigs, cameras) update in the editor. You can see everything moving without having to press play or use the inspection camera. All your composure passes display correct previews so it is easier to adjust camera rigs, mattes and other elements of your virtual set. There is no need to use the inspection camera or go back and forth between edit and play.

• Live Link lets you set the tracking delay by fractions of a second instead of per frame. One setting covers all your trackers.

• The older "motion controller" tracker actors are still available if you don't want to use LiveLink. All of your steam trackers must use one or the other, not both.

• VPDemoMap is now setup to use LiveLink. If you want to use the old motion controller setup, use VPMotionControllerMap and turn off the LiveLinkXR plugin.

• I have had some problems with steam bindings when using 4.26, this release worked for me but you may have to re-make your steam bindings when you update. I would recommend using LiveLink, you don't need to setup any steam bindings for it and it's nicer.

• There is a different "Measuring Gadget" and "Multiple Tracker Telemetry" actor for use with LiveLink. Both still require you to press play to use them.

For More Information

Take a look at tutorial 11 on my YouTube channel

Release 5

Last Unreal 4.25 release!

Release 5 is for Unreal 4.25, unless bugs are detected this will be the last release before moving to Unreal 4.26 which has a lot of problem solving Virtual Production Features.

This release has a lot of reorganizing of folders, renaming actors and deleting old obsolete things. It should be much cleaner and easier to use. It also features several systems for debugging tracking problems.

TCP Socket Plugin

When you load the project it will ask you to install "TCP Socket Plugin", free on Unreal Marketplace from SpartanTools. If you don't install it, everything but telemetry will still work.

This Release Includes

• Added a VPStudioForeground pass to composure for both cameras and updated the compositing materials to handle them. Any actors rendered in VPStudioForeground will appear in front of your talent. To put an actor in the foreground pass, select it in the world outliner then go to Window->Layers, click ForegroundObjects then right click and choose "add selected objects to selected layers".

• To improve your frame rate, be sure to turn off any composure passes you are NOT using. If you only have one camera, turn off all the passes for camera 2, if you are not using the foreground pass turn it off. If you have a large green screen and don't need the garbage matte, turn that off also. Every pass costs a some rendering time even if there is very little in it, so this helps. Just go to the Window->Composure Compositing and click the eyeball next to the passes to turn them on and off.

• To reduce jittering, make sure you are running at a fixed frame rate and always hitting it. Either set the project for a fixed frame rate (the same as the camera) or setup the project to synchronize to the camera frame rate if your device supports it.

• ViveTracker actor was renamed "Tracker". Now it works with Vive Tracker Pucks and Vive .Hand Controllers, displaying the right 3d model with tracking delay applied. There are some new settings in the tracker details, described below.

• ViveTracker's "Tracker Type" setting controls which tracker model is displayed. It also adjusts the transform to have what most people consider the right "up" and "front" orientation for the type of tracker. You can update this to support more types of trackers or different transform adjustments.

• If ViveTracker's "Tracker is Lockable" setting is checked the tracker can be turned off or "locked down" at any time by pressing a key. If you are shooting with stationary cameras you can have everything unlocked while you setup your camera positions, then lock them to prevent any tracking jitter. Don't check this for trackers attached to things you want to move during the shoot, like a hand-held camera or other object.

• If the ViveTracker's "Draw Tracker" box is checked, the correct model for the tracker will be displayed. This can be a big help seeing if your camera delay and camera alignment is correct. The 3d model of the tracker should exactly cover the image of a tracker you hold in your hand and not be ahead of or behind when you move the tracker in the real world.

• ViveTracker's "Telemetry Order" is used for sending your tracking info to a program outside of Unreal for monitoring.

• The "MotionStats" actor is for testing trackers. Add a MotionStats actor to the level and use the "attach to" function to attach it to a tracker. Make sure the tracker is stationary, then press "play". MotionStats will record movement for a bit and then print out the average, minimum, maximum, spread and standard deviation values for X, Y, Z, Roll, Pitch and Yaw of the actor, these will appear on the screen and in Window->Developer Tools->Output Log. Standard deviation is a measure of how much "jitter" the tracker has, lower is better. If you want to test multiple trackers all at once, just create and attach a MotionStats actor to each one of them. You can set the number of frames to record (1000 is the default, 15-40 seconds depending on frame rate). MotionStats will work when attached to any actor but is mostly useful for testing trackers.

• There a system for sending tracker and other information (Telemetry) to another program that graphs it in real time. This is useful for detecting and troubleshooting problems with tracking. See the VPStudio/TelemetryViewer folder for more detailed documentation on this and the SingleTlemetrySender and MultipleTrackerTelemetry actors.

• The SingleTelemetrySender actor will send real-time X,Y,Z,Roll,Pitch,Yaw data to TelemetryViewer. Add to your level and drop or "attach" it to the actor you want telemetry from. Only create ONE of these or your telemetry may be corrupted. See the SingleTelemetrySenderExample level for a sample.

• The MultipleTrackerTelemetry actor will find all your Tracker actors and send their telemtry to TelemetryViewer at once. Only create one of these. See the MultipleTrackerTelemetryExample level for a sample.

• The ExperimentalTelemetryTracker is not useful for you, it's for testing some ideas on filtering tracker data.

• The mattes have also changed. Instead of the "flat green screen" there are now separate mattes for a greenscreen on the wall, one on the floor and one for a desktop. The origin of each matte is now on the left hand corner. This makes lining them up with the world a little easier, just get the edge of the matte in the right place and adjust the size untill it's right. Once you have the ones you want in your level, just add them to the "Garbage Matte Layer" in the Layers window and they will be used.

• The "MeasuringGrid" actor draws a grid of squares to help you align cameras with a real world object like a checkerboard or cutting mat with a grid on it. In the details set the size of the squares in cm and the number of squares horizontal and vertical. To use just drop into the level. You can attach it to a VivePuck actor and the grid will go wherever the Vive Puck goes. The "Grid Adjustment" node inside the actor lets you rotate or shift the grid. There are some checkboxes to turn drawing of the grid and the axis guide on and off.

• The "AutoRig" actor is a generic camera rig you can quickly setup to work with most camera rigs. The "measured" settings should be set to the measurements of your rig, then the "adjust" elements can be used to make fine adjustments if you find the measurements to be slightly off. This also works with "MeasuringGadget" to automatically measure your rig.

• The "MeasuringGadget" actor uses a second vive tracker to measure your camera rig and apply it's settings to an AutoRig. This lets you setup your rig very quickly. I will put a link to a video tutorial here when it's ready.

• The "LaserPointer" actor projects a laser beam forward till it hits something. To use, just attach it to a tracker actor. The blueprint for this actor finds a lot of information about the actor it hits. You could add to the blueprint functions to select, change the target actor or turn the laser beam on and off. You can set the maximum length of the beam in details.

For more information

See the Readme file in the VPStudio directory and tutorials 7 through 10 on my youtube channel.

Release 4

This release contains the blueprints for the flying logo and flying TV screens seen in my ESports demo set. This shows how I am using an OSC control surface on my phone to control these features

If you are using my VPStudio project for something important, I suggest you wait till I have a complete release 3 for you along with tutorials before you upgrade. Please backup your old VPStudio project before updating to this one, just in case there are bugs.

This release includes

• An experimental blueprint (measureme) for measuring your camera rig using a second vive tracker
• Ability to control some features using an OpenSoundControl control surface like TouchOSC
• The blueprint for the flying screens you saw in the ESports Set demo
• The blueprint for the flying logo you saw in the ESports Set demo
• The TouchOSC setup file I used for this
• A change to the settings of the garbage matte scene capture component in composure which should make it faster

Please note this does not include any of the fancy sets you have seen in my YouTube demos, these all come from the Epic Unreal Marketplace and the license does not allow me to distribute them here. Some of these items are in the "permanently free" collection on the Marketplace so you can get them at no charge.

Release Candidate 3

This is a release candidate and is not fully tested

I'm putting this out as a release candidate because it's been so hot here and the air quality so bad that I haven't been able to film tutorials on this or do my usual amount of quality testing. Some people have been wanting to see how I did OSC support and some of the effects from the "ESports Set" on my youtube channel, so this release is for them and other adventurous folks.

If you are using my VPStudio project for something important, I suggest you wait till I have a complete release 3 for you along with tutorials before you upgrade. Please backup your old VPStudio project before updating to this one, just in case there are bugs.

Please report any problems you have with this release candidate so I can fix them.

This release includes

• An experimental blueprint (measureme) for measuring your camera rig using a second vive tracker
• Ability to control some features using an OpenSoundControl control surface like TouchOSC
• The blueprint for the flying screens you saw in the ESports Set demo
• The blueprint for the flying logo you saw in the ESports Set demo
• The TouchOSC setup file I used for this
• A change to the settings of the garbage matte scene capture component in composure which should make it faster

Please note this does not include any of the fancy sets you have seen in my YouTube demos, these all come from the Epic Unreal Marketplace and the license does not allow me to distribute them here. Some of these items are in the "permanently free" collection on the Marketplace so you can get them at no charge.

Release 2 minor fixes

This release contains fixes to a few minor bugs I discovered when doing the "Use Your Own Sets" tutorial

• Set all the chroma key, despil and erode passes to be on by default

• The I key was mapped to two functions (inspection camera and hide trackers) remapped so hide trackers is now K key.

• The two AJA media sources should have been part of the core map, moved them there.

Release 1 major improvements

I'm starting to number releases properly, starting with 1 and going up by 1 with each new release. Should make it easier to keep things straight. In this release VPStudio has a number of usability and feature improvements and was used to film Tutorial 4 and the mad scientist lab demo on my youtube channel. I'm hoping you will find this release easier to setup and use, I will be posting a few more tutorials on how to use it with your own content soon.

• Be sure you are on Unreal 4.25.2 or higher earlier versions of 4.25 had a bug that could cause a crash when switching levels.

• Folder structure has changed, everything is now in a top level folder called VPStudioCore. This is so when you migrate your own content into this project it will be kept separate from all the VPStudio stuff.

• The core virtual production stuff is by itself in a sublevel now. You can import your own level into the project, then just add the virtual production sublevel to it (see the levels window) and you're ready to go.

• Talent marks now have a "sort order" variable you can set. VPStudio will sort your talent marks based on this variable. It's a float to make it easy to insert a talent mark between existing ones (ie: if you have mark 1 and mark 2 you can insert between them by setting the number on the new mark to 1.5) The name of the talent mark can be anything you want (ie: BedroomMark, KitchenMark...etc)

• All your studio elements (trackers, cameras, rigs...etc) should be attached to one talent mark. If you have accidentally connected some to different marks, they will now be found and moved to the first one when you press play.

• Added some more adjustments to my camera rigs because the real world rigs turned out to be slightly misaligned on a couple other axies.

• Fixed a problem with talent mark man, made it inherit from talent mark.

• Added a two meter height scale, marks on the right side are at 5'8" and 6'

• I added a few more props to the demo level to make it slightly more interesting. Just a big studio light and a camera on a tripod.

• Changed the color of the plumbline graphic to red for better contrast when trying to line up with real world objects.

• The right hand vive controller can now be used to control some of the demo functions. Clicking left or right trackpad will select camera 1 or 2. Pulling the trigger moves to the next talent mark. Clicking down on the trackpad or pressing B on the keyboard will take s screenshot (this is useful for checking the key, focus...etc if you are working by yourself).

• Added materials on things like the floor and the spheres/cones/cubes in the scene. This makes it easier to see all the objects. The green screen object is now green.

• The talent mark texture now has an arrow showing which way the talent should be facing.

• The H key will bring a HUD with some alignment guides that go through the center of the screen and one quarter the way to either side of it. These are useful when trying to align cameras/trackers if your camera doesn't have a screen or the screen is hard to see.

Overdue Release

This is the release was used to do the science lab demo on my my youtube channel and has the latest set of features. All the changes are in the VPStudio directory, nothing else has changed. This is the last release with the current folder structure, the next one will have all the folders moved and the levels rebuilt to make them easier to add to your own Unreal projects.

• The right hand vive controller can now be used to control some of the demo functions. Clicking left or right trackpad will select camera 1 or 2. Pulling the trigger moves to the next talent mark. Clicking down on the trackpad or pressing B on the keyboard will take s screenshot (this is useful for checking the key, focus...etc if you are working by yourself).

• I've put materials on the floor, spheres, cones and cubes in the scene. This makes it easier to see all the objects. The green screen object is now green.

• The talent mark texture now has an arrow pointing the way the talent should be facing.

This release also all the other VPStudio revisions from the last couple of months including.

6/28/2020

Major refactoring of the tracker and camera rig system to remove unnecessary rotations and transforms. This was because vive trackers are setup so that their "front" or X axis when they are flat on the table is pointing up. This caused any mesh or other object connected to them to be pointing the wrong way. The tracker objects in VPStudio have had a 270 degree Y axis rotation added to their output to fix this. Now when the tracker is setting flat on a table, up is the Z axis same in unreal. The "front" positive X axis points in the direction opposite the USB plug. If you used one of my earlier examples to build your own camera rig, you probably have this 270 degree rotation as the first node in your rig, this is no longer needed, you can set the rotations to zeros.

Added measuring and alignment tools. Actors that can be attached to a tracker or placed in the world for reference. Attach AxisGuide to anything to see it's local coordinates. AxisGuideWorld will always show the world x/y/z no matter what it is attached to or how it is rotated. There are two 1m objects you can use as references when aligning the camera with the virtual world. These are examples, you should find a real world box and a square of material some fixed size and make your own versions of these references in that size. Then you can compare the real world object and CG objects in a composite to see if they line up. Both of my objects have the origin at top-center so if you attach them to a tracker object in unreal and set a Vive tracker on the real world object the CG reference object will appear in the same place. There is also an actor called PlumbLine which will always draw a vertical line no matter what it is attached to. If you hang a real plumb line from a vive tracker, the two vertical lines should match. An axis guide attached to CameraRig will show the entrance pupil component (where the camera would be attached). If you attach one to the tracker it will show the output of the tracker with any delay included.

The camera models have been adjusted so their origin is at the entrance pupil of the lens. The A7R4 has a 24-70 F2.8 G-Master zoom attached with the entrance pupil set for it set to 24mm. The UNCS3CA is setup for a 16-35 F2.8 G-Master with the zoom set to 16mm. Note that camera models are just for reference when building camera rigs and setting up the studio. They don't have anything to do with the alignment of the composite cameras so their measurements don't have to be exact.

There are 3 pre-built camera rigs, these represent the hardware used to attach your tracker to your camera and must have correct measurements. One end of the rig is the point where the tracker attaches, the other is where the entrance pupil of your lens is. You can make your own rigs however you like. I have found it is helpful to put joints wherever your real-world rig can rotate or twist (like at a tripod screw or ballhead) and set them up as I have so you can make small adjustments easily. If the real-world rig is out of alignment by as little as a degree it will be noticable, these adjustments let you correct that without having to fight to get the real-world rig parts perfectly aligned. My rigs include the mesh of the vive tracker, this is just to make it more clear how the rigs are built, when you build your own you can leave the tracker mesh out.

Added a two-sided white material with a red X going through the center. These are useful when making alignment objects. It's also used on the greenscreen model to make it visible from both sides.

Added meshes for a Sony A7R4 camera and a VIVE tracker, these are not precise models but are fairly accurate. Both models didn't come with the origin where it should be. To have them appear in the right places, the Vive model needs to have it's origin at the tripod screw and the A7R4 needs it at the entrance pupil location of the lens.

Added TalentMarkMan with the unreal mannequin standing on it as a size reference. This man is 193cm tall or around 6'3". I suggest you rescale this to match your your talent's height so it will be a good size reference when putting the talent mark in a level.

Added VivePuckNoRotate which outputs a tracker position without any rotation.

The "entrance pupil" map was used in filming of the tutorials but at the moment is probably not useful for anyone.

There is a "laser pointer" actor. It contains a motion controller you can hook to any tracking device, by default it is hooked to the right vive controller. This actor will have a red laser coming from the tracker and stopping on the first object it hits. Not really useful yet, but it is fun and can be used to point out objects in a virtual set if you are doing a tutorial. Inside the blueprint it returns the actor the beam hit.

Added keys to toggle visibility of TalentMark, MeasuringGuide, CameraModel, CameraRig and Tracker actors. Sometimes these are visible when filming so you want to turn them on and off easily. Current default is ON.

6/14/2020

Completely revised the tracking and camera system. The system is similar to the built-in unreal "camera rig crane" but is implemented in blueprints instead of C++ code. It is broken into several actors. There is a tracker which handles the motion controllers (and any delay), one or more "rigs" that represent the structure attaching the tracker to the camera, the VPCamera which is the CineCameraActor and another actor with a model of the camera. This change allows for multiple types of "tracker" actor such as Optitrack or a mechanical tracker. It also allows multiple rigs to attach the tracker to the camera such as a hot-shoe mount or a ball head. The rigs allow for the tracker to be setup in most any position and can have degrees of freedom which can be adjusted to account for something such as a ball head that is not perfectly aligned. The default has the tracker, rig and camera models all visible but they can be easily turned off or set to be "hidden in game".

Do not touch any transform nodes in a rig while it is running, this will detach the rig from the tracker and stop it from working. Each rig is made mostly of solid parts, any place in the rig that can move will have a variable in details for adjusting it to correct for small alignment errors.

Both maps were updated to use the new camera components. The virtual camera map is using the "sample" camera rig right now, if you are using this map you will want to make a rig that matches your camera.

The rig system could also be used to represent a fixed tripod with tracked joints, a crane/jib or something like a robot with a camera on the end of the arm.

For Migrate Tutorial

This release has some minor internal changes to make it work correctly with the migration tutorial.