Tutorial Composer

Create Project

Open the unreal engine launcher and create a new C++ Project. You may use any project template type if you are daring. However, we will use the Third Person project template for the tutorial, so select that project type. Once the project is completed, follow the installing Reasonable Planning AI instructions.

Create the Game Assets

Follow the instructions to Create Tutorial Game Assets.

Design the AI

Before starting with RPAI it is wise to design the AI first. The reason we design up front is to determine what our state properties ought to be.

For this tutorial the archetype we will design is an AI that exercises and rests. This is a rather basic set of goals a human can have and is a good starting point for learning RPAI. Exercise is broad enough of a goal with some data we can use to determine enough, too much, or unnecessary exercise. Resting is mostly the inverse but may overlap and enables us to have a fuzzy opposite goal.

From these two goals we can define a series of actions that can accomplish them but not necessarily linked to the goals. For instance, walking is an action that can be used to accomplish both goals - walking to exercise and walking to a chair to rest. From here we have a simple state we can use that supports the two goals and the following actions: move, idle, jump

---
title: State Properties
---
classDiagram
   State --o APainCausingVolume : `GetActorLocation() -> NearestWorkout`
   State --o APainCausingVolume : `GetActorLocation() -> NearestRest`
   State --o ATutorialAICharacter : `Energy -> GetCurrentEnergy()`
   State --o ATutorialAICharacter : `Strength -> GetCurrentStrength()`
   State --o ATutorialAICharacter : `GetActorLocation() -> TargetLocation`
   class State {
      +float Energy
      +float Strength
      +FVector CurrentLocation
      +FVector NearestWorkout
      +FVector NearestRest
   }
   class ATutorialAICharacter {
      +FVector GetActorLocation()
      +float GetCurrentEnergy()
      +float GetCurrentStrength()
   }
   class APainCausingVolume {
      +FVector GetActorLocation()
      +float DamagePerSec
   }

Loading

Define the State

Create a new Blueprint with a parent class of RpaiState_Reflection named TutorialAIState. This state is used to create snapshots of the game from the AI character's perspective. It is also used to create mutated snapshots for planning goals.

The state will implement most of the diagram we designed previously.

Add the following properties to the blueprint as seen below.

Now update the Blueprint Graph to create snapshots of the game world on demand.

RPAI Composer Behavior

The next step of the project is to create and configure the RPAI Composer Behavior data asset. This asset is the primary method of defining new AI in a data driven manner.

Create the Asset

Open Source Edition

Open the Content Browser. Once open, right click to open the context menu to create a new asset. Select Miscellaneous, then select Data Asset.

In the menu select RPAI Composer Behavior

Marketplace Edition

Open the content Browser. Once open, right click the content browser to open the context menu for a new asset. Select Reasonable Planning AI, in the expanded menu select RPAI Composer Behavior

TIP In the Data Asset picker, you can type in the search bar "RPAI" to find the asset quickly.

Name the new Data Asset Tutorial Behavior

Setup the Behavior

Configuring the behavior requires setting five properties. These properties are the state type, the reasoner, the planner, the goals, and the actions.

Configure the State Type

The state type is used as the data bridge between the game world and the AI's reasoner and planner. It is also used as a method of passing information to action tasks for execution.

For the ConstructedStateType property, click the dropdown and select TutorialAIState.

Configure the Reasoner

The reasoner will select from all of the configured goals and select the most valuable goal to pursue.

For the Reasoner property select Rpai Reasoner Dual Utility. By default there are two different Reasoners installed in the RPAI plugin. They are Rpai Reasoner Absolute and Rpai Reasoner Dual Utility. Both are viable options. The difference between each is subtle. The absolute reasoner will always select the highest scoring - most valuable - goal. The dual utility reasoner applies some extra logic by grouping by goal category and randomly selected a subset of the highest scoring goals.

Leave all settings of the reasoner as the default settings.

Configure the Planner

Given a goal and a list of the configured actions, the planner will output a set of actions to perform to accomplish the desired goal.

For the Planner select Rpai Planner AStar. By default there are two different Planners installed in the RPAI plugin. They are Rpai Planner AStar and Rpai Planner HUG. The former is a traditional Goal Oriented Action Planning type of planner. It will find a path to accomplish the given goal with given actions. The latter is a modified A-Star that uses an embedded reasoner to "short-circuit" a the final plan if enough of a goal divergence occurs.

Leave all settings of the planner as the default settings.

Checkpoint - Configuration

If you have followed the steps correctly, your setup should look similar to the image below.

Configure Goals

The goals are the abstract concepts of something the AI can accomplish.

For this tutorial we will configure two goals for the AI to choose between. These two goals will be Exercise or Rest.

The Exercise Goal

Add a new Goal to the Behavior in the Goals section. Set the class to Rpai Composer Goal

The Goals array supports any class sub typed from the Core library URpaiGoalBase class. Therefore you can incorporate custom C++ or Blueprint Goals if desired.

The logic behind determining how much effort is left to accomplish this goal is the distance to the nearest exercise location and a set value of strength.

Distance

The below configuration will determine the effort left to accomplish the given goal. Effort for the Exercise Goal is determined by the distance to an interaction and the difference between the minimum desired strength and current strength.

• For Distance Calculator select Rpai Distance Add All. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary.
  • Add a new Sub Distances
  • For the new distance, select Rpai Distance State. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary.
    • For Right Hand Side State Reference Key, expand details and set State Key Name to CurrentLocation and Expected Value Type to Vector
    • For Left Hand Side State Reference Key, expand details and set State Key Name to NearestWorkout and Expected Value Type to Vector
  • Add a new Sub Distances
  • For the new distance, select Rpai Distance Float. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary.
    • For Left Hand Side State Reference Key, expand details and set State Key Name to Strength and Expected Value Type to Float
    • For RHS, set the value to 80.

Weight

The weight determines how valuable it is to accomplish a goal. It is not tied to the distance but may include attributes used by the distance algorithm. For this tutorial the value of the Exercise goal is determined by the gap between strength and the maximum for strength. The higher the gap, the more the value to pursue. Perhaps this is because if it reaches 0 the AI will wither and die!

Think About This We just noted that the further from having high strength acts as a motivator for an AI to pursue a goal. Why is that? We gave the example that death would be imminent if strength hit 0. Perhaps in a more complex configuration the motivation would be less dire - such as feeling more powerful or safe due to self-improvement. It is important to think of the high level abstract reasons a goal exists to remove goal saturation. In a more sophisticated environment, this goal would not be Exercise but rather self improvement. Then our weights, distances, and other attributes would be more complex. However, this would open up dynamism in the actions executed.

• For Weight select Rpai Weight Distance. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary.
  • For Distance select Rpai Weight Distance Float
  • For the new distance, select Rpai Distance Float. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary.
    • For Left Hand Side State Reference Key, expand details and set State Key Name to Strength and Expected Value Type to Float
    • For RHS, set the value to 100.

Is Applicable

The Is Applicable property is used to determine the possibility of a Goal. If it is impossible for a goal to be pursued based on the current game conditions - then this configuration detects such scenarios.

In our game we will later define a rule that in order to use exercise equipment, the Energy of the character must be at least ten (10). Therefore it is not possible to pursue this goal if the agent has less than 10 energy.

• For Is Applicable set the value to State Query Compare To Float
  • For Queried State, expand details and set State Key Name to Energy and Expected Value Type to Float
  • For Comparison Operation, set the value to Greater Than or Equal To
  • For Value To Compare, set the value to 10

Is In Desired State Query

The Is In Desired State Query property is used by heuristics to determine if a goal is finished based on the projected permutations of a plan's actions. This configuration must be fuzzy and allow for ranges of values or else it is too precise and unreachable. Simplicity is key.

For the Exercise goal we will determine the goal is in a desired state when the Strength attribute is greater than or equal to the minimum value defined in the distance calculator: 80.

• For Is In Desired State Query set the value to State Query Compare To Float
  • For Queried State, expand details and set State Key Name to Strength and Expected Value Type to Float
  • For Comparison Operation, set the value to Greater Than or Equal To
  • For Value To Compare, set the value to 80

Final Step

Set the Goal Name to Exercise. This will make it easier to find in the list of goals when it is not expanded.

The Rest Goal

Add a new Goal to the Behavior in the Goals section. The logic behind determining how much effort is left to accomplish this goal is the distance to the nearest rest location and a set value of energy.

Distance

The below configuration will determine the effort left to accomplish the given goal. Effort for the Rest Goal is determined by the distance to an interaction and the difference between the minimum desired energy and current energy.

• For Distance Calculator select Rpai Distance Add All. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary.
  • Add a new Sub Distances
  • For the new distance, select Rpai Distance State. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary.
    • For Right Hand Side State Reference Key, expand details and set State Key Name to CurrentLocation and Expected Value Type to Vector
    • For Left Hand Side State Reference Key, expand details and set State Key Name to NearestRest and Expected Value Type to Vector
  • Add a new Sub Distances
  • For the new distance, select Rpai Distance Float. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary.
    • For Left Hand Side State Reference Key, expand details and set State Key Name to Energy and Expected Value Type to Float
    • For RHS, set the value to 80.

Weight

The weight determines how valuable it is to accomplish a goal. It is not tied to the distance but may include attributes used by the distance algorithm. For this tutorial the value of the Rest goal is determined by the gap between energy and the maximum for energy.

• For Weight select Rpai Weight Distance. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary.
  • For Distance select Rpai Weight Distance Float
  • For the new distance, select Rpai Distance Float. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary.
    • For Left Hand Side State Reference Key, expand details and set State Key Name to **Energy **and Expected Value Type to Float
    • For RHS, set the value to 100.

Is Applicable

This goal is always applicable, so we will use a configuration that will always evaluate to true. Set the Is Applicable Query dropdown to Rpai State Query Any. Leave all properties empty. This configuration results in an always true configuration.

Is In Desired State Query

The Is In Desired State Query property is used by heuristics to determine if a goal is finished based on the projected permutations of a plan's actions. This configuration must be fuzzy and allow for ranges of values or else it is too precise and unreachable. Simplicity is key.

For the Rest goal we will determine the goal is in a desired state when the Energy attribute is greater than or equal to the minimum value defined in the distance calculator: 80.

• For Is In Desired State Query set the value to Rpai State Query Compare To Float
  • For Queried State, expand details and set State Key Name to Energy and Expected Value Type to Float
  • For Comparison Operation, set the value to Greater Than or Equal To
  • For Value To Compare, set the value to 80

Final Step

Change the Goal Name to Rest. This will make it simpler to find in the editor when scanning goals.

Configure Actions

The actions are all of the granular things an AI can do in the game world.

For this tutorial we will define four actions: Move To Equipment, Move To Rest, Idle, Jump

Move To Equipment Action

The Move To Equipment Action is an example of a specific action. It will use specific state data to execution AI Navigation towards a specific point. It also demonstrates fuzzy mutation on the state for planning.

Add a new Action to the Behavior in the Actions array. Set the new value to Rpai Composer Action.

The Actions array supports any class sub typed from the Core library URpaiActionBase class. Therefore you can incorporate custom C++ or Blueprint Actions if desired.

Weight Algorithm

The weight algorithm determines the amount of effort - or cost - an action will incur. For the move action, the weight will be the distance between the current location and the target location.

• For Weight select Rpai Weight Distance. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary.
  • For Distance select Rpai Distance State
    • For Left Hand Side State Reference Key, expand details and set State Key Name to CurrentLocation and Expected Value Type to "Vector"
    • For Right Hand Side State Reference Key, expand details and set State Key Name to NearestWorkout and Expected Value Type to Vector

Apply To State Mutators

The mutators will project future state onto the state during planning. For this action, the fuzzy result would be the character is at the defined location.

Add a mutator to the Apply To State Mutators array.

• Set the class to Rpai State Mutator Copy State
  • For State Property To Mutate, expand details and set State Key Name to CurrentLocation and Expected Value Type to Vector
  • For State Property To Copy, expand details and set State Key Name to NearestWorkout and Expected Value Type to Vector

Is Applicable Query

The is applicable query serves two purposes. These purposes are to determine if an action is possible or relevant. For instance, if the character is already at the desired location, this action is no longer relevant.

• Set the class to Rpai State Query Compare Distance Float
  • Set the Comparison Operation to Greater Than or Equal To
  • Set the RHS value to 100
  • Set the Distance class to Rpai Distance State
    • For Left Hand Side State Reference Key, expand details and set State Key Name to CurrentLocation and Expected Value Type to Vector
    • For RHS, expand details and set State Key Name to NearestWorkout and Expected Value Type to Vector

Action Task

The action task is what will actually execute in the game world when this action is executed after planning. RPAI includes a lot of Action Tasks out of the box. For this tutorial we will use the Rpai Action Task Move To action task.

• Set the Action Task class to Rpai Action Task Move To
  • Set Acceptable Radius as 100
  • Set Reach Test Includes Agent Radius to false
  • Set Reach Test Includes Goal Radius to false
  • Set Allow Partial Path to false
  • For Action Task State Key Value Reference expand details if necessary.
    • Set State Key Name to NearestWorkout
    • Set Expected Value Type to Vector

By setting Allow Partial Path to false, the Action Task will cancel not only the current action, but all planned Actions remaining. So ensure this is the behavior you want for your AI before setting it in a real game.

Final Step

Set the Action Name to Walk To Workout to make it easier to find in the array of actions. This will also be visible in debug views.

Move To Rest Action

The Move To Rest Action is similar to the Move To Equipment Action. The only difference is the state key used. In a larger project it may be more beneficial to create a pre-configured action.

NOTE In Open Source RPAI Composer the only way to create a pre-configured Action - or any type - is by creating a subclass of it in C++ or Blueprints. It is possible to create a data asset in C++. With the Marketplace edition of RPAI one can create pre-configured Data Assets of the various RPAI Composer objects.

Add a new Action to the Behavior in the Actions array. Set the new value to Rpai Composer Action.

Weight Algorithm

The weight algorithm determines the amount of effort - or cost - an action will incur. For the move action, the weight will be the distance between the current location and the target location.

• For Weight select Rpai Weight Distance. Select the arrow to the left to expand details if necessary, also expand the Rpai details if necessary.
  • For Distance select Rpai Distance State
    • For Left Hand Side State Reference Key, expand details and set State Key Name to CurrentLocation and Expected Value Type to Vector
    • For RHS, expand details and set State Key Name to NearestRest and Expected Value Type to Vector

Apply To State Mutators

The mutators will project future state onto the state during planning. For this action, the fuzzy result would be the character is at the defined location.

Add a mutator to the Apply To State Mutators array.

• Set the class to Rpai State Mutator Copy State
  • For State Property To Mutate, expand details and set State Key Name to CurrentLocation and Expected Value Type to Vector
  • For State Property To Copy, expand details and set State Key Name to NearestRest and Expected Value Type to Vector

Is Applicable Query

The is applicable query serves two purposes. These purposes are to determine if an action is possible or relevant. For instance, if the character is already at the desired location, this action is no longer relevant.

• Set the class to Rpai State Query Compare Distance Float
  • Set the Comparison Operation to "Greater Than or Equal To**
  • Set the RHS value to 100
  • Set the Distance class to Rpai Distance State
    • For Left Hand Side State Reference Key, expand details and set State Key Name to CurrentLocation and Expected Value Type to Vector
    • For RHS, expand details and set State Key Name to NearestRest and Expected Value Type to Vector

Action Task

The action task is what will actually execute in the game world when this action is executed after planning. RPAI includes a lot of Action Tasks out of the box. For this tutorial we will use the Rpai Action Task Move To action task.

• Set the Action Task class to Rpai Action Task Move To
  • Set Acceptable Radius as 100
  • Set Reach Test Includes Agent Radius to false
  • Set Reach Test Includes Goal Radius to false
  • Set Allow Partial Path to false
  • For Action Task State Key Value Reference expand details if necessary.
    • Set State Key Name to NearestRest
    • Set Expected Value Type to Vector

Final Step

Set the Action Name to Walk to Rest

Exercise Action

The "Exercise Action" will be an action that is sometimes relevant, drains energy, and increases strength.

Add a new Action to the Behavior in the Actions array. Set the new value to Rpai Composer Action.

Weight Algorithm

The effort to execute this action is also dynamic. We will increase effort the further the Pawn is from an exercise location.

• Set the Weight Algorithm to Rpai Weight Select
  • For Default set the class value to Rpai Weight Constant Float
    • Set Constant Weight to 1.0
  • Add an element to Selections
    • For Selection Query set the class to Rpai State Query Compare Distance Float
      • Set Comparison Operation to Less than or Equal To
      • Set Distance to Rpai Distance State
        • For Left Hand Side State Reference Key, expand details and set State Key Name to CurrentLocation and Expected Value Type to Vector
        • For RHS, expand details and set State Key Name to NearestWorkout and Expected Value Type to Vector
      • Set RHS to 300
  • For Selection Weight set the class to Rpai Weight Constant Float
    • Set Constant Weight to 0.0

Apply To State Mutators

For this tutorial idling is a method of resting. Therefore, it will increase energy.

• Add an element to Apply to State Mutators
  • Set the class to Rpai State Mutator Add Float
    • Modify State Property To Mutate
      • Set State Key Name to Energy
      • Set Expected Value Type to Float
    • Set Value to Add to -20.0
• Add an element to Apply to State Mutators
  • Set the class to Rpai State Mutator Add Float
    • Modify State Property To Mutate
      • Set State Key Name to Strength
      • Set Expected Value Type to Float
    • Set Value to Add to 5.0

Is Applicable Query

Exercise is only applicable if within a workout location.

• For Is Applicable Query set the class to Rpai State Query Compare Distance Float
  • Set Comparison Operation to Less than or Equal To
  • Set Distance to Rpai Distance State
    • For Left Hand Side State Reference Key, expand details and set State Key Name to CurrentLocation and Expected Value Type to Vector
    • For RHS, expand details and set State Key Name to NearestWorkout and Expected Value Type to Vector
  • Set RHS to 300.0

Action Task

The Action Task is the code that gets executed during the game when the planned action is to be acted upon. Exercise could be exciting in a real game - such as playing an animation or interacting with a game object. For the purpose of the tutorial the Pawn will simply wait in the workout location.

• Set Action Task to Rpai Action Task Wait
  • Set Wait Time Seconds to 10
  • Set Random Deviation to 0

Final Step

Set the Action Name to Workout. This is the name that will appear in the debugger and editor.

Idle Action

The "Idle Action" will be an action that is always relevant and restores energy. It is less effort to idle in a rest location, but it is always applicable to idle and gain a benefit.

Add a new Action to the Behavior in the Actions array. Set the new value to Rpai Composer Action.

Weight Algorithm

The effort to execute this action is dynamic. We will increase effort the further the Pawn is from a rest location, but not so much so that it will always choose to walk to a rest location to idle. We will use a Select to accomplish this.

NOTE Order matters for the Selections array. It will evaluate "top-down". The logic of evaluation is similar to an else-if statement. Default is selected if none of the State Queries resolve successfully. That means Default is treated as the final else.

• Set the Weight Algorithm to Rpai Weight Select
  • For Default set the class value to Rpai Weight Constant Float
    • Set Constant Weight to 0.1
  • Add an element to Selections
    • For Selection Query set the class to Rpai State Query Compare Distance Float
      • Set Comparison Operation to Less than or Equal To
      • Set Distance to Rpai Distance State
        • For Left Hand Side State Reference Key, expand details and set State Key Name to CurrentLocation and Expected Value Type to Vector
        • For RHS, expand details and set State Key Name to NearestRest and Expected Value Type to Vector
      • Set RHS to 300
  • For Selection Weight set the class to Rpai Weight Constant Float
    • Set Constant Weight to 0.0
  • Add an element to Selections
    • For Selection Query set the class to Rpai State Query Compare Distance Float
      • Set Comparison Operation to Less than or Equal To
      • Set Distance to Rpai Distance State
        • For Left Hand Side State Reference Key, expand details and set State Key Name to CurrentLocation and Expected Value Type to Vector
        • For RHS, expand details and set State Key Name to NearestRest and Expected Value Type to Vector
      • Set RHS to 3000
  • For Selection Weight set the class to Rpai Weight Constant Float
    • Set Constant Weight to 0.5

Apply To State Mutators

For this tutorial idling is a method of resting. Therefore, it will increase energy.

• Add an element to Apply to State Mutators
  • Set the class to Rpai State Mutator Add Float
    • Modify State Property To Mutate
      • Set State Key Name to Energy
      • Set Expected Value Type to Float
    • Set Value to Add to 10

Is Applicable Query

Idling is always applicable and thus will always return true. Set the Is Applicable Query dropdown to Rpai State Query Every. Leave all properties empty. This configuration results in an "always true" configuration.

Action Task

The Action Task is the code that gets executed during the game when the planned action is to be acted upon. Idling is nothing more than waiting. Therefore we will configure it to use the Wait Action Task.

• Set Action Task to Rpai Action Task Wait
  • Set Wait Time Seconds to 5
  • Set Random Deviation to 2

Final Step

Set the Action Name to Idle. This is the name that will appear in the debugger and editor.

Configure the Tutorial AI Controller

Open the Tutorial AI Character Blueprint. Open Details for the Rpai Brain Component component. For Behavior select Tutorial Behavior

Play!

Now press play and watch your AI decide what to do, execute, and re-plan in a continuous loop! If you want to see all of the internal workings, learn how to debug Reasonable Planning AI.