Support running the user controls system in a different timestep than the Tnua and (/or?) physics backend systems

[idanarye]

@janhohenheim mentioned in Discord that input mechanisms (like just_pressed) don't work well in fixed timesteps. This is a problem because it often makes sense to run the physics engine in fixed timestep. Crates like Leafwing Input Manager can handle this, but it may also be desirable to just allow the user input system run "out of sync" with Tnua's internal systems (that will still need to run in the same timestep as the physics backend)

I've made this a discussion instead of an issue because there are several issues that need to be resolved when the user control systems are not in sync with Tnua's internal systems, and a discussion can have multiple comments each with its own replies which we can use to discuss these various aspects.

Also note that this is mostly a nice-to-have - I'm not sure this can be reasonably done, and it's always possible to just keep the user control system in the physics timestep and just use LWIM.

[idanarye]

Problem: actions need to be fed each frame

User control systems are not expected to explicitly tell Tnua that the action ends. Instead, the action ends when it is no longer fed. For example - when the user releases the jump button, the user control system stops invoking controller.action(TnuaBuiltinJump { ... }), Tnua sees that it no longer receives the action and informs the action that it should end (note that it doesn't mean that the action is going to end immediately. In case of TnuaBuiltinJump the action continues and applies forces to shorten the jump. The action only ends when the character lands or when it gets cancelled into another action)

If the user control system runs in a different timestep from the internal system, the internal system may have two consecutive steps without the user control system running between them, which means that the second step will see that the action was not fed (because the user control system was not there to feed it) after the first step already marked the action as fed.

[idanarye]

"Internally" Tnua does not know the semantics of the schedule it runs in. It gets a schedule given to it and just runs everything there. If you add TnuaControllerPlugin::new(Update) it adds everything to the Update schedule and if you add TnuaControllerPlugin::new(FixedUpdate) it adds everything to the FixedUpdate schedule. It has no way to know that in the former it should drain in RunFixedMainLoop while in the latter it should drain in FixedUpdate.

And I don't want to make the user supply that knowledge either. I was not familiar with RunFixedMainLoop (I assumed something was running the fixed timestep schedules, but did not know exactly what) and I won't assume the average Tnua user wants to know about these details.

One ways to make it work though it to put the responsibility of supplying that knowleldge on the integration layer. That is - TnuaAvian3dPlugin will know about how Avian works and TnuaRapier3dPlugin will know how Rapier works. I'll remove the ability to freely select the schedule, forcing users instead to use a preset that must match the one they registered the physics engine with. So either:

app.add_plugins((
    PhysicsPlugin::default(), // Using 0.1, so it's still Update
    TnuaAvian3dPlugin::Update,
));

// OR

app.add_plugins((
    PhysicsPlugin::new(FixedUpdate), // Using 0.1, so it's still Update
    TnuaAvian3dPlugin::FixedUpdate,
));

(maybe I'll add a Custom option to that enum, for manually specifying schedules)

The physics integration plugin will have the responsibility of registering a TnuaSchedulesConfiguration resource, which will contain multiple schedules. TnuaControllerPlugin - and the rest of the main crate's plugins - will have to use that resource to determine which schedules to register their systems on.

---

This... is starting to get complicated...

[janhohenheim]

In the past we've done a bad job at presenting and/or documenting this anywhere, but RunFixedMainLoop is essential to know when working with fixed timesteps imo, and the official example reflects this on main:
https://github.com/bevyengine/bevy/blob/main/examples/movement/physics_in_fixed_timestep.rs#L98

Bevy now supports a RunFixedMainLoopSystem::BeforeFixedMainLoop system set for a reason in its API, so I would not try to hide that from the user. Users of Avian 0.2.0 need to know about this anyways, with or without Tnua, if they wish that their changes apply to the current frame and don't lag behind.

Disclaimer: I made the changes in Bevy mentioned above. My goal was to make exactly this workflow less arcane and weird to work with, and to improve the documentation about how to use these schedules.

[janhohenheim]

That said, I see the problem. Supporting both Update and FixedPostUpdate as a schedule for Tnua means we cannot just run these basis draining functions without careful consideration.
I suppose we could also just tell the user upfront "hey, this is probably not going to work as you expect if you use just_pressed when using fixed timesteps, e.g. when using Avian's default config. Please just use LWIM, thanks".
LWIM will get upstreamed at some point in the future anyways, so we may as well embrace it. If we go this route, it may be didactically valuable to also use LWIM in the examples as a dev-dependency.

[idanarye]

I always worry about adding Bevy plugins as dev-dependencies (actually, with Tnua's workspace structure, it'd be a proper dependency of the demos crate) because then when Bevy updates I have to wait for them to update before I can build the demos.

[janhohenheim]

That's completely justified, yeah.

[idanarye]

Problem: user control systems may want to interact with the sensor

This, for example, is how ghost platforms work - the user control system needs to modify the TnuaProximitySensor and overwrite its output with data from TnuaGhostSensor. This only works when the user control system runs in TnuaUserControlsSystemSet - between the physics integration layer updating the proximity sensor and Tnua's controller system reading it. If the user control system runs in a different schedule (doesn't even have to be a different timestep!) - Tnua will overwrite these changes to output before it gets a chance to read what the user control system has set.

There are also issues with some "lags" if the user control systems wants to utilize information from the proximity sensor output (e.g. make a bigger jump when standing on entity marked as trampoline)

[janhohenheim]

That is a good use-case for having the actions be in the fixed update, yeah

[idanarye]

Problem: using the action flow status

TnuaController has a method called action_flow_status which is used for things like air action counting. One of the states is can return is ActionStarted which indicates the first frame of the action (because special things may be needed to be done then). If the user control system lives in a different timestep, it may miss it.

The user control system may also miss states like ActionEnded or Cancelled, which are not limited to a single step but may still, in practice, not last long enough for the user control system to notice them and act upon them.

[janhohenheim]

This could potentially be fixed by triggering observers on these special state changed

[idanarye]

Not sure about that. While these transitions don't happen every frame, they happen often enough (and are parallelable enough) that I don't feel comfortable using a stop-the-world mechanism like observers for them. If anything, forcing users to update them in a separate system that runs in the fixed update would be more reasonable than forcing users to update them in observers.

[janhohenheim]

Then using regular events would work, as the user can themselves schedule the readers wherever they want.