Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strange bug: only left Babylon.js VirtualJoysticksCamera joystick not working & workaround #215

Open
SugarRayLua opened this issue Jan 21, 2024 · 0 comments

Comments

@SugarRayLua
Copy link

SugarRayLua commented Jan 21, 2024

I've read on the forum that there currently maybe incompatibilities between Babylon.js camera control inputs and microStudio. I really was hoping to use Babylon.js's VirtualJoystickCamera to create 3D projects that can be controlled easily on mobile devices without keyboards. For unknown reasons, one of the Babylon.js VirtualJoysticksCamera joysticks doesn't work (the left joystick; the right joystick works fine). I debugged the problem and found that the left joystick sense input fine (it moves on the screen when the user touches it, and through code, I can determine that the left joystick is recording joystick movement; however, for unclear reasons, Babylon.js is not translating the camera movement with the output from the left joystick despite both the left and right joysticks using similar code to transform the camera (relevant section of Babylon.js code found here):

https://github.com/BabylonJS/Babylon.js/blob/master/packages/dev/core/src/Cameras/Inputs/freeCameraVirtualJoystickInput.ts

I therefore tried creating a workaround for the problem through code by manually obtaining the left joystick's output and then calling a function inside of microStudio which mimics Babylon.js's code above and that restores the left joystick's function. Code below (also submitted as a project for the community so can use the workaround now in their projects):

init = function()
  scene = new BABYLON.Scene()
  camera = new BABYLON.VirtualJoysticksCamera("camera", new BABYLON.Vector3(0, 5, -10), scene)
  camera.setTarget(BABYLON.Vector3.Zero())
  //attach default camera joystick controls but left joystick doesn't work yet on microStudio
  camera.attachControl(true)
  light = new BABYLON.HemisphericLight("light", new BABYLON.Vector3(0, 1, 0), scene)
  assetsManager = new BABYLON.AssetsManager(scene)
  meshTask = assetsManager.addMeshTask("task1","","https://assets.babylonjs.com/meshes/","both_houses_scene.babylon")
  // meshes are loaded asynch so define a callback function to do when meshes loaded
  // (in example below you could do something in the callback) 
  meshTask.onSuccess = function(task) end
  // need to 'load' the AssetsManager to start the load
  assetsManager.load()
  // define an object to hold information from the non-working left Joystick (senses input but doesn't move)
  leftJoystick = camera.inputs.attached.virtualJoystick.getLeftJoystick()
  // decrease sensitivity of joystick controls (default 2000.0 seemed too sensitive; lower # more sensitive)
  camera.angularSensibilityX = 20000.0
  camera.angulatSensibilityY = 20000.0
  // slow down camera inertia for better joystick feel (default = 0.9; higher # faster)
  camera.inertia = 0.8
end

update = function()
  // leftJoystickMovement function is microStudio workaround that captures left input input & restores movement)
  leftJoystickMovement()
  // added the following microStudio keyboard controls to sense keyboard movement if using desktop 
  if keyboard.UP then 
    camera.cameraDirection.addInPlace(camera.getDirection(BABYLON.Vector3.Forward()).scale(0.01))
  elsif keyboard.DOWN then 
    camera.cameraDirection.addInPlace(camera.getDirection(BABYLON.Vector3.Backward()).scale(0.01))
  elsif keyboard.LEFT then
    camera.cameraDirection.addInPlace(camera.getDirection(BABYLON.Vector3.Left()).scale(0.01))
  elsif keyboard.RIGHT then 
    camera.cameraDirection.addInPlace(camera.getDirection(BABYLON.Vector3.Right()).scale(0.01))
  elsif keyboard.A then
    rotateCamera("left")
  elsif keyboard.D then
    rotateCamera("right")
  elsif keyboard.W then
    rotateCamera("up")
  elsif keyboard.S then
    rotateCamera("down")
  elsif keyboard.Q then
    rotateCamera("rollLeft")
  elsif keyboard.E then
    rotateCamera("rollRight")
  end
end

draw = function()
  screen.render(scene)
end

rotateCamera = function(direction)
  // this function rotates camera similar to how Babylon.js code rotates camera
  local currentRotation = BABYLON.Quaternion.RotationYawPitchRoll(camera.rotation.y, camera.rotation.x, camera.rotation.z)
  if direction == "left" then
    // uses radians (2*PI radians = 360 degrees)
    rotationChange = BABYLON.Quaternion.RotationAxis(BABYLON.Axis.Y, -PI/180)
  elsif direction == "right" then
    rotationChange = BABYLON.Quaternion.RotationAxis(BABYLON.Axis.Y, PI/180)
  elsif direction == "up" then
    rotationChange = BABYLON.Quaternion.RotationAxis(BABYLON.Axis.X, -PI/180)
  elsif direction == "down" then
    rotationChange = BABYLON.Quaternion.RotationAxis(BABYLON.Axis.X, PI/180)
  elsif direction == "rollLeft" then
    rotationChange = BABYLON.Quaternion.RotationAxis(BABYLON.Axis.Z, -PI/180)
  elsif direction == "rollRight" then
    rotationChange = BABYLON.Quaternion.RotationAxis(BABYLON.Axis.Z, PI/180)
  end
  currentRotation.multiplyInPlace(rotationChange)
  currentRotation.toEulerAnglesToRef(camera.rotation) 
end

leftJoystickMovement = function()
  // note: in Babylon.js actual code, camera.speed * 50 but seemed too fast in microStudio so didn't * 50
  local speed = camera.speed 
  local cameraTransform = BABYLON.Matrix.RotationYawPitchRoll(camera.rotation.y, camera.rotation.x, 0)
  local deltaTransform = BABYLON.Vector3.TransformCoordinates(new BABYLON.Vector3(leftJoystick.deltaPosition.x * speed, leftJoystick.deltaPosition.y * speed, leftJoystick.deltaPosition.z * speed), cameraTransform)
  camera.cameraDirection = camera.cameraDirection.add(deltaTransform)
end

Not sure what exactly is preventing Babylon.js' code from actually translating the camera's position when running in microStudio (works when running in Babylon.js playground) yet still working for the right joystick but thought helpful to submit the issue as it would seem challenging to create a virtual joystick for Babylon.js from scratch (the microStudio button libraries don't seem to work when using Babylon graphics [2d vs. 3d canvas?], and it doesn't seem that microStudio comes with Babylon's AdvancedDynamicTexture extension that allows GUI creation on a 3d canvas].

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant