The first thing about music playing is to get rid of the controllers, because in reality we are not playing instruments with a controller. We embedded Oculus' gesture API to implement gesture control functionality. Moreover, we tuned as many joints as possible for different triggers of the playing process to make the interaction more natural and smooth. Natural interaction has always been our main concern, we want to design a game for children with low learning costs.
We believe that visual effects add more variety and interest to the interaction between children and music. Therefore, we have incorporated unique visual effects for each musical interaction. Taking circular visual effects as an example: The program checks, during each play, whether the current hand movement speed, `currentSpeed`, is greater than or equal to 0. If the hand is moving in the positive direction, the `MoveCount` counter is incremented by 1. This allows the program to generate pink and blue circles sequentially based on the speed of the play.
We believe we need to restore the logic of real instrument interaction, so we set up various triggers such as acceleration trigger, height trigger, and force trigger to restore the real physics. We also envisioned designing VR gloves to better utilize sensors and haptic feedback.
During the development and exploration of our demo, we conducted some user testing. The results helped us understand how users subconsciously want to interact with different shapes, how users' posture and position affect their gestures, and whether MUSO is Easy to use.