So, what can we use the collected mana for?
What I'd like to implement as a ranged attack is a classic energy fireball. It fits the "stolen energy" theme and it should be cool to make with hand tracking.
Even in this case, we might do all kind of variations with little changes (for example, making it more or less powerful depending on the "charge" time, which would deplete different amounts of mana).
The way I imagined the firing is:
recognize a combined hands pose to start the charge
when charged, allow to either
fire it, with a second hand pose (depleting the mana) or
cancel it (keeping the mana), by dropping the charge pose
In a more sophisticate version of the enemy AI, they could switch to a flee/dodge behaviour if they see the player charging the fireball, but let's just keep note of this for the future.
I'm pretty sure that to keep things fun some form of targeting help will be needed (the need to be 100% precise in aiming the fireball at moving targets might make things frustrating), but for now let's focus on actually shooting the fireball.
This is a classic job for a simple FSM.
As input to the FSM, we can basically have a single value indicating
"fireball charge" pose recognized
"fireball shoot" pose recognized
none of the two poses recognized
All the remaining logic is a matter of switching between states depending on the current input and some timers, and adding a bit of procedural animation (and, later, some VFX).
I'll deal with the pose recognition later, for now I'll test using two keys for the "charge" and "shoot" poses: later I'll substitute the act of keeping the key pressed with the act of keeping the hands pose.
When starting to implement some moderately complex interaction like this, cutting the iteration time is crucial, because you want to try different values for the many timings involved: ideally, I would have an in-VR UI to tune values and a persistence system allowing me to save them from "play mode". When I'll start building the actual game, something like that will be one of the first things to get implemented: it's a good investment of time, because the sooner you make it, the sooner you can use it to accelerate the development (and make it more pleasing).
That said, for this prototype, a bunch of keys and some public values exposed in the Unity inspector, together with "paste component values", will be good enough.
While implementing, some question arose:
what happens when the fireball hits an enemy? Does it go through and keeps destroying enemies, or ends it life there? What if it bounced around? How long does it live if not hitting anything?
Let's keep things simple and, for now, just make the fireball explode when hitting something, and disappear after a few seconds if not hitting anything.
After a bit of fiddling and tuning, I got to a fireball launch that feels "good enough" for now.
It's not perfect by far, but with a little practice it's not that hard to hit enemies at a reasonable distance.
Actually, I think I'll skip doing the bit of auto-targeting/homing missile behaviour that I was expecting to be crucial.
Notice that even this time a good debug visualization was crucial.
I added a gesture to enable/disable the debug mode, which shouls a series of indicatos and pauses the enemies.
There's two small spheres between the wrists and the palms. The direction between these two points is than added to the direction of the gaze, which help in directing the fireball not only where your hands are pointing, but also where you are looking.
The result of this sum is shown by the orientation of the small blue arrow.
To charge the fireball, the hands must be open and parallel, with the wrist not too far. After charging, to shoot the fireball, one must widen the angle between the hands.
Here's a video showing a bit of testing. Of course, for testing purposes, I skipped the mana depletion. As you will see, it's not perfect, but a decent start.