Advanced Prototyping - Week #29
Smart particles basics
2024-10-21 - Smart particles design
The smart particles should be one of the two subsystems related to the player avatar (the other is the "skeleton" management discussed in the week 27 and week 28 articles).
They should act as both an embodiment of the player when in active gameplay, and as a hint system teaching him gestures and other gameplay actions.
At least, this is what I'm imagining at the moment: the idea is to start somewhere and extend the usage as the gameplay needs change.
So where do we start? Let's extract from the design done in week 25 a list of things that the smart particles should be able to do:
catch the attention of the player by going into their field of view
guide the player towards a specific spot
show/suggest a gesture to perform
attach to the skeleton avatar and "follow" it
detach from the skeleton avatar
Ideally, the smart particles should transition smoothly from one activity to another, morphing between shapes and effects as needed.
But that's for later, when I'm going to work on presentation and VFX. As usual, instead, I'm going to focus on the logic first, and just implement a quick placeholder visualization.
I'm still prototyping, after all.
I'm going to define some state identifiers, with a brief description of the logic that the state should implement and the input it would need.
`
captivate`: the particles should catch the player's eye. A basic way to do it is move towards the centre of the player's field of view, at a certain distance, and idle around there.input: the player state data
`
guide_move`: the particles should guide the player towards a certain spot.input: the player state data and the destination spot (a location and a hexagonal slice identifier should suffice)
`
guide_look`: the particles should invite the player to look towards a certain pointinput: the player state data and the destination point
`
show_anim`: the particles should morph to a detached body (or maybe just hands?) which playbacks a captured animation, inviting the player to emulate itinput: where to perform the animation, and what animation to perform
`
skeleton_follow`: the particles follow the skeleton avatar, which should be the "normal" state during active gameplayinput: the player body data needed to match/follow it
`
skeleton_idle`: the particles don't follow the skeleton avatar, and stay idle wherever they were when they entered the stateinput: probably none
`
invisible`: the particles are not shown (startup state)
From what I described, this subsystem sounds rather complex compared to other things I've implemented so far.
When something is complex, one should try to further divide into manageable subtasks, which can be reasoned about (and maybe implemented) independently. And in fact, each of these states might end up having its own nested state machine.
One step at a time, I'll get there.
2024-10-22 - Adding a new subsystem
As discussed, the smart particles are an important and quite complex element, so I'm going to introduce a new subsystem to take care of them.
This means, as usual, defining their state management and presentation code. The first step is drafting these elements with empty stubs that I will fill along the way.
At what "level" should the smart particles operate and, consequently, where should I put their state? Application or game?
Like the avatar state, I'm pretty confident they should go in the application state, as I need them working properly outside of actual gameplay, in the limbo/menu zones too.
I'm not sure yet if I'm going to need some kind of "gameplay only" data override to handle them in-game, but I'll figure that out in the future.
Ok then, I defined
`
SmartParticles` and `SmartParticlesManager` (state management)`
SmartParticlesBhv` (presentation)`
TestSmartParticlesBhv` (testing)
...and I wired things up so that the `SmartParticlesManager` is nested inside of `AppStateManager`.
This initial setup should be ok: I checked that the game runs and that all data is initialized properly.
The new subsystem currently does nothing, but also doing nothing is something, in the sense that it's not crashing the game.
2024-10-23 - Energy bars refactoring
Yesterday I discussed how the smart particles data had a bigger scope that the gameplay data, because they also need to run outside of gameplay.
But there's a little thing that bothers me, and it's the energy bars, for which the opposite is true. They are now always visible, while they should be a "gameplay only" feature.
Let's look again at our table about player management:
My initial idea was that health and mana should be represented through the particles following the skeleton.
I'm not sure exactly how that is going to happen, in terms of visualization, but in terms of structure and update logic, moving the energy bars management to the smart particles code I introduced yesterday sounds like a small step in the right direction. Or, at least, in a less wrong direction.
So, as a first step in implementing the smart particles subsystem, I'm going to move to `SmartParticlesBhv` (taking care of presentation) the references to the energy bar nodes and their management.
I will make it so they are only visible in the `skeleton_follow` and `skeleton_idle` states, where they should behave like the particles: following the avatar, or standing still.
To be able to test this, I will also implement a restricted version of the `SmartParticlesManager` FSM, limited to these states and inputs:
Of course, to be able to test it, I will also add the `startFollowingSkel` and `stopFollowingSkel` input signals as buttons of the `TestSmartParticlesBhv` script.
Ok, I did what planned. The only problem is that there's a mismatch between the hand pose that I'm using to position the energy bars and the upper body skeleton. I think it's the usual mismatch that was introduced at some point updating the Movement SDK, and that I already discussed a few times.
I have two options: I change the positioning of the energy bar to use the (slightly wrong) bone pose as taken from the skeleton, or I try to fix the mismatch once and for all.
Well, today I won't do either: it's getting late.
2024-10-24 - Attached and detached particles
The most obvious step to do today is adding back the "particles avatar" I recently disabled, which in my vision is also part of the smart particles system, and specifically of the two states that I decided to use as a starting point: `skeleton_follow` and `skeleton_idle`.
What I need to do should be pretty simple: give control of that "particles avatar" element to the script orchestrating the smart particles presentation (`SmartParticlesBhv`) and
setting it hidden in the `
invisible` stateupdate it so that it follows the player in the `
skeleton_follow` statekeeping it visible, but not updating it with body tracking data, when in the `
skeleton_idle` state
Well, that was quick as expected, for once! And not interesting enough to capture a video, but here's an image.
Obviously, the particles intersect with the skeleton mesh, and the result is an ugly mess.
But that doesn't bother me right now - it's still just a placeholder visualization.
Instead, there are two more significant things I want to work on.
For starters, I currently have more than one `OVRBody` script in the scene, because the quick thing to do, starting from the initial sample, was fetching the body tracking data that way.
But I did significant work (weeks 16-17-18-19 and the extra article "Deep dive into body tracking") to be able to drive a skinned character mesh from body tracking data saved in a custom motion clip, and I took care of that adding an indirection layer between sourcing the body tracking data and animating the skinned mesh.
Then, I only used that data path for playing saved motion clip, but I could do the same with live data. Why bother?
Well, as I'm starting to have multiple skinned meshed driven by the same tracking data, I can easily avoid some useless data duplication and processing. But a possible improvement in performances is not the primary factor: it's more about decoupling subsystems and keeping the data flow tidy.
It makes sense to start separating and keeping minimal the parts that take input data, and that use a specific API to do so. Today such data comes from the `OVRBody` script, tomorrow it might be something provided by Pico, Sony, Apple, Valve...
I know that I'm just prototyping, but I guess today I feel positive about the distant future.
Done! Now I have a single instance of the body tracking data, and I use that to control two separate skinned character meshes, the "skeleton" and the "particles".
And I have an idea for a third one, but we'll discuss that tomorrow.
What's the second think I wanted to fix?
There's a problem with the "particles" character that I've been ignoring so far: sometimes, when I move/rotate my head while testing, the VFX particles disappear for no apparent reason. When that happens, moving back causes them to become visible again.
Let's see if I manage to remove this annoying glitch.
Ok, after a brief search, I found about a culling parameter that one can set in the VFX graph to avoid processing "out of bounds" particles.
There were some fixed bounds set, centered on the origin of the world. So, when I moved outside of those bounds, the particles disappeared.
To try a fix, I changed the center of the bounds so that it follows the main camera.
Well, it works. It's not great because I should pass in as parameter the position to use as center, and not make it dependent on the camera. But it's super late and I'm happy enough of having found the cause of the problem - a better fix won't take much time.
2024-10-25 - Yet another character instance
While implementing the body tracking data "routing" to multiple avatars ("skeleton" and "particles"), yesterday, I wondered if that could also be useful for other things.
Well, there was an obvious one: when the player is using a portal, I can have another avatar instance placed on the "destination" platform, so that the part of player avatar which is going through the portal can be properly seen.
Let's explain it a little better, starting from the beginning.
Right now, this is what happens when you enter a portal: parts of your avatar go behind the portal mesh which is showing the texture rendered by the "portal camera" as discussed in the past:
I didn't mention this until now, but honestly it's something that has been bothering me a little since I first implemented the most basic form of portal, during the PoC prototyping days.
Now, if I put a clone of the player avatar positioned on the destination platform, at the same platform-relative position, I should be able to see the parts which are now hidden by the portal. More precisely, I'm going to see the copy of those parts, coming from the clone.
Let's give it a try.
Ok, getting the basics working was relatively easy. I just had to create the copy, animate it with the same body tracking data, like I did yesterday, and apply the proper offset. Let's see where I'm at:
The video should show "the trick" pretty well. To test more effectively, I disabled the "particles avatar" and configured the "cloned" character as activated (meaning that they have the brighter material), which makes it obvious what mesh you're actually seeing when throwing your hands into the portal.
While I'm sure one could fake this in terms of rendering, doing some work on the shaders involved, there are advantages to the naïve approach I'm using. For example, is easy to have a slightly different mesh for the "other side" of the portal, or to make it interactive.
This needs some thinking in terms of gameplay, but I'm not far from being able to punch an enemy through a portal, for example.
Anyway, I'm not done yet. I need to
make the cloned character visible only when it's behind the portal
set the cloned character layer so that it works properly with the zone crossing, when opening a bridge portal
set the cloned character to the same state of the original (activated or not)
re-enable the particles avatar too
All done! Here's me putting my hands into two portals, a "normal" and a "bridge" one.
Of course, the difference is a bit less obvious now that I'm not using the "activated" material for the avatar on the other side of the portal. Still, the particles are only on the primary avatar, and the health/mana bar too. That's why the moment one gets teleported, the particles appear back (instantly) on the arms - that's when the cloned avatar disappear, and you're only seeing the primary one.
While testing, I remembered that didn't bother fixing the health/mana bar that I poorly attached to the player on Wednesday.
I also realized that I have an opportunity to link the smart particles to the portal management in a more sensible way (in terms of design and game world consistency).
But... I'm going to save this discussion for next time!