2024-09-23 - Measuring mirror proximity
Let's continue from where we stopped. I need to
detect if the player is touching/approaching the "mirrored" skeleton
implement the special "mirror portal" logic
I would also like to do some kind of VFX on the portal surface, to make it react to the player's body, but that's not a priority at this stage.
I think the most natural thing to do is reaching into the mirror with the hands, but somebody could try with the head too. I mean, I guess they could also use the elbows.
I'd like to have an indication of the minimum current distance between body and mirror, so that I can modulate some effect to ease into the "threshold" point where the transition happens.
Basically, the problem reduces to calculating the distance between a point (a body part) and a plane (the mirror). Once I can do that, doing it for multiple elements and considering the minimum is trivial.
I did a quick test considering head and hands, and showing the distance values in the debug panel.
The Unity API offers a `Plane
` class with a `GetDistanceToPoint
` method, so I just had to initialize a `Plane
` with the mirror position/normal, and fetch the head/hands position from the body input data.
For example, that's the code for the head distance:
Plane p = new Plane(m_vMirrorNormal, m_rMirrorTr.position);
Vector3 vHeadPos = BodyInputManagerBhv.i().getHeadPose().position;
float fHeadDistance = p.GetDistanceToPoint(vHeadPos);
Of course, I could do better, considering the actual meshes geometries, but that sounds like something for a future polishing phase, and after having the final 3D model so that I can fine-tune the threshold distances.
2024-09-24 - Mirror proximity indicator
I'm going to bring the test done yesterday to a state which I can actually use in the game logic.
I'm going to define two values which I will be able to easily tune:
`
farMirrorActivation
``
nearMirrorActivation
`
I will compare the minimum distance of those measured from body to mirror with this range, and derive a 0-1 value from there.
I will then use this value to modulate a "pre-transition" effect, and finally, when the value reaches 1, I will activate the "mirror portal" and teleport the player to the menuZone.
Calculating the 0-1 value was trivial, and I also did some restructuring so that the processing is done in the "gameplay logic" part of the code.
Yesterday, for testing purpose, I cheated a bit and did it accessing the body tracking directly and not from the "state management" layer.
That took some time, as I also ran into a bug and into something I had only partially implemented and had to be completed before usage. So, that's it for today!
2024-09-25 - Mirrored character placement
I have a mirror proximity indicator which I can use to activate the teleport to the menu zone, and I have the "mirrored" character.
But I don't have the "mirror portal" yet.
To get started, I'd like to use the bridge portal system I already have implemented, and just change some details about how the portal opens or activates the teleportation of the player.
If I manually open the bridge portal to the limbo zone, of course, the mirrored character becomes covered by the portal: to focus on the "mirroring" I temporarily placed it on the limbo platform, together with the player avatar.
It's finally time to move it to the destination platform, in the menu zone, so that it can show up only inside the portal.
Ok, I added a bit of logic to fetch the menu platform placement and used it to adjust the position of the mirrored character.
I also did the layer setup that I usually do to properly cull objects depending on the camera which is drawing them (the "normal" camera, or the camera for the portal render texture).
I also visualized the "proximity" index by setting the screen fade opacity, just for testing.
You might notice a problem, which I already spotted a long time ago but haven't faced yet.
The "content" shown in the "distant" bridge portal is wrong. It doesn't feel like the portal is coming forward, but like it is a strange window into another space, which feels right only when completely opened. The effect is even worse in the headset, because of stereoscopy: you see the "close" mirrored avatar, but confined in the "far" hexagon, and your brain and your eyes complain.
In my next iteration on the portal rendering, I will get it fixed. I'm tempted to try and do it now, but I don't want to wander off the path.
...and the Meta Connect keynote is about to start, so that's it for today!
2024-09-26 - Proximity indicator VFX
Well, the Meta Connect news about the Orion AR glasses prototype were pretty cool. On the VR/gaming side, nothing spectacular, but some good little things. And I think the Quest 3S has potential to further expand the VR user-base, which of course is a good thing for the medium. And more potential players for _Particular Reality_, yay!
So, let's get back at work, or it will never ship!
We have the mirrored character properly positioned, we have an indicator of the player proximity to the "mirror portal", and we have an avatar ready to change visualization mode.
Two things are still missing:
some kind of effect to bind to the proximity indicator: yesterday I used the opacity of the white quad for quick testing, but that's reserved for the mode transition when going from limbo to menu (and changing avatar)
some custom logic for the special "mirror" portal behaviour: it must open without requiring the opening gesture, and the teleport should happen depending on the proximity indicator and not the portal crossing
Let's proceed in order.
So, what kind of effect do I want to activate depending on the proximity indicator?
I think that it should be something on the portal itself. The player should understand that by reaching into it something might happen.
I don't feel very creative in terms of VFX today, so I went for something super-simple that I will definitely replace at some point.
I passed in a `float
` value to the shader that draws the portal contents, and used the value to fade between greyscale and normal color.
Simple enough to take the chance and throw in a fragment shader snippet:
float lum = dot(col.rgb, float3(0.2126, 0.7152, 0.0722));
float3 gray = float3(lum, lum, lum);
float3 c = lerp(gray, col.rgb, _ActivationLevel);
col.rgb = c;
I calculate the luminance of the color, and use it to define an appropriate gray value.
Then, I linearly interpolate between the "normal" color and the grey depending on the `_ActivationLevel
` passed from outside.
If you're wondering about how the luminance calculation works, and what does the dot product have to do with it, the answer is simple enough to include here.
The luminance is the "perceived brightness" of a color, and can be calculated by modulating the values of its RGB channels for some constants.
For a "linear" RGB color `c
`, the luminance `lum
` can be calculated like this:
lum = (0.2126 * c.r) + (0.7152 * c.g) + (0.0722 * c.b)
If you remember the dot product formula, you already understood why that works.
If you don't, well, for two 3D vectors `a
` and `b
`:
dot(a, b) == (a.x * b.x) + (a.y * b.y) + (a.z * b.z)
So, it's exactly the computation we need for the luminance, with `a = (0.2126, 0.7152, 0.0722)
` and `b = c.rgb
`.
2024-09-27 - Mirror portal activation
It's finally time to face the core challenge, which is implementing custom logic for the "mirror portal".
I already have a (bridge) portal configured to bring the player to the menuZone.
Right now, all the portals work like this:
when the player is in a suitable position of the platform and performs the "opening" gesture, they appear/open
when they are crossed, the player is teleported to the destination location
So, the player positioning check, and the teleportation to the destination, also apply to the special "mirror" portal.
Instead, I need to provide alternatives for:
opening the portal (it should open automatically)
activating the player teleportation (it should activate when the proximity indicator reaches value 1)
As first step, I'm going to be shameless and hardcode the special logic behind a couple of conditionals. I have a very simple way to check if the portal available on the current platform is the "mirror" portal, which is just checking if the source or target zone of the portal is the limboZone.
I added the logic and it worked but... not everything went smoothly as expected! When I get to the destination platform, I find myself in a state which is detected as crossing back from the menu to the limbo with the "normal" bridge portal logic. I didn't think about it, but it makes sense if I consider how the normal "portal crossing" detection works.
Considering that I'm changing the way one exits the menu, I can remove that bridge portal from the menuZone definition and just avoid the problem.
Cool, it works! Time to test a Quest build in standalone mode and see if there are any problems that don't show up testing in-editor via Meta Link.
Guess what? It didn't work well! I think it's something related to the calibration offset... let's try to figure it out.
Ok, after a bit of debugging I managed to get it working consistently. There were a couple of things about how the reflection was calculated, and a world/local position I got wrong but worked anyway if there was no calibration offset, which explains what happened.
Let's see a capture on device - I disabled the passthrough for the capture, because the room is a mess, but that works too.
And another week is gone! I'm quite happy about the progress, but the code quality isn't great because I rushed things a bit. I wanted to see it done before the week ended, and I did it, but I took some shortcuts.
Next week I'm going to do a bit of refactoring and clean-up, before proceeding with the smart particles.