2024-03-18 - Handling multiple clips
Let's start the week doing a bit of refactoring and clean-up of last week work.
When I sketch an implementation I sometimes put multiple classes in one file (in this case, I had in a single file `AAnimClip
`, `AnimClip
`, and `AnimClipCapture
`, for example).
I put each class into its own file and also ended up renaming:
`
UpperBodyHeaderData
` to `UpperBodyAnimClipHeader
``
UpperBodyFrameData
` to `UpperBodyAnimClipFrame
`
I also removed a bit of test code not needed anymore.
Time to attack the TODO list written at the end of last week.
First, I want to be able to handle multiple clips through the UI.
I'm going to enumerate the directory where I'm saving the clips and create a slot in the UI for each clip. Then, I'll add an extra "empty slot" so that one can always add a new clip.
For the slots, I'm going to use the scrolling panel from the sample canvas I started from last week.
Giving the clips proper names is a problem for another day: to get started I'll automatically assign them a name based on the current date and time.
I managed to get it working as I imagined:
the empty slot at the beginning only allows you to start a recording
while recording, you can't change slot
as long as you're not recording, you can pick any slot and play its clip
you can overwrite a slot with a new recording
Here's me doing a little testing:
There are some details I put a bit of extra time into, like the fact that, when you finish recording a clip, it automatically gets selected, or the fact that the naming and the directory listing cause the slots to be naturally ordered: a new clip always get inserted at the beginning.
The UI panel is not great, though: saving and loading a clip are blocking operations, which is very lame (see the hourglass briefly appearing, when the application stops responding for a moment?).
As this is just for internal use, the wise option should be to not care and leave it like this, but I'm not sure I will be able to live with it. I'll probably end up fixing it.
Still, there's more important things to do first... like, going to sleep.
2024-03-19 - Virtual keyboard, clip deletion
The default naming based on date/time is practical, but I really want to be able to rename clips.
This sounds like a good opportunity to try out the virtual keyboard offered by the SDK. Let's see if it's quick and painless to do.
Well, it wasn't painless nor quick.
I couldn't understand why I couldn't get the keyboard to visualize in my scene, until I went with the debugger and noticed that, in the SDK code, this part takes a very long amount of time:
var loadGlbCoroutine =
_gltfLoader.LoadGLBCoroutine(supportAnimation: true, loadMips: true);
while (loadGlbCoroutine.MoveNext())
{
yield return loadGlbCoroutine.Current;
}
In my eyes, it simply wasn't working. Instead, I just had to wait. I figured it out by chance, by leaving the game running while I was grabbing a packet of crisps: when I went back to the pc, to my surprise, there was the keyboard waiting me in the scene, finally loaded and visible.
I'm still not sure what's going on, because if I try the sample scene, the loading is (relatively) quick.
While I am usually finding the SDK stuff pretty good and well documented, I didn't have a good experience with the virtual keyboard.
Maybe I'm skipping some kind of small but crucial detail, but anyway, other things about the virtual keyboard don't feel good/right/polished: the keyboard positioning system, the way to show or hide it, the binding to UI controls.
I don't want to spend more time on this for now, at least before completing other features first, so I'm gonna accept my failure and move on.
Ending the day with no progress at all bothered me a little, so I decided to put in another hour of work before going to bed. I went for an easy target and added a button to delete a clip.
Let's see:
It's not perfect: it's a little bit hard to activate, and it could cause an unwanted deletion if pressed by mistake.
A deletion should either ask confirmation to the user, or require a very explicit activation (like, a long press or dragging a sliding button). Something to fix in the future.
Anyway, it does the job, and now the day doesn't feel totally wasted.
2024-03-20 - Slicing clips
Today I'd like to add the possibility of cutting the clips through my UI.
Usually, when recording a clip, there's always a bit of clean-up to be done at the beginning and at the end, because of the gestures needed to start and stop the recording itself.
It's possible to avoid the problem by having someone that starts and stops the recording at the pc, or minimize it by automatically cutting a certain amount of time at the beginning (showing a countdown) and at the end (good luck with that), but I don't want to have these restrictions.
Another thing that I want is being able to easily obtain perfectly looping clips, so I think I'm going to let the user select two timestamps, to be used as beginning and end of the playback, and allow them to preview the playback of the selected slice (before cutting it as a new clip).
An UI component that should be suitable for the task, that I've already used in the past in other projects, is the range slider provided by the `com.unity.uiextensions
` package, available here.
Of course, I must also implement the back-end functionality (the creation of a clip from another), passing the two timestamps, but that should be trivial.
I was almost done, I just had to add a checkbox to define if the playback of the clip had to be locked to the clip selection as defined by the range slider.
Then Unity crashed on a CTRL+Z, and considering that it's late and how long the editor takes to open the project... no demo video for today!
2024-03-21 - Playback options
Ok, time to complete the "play selection" checkbox that I was doing yesterday, before the crash.
Or more precisely, to do it again, as I had not been saving the scene for a while.
I added the "selection" checkbox, to force the playback to be restricted to the slice of clip selected.
This allows a precise preview of what clip one would obtain by cutting the current selection.
I also added a "loop" checkbox, so that the playback restarts automatically only if desired.
While I was at it, I added a "go to beginning" and a "go to ending" button. These also change their behaviour depending on if the "selection" checkbox is on.
I changed the buttons labels to make things a little more compact (for example, "Play" is now ">"). Yeah, they could use some proper icons, but players will never see this panel, so I don't really care.
Here's a video of me testing the new features: I capture a clip while throwing a punch, standing, then I sit and clean-up the capture cutting only the meaningful part of the clip, testing that the controls work properly. At the end, I cut the clip and verify that the original clip is intact, and that the new one features the sliced selection.
I'm pretty happy about the result, it's quite convenient to use.
There's only a couple of details I'd like to take care of before I move to the next feature.
I want to be able to tune the playback speed, and maybe even having the possibility to tune the capture interval and the frame interval of the saved clips could be useful.
Additionally, I want to have the possibility to have another "reference" skeleton visible while recording a clip: this will allow me to be sure that I can record clips that can work well together, and that can loop cleanly, with beginning and ending in roughly the same pose.
2024-03-22 - Playback speed
To tune the playback speed, I'm going to add a multiplier and some kind of knob on the UI to adjust its values.
A slider is not good enough because it would force me to define a maximum speed. And even if I could probably hardcode a reasonable value (like, I don't know, 5?), the higher the value, the more I would lose in terms of precision on the slider. And precision is important because I expect some minor adjustments most of the time (like, a slight speedup, at 1.2X speed, or a minor slowdown, at 0.9X...).
I tried using the knob control offered by the UI extensions package I installed for the range slider.
It didn't work well, and I couldn't see any obvious mistakes on my part. Maybe it would require some editing to work properly in VR?
I glanced at the code but I really didn't feel motivated to try to understand what the problem was.
I must remember myself that I'm only doing an UI panel for internal use, and I shouldn't spend much time on things players will never see.
So, I decided to scrap the knob and use a standard UI widget, the `TMP_InputField
`.
An input field will allow me to easily set precise values when (if?) I manage to have the virtual keyboard working.
Meanwhile, I can easily implement a fine tuning of the input field value based on cursor dragging.
I just need to pinch on the input field, and drag the cursor left or right to have the value adjusted smoothly.
It's not perfect, because to prevent the normal dragging interactions in the text field (which select the current text value, as you can imagine) I did set it as non interactable (but it still raises the low level events I'm using). So it's greyed out, which is counterintuitive.
Still, it's practical, and didn't take long - which is exactly how much time I have available today.
Let's see the playback speed change in a short demo video:
Not that bad. And another week is over.
Where to start next time?
I still need to add the tuning of the capture frequency, which shouldn't take long if I use the same UI control I used for the speed.
I want to be able to visualize a reference skeleton when recording clips, to help in capturing looping clips
I need to be able to playback a clip on a skinned mesh and not only using the debug gizmos (which only work in editor!)
after doing that, I need to test everything with an Android build running standalone on Quest (I've been using Airlink for now)
This is probably going to take another week, but then I should be 100% done with body tracking capture and playback. Unfortunately, next week I'm going to be super busy, so I need to take another little break: see you in two weeks for the next DevLog.