Since I implemented support for OpenXR, the performance of the game went down. Enough to be noticable. I decided not to investigate it immediately as I knew that with lots of new things coming the performance could be still affected and investigating what's going on there takes some time.

Finally, though, I decided to see why the game stutters much more than before. One of the things that I was suspecting was that when switching from VRAPI to OpenXR, I was also switching from Extra Latency mode to synchronised mode.

What's Extra Latency mode? Let me just explain how games work. Games process gameplay - all the action, then they render it and only then they show it to the player. Long time ago, gameplay + rendering was done in one frame and then displayed. You were seeing what happened almost immediately. As rendering was outsourced from CPU to GPU, it was possible to have CPU do gameplay in one frame, while GPU was rendering the results from the previous frame. The game was kind of lagging one frame behind the actual action. But this made it possible to show action at twice the framerate, so while it was "lagging" because there was one frame of delay, when compared to the old method, it was displaying the action with exactly the same delay (time wise) but was doing it twice as often.

Extra Latency mode introduces one frame of delay between processing gameplay and showing the results. This means that there's an extra frame added. One extra frame of delay. Why would anyone do so? Because in many cases of small hitches, when one frame is processed a little bit longer, it allows to "borrow some free time" from the next frame - ie. the next frame gets a little bit less time to process everything but most of the time it works out pretty well. The interesting part here is that something sometimes requires more CPU power and may pause some threads for a moment. It might be Quest's OS, it might be VR faries, it might be some bug. Most likely it is Quest's OS. If you don't have lots of extra time available and you have no hitches, you may still notice stuttering. With Extra Latency mode such a almost random hitches could be avoided better.

At least I thought it could be due to that.

Well, I still get the Quest's OS pause some of my threads (at least it appears so) but the problem was actually somewhere else. And that's why it is important to get the thread priorities right and not to fit into frame at 100% but leave some CPU power.

Actually, there was a bunch of problems.

And I think that a proper code review could question some decisions and would help me to avoid the problems you're to read about. But doing self code reviews rarely works.

There are some images below and it's best to open them separately. In the left bottom corner there is a stack of things being done with how much time do they take.

The first, CPU hogging synchronisation

The game was stuttering even when you were standing in an empty room, staring at a wall, no enemies spawned, rendering switched to debug ultra low, no sounds processed. And CPU was at level 4. When I noticed that I knew that there is something seriously wrong going on.

I will skip the investigation and just go to the conclussion: I have a mechanism that I should really change that I use for threads synchronisation. The problem with it is that while it waits, it hogs CPU. It constantly checks if it can move forward. This is the most basic error. The worst kind of. Something that should be avoided at all times and yet, I somehow let it through. For very very very short periods of time it could be temporarily acceptable (it shouldn't in general, but if you have tons of stuff to do, sometimes you just go with the low-hanging fruit solution only to fix it later) but if you're waiting 1/3 of a frame? A big NO.

I modified the mechanism to make it wait patiently (I still need a better solution here, going to introduce it soon). And just like that, the framerate stabilised, CPU level went down to 2 (the minimal value requested by the game, and it usually sits at 2).

But that was not over, I was still going to leave that empty room and encounter more problems affecting the performance.

The second, lots of proximity mines left alone with their thoughts

When I looked into the graph, I noticed that AI logic takes a significant amount of time to process. But there was no single culprit. It was a lot of object. And most of them were proximity mines. They were updating their AI logic every frame, no matter how far from the player they were. For no particular reason. I am a big fan of "what you don't see, doesn't need to happen continuously" approach. When I write code for anything I always assume that it could be skipped for a few frames. And if it shouldn't, it should request to be updated every frame. I already use such an approach for many NPCs. Actually almost all fighting NPCs. But some other were skipped.

Modified code to include other NPCs. But then I thought that it's not enough. There were lots of proximity mines throughout the levels. Way too many. The way they were spawned was not allowing to control their overal number -  I changed that. This way I was able to control the number, their density, where they are and I was able to avoid having too many items in general.

The third, maintenance work done at full power all the time

During each frame, there is some, I call it "maintenance work", that has to be performed. This checks how far certain rooms are, what objects should advance etc. The "how far away room from the player is" part I optimised a long time ago. It takes a very little time right now. But then, there is another thing that has to be done, checking what objects should be suspended and what not. All objects can be declared as "suspendable" or "always active". If an object is suspendable, if it is far enough from the player (room wise speaking, far enough from a room visible to the player), it is suspended. If the player approaches it, it gets reactivated. Many objects are removed though when they are too far away by the spawn manager. There is also a special room that holds objects that are used by game scripts that should remain inactive until required.

Such an operation was being done every single frame. But in many cases it it not actually required. The visible rooms should activate all objects immediately. The "locker" room should suspend them. But all the other? Right now I check up to 30 rooms (including visible + special), at least 5 of non visible (if too many visible + special). When there are 200 rooms generated, if everything goes wrong (too many visible rooms), each room will still be processed every half a second.

So far, all these bugs could be handled with a code review. Now we're moving into "you need to remember that procedural generation may get ouf of hand" territory.

The fourth, hundreds of airships doing nothing

Before I noticed the problem with proximity mines, there was another one. Airships. Actually, there is a special room where airships move and all airships you see (except those who are part of the gameplay) are just proxies. The proxies' logics were not updated so often but the sound system was checking them all the time, because their sound could propagate quite well into the interiors. And reversing the situation, where player was, there were lots of airships' sounds coming from many directions. Even though they were so silent that at that distance they were nothing more than some background faint noise. And not only that but also their movement was requiring them to be in sync with the actual ships.

For time being I decided to avoid having airship proxies visible through windows. If no one notices, I'll leave it like that. I mean, the vista through the windows is not the real place anyway - there are no enemies, no items. It's a model. If someone notices or it bugs me enough, I will think about some easy to implement solution.

But why suddenly there were so many windows?

The fifth, inaccessible parts of the world that were larger than where you could walk through

By a mistake I added a small room that was adjacent to where you could walk but it was itself inaccessible because there were bars on the way. The moment I saw this, I loved the idea of not only corridors stretching out beyond such doors but also rooms. I modifed the room to allow it to include further corridors and rooms. Only to find out that it was constantly creating an endless maze. There were thousands of rooms being created and the player couldn't move somewhere else because the game kept to create these rooms. I limited that and it was okay for a while.

Until I decided that I should drop the bars for inaccessible rooms and actually make the rooms larger. With an extra short corridor type added, the thing that you can see through the bars looks quite good. Like a different part of the level. It really makes you feel that the place is much larger than it is. The problem was that each room could have extra windows. And each window required vista (which is just another room). And each vista required airship proxies.

But even with the removal of airship proxies, when you entered some of the places (for example, the well), there were so many rooms to be rendered that just processing that was hitting CPU significantly and unnecessarily. I played around with limiting when windows can be added and how many rooms can be generated. With that I had to cut one thing that I liked. Items and enemies behind the bars. There are still enemies sometimes appearing there but are less likely to encounter. And no items at all - you would not be able to pick them up anyway.

There are still some problems with the performance but they seem to be more occassional and related to actual action happening. While I want to keep the game running as smooth as possible, I shouldn't focus too much on performance or fixing random bugs as I actually need to add more content.

But in general, I do enjoy working on performance. When you start you just have "something slows down the game" and you investigate various stuff, finding what could be the cause, then why it is slowing down and finally what can be done about it. It's a work that is composed of playing the game, checking performance, looking into logs and tools. For most of the time, it seems like you're not doing anything productive. You just play the game, have a disappointed look on your face and curse a bit. But the more information you have gathered, the more you know about the actual problem and when you finally figure it out and see an improvement, it feels really good. But even the investigation part is not that sad and gloomy experience. It's more like exploration of the game and code but from a completely different perspective.

Anyway, a new demo update should land soon and a devlog post will be there as well. There I will write a bit more about what's next big thing on the list.

Because the small things are sorting out issues with Windows builds - there are a few problems with various controllers (and I hope that I have all of them at home and they actually work!)