Greetings Awoken It's been a minute since our last update, and we wanted to say thanks for hanging in there with us. We've been deep in the workshop, so to speak, and we’re excited to finally come up for air and show you all the things we've been building for Early Access. Right now there is a huge emphasis on bug fixing and removing as many issues hindering the experience on the tech side as possible. We are eternally grateful for all the amazing feedback, reviews and bug reports. This information is super important at this stage as it helps us to make the game as stable and problem-free as possible ahead of full launch. Feature-wise there is the much-anticipated map table, which was part of our roadmap. V1 of this will be the lightweight version, this won't track progression on each island yet, but will display all islands in the area to make navigation much easier! Additionally, we've introduced a new comfort system. Some items boost comfort (e.g., decorations, furniture), while others have a discomfort value if close-by (e.g., engines). This means you'll need to think about separating different areas of your ship to maximize your crew's well-being and earn a "rested" bonus! We are adding some quality of life improvements including full implementation of key rebindings, renaming storage and an option to disable screenshake. The team has also been working on another big balance pass on health and damage, to improve combat experience in more difficult areas.
We have been refactoring parts of storage/inventory, specifically to fix issues with moving items or losing them when splitting stacks, upgrading schematics & reloading the game.
Alongside these improvements there are a few other highlights which we will cover in our latest Skywatch later this week. As well as all the work we are doing towards full launch.
The biggest thing our tech team is solving is definitely the potential loss of progress that affects some players, this has been a really tough egg to crack, with many variables at play. Of course we are working to resolve this completely before launch, but we'd like to go into a bit more detail so you can understand what is involved.
As the problem is such a complex challenge, it’s best to hear this info from the Developer’s POV. Here are Tom’s Dev notes on the issue.
Lost Skies has a few problem areas that need to be addressed in order to become robust at scale (for larger worlds / entity counts / higher player numbers). These are the issues that present some form of instability or excessive network traffic :
All players are always checked out for everyone playing. This means everything players are up to no matter where they are on the map, is synchronised over the network. This creates excessive state sync for players that could be on other islands so the data is not relevant to you.
Ships have a similar issue, they are always checked out for everyone, but one reason for that is their initial state is quite large so accepting a minor spike in network traffic once per ship spawn was preferred. However, they are expensive to have around on the CPU as they are made of hundreds of objects, especially when players start experimenting with multiple ships.
World Save data: This is a big one. There have been reports of data loss, corruption and worlds ending up in a volatile state when reloading from saves. As it stands, saving of all world data is handled by the host of the session, which makes sense. But save data for individual entities is grouped per island. So when a save event is raised, clients on islands the host is not on have to send huge amounts of data back to the host including every entity on the island, even if 90% of it has not changed.
Island-based checkouts: All network entities on an island are permanently checked out when you are on that island. We have the concept of an 'island authority' client. That is the person who is in charge of all the entities spawned by the island. Some areas do utilise distributed authority like trees, certain puzzle elements, etc, but AI for example, is all controlled by the client chosen as that island's authority. The main reason for this is data consistency, as all buildables, AI, critters, loot etc, is part of the island data that is saved, it all needs to be maintained by a single owner that then sends back the data to the host (or just saves locally if they are host). We suspect these large multiple packet sends of save data from multiple people all coming back to the host can be problematic, especially on an exit game event. As clients may disconnect before the save data has been sent completely due to it taking multiple frames to send everything.
There's some other miscellaneous things, like large message service sending ship data to every player regardless if they already have the ship checked out, so by the time the 6th player has joined, the ship data is getting sent from the host to 6 players, even though only 1 needs it this becomes pretty detrimental, imagine a 6th player joining a world that has 10 ships, that would be a long time without state sync while the commands are being sent/received.
The solution for all of these has taken a while because they all touch on low-level systems that affect each other. You can't make AI selectively despawned or distributed authority on the island without first changing how the island data is saved for example.
The work is ongoing but I can share with you the work in progress and the plan for 1.0
Players now have a moving live query so they can inform the replication server which objects it cares about that it does not have authority over. (Such as other players) and their state will not be sent over the network if they are outside the query.
The more entities that get moved over to a distributed freeform model (rather than all things attached to an island having authority by a single client), the more benefit from this live query change. AI is largely working, but ships have the issue of large data within their save data so having them despawn and respawn regularly would use more bandwidth than we want, so we are actively working on optimizing ship data sizes at the moment to make this feasible (certain things like Heralds and sky whales will stay as global entities so you can see them from afar and are part of the skyscape. Bandwidth optimization will happen where it can at large distances)
I am currently moving all of our save-data from our large per-island data storage to a KVP database where each entity is its own entry. The idea of this is that all entities can be stored for every island as individual database entries, we can easily have 200,000 entities or more in a single world without any data issue and the best part is we don't need to update everything as the world save is not this gigantic blob of data. We can just update small batches of entities as we need.
The next video shows a simulation of a 50 km-sized world split into 750-meter cells and entities are loaded straight from the database, and saved when the cells are unpopulated by players. Entities are only simulated when they are in a populated cell, else authority is dropped and their state is written to the database on the fly.
Viewing and debugging world data is now much simpler thanks to tools like LiteDB Explorer:
For 1.0 the plan is for all world-data to be moved to a database model, and generalised into these tables :
World meta-data
Generic Entity Data (ore spawners, ai, death loot, buildables, critters, etc now no longer linked to an 'island')
Ships
Shipwrecks
Island-specific data (1 entry per island : Loot seed, removed trees, puzzles state, etc)
Boss Data
Once this is handled, the host will be responsible for all spawning based on cell queries into the database from all remote players, these will then be authority switched to the player that requested them. when they drop into unpopulated they will be abandoned, then at a regular interval, the host will loop over the abandoned entities and save their state to the database. This fixes many of the issues with players leaving before sending the save data they are authoritative over, it will also be more incremental and because of the database structure, any corruption in a single entity will not destroy the whole world.
Further more, I also plan to utilise the state sync within the replication server (rather than relying on remote players to send data of everything on an island back to the host when they exit the game or leave the island), as the replication server is aware of entities and the state it can use that to augment the saving and reduce bandwidth further. This will be applicable for 90% of entities, but a few have non state synced data that will still need to be sent (Inventories of storage and ship data for 2 examples) we either have to find a way to fit the data into state sync for these things, or still keep large messages for those. - Tom Thanks for reading. Expect a new Skywatch later this week.