Outdated Ships Bug Explanation

As requested, here's an in-depth explanation of what caused the AI in Airships: Conquer the Skies to build outdated ships. I've tried my best to keep it relatively non-technical.

So in the game, the AI builds airships (and landships and buildings), and it researches technologies. Of course, it should use these new technologies in the construction of better ships. But a lot of the time, this was simply not happening, with the AI still deploying muskets and grenades late into the game.

After multiple tries, I finally figured out what was causing this bug, and also why it was so hard to diagnose.

First, some context: compared to other real-time strategy games, units in A:CtS are much more complicated. Instead of a single entity with one HP pool, an airship consists of dozens or hundreds of modules and tiles, all with their own stats. This is what makes the game fun, but it also causes performance problems.

This means that when the AI wants to build a ship, it's expensive to check every ship to see if it can be built. So instead, the AI keeps a list of ships that are buildable, and a list of techs it had when it made that list.

Whenever the AI wants to build a ship, it checks whether its list of ship techs is the same as the list of techs it's researched. If they're not the same, it's developed some new technology, so it re-checks all the ships and makes a new list of ships to build, and updates its list of ship techs with its current technologies.

So, step by step:

Ships List          Ship Tech List      Current Tech List
│                   │                   │
HMS Murderface      Grenades            Grenades
HMS Stabber         Wooden Armour       Wooden Armour
HMS Not NiceNew tech researched and added to Current Tech List.Ships List          Ship Tech List      Current Tech List
│                   │                   │
HMS Murderface      Grenades            Grenades
HMS Stabber         Wooden Armour       Wooden Armour
HMS Not Nice                            CannonsShip Tech List and Current Tech List are different, so the AI updates the Ships List.Ships List          Ship Tech List      Current Tech List
│                   │                   │
HMS Murderface II   Grenades            Grenades
HMS Stabber         Wooden Armour       Wooden Armour
HMS Katana                              CannonsShip Tech List is updated to be the same as Current Tech List.Ships List          Ship Tech List      Current Tech List
│                   │                   │
HMS Murderface II   Grenades            Grenades
HMS Stabber         Wooden Armour       Wooden Armour
HMS Katana          Cannons             Cannons

That last step is where the bug was: It should have made a copy of its current technology list and filed it away as the ship technology list. Instead, it just used the same list, the "live" one. So now, whenever it researched a new technology, it would be instantly in the ship tech list as well. Meaning that whenever it checked, the two lists would be the same, and so it would never notice technology had changed, and would never rebuild its ship list.

Bug: Ship Tech List is the same as Current Tech List.Ships List          Ship Tech List      Current Tech List
│                   └───────────────────┤
HMS Murderface                          Grenades
HMS Stabber                             Wooden Armour
HMS Not NiceNew tech researched and added to the tech list.Ships List          Ship Tech List      Current Tech List
│                   └───────────────────┤
HMS Murderface                          Grenades
HMS Stabber                             Wooden Armour
HMS Not Nice                            CannonsShip Tech List equals Current Tech List, and so Ships List is not updated.

And so, the ship list stayed the same, with the same old ship designs, and the AI did not actually benefit from the research it was doing.

The fix was very simple: just make a copy of the tech list rather than use it directly. But what caused the bug in the first place? An optimization pass I did three years ago. These tech lists are used all over the place in the game, as they need to be consulted frequently to check if something is available. I discovered that the game spent a lot of time making pointless copies of them, and so I got rid of all that copying - including in this place, where there was a very good reason for it!

So why did this take so long to track down? You might have noticed that there were multiple previous updates where I claimed to have fixed this bug. Well, to be able to fix a bug, you have to first observe it in the wild. There's a bunch of confounding factors: perhaps the AI has outdated ships because it doesn't have the money to upgrade them, or they haven't made it to a shipyard because they've been too busy fighting.

And there's a lot of moving parts to ship management: I long thought the code for upgrading, rather than building new ships, was broken. So I asked players to send me saves of games where the AI had failed to build modern ships.

Here's the thing: the game doesn't store the ship list in the save file. It doesn't have to, because it can recreate the list, right? So when you loaded a save game with this bug, the bug would go away as the ship list would be freshly recreated and up to date. So I'd inspect the game and see correct ship lists, and hence assume the problem was with using the lists, not the lists themselves. The bug took so long to find because it was covering its tracks.

So here you have it: optimization always carries the risk of introducing new bugs, and sometimes bugs hide from you.