A new way to change a Partition Key in CosmosDB

A Cosmos DB partition key is immutable and cannot be changed. This is not a drama, but it is crucial to remember.

Applications evolve over time, and a solution that was good enough initially may degrade. In the context of Cosmos DB, a wrong partition key can result in cross-partition queries or hot partitions (while others are idle). In turn, cross-partition queries lead to increased RU consumption, response time, and budget. These factors become even more painful if your database is extensive and highly loaded.

As I mentioned earlier, you cannot change the partition key online, but you can still create a new container and migrate data from the old one. I call this process a re-partition, and it consists of several steps.

1. Create a new container (target).
2. Choose a new partition key.
3. Write a console app or a one-time off Function that migrates the data.
4. Stop any write operations to the source container.
5. Migrate the data from source to destination.
6. Retarget consumers to the target container.

It is not a walk in the park, but it is not rocket science too.

A Game Changer?

The good news is that a recently announced Azure feature may significantly simplify steps #1, #3, and #5. The feature is called Changing the partition key of a container (in Azure Cosmos DB).

You no longer have to write migration code yourself. Instead, you kick off the migration process via the portal UI. Azure allocates a temporary virtual machine (4 CPU, 16 GB) and performs the task in the background. This not only saves developers time but also decreases the risk of a mistake. The less code we write, the better, isn’t it? Also, a new destination container gets created as a copy of a source container, thus inheriting all the settings besides a partition key.

I want to emphasize that the essential steps #2, #4, and #6 remain on us. We are still responsible for selecting the “right” partition key, so this feature is just a syntax sugar spilled in the cloud atmosphere. However, this sugar automates the routine, accelerating the team. So, I found it extremely useful.

Conclusion

The more honest name for this feature would be “Migrating a container with a partition key change in a semi-automated way” because that’s what it does. However, marketing science requires the title to be a bit clickbeity 😊.

Anyway, Changing the partition key in a container is already in preview (as of April 2024), and it is nice to be aware of such a capability.