I'm a really curious person and ever since I got into programming and software engineering it's got worse.
When Microsoft announced the Bulk Executor library for CosmosDB, I instantly got suspicious. If it could deliver what they promise then it would be a great addition for Cosmonaut. I really wanted to see how a single operation can perform better in both performance and cost compared to many individual requests (which was the only way to do something like that until now).
Sadly, the Nuget package is targeting .NET Framework 4.5.1 so that's not gonna work for me. That's fine, let's check the code and potentially adapt it for .NET Standard. Surprise! The code is not published yet.
Time for Plan B.
What Microsoft says
When a bulk operation to import or update documents is triggered with a batch of entities, they are initially shuffled into buckets corresponding to their Azure Cosmos DB partition key range. Within each bucket that corresponds to a partition key range, they are broken down into mini-batches and each mini-batch act as a payload that is committed on the server-side. The BulkExecutor library has built in optimizations for concurrent execution of these mini-batches both within and across partition key ranges. Following image illustrates how BulkExecutory batches data into different partition keys.
Under the hood
It's always stored procedures, Nick Chapsas 2018
So as you probably know, CosmosDB supports user-defined stored procedures. It turns out that's just the tip of the iceberg. CosmosDB also supports several system-defined stored procedures.
That's exactly how the Bulk Executor operates, by using three system defined stored procedures.
Let's look past the naming inconsistency between these stored procedures.
What's gonna happen if you use any of the respective methods for these operations.
First and foremost, during object creation the Bulk Executor will add
.NET/BulkExecutorLibrary/1.0.1 at the
UserAgentSuffix of the connection policy. That's probably used for something internally that we can't possibly know. I wonder however if that affects the cost of the operation or it's just used for Microsoft to see how much the library is used.
Then it will fetch the partition map of the specified collection in order to calculate the partition key range ids and the partition degree of concurrency. These two will happen regardless the operation that will be executed.
During import the documents provided will be sorted into partition buckets. A page size will be calculated based on the documents size and that will be used page after page in a loop.
After that the system will also create
mini batches. A loop will go through the documents to be imported by partition and the partition range id will be used to get the list of documents to import by partition. Each document will have it's byte count calculated and if its size is more than the ideal mini batch size an information about that will be traced (no idea why, hmm).
Once these two things are done and we have some soft of list populated, the executor will create a
BatchUpdater and a
CongestionController per partition key range id.
BatchInserter appears to be what will run the stored procedures and will do all the response handling if something goes right or wrong with them. The
CongestionController on the other hand will handle all the concurrency logic, throttling and aggregate all the metrics that will be returned at the end for the operation. It is what ultimately will trigger the
BatchUpdater logic and it will add all the extra congestion logic on top. It is essentially the AIMD-style congestion control mechanism that they said the library has.
This is pretty straightforward. No minibatches needed. Just one
BatchDeleter per partition key range id and wait all delete stored procedures to be executed.
So, I could be wrong, but what the Bulk Executor really seems to be is a library that will use 3
system-defined stored procedures that have potentially special cost around them. They will be split in very thin slices and then executed asynchronously and parallel-ly (if that's even a real word) while also handling internally all the congestion that comes with parallel execution.
It's pretty neat and I can't wait until they release the .NET Standard version.
Find more (well it's actually less but whatever) about the Azure Cosmos DB BulkExecutor here