Fix keyset pagination with monotonic UUIDv7-like task IDs#1215
Fix keyset pagination with monotonic UUIDv7-like task IDs#1215stephentoub merged 24 commits intomainfrom
Conversation
Co-authored-by: eiriktsarpalis <2813363+eiriktsarpalis@users.noreply.github.com>
Co-authored-by: eiriktsarpalis <2813363+eiriktsarpalis@users.noreply.github.com>
Co-authored-by: eiriktsarpalis <2813363+eiriktsarpalis@users.noreply.github.com>
eiriktsarpalis
left a comment
There was a problem hiding this comment.
@copilot can you come up with a test that reliably checks if tasks with the same timestamp are handled correctly? You might want to look at using a custom time provider to achieve this.
Co-authored-by: eiriktsarpalis <2813363+eiriktsarpalis@users.noreply.github.com>
Added in 0fd7769. I added a |
|
@copilot Adding a TimeProvider NuGet dependency to improve testing isn't a workable solution. Let's revert that final commit. We can keep the new test by linking InMemoryTaskProvider.cs in the test project, and conditionally including code that accepts time provider only in testing. |
… test-only TimeProvider support Co-authored-by: eiriktsarpalis <2813363+eiriktsarpalis@users.noreply.github.com>
Done in b8fa0d8. Reverted the TimeProvider NuGet dependency from production code. Now using conditional compilation ( |
Co-authored-by: eiriktsarpalis <2813363+eiriktsarpalis@users.noreply.github.com>
…f UUIDv7-like format Co-authored-by: eiriktsarpalis <2813363+eiriktsarpalis@users.noreply.github.com>
|
@copilot please update the PR and description to accurately reflect the current state. |
Updated the PR description to reflect the current implementation: monotonic IDs using 64-bit timestamp ticks + 64-bit counter, formatted as a 32-character hex string. |
Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
Fixes a sporadic test failure in
ListTasksAsync_KeysetPaginationMaintainsConsistencyWithNewTaskscaused by tasks with identical timestamps having non-deterministic ordering when using random GUID-based task IDs.Root Cause
The keyset pagination was using
CreatedAttimestamp andTaskId(random GUID) for ordering. When multiple tasks had identical timestamps (due to low resolution ofDateTimeOffset.UtcNow), the random GUIDs provided no guaranteed ordering, causing tasks created after a cursor was issued to potentially sort before the cursor boundary.Solution
Replace random GUID task IDs with monotonic IDs composed of:
DateTimeOffset.UtcNow.Ticks)Interlocked.Increment)Formatted as a 32-character hex string:
{ticks:x16}{counter:x16}This guarantees strict monotonicity - every new task ID is greater than all previous IDs, regardless of timestamp resolution.
Changes Made
IdHelpers.CreateMonotonicId(DateTimeOffset timestamp): New helper method that generates monotonic IDs from timestamp ticks and a global counterTaskIdinstead ofCreatedAt|TaskIdsince the ID is inherently time-ordered#if MCP_TEST_TIME_PROVIDER) allows injectingFakeTimeProviderin tests without adding production dependenciesListTasksAsync_KeysetPaginationWorksWithIdenticalTimestamps- verifies pagination works when all tasks have identical timestampsListTasksAsync_TasksCreatedAfterFirstPageWithSameTimestampAppearInSecondPage- verifies tasks created after first page correctly appear in subsequent pagesTesting
Original prompt
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.