Skip to content

Fix keyset pagination with monotonic UUIDv7-like task IDs#1215

Merged
stephentoub merged 24 commits intomainfrom
copilot/fix-keyset-pagination-issue
Jan 30, 2026
Merged

Fix keyset pagination with monotonic UUIDv7-like task IDs#1215
stephentoub merged 24 commits intomainfrom
copilot/fix-keyset-pagination-issue

Conversation

Copy link
Contributor

Copilot AI commented Jan 30, 2026

Fixes a sporadic test failure in ListTasksAsync_KeysetPaginationMaintainsConsistencyWithNewTasks caused by tasks with identical timestamps having non-deterministic ordering when using random GUID-based task IDs.

Root Cause

The keyset pagination was using CreatedAt timestamp and TaskId (random GUID) for ordering. When multiple tasks had identical timestamps (due to low resolution of DateTimeOffset.UtcNow), the random GUIDs provided no guaranteed ordering, causing tasks created after a cursor was issued to potentially sort before the cursor boundary.

Solution

Replace random GUID task IDs with monotonic IDs composed of:

  • 64-bit timestamp (DateTimeOffset.UtcNow.Ticks)
  • 64-bit counter (globally monotonically increasing via Interlocked.Increment)

Formatted as a 32-character hex string: {ticks:x16}{counter:x16}

This guarantees strict monotonicity - every new task ID is greater than all previous IDs, regardless of timestamp resolution.

Changes Made

  • IdHelpers.CreateMonotonicId(DateTimeOffset timestamp): New helper method that generates monotonic IDs from timestamp ticks and a global counter
  • Simplified pagination cursor: Now uses only TaskId instead of CreatedAt|TaskId since the ID is inherently time-ordered
  • TimeProvider support for testing: Conditional compilation (#if MCP_TEST_TIME_PROVIDER) allows injecting FakeTimeProvider in tests without adding production dependencies
  • New tests:
    • ListTasksAsync_KeysetPaginationWorksWithIdenticalTimestamps - verifies pagination works when all tasks have identical timestamps
    • ListTasksAsync_TasksCreatedAfterFirstPageWithSameTimestampAppearInSecondPage - verifies tasks created after first page correctly appear in subsequent pages

Testing

  • All 52 InMemoryMcpTaskStore tests pass
  • Build succeeds for all target frameworks (net10.0, net9.0, net8.0, netstandard2.0)
Original prompt

This section details on the original issue you should resolve

<issue_title>ListTasksAsync_KeysetPaginationMaintainsConsistencyWithNewTasks failing sporadically in CI</issue_title>
<issue_description>```
[xUnit.net 00:00:25.03] ModelContextProtocol.Tests.Server.InMemoryMcpTaskStoreTests.ListTasksAsync_KeysetPaginationMaintainsConsistencyWithNewTasks [FAIL]
Failed ModelContextProtocol.Tests.Server.InMemoryMcpTaskStoreTests.ListTasksAsync_KeysetPaginationMaintainsConsistencyWithNewTasks [11 ms]
Error Message:
Assert.NotNull() Failure: Value is null
Stack Trace:
at ModelContextProtocol.Tests.Server.InMemoryMcpTaskStoreTests.<ListTasksAsync_KeysetPaginationMaintainsConsistencyWithNewTasks>d__41.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)


<agent_instructions>See if you can address the comment I made here: https://github.com/modelcontextprotocol/csharp-sdk/issues/1199#issuecomment-3822436859. We might want to issue a test that checks for this, although that might not be feasible without having the store accept a custom time provider.</agent_instructions>

## Comments on the Issue (you are @copilot in this section)

<comments>
<comment_new><author>@ericstj</author><body>
I hit this myself - I think this is suspect:
https://github.com/modelcontextprotocol/csharp-sdk/blob/7c85ac6706d9c22ba9b2e2229c785da2b62b7b8d/src/ModelContextProtocol.Core/Server/InMemoryMcpTaskStore.cs#L344

What if tasks have identical `CreatedAt` values, it will then compare the Ids, which are Guid strings and have no numerical relationship.  I think `DateTimeOffset.UtcNow` doesn't have a high-enough resolution to ensure tasks won't get the same `CreatedAt` value, then there is nothing to ensure a task added later (with same `CreatedAt`) might sort lower than a cursor boundary already handed out.
https://github.com/modelcontextprotocol/csharp-sdk/blob/7c85ac6706d9c22ba9b2e2229c785da2b62b7b8d/src/ModelContextProtocol.Core/Server/InMemoryMcpTaskStore.cs#L420

The test would fail in this case, if all of the last 5 tasks added after the first cursor boundary was established sorted below that boundary.

@eiriktsarpalis can you review this algorithm and see what you intended for this?  Did you mean to use something other than a GUID that might ensure monotonic increasing Task IDs?</body></comment_new>
<comment_new><author>@eiriktsarpalis</author><body>
This check was meant to be run after the sorting operation that currently happens in line 349. Pretty certain it used to do this, so wondering if copilot at some point reshuffled this part without me noticing.</body></comment_new>
</comments>


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: eiriktsarpalis <2813363+eiriktsarpalis@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix sporadic failure in ListTasksAsync_KeysetPagination test Fix keyset pagination with monotonically increasing task IDs Jan 30, 2026
Copilot AI requested a review from eiriktsarpalis January 30, 2026 08:33
Co-authored-by: eiriktsarpalis <2813363+eiriktsarpalis@users.noreply.github.com>
Co-authored-by: eiriktsarpalis <2813363+eiriktsarpalis@users.noreply.github.com>
Copy link
Member

@eiriktsarpalis eiriktsarpalis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot can you come up with a test that reliably checks if tasks with the same timestamp are handled correctly? You might want to look at using a custom time provider to achieve this.

Co-authored-by: eiriktsarpalis <2813363+eiriktsarpalis@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jan 30, 2026

@copilot can you come up with a test that reliably checks if tasks with the same timestamp are handled correctly? You might want to look at using a custom time provider to achieve this.

Added in 0fd7769. I added a TimeProvider parameter to InMemoryMcpTaskStore and created a test ListTasksAsync_KeysetPaginationWorksWithIdenticalTimestamps that uses FakeTimeProvider to create 10 tasks with exactly the same timestamp, then verifies that keyset pagination returns all tasks exactly once across two pages.

@eiriktsarpalis
Copy link
Member

@copilot Adding a TimeProvider NuGet dependency to improve testing isn't a workable solution. Let's revert that final commit. We can keep the new test by linking InMemoryTaskProvider.cs in the test project, and conditionally including code that accepts time provider only in testing.

… test-only TimeProvider support

Co-authored-by: eiriktsarpalis <2813363+eiriktsarpalis@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jan 30, 2026

@copilot Adding a TimeProvider NuGet dependency to improve testing isn't a workable solution. Let's revert that final commit. We can keep the new test by linking InMemoryTaskProvider.cs in the test pr...

Done in b8fa0d8. Reverted the TimeProvider NuGet dependency from production code. Now using conditional compilation (#if MCP_TEST_TIME_PROVIDER) in InMemoryMcpTaskStore.cs - the test project defines this constant, links the source file, and adds the Microsoft.Extensions.TimeProvider.Testing package, enabling the test with FakeTimeProvider without impacting the production assembly.

Copilot AI changed the title Fix keyset pagination with monotonically increasing task IDs Fix keyset pagination by sorting before cursor filter Jan 30, 2026
Co-authored-by: eiriktsarpalis <2813363+eiriktsarpalis@users.noreply.github.com>
…f UUIDv7-like format

Co-authored-by: eiriktsarpalis <2813363+eiriktsarpalis@users.noreply.github.com>
@eiriktsarpalis eiriktsarpalis marked this pull request as ready for review January 30, 2026 19:33
@eiriktsarpalis
Copy link
Member

@copilot please update the PR and description to accurately reflect the current state.

Copy link
Contributor Author

Copilot AI commented Jan 30, 2026

@copilot please update the PR and description to accurately reflect the current state.

Updated the PR description to reflect the current implementation: monotonic IDs using 64-bit timestamp ticks + 64-bit counter, formatted as a 32-character hex string.

stephentoub
stephentoub previously approved these changes Jan 30, 2026
Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
@stephentoub stephentoub enabled auto-merge (squash) January 30, 2026 22:09
@stephentoub stephentoub disabled auto-merge January 30, 2026 22:09
@stephentoub stephentoub merged commit e47f884 into main Jan 30, 2026
9 of 10 checks passed
@stephentoub stephentoub deleted the copilot/fix-keyset-pagination-issue branch January 30, 2026 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ListTasksAsync_KeysetPaginationMaintainsConsistencyWithNewTasks failing sporadically in CI

4 participants