Add support for block-until-commit responses

[eddyashton]

This has taken a few different forms as I tried to work out what the right API was. I think this is a neat approach, and roughly minimal for apps to implement - a new modifier when installing an endpoint that defers the response until commit, and a default callback that inserts an error response if the TxID is invalidated.

A handful of outstanding items, probably for future PRs but up-for-discussion:

• Better testing, including one of an invalidated (straddling an election) request.
• Support for a timeout response. If the terminal state (Committed or Invalid) is not reached for a long time, the caller may still want to know the assigned TxID, rather than simply getting a client HTTP timeout.
• Support for inserting receipts with the committed response. There's both semantic churn here - I want to lock down how any request works, separately from how some might wait further for a receipt - and some decisions about how we ensure fast-path retention of recent receipts.
• Variants of more default/built-in endpoints that are respond-on-commit.
• Naming.

[Copilot]

Pull request overview

This pull request adds support for endpoints that defer their HTTP responses until the transaction is globally committed (or invalidated) by the consensus protocol. The implementation introduces a commit callback subsystem, modifies the task system to use shared ownership for resumable tasks, and integrates these changes throughout the RPC stack.

• Introduces CommitCallbackSubsystem to manage callbacks triggered when transactions reach terminal consensus states
• Changes Resumable from unique_ptr to shared_ptr to support capturing in lambda callbacks
• Adds set_consensus_committed_function to endpoint configuration API allowing endpoints to defer responses

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/e2e_logging.py	Adds test comparing response timing between blocking and non-blocking endpoints
src/tasks/task_system.cpp	Updates function signature to accept Resumable by value instead of rvalue reference
src/tasks/resumable.h	Changes Resumable from unique_ptr to shared_ptr to enable shared ownership
src/node/rpc_context_impl.h	Adds optional respond_on_commit field to track deferred responses
src/node/commit_callback_subsystem.h	New subsystem managing callbacks for committed/invalidated transactions
src/node/commit_callback_interface.h	Interface definition for commit callback subsystem
src/http/http_session.h	Implements response deferral by pausing tasks and registering commit callbacks
src/endpoints/endpoint_registry.cpp	Adds default handler for commit responses that reports invalidated transactions
src/endpoints/endpoint.cpp	Implements setter for consensus_committed_func
src/node/node_state.h	Integrates commit_callbacks subsystem into node state
src/enclave/rpc_sessions.h	Passes commit callbacks subsystem to HTTP sessions
src/enclave/enclave.h	Creates and installs commit callback subsystem during initialization
src/consensus/aft/raft.h	Integrates callback execution when transactions commit
src/node/rpc/frontend.h	Sets respond_on_commit when endpoint has consensus_committed_func
samples/apps/logging/logging.cpp	Demonstrates blocking endpoints with example /log/blocking/private endpoints
include/ccf/tx_status.h	Adds FinalTxStatus enum containing only terminal transaction states
include/ccf/endpoint_registry.h	Declares default_respond_on_commit_func in public API
include/ccf/endpoint_context.h	Defines ConsensusCommittedEndpointFunction type
include/ccf/endpoint.h	Adds consensus_committed_func field and setter to Endpoint
CHANGELOG.md	Documents the new feature and adds formatting changes to older sections

Comments suppressed due to low confidence (5)

CHANGELOG.md:2160

• This adds an extra blank line that appears unintentional. This section already had proper formatting and this change doesn't improve readability.

  - The new `command.start.service_configuration.maximum_node_certificate_validity_days` (defaults to 365 days) configuration entry sets the maximum validity period allowed for node certificates.

CHANGELOG.md:1874

• This adds an extra blank line that appears unintentional. This section already had proper formatting and this change doesn't improve readability.

  - The new `node_certificate.initial_validity_days` (defaults to 1 day) configuration entry lets operators set the initial validity period for the node certificate (valid from the current system time).

CHANGELOG.md:2908

• This adds an extra blank line that appears unintentional. The surrounding sections don't use blank lines to separate items in this way.
  CHANGELOG.md:2913
• This adds an extra blank line that appears unintentional. The surrounding sections don't use blank lines to separate items in this way.

- Governance

CHANGELOG.md:2918

• This adds an extra blank line that appears unintentional. The surrounding sections don't use blank lines to separate items in this way.