Remove dependency on tokio/rt and tokio::spawn(...)

[AadamZ5]

These changes aim to remove most calls to tokio::spawn(...) and other runtime-intertwined calls. These changes are to support runtime-agnosticism.

Heads Up: There are a lot of changes, and I acknowledge that may make this PR unsuitable to merge.

Motivation and Context

This will allow the project to be used in other environments where tokio/rt is not available or cannot be used. I originally raised this issue in #290, but some real demand for this particular change may be present in #379.

Ultimately, the goal is to remove dependency on the Tokio runtime. However, as I progress, I see a few repeated smaller sub-issues I am trying to tackle to accomplish the larger goal:

• Remove calls to tokio::spawn(...) (which mostly encompasses below)
• Remove async side effects (via tokio::spawn(...)) in Drop implementations
• Give control to the caller for how to run asynchronous background work
• Make child-process adaptable to different async implementations

After my initial changes, there is a lot of the following refactor:

let client = ().serve(transport).await?;
// use client...

turns into

let (client, work) = ().serve(transport).await?;
// now the library user must explicitly spawn the work loop for the service. 
tokio::spawn(work);
// use client...

How Has This Been Tested?

This refactor should be testable via the existing unit tests. As development continues, we can add more unit tests if needed. I also hope to perform real smoke-test, minimally using the already existing examples in this repo.

Breaking Changes

Yes, users will most likely need to update their code. This will very likely be a breaking change.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update

Checklist

• I have read the MCP Documentation
• My code follows the repository's style guidelines
• New and existing tests pass locally
• I have added appropriate error handling
• I have added or updated documentation as needed

Additional context

This is an ongoing draft. Please feel free to interject with comments or concerns.

Could eventually close #290

[AadamZ5]

Next note for myself: crates/rmcp/src/transport/child_process.rs will need a lot of consideration about how to refactor to be runtime-agnostic or at least adaptable. We could include feature-guarded implementations of a new ChildProcessRunner trait, perhaps wrapped in a common struct to ease usage.

Idea:

/// Underlying implementation and contract about what a command runner needs to provide
pub trait ChildProcessRunner {
   pub fn send_input(&self, input &[u8]) -> Result<...>;

   pub fn get_output_stream(&self) -> Result<PinnedStream<Vec<u8>>, ...>;
   pub fn get_exit_code(&self) -> Result<u8, ...>;
}

/// The `ChildProcess` struct would "control" the runner, and properly orchestrate methods calls (like
/// ensuring only one call to `get_output_stream(...)` etc)
///
/// This would simplify usage for the caller or library user
pub struct ChildProcess {
   inner: Box<dyn ChildProcessRunner>;
}

Then we could provide different implementations for the command runner, or provide a way for the library consumer to provide their own.

All of the ChildProcess usage might need to be behind a feature-flag.

Edit: I discovered the project is using the process_wrap crate, going to learn about that first before I make any major changes :)

[AadamZ5]

The new ChildProcess API seems to be functional. We can later write an async-std implementation now without changing too many concrete types 😀 I hope that refactor is not too disruptive from a user standpoint. I did try to update and run unit tests involving ChildProcess stuff. It could use a little more documentation about how to write one's own implementation for command running.

Still need to continue eliminating calls to tokio::spawn(...). There are plenty still, but a manageable amount! And fyi, I am not removing calls to tokio::spawn(...) in examples or tests. I think those are typically appropriate places to setup and use a runtime.

[AadamZ5]

Note for me:

Need to find all references to ServiceExt::serve(...) (in crates/rmcp/src/service.rs) and refactor into tuple unboxing and spawning work

[AadamZ5]

We need to remove calls to tokio::time::timeout(...) since it cannot work if it is not inside of a Tokio runtime.

There are a few portable candidates, but I will limit it to these two for now:

• futures-timeout: A very small and focused timeout library for futures. It spawns it's own thread to manage timers independently from any runtime. (wasm compatible)
• fastimer: Similar small library, focused on timing-out futures with it's own state managed in a separate thread.

I have personally used futures-timeout before, and enjoy it's simple timeout API, and it is also wasm-compatible. It also has slightly more all-time downloads. I will go with that.

[AadamZ5]

Okay, I think this is largely done. All async work loops are now returned at construction of services or clients. Timeouts have also been changed to futures-timeout.

Just merged in latest main into my dev branch and resolved any conflicts.

cargo test --all-features passes with no failures inside of the dev container 💪

I have not verified all examples yet, will try to do that on this weekend at the latest. They did receive a pass part-way through my refactoring, but not a final pass yet.