Bottleneck Transformer Draft, Multimodal Fusion

[Rian354]

Summary

This PR introduces the BottleneckTransformer model to PyHealth, allowing for the efficient multimodal fusion via shared bottleneck tokens, based on the NeurIPS '21 paper Attention Bottlenecks for Multimodal Fusion.

Changes

• pyhealth/models/bottleneck_transformer.py:

  • Added MultimodalBottleneckTransformerEncoder to process independent modalities before fusing them at a specified fusion_startidx.
  • Added BottleneckTransformer which integrates directly with PyHealth's EmbeddingModel and dataset input_schema dynamically tracking modalities via feature_keys.

• pyhealth/models/__init__.py:

  • Exported the new model components.

• tests/core/test_bottleneck_transformer.py:

  • Comprehensive unit tests covering instantiation, forward/backward passes, and dimension matching.

• docs/api/models/pyhealth.models.BottleneckTransformer.rst & docs/api/models.rst:

  • Added full API documentation for BottleneckTransformer maintaining uniformity with existing model documentation.

• examples/bottleneck_transformer_tutorial.ipynb:

  • Provided a tutorial notebook demonstrating end to end multimodal predictions using the new model.