Skip to content

S3 Directory Support#3304

Merged
jterapin merged 104 commits intoversion-3from
s3-directory-support
Mar 3, 2026
Merged

S3 Directory Support#3304
jterapin merged 104 commits intoversion-3from
s3-directory-support

Conversation

@jterapin
Copy link
Contributor

@jterapin jterapin commented Oct 13, 2025

Adds directory upload/download to Transfer Manager.

upload_directory

  • Upload all files from a local directory to S3
  • Optional recursive traversal of subdirectories
  • Symlink handling with circular reference detection
  • S3 key prefix support
  • Filter callback to selectively upload files
  • Request callback to modify upload parameters per file
  • Progress callback for transfer monitoring
  • Configurable failure handling (fail-fast or continue on error)

download_directory

  • Download all objects from an S3 bucket/prefix to a local directory
  • S3 prefix stripping for clean local paths
  • Path traversal detection
  • Filter callback to selectively download objects
  • Request callback to modify download parameters per object
  • Progress callback for transfer monitoring
  • Configurable failure handling (fail-fast or continue on error)
  • Automatic directory creation for nested structures

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

  1. To make sure we include your contribution in the release notes, please make sure to add description entry for your changes in the "unreleased changes" section of the CHANGELOG.md file (at corresponding gem). For the description entry, please make sure it lives in one line and starts with Feature or Issue in the correct format.

  2. For generated code changes, please checkout below instructions first:
    https://github.com/aws/aws-sdk-ruby/blob/version-3/CONTRIBUTING.md

Thank you for your contribution!

@github-actions
Copy link

github-actions bot commented Oct 13, 2025

Detected 1 possible performance regressions:

  • aws-sdk-s3.gem_size_kb - z-score regression: 549.5 -> 555.0. Z-score: Infinity

Copy link
Contributor

@alextwoods alextwoods left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - just minor comments.

@jterapin jterapin merged commit db9d80f into version-3 Mar 3, 2026
28 checks passed
@jterapin jterapin deleted the s3-directory-support branch March 3, 2026 18:18
jterapin added a commit that referenced this pull request Mar 3, 2026
@jterapin jterapin restored the s3-directory-support branch March 3, 2026 19:33
@jterapin jterapin deleted the s3-directory-support branch March 3, 2026 19:44
jterapin added a commit that referenced this pull request Mar 3, 2026
@jterapin jterapin mentioned this pull request Mar 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants