config_wrangler.config_templates.aws.s3_bucket module

class config_wrangler.config_templates.aws.s3_bucket.OverwriteModes(*values)

Bases: Enum

Enumeration of file overwrite behavior modes for S3 operations.

ALWAYS_OVERWRITE

Always overwrite the target file regardless of timestamps or sizes.

Type:

auto

OVERWRITE_OLDER

Only overwrite if source is newer or sizes differ.

Type:

auto

NEVER_OVERWRITE

Never overwrite existing files.

Type:

auto

ALWAYS_OVERWRITE = 1

NEVER_OVERWRITE = 3

OVERWRITE_OLDER = 2

exception config_wrangler.config_templates.aws.s3_bucket.S3ClientError(message: str, original_error: ClientError | None)

Bases: ClientError

Custom exception for S3 client errors with enhanced error context.

Wraps boto3 ClientError exceptions with additional context while preserving the original error response and operation name.

Parameters:

• message (str) – Descriptive error message.

• original_error (Optional[ClientError]) – Original boto3 ClientError, if available.

__init__(message: str, original_error: ClientError | None)

pydantic model config_wrangler.config_templates.aws.s3_bucket.S3_Bucket

Bases: AWS_Session

Provides pathlib-like interface for AWS S3 bucket operations.

Extends AWS_Session to provide file-system-like operations for S3 buckets, including upload, download, listing, and manipulation of S3 objects.

bucket_name

The name of the S3 bucket.

Type:

str

key

The S3 object key (path within bucket). Default is None.

Type:

str | None, optional

treat_as_folder

Whether to treat the key as a folder prefix. Default is False.

Type:

bool, optional

OverwriteModes

Reference to OverwriteModes enum for controlling overwrite behavior.

Type:

ClassVar[type[OverwriteModes]]

Examples

>>> bucket = S3_Bucket(bucket_name='my-bucket')
>>> bucket.exists('path/to/file.txt')
True
>>> file_obj = bucket / 'path/to/file.txt'
>>> file_obj.download_file(local_filename='local_file.txt')

Config:

• validate_default: bool = True

• validate_assignment: bool = True

• validate_credentials: bool = True

Fields:

• bucket_name (str)

• iam_role ()

• keepass ()

• keepass_config ()

• keepass_group ()

• keepass_title ()

• key (str | None)

• keyring_section ()

• password_source ()

• raw_password ()

• region_name ()

• treat_as_folder (bool)

• user_id ()

• validate_password_on_load ()

Validators:

• check_model » all fields

class CompareResult(*values)

Bases: Enum

Enumeration of file comparison results between S3 and local files.

DIFFERENT_SIZE

Files have different sizes.

Type:

auto

LOCAL_NEWER

Local file has a newer modification timestamp.

Type:

auto

SAME_TIMES

Files have identical modification timestamps.

Type:

auto

LOCAL_OLDER

Local file has an older modification timestamp.

Type:

auto

DIFFERENT_SIZE = 1

LOCAL_NEWER = 2

LOCAL_OLDER = 4

SAME_TIMES = 3

class OverwriteModes(*values)

Bases: Enum

Enumeration of file overwrite behavior modes for S3 operations.

ALWAYS_OVERWRITE

Always overwrite the target file regardless of timestamps or sizes.

Type:

auto

OVERWRITE_OLDER

Only overwrite if source is newer or sizes differ.

Type:

auto

NEVER_OVERWRITE

Never overwrite existing files.

Type:

auto

ALWAYS_OVERWRITE = 1

NEVER_OVERWRITE = 3

OVERWRITE_OLDER = 2

add_child(name: str, child_object: ConfigHierarchy)

Set this configuration as a child in the hierarchy of another config. For any programmatically created config objects this is required so that the new object ‘knows’ where it lives in the hierarchy – most importantly so that it can find the hierarchies root object.

as_bucket() → S3_Bucket

Create a new S3_Bucket instance with only the bucket name.

Returns:

S3_Bucket instance without key, folder, or file_name attributes.

Return type:

S3_Bucket

assume_role()

validator check_model  »  all fields

content_type() → str

Get the content type (MIME type) of the S3 object.

Returns:

The content type string (e.g., ‘text/plain’, ‘application/json’).

Return type:

str

copy_to(target: str | S3_Bucket_Key, version_id: str | None = None)

Copy this S3 object to another S3 location.

Parameters:

• target (str | S3_Bucket_Key) – Target S3 location (URI string or S3_Bucket_Key instance).

• version_id (str | None, optional) – Specific version ID to copy. If None, copies the latest version.

delete(key: str | PurePosixPath | None = None, version_id: str | None = None)

Delete an S3 object.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key to delete. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID to delete. If None, deletes the latest version.

delete_by_key(key: str | PurePosixPath, version_id: str | None = None)

Delete an S3 object by key.

Deprecated since version Use: delete() instead.

Parameters:

• key (str | PurePosixPath) – S3 key to delete.

• version_id (str | None, optional) – Specific version ID to delete.

download_file(*, local_filename: str | Path, key: str | PurePosixPath | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, create_parents: bool = True, overwrite_mode: OverwriteModes = OverwriteModes.OVERWRITE_OLDER)

Download an S3 object to a local file.

Parameters:

• local_filename (Path | str) – Path where the downloaded file will be saved.

• key (str | PurePosixPath | None, optional) – S3 key to download. If None, uses the instance key.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the download.

• create_parents (bool, optional) – Whether to create parent directories. Default is True.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is OVERWRITE_OLDER.

Raises:

• S3ClientError – If the S3 object does not exist or download fails.

• ValueError – If a non-blank key is not specified for the download.

download_files(*, local_path: str | Path, key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, create_parents: bool = True, overwrite_mode: OverwriteModes = OverwriteModes.OVERWRITE_OLDER) → Iterable[Path]

Download multiple S3 objects under a key prefix to a local directory.

Parameters:

• local_path (str | Path) – Local directory path where files will be downloaded.

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder prefix. If None, uses instance setting.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for downloads.

• create_parents (bool, optional) – Whether to create parent directories. Default is True.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is OVERWRITE_OLDER.

Returns:

List of local file paths that were downloaded.

Return type:

Iterable[Path]

exists(key: str | PurePosixPath | None = None, version_id: str | None = None) → bool

Check if an S3 object exists.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key to check. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID to check. If None, checks the latest version.

Returns:

True if the object exists, False otherwise.

Return type:

bool

Raises:

ClientError – If there’s an error other than 404/NoSuchKey.

find_objects(key: str | PurePosixPath | None = None) → BucketObjectsCollection

List S3 objects under a key prefix.

Deprecated since version Use: list_objects() instead.

Parameters:

key (str | PurePosixPath | None, optional) – S3 key prefix.

Returns:

Collection of S3 ObjectSummary objects.

Return type:

BucketObjectsCollection

full_item_name(item_name: str | None = None, delimiter: str = ' -> ')

The fully qualified name of this config item in the config hierarchy.

get_boto3_bucket() → Bucket

Get the boto3 Bucket resource object.

Returns:

The boto3 Bucket resource.

Return type:

Bucket

Raises:

ClientError – If the bucket does not exist (error code NoSuchBucket).

get_bucket_region() → str

Get the region_name from the actual S3 bucket definition.

Note

This can differ from the region_name attribute specified when constructing this class for example via a config file. That region_name attribute is used for establishing the AWS session.

NOTE 2:

If you don’t specify a Region, when the bucket is created it will have used the US East (N. Virginia) Region (us-east-1). So this method defaults to that if it does not find a LocationConstraint.

get_bucket_region_name() → str

Get the AWS region name where the bucket is located.

Returns:

AWS region name (e.g., ‘us-east-1’, ‘us-west-2’).

Return type:

str

Notes

Buckets in Region us-east-1 have a LocationConstraint of null.

get_copy(copied_by: str = 'get_copy') → AWS_Session

Copy this configuration. Useful when you need to programmatically modify a configuration without modifying the original base configuration.

get_object(key: str | PurePosixPath | None = None, version_id: str | None = None) → Object | ObjectVersion

Get a boto3 Object or ObjectVersion resource with caching.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID. If None, returns the latest version Object.

Returns:

The boto3 Object or ObjectVersion resource.

Return type:

Object | ObjectVersion

get_object_uncached(key: str | PurePosixPath | None = None, version_id: str | None = None) → Object | ObjectVersion

Get a boto3 Object or ObjectVersion resource without caching.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID. If None, returns the latest version Object.

Returns:

The boto3 Object or ObjectVersion resource.

Return type:

Object | ObjectVersion

get_password() → str

Get the password for this resource. password_source controls where it looks for the password. If that is None, then the root level passwords container is checked for password_source value.

get_request_v4_authorizer(service: str = None) → requests.auth.AuthBase

get_s3_bucket_key(key: str | PurePosixPath | None = None) → S3_Bucket_Key

Get an S3_Bucket_Key instance for the specified key.

Parameters:

key (str | PurePosixPath | None, optional) – S3 key. If None, uses the instance key.

Returns:

S3_Bucket_Key instance representing the key.

Return type:

S3_Bucket_Key

get_secrets_manager()

get_service_client(service: str)

get_service_resource(service: str)

get_session_expiry()

get_ssm()

is_file()

Check if this S3 object represents a file (not a directory marker).

Returns:

True if it’s a file, False if it’s a directory marker or doesn’t exist.

Return type:

bool

Notes

S3 doesn’t have true directories, but some tools create directory markers with content type ‘application/x-directory’.

is_relative_to(other: S3_Bucket)

Check if this S3 path is relative to another.

Parameters:

other (S3_Bucket) – Another S3_Bucket instance to compare with.

Returns:

True if this path is relative to the other path, False otherwise.

Return type:

bool

iterdir() → Iterable[S3_Bucket_Key]

Return the S3_Bucket_Key objects contained in the in/under this object.

joinpath(*others) → S3_Bucket_Key | S3_Bucket_Folder

Join multiple path components.

Parameters:

*others – Path components to join.

Returns:

New S3 path object with all components joined.

Return type:

S3_Bucket_Key | S3_Bucket_Folder

key_exists(key: str | PurePosixPath) → bool

Check if an S3 object exists at the specified key.

Deprecated since version Use: exists() instead.

Parameters:

key (str | PurePosixPath) – S3 key to check.

Returns:

True if the object exists, False otherwise.

Return type:

bool

list_object_keys(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → List[str]

List S3 object keys under a key prefix.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder. If None, uses instance setting.

Returns:

List of S3 object keys.

Return type:

List[str]

list_object_paths(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → List[PurePosixPath]

Return the relative paths of objects contained in/under this object.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder. If None, uses instance setting.

Returns:

List of relative paths (PurePosixPath) of objects under the key prefix.

Return type:

List[PurePosixPath]

list_objects(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → BucketObjectsCollection

List S3 objects under a key prefix.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder by appending ‘/’. If None, uses instance setting.

Returns:

Collection of S3 ObjectSummary objects matching the prefix.

Return type:

BucketObjectsCollection

Raises:

S3ClientError – If the bucket or key doesn’t exist, or other errors occur.

model_dump_non_private(*, mode: Literal['json', 'python'] | str = 'python', exclude: Set[str] | None = None) → dict[str, Any]

nav_to_bucket(bucket_name) → S3_Bucket

nav_to_s3_link(s3_uri: str) → S3_Bucket_Key

open(mode: str = 'r', encoding: str | None = None, errors: str | None = None) → IOBase

Open the S3 object as a file-like object for reading.

Parameters:

• mode (str, optional) – File mode (‘r’ for text, ‘rb’ for binary). Default is ‘r’.

• encoding (str | None, optional) – Text encoding (for text mode only).

• errors (str | None, optional) – Error handling strategy for encoding (for text mode only).

Returns:

File-like object (TextIOWrapper for text mode, BufferedReader for binary).

Return type:

io.IOBase

Raises:

ValueError – If mode is empty or doesn’t start with ‘r’ (only read mode is supported).

read_bytes() → bytes

Read the S3 object contents as bytes.

Returns:

The complete file contents as bytes.

Return type:

bytes

read_text(encoding='utf-8', errors=None) → str

Read the S3 object contents as text.

Parameters:

• encoding (str, optional) – Text encoding. Default is ‘utf-8’.

• errors (str | None, optional) – Error handling strategy for encoding.

Returns:

The complete file contents as a string.

Return type:

str

rename(target: str)

Rename (move) this S3 object to a new key.

Parameters:

target (str) – Target S3 key or URI.

Notes

This copies the object to the new location then deletes the original.

set_as_child(name: str, other_config_item: ConfigHierarchy)

set_session(session: Session)

static split_s3_uri(s3_uri: str) → Tuple[str, str]

static translate_config_data(config_data: MutableMapping)

Children classes can provide translation logic to allow older config files to be used with newer config class definitions.

unlink(key: str | PurePosixPath | None = None, version_id: str | None = None, missing_ok=False)

Delete an S3 object (pathlib-style interface).

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key to delete. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID to delete. If None, deletes the latest version.

• missing_ok (bool, optional) – If True, don’t raise an error if the object doesn’t exist. Default is False.

Raises:

ClientError – If the object doesn’t exist and missing_ok is False, or other errors occur.

upload_file(*, local_filename: str | Path, key: str | PurePosixPath | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, overwrite_mode: OverwriteModes = OverwriteModes.ALWAYS_OVERWRITE)

Upload a local file to S3.

Parameters:

• local_filename (str | Path) – The local file to upload.

• key (str | PurePosixPath | None, optional) – S3 key to upload to. If None, uses the instance key.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 upload_file (e.g., metadata, ACL).

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the upload. Defaults to 5GB multipart threshold.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is ALWAYS_OVERWRITE.

with_name(name: str)

Return a new S3_Bucket_Key with the file name changed.

Parameters:

name (str) – The new file name.

Returns:

New S3_Bucket_Key with the modified name.

Return type:

S3_Bucket_Key

with_stem(stem: str)

Return a new S3_Bucket_Key with the file stem changed.

Parameters:

stem (str) – The new file stem (name without extension).

Returns:

New S3_Bucket_Key with the modified stem.

Return type:

S3_Bucket_Key

with_suffix(suffix: str)

Return a new S3_Bucket_Key with the file suffix changed.

Parameters:

suffix (str) – The new file extension (e.g., ‘.txt’).

Returns:

New S3_Bucket_Key with the modified suffix.

Return type:

S3_Bucket_Key

pydantic model config_wrangler.config_templates.aws.s3_bucket.S3_Bucket_Folder

Bases: S3_Bucket

Represents a folder within an S3 bucket.

folder

The folder path within the bucket.

Type:

str

treat_as_folder

Always True for this class.

Type:

bool

Config:

• validate_default: bool = True

• validate_assignment: bool = True

• validate_credentials: bool = True

Fields:

• bucket_name ()

• folder (str)

• iam_role ()

• keepass ()

• keepass_config ()

• keepass_group ()

• keepass_title ()

• key ()

• keyring_section ()

• password_source ()

• raw_password ()

• region_name ()

• treat_as_folder (bool)

• user_id ()

• validate_password_on_load ()

Validators:

• check_model » all fields

• validate_folder » folder

class CompareResult(*values)

Bases: Enum

Enumeration of file comparison results between S3 and local files.

DIFFERENT_SIZE

Files have different sizes.

Type:

auto

LOCAL_NEWER

Local file has a newer modification timestamp.

Type:

auto

SAME_TIMES

Files have identical modification timestamps.

Type:

auto

LOCAL_OLDER

Local file has an older modification timestamp.

Type:

auto

DIFFERENT_SIZE = 1

LOCAL_NEWER = 2

LOCAL_OLDER = 4

SAME_TIMES = 3

class OverwriteModes(*values)

Bases: Enum

Enumeration of file overwrite behavior modes for S3 operations.

ALWAYS_OVERWRITE

Always overwrite the target file regardless of timestamps or sizes.

Type:

auto

OVERWRITE_OLDER

Only overwrite if source is newer or sizes differ.

Type:

auto

NEVER_OVERWRITE

Never overwrite existing files.

Type:

auto

ALWAYS_OVERWRITE = 1

NEVER_OVERWRITE = 3

OVERWRITE_OLDER = 2

add_child(name: str, child_object: ConfigHierarchy)

Set this configuration as a child in the hierarchy of another config. For any programmatically created config objects this is required so that the new object ‘knows’ where it lives in the hierarchy – most importantly so that it can find the hierarchies root object.

as_bucket() → S3_Bucket

Create a new S3_Bucket instance with only the bucket name.

Returns:

S3_Bucket instance without key, folder, or file_name attributes.

Return type:

S3_Bucket

assume_role()

validator check_model  »  all fields

content_type() → str

Get the content type (MIME type) of the S3 object.

Returns:

The content type string (e.g., ‘text/plain’, ‘application/json’).

Return type:

str

copy_to(target: str | S3_Bucket_Key, version_id: str | None = None)

Copy this S3 object to another S3 location.

Parameters:

• target (str | S3_Bucket_Key) – Target S3 location (URI string or S3_Bucket_Key instance).

• version_id (str | None, optional) – Specific version ID to copy. If None, copies the latest version.

delete(key: str | PurePosixPath | None = None, version_id: str | None = None)

Delete an S3 object.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key to delete. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID to delete. If None, deletes the latest version.

delete_by_key(key: str | PurePosixPath, version_id: str | None = None)

Delete an S3 object by key.

Deprecated since version Use: delete() instead.

Parameters:

• key (str | PurePosixPath) – S3 key to delete.

• version_id (str | None, optional) – Specific version ID to delete.

download_file(*, local_filename: str | Path, key: str | PurePosixPath | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, create_parents: bool = True, overwrite_mode: OverwriteModes = OverwriteModes.OVERWRITE_OLDER)

Download an S3 object to a local file.

Parameters:

• local_filename (Path | str) – Path where the downloaded file will be saved.

• key (str | PurePosixPath | None, optional) – S3 key to download. If None, uses the instance key.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the download.

• create_parents (bool, optional) – Whether to create parent directories. Default is True.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is OVERWRITE_OLDER.

Raises:

• S3ClientError – If the S3 object does not exist or download fails.

• ValueError – If a non-blank key is not specified for the download.

download_files(*, local_path: str | Path, key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, create_parents: bool = True, overwrite_mode: OverwriteModes = OverwriteModes.OVERWRITE_OLDER) → Iterable[Path]

Download multiple S3 objects under a key prefix to a local directory.

Parameters:

• local_path (str | Path) – Local directory path where files will be downloaded.

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder prefix. If None, uses instance setting.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for downloads.

• create_parents (bool, optional) – Whether to create parent directories. Default is True.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is OVERWRITE_OLDER.

Returns:

List of local file paths that were downloaded.

Return type:

Iterable[Path]

download_folder_file(key_suffix: str | PurePosixPath, local_filename: str | Path, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, create_parents: bool = True)

Download a file from a key within this folder.

Deprecated since version Use: (my_folder / key_suffix).download_file() instead.

Parameters:

• key_suffix (str | PurePosixPath) – Key suffix to add after the folder name.

• local_filename (str | Path) – Path where the downloaded file will be saved.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the download.

• create_parents (bool, optional) – Whether to create parent directories. Default is True.

exists(key: str | PurePosixPath | None = None, version_id: str | None = None) → bool

Check if an S3 object exists.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key to check. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID to check. If None, checks the latest version.

Returns:

True if the object exists, False otherwise.

Return type:

bool

Raises:

ClientError – If there’s an error other than 404/NoSuchKey.

find_objects(key: str | PurePosixPath | None = None) → BucketObjectsCollection

List S3 objects under a key prefix.

Deprecated since version Use: list_objects() instead.

Parameters:

key (str | PurePosixPath | None, optional) – S3 key prefix.

Returns:

Collection of S3 ObjectSummary objects.

Return type:

BucketObjectsCollection

full_item_name(item_name: str | None = None, delimiter: str = ' -> ')

The fully qualified name of this config item in the config hierarchy.

get_boto3_bucket() → Bucket

Get the boto3 Bucket resource object.

Returns:

The boto3 Bucket resource.

Return type:

Bucket

Raises:

ClientError – If the bucket does not exist (error code NoSuchBucket).

get_bucket_region() → str

Get the region_name from the actual S3 bucket definition.

Note

This can differ from the region_name attribute specified when constructing this class for example via a config file. That region_name attribute is used for establishing the AWS session.

NOTE 2:

If you don’t specify a Region, when the bucket is created it will have used the US East (N. Virginia) Region (us-east-1). So this method defaults to that if it does not find a LocationConstraint.

get_bucket_region_name() → str

Get the AWS region name where the bucket is located.

Returns:

AWS region name (e.g., ‘us-east-1’, ‘us-west-2’).

Return type:

str

Notes

Buckets in Region us-east-1 have a LocationConstraint of null.

get_copy(copied_by: str = 'get_copy') → AWS_Session

Copy this configuration. Useful when you need to programmatically modify a configuration without modifying the original base configuration.

get_object(key: str | PurePosixPath | None = None, version_id: str | None = None) → Object | ObjectVersion

Get a boto3 Object or ObjectVersion resource with caching.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID. If None, returns the latest version Object.

Returns:

The boto3 Object or ObjectVersion resource.

Return type:

Object | ObjectVersion

get_object_uncached(key: str | PurePosixPath | None = None, version_id: str | None = None) → Object | ObjectVersion

Get a boto3 Object or ObjectVersion resource without caching.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID. If None, returns the latest version Object.

Returns:

The boto3 Object or ObjectVersion resource.

Return type:

Object | ObjectVersion

get_password() → str

Get the password for this resource. password_source controls where it looks for the password. If that is None, then the root level passwords container is checked for password_source value.

get_request_v4_authorizer(service: str = None) → requests.auth.AuthBase

get_s3_bucket_key(key: str | PurePosixPath | None = None) → S3_Bucket_Key

Get an S3_Bucket_Key instance for the specified key.

Parameters:

key (str | PurePosixPath | None, optional) – S3 key. If None, uses the instance key.

Returns:

S3_Bucket_Key instance representing the key.

Return type:

S3_Bucket_Key

get_secrets_manager()

get_service_client(service: str)

get_service_resource(service: str)

get_session_expiry()

get_ssm()

is_file()

Check if this S3 object represents a file (not a directory marker).

Returns:

True if it’s a file, False if it’s a directory marker or doesn’t exist.

Return type:

bool

Notes

S3 doesn’t have true directories, but some tools create directory markers with content type ‘application/x-directory’.

is_relative_to(other: S3_Bucket)

Check if this S3 path is relative to another.

Parameters:

other (S3_Bucket) – Another S3_Bucket instance to compare with.

Returns:

True if this path is relative to the other path, False otherwise.

Return type:

bool

iterdir() → Iterable[S3_Bucket_Key]

Return the S3_Bucket_Key objects contained in the in/under this object.

joinpath(*others) → S3_Bucket_Key | S3_Bucket_Folder

Join multiple path components.

Parameters:

*others – Path components to join.

Returns:

New S3 path object with all components joined.

Return type:

S3_Bucket_Key | S3_Bucket_Folder

key_exists(key: str | PurePosixPath) → bool

Check if an S3 object exists at the specified key.

Deprecated since version Use: exists() instead.

Parameters:

key (str | PurePosixPath) – S3 key to check.

Returns:

True if the object exists, False otherwise.

Return type:

bool

list_object_keys(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → List[str]

List S3 object keys under a key prefix.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder. If None, uses instance setting.

Returns:

List of S3 object keys.

Return type:

List[str]

list_object_paths(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → List[PurePosixPath]

Return the relative paths of objects contained in/under this object.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder. If None, uses instance setting.

Returns:

List of relative paths (PurePosixPath) of objects under the key prefix.

Return type:

List[PurePosixPath]

list_objects(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → BucketObjectsCollection

List S3 objects under a key prefix.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder by appending ‘/’. If None, uses instance setting.

Returns:

Collection of S3 ObjectSummary objects matching the prefix.

Return type:

BucketObjectsCollection

Raises:

S3ClientError – If the bucket or key doesn’t exist, or other errors occur.

model_dump_non_private(*, mode: Literal['json', 'python'] | str = 'python', exclude: Set[str] | None = None) → dict[str, Any]

nav_to_bucket(bucket_name) → S3_Bucket

nav_to_s3_link(s3_uri: str) → S3_Bucket_Key

open(mode: str = 'r', encoding: str | None = None, errors: str | None = None) → IOBase

Open the S3 object as a file-like object for reading.

Parameters:

• mode (str, optional) – File mode (‘r’ for text, ‘rb’ for binary). Default is ‘r’.

• encoding (str | None, optional) – Text encoding (for text mode only).

• errors (str | None, optional) – Error handling strategy for encoding (for text mode only).

Returns:

File-like object (TextIOWrapper for text mode, BufferedReader for binary).

Return type:

io.IOBase

Raises:

ValueError – If mode is empty or doesn’t start with ‘r’ (only read mode is supported).

read_bytes() → bytes

Read the S3 object contents as bytes.

Returns:

The complete file contents as bytes.

Return type:

bytes

read_text(encoding='utf-8', errors=None) → str

Read the S3 object contents as text.

Parameters:

• encoding (str, optional) – Text encoding. Default is ‘utf-8’.

• errors (str | None, optional) – Error handling strategy for encoding.

Returns:

The complete file contents as a string.

Return type:

str

rename(target: str)

Rename (move) this S3 object to a new key.

Parameters:

target (str) – Target S3 key or URI.

Notes

This copies the object to the new location then deletes the original.

set_as_child(name: str, other_config_item: ConfigHierarchy)

set_session(session: Session)

static split_s3_uri(s3_uri: str) → Tuple[str, str]

static translate_config_data(config_data: MutableMapping)

Children classes can provide translation logic to allow older config files to be used with newer config class definitions.

unlink(key: str | PurePosixPath | None = None, version_id: str | None = None, missing_ok=False)

Delete an S3 object (pathlib-style interface).

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key to delete. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID to delete. If None, deletes the latest version.

• missing_ok (bool, optional) – If True, don’t raise an error if the object doesn’t exist. Default is False.

Raises:

ClientError – If the object doesn’t exist and missing_ok is False, or other errors occur.

upload_file(*, local_filename: str | Path, key: str | PurePosixPath | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, overwrite_mode: OverwriteModes = OverwriteModes.ALWAYS_OVERWRITE)

Upload a local file to S3.

Parameters:

• local_filename (str | Path) – The local file to upload.

• key (str | PurePosixPath | None, optional) – S3 key to upload to. If None, uses the instance key.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 upload_file (e.g., metadata, ACL).

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the upload. Defaults to 5GB multipart threshold.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is ALWAYS_OVERWRITE.

upload_folder_file(local_filename: str | Path, key_suffix: str | PurePosixPath, extra_args: dict | None = None, transfer_config: TransferConfig | None = None)

Upload a local file to a key within this folder.

Deprecated since version Use: (my_folder / key_suffix).upload_file() instead.

Parameters:

• local_filename (str | Path) – Path to the local file to upload.

• key_suffix (str | PurePosixPath) – Key suffix to add after the folder name.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 upload_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the upload.

validator validate_folder  »  folder

Validate and convert the folder to a string.

Parameters:

v (Any) – Folder value to validate.

Returns:

Validated folder string (empty string for root folder).

Return type:

str

with_name(name: str)

Return a new S3_Bucket_Key with the file name changed.

Parameters:

name (str) – The new file name.

Returns:

New S3_Bucket_Key with the modified name.

Return type:

S3_Bucket_Key

with_stem(stem: str)

Return a new S3_Bucket_Key with the file stem changed.

Parameters:

stem (str) – The new file stem (name without extension).

Returns:

New S3_Bucket_Key with the modified stem.

Return type:

S3_Bucket_Key

with_suffix(suffix: str)

Return a new S3_Bucket_Key with the file suffix changed.

Parameters:

suffix (str) – The new file extension (e.g., ‘.txt’).

Returns:

New S3_Bucket_Key with the modified suffix.

Return type:

S3_Bucket_Key

pydantic model config_wrangler.config_templates.aws.s3_bucket.S3_Bucket_Folder_File

Bases: S3_Bucket_Folder

Represents a unique folder & file within an S3 bucket.

Similar to S3_Bucket_Key but uses folder + file_name instead of a single key.

file_name

The file name within the folder.

Type:

str

treat_as_folder

Always False for this class.

Type:

bool

Config:

• validate_default: bool = True

• validate_assignment: bool = True

• validate_credentials: bool = True

Fields:

• bucket_name ()

• file_name (str)

• folder ()

• iam_role ()

• keepass ()

• keepass_config ()

• keepass_group ()

• keepass_title ()

• key ()

• keyring_section ()

• password_source ()

• raw_password ()

• region_name ()

• treat_as_folder (bool)

• user_id ()

• validate_password_on_load ()

Validators:

• check_model » all fields

• validate_file_name » file_name

• validate_folder » folder

class CompareResult(*values)

Bases: Enum

Enumeration of file comparison results between S3 and local files.

DIFFERENT_SIZE

Files have different sizes.

Type:

auto

LOCAL_NEWER

Local file has a newer modification timestamp.

Type:

auto

SAME_TIMES

Files have identical modification timestamps.

Type:

auto

LOCAL_OLDER

Local file has an older modification timestamp.

Type:

auto

DIFFERENT_SIZE = 1

LOCAL_NEWER = 2

LOCAL_OLDER = 4

SAME_TIMES = 3

class OverwriteModes(*values)

Bases: Enum

Enumeration of file overwrite behavior modes for S3 operations.

ALWAYS_OVERWRITE

Always overwrite the target file regardless of timestamps or sizes.

Type:

auto

OVERWRITE_OLDER

Only overwrite if source is newer or sizes differ.

Type:

auto

NEVER_OVERWRITE

Never overwrite existing files.

Type:

auto

ALWAYS_OVERWRITE = 1

NEVER_OVERWRITE = 3

OVERWRITE_OLDER = 2

add_child(name: str, child_object: ConfigHierarchy)

Set this configuration as a child in the hierarchy of another config. For any programmatically created config objects this is required so that the new object ‘knows’ where it lives in the hierarchy – most importantly so that it can find the hierarchies root object.

as_bucket() → S3_Bucket

Create a new S3_Bucket instance with only the bucket name.

Returns:

S3_Bucket instance without key, folder, or file_name attributes.

Return type:

S3_Bucket

assume_role()

validator check_model  »  all fields

content_type() → str

Get the content type (MIME type) of the S3 object.

Returns:

The content type string (e.g., ‘text/plain’, ‘application/json’).

Return type:

str

copy_to(target: str | S3_Bucket_Key, version_id: str | None = None)

Copy this S3 object to another S3 location.

Parameters:

• target (str | S3_Bucket_Key) – Target S3 location (URI string or S3_Bucket_Key instance).

• version_id (str | None, optional) – Specific version ID to copy. If None, copies the latest version.

delete(key: str | PurePosixPath | None = None, version_id: str | None = None)

Delete an S3 object.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key to delete. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID to delete. If None, deletes the latest version.

delete_by_key(key: str | PurePosixPath, version_id: str | None = None)

Delete an S3 object by key.

Deprecated since version Use: delete() instead.

Parameters:

• key (str | PurePosixPath) – S3 key to delete.

• version_id (str | None, optional) – Specific version ID to delete.

download_file(*, local_filename: str | Path, key: str | PurePosixPath | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, create_parents: bool = True, overwrite_mode: OverwriteModes = OverwriteModes.OVERWRITE_OLDER)

Download an S3 object to a local file.

Parameters:

• local_filename (Path | str) – Path where the downloaded file will be saved.

• key (str | PurePosixPath | None, optional) – S3 key to download. If None, uses the instance key.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the download.

• create_parents (bool, optional) – Whether to create parent directories. Default is True.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is OVERWRITE_OLDER.

Raises:

• S3ClientError – If the S3 object does not exist or download fails.

• ValueError – If a non-blank key is not specified for the download.

download_files(*, local_path: str | Path, key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, create_parents: bool = True, overwrite_mode: OverwriteModes = OverwriteModes.OVERWRITE_OLDER) → Iterable[Path]

Download multiple S3 objects under a key prefix to a local directory.

Parameters:

• local_path (str | Path) – Local directory path where files will be downloaded.

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder prefix. If None, uses instance setting.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for downloads.

• create_parents (bool, optional) – Whether to create parent directories. Default is True.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is OVERWRITE_OLDER.

Returns:

List of local file paths that were downloaded.

Return type:

Iterable[Path]

download_folder_file(key_suffix: str | PurePosixPath, local_filename: str | Path, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, create_parents: bool = True)

Download a file from a key within this folder.

Deprecated since version Use: (my_folder / key_suffix).download_file() instead.

Parameters:

• key_suffix (str | PurePosixPath) – Key suffix to add after the folder name.

• local_filename (str | Path) – Path where the downloaded file will be saved.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the download.

• create_parents (bool, optional) – Whether to create parent directories. Default is True.

download_specified_file(local_filename: str | Path, extra_args: dict | None = None, transfer_config: TransferConfig | None = None)

Download this S3 object to a local file.

Deprecated since version Use: download_file() instead.

Parameters:

• local_filename (str | Path) – Path where the downloaded file will be saved.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the download.

exists(key: str | PurePosixPath | None = None, version_id: str | None = None) → bool

Check if an S3 object exists.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key to check. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID to check. If None, checks the latest version.

Returns:

True if the object exists, False otherwise.

Return type:

bool

Raises:

ClientError – If there’s an error other than 404/NoSuchKey.

find_objects(key: str | PurePosixPath | None = None) → BucketObjectsCollection

List S3 objects under a key prefix.

Deprecated since version Use: list_objects() instead.

Parameters:

key (str | PurePosixPath | None, optional) – S3 key prefix.

Returns:

Collection of S3 ObjectSummary objects.

Return type:

BucketObjectsCollection

full_item_name(item_name: str | None = None, delimiter: str = ' -> ')

The fully qualified name of this config item in the config hierarchy.

get_boto3_bucket() → Bucket

Get the boto3 Bucket resource object.

Returns:

The boto3 Bucket resource.

Return type:

Bucket

Raises:

ClientError – If the bucket does not exist (error code NoSuchBucket).

get_bucket_region() → str

Get the region_name from the actual S3 bucket definition.

Note

This can differ from the region_name attribute specified when constructing this class for example via a config file. That region_name attribute is used for establishing the AWS session.

NOTE 2:

If you don’t specify a Region, when the bucket is created it will have used the US East (N. Virginia) Region (us-east-1). So this method defaults to that if it does not find a LocationConstraint.

get_bucket_region_name() → str

Get the AWS region name where the bucket is located.

Returns:

AWS region name (e.g., ‘us-east-1’, ‘us-west-2’).

Return type:

str

Notes

Buckets in Region us-east-1 have a LocationConstraint of null.

get_copy(copied_by: str = 'get_copy') → AWS_Session

Copy this configuration. Useful when you need to programmatically modify a configuration without modifying the original base configuration.

get_object(key: str | PurePosixPath | None = None, version_id: str | None = None) → Object | ObjectVersion

Get a boto3 Object or ObjectVersion resource with caching.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID. If None, returns the latest version Object.

Returns:

The boto3 Object or ObjectVersion resource.

Return type:

Object | ObjectVersion

get_object_uncached(key: str | PurePosixPath | None = None, version_id: str | None = None) → Object | ObjectVersion

Get a boto3 Object or ObjectVersion resource without caching.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID. If None, returns the latest version Object.

Returns:

The boto3 Object or ObjectVersion resource.

Return type:

Object | ObjectVersion

get_password() → str

Get the password for this resource. password_source controls where it looks for the password. If that is None, then the root level passwords container is checked for password_source value.

get_request_v4_authorizer(service: str = None) → requests.auth.AuthBase

get_s3_bucket_key(key: str | PurePosixPath | None = None) → S3_Bucket_Key

Get an S3_Bucket_Key instance for the specified key.

Parameters:

key (str | PurePosixPath | None, optional) – S3 key. If None, uses the instance key.

Returns:

S3_Bucket_Key instance representing the key.

Return type:

S3_Bucket_Key

get_secrets_manager()

get_service_client(service: str)

get_service_resource(service: str)

get_session_expiry()

get_ssm()

is_file()

Check if this S3 object represents a file (not a directory marker).

Returns:

True if it’s a file, False if it’s a directory marker or doesn’t exist.

Return type:

bool

Notes

S3 doesn’t have true directories, but some tools create directory markers with content type ‘application/x-directory’.

is_relative_to(other: S3_Bucket)

Check if this S3 path is relative to another.

Parameters:

other (S3_Bucket) – Another S3_Bucket instance to compare with.

Returns:

True if this path is relative to the other path, False otherwise.

Return type:

bool

iter_versions() → Iterator[S3_Bucket_Key_Version]

Method inherited from S3_Bucket_Key

iterdir() → Iterable[S3_Bucket_Key]

Return the S3_Bucket_Key objects contained in the in/under this object.

joinpath(*others) → S3_Bucket_Key | S3_Bucket_Folder

Join multiple path components.

Parameters:

*others – Path components to join.

Returns:

New S3 path object with all components joined.

Return type:

S3_Bucket_Key | S3_Bucket_Folder

key_exists(key: str | PurePosixPath) → bool

Check if an S3 object exists at the specified key.

Deprecated since version Use: exists() instead.

Parameters:

key (str | PurePosixPath) – S3 key to check.

Returns:

True if the object exists, False otherwise.

Return type:

bool

list_object_keys(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → List[str]

List S3 object keys under a key prefix.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder. If None, uses instance setting.

Returns:

List of S3 object keys.

Return type:

List[str]

list_object_paths(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → List[PurePosixPath]

Return the relative paths of objects contained in/under this object.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder. If None, uses instance setting.

Returns:

List of relative paths (PurePosixPath) of objects under the key prefix.

Return type:

List[PurePosixPath]

list_objects(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → BucketObjectsCollection

List S3 objects under a key prefix.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder by appending ‘/’. If None, uses instance setting.

Returns:

Collection of S3 ObjectSummary objects matching the prefix.

Return type:

BucketObjectsCollection

Raises:

S3ClientError – If the bucket or key doesn’t exist, or other errors occur.

model_dump_non_private(*, mode: Literal['json', 'python'] | str = 'python', exclude: Set[str] | None = None) → dict[str, Any]

model_post_init(_S3_Bucket_Folder_File__context: Any) → None

Initialize the key attribute after model creation.

Parameters:

__context (Any) – Pydantic context (unused).

nav_to_bucket(bucket_name) → S3_Bucket

nav_to_s3_link(s3_uri: str) → S3_Bucket_Key

open(mode: str = 'r', encoding: str | None = None, errors: str | None = None) → IOBase

Open the S3 object as a file-like object for reading.

Parameters:

• mode (str, optional) – File mode (‘r’ for text, ‘rb’ for binary). Default is ‘r’.

• encoding (str | None, optional) – Text encoding (for text mode only).

• errors (str | None, optional) – Error handling strategy for encoding (for text mode only).

Returns:

File-like object (TextIOWrapper for text mode, BufferedReader for binary).

Return type:

io.IOBase

Raises:

ValueError – If mode is empty or doesn’t start with ‘r’ (only read mode is supported).

read_bytes() → bytes

Read the S3 object contents as bytes.

Returns:

The complete file contents as bytes.

Return type:

bytes

read_text(encoding='utf-8', errors=None) → str

Read the S3 object contents as text.

Parameters:

• encoding (str, optional) – Text encoding. Default is ‘utf-8’.

• errors (str | None, optional) – Error handling strategy for encoding.

Returns:

The complete file contents as a string.

Return type:

str

rename(target: str)

Rename (move) this S3 object to a new key.

Parameters:

target (str) – Target S3 key or URI.

Notes

This copies the object to the new location then deletes the original.

set_as_child(name: str, other_config_item: ConfigHierarchy)

set_session(session: Session)

static split_s3_uri(s3_uri: str) → Tuple[str, str]

static translate_config_data(config_data: MutableMapping)

Children classes can provide translation logic to allow older config files to be used with newer config class definitions.

unlink(key: str | PurePosixPath | None = None, version_id: str | None = None, missing_ok=False)

Delete an S3 object (pathlib-style interface).

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key to delete. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID to delete. If None, deletes the latest version.

• missing_ok (bool, optional) – If True, don’t raise an error if the object doesn’t exist. Default is False.

Raises:

ClientError – If the object doesn’t exist and missing_ok is False, or other errors occur.

upload_file(*, local_filename: str | Path, key: str | PurePosixPath | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, overwrite_mode: OverwriteModes = OverwriteModes.ALWAYS_OVERWRITE)

Upload a local file to S3.

Parameters:

• local_filename (str | Path) – The local file to upload.

• key (str | PurePosixPath | None, optional) – S3 key to upload to. If None, uses the instance key.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 upload_file (e.g., metadata, ACL).

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the upload. Defaults to 5GB multipart threshold.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is ALWAYS_OVERWRITE.

upload_folder_file(local_filename: str | Path, key_suffix: str | PurePosixPath, extra_args: dict | None = None, transfer_config: TransferConfig | None = None)

Upload a local file to a key within this folder.

Deprecated since version Use: (my_folder / key_suffix).upload_file() instead.

Parameters:

• local_filename (str | Path) – Path to the local file to upload.

• key_suffix (str | PurePosixPath) – Key suffix to add after the folder name.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 upload_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the upload.

upload_specified_file(local_filename: str | Path, extra_args: dict | None = None, transfer_config: TransferConfig | None = None)

Upload a local file to this S3 folder/file location.

Deprecated since version Use: upload_file() instead.

Parameters:

• local_filename (str | Path) – Path to the local file to upload.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 upload_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the upload.

validator validate_file_name  »  file_name

Validate and convert the file_name to a string.

Parameters:

v (Any) – File name value to validate.

Returns:

Validated file name string.

Return type:

str

Raises:

ConfigError – If the file name is an empty string.

validator validate_folder  »  folder

Validate and convert the folder to a string.

Parameters:

v (Any) – Folder value to validate.

Returns:

Validated folder string (empty string for root folder).

Return type:

str

with_name(name: str)

Return a new S3_Bucket_Key with the file name changed.

Parameters:

name (str) – The new file name.

Returns:

New S3_Bucket_Key with the modified name.

Return type:

S3_Bucket_Key

with_stem(stem: str)

Return a new S3_Bucket_Key with the file stem changed.

Parameters:

stem (str) – The new file stem (name without extension).

Returns:

New S3_Bucket_Key with the modified stem.

Return type:

S3_Bucket_Key

with_suffix(suffix: str)

Return a new S3_Bucket_Key with the file suffix changed.

Parameters:

suffix (str) – The new file extension (e.g., ‘.txt’).

Returns:

New S3_Bucket_Key with the modified suffix.

Return type:

S3_Bucket_Key

pydantic model config_wrangler.config_templates.aws.s3_bucket.S3_Bucket_Key

Bases: S3_Bucket

Represents a unique file (key) within an S3 bucket.

Subclass of S3_Bucket that additionally allows a key to be specified. This can represent either a “folder” like key prefix or an individual S3 object (file). Also provides cached access to object metadata like size and modification time.

key

The S3 object key (required). Must be a valid S3 key string.

Type:

str

Config:

• validate_default: bool = True

• validate_assignment: bool = True

• validate_credentials: bool = True

Fields:

• bucket_name ()

• iam_role ()

• keepass ()

• keepass_config ()

• keepass_group ()

• keepass_title ()

• key (str)

• keyring_section ()

• password_source ()

• raw_password ()

• region_name ()

• treat_as_folder ()

• user_id ()

• validate_password_on_load ()

Validators:

• check_model » all fields

• validate_key » key

class CompareResult(*values)

Bases: Enum

Enumeration of file comparison results between S3 and local files.

DIFFERENT_SIZE

Files have different sizes.

Type:

auto

LOCAL_NEWER

Local file has a newer modification timestamp.

Type:

auto

SAME_TIMES

Files have identical modification timestamps.

Type:

auto

LOCAL_OLDER

Local file has an older modification timestamp.

Type:

auto

DIFFERENT_SIZE = 1

LOCAL_NEWER = 2

LOCAL_OLDER = 4

SAME_TIMES = 3

class OverwriteModes(*values)

Bases: Enum

Enumeration of file overwrite behavior modes for S3 operations.

ALWAYS_OVERWRITE

Always overwrite the target file regardless of timestamps or sizes.

Type:

auto

OVERWRITE_OLDER

Only overwrite if source is newer or sizes differ.

Type:

auto

NEVER_OVERWRITE

Never overwrite existing files.

Type:

auto

ALWAYS_OVERWRITE = 1

NEVER_OVERWRITE = 3

OVERWRITE_OLDER = 2

add_child(name: str, child_object: ConfigHierarchy)

Set this configuration as a child in the hierarchy of another config. For any programmatically created config objects this is required so that the new object ‘knows’ where it lives in the hierarchy – most importantly so that it can find the hierarchies root object.

as_bucket() → S3_Bucket

Create a new S3_Bucket instance with only the bucket name.

Returns:

S3_Bucket instance without key, folder, or file_name attributes.

Return type:

S3_Bucket

assume_role()

validator check_model  »  all fields

compare_to_file(local_filename: Path) → CompareResult

Compare this S3 object to a local file.

Parameters:

local_filename (Path) – Path to the local file to compare against.

Returns:

The comparison result indicating size differences or relative age.

Return type:

S3_Bucket.CompareResult

content_type() → str

Get the content type (MIME type) of the S3 object.

Returns:

The content type string (e.g., ‘text/plain’, ‘application/json’).

Return type:

str

copy_to(target: str | S3_Bucket_Key, version_id: str | None = None)

Copy this S3 object to another S3 location.

Parameters:

• target (str | S3_Bucket_Key) – Target S3 location (URI string or S3_Bucket_Key instance).

• version_id (str | None, optional) – Specific version ID to copy. If None, copies the latest version.

delete(key: str | PurePosixPath | None = None, version_id: str | None = None)

Delete an S3 object.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key to delete. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID to delete. If None, deletes the latest version.

delete_by_key(key: str | PurePosixPath, version_id: str | None = None)

Delete an S3 object by key.

Deprecated since version Use: delete() instead.

Parameters:

• key (str | PurePosixPath) – S3 key to delete.

• version_id (str | None, optional) – Specific version ID to delete.

download_file(*, local_filename: str | Path, key: str | PurePosixPath | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, create_parents: bool = True, overwrite_mode: OverwriteModes = OverwriteModes.OVERWRITE_OLDER)

Download an S3 object to a local file.

Parameters:

• local_filename (Path | str) – Path where the downloaded file will be saved.

• key (str | PurePosixPath | None, optional) – S3 key to download. If None, uses the instance key.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the download.

• create_parents (bool, optional) – Whether to create parent directories. Default is True.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is OVERWRITE_OLDER.

Raises:

• S3ClientError – If the S3 object does not exist or download fails.

• ValueError – If a non-blank key is not specified for the download.

download_files(*, local_path: str | Path, key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, create_parents: bool = True, overwrite_mode: OverwriteModes = OverwriteModes.OVERWRITE_OLDER) → Iterable[Path]

Download multiple S3 objects under a key prefix to a local directory.

Parameters:

• local_path (str | Path) – Local directory path where files will be downloaded.

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder prefix. If None, uses instance setting.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for downloads.

• create_parents (bool, optional) – Whether to create parent directories. Default is True.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is OVERWRITE_OLDER.

Returns:

List of local file paths that were downloaded.

Return type:

Iterable[Path]

download_specified_file(local_filename: str | Path, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, create_parents: bool = True)

Download this S3 object to a local file.

Deprecated since version Use: download_file() instead.

Parameters:

• local_filename (str | Path) – Path where the downloaded file will be saved.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the download.

• create_parents (bool, optional) – Whether to create parent directories. Default is True.

exists(key: str | PurePosixPath | None = None, version_id: str | None = None) → bool

Check if an S3 object exists.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key to check. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID to check. If None, checks the latest version.

Returns:

True if the object exists, False otherwise.

Return type:

bool

Raises:

ClientError – If there’s an error other than 404/NoSuchKey.

find_objects(key: str | PurePosixPath | None = None) → BucketObjectsCollection

List S3 objects under a key prefix.

Deprecated since version Use: list_objects() instead.

Parameters:

key (str | PurePosixPath | None, optional) – S3 key prefix.

Returns:

Collection of S3 ObjectSummary objects.

Return type:

BucketObjectsCollection

full_item_name(item_name: str | None = None, delimiter: str = ' -> ')

The fully qualified name of this config item in the config hierarchy.

get_boto3_bucket() → Bucket

Get the boto3 Bucket resource object.

Returns:

The boto3 Bucket resource.

Return type:

Bucket

Raises:

ClientError – If the bucket does not exist (error code NoSuchBucket).

get_bucket_region() → str

Get the region_name from the actual S3 bucket definition.

Note

This can differ from the region_name attribute specified when constructing this class for example via a config file. That region_name attribute is used for establishing the AWS session.

NOTE 2:

If you don’t specify a Region, when the bucket is created it will have used the US East (N. Virginia) Region (us-east-1). So this method defaults to that if it does not find a LocationConstraint.

get_bucket_region_name() → str

Get the AWS region name where the bucket is located.

Returns:

AWS region name (e.g., ‘us-east-1’, ‘us-west-2’).

Return type:

str

Notes

Buckets in Region us-east-1 have a LocationConstraint of null.

get_copy(copied_by: str = 'get_copy') → AWS_Session

Copy this configuration. Useful when you need to programmatically modify a configuration without modifying the original base configuration.

get_object(key: str | PurePosixPath | None = None, version_id: str | None = None) → Object | ObjectVersion

Get a boto3 Object or ObjectVersion resource with caching.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID. If None, returns the latest version Object.

Returns:

The boto3 Object or ObjectVersion resource.

Return type:

Object | ObjectVersion

get_object_uncached(key: str | PurePosixPath | None = None, version_id: str | None = None) → Object | ObjectVersion

Get a boto3 Object or ObjectVersion resource without caching.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID. If None, returns the latest version Object.

Returns:

The boto3 Object or ObjectVersion resource.

Return type:

Object | ObjectVersion

get_password() → str

Get the password for this resource. password_source controls where it looks for the password. If that is None, then the root level passwords container is checked for password_source value.

get_request_v4_authorizer(service: str = None) → requests.auth.AuthBase

get_s3_bucket_key(key: str | PurePosixPath | None = None) → S3_Bucket_Key

Get an S3_Bucket_Key instance for the specified key.

Parameters:

key (str | PurePosixPath | None, optional) – S3 key. If None, uses the instance key.

Returns:

S3_Bucket_Key instance representing the key.

Return type:

S3_Bucket_Key

get_secrets_manager()

get_service_client(service: str)

get_service_resource(service: str)

get_session_expiry()

get_ssm()

is_file()

Check if this S3 object represents a file (not a directory marker).

Returns:

True if it’s a file, False if it’s a directory marker or doesn’t exist.

Return type:

bool

Notes

S3 doesn’t have true directories, but some tools create directory markers with content type ‘application/x-directory’.

is_relative_to(other: S3_Bucket)

Check if this S3 path is relative to another.

Parameters:

other (S3_Bucket) – Another S3_Bucket instance to compare with.

Returns:

True if this path is relative to the other path, False otherwise.

Return type:

bool

iter_versions() → Iterator[S3_Bucket_Key_Version]

Iterate over all versions of this S3 object key.

Yields S3_Bucket_Key_Version objects for each version that exists in S3. Versions are returned in the order provided by S3 (typically newest first).

Raises:

ClientError – If there’s an error accessing the S3 bucket or versions.

iterdir() → Iterable[S3_Bucket_Key]

Return the S3_Bucket_Key objects contained in the in/under this object.

joinpath(*others) → S3_Bucket_Key | S3_Bucket_Folder

Join multiple path components.

Parameters:

*others – Path components to join.

Returns:

New S3 path object with all components joined.

Return type:

S3_Bucket_Key | S3_Bucket_Folder

key_exists(key: str | PurePosixPath) → bool

Check if an S3 object exists at the specified key.

Deprecated since version Use: exists() instead.

Parameters:

key (str | PurePosixPath) – S3 key to check.

Returns:

True if the object exists, False otherwise.

Return type:

bool

latest_version() → S3_Bucket_Key_Version

Get the latest version of this S3 object.

Returns:

The latest version of this S3 object.

Return type:

S3_Bucket_Key_Version

Raises:

FileNotFoundError – If no latest version exists or multiple latest versions are found.

list_object_keys(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → List[str]

List S3 object keys under a key prefix.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder. If None, uses instance setting.

Returns:

List of S3 object keys.

Return type:

List[str]

list_object_paths(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → List[PurePosixPath]

Return the relative paths of objects contained in/under this object.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder. If None, uses instance setting.

Returns:

List of relative paths (PurePosixPath) of objects under the key prefix.

Return type:

List[PurePosixPath]

list_objects(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → BucketObjectsCollection

List S3 objects under a key prefix.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder by appending ‘/’. If None, uses instance setting.

Returns:

Collection of S3 ObjectSummary objects matching the prefix.

Return type:

BucketObjectsCollection

Raises:

S3ClientError – If the bucket or key doesn’t exist, or other errors occur.

model_dump_non_private(*, mode: Literal['json', 'python'] | str = 'python', exclude: Set[str] | None = None) → dict[str, Any]

nav_to_bucket(bucket_name) → S3_Bucket

nav_to_s3_link(s3_uri: str) → S3_Bucket_Key

open(mode: str = 'r', encoding: str | None = None, errors: str | None = None) → IOBase

Open the S3 object as a file-like object for reading.

Parameters:

• mode (str, optional) – File mode (‘r’ for text, ‘rb’ for binary). Default is ‘r’.

• encoding (str | None, optional) – Text encoding (for text mode only).

• errors (str | None, optional) – Error handling strategy for encoding (for text mode only).

Returns:

File-like object (TextIOWrapper for text mode, BufferedReader for binary).

Return type:

io.IOBase

Raises:

ValueError – If mode is empty or doesn’t start with ‘r’ (only read mode is supported).

read_bytes() → bytes

Read the S3 object contents as bytes.

Returns:

The complete file contents as bytes.

Return type:

bytes

read_text(encoding='utf-8', errors=None) → str

Read the S3 object contents as text.

Parameters:

• encoding (str, optional) – Text encoding. Default is ‘utf-8’.

• errors (str | None, optional) – Error handling strategy for encoding.

Returns:

The complete file contents as a string.

Return type:

str

rename(target: str)

Rename (move) this S3 object to a new key.

Parameters:

target (str) – Target S3 key or URI.

Notes

This copies the object to the new location then deletes the original.

set_as_child(name: str, other_config_item: ConfigHierarchy)

set_session(session: Session)

static split_s3_uri(s3_uri: str) → Tuple[str, str]

static translate_config_data(config_data: MutableMapping)

Children classes can provide translation logic to allow older config files to be used with newer config class definitions.

unlink(key: str | PurePosixPath | None = None, version_id: str | None = None, missing_ok=False)

Delete an S3 object (pathlib-style interface).

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key to delete. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID to delete. If None, deletes the latest version.

• missing_ok (bool, optional) – If True, don’t raise an error if the object doesn’t exist. Default is False.

Raises:

ClientError – If the object doesn’t exist and missing_ok is False, or other errors occur.

upload_file(*, local_filename: str | Path, key: str | PurePosixPath | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, overwrite_mode: OverwriteModes = OverwriteModes.ALWAYS_OVERWRITE)

Upload a local file to S3.

Parameters:

• local_filename (str | Path) – The local file to upload.

• key (str | PurePosixPath | None, optional) – S3 key to upload to. If None, uses the instance key.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 upload_file (e.g., metadata, ACL).

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the upload. Defaults to 5GB multipart threshold.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is ALWAYS_OVERWRITE.

upload_specified_file(local_filename: str | Path, extra_args: dict | None = None, transfer_config: TransferConfig | None = None)

Upload a local file to this S3 key.

Deprecated since version Use: upload_file() instead.

Parameters:

• local_filename (str | Path) – Path to the local file to upload.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 upload_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the upload.

validator validate_key  »  key

Validate and convert the key to a string.

Parameters:

v (Any) – Key value to validate.

Returns:

Validated key string.

Return type:

str

version(version_id: str) → S3_Bucket_Key_Version

Get a specific version of this S3 object by version ID.

Parameters:

version_id (str) – The version ID to retrieve.

Returns:

The S3_Bucket_Key_Version instance for the specified version.

Return type:

S3_Bucket_Key_Version

Raises:

FileNotFoundError – If the specified version ID does not exist.

version_map() → dict[str, S3_Bucket_Key_Version]

Get a mapping of version IDs to S3_Bucket_Key_Version instances.

Returns:

Dictionary mapping version IDs to S3_Bucket_Key_Version objects.

Return type:

dict[str, S3_Bucket_Key_Version]

with_name(name: str)

Return a new S3_Bucket_Key with the file name changed.

Parameters:

name (str) – The new file name.

Returns:

New S3_Bucket_Key with the modified name.

Return type:

S3_Bucket_Key

with_stem(stem: str)

Return a new S3_Bucket_Key with the file stem changed.

Parameters:

stem (str) – The new file stem (name without extension).

Returns:

New S3_Bucket_Key with the modified stem.

Return type:

S3_Bucket_Key

with_suffix(suffix: str)

Return a new S3_Bucket_Key with the file suffix changed.

Parameters:

suffix (str) – The new file extension (e.g., ‘.txt’).

Returns:

New S3_Bucket_Key with the modified suffix.

Return type:

S3_Bucket_Key

pydantic model config_wrangler.config_templates.aws.s3_bucket.S3_Bucket_Key_Version

Bases: S3_Bucket_Key

Represents a specific version of an S3 object key. Extends S3_Bucket_Key with version_id to uniquely identify a version.

Config:

• validate_default: bool = True

• validate_assignment: bool = True

• validate_credentials: bool = True

Fields:

• bucket_name ()

• iam_role ()

• is_latest (bool)

• keepass ()

• keepass_config ()

• keepass_group ()

• keepass_title ()

• key ()

• keyring_section ()

• password_source ()

• raw_password ()

• region_name ()

• treat_as_folder (bool)

• user_id ()

• validate_password_on_load ()

• version_id (str)

Validators:

• check_model » all fields

• validate_key » key

• validate_version_id » version_id

class CompareResult(*values)

Bases: Enum

Enumeration of file comparison results between S3 and local files.

DIFFERENT_SIZE

Files have different sizes.

Type:

auto

LOCAL_NEWER

Local file has a newer modification timestamp.

Type:

auto

SAME_TIMES

Files have identical modification timestamps.

Type:

auto

LOCAL_OLDER

Local file has an older modification timestamp.

Type:

auto

DIFFERENT_SIZE = 1

LOCAL_NEWER = 2

LOCAL_OLDER = 4

SAME_TIMES = 3

class OverwriteModes(*values)

Bases: Enum

Enumeration of file overwrite behavior modes for S3 operations.

ALWAYS_OVERWRITE

Always overwrite the target file regardless of timestamps or sizes.

Type:

auto

OVERWRITE_OLDER

Only overwrite if source is newer or sizes differ.

Type:

auto

NEVER_OVERWRITE

Never overwrite existing files.

Type:

auto

ALWAYS_OVERWRITE = 1

NEVER_OVERWRITE = 3

OVERWRITE_OLDER = 2

add_child(name: str, child_object: ConfigHierarchy)

Set this configuration as a child in the hierarchy of another config. For any programmatically created config objects this is required so that the new object ‘knows’ where it lives in the hierarchy – most importantly so that it can find the hierarchies root object.

as_bucket() → S3_Bucket

Create a new S3_Bucket instance with only the bucket name.

Returns:

S3_Bucket instance without key, folder, or file_name attributes.

Return type:

S3_Bucket

assume_role()

validator check_model  »  all fields

compare_to_file(local_filename: Path) → CompareResult

Compare this S3 object to a local file.

Parameters:

local_filename (Path) – Path to the local file to compare against.

Returns:

The comparison result indicating size differences or relative age.

Return type:

S3_Bucket.CompareResult

content_type() → str

Get the content type (MIME type) of the S3 object.

Returns:

The content type string (e.g., ‘text/plain’, ‘application/json’).

Return type:

str

copy_to(target: str | S3_Bucket_Key, version_id: str | None = None)

Copy this specific version of the S3 object to another location.

Parameters:

• target (str | S3_Bucket_Key) – Target S3 location.

• version_id (str | None, optional) – Version ID to copy. If None, uses instance version_id.

delete(key: str | PurePosixPath | None = None, version_id: str | None = None)

Delete this specific version of the S3 object.

Parameters:

• key (str | PurePosixPath | None, optional) – Ignored for this subclass (always uses instance key).

• version_id (str | None, optional) – Version ID to delete. If None, uses instance version_id.

delete_by_key(key: str | PurePosixPath, version_id: str | None = None)

Delete an S3 object by key.

Deprecated since version Use: delete() instead.

Parameters:

• key (str | PurePosixPath) – S3 key to delete.

• version_id (str | None, optional) – Specific version ID to delete.

download_file(*, local_filename: str | Path, key: str | PurePosixPath | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, create_parents: bool = True, overwrite_mode: OverwriteModes = OverwriteModes.OVERWRITE_OLDER)

Download this specific version of the S3 object to a local file.

Parameters:

• local_filename (str | Path) – Path where the downloaded file will be saved.

• key (str | PurePosixPath | None, optional) – Ignored for this subclass.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file. VersionId is automatically added.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the download.

• create_parents (bool, optional) – Whether to create parent directories. Default is True.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is OVERWRITE_OLDER.

download_files(*, local_path: str | Path, key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, create_parents: bool = True, overwrite_mode: OverwriteModes = OverwriteModes.OVERWRITE_OLDER) → Iterable[Path]

Download multiple S3 objects under a key prefix to a local directory.

Parameters:

• local_path (str | Path) – Local directory path where files will be downloaded.

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder prefix. If None, uses instance setting.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for downloads.

• create_parents (bool, optional) – Whether to create parent directories. Default is True.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is OVERWRITE_OLDER.

Returns:

List of local file paths that were downloaded.

Return type:

Iterable[Path]

download_specified_file(local_filename: str | Path, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, create_parents: bool = True)

Download this S3 object to a local file.

Deprecated since version Use: download_file() instead.

Parameters:

• local_filename (str | Path) – Path where the downloaded file will be saved.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 download_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the download.

• create_parents (bool, optional) – Whether to create parent directories. Default is True.

exists(key: str | PurePosixPath | None = None, version_id: str | None = None) → bool

Check if this specific version of the S3 object exists.

Parameters:

• key (str | PurePosixPath | None, optional) – Ignored for this subclass (always uses instance key).

• version_id (str | None, optional) – Version ID to check. If None, uses instance version_id.

Returns:

True if the version exists, False otherwise.

Return type:

bool

find_objects(key: str | PurePosixPath | None = None) → BucketObjectsCollection

List S3 objects under a key prefix.

Deprecated since version Use: list_objects() instead.

Parameters:

key (str | PurePosixPath | None, optional) – S3 key prefix.

Returns:

Collection of S3 ObjectSummary objects.

Return type:

BucketObjectsCollection

full_item_name(item_name: str | None = None, delimiter: str = ' -> ')

The fully qualified name of this config item in the config hierarchy.

get_boto3_bucket() → Bucket

Get the boto3 Bucket resource object.

Returns:

The boto3 Bucket resource.

Return type:

Bucket

Raises:

ClientError – If the bucket does not exist (error code NoSuchBucket).

get_bucket_region() → str

Get the region_name from the actual S3 bucket definition.

Note

This can differ from the region_name attribute specified when constructing this class for example via a config file. That region_name attribute is used for establishing the AWS session.

NOTE 2:

If you don’t specify a Region, when the bucket is created it will have used the US East (N. Virginia) Region (us-east-1). So this method defaults to that if it does not find a LocationConstraint.

get_bucket_region_name() → str

Get the AWS region name where the bucket is located.

Returns:

AWS region name (e.g., ‘us-east-1’, ‘us-west-2’).

Return type:

str

Notes

Buckets in Region us-east-1 have a LocationConstraint of null.

get_copy(copied_by: str = 'get_copy') → AWS_Session

Copy this configuration. Useful when you need to programmatically modify a configuration without modifying the original base configuration.

get_object(key: str | PurePosixPath | None = None, version_id: str | None = None) → Object | ObjectVersion

Get a boto3 Object or ObjectVersion resource with caching.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key. If None, uses the instance key.

• version_id (str | None, optional) – Specific version ID. If None, returns the latest version Object.

Returns:

The boto3 Object or ObjectVersion resource.

Return type:

Object | ObjectVersion

get_object_uncached(key: str | PurePosixPath | None = None, version_id: str | None = None) → Object | ObjectVersion

Get the versioned object from S3 without caching.

Parameters:

• key (str | PurePosixPath | None, optional) – Ignored for this subclass (always uses instance key).

• version_id (str | None, optional) – Version ID to retrieve. If None, uses instance version_id.

Returns:

The boto3 ObjectVersion resource.

Return type:

Object | ObjectVersion

get_password() → str

Get the password for this resource. password_source controls where it looks for the password. If that is None, then the root level passwords container is checked for password_source value.

get_request_v4_authorizer(service: str = None) → requests.auth.AuthBase

get_s3_bucket_key(key: str | PurePosixPath | None = None) → S3_Bucket_Key

Get an S3_Bucket_Key instance for the specified key.

Parameters:

key (str | PurePosixPath | None, optional) – S3 key. If None, uses the instance key.

Returns:

S3_Bucket_Key instance representing the key.

Return type:

S3_Bucket_Key

get_secrets_manager()

get_service_client(service: str)

get_service_resource(service: str)

get_session_expiry()

get_ssm()

is_file()

Check if this S3 object represents a file (not a directory marker).

Returns:

True if it’s a file, False if it’s a directory marker or doesn’t exist.

Return type:

bool

Notes

S3 doesn’t have true directories, but some tools create directory markers with content type ‘application/x-directory’.

is_relative_to(other: S3_Bucket)

Check if this S3 path is relative to another.

Parameters:

other (S3_Bucket) – Another S3_Bucket instance to compare with.

Returns:

True if this path is relative to the other path, False otherwise.

Return type:

bool

iter_versions() → Iterator[S3_Bucket_Key_Version]

Iterate over all versions of this S3 object key.

Yields S3_Bucket_Key_Version objects for each version that exists in S3. Versions are returned in the order provided by S3 (typically newest first).

Raises:

ClientError – If there’s an error accessing the S3 bucket or versions.

iterdir() → Iterable[S3_Bucket_Key]

Return the S3_Bucket_Key objects contained in the in/under this object.

joinpath(*others) → S3_Bucket_Key | S3_Bucket_Folder

Join multiple path components.

Parameters:

*others – Path components to join.

Returns:

New S3 path object with all components joined.

Return type:

S3_Bucket_Key | S3_Bucket_Folder

key_exists(key: str | PurePosixPath) → bool

Check if an S3 object exists at the specified key.

Deprecated since version Use: exists() instead.

Parameters:

key (str | PurePosixPath) – S3 key to check.

Returns:

True if the object exists, False otherwise.

Return type:

bool

latest_version() → S3_Bucket_Key_Version

Get the latest version of this S3 object.

Returns:

The latest version of this S3 object.

Return type:

S3_Bucket_Key_Version

Raises:

FileNotFoundError – If no latest version exists or multiple latest versions are found.

list_object_keys(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → List[str]

List S3 object keys under a key prefix.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder. If None, uses instance setting.

Returns:

List of S3 object keys.

Return type:

List[str]

list_object_paths(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → List[PurePosixPath]

Return the relative paths of objects contained in/under this object.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder. If None, uses instance setting.

Returns:

List of relative paths (PurePosixPath) of objects under the key prefix.

Return type:

List[PurePosixPath]

list_objects(key: str | PurePosixPath | None = None, treat_as_folder: bool | None = None) → BucketObjectsCollection

List S3 objects under a key prefix.

Parameters:

• key (str | PurePosixPath | None, optional) – S3 key prefix. If None, uses the instance key.

• treat_as_folder (bool | None, optional) – Whether to treat the key as a folder by appending ‘/’. If None, uses instance setting.

Returns:

Collection of S3 ObjectSummary objects matching the prefix.

Return type:

BucketObjectsCollection

Raises:

S3ClientError – If the bucket or key doesn’t exist, or other errors occur.

model_dump_non_private(*, mode: Literal['json', 'python'] | str = 'python', exclude: Set[str] | None = None) → dict[str, Any]

nav_to_bucket(bucket_name) → S3_Bucket

nav_to_s3_link(s3_uri: str) → S3_Bucket_Key

open(mode: str = 'r', encoding: str | None = None, errors: str | None = None) → IOBase

Open the S3 object as a file-like object for reading.

Parameters:

• mode (str, optional) – File mode (‘r’ for text, ‘rb’ for binary). Default is ‘r’.

• encoding (str | None, optional) – Text encoding (for text mode only).

• errors (str | None, optional) – Error handling strategy for encoding (for text mode only).

Returns:

File-like object (TextIOWrapper for text mode, BufferedReader for binary).

Return type:

io.IOBase

Raises:

ValueError – If mode is empty or doesn’t start with ‘r’ (only read mode is supported).

read_bytes() → bytes

Read the S3 object contents as bytes.

Returns:

The complete file contents as bytes.

Return type:

bytes

read_text(encoding='utf-8', errors=None) → str

Read the S3 object contents as text.

Parameters:

• encoding (str, optional) – Text encoding. Default is ‘utf-8’.

• errors (str | None, optional) – Error handling strategy for encoding.

Returns:

The complete file contents as a string.

Return type:

str

rename(target: str)

Rename (move) this S3 object to a new key.

Parameters:

target (str) – Target S3 key or URI.

Notes

This copies the object to the new location then deletes the original.

set_as_child(name: str, other_config_item: ConfigHierarchy)

set_session(session: Session)

static split_s3_uri(s3_uri: str) → Tuple[str, str]

static translate_config_data(config_data: MutableMapping)

Children classes can provide translation logic to allow older config files to be used with newer config class definitions.

unlink(key: str | PurePosixPath | None = None, version_id: str | None = None, missing_ok=False)

Delete this specific version of the S3 object (pathlib-style).

Parameters:

• key (str | PurePosixPath | None, optional) – Ignored for this subclass (always uses instance key).

• version_id (str | None, optional) – Version ID to delete. If None, uses instance version_id.

• missing_ok (bool, optional) – If True, don’t raise an error if the version doesn’t exist. Default is False.

upload_file(*, local_filename: str | Path, key: str | PurePosixPath | None = None, extra_args: dict | None = None, transfer_config: TransferConfig | None = None, overwrite_mode: OverwriteModes = OverwriteModes.ALWAYS_OVERWRITE)

Upload a local file to S3.

Parameters:

• local_filename (str | Path) – The local file to upload.

• key (str | PurePosixPath | None, optional) – S3 key to upload to. If None, uses the instance key.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 upload_file (e.g., metadata, ACL).

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the upload. Defaults to 5GB multipart threshold.

• overwrite_mode (OverwriteModes, optional) – Controls overwrite behavior. Default is ALWAYS_OVERWRITE.

upload_specified_file(local_filename: str | Path, extra_args: dict | None = None, transfer_config: TransferConfig | None = None)

Upload a local file to this S3 key.

Deprecated since version Use: upload_file() instead.

Parameters:

• local_filename (str | Path) – Path to the local file to upload.

• extra_args (dict | None, optional) – Extra arguments to pass to boto3 upload_file.

• transfer_config (TransferConfig | None, optional) – Transfer configuration for the upload.

validator validate_key  »  key

Validate and convert the key to a string.

Parameters:

v (Any) – Key value to validate.

Returns:

Validated key string.

Return type:

str

validator validate_version_id  »  version_id

Validate and convert the version_id to a string.

Parameters:

v (Any) – Version ID value to validate.

Returns:

Validated version ID string, or None if ‘null’.

Return type:

str | None

version(version_id: str) → S3_Bucket_Key_Version

Get a specific version of this S3 object by version ID.

Parameters:

version_id (str) – The version ID to retrieve.

Returns:

The S3_Bucket_Key_Version instance for the specified version.

Return type:

S3_Bucket_Key_Version

Raises:

FileNotFoundError – If the specified version ID does not exist.

version_map() → dict[str, S3_Bucket_Key_Version]

Get a mapping of version IDs to S3_Bucket_Key_Version instances.

Returns:

Dictionary mapping version IDs to S3_Bucket_Key_Version objects.

Return type:

dict[str, S3_Bucket_Key_Version]

with_name(name: str)

Return a new S3_Bucket_Key with the file name changed.

Parameters:

name (str) – The new file name.

Returns:

New S3_Bucket_Key with the modified name.

Return type:

S3_Bucket_Key

with_stem(stem: str)

Return a new S3_Bucket_Key with the file stem changed.

Parameters:

stem (str) – The new file stem (name without extension).

Returns:

New S3_Bucket_Key with the modified stem.

Return type:

S3_Bucket_Key

with_suffix(suffix: str)

Return a new S3_Bucket_Key with the file suffix changed.

Parameters:

suffix (str) – The new file extension (e.g., ‘.txt’).

Returns:

New S3_Bucket_Key with the modified suffix.

Return type:

S3_Bucket_Key