deirokay.fs.S3FileSystem

class deirokay.fs.S3FileSystem(path: Optional[str] = None, bucket: Optional[str] = None, prefix_or_key: Optional[str] = None, client: Optional[client] = None)[source]

Bases: FileSystem

FileSystem wrapper for objects stored in AWS S3.

Methods

`import_as_python_module`	Import file as a Python module.
`isdir`	Return True if the path is a directory.
`ls`	List files in a prefix or folder.
`mkdir`	Create directory using Path.mkdir method.
`open`	Open file.
`read`	Read a file as text.
`read_dict`	Read and parse a dict-like file from either YAML or JSON format.
`read_json`	Read and parse a JSON file as a Python dict.
`read_yaml`	Read and parse a YAML file as a Python dict.
`write_dict`	Serialize and write a Python dict to either YAML or JSON file.
`write_json`	Serialize and write a Python dict to a JSON file.
`write_yaml`	Serialize and write a Python dict to a YAML file.

Attributes

LIST_OBJECTS_MAX_KEYS

LIST_OBJECTS_MAX_KEYS = 1000

__truediv__(rest: str) → FileSystem

Create another FileSystem object by ‘/’-joining a FileSystem object with a string.

Parameters: rest (str) – The rest of the file path.
Returns: The same FileSystem subclass as the original object.
Return type: FileSystem
Raises: TypeError – rest should be a str.

import_as_python_module() → module[source]: Import file as a Python module.

isdir() → bool

Return True if the path is a directory.

Returns: Whether or not the path is a directory
Return type: bool
Raises: NotImplementedError – Operation not valid or not implemented.

ls(recursive: bool = False, files_only: bool = False, reverse: bool = False, limit: Optional[int] = None) → Sequence[S3FileSystem][source]

List files in a prefix or folder.

Parameters

recursive (bool, optional) – Whether or not to list subfolders recursively, by default False
files_only (bool, optional) – List only files, ignore folders/prefixes, by default False.

mkdir(*args, **kwargs)

Create directory using Path.mkdir method. Arguments are passed directly to this method.

Raises: NotImplementedError – Operator not valid or not implemented.

open(mode: Literal['r', 'w'], *args, **kwargs) → IO[source]: Open file.

read(*args, **kwargs) → str: Read a file as text.

read_dict(*args, **kwargs) → dict

Read and parse a dict-like file from either YAML or JSON format.

Returns: Python dictionary with the file content.
Return type: dict

read_json() → dict

Read and parse a JSON file as a Python dict.

Returns: Python dictionary with the file content.
Return type: dict

read_yaml() → dict

Read and parse a YAML file as a Python dict.

Returns: Python dictionary with the file content.
Return type: dict

write_dict(*args, **kwargs) → None: Serialize and write a Python dict to either YAML or JSON file.

write_json(doc: dict, **kwargs) → None

Serialize and write a Python dict to a JSON file.

Parameters: doc (dict) – The Python dict.

write_yaml(doc: dict, **kwargs) → None

Serialize and write a Python dict to a YAML file.

Parameters: doc (dict) – The Python dict.