megfile.sftp module

megfile.sftp.is_sftp(path: Union[str, os.PathLike]) → bool

Test if a path is sftp path

Parameters

path – Path to be tested

Returns

True of a path is sftp path, else False

megfile.sftp.sftp_readlink(path: Union[str, os.PathLike]) → str

Return a SftpPath instance representing the path to which the symbolic link points. :param path: Given path :returns: Return a SftpPath instance representing the path to which the symbolic link points.

megfile.sftp.sftp_glob(path: Union[str, os.PathLike], recursive: bool = True, missing_ok: bool = True) → List[str]

Return path list in ascending alphabetical order, in which path matches glob pattern

1. If doesn’t match any path, return empty list

   Notice: glob.glob in standard library returns [‘a/’] instead of empty list when pathname is like a/**, recursive is True and directory ‘a’ doesn’t exist. fs_glob behaves like glob.glob in standard library under such circumstance.

2. No guarantee that each path in result is different, which means:

   Assume there exists a path /a/b/c/b/d.txt use path pattern like /**/b/**/*.txt to glob, the path above will be returned twice

3. ** will match any matched file, directory, symlink and ‘’ by default, when recursive is True

4. fs_glob returns same as glob.glob(pathname, recursive=True) in acsending alphabetical order.

5. Hidden files (filename stars with ‘.’) will not be found in the result

Parameters

• path – Given path

• pattern – Glob the given relative pattern in the directory represented by this path

• recursive – If False, ** will not search directory recursively

• missing_ok – If False and target path doesn’t match any file, raise FileNotFoundError

Returns

A list contains paths match pathname

megfile.sftp.sftp_iglob(path: Union[str, os.PathLike], recursive: bool = True, missing_ok: bool = True) → Iterator[str]

Return path iterator in ascending alphabetical order, in which path matches glob pattern

1. If doesn’t match any path, return empty list

   Notice: glob.glob in standard library returns [‘a/’] instead of empty list when pathname is like a/**, recursive is True and directory ‘a’ doesn’t exist. fs_glob behaves like glob.glob in standard library under such circumstance.

2. No guarantee that each path in result is different, which means:

   Assume there exists a path /a/b/c/b/d.txt use path pattern like /**/b/**/*.txt to glob, the path above will be returned twice

3. ** will match any matched file, directory, symlink and ‘’ by default, when recursive is True

4. fs_glob returns same as glob.glob(pathname, recursive=True) in acsending alphabetical order.

5. Hidden files (filename stars with ‘.’) will not be found in the result

Parameters

• path – Given path

• pattern – Glob the given relative pattern in the directory represented by this path

• recursive – If False, ** will not search directory recursively

• missing_ok – If False and target path doesn’t match any file, raise FileNotFoundError

Returns

An iterator contains paths match pathname

megfile.sftp.sftp_glob_stat(path: Union[str, os.PathLike], recursive: bool = True, missing_ok: bool = True) → Iterator[megfile.pathlike.FileEntry]

Return a list contains tuples of path and file stat, in ascending alphabetical order, in which path matches glob pattern

1. If doesn’t match any path, return empty list

   Notice: glob.glob in standard library returns [‘a/’] instead of empty list when pathname is like a/**, recursive is True and directory ‘a’ doesn’t exist. sftp_glob behaves like glob.glob in standard library under such circumstance.

2. No guarantee that each path in result is different, which means:

   Assume there exists a path /a/b/c/b/d.txt use path pattern like /**/b/**/*.txt to glob, the path above will be returned twice

3. ** will match any matched file, directory, symlink and ‘’ by default, when recursive is True

4. fs_glob returns same as glob.glob(pathname, recursive=True) in acsending alphabetical order.

5. Hidden files (filename stars with ‘.’) will not be found in the result

Parameters

• path – Given path

• pattern – Glob the given relative pattern in the directory represented by this path

• recursive – If False, ** will not search directory recursively

• missing_ok – If False and target path doesn’t match any file, raise FileNotFoundError

Returns

A list contains tuples of path and file stat, in which paths match pathname

megfile.sftp.sftp_resolve(path: Union[str, os.PathLike], strict=False) → str

Equal to fs_realpath

Parameters

• path – Given path

• strict – Ignore this parameter, just for compatibility

Returns

Return the canonical path of the specified filename, eliminating any symbolic links encountered in the path.

Return type

SftpPath

megfile.sftp.sftp_download(src_url: Union[str, os.PathLike], dst_url: Union[str, os.PathLike], callback: Optional[Callable[int, None]] = None, followlinks: bool = False)

File download

megfile.sftp.sftp_upload(src_url: Union[str, os.PathLike], dst_url: Union[str, os.PathLike], callback: Optional[Callable[int, None]] = None, followlinks: bool = False)

File upload

megfile.sftp.sftp_path_join(path: Union[str, os.PathLike], *other_paths: Union[str, os.PathLike]) → str

Concat 2 or more path to a complete path

Parameters

• path – Given path

• other_paths – Paths to be concatenated

Returns

Concatenated complete path

Note

The difference between this function and os.path.join is that this function ignores left side slash (which indicates absolute path) in other_paths and will directly concat. e.g. os.path.join(‘/path’, ‘to’, ‘/file’) => ‘/file’, but sftp_path_join(‘/path’, ‘to’, ‘/file’) => ‘/path/to/file’

megfile.sftp.sftp_concat(src_paths: List[Union[str, os.PathLike]], dst_path: Union[str, os.PathLike]) → None

Concatenate sftp files to one file.

Parameters

• src_paths – Given source paths

• dst_path – Given destination path

megfile.sftp.sftp_exists(path: Union[str, os.PathLike], followlinks: bool = False) → bool

Test if the path exists

Parameters

• path – Given path

• followlinks – False if regard symlink as file, else True

Returns

True if the path exists, else False

megfile.sftp.sftp_getmtime(path: Union[str, os.PathLike], follow_symlinks: bool = False) → float

Get last-modified time of the file on the given path (in Unix timestamp format). If the path is an existent directory, return the latest modified time of all file in it.

Parameters

path – Given path

Returns

last-modified time

megfile.sftp.sftp_getsize(path: Union[str, os.PathLike], follow_symlinks: bool = False) → int

Get file size on the given file path (in bytes). If the path in a directory, return the sum of all file size in it, including file in subdirectories (if exist). The result excludes the size of directory itself. In other words, return 0 Byte on an empty directory path.

Parameters

path – Given path

Returns

File size

megfile.sftp.sftp_isdir(path: Union[str, os.PathLike], followlinks: bool = False) → bool

Test if a path is directory

Note

The difference between this function and os.path.isdir is that this function regard symlink as file

Parameters

• path – Given path

• followlinks – False if regard symlink as file, else True

Returns

True if the path is a directory, else False

megfile.sftp.sftp_isfile(path: Union[str, os.PathLike], followlinks: bool = False) → bool

Test if a path is file

Note

The difference between this function and os.path.isfile is that this function regard symlink as file

Parameters

• path – Given path

• followlinks – False if regard symlink as file, else True

Returns

True if the path is a file, else False

megfile.sftp.sftp_listdir(path: Union[str, os.PathLike]) → List[str]

Get all contents of given sftp path. The result is in acsending alphabetical order.

Parameters

path – Given path

Returns

All contents have in the path in acsending alphabetical order

megfile.sftp.sftp_load_from(path: Union[str, os.PathLike]) → BinaryIO

Read all content on specified path and write into memory

User should close the BinaryIO manually

Parameters

path – Given path

Returns

Binary stream

megfile.sftp.sftp_makedirs(path: Union[str, os.PathLike], mode=511, parents: bool = False, exist_ok: bool = False)

make a directory on sftp, including parent directory

If there exists a file on the path, raise FileExistsError

Parameters

• path – Given path

• mode – If mode is given, it is combined with the process’ umask value to determine the file mode and access flags.

• parents – If parents is true, any missing parents of this path are created as needed;

If parents is false (the default), a missing parent raises FileNotFoundError. :param exist_ok: If False and target directory exists, raise FileExistsError :raises: FileExistsError

megfile.sftp.sftp_realpath(path: Union[str, os.PathLike]) → str

Return the real path of given path

Parameters

path – Given path

Returns

Real path of given path

megfile.sftp.sftp_rename(src_path: Union[str, os.PathLike], dst_path: Union[str, os.PathLike]) → megfile.sftp_path.SftpPath

rename file on sftp

Parameters

• src_path – Given path

• dst_path – Given destination path

megfile.sftp.sftp_move(src_path: Union[str, os.PathLike], dst_path: Union[str, os.PathLike]) → megfile.sftp_path.SftpPath

move file on sftp

Parameters

• src_path – Given path

• dst_path – Given destination path

megfile.sftp.sftp_remove(path: Union[str, os.PathLike], missing_ok: bool = False) → None

Remove the file or directory on sftp

Parameters

• path – Given path

• missing_ok – if False and target file/directory not exists, raise FileNotFoundError

megfile.sftp.sftp_scan(path: Union[str, os.PathLike], missing_ok: bool = True, followlinks: bool = False) → Iterator[str]

Iteratively traverse only files in given directory, in alphabetical order. Every iteration on generator yields a path string.

If path is a file path, yields the file only If path is a non-existent path, return an empty generator If path is a bucket path, return all file paths in the bucket

Parameters

• path – Given path

• missing_ok – If False and there’s no file in the directory, raise FileNotFoundError

Returns

A file path generator

megfile.sftp.sftp_scan_stat(path: Union[str, os.PathLike], missing_ok: bool = True, followlinks: bool = False) → Iterator[megfile.pathlike.FileEntry]

Iteratively traverse only files in given directory, in alphabetical order. Every iteration on generator yields a tuple of path string and file stat

Parameters

• path – Given path

• missing_ok – If False and there’s no file in the directory, raise FileNotFoundError

Returns

A file path generator

megfile.sftp.sftp_scandir(path: Union[str, os.PathLike]) → Iterator[megfile.pathlike.FileEntry]

Get all content of given file path.

Parameters

path – Given path

Returns

An iterator contains all contents have prefix path

megfile.sftp.sftp_stat(path: Union[str, os.PathLike], follow_symlinks=True) → megfile.pathlike.StatResult

Get StatResult of file on sftp, including file size and mtime, referring to fs_getsize and fs_getmtime

Parameters

path – Given path

Returns

StatResult

megfile.sftp.sftp_lstat(path: Union[str, os.PathLike]) → megfile.pathlike.StatResult

Get StatResult of file on sftp, including file size and mtime, referring to fs_getsize and fs_getmtime

Parameters

path – Given path

Returns

StatResult

megfile.sftp.sftp_unlink(path: Union[str, os.PathLike], missing_ok: bool = False) → None

Remove the file on sftp

Parameters

• path – Given path

• missing_ok – if False and target file not exists, raise FileNotFoundError

megfile.sftp.sftp_walk(path: Union[str, os.PathLike], followlinks: bool = False) → Iterator[Tuple[str, List[str], List[str]]]

Generate the file names in a directory tree by walking the tree top-down. For each directory in the tree rooted at directory path (including path itself), it yields a 3-tuple (root, dirs, files).

root: a string of current path dirs: name list of subdirectories (excluding ‘.’ and ‘..’ if they exist) in ‘root’. The list is sorted by ascending alphabetical order files: name list of non-directory files (link is regarded as file) in ‘root’. The list is sorted by ascending alphabetical order

If path not exists, or path is a file (link is regarded as file), return an empty generator

Note

Be aware that setting followlinks to True can lead to infinite recursion if a link points to a parent directory of itself. fs_walk() does not keep track of the directories it visited already.

Parameters

• path – Given path

• followlinks – False if regard symlink as file, else True

Returns

A 3-tuple generator

megfile.sftp.sftp_getmd5(path: Union[str, os.PathLike], recalculate: bool = False, followlinks: bool = True)

Calculate the md5 value of the file

Parameters

• path – Given path

• recalculate – Ignore this parameter, just for compatibility

• followlinks – Ignore this parameter, just for compatibility

returns: md5 of file

megfile.sftp.sftp_symlink(src_path: Union[str, os.PathLike], dst_path: Union[str, os.PathLike]) → None

Create a symbolic link pointing to src_path named dst_path.

Parameters

• src_path – Given path

• dst_path – Desination path

megfile.sftp.sftp_islink(path: Union[str, os.PathLike]) → bool

Test whether a path is a symbolic link

Parameters

path – Given path

Returns

If path is a symbolic link return True, else False

Return type

bool

megfile.sftp.sftp_save_as(file_object: BinaryIO, path: Union[str, os.PathLike])

Write the opened binary stream to path If parent directory of path doesn’t exist, it will be created.

Parameters

• path – Given path

• file_object – stream to be read

megfile.sftp.sftp_open(path: Union[str, os.PathLike], mode: str = 'r', buffering=-1, encoding: Optional[str] = None, errors: Optional[str] = None, **kwargs) → IO[AnyStr]

Open a file on the path.

Parameters

• path – Given path

• mode – Mode to open file

• buffering – buffering is an optional integer used to set the buffering policy.

• encoding – encoding is the name of the encoding used to decode or encode the file. This should only be used in text mode.

• errors – errors is an optional string that specifies how encoding and decoding errors are to be handled—this cannot be used in binary mode.

Returns

File-Like object

megfile.sftp.sftp_chmod(path: Union[str, os.PathLike], mode: int, follow_symlinks: bool = True)

Change the file mode and permissions, like os.chmod().

Parameters

• path – Given path

• mode – the file mode you want to change

• followlinks – Ignore this parameter, just for compatibility

megfile.sftp.sftp_absolute(path: Union[str, os.PathLike]) → megfile.sftp_path.SftpPath

Make the path absolute, without normalization or resolving symlinks. Returns a new path object

megfile.sftp.sftp_rmdir(path: Union[str, os.PathLike])

Remove this directory. The directory must be empty.

megfile.sftp.sftp_copy(src_path: Union[str, os.PathLike], dst_path: Union[str, os.PathLike], callback: Optional[Callable[int, None]] = None, followlinks: bool = False)

Copy the file to the given destination path.

Parameters

• src_path – Given path

• dst_path – The destination path to copy the file to.

• callback – An optional callback function that takes an integer parameter and is called periodically during the copy operation to report the number of bytes copied.

• followlinks – Whether to follow symbolic links when copying directories.

Raises

• IsADirectoryError – If the source is a directory.

• OSError – If there is an error copying the file.

megfile.sftp.sftp_sync(src_path: Union[str, os.PathLike], dst_path: Union[str, os.PathLike], followlinks: bool = False, force: bool = False)

Copy file/directory on src_url to dst_url

Parameters

• src_path – Given path

• dst_url – Given destination path

• followlinks – False if regard symlink as file, else True

• force – Sync file forcely, do not ignore same files