utils

This notebook was generated from the following notebooks:

Utils

This notebook was generated from _00_utils.ipynb)

Basic Utils

---

source

arein

 arein (vals:Iterable, refs:Iterable)

---

source

isin

 isin (val:Any, vals:Iterable)

---

source

allsametype

 allsametype (vals:Any)

---

source

allinstance

 allinstance (vals:Any, dtype:Union[Type,TypeAlias]=typing.Any)

---

source

isiter

 isiter (val:Any)

---

source

str_part_end

 str_part_end (s:str, sep:Optional[str]=' ')

---

source

str_part_mid

 str_part_mid (s:str, sep:Optional[str]=' ')

---

source

str_part_pre

 str_part_pre (s:str, sep:Optional[str]=' ')

---

source

str_split_get

 str_split_get (s:str, sep:str='_', idx:int=0)

---

source

strip_punc

 strip_punc (s:str)

---

source

str_just_numeric

 str_just_numeric (s:str)

Filters a string for just numeric values

---

source

str_just_alpha

 str_just_alpha (s:str)

Filters a string for just alpha values

Argument and Key-Word Argument Wrangling

---

source

get_func_params

 get_func_params (fn:Callable, drop_self:Optional[bool]=True,
                  drop_before:Optional[int]=0,
                  drop_idxs:Optional[List[int]]=[],
                  drop_names:Optional[List[str]]=[],
                  drop_after:Optional[int]=None)

---

source

wrangle_kwargs_for_class

 wrangle_kwargs_for_class (cls:Callable, defaults:Optional[dict]=None,
                           **kwargs:Optional[dict])

---

source

wrangle_kwargs_for_func

 wrangle_kwargs_for_func (fn:Callable, defaults:Optional[dict]=None,
                          **kwargs:Optional[dict])

---

source

filter_kwargs_for_class

 filter_kwargs_for_class (cls:Callable, **kwargs:Optional[dict])

---

source

filter_kwargs_for_func

 filter_kwargs_for_func (fn:Callable, **kwargs:Optional[dict])

---

source

copy_to_clipboard

 copy_to_clipboard (text:str)

Files

This notebook was generated from _01_files.ipynb)

PathLib Interoperabliilty

---

source

sort_directory_first

 sort_directory_first (path:Union[pathlib.Path,pathlib.PosixPath])

---

source

sort_file_first

 sort_file_first (path:Union[pathlib.Path,pathlib.PosixPath])

---

source

to_abs_expanded

 to_abs_expanded
                  (dirname:Union[str,pathlib.Path,pathlib.PosixPath,NoneTy
                  pe]=None)

---

source

path_to_str

 path_to_str (path:Union[pathlib.Path,pathlib.PosixPath])

Convert a pathlib.Path object to a string

---

source

str_to_path

 str_to_path (path:str)

Convert a string to a pathlib.Path object

---

source

is_path

 is_path (path:Any, existance:Optional[bool])

Check if the input is a path

---

source

is_pathlike

 is_pathlike (path:Any)

Check if the input is a path

---

source

is_pathlib

 is_pathlib (path:Any)

Check if the input is a path

User Utils

---

source

collapse_user

 collapse_user (path:str)

---

source

get_user

 get_user ()

Extensions

---

source

drop_ext

 drop_ext (filename:str, extension:Optional[str]=None)

---

source

check_ext

 check_ext (filename:str, extension:str)

---

source

is_tarball

 is_tarball (filename:str)

---

source

is_targz

 is_targz (filename:str)

---

source

is_gz

 is_gz (filename:str)

---

source

is_tar

 is_tar (filename:str)

---

source

get_gz_files_in_dir

 get_gz_files_in_dir (dirname:str)

---

source

filter_for_gz_files

 filter_for_gz_files (files:List[str])

Decompression

---

source

undo_gz

 undo_gz (filename:str, remove:bool=False)

/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/fastcore/docscrape.py:225: UserWarning: Unknown section Notes
  else: warn(msg)

---

source

decompress_gunzip

 decompress_gunzip (filename:str, remove:bool=False)

	Type	Default	Details
filename	str
remove	bool	False
Returns	Tuple		The name of the decompressed file e.g. ~/Downloads/fluentbio.tsv.gz would yield ~/Downloads/fluentbio.tsv

---

source

decompress_tarball

 decompress_tarball (filename:str, remove:bool=False)

	Type	Default	Details
filename	str
remove	bool	False
Returns	Tuple		The name of the archive e.g. ~/Downloads/fluentbio.tar.gz would yield ~/Downloads/fluentbio

---

source

RecursiveDecompressor

 RecursiveDecompressor (dirname:Union[pathlib.Path,pathlib.PosixPath],
                        entries:Optional[List[str]]=<factory>, strategy:Op
                        tional[Callable[[str,bool],str]]=<function
                        undo_gz>, remove:Optional[bool]=True,
                        progress:Optional[Any]=None)

Directories

---

source

dir_dirs

 dir_dirs (dirname:str)

---

source

make_missing_dirs

 make_missing_dirs (dirs:List[str])

---

source

decompress_tarball_of_gunzipped_files

 decompress_tarball_of_gunzipped_files (filename:str,
                                        desc:Optional[str]=None,
                                        remove:Optional[bool]=False)

---

source

decompress_directory_of_gunzipped_files

 decompress_directory_of_gunzipped_files (dirname:str,
                                          desc:Optional[str]=None,
                                          remove:Optional[bool]=False)

Streaming

---

source

stream_file

 stream_file (uri:str, filename:Optional[str]=None,
              desc:Optional[str]=None)

	Type	Default	Details
uri	str		The URI to download
filename	Optional	None	The fullpath name of the file to download. Defaults to ~/Downloads/os.path.basename(uri).
desc	Optional	None	The description of the tqdm progress bar. Defaults to os.path.basename(uri).
Returns	None

Downloads

---

source

download_and_decompress_tarball_of_gunzipped_files

 download_and_decompress_tarball_of_gunzipped_files (uri:str,
                                                     download_dir:str=None
                                                     , desc:Optional[str]=
                                                     None, remove:Optional
                                                     [bool]=False)

Temporary Files

---

source

make_temp_file

 make_temp_file (**kwargs:Any)

URLs

---

source

urljoin

 urljoin (*parts:str)

Slice

This notebook was generated from _02_slice.ipynb)

Slice

Helps convert slices to its numeric values

---

source

Slice

 Slice (slc:slice=<factory>)

A class for representing a slice and providing conversion to other formats.

Director

This notebook was generated from _03_directory.ipynb)

Directory Viewer

Default Functions

Init Tree

---

source

init_tree

 init_tree (dirname)

---

source

rich_init_tree

 rich_init_tree (dirname)

---

source

base_init_tree

 base_init_tree (dirname:str)

Walked Entry to Str

---

source

entry_fn

 entry_fn (path:pathlib.Path, prefix:str='', pointer:str='',
           suffix:str='')

---

source

rich_entry_fn

 rich_entry_fn (path:pathlib.Path, prefix:str='', pointer:str='',
                suffix:str='')

---

source

base_entry_fn

 base_entry_fn (path:pathlib.Path, prefix:str='', pointer:str='',
                suffix:str='')

Walk Function

---

source

walk_dir_tree

 walk_dir_tree (dirname:Union[str,pathlib.Path,pathlib.PosixPath],
                prefix:str='', hidden:Optional[bool]=False,
                tree:Optional[rich.tree.Tree]=None, entry_fn:Callable[[For
                wardRef('PathLike'),Optional[str],Optional[str],Optional[s
                tr]],str]=<function entry_fn>)

A recursive generator, given a directory Path object will yield a visual tree structure line by line with each line prefixed by the same characters Notes —– Adapted from https://stackoverflow.com/a/59109706/5623899

DirectoryTree

walks through directory

---

source

DirectoryTree

 DirectoryTree (dirname:str, hidden:Optional[bool]=False, entry_fn:Optiona
                l[Callable[[ForwardRef('PathLike'),Optional[str],Optional[
                str],Optional[str]],str]]=<function entry_fn>)

---

source

RichDirectory

 RichDirectory (dirname:str, hidden:Optional[bool]=False, entry_fn:Optiona
                l[Callable[[ForwardRef('PathLike'),Optional[str],Optional[
                str],Optional[str]],str]]=<function entry_fn>,
                console:Optional[rich.console.Console]=<factory>)

---

source

Directory

 Directory (dirname:Union[str,pathlib.Path,pathlib.PosixPath],
            hidden:Optional[bool]=False, entry_fn:Optional[Callable[[Forwa
            rdRef('PathLike'),Optional[str],Optional[str],Optional[str]],s
            tr]]=<function entry_fn>)

Filter matrix director

This notebook was generated from _04_filter_matrix_directory.ipynb)

Filter Matrix Directory Viewer

Guards

This notebook was generated from _05_guards.ipynb)

Guards

Numpy and Pandas

---

source

is_np

 is_np (arr_q:Union[list,pandas.core.series.Series,numpy.ndarray,Iterable[
        numbers.Number],Any,List[int],List[Union[bool,numpy.bool_,Literal[
        0],Literal[1]]],pandas.core.indexes.base.Index])

Checks whether or not arr_q is a ndarray

	Type	Details
arr_q	Union	object to check whether or not it is a SeriesLike.
Returns	bool

---

source

is_series_like

 is_series_like (series_q:Union[list,pandas.core.series.Series,numpy.ndarr
                 ay,Iterable[numbers.Number],Any,List[int],List[Union[bool
                 ,numpy.bool_,Literal[0],Literal[1]]],pandas.core.indexes.
                 base.Index])

Checks whether or not series_q is SeriesLike i.e. something that is probably data.

	Type	Details
series_q	Union	object to check whether or not it is a SeriesLike.
Returns	bool

---

source

is_series

 is_series (arr:Union[list,pandas.core.series.Series,numpy.ndarray,Iterabl
            e[numbers.Number],Any,List[int],List[Union[bool,numpy.bool_,Li
            teral[0],Literal[1]]],pandas.core.indexes.base.Index])

Checks whether or not arr is a pd.Series

	Type	Details
arr	Union	object to check whether or not it is a pd.Series.
Returns	bool

---

source

undo_npmatrix

 undo_npmatrix (arr:Union[list,pandas.core.series.Series,numpy.ndarray,Ite
                rable[numbers.Number],Any,List[int],List[Union[bool,numpy.
                bool_,Literal[0],Literal[1]]],pandas.core.indexes.base.Ind
                ex])

Given a tensor converts it to a numpy array

	Type	Details
arr	Union
Returns	Union

---

source

is_matrix

 is_matrix (arr:Union[list,pandas.core.series.Series,numpy.ndarray,Iterabl
            e[numbers.Number],Any,List[int],List[Union[bool,numpy.bool_,Li
            teral[0],Literal[1]]],pandas.core.indexes.base.Index])

Checks whether or not arr is a np.matrix

	Type	Details
arr	Union	object to check whether or not it is a np.matrix.
Returns	bool

/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/fastcore/docscrape.py:225: UserWarning: Unknown section See Also
  else: warn(msg)

---

source

is_torch

 is_torch (tensor_q:Union[list,pandas.core.series.Series,numpy.ndarray,Ite
           rable[numbers.Number],Any,List[int],List[Union[bool,numpy.bool_
           ,Literal[0],Literal[1]]],pandas.core.indexes.base.Index])

Alias for is_tensor.

	Type	Details
tensor_q	Union	object to check whether or not it is a pytorch tensor
Returns	bool

---

source

is_tensor

 is_tensor (tensor_q:Union[list,pandas.core.series.Series,numpy.ndarray,It
            erable[numbers.Number],Any,List[int],List[Union[bool,numpy.boo
            l_,Literal[0],Literal[1]]],pandas.core.indexes.base.Index])

Checks whether or not tensor_q is a pytorch tensor

	Type	Details
tensor_q	Union	object to check whether or not it is a pytorch tensor
Returns	bool

---

source

is_mps

 is_mps (tensor:Union[Iterable[numbers.Number],Any])

Checks whether or not tensor is on cpu

	Type	Details
tensor	Union	object to check whether or not it is on cpu.
Returns	bool

---

source

is_cpu

 is_cpu (tensor:Union[Iterable[numbers.Number],Any])

Checks whether or not tensor is on cpu

	Type	Details
tensor	Union	object to check whether or not it is on cpu.
Returns	bool

---

source

is_device

 is_device (device_q:Union[str,NoneType,Any])

Checks whether or not device_q is a valid pytorch device type.

	Type	Details
device_q	Union	object to check whether or not it is a pytorch device.
Returns	bool

---

source

to_ndarray

 to_ndarray (arr)

---

source

undo_sparse

 undo_sparse (arr:Union[list,pandas.core.series.Series,numpy.ndarray,Itera
              ble[numbers.Number],Any,List[int],List[Union[bool,numpy.bool
              _,Literal[0],Literal[1]]],pandas.core.indexes.base.Index])

Given a arr tries to make it a dense array

	Type	Details
arr	Union
Returns	Union

Torch utils

This notebook was generated from _06_torch_utils.ipynb)

Torch Utils

Adata

This notebook was generated from _07_adata.ipynb)

Adata

---

source

AdataExtractor

 AdataExtractor (adata:Any, layer:Optional[str]='X_magic',
                 x_emb:Optional[str]='X_phate',
                 dim_str:Optional[str]=None, use_hvg:Optional[bool]=True)

Ex

This notebook was generated from _09_exp.ipynb)

/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/fastcore/docscrape.py:225: UserWarning: Unknown section Arguments:
  else: warn(msg)

---

source

find_exps

 find_exps (path, params)

---

source

is_config_subset

 is_config_subset (truth, params)

---

source

setup_exp

 setup_exp (path, params, name=None)

---

source

save_exp_params

 save_exp_params (path, params, logger=None)

---

source

load_exp_params

 load_exp_params (path)

/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/fastcore/docscrape.py:225: UserWarning: potentially wrong underline length... 
Returns: 
---------- in 
Returns:
----------...
  else: warn(msg)
/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/fastcore/docscrape.py:225: UserWarning: Unknown section Returns:
  else: warn(msg)

---

source

gen_exp_name

 gen_exp_name (name=None)

/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/fastcore/docscrape.py:225: UserWarning: potentially wrong underline length... 
Notes: 
---------- in 
Notes:
----------...
  else: warn(msg)
/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/fastcore/docscrape.py:225: UserWarning: Unknown section Notes:
  else: warn(msg)

---

source

list_exps

 list_exps (path)

---

source

exp_param_filename

 exp_param_filename (path)

---

source

exp_log_filename

 exp_log_filename (path)

---

source

config_exp_logger

 config_exp_logger (path)

Archive

This notebook was generated from _10_archive.ipynb)

Directory Viewer

Archive Downloader

• Directory defined in _02_utils/_03_directory.ipynb
• ConsoleType defined in _02_utils/_03_directory.ipynb
• get_console imported in _02_utils/_03_directory.ipynb
• is_rich_available defined in _02_utils/_08_archive.ipynb
• urljoin defined in _02_utils/_01_files.ipynb
• parse_url imported in _02_utils/_01_files.ipynb

---

source

ArchiveDownloader

 ArchiveDownloader (rootdir:str, archive:str,
                    entries:Union[str,list[str]], savedir:str,
                    extract:bool=False, cleanup:bool=False,
                    compound_archive:bool=False,
                    archives:Optional[list[str]]=None,
                    console:Optional[rich.console.Console]=None,
                    progress:Optional[rich.progress.Progress]=None)

Example

from iza.static import AMAZON_BUCKET_FLUENTBIO

downloader = ArchiveDownloader(
    rootdir = AMAZON_BUCKET_FLUENTBIO,
    archive = 'public-datasets/pbmc/',
    entries = ['combined.html', 'filtered_matrix.tar.gz'],
    savedir = '~/Downloads/fluent_bio',  extract=True, cleanup=True
)
downloader.execute()

Typer