Utility Functions¶
Utility functions useful in the implementation and testing of the Synapse client.
-
synapseclient.core.utils.
as_url
(s)¶ Tries to convert the input into a proper URL.
-
synapseclient.core.utils.
caller_module_name
(current_frame)¶ - Parameters
current_frame – use inspect.currentframe().
- Returns
the name of the module calling the function, foo(), in which this calling_module() is invoked. Ignores callers that belong in the same module as foo()
-
synapseclient.core.utils.
concrete_type_of
(obj: collections.abc.Mapping)¶ Return the concrete type of an object representing a Synapse entity. This is meant to operate either against an actual Entity object, or the lighter weight dictionary returned by Synapse#getChildren, both of which are Mappings.
-
class
synapseclient.core.utils.
deprecated_keyword_param
(keywords, version, reason)¶ A decorator to use to warn when a keyword parameter from a function has been deprecated and is intended for future removal. Will emit a warning such a keyword is passed.
-
synapseclient.core.utils.
download_file
(url, localFilepath=None)¶ Downloads a remote file.
- Parameters
localFilePath – May be None, in which case a temporary file is created
- Returns
localFilePath
-
synapseclient.core.utils.
equal_paths
(path1, path2)¶ Compare file paths in a platform neutral way
-
synapseclient.core.utils.
extract_filename
(content_disposition_header, default_filename=None)¶ Extract a filename from an HTTP content-disposition header field.
See this memo and this package for cryptic details.
-
synapseclient.core.utils.
extract_prefix
(keys)¶ Takes a list of strings and extracts a common prefix delimited by a dot, for example:
extract_prefix(["entity.bang", "entity.bar", "entity.bat"])
entity.
-
synapseclient.core.utils.
extract_synapse_id_from_query
(query)¶ An unfortunate hack to pull the synapse ID out of a table query of the form “select column1, column2 from syn12345 where….” needed to build URLs for table services.
-
synapseclient.core.utils.
extract_user_name
(profile)¶ Extract a displayable user name from a user’s profile
-
synapseclient.core.utils.
extract_zip_file_to_directory
(zip_file, zip_entry_name, target_dir)¶ Extracts a specified file in a zip to the specified directory :param zip_file: an opened zip file. e.g. “with zipfile.ZipFile(zipfilepath) as zip_file:” :param zip_entry_name: the name of the file to be extracted from the zip e.g. folderInsideZipIfAny/fileName.txt :param target_dir: the directory to which the file will be extracted
- Returns
full path to the extracted file
-
synapseclient.core.utils.
file_url_to_path
(url, verify_exists=False)¶ Convert a file URL to a path, handling some odd cases around Windows paths.
- Parameters
url – a file URL
verify_exists – If true, return an populated dict only if the resulting file path exists on the local file system.
- Returns
a path or None if the URL is not a file URL.
-
synapseclient.core.utils.
find_data_file_handle
(bundle)¶ Return the fileHandle whose ID matches the dataFileHandleId in an entity bundle
-
synapseclient.core.utils.
format_time_interval
(seconds)¶ Format a time interval given in seconds to a readable value, e.g. “5 minutes, 37 seconds”.
-
synapseclient.core.utils.
from_unix_epoch_time
(ms)¶ Returns a Datetime object given milliseconds since midnight Jan 1, 1970.
-
synapseclient.core.utils.
from_unix_epoch_time_secs
(secs)¶ Returns a Datetime object given milliseconds since midnight Jan 1, 1970.
-
synapseclient.core.utils.
get_properties
(entity)¶ Returns the dictionary of properties of the given Entity.
-
synapseclient.core.utils.
guess_file_name
(string)¶ Tries to derive a filename from an arbitrary string.
-
synapseclient.core.utils.
id_of
(obj)¶ Try to figure out the Synapse ID of the given object.
- Parameters
obj – May be a string, Entity object, or dictionary
- Returns
The ID or throws an exception
-
synapseclient.core.utils.
is_base64_encoded
(input_string)¶ Return whether the given input string appears to be base64 encoded
-
synapseclient.core.utils.
is_date
(dt)¶ Objects of class datetime.date and datetime.datetime will be recognized as dates
-
synapseclient.core.utils.
is_in_path
(id, path)¶ Determines whether id is in the path as returned from /entity/{id}/path
- Parameters
id – synapse id string
path – object as returned from ‘/entity/{id}/path’
- Returns
True or False
-
synapseclient.core.utils.
is_json
(content_type)¶ detect if a content-type is JSON
-
synapseclient.core.utils.
is_same_base_url
(url1, url2)¶ Compares two urls to see if they are the same excluding up to the base path
- Parameters
url1 – a URL
url2 – a second URL
- Returns
Boolean
-
synapseclient.core.utils.
is_synapse_id
(obj)¶ If the input is a Synapse ID return it, otherwise return None
-
synapseclient.core.utils.
is_url
(s)¶ Return True if the string appears to be a valid URL.
-
synapseclient.core.utils.
itersubclasses
(cls, _seen=None)¶ http://code.activestate.com/recipes/576949/ (r3)
itersubclasses(cls)
Generator over all subclasses of a given class, in depth first order.
>>> list(itersubclasses(int)) == [bool] True >>> class A(object): pass >>> class B(A): pass >>> class C(A): pass >>> class D(B,C): pass >>> class E(D): pass >>> >>> for cls in itersubclasses(A): ... print(cls.__name__) B D E C >>> # get ALL (new-style) classes currently defined >>> [cls.__name__ for cls in itersubclasses(object)] ['type', ...'tuple', ...]
-
synapseclient.core.utils.
make_bogus_binary_file
(n=1048576, filepath=None, printprogress=False)¶ Makes a bogus binary data file for testing. It is the caller’s responsibility to clean up the file when finished.
- Parameters
n – How many bytes to write
- Returns
The name of the file
-
synapseclient.core.utils.
make_bogus_data_file
(n=100, seed=None)¶ Makes a bogus data file for testing. It is the caller’s responsibility to clean up the file when finished.
- Parameters
n – How many random floating point numbers to be written into the file, separated by commas
seed – Random seed for the random numbers
- Returns
The name of the file
-
synapseclient.core.utils.
md5_for_file
(filename, block_size=2097152, callback=None)¶ Calculates the MD5 of the given file. See source.
- Parameters
filename – The file to read in
block_size – How much of the file to read in at once (bytes). Defaults to 2 MB
callback – The callback function that help us show loading spinner on terminal. Defaults to None
- Returns
The MD5
-
synapseclient.core.utils.
normalize_path
(path)¶ Transforms a path into an absolute path with forward slashes only.
-
synapseclient.core.utils.
normalize_whitespace
(s)¶ Strips the string and replace all whitespace sequences and other non-printable characters with a single space.
-
synapseclient.core.utils.
printTransferProgress
(transferred, toBeTransferred, prefix='', postfix='', isBytes=True, dt=None, previouslyTransferred=0)¶ Prints a progress bar
- Parameters
transferred – a number of items/bytes completed
toBeTransferred – total number of items/bytes when completed
prefix – String printed before progress bar
postfix – String printed after progress bar
isBytes – A boolean indicating whether to convert bytes to kB, MB, GB etc.
dt – The time in seconds that has passed since transfer started is used to calculate rate
previouslyTransferred – the number of bytes that were already transferred before this transfer began (e.g. someone ctrl+c’d out of an upload and restarted it later)
-
synapseclient.core.utils.
query_limit_and_offset
(query, hard_limit=1000)¶ Extract limit and offset from the end of a query string.
- Returns
A triple containing the query with limit and offset removed, the limit at most equal to the hard_limit, and the offset which defaults to 1
-
synapseclient.core.utils.
snake_case
(string)¶ Convert the given string from CamelCase to snake_case
-
synapseclient.core.utils.
threadsafe_generator
(f)¶ A decorator that takes a generator function and makes it thread-safe. See: http://anandology.com/blog/using-iterators-and-generators/
-
class
synapseclient.core.utils.
threadsafe_iter
(it)¶ Takes an iterator/generator and makes it thread-safe by serializing call to the next method of given iterator/generator. See: http://anandology.com/blog/using-iterators-and-generators/
-
synapseclient.core.utils.
to_list
(value)¶ Convert the value (an iterable or a scalar value) to a list.
-
synapseclient.core.utils.
to_unix_epoch_time
(dt)¶ Convert either datetime.date or datetime.datetime objects to UNIX time.
-
synapseclient.core.utils.
to_unix_epoch_time_secs
(dt)¶ Convert either datetime.date or datetime.datetime objects to UNIX time.
-
synapseclient.core.utils.
topolgical_sort
(graph)¶ Given a graph in the form of a dictionary returns a sorted list
Adapted from: http://blog.jupo.org/2012/04/06/topological-sorting-acyclic-directed-graphs/
- Parameters
graph – a dictionary with values containing lists of keys referencing back into the dictionary
- Returns
sorted list of items
-
synapseclient.core.utils.
touch
(path, times=None)¶ Make sure a file exists. Update its access and modified times.
-
synapseclient.core.utils.
unique_filename
(path)¶ Returns a unique path by appending (n) for some number n to the end of the filename.