Utility Functions

Utility functions useful in the implementation and testing of the Synapse client.

synapseclient.core.utils.as_url(s)

Tries to convert the input into a proper URL.

synapseclient.core.utils.caller_module_name(current_frame)
Parameters

current_frame – use inspect.currentframe().

Returns

the name of the module calling the function, foo(), in which this calling_module() is invoked. Ignores callers that belong in the same module as foo()

synapseclient.core.utils.concrete_type_of(obj: collections.abc.Mapping)

Return the concrete type of an object representing a Synapse entity. This is meant to operate either against an actual Entity object, or the lighter weight dictionary returned by Synapse#getChildren, both of which are Mappings.

class synapseclient.core.utils.deprecated_keyword_param(keywords, version, reason)

A decorator to use to warn when a keyword parameter from a function has been deprecated and is intended for future removal. Will emit a warning such a keyword is passed.

synapseclient.core.utils.download_file(url, localFilepath=None)

Downloads a remote file.

Parameters

localFilePath – May be None, in which case a temporary file is created

Returns

localFilePath

synapseclient.core.utils.equal_paths(path1, path2)

Compare file paths in a platform neutral way

synapseclient.core.utils.extract_filename(content_disposition_header, default_filename=None)

Extract a filename from an HTTP content-disposition header field.

See this memo and this package for cryptic details.

synapseclient.core.utils.extract_prefix(keys)

Takes a list of strings and extracts a common prefix delimited by a dot, for example:

extract_prefix(["entity.bang", "entity.bar", "entity.bat"])

entity.

synapseclient.core.utils.extract_synapse_id_from_query(query)

An unfortunate hack to pull the synapse ID out of a table query of the form “select column1, column2 from syn12345 where….” needed to build URLs for table services.

synapseclient.core.utils.extract_user_name(profile)

Extract a displayable user name from a user’s profile

synapseclient.core.utils.extract_zip_file_to_directory(zip_file, zip_entry_name, target_dir)

Extracts a specified file in a zip to the specified directory :param zip_file: an opened zip file. e.g. “with zipfile.ZipFile(zipfilepath) as zip_file:” :param zip_entry_name: the name of the file to be extracted from the zip e.g. folderInsideZipIfAny/fileName.txt :param target_dir: the directory to which the file will be extracted

Returns

full path to the extracted file

synapseclient.core.utils.file_url_to_path(url, verify_exists=False)

Convert a file URL to a path, handling some odd cases around Windows paths.

Parameters

  • url – a file URL

  • verify_exists – If true, return an populated dict only if the resulting file path exists on the local file system.

Returns

a path or None if the URL is not a file URL.

synapseclient.core.utils.find_data_file_handle(bundle)

Return the fileHandle whose ID matches the dataFileHandleId in an entity bundle

synapseclient.core.utils.format_time_interval(seconds)

Format a time interval given in seconds to a readable value, e.g. “5 minutes, 37 seconds”.

synapseclient.core.utils.from_unix_epoch_time(ms)

Returns a Datetime object given milliseconds since midnight Jan 1, 1970.

synapseclient.core.utils.from_unix_epoch_time_secs(secs)

Returns a Datetime object given milliseconds since midnight Jan 1, 1970.

synapseclient.core.utils.get_properties(entity)

Returns the dictionary of properties of the given Entity.

synapseclient.core.utils.guess_file_name(string)

Tries to derive a filename from an arbitrary string.

synapseclient.core.utils.id_of(obj)

Try to figure out the Synapse ID of the given object.

Parameters

obj – May be a string, Entity object, or dictionary

Returns

The ID or throws an exception

synapseclient.core.utils.is_base64_encoded(input_string)

Return whether the given input string appears to be base64 encoded

synapseclient.core.utils.is_date(dt)

Objects of class datetime.date and datetime.datetime will be recognized as dates

synapseclient.core.utils.is_in_path(id, path)

Determines whether id is in the path as returned from /entity/{id}/path

Parameters

  • id – synapse id string

  • path – object as returned from ‘/entity/{id}/path’

Returns

True or False

synapseclient.core.utils.is_json(content_type)

detect if a content-type is JSON

synapseclient.core.utils.is_same_base_url(url1, url2)

Compares two urls to see if they are the same excluding up to the base path

Parameters

  • url1 – a URL

  • url2 – a second URL

Returns

Boolean

synapseclient.core.utils.is_synapse_id(obj)

If the input is a Synapse ID return it, otherwise return None

synapseclient.core.utils.is_url(s)

Return True if the string appears to be a valid URL.

synapseclient.core.utils.itersubclasses(cls, _seen=None)

http://code.activestate.com/recipes/576949/ (r3)

itersubclasses(cls)

Generator over all subclasses of a given class, in depth first order.

>>> list(itersubclasses(int)) == [bool]
True
>>> class A(object): pass
>>> class B(A): pass
>>> class C(A): pass
>>> class D(B,C): pass
>>> class E(D): pass
>>>
>>> for cls in itersubclasses(A):
...     print(cls.__name__)
B
D
E
C
>>> # get ALL (new-style) classes currently defined
>>> [cls.__name__ for cls in itersubclasses(object)] 
['type', ...'tuple', ...]
synapseclient.core.utils.make_bogus_binary_file(n=1048576, filepath=None, printprogress=False)

Makes a bogus binary data file for testing. It is the caller’s responsibility to clean up the file when finished.

Parameters

n – How many bytes to write

Returns

The name of the file

synapseclient.core.utils.make_bogus_data_file(n=100, seed=None)

Makes a bogus data file for testing. It is the caller’s responsibility to clean up the file when finished.

Parameters

  • n – How many random floating point numbers to be written into the file, separated by commas

  • seed – Random seed for the random numbers

Returns

The name of the file

synapseclient.core.utils.md5_for_file(filename, block_size=2097152, callback=None)

Calculates the MD5 of the given file. See source.

Parameters

  • filename – The file to read in

  • block_size – How much of the file to read in at once (bytes). Defaults to 2 MB

  • callback – The callback function that help us show loading spinner on terminal. Defaults to None

Returns

The MD5

synapseclient.core.utils.normalize_path(path)

Transforms a path into an absolute path with forward slashes only.

synapseclient.core.utils.normalize_whitespace(s)

Strips the string and replace all whitespace sequences and other non-printable characters with a single space.

synapseclient.core.utils.printTransferProgress(transferred, toBeTransferred, prefix='', postfix='', isBytes=True, dt=None, previouslyTransferred=0)

Prints a progress bar

Parameters

  • transferred – a number of items/bytes completed

  • toBeTransferred – total number of items/bytes when completed

  • prefix – String printed before progress bar

  • postfix – String printed after progress bar

  • isBytes – A boolean indicating whether to convert bytes to kB, MB, GB etc.

  • dt – The time in seconds that has passed since transfer started is used to calculate rate

  • previouslyTransferred – the number of bytes that were already transferred before this transfer began (e.g. someone ctrl+c’d out of an upload and restarted it later)

synapseclient.core.utils.query_limit_and_offset(query, hard_limit=1000)

Extract limit and offset from the end of a query string.

Returns

A triple containing the query with limit and offset removed, the limit at most equal to the hard_limit, and the offset which defaults to 1

synapseclient.core.utils.snake_case(string)

Convert the given string from CamelCase to snake_case

synapseclient.core.utils.threadsafe_generator(f)

A decorator that takes a generator function and makes it thread-safe. See: http://anandology.com/blog/using-iterators-and-generators/

class synapseclient.core.utils.threadsafe_iter(it)

Takes an iterator/generator and makes it thread-safe by serializing call to the next method of given iterator/generator. See: http://anandology.com/blog/using-iterators-and-generators/

synapseclient.core.utils.to_list(value)

Convert the value (an iterable or a scalar value) to a list.

synapseclient.core.utils.to_unix_epoch_time(dt)

Convert either datetime.date or datetime.datetime objects to UNIX time.

synapseclient.core.utils.to_unix_epoch_time_secs(dt)

Convert either datetime.date or datetime.datetime objects to UNIX time.

synapseclient.core.utils.topolgical_sort(graph)

Given a graph in the form of a dictionary returns a sorted list

Adapted from: http://blog.jupo.org/2012/04/06/topological-sorting-acyclic-directed-graphs/

Parameters

graph – a dictionary with values containing lists of keys referencing back into the dictionary

Returns

sorted list of items

synapseclient.core.utils.touch(path, times=None)

Make sure a file exists. Update its access and modified times.

synapseclient.core.utils.unique_filename(path)

Returns a unique path by appending (n) for some number n to the end of the filename.