Command Line Client
The Synapse Python Client can be used from the command line via the synapse command.
Note
The command line client is installed along with Installation of the Synapse Python client.
Usage
For help, type:
synapse -h.
For help on specific commands, type:
synapse [command] -h
usage: synapse [-h] [--version] [-u SYNAPSEUSER] [-p SYNAPSEPASSWORD] [-c CONFIGPATH] [--debug] [--silent] [-s]
{get,manifest,sync,store,add,mv,cp,get-download-list,associate,delete,query,submit,show,cat,list,config,set-provenance,get-provenance,set-annotations,get-annotations,create,store-table,onweb,login,test-encoding,get-sts-token,migrate}
...
options
- --version
show program’s version number and exit
- -u, --username
Username used to connect to Synapse
- -p, --password
Password, api key, or token used to connect to Synapse
- -c, --configPath
Path to configuration file used to connect to Synapse [default: “/Users/tyu/.synapseConfig”]
Default: “/Users/tyu/.synapseConfig”
- --debug
“Set to debug mode, additional output and error messages are printed to the console”
Default: False
- --silent
“Set to silent mode, console output is suppressed”
Default: False
- -s, --skip-checks
suppress checking for version upgrade messages and endpoint redirection
Default: False
commands
The following commands are available:
- subparser
Possible choices: get, manifest, sync, store, add, mv, cp, get-download-list, associate, delete, query, submit, show, cat, list, config, set-provenance, get-provenance, set-annotations, get-annotations, create, store-table, onweb, login, test-encoding, get-sts-token, migrate
For additional help: “synapse <COMMAND> -h”
Sub-commands:
get
downloads a file from Synapse
synapse get [-h] [-q queryString] [-v VERSION] [-r] [--followLink] [--limitSearch projId] [--downloadLocation path] [--multiThreaded]
[--manifest {all,root,suppress}]
[local path]
Positional Arguments
- local path
Synapse ID of form syn123 of desired data object.
options
- -q, --query
Optional query parameter, will fetch all of the entities returned by a query (see query for help).
- -v, --version
Synapse version number of entity to retrieve. Defaults to most recent version.
- -r, --recursive
Fetches content in Synapse recursively contained in the parentId specified by id.
Default: False
- --followLink
Determines whether the link returns the target Entity.
Default: False
- --limitSearch
Synapse ID of a container such as project or folder to limit search for files if using a path.
- --downloadLocation
Directory to download file to [default: “./”].
Default: “./”
- --multiThreaded
Download file using a multiple threaded implementation. This flag will be removed in the future when multi-threaded download is deemed fully stable and becomes the default implementation.
Default: True
- --manifest
Possible choices: all, root, suppress
Determines whether creating manifest file automatically.
Default: “all”
manifest
Generate manifest for uploading directory tree to Synapse.
synapse manifest [-h] --parent-id syn123 [--manifest-file OUTPUT] PATH
Positional Arguments
- PATH
A path to a file or folder whose manifest will be generated.
options
- --parent-id
Synapse ID of project or folder where to upload data.
- --manifest-file
A TSV output file path where the generated manifest is stored. (default: stdout)
sync
Synchronize files described in a manifest to Synapse
synapse sync [-h] [--dryRun] [--sendMessages] [--retries INT] FILE
Positional Arguments
- FILE
A tsv file with file locations and metadata to be pushed to Synapse. See https://python-docs.synapse.org/build/html/synapseutils.html#synapseutils.sync.syncToSynapse for details on the format of a manifest.
options
- --dryRun
Perform validation without uploading.
Default: False
- --sendMessages
Send notifications via Synapse messaging (email) at specific intervals, on errors and on completion.
Default: False
- --retries
Default: 4
store
uploads and adds a file to Synapse
synapse store [-h] (--parentid syn123 | --id syn123 | --type TYPE) [--name NAME] [--description DESCRIPTION | --descriptionFile DESCRIPTION_FILE_PATH]
[--used [target ...]] [--executed [target ...]] [--limitSearch projId] [--noForceVersion] [--annotations ANNOTATIONS] [--replace]
[FILE]
Positional Arguments
- FILE
file to be added to synapse.
options
- --parentid, --parentId
Synapse ID of project or folder where to upload data (must be specified if –id is not used.
- --id
Optional Id of entity in Synapse to be updated.
- --type
Type of object, such as “File”, “Folder”, or “Project”, to create in Synapse. Defaults to “File”
Default: “File”
- --name
Name of data object in Synapse
- --description
Description of data object in Synapse.
- --descriptionFile
Path to a markdown file containing description of project/folder
- --used
Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) from which the specified entity is derived
- --executed
Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) that was executed to generate the specified entity
- --limitSearch
Synapse ID of a container such as project or folder to limit search for provenance files.
- --noForceVersion
Do not force a new version to be created if the contents of the file have not changed. The default is a new version is created.
Default: False
- --annotations
Annotations to add as a JSON formatted string, should evaluate to a dictionary (key/value pairs). Example: ‘{“foo”: 1, “bar”:”quux”}’
- --replace
Replace all existing annotations with the given annotations
Default: False
add
uploads and adds a file to Synapse
synapse add [-h] (--parentid syn123 | --id syn123 | --type TYPE) [--name NAME] [--description DESCRIPTION | --descriptionFile DESCRIPTION_FILE_PATH]
[--used [target ...]] [--executed [target ...]] [--limitSearch projId] [--noForceVersion] [--annotations ANNOTATIONS] [--replace]
[FILE]
Positional Arguments
- FILE
file to be added to synapse.
options
- --parentid, --parentId
Synapse ID of project or folder where to upload data (must be specified if –id is not used.
- --id
Optional Id of entity in Synapse to be updated.
- --type
Type of object, such as “File”, “Folder”, or “Project”, to create in Synapse. Defaults to “File”
Default: “File”
- --name
Name of data object in Synapse
- --description
Description of data object in Synapse.
- --descriptionFile
Path to a markdown file containing description of project/folder
- --used
Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) from which the specified entity is derived
- --executed
Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) that was executed to generate the specified entity
- --limitSearch
Synapse ID of a container such as project or folder to limit search for provenance files.
- --noForceVersion
Do not force a new version to be created if the contents of the file have not changed. The default is a new version is created.
Default: False
- --annotations
Annotations to add as a JSON formatted string, should evaluate to a dictionary (key/value pairs). Example: ‘{“foo”: 1, “bar”:”quux”}’
- --replace
Replace all existing annotations with the given annotations
Default: False
mv
Moves a file/folder in Synapse
synapse mv [-h] --id syn123 --parentid syn123
options
- --id
Id of entity in Synapse to be moved.
- --parentid, --parentId
Synapse ID of project or folder where file/folder will be moved
cp
Copies specific versions of synapse content such as files, folders and projects by recursively copying all sub-content
synapse cp [-h] --destinationId syn123 [--version 1] [--setProvenance traceback] [--updateExisting] [--skipCopyAnnotations] [--excludeTypes [file table ...]]
[--skipCopyWiki]
syn123
Positional Arguments
- syn123
Id of entity in Synapse to be copied.
options
- --destinationId
Synapse ID of project or folder where file will be copied to.
- --version, -v
Synapse version number of File or Link to retrieve. This parameter cannot be used when copying Projects or Folders. Defaults to most recent version.
- --setProvenance
Has three values to set the provenance of the copied entity-traceback: Sets to the source entityexisting: Sets to source entity’s original provenance (if it exists)None/none: No provenance is set
Default: “traceback”
- --updateExisting
Will update the file if there is already a file that is named the same in the destination
Default: False
- --skipCopyAnnotations
Do not copy the annotations
Default: False
- --excludeTypes
Accepts a list of entity types (file, table, link) which determines which entity types to not copy.
Default: []
- --skipCopyWiki
Do not copy the wiki pages
Default: False
get-download-list
Download files from the Synapse download cart
synapse get-download-list [-h] [--downloadLocation path]
options
- --downloadLocation
Directory to download file to [default: “./”].
Default: “./”
associate
Associate local files with the files stored in Synapse so that calls to “synapse get” and “synapse show” don’t re-download the files but use the already existing file.
synapse associate [-h] [--limitSearch projId] [-r] path
Positional Arguments
- path
local file path
options
- --limitSearch
Synapse ID of a container such as project or folder to limit search to.
- -r
Perform recursive association with all local files in a folder
Default: False
delete
removes a dataset from Synapse
synapse delete [-h] [--version VERSION] syn123
Positional Arguments
- syn123
Synapse ID of form syn123 of desired data object
options
- --version
Version number to delete of given entity.
query
Performs SQL like queries on Synapse
synapse query [-h] [string ...]
Positional Arguments
- string
A query string. Note that when using the command line query strings must be passed intact as a single string. In most shells this can mean wrapping the query in quotes as appropriate and escaping any quotes that may appear within the query string itself. Example:
synapse query "select \"column has spaces\" from syn123"
See https://rest-docs.synapse.org/rest/org/sagebionetworks/repo/web/controller/TableExamples.html’ for more information
submit
submit an entity or a file for evaluation
synapse submit [-h] [--evaluationID EVALUATIONID] [--evaluationName EVALUATIONNAME] [--entity ENTITY] [--file FILE] [--parentId PARENTID] [--name NAME]
[--teamName TEAMNAME] [--submitterAlias ALIAS] [--used [target ...]] [--executed [target ...]] [--limitSearch projId]
options
- --evaluationID, --evaluationId, --evalID
Evaluation ID where the entity/file will be submitted
- --evaluationName, --evalN
Evaluation Name where the entity/file will be submitted
- --entity, --eid, --entityId, --id
Synapse ID of the entity to be submitted
- --file, -f
File to be submitted to the challenge
- --parentId, --parentid, --parent
Synapse ID of project or folder where to upload data
- --name
Name of the submission
- --teamName, --team
Submit of behalf of a registered team
- --submitterAlias, --alias
A nickname, possibly for display in leaderboards
- --used
Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) from which the specified entity is derived
- --executed
Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) that was executed to generate the specified entity
- --limitSearch
Synapse ID of a container such as project or folder to limit search for provenance files.
show
show metadata for an entity
synapse show [-h] [--limitSearch projId] syn123
Positional Arguments
- syn123
Synapse ID of form syn123 of desired synapse object
options
- --limitSearch
Synapse ID of a container such as project or folder to limit search for provenance files.
cat
prints a dataset from Synapse
synapse cat [-h] [-v VERSION] syn123
Positional Arguments
- syn123
Synapse ID of form syn123 of desired data object
options
- -v, --version
Synapse version number of entity to display. Defaults to most recent version.
list
List Synapse entities contained by the given Project or Folder. Note: May not be supported in future versions of the client.
synapse list [-h] [-r] [-l] [-m] syn123
Positional Arguments
- syn123
Synapse ID of a project or folder
options
- -r, --recursive
recursively list contents of the subtree descending from the given Synapse ID
Default: False
- -l, --long
List synapse entities in long format
Default: False
- -m, --modified
List modified by and modified date
Default: False
config
Create or modify a Synapse configuration file
synapse config [-h]
set-provenance
create provenance records
synapse set-provenance [-h] --id syn123 [--name NAME] [--description DESCRIPTION] [-o [OUTPUT_FILE]] [--used [target ...]] [--executed [target ...]]
[--limitSearch projId]
options
- --id
Synapse ID of entity whose provenance we are accessing.
- --name
Name of the activity that generated the entity
- --description
Description of the activity that generated the entity
- -o, --output
Output the provenance record in JSON format
- --used
Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) from which the specified entity is derived
- --executed
Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) that was executed to generate the specified entity
- --limitSearch
Synapse ID of a container such as project or folder to limit search for provenance files.
get-provenance
show provenance records
synapse get-provenance [-h] --id syn123 [--version version] [-o [OUTPUT_FILE]]
options
- --id
Synapse ID of entity whose provenance we are accessing.
- --version
version of Synapse entity whose provenance we are accessing.
- -o, --output
Output the provenance record in JSON format
set-annotations
create annotations records
synapse set-annotations [-h] --id syn123 --annotations ANNOTATIONS [-r]
options
- --id
Synapse ID of entity whose annotations we are accessing.
- --annotations
Annotations to add as a JSON formatted string, should evaluate to a dictionary (key/value pairs). Example: ‘{“foo”: 1, “bar”:”quux”}’
- -r, --replace
Replace all existing annotations with the given annotations
Default: False
get-annotations
show annotations records
synapse get-annotations [-h] --id syn123 [-o [OUTPUT_FILE]]
options
- --id
Synapse ID of entity whose annotations we are accessing.
- -o, --output
Output the annotations record in JSON format
create
Creates folders or projects on Synapse
synapse create [-h] [--parentid syn123] --name NAME [--description DESCRIPTION | --descriptionFile DESCRIPTION_FILE_PATH] type
Positional Arguments
- type
Type of object to create in synapse one of {Project, Folder}
options
- --parentid, --parentId
Synapse ID of project or folder where to place folder [not used with project]
- --name
Name of folder/project.
- --description
Description of project/folder
- --descriptionFile
Path to a markdown file containing description of project/folder
store-table
Creates a Synapse Table given a csv
synapse store-table [-h] --name NAME [--parentid syn123] [--csv foo.csv]
options
- --name
Name of Table
- --parentid, --parentId
Synapse ID of project
- --csv
Path to csv
onweb
opens Synapse website for Entity
synapse onweb [-h] id
Positional Arguments
- id
Synapse id
login
login to Synapse and (optionally) cache credentials
synapse login [-h] [-u SYNAPSEUSER] [-p SYNAPSEPASSWORD] [--rememberMe]
options
- -u, --username
Username used to connect to Synapse
- -p, --password
This will be deprecated. Password or api key used to connect to Synapse.
- --rememberMe, --remember-me
Cache credentials for automatic authentication on future interactions with Synapse
Default: False
test-encoding
test character encoding to help diagnose problems
synapse test-encoding [-h]
get-sts-token
Get an STS token for access to AWS S3 storage underlying Synapse
synapse get-sts-token [-h] [-o {json,boto,shell,bash,cmd,powershell}] id {read_write,read_only}
Positional Arguments
- id
Synapse id
- permission
Possible choices: read_write, read_only
options
- -o, --output
Possible choices: json, boto, shell, bash, cmd, powershell
Default: “shell”
migrate
Migrate Synapse entities to a different storage location
synapse migrate [-h] [--source_storage_location_ids [SOURCE_STORAGE_LOCATION_IDS ...]] [--file_version_strategy FILE_VERSION_STRATEGY] [--include_table_files]
[--continue_on_error] [--csv_log_path CSV_LOG_PATH] [--dryRun] [--force]
id dest_storage_location_id db_path
Positional Arguments
- id
Synapse id
- dest_storage_location_id
Destination Synapse storage location id
- db_path
Local system path where a record keeping file can be stored
options
- --source_storage_location_ids
Source Synapse storage location ids. If specified only files in these storage locations will be migrated.
- --file_version_strategy
- one of ‘new’, ‘latest’, ‘all’, ‘skip’
new creates a new version of each entity, latest migrates the most recent version, all migrates all versions, skip avoids migrating file entities (use when exclusively targeting table attached files
Default: “new”
- --include_table_files
Include table attached files when migrating
Default: False
- --continue_on_error
Whether to continue processing other entities if migration of one fails
Default: False
- --csv_log_path
Path where to log a csv documenting the changes from the migration
- --dryRun
Dry run, files will be indexed by not migrated
Default: False
- --force
Bypass interactive prompt confirming migration
Default: False