Command Line Client

The Synapse Python Client can be used from the command line via the synapse command.

Note

The command line client is installed along with Installation of the Synapse Python client.

Usage

For help, type:

synapse -h.

For help on specific commands, type:

synapse [command] -h

usage: synapse [-h] [--version] [-u SYNAPSEUSER] [-p SYNAPSEPASSWORD] [-c CONFIGPATH] [--debug] [--silent] [-s]
               {get,manifest,sync,store,add,mv,cp,get-download-list,associate,delete,query,submit,show,cat,list,config,set-provenance,get-provenance,set-annotations,get-annotations,create,store-table,onweb,login,test-encoding,get-sts-token,migrate}
               ...

options

--version

show program’s version number and exit

-u, --username

Username used to connect to Synapse

-p, --password

Password, api key, or token used to connect to Synapse

-c, --configPath

Path to configuration file used to connect to Synapse [default: “/Users/tyu/.synapseConfig”]

Default: “/Users/tyu/.synapseConfig”

--debug

“Set to debug mode, additional output and error messages are printed to the console”

Default: False

--silent

“Set to silent mode, console output is suppressed”

Default: False

-s, --skip-checks

suppress checking for version upgrade messages and endpoint redirection

Default: False

commands

The following commands are available:

subparser

Possible choices: get, manifest, sync, store, add, mv, cp, get-download-list, associate, delete, query, submit, show, cat, list, config, set-provenance, get-provenance, set-annotations, get-annotations, create, store-table, onweb, login, test-encoding, get-sts-token, migrate

For additional help: “synapse <COMMAND> -h”

Sub-commands:

get

downloads a file from Synapse

synapse get [-h] [-q queryString] [-v VERSION] [-r] [--followLink] [--limitSearch projId] [--downloadLocation path] [--multiThreaded]
            [--manifest {all,root,suppress}]
            [local path]

Positional Arguments

local path: Synapse ID of form syn123 of desired data object.

options

-q, --query

Optional query parameter, will fetch all of the entities returned by a query (see query for help).

-v, --version

Synapse version number of entity to retrieve. Defaults to most recent version.

-r, --recursive

Fetches content in Synapse recursively contained in the parentId specified by id.

Default: False

--followLink

Determines whether the link returns the target Entity.

Default: False

--limitSearch

Synapse ID of a container such as project or folder to limit search for files if using a path.

--downloadLocation

Directory to download file to [default: “./”].

Default: “./”

--multiThreaded

Download file using a multiple threaded implementation. This flag will be removed in the future when multi-threaded download is deemed fully stable and becomes the default implementation.

Default: True

--manifest

Possible choices: all, root, suppress

Determines whether creating manifest file automatically.

Default: “all”

manifest

Generate manifest for uploading directory tree to Synapse.

synapse manifest [-h] --parent-id syn123 [--manifest-file OUTPUT] PATH

Positional Arguments

PATH: A path to a file or folder whose manifest will be generated.

options

--parent-id: Synapse ID of project or folder where to upload data.
--manifest-file: A TSV output file path where the generated manifest is stored. (default: stdout)

sync

Synchronize files described in a manifest to Synapse

synapse sync [-h] [--dryRun] [--sendMessages] [--retries INT] FILE

Positional Arguments

FILE: A tsv file with file locations and metadata to be pushed to Synapse. See https://python-docs.synapse.org/build/html/synapseutils.html#synapseutils.sync.syncToSynapse for details on the format of a manifest.

options

--dryRun

Perform validation without uploading.

Default: False

--sendMessages

Send notifications via Synapse messaging (email) at specific intervals, on errors and on completion.

Default: False

--retries

Default: 4

store

uploads and adds a file to Synapse

synapse store [-h] (--parentid syn123 | --id syn123 | --type TYPE) [--name NAME] [--description DESCRIPTION | --descriptionFile DESCRIPTION_FILE_PATH]
              [--used [target ...]] [--executed [target ...]] [--limitSearch projId] [--noForceVersion] [--annotations ANNOTATIONS] [--replace]
              [FILE]

Positional Arguments

FILE: file to be added to synapse.

options

--parentid, --parentId

Synapse ID of project or folder where to upload data (must be specified if –id is not used.

--id

Optional Id of entity in Synapse to be updated.

--type

Type of object, such as “File”, “Folder”, or “Project”, to create in Synapse. Defaults to “File”

Default: “File”

--name

Name of data object in Synapse

--description

Description of data object in Synapse.

--descriptionFile

Path to a markdown file containing description of project/folder

--used

Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) from which the specified entity is derived

--executed

Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) that was executed to generate the specified entity

--limitSearch

Synapse ID of a container such as project or folder to limit search for provenance files.

--noForceVersion

Do not force a new version to be created if the contents of the file have not changed. The default is a new version is created.

Default: False

--annotations

Annotations to add as a JSON formatted string, should evaluate to a dictionary (key/value pairs). Example: ‘{“foo”: 1, “bar”:”quux”}’

--replace

Replace all existing annotations with the given annotations

Default: False

add

uploads and adds a file to Synapse

synapse add [-h] (--parentid syn123 | --id syn123 | --type TYPE) [--name NAME] [--description DESCRIPTION | --descriptionFile DESCRIPTION_FILE_PATH]
            [--used [target ...]] [--executed [target ...]] [--limitSearch projId] [--noForceVersion] [--annotations ANNOTATIONS] [--replace]
            [FILE]

Positional Arguments

FILE: file to be added to synapse.

options

--parentid, --parentId

Synapse ID of project or folder where to upload data (must be specified if –id is not used.

--id

Optional Id of entity in Synapse to be updated.

--type

Type of object, such as “File”, “Folder”, or “Project”, to create in Synapse. Defaults to “File”

Default: “File”

--name

Name of data object in Synapse

--description

Description of data object in Synapse.

--descriptionFile

Path to a markdown file containing description of project/folder

--used

Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) from which the specified entity is derived

--executed

Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) that was executed to generate the specified entity

--limitSearch

Synapse ID of a container such as project or folder to limit search for provenance files.

--noForceVersion

Do not force a new version to be created if the contents of the file have not changed. The default is a new version is created.

Default: False

--annotations

Annotations to add as a JSON formatted string, should evaluate to a dictionary (key/value pairs). Example: ‘{“foo”: 1, “bar”:”quux”}’

--replace

Replace all existing annotations with the given annotations

Default: False

mv

Moves a file/folder in Synapse

synapse mv [-h] --id syn123 --parentid syn123

options

--id: Id of entity in Synapse to be moved.
--parentid, --parentId: Synapse ID of project or folder where file/folder will be moved

cp

Copies specific versions of synapse content such as files, folders and projects by recursively copying all sub-content

synapse cp [-h] --destinationId syn123 [--version 1] [--setProvenance traceback] [--updateExisting] [--skipCopyAnnotations] [--excludeTypes [file table ...]]
           [--skipCopyWiki]
           syn123

Positional Arguments

syn123: Id of entity in Synapse to be copied.

options

--destinationId

Synapse ID of project or folder where file will be copied to.

--version, -v

Synapse version number of File or Link to retrieve. This parameter cannot be used when copying Projects or Folders. Defaults to most recent version.

--setProvenance

Has three values to set the provenance of the copied entity-traceback: Sets to the source entityexisting: Sets to source entity’s original provenance (if it exists)None/none: No provenance is set

Default: “traceback”

--updateExisting

Will update the file if there is already a file that is named the same in the destination

Default: False

--skipCopyAnnotations

Do not copy the annotations

Default: False

--excludeTypes

Accepts a list of entity types (file, table, link) which determines which entity types to not copy.

Default: []

--skipCopyWiki

Do not copy the wiki pages

Default: False

get-download-list

Download files from the Synapse download cart

synapse get-download-list [-h] [--downloadLocation path]

options

--downloadLocation

Directory to download file to [default: “./”].

Default: “./”

associate

Associate local files with the files stored in Synapse so that calls to “synapse get” and “synapse show” don’t re-download the files but use the already existing file.

synapse associate [-h] [--limitSearch projId] [-r] path

Positional Arguments

path: local file path

options

--limitSearch

Synapse ID of a container such as project or folder to limit search to.

-r

Perform recursive association with all local files in a folder

Default: False

delete

removes a dataset from Synapse

synapse delete [-h] [--version VERSION] syn123

Positional Arguments

syn123: Synapse ID of form syn123 of desired data object

options

--version: Version number to delete of given entity.

query

Performs SQL like queries on Synapse

synapse query [-h] [string ...]

Positional Arguments

string

A query string. Note that when using the command line query strings must be passed intact as a single string. In most shells this can mean wrapping the query in quotes as appropriate and escaping any quotes that may appear within the query string itself. Example:

synapse query "select \"column has spaces\" from syn123"

See https://rest-docs.synapse.org/rest/org/sagebionetworks/repo/web/controller/TableExamples.html’ for more information

submit

submit an entity or a file for evaluation

synapse submit [-h] [--evaluationID EVALUATIONID] [--evaluationName EVALUATIONNAME] [--entity ENTITY] [--file FILE] [--parentId PARENTID] [--name NAME]
               [--teamName TEAMNAME] [--submitterAlias ALIAS] [--used [target ...]] [--executed [target ...]] [--limitSearch projId]

options

--evaluationID, --evaluationId, --evalID: Evaluation ID where the entity/file will be submitted
--evaluationName, --evalN: Evaluation Name where the entity/file will be submitted
--entity, --eid, --entityId, --id: Synapse ID of the entity to be submitted
--file, -f: File to be submitted to the challenge
--parentId, --parentid, --parent: Synapse ID of project or folder where to upload data
--name: Name of the submission
--teamName, --team: Submit of behalf of a registered team
--submitterAlias, --alias: A nickname, possibly for display in leaderboards
--used: Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) from which the specified entity is derived
--executed: Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) that was executed to generate the specified entity
--limitSearch: Synapse ID of a container such as project or folder to limit search for provenance files.

show

show metadata for an entity

synapse show [-h] [--limitSearch projId] syn123

Positional Arguments

syn123: Synapse ID of form syn123 of desired synapse object

options

--limitSearch: Synapse ID of a container such as project or folder to limit search for provenance files.

cat

prints a dataset from Synapse

synapse cat [-h] [-v VERSION] syn123

Positional Arguments

syn123: Synapse ID of form syn123 of desired data object

options

-v, --version: Synapse version number of entity to display. Defaults to most recent version.

list

List Synapse entities contained by the given Project or Folder. Note: May not be supported in future versions of the client.

synapse list [-h] [-r] [-l] [-m] syn123

Positional Arguments

syn123: Synapse ID of a project or folder

options

-r, --recursive

recursively list contents of the subtree descending from the given Synapse ID

Default: False

-l, --long

List synapse entities in long format

Default: False

-m, --modified

List modified by and modified date

Default: False

config

Create or modify a Synapse configuration file

synapse config [-h]

set-provenance

create provenance records

synapse set-provenance [-h] --id syn123 [--name NAME] [--description DESCRIPTION] [-o [OUTPUT_FILE]] [--used [target ...]] [--executed [target ...]]
                       [--limitSearch projId]

options

--id: Synapse ID of entity whose provenance we are accessing.
--name: Name of the activity that generated the entity
--description: Description of the activity that generated the entity
-o, --output: Output the provenance record in JSON format
--used: Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) from which the specified entity is derived
--executed: Synapse ID, a url, or a local file path (of a file previouslyuploaded to Synapse) that was executed to generate the specified entity
--limitSearch: Synapse ID of a container such as project or folder to limit search for provenance files.

get-provenance

show provenance records

synapse get-provenance [-h] --id syn123 [--version version] [-o [OUTPUT_FILE]]

options

--id: Synapse ID of entity whose provenance we are accessing.
--version: version of Synapse entity whose provenance we are accessing.
-o, --output: Output the provenance record in JSON format

set-annotations

create annotations records

synapse set-annotations [-h] --id syn123 --annotations ANNOTATIONS [-r]

options

--id

Synapse ID of entity whose annotations we are accessing.

--annotations

Annotations to add as a JSON formatted string, should evaluate to a dictionary (key/value pairs). Example: ‘{“foo”: 1, “bar”:”quux”}’

-r, --replace

Replace all existing annotations with the given annotations

Default: False

get-annotations

show annotations records

synapse get-annotations [-h] --id syn123 [-o [OUTPUT_FILE]]

options

--id: Synapse ID of entity whose annotations we are accessing.
-o, --output: Output the annotations record in JSON format

create

Creates folders or projects on Synapse

synapse create [-h] [--parentid syn123] --name NAME [--description DESCRIPTION | --descriptionFile DESCRIPTION_FILE_PATH] type

Positional Arguments

type: Type of object to create in synapse one of {Project, Folder}

options

--parentid, --parentId: Synapse ID of project or folder where to place folder [not used with project]
--name: Name of folder/project.
--description: Description of project/folder
--descriptionFile: Path to a markdown file containing description of project/folder

store-table

Creates a Synapse Table given a csv

synapse store-table [-h] --name NAME [--parentid syn123] [--csv foo.csv]

options

--name: Name of Table
--parentid, --parentId: Synapse ID of project
--csv: Path to csv

onweb

opens Synapse website for Entity

synapse onweb [-h] id

Positional Arguments

id: Synapse id

login

login to Synapse and (optionally) cache credentials

synapse login [-h] [-u SYNAPSEUSER] [-p SYNAPSEPASSWORD] [--rememberMe]

options

-u, --username

Username used to connect to Synapse

-p, --password

This will be deprecated. Password or api key used to connect to Synapse.

--rememberMe, --remember-me

Cache credentials for automatic authentication on future interactions with Synapse

Default: False

test-encoding

test character encoding to help diagnose problems

synapse test-encoding [-h]

get-sts-token

Get an STS token for access to AWS S3 storage underlying Synapse

synapse get-sts-token [-h] [-o {json,boto,shell,bash,cmd,powershell}] id {read_write,read_only}

Positional Arguments

id: Synapse id
permission: Possible choices: read_write, read_only

options

-o, --output

Possible choices: json, boto, shell, bash, cmd, powershell

Default: “shell”

migrate

Migrate Synapse entities to a different storage location

synapse migrate [-h] [--source_storage_location_ids [SOURCE_STORAGE_LOCATION_IDS ...]] [--file_version_strategy FILE_VERSION_STRATEGY] [--include_table_files]
                [--continue_on_error] [--csv_log_path CSV_LOG_PATH] [--dryRun] [--force]
                id dest_storage_location_id db_path

Positional Arguments

id: Synapse id
dest_storage_location_id: Destination Synapse storage location id
db_path: Local system path where a record keeping file can be stored

options

--source_storage_location_ids

Source Synapse storage location ids. If specified only files in these storage locations will be migrated.

--file_version_strategy

one of ‘new’, ‘latest’, ‘all’, ‘skip’: new creates a new version of each entity, latest migrates the most recent version, all migrates all versions, skip avoids migrating file entities (use when exclusively targeting table attached files

Default: “new”

--include_table_files

Include table attached files when migrating

Default: False

--continue_on_error

Whether to continue processing other entities if migration of one fails

Default: False

--csv_log_path

Path where to log a csv documenting the changes from the migration

--dryRun

Dry run, files will be indexed by not migrated

Default: False

--force

Bypass interactive prompt confirming migration

Default: False