MaterializedView¶
Contained within this file are experimental interfaces for working with the Synapse Python Client. Unless otherwise noted these interfaces are subject to change at any time. Use at your own risk.
API reference¶
synapseclient.models.MaterializedView
dataclass
¶
Bases: MaterializedViewSynchronousProtocol
, AccessControllable
, TableBase
, ViewStoreMixin
, DeleteMixin
, GetMixin
, QueryMixin
A materialized view is a type of table that is automatically built from a Synapse
SQL query. Its content is read only and based off the defining_sql
attribute.
The SQL of the materialized view may contain JOIN clauses on multiple tables.
A MaterializedView
object represents this concept in Synapse:
https://rest-docs.synapse.org/rest/org/sagebionetworks/repo/model/table/MaterializedView.html
ATTRIBUTE | DESCRIPTION |
---|---|
id |
The unique immutable ID for this entity. Once issued, this ID is guaranteed to never change or be re-issued. |
name |
The name of this entity. Must be 256 characters or less. Names may only contain: letters, numbers, spaces, underscores, hyphens, periods, plus signs, apostrophes, and parentheses. |
description |
The description of this entity. Must be 1000 characters or less. |
etag |
Synapse employs an Optimistic Concurrency Control (OCC) scheme to handle concurrent updates. Since the E-Tag changes every time an entity is updated it is used to detect when a client's current representation of an entity is out-of-date. |
created_on |
The date this entity was created. |
modified_on |
The date this entity was last modified. In YYYY-MM-DD-Thh:mm:ss.sssZ format. |
created_by |
The ID of the user that created this entity. |
modified_by |
The ID of the user that last modified this entity. |
parent_id |
The ID of the Entity that is the parent of this entity. |
version_number |
The version number issued to this version on the object. |
version_label |
The version label for this entity. |
version_comment |
The version comment for this entity. |
is_latest_version |
If this is the latest version of the object. |
columns |
(Read Only) The columns of a materialized view are dynamic based on the select statement of the definingSQL. This list of columnIds is for read-only purposes.
TYPE:
|
is_search_enabled |
When creating or updating a table or view specifies if full text search should be enabled. |
defining_sql |
The synapse SQL statement that defines the data in the materialized view. |
annotations |
Additional metadata associated with the entityview. The key is
the name of your desired annotations. The value is an object containing a
list of values (use empty list to represent no values for key) and the
value type associated with all values in the list. To remove all
annotations set this to an empty dict
TYPE:
|
activity |
The Activity model represents the main record of Provenance in Synapse. |
Create a new materialized view with a defining SQL query.
from synapseclient import Synapse
from synapseclient.models import MaterializedView
syn = Synapse()
syn.login()
materialized_view = MaterializedView(
name="My Materialized View",
description="A test materialized view",
parent_id="syn12345",
defining_sql="SELECT * FROM syn67890"
)
materialized_view = materialized_view.store()
print(f"Created Materialized View with ID: {materialized_view.id}")
Update the defining SQL of an existing materialized view.
from synapseclient import Synapse
from synapseclient.models import MaterializedView
syn = Synapse()
syn.login()
materialized_view = MaterializedView(id="syn12345").get()
materialized_view.defining_sql = "SELECT column1, column2 FROM syn67890"
materialized_view = materialized_view.store()
print("Updated Materialized View defining SQL.")
Delete a materialized view.
from synapseclient import Synapse
from synapseclient.models import MaterializedView
syn = Synapse()
syn.login()
materialized_view = MaterializedView(id="syn12345")
materialized_view.delete()
print("Deleted Materialized View.")
Query data from a materialized view.
from synapseclient import Synapse
from synapseclient.models import query
syn = Synapse()
syn.login()
query_result = query("SELECT * FROM syn66080386")
print(query_result)
Retrieve and update annotations for a materialized view.
from synapseclient import Synapse
from synapseclient.models import MaterializedView
syn = Synapse()
syn.login()
materialized_view = MaterializedView(id="syn12345").get()
materialized_view.annotations["key1"] = ["value1"]
materialized_view.annotations["key2"] = ["value2"]
materialized_view.store()
print("Updated annotations for Materialized View.")
Create a materialized view with a JOIN clause.
from synapseclient import Synapse
from synapseclient.models import MaterializedView
syn = Synapse()
syn.login()
defining_sql = '''
SELECT t1.column1 AS new_column1, t2.column2 AS new_column2
FROM syn12345 t1
JOIN syn67890 t2
ON t1.id = t2.foreign_id
'''
materialized_view = MaterializedView(
name="Join Materialized View",
description="A materialized view with a JOIN clause",
parent_id="syn11111",
defining_sql=defining_sql,
)
materialized_view = materialized_view.store()
print(f"Created Materialized View with ID: {materialized_view.id}")
Create a materialized view with a LEFT JOIN clause.
from synapseclient import Synapse
from synapseclient.models import MaterializedView
syn = Synapse()
syn.login()
defining_sql = '''
SELECT t1.column1 AS new_column1, t2.column2 AS new_column2
FROM syn12345 t1
LEFT JOIN syn67890 t2
ON t1.id = t2.foreign_id
'''
materialized_view = MaterializedView(
name="Left Join Materialized View",
description="A materialized view with a LEFT JOIN clause",
parent_id="syn11111",
defining_sql=defining_sql,
)
materialized_view = materialized_view.store()
print(f"Created Materialized View with ID: {materialized_view.id}")
Create a materialized view with a RIGHT JOIN clause.
from synapseclient import Synapse
from synapseclient.models import MaterializedView
syn = Synapse()
syn.login()
defining_sql = '''
SELECT t1.column1 AS new_column1, t2.column2 AS new_column2
FROM syn12345 t1
RIGHT JOIN syn67890 t2
ON t1.id = t2.foreign_id
'''
materialized_view = MaterializedView(
name="Right Join Materialized View",
description="A materialized view with a RIGHT JOIN clause",
parent_id="syn11111",
defining_sql=defining_sql,
)
materialized_view = materialized_view.store()
print(f"Created Materialized View with ID: {materialized_view.id}")
Create a materialized view with a UNION clause.
from synapseclient import Synapse
from synapseclient.models import MaterializedView
syn = Synapse()
syn.login()
defining_sql = '''
SELECT column1 AS new_column1, column2 AS new_column2
FROM syn12345
UNION
SELECT column1 AS new_column1, column2 AS new_column2
FROM syn67890
'''
materialized_view = MaterializedView(
name="Union Materialized View",
description="A materialized view with a UNION clause",
parent_id="syn11111",
defining_sql=defining_sql,
)
materialized_view = materialized_view.store()
print(f"Created Materialized View with ID: {materialized_view.id}")
Source code in synapseclient/models/materializedview.py
316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 |
|
Functions¶
store_async
async
¶
store_async(dry_run: bool = False, *, job_timeout: int = 600, synapse_client: Optional[Synapse] = None) -> Self
Asynchronously store non-row information about a MaterializedView including the annotations.
Note: Columns in a MaterializedView are determined by the defining_sql
attribute. To update
the columns, you must update the defining_sql
and store the view.
PARAMETER | DESCRIPTION |
---|---|
dry_run
|
If True, will not actually store the table but will log to the console what would have been stored.
TYPE:
|
job_timeout
|
The maximum amount of time to wait for a job to complete.
This is used when updating the table schema. If the timeout
is reached a
TYPE:
|
synapse_client
|
If not passed in and caching was not disabled by
|
RETURNS | DESCRIPTION |
---|---|
Self
|
The MaterializedView instance stored in synapse. |
Create a new materialized view with a defining SQL query.
import asyncio
from synapseclient import Synapse
from synapseclient.models import MaterializedView
async def main():
syn = Synapse()
await syn.login_async()
materialized_view = MaterializedView(
name="My Materialized View",
description="A test materialized view",
parent_id="syn12345",
defining_sql="SELECT * FROM syn67890"
)
materialized_view = await materialized_view.store_async()
print(f"Created Materialized View with ID: {materialized_view.id}")
asyncio.run(main())
Source code in synapseclient/models/materializedview.py
730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 |
|
get_async
async
¶
get_async(include_columns: bool = True, include_activity: bool = False, *, synapse_client: Optional[Synapse] = None) -> Self
Asynchronously get the metadata about the MaterializedView from synapse.
PARAMETER | DESCRIPTION |
---|---|
include_columns
|
If True, will include fully filled column objects in the
TYPE:
|
include_activity
|
If True the activity will be included in the MaterializedView if it exists. Defaults to False.
TYPE:
|
synapse_client
|
If not passed in and caching was not disabled by
|
RETURNS | DESCRIPTION |
---|---|
Self
|
The MaterializedView instance stored in synapse. |
Retrieve a materialized view by ID.
import asyncio
from synapseclient import Synapse
from synapseclient.models import MaterializedView
async def main():
syn = Synapse()
await syn.login_async()
materialized_view = await MaterializedView(id="syn12345").get_async()
print(materialized_view)
asyncio.run(main())
Source code in synapseclient/models/materializedview.py
785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 |
|
delete_async
async
¶
Asynchronously delete the materialized view from synapse. This is not version specific. If you'd like to delete a specific version of the materialized view you must use the synapseclient.api.delete_entity function directly.
PARAMETER | DESCRIPTION |
---|---|
synapse_client
|
If not passed in and caching was not disabled by
|
RETURNS | DESCRIPTION |
---|---|
None
|
None |
Delete a materialized view.
import asyncio
from synapseclient import Synapse
from synapseclient.models import MaterializedView
async def main():
syn = Synapse()
await syn.login_async()
materialized_view = MaterializedView(id="syn12345")
await materialized_view.delete_async()
print("Deleted Materialized View.")
asyncio.run(main())
Source code in synapseclient/models/materializedview.py
832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 |
|
query_async
async
staticmethod
¶
query_async(query: str, include_row_id_and_row_version: bool = True, convert_to_datetime: bool = False, download_location=None, quote_character='"', escape_character='\\', line_end=str(linesep), separator=',', header=True, *, synapse_client: Optional[Synapse] = None, **kwargs) -> DATA_FRAME_TYPE
Query for data on a table stored in Synapse. The results will always be
returned as a Pandas DataFrame unless you specify a download_location
in which
case the results will be downloaded to that location. There are a number of
arguments that you may pass to this function depending on if you are getting
the results back as a DataFrame or downloading the results to a file.
PARAMETER | DESCRIPTION |
---|---|
query
|
The query to run. The query must be valid syntax that Synapse can understand. See this document that describes the expected syntax of the query: https://rest-docs.synapse.org/rest/org/sagebionetworks/repo/web/controller/TableExamples.html
TYPE:
|
include_row_id_and_row_version
|
If True the
TYPE:
|
convert_to_datetime
|
(DataFrame only) If set to True, will convert all Synapse DATE columns from UNIX timestamp integers into UTC datetime objects
TYPE:
|
download_location
|
(CSV Only) If set to a path the results will be downloaded to that directory. The results will be downloaded as a CSV file. A path to the downloaded file will be returned instead of a DataFrame.
DEFAULT:
|
quote_character
|
(CSV Only) The character to use to quote fields. The default is a double quote.
DEFAULT:
|
escape_character
|
(CSV Only) The character to use to escape special characters. The default is a backslash.
DEFAULT:
|
line_end
|
(CSV Only) The character to use to end a line. The default is the system's line separator. |
separator
|
(CSV Only) The character to use to separate fields. The default is a comma.
DEFAULT:
|
header
|
(CSV Only) If set to True the first row will be used as the header row. The default is True.
DEFAULT:
|
**kwargs
|
(DataFrame only) Additional keyword arguments to pass to pandas.read_csv. See https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html for complete list of supported arguments. This is exposed as internally the query downloads a CSV from Synapse and then loads it into a dataframe.
DEFAULT:
|
synapse_client
|
If not passed in and caching was not disabled by
|
RETURNS | DESCRIPTION |
---|---|
DATA_FRAME_TYPE
|
The results of the query as a Pandas DataFrame or a path to the downloaded |
DATA_FRAME_TYPE
|
query results if |
Querying for data
This example shows how you may query for data in a table and print out the results.
import asyncio
from synapseclient import Synapse
from synapseclient.models import query_async
syn = Synapse()
syn.login()
async def main():
results = await query_async(query="SELECT * FROM syn1234")
print(results)
asyncio.run(main())
Source code in synapseclient/models/mixins/table_components.py
2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 |
|
query_part_mask_async
async
staticmethod
¶
query_part_mask_async(query: str, part_mask: int, *, synapse_client: Optional[Synapse] = None) -> QueryResultBundle
Query for data on a table stored in Synapse. This is a more advanced use case
of the query
function that allows you to determine what addiitional metadata
about the table or query should also be returned. If you do not need this
additional information then you are better off using the query
function.
The query for this method uses this Rest API: https://rest-docs.synapse.org/rest/POST/entity/id/table/query/async/start.html
PARAMETER | DESCRIPTION |
---|---|
query
|
The query to run. The query must be valid syntax that Synapse can understand. See this document that describes the expected syntax of the query: https://rest-docs.synapse.org/rest/org/sagebionetworks/repo/web/controller/TableExamples.html
TYPE:
|
part_mask
|
The bitwise OR of the part mask values you want to return in the results. The following list of part masks are implemented to be returned in the results:
TYPE:
|
synapse_client
|
If not passed in and caching was not disabled by
|
RETURNS | DESCRIPTION |
---|---|
QueryResultBundle
|
The results of the query as a Pandas DataFrame. |
Querying for data with a part mask
This example shows how to use the bitwise OR
of Python to combine the
part mask values and then use that to query for data in a table and print
out the results.
In this case we are getting the results of the query, the count of rows, and the last updated on date of the table.
import asyncio
from synapseclient import Synapse
from synapseclient.models import query_part_mask_async
syn = Synapse()
syn.login()
QUERY_RESULTS = 0x1
QUERY_COUNT = 0x2
LAST_UPDATED_ON = 0x80
# Combine the part mask values using bitwise OR
part_mask = QUERY_RESULTS | QUERY_COUNT | LAST_UPDATED_ON
async def main():
result = await query_part_mask_async(query="SELECT * FROM syn1234", part_mask=part_mask)
print(result)
asyncio.run(main())
Source code in synapseclient/models/mixins/table_components.py
2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 |
|
get_permissions
¶
get_permissions(*, synapse_client: Optional[Synapse] = None) -> Permissions
Get the permissions that the caller has on an Entity.
PARAMETER | DESCRIPTION |
---|---|
synapse_client
|
If not passed in and caching was not disabled by
|
RETURNS | DESCRIPTION |
---|---|
Permissions
|
A Permissions object |
Using this function:
Getting permissions for a Synapse Entity
from synapseclient import Synapse
from synapseclient.models import File
syn = Synapse()
syn.login()
permissions = File(id="syn123").get_permissions()
Getting access types list from the Permissions object
permissions.access_types
Source code in synapseclient/models/protocols/access_control_protocol.py
18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
|
get_acl
¶
Get the ACL that a user or group has on an Entity.
PARAMETER | DESCRIPTION |
---|---|
principal_id
|
Identifier of a user or group (defaults to PUBLIC users)
TYPE:
|
synapse_client
|
If not passed in and caching was not disabled by
|
RETURNS | DESCRIPTION |
---|---|
List[str]
|
An array containing some combination of ['READ', 'UPDATE', 'CREATE', 'DELETE', 'DOWNLOAD', 'MODERATE', 'CHANGE_PERMISSIONS', 'CHANGE_SETTINGS'] or an empty array |
Source code in synapseclient/models/protocols/access_control_protocol.py
57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 |
|
set_permissions
¶
set_permissions(principal_id: int = None, access_type: List[str] = None, modify_benefactor: bool = False, warn_if_inherits: bool = True, overwrite: bool = True, *, synapse_client: Optional[Synapse] = None) -> Dict[str, Union[str, list]]
Sets permission that a user or group has on an Entity. An Entity may have its own ACL or inherit its ACL from a benefactor.
PARAMETER | DESCRIPTION |
---|---|
principal_id
|
Identifier of a user or group.
TYPE:
|
access_type
|
Type of permission to be granted. One or more of CREATE, READ, DOWNLOAD, UPDATE, DELETE, CHANGE_PERMISSIONS. Defaults to ['READ', 'DOWNLOAD'] |
modify_benefactor
|
Set as True when modifying a benefactor's ACL. The term 'benefactor' is used to indicate which Entity an Entity inherits its ACL from. For example, a newly created Project will be its own benefactor, while a new FileEntity's benefactor will start off as its containing Project. If the entity already has local sharing settings the benefactor would be itself. It may also be the immediate parent, somewhere in the parent tree, or the project itself.
TYPE:
|
warn_if_inherits
|
When
TYPE:
|
overwrite
|
By default this function overwrites existing permissions for the specified user. Set this flag to False to add new permissions non-destructively.
TYPE:
|
synapse_client
|
If not passed in and caching was not disabled by
|
RETURNS | DESCRIPTION |
---|---|
Dict[str, Union[str, list]]
|
An Access Control List object |
Setting permissions
Grant all registered users download access
from synapseclient import Synapse
from synapseclient.models import File
syn = Synapse()
syn.login()
File(id="syn123").set_permissions(principal_id=273948, access_type=['READ','DOWNLOAD'])
Grant the public view access
from synapseclient import Synapse
from synapseclient.models import File
syn = Synapse()
syn.login()
File(id="syn123").set_permissions(principal_id=273949, access_type=['READ'])
Source code in synapseclient/models/protocols/access_control_protocol.py
78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 |
|