Transformations¶

TransformationsAPI¶

Create transformations¶

TransformationsAPI.create(transformation: Union[cognite.client.data_classes.transformations.Transformation, Sequence[cognite.client.data_classes.transformations.Transformation]]) → Union[cognite.client.data_classes.transformations.Transformation, cognite.client.data_classes.transformations.TransformationList]¶

Create one or more transformations.

Parameters:	transformation (Union[Transformation, List[Transformation]]) – Transformation or list of transformations to create.
Returns:	Created transformation(s)

Examples

Create new transformations:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes import Transformation, TransformationDestination
>>> from cognite.client.data_classes.transformations.common import ViewInfo, EdgeType, DataModelInfo
>>> c = CogniteClient()
>>> transformations = [
>>>     Transformation(
>>>         name="transformation1",
>>>         destination=TransformationDestination.assets()
>>>     ),
>>>     Transformation(
>>>         name="transformation2",
>>>         destination=TransformationDestination.raw("myDatabase", "myTable")
>>>     ),
>>>      Transformation(
>>>      name="transformation3",
>>>      view = ViewInfo(space="TypeSpace", external_id="TypeExtId", version="version"),
>>>      destination=TransformationDestination.nodes(view, "InstanceSpace")
>>>      ),
>>>      Transformation(
>>>      name="transformation4",
>>>      view = ViewInfo(space="TypeSpace", external_id="TypeExtId", version="version"),
>>>      destination=TransformationDestination.edges(view, "InstanceSpace")
>>>      ),
>>>      Transformation(
>>>      name="transformation5",
>>>      edge_type = EdgeType(space="TypeSpace", external_id="TypeExtId"),
>>>      destination=TransformationDestination.edges(edge_type,"InstanceSpace")
>>>      ),
>>>      Transformation(
>>>      name="transformation6",
>>>      data_model = DataModelInfo(space="modelSpace", external_id="modelExternalId",version="modelVersion",destination_type="viewExternalId"),
>>>      destination=TransformationDestination.instances(data_model,"InstanceSpace")
>>>      ),
>>>      Transformation(
>>>      name="transformation7",
>>>      data_model = DataModelInfo(space="modelSpace", external_id="modelExternalId",version="modelVersion",destination_type="viewExternalId", destination_relationship_from_type="connectionPropertyName"),
>>>      destination=TransformationDestination.instances(data_model,"InstanceSpace")
>>>      ),
>>> ]
>>> res = c.transformations.create(transformations)

Retrieve transformations by id¶

TransformationsAPI.retrieve(id: Optional[int] = None, external_id: Optional[str] = None) → Optional[cognite.client.data_classes.transformations.Transformation]¶

Retrieve a single transformation by id.

Parameters:	id (int, optional) – ID
Returns:	Requested transformation or None if it does not exist.
Return type:	Optional[Transformation]

Examples

Get transformation by id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.retrieve(id=1)

Get transformation by external id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.retrieve(external_id="1")

TransformationsAPI.retrieve_multiple(ids: Optional[Sequence[int]] = None, external_ids: Optional[Sequence[str]] = None, ignore_unknown_ids: bool = False) → cognite.client.data_classes.transformations.TransformationList¶

Retrieve multiple transformations.

Parameters:	ids (List[int]) – List of ids to retrieve. external_ids (List[str]) – List of external ids to retrieve. ignore_unknown_ids (bool) – Ignore IDs and external IDs that are not found rather than throw an exception.
Returns:	Requested transformation or None if it does not exist.
Return type:	TransformationList

Examples

Get multiple transformations:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.retrieve_multiple(ids=[1,2,3], external_ids=['transform-1','transform-2'])

Run transformations by id¶

TransformationsAPI.run(transformation_id: Optional[int] = None, transformation_external_id: Optional[str] = None, wait: bool = True, timeout: Optional[float] = None) → cognite.client.data_classes.transformations.jobs.TransformationJob¶

Run a transformation.

Parameters:

transformation_id (int) – Transformation internal id
transformation_external_id (str) – Transformation external id
wait (bool) – Wait until the transformation run is finished. Defaults to True.
timeout (Optional[float]) – maximum time (s) to wait, default is None (infinite time). Once the timeout is reached, it returns with the current status. Won’t have any effect if wait is False.

Returns:

Created transformation job

Examples

Run transformation to completion by id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> res = c.transformations.run(transformation_id = 1)

Start running transformation by id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> res = c.transformations.run(transformation_id = 1, wait = False)

TransformationsAPI.run_async(transformation_id: Optional[int] = None, transformation_external_id: Optional[str] = None, timeout: Optional[float] = None) → cognite.client.data_classes.transformations.jobs.TransformationJob¶

Run a transformation to completion asynchronously.

Parameters:	transformation_id (int) – internal Transformation id transformation_external_id (str) – external Transformation id timeout (Optional[float]) – maximum time (s) to wait, default is None (infinite time). Once the timeout is reached, it returns with the current status.
Returns:	Completed (if finished) or running (if timeout reached) transformation job.

Examples

Run transformation asyncronously by id:

>>> import asyncio
>>> from cognite.client import CogniteClient
>>>
>>> c = CogniteClient()
>>>
>>> async def run_transformation():
>>>     res = await c.transformations.run_async(id = 1)
>>>
>>> loop = asyncio.get_event_loop()
>>> loop.run_until_complete(run_transformation())
>>> loop.close()

Preview transformations¶

TransformationsAPI.preview(query: Optional[str] = None, convert_to_string: bool = False, limit: int = 100, source_limit: Optional[int] = 100, infer_schema_limit: Optional[int] = 1000) → cognite.client.data_classes.transformations.TransformationPreviewResult¶

Preview the result of a query.

Parameters:

query (str) – SQL query to run for preview.
convert_to_string (bool) – Stringify values in the query results, default is False.
limit (int) – Maximum number of rows to return in the final result, default is 100.
source_limit (Union[int,str]) – Maximum number of items to read from the data source or None to run without limit, default is 100.
infer_schema_limit – Limit for how many rows that are used for inferring result schema, default is 1000.

Returns:

Result of the executed query

Examples

Preview transformation results as schema and list of rows:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> query_result = c.transformations.preview(query="select * from _cdf.assets")

Preview transformation results as pandas dataframe:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> df = c.transformations.preview(query="select * from _cdf.assets").to_pandas()

Cancel transformation run by id¶

TransformationsAPI.cancel(transformation_id: Optional[int] = None, transformation_external_id: Optional[str] = None) → None¶

Cancel a running transformation.

Parameters:	transformation_id (int) – Transformation internal id transformation_external_id (str) – Transformation external id

Examples

Wait transformation for 1 minute and cancel if still running:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes import TransformationJobStatus
>>> c = CogniteClient()
>>>
>>> res = c.transformations.run(id = 1, timeout = 60.0)
>>> if res.status == TransformationJobStatus.RUNNING:
>>>     res.cancel()

List transformations¶

TransformationsAPI.list(include_public: bool = True, name_regex: Optional[str] = None, query_regex: Optional[str] = None, destination_type: Optional[str] = None, conflict_mode: Optional[str] = None, cdf_project_name: Optional[str] = None, has_blocked_error: Optional[bool] = None, created_time: Union[Dict[str, Any], cognite.client.data_classes.shared.TimestampRange, None] = None, last_updated_time: Union[Dict[str, Any], cognite.client.data_classes.shared.TimestampRange, None] = None, data_set_ids: Optional[List[int]] = None, data_set_external_ids: Optional[List[str]] = None, tags: Optional[cognite.client.data_classes.transformations.TagsFilter] = None, limit: Optional[int] = 25) → cognite.client.data_classes.transformations.TransformationList¶

List all transformations.

Parameters:	include_public (bool) – Whether public transformations should be included in the results. (default true). name_regex (str) – Regex expression to match the transformation name query_regex (str) – Regex expression to match the transformation query destination_type (str) – Transformation destination resource name to filter by. conflict_mode (str) – Filters by a selected transformation action type: abort/create, upsert, update, delete cdf_project_name (str) – Project name to filter by configured source and destination project has_blocked_error (bool) – Whether only the blocked transformations should be included in the results. created_time (Union[Dict[str, Any], TimestampRange]) – Range between two timestamps last_updated_time (Union[Dict[str, Any], TimestampRange]) – Range between two timestamps data_set_ids (List[int]) – Return only transformations in the specified data sets with these ids. data_set_external_ids (List[str]) – Return only transformations in the specified data sets with these external ids. tags (TagsFilter) – Return only the resource matching the specified tags constraints. It only supports ContainsAny as of now. limit (int) – Limits the number of results to be returned. To retrieve all results use limit=-1, default limit is 25.
Returns:	List of transformations
Return type:	TransformationList

Example

List transformations:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> transformations_list = c.transformations.list()

Update transformations¶

TransformationsAPI.update(item: Union[cognite.client.data_classes.transformations.Transformation, cognite.client.data_classes.transformations.TransformationUpdate, Sequence[Union[cognite.client.data_classes.transformations.Transformation, cognite.client.data_classes.transformations.TransformationUpdate]]]) → Union[cognite.client.data_classes.transformations.Transformation, cognite.client.data_classes.transformations.TransformationList]¶

Update one or more transformations

Parameters:	item (Union[Transformation, TransformationUpdate, List[Union[Transformation, TransformationUpdate]]]) – Transformation(s) to update
Returns:	Updated transformation(s)
Return type:	Union[Transformation, TransformationList]

Examples

Update a transformation that you have fetched. This will perform a full update of the transformation:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> transformation = c.transformations.retrieve(id=1)
>>> transformation.query = "SELECT * FROM _cdf.assets"
>>> res = c.transformations.update(transformation)

Perform a partial update on a transformation, updating the query and making it private:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes import TransformationUpdate
>>> c = CogniteClient()
>>> my_update = TransformationUpdate(id=1).query.set("SELECT * FROM _cdf.assets").is_public.set(False)
>>> res = c.transformations.update(my_update)

Delete transformations¶

TransformationsAPI.delete(id: Union[int, Sequence[int], None] = None, external_id: Union[str, Sequence[str], None] = None, ignore_unknown_ids: bool = False) → None¶

Delete one or more transformations.

Parameters:	id (Union[int, List[int]) – Id or list of ids. external_id (Union[str, List[str]]) – External ID or list of external ids. ignore_unknown_ids (bool) – Ignore IDs and external IDs that are not found rather than throw an exception.
Returns:	None

Example

Delete transformations by id or external id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> c.transformations.delete(id=[1,2,3], external_id="function3")

Transformation Schedules¶

Create transformation Schedules¶

TransformationSchedulesAPI.create(schedule: Union[cognite.client.data_classes.transformations.schedules.TransformationSchedule, Sequence[cognite.client.data_classes.transformations.schedules.TransformationSchedule]]) → Union[cognite.client.data_classes.transformations.schedules.TransformationSchedule, cognite.client.data_classes.transformations.schedules.TransformationScheduleList]¶

Schedule the specified transformation with the specified configuration(s).

Parameters:	schedule (Union[TransformationSchedule, Sequence[TransformationSchedule]]) – Configuration or list of configurations of the schedules to create.
Returns:	Created schedule(s)

Examples

Create new schedules:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes import TransformationSchedule
>>> c = CogniteClient()
>>> schedules = [TransformationSchedule(id = 1, interval = "0 * * * *"), TransformationSchedule(external_id="transformation2", interval = "5 * * * *"))]
>>> res = c.transformations.schedules.create(schedules)

Retrieve transformation schedules¶

TransformationSchedulesAPI.retrieve(id: Optional[int] = None, external_id: Optional[str] = None) → Optional[cognite.client.data_classes.transformations.schedules.TransformationSchedule]¶

Retrieve a single transformation schedule by the id or external id of its transformation.

Parameters:	id (int, optional) – transformation ID external_id (str, optional) – transformation External ID
Returns:	Requested transformation schedule or None if it does not exist.
Return type:	Optional[TransformationSchedule]

Examples

Get transformation schedule by transformation id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.schedules.retrieve(id=1)

Get transformation schedule by transformation external id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.schedules.retrieve(external_id="1")

Retrieve multiple transformation schedules¶

TransformationSchedulesAPI.retrieve_multiple(ids: Optional[Sequence[int]] = None, external_ids: Optional[Sequence[str]] = None, ignore_unknown_ids: bool = False) → cognite.client.data_classes.transformations.schedules.TransformationScheduleList¶

Retrieve multiple transformation schedules by the ids or external ids of the corresponding transformations.

Parameters:	ids (int, optional) – transformation IDs external_ids (str, optional) – transformation External IDs ignore_unknown_ids (bool) – Ignore IDs and external IDs that are not found rather than throw an exception.
Returns:	Requested transformation schedules.
Return type:	TransformationScheduleList

Examples

Get transformation schedules by transformation ids:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.schedules.retrieve_multiple(ids=[1, 2, 3])

Get transformation schedules by transformation external ids:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.schedules.retrieve_multiple(external_ids=["t1", "t2"])

List transformation schedules¶

TransformationSchedulesAPI.list(include_public: bool = True, limit: Optional[int] = 25) → cognite.client.data_classes.transformations.schedules.TransformationScheduleList¶

List all transformation schedules.

Parameters:	include_public (bool) – Whether public transformations should be included in the results. (default true). cursor (str) – Cursor for paging through results. limit (int) – Limits the number of results to be returned. To retrieve all results use limit=-1, default limit is 25.
Returns:	List of schedules
Return type:	TransformationScheduleList

Example

List schedules:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> schedules_list = c.transformations.schedules.list()

Update transformation schedules¶

TransformationSchedulesAPI.update(item: Union[cognite.client.data_classes.transformations.schedules.TransformationSchedule, cognite.client.data_classes.transformations.schedules.TransformationScheduleUpdate, Sequence[Union[cognite.client.data_classes.transformations.schedules.TransformationSchedule, cognite.client.data_classes.transformations.schedules.TransformationScheduleUpdate]]]) → Union[cognite.client.data_classes.transformations.schedules.TransformationSchedule, cognite.client.data_classes.transformations.schedules.TransformationScheduleList]¶

Update one or more transformation schedules

Parameters:	item (Union[TransformationSchedule, TransformationScheduleUpdate, Sequence[Union[TransformationSchedule, TransformationScheduleUpdate]]]) – Transformation schedule(s) to update
Returns:	Updated transformation schedule(s)
Return type:	Union[TransformationSchedule, TransformationScheduleList]

Examples

Update a transformation schedule that you have fetched. This will perform a full update of the schedule:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> transformation_schedule = c.transformations.schedules.retrieve(id=1)
>>> transformation_schedule.is_paused = True
>>> res = c.transformations.schedules.update(transformation_schedule)

Perform a partial update on a transformation schedule, updating the interval and unpausing it:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes import TransformationScheduleUpdate
>>> c = CogniteClient()
>>> my_update = TransformationScheduleUpdate(id=1).interval.set("0 * * * *").is_paused.set(False)
>>> res = c.transformations.schedules.update(my_update)

Delete transformation schedules¶

TransformationSchedulesAPI.delete(id: Union[int, Sequence[int], None] = None, external_id: Union[str, Sequence[str], None] = None, ignore_unknown_ids: bool = False) → None¶

Unschedule one or more transformations

Parameters:	id (Union[int, Sequence[int]) – Id or list of ids external_id (Union[str, Sequence[str]]) – External ID or list of external ids ignore_unknown_ids (bool) – Ignore IDs and external IDs that are not found rather than throw an exception.
Returns:	None

Examples

Delete schedules by id or external id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> c.transformations.schedules.delete(id=[1,2,3], external_id="3")

Transformation Notifications¶

Create transformation notifications¶

TransformationNotificationsAPI.create(notification: Union[cognite.client.data_classes.transformations.notifications.TransformationNotification, Sequence[cognite.client.data_classes.transformations.notifications.TransformationNotification]]) → Union[cognite.client.data_classes.transformations.notifications.TransformationNotification, cognite.client.data_classes.transformations.notifications.TransformationNotificationList]¶

Subscribe for notifications on the transformation errors.

Parameters:	notification (Union[TransformationNotification, Sequence[TransformationNotification]]) – Notification or list of notifications to create.
Returns:	Created notification(s)

Examples

Create new notifications:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes import TransformationNotification
>>> c = CogniteClient()
>>> notifications = [TransformationNotification(transformation_id = 1, destination="[email protected]"), TransformationNotification(transformation_external_id="transformation2", destination="[email protected]"))]
>>> res = c.transformations.notifications.create(notifications)

List transformation notifications¶

TransformationNotificationsAPI.list(transformation_id: Optional[int] = None, transformation_external_id: Optional[str] = None, destination: Optional[str] = None, limit: Optional[int] = 25) → cognite.client.data_classes.transformations.notifications.TransformationNotificationList¶

List notification subscriptions.

Parameters:	transformation_id (Optional[int]) – Filter by transformation internal numeric ID. transformation_external_id (str) – Filter by transformation externalId. destination (str) – Filter by notification destination. limit (int) – Limits the number of results to be returned. To retrieve all results use limit=-1, default limit is 25.
Returns:	List of transformation notifications
Return type:	TransformationNotificationList

Example

List all notifications:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> notifications_list = c.transformations.notifications.list()

List all notifications by transformation id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> notifications_list = c.transformations.notifications.list(transformation_id = 1)

Delete transformation notifications¶

TransformationNotificationsAPI.delete(id: Union[int, Sequence[int], None] = None) → None¶

Deletes the specified notification subscriptions on the transformation. Does nothing when the subscriptions already don’t exist

Parameters:	id (Union[int, Sequence[int]) – Id or list of transformation notification ids
Returns:	None

Examples

Delete schedules by id or external id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> c.transformations.notifications.delete(id=[1,2,3])

Transformation Jobs¶

Retrieve transformation jobs¶

TransformationJobsAPI.retrieve(id: int) → Optional[cognite.client.data_classes.transformations.jobs.TransformationJob]¶

Retrieve a single transformation job by id.

Parameters:	id (int) – Job internal Id
Returns:	Requested transformation job or None if it does not exist.
Return type:	Optional[TransformationJob]

Examples

Get transformation job by id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.jobs.retrieve(id=1)

TransformationJobsAPI.retrieve_multiple(ids: Sequence[int], ignore_unknown_ids: bool = False) → cognite.client.data_classes.transformations.jobs.TransformationJobList¶

Retrieve multiple transformation jobs by id.

Parameters:	ids (Sequence[int]) – Job internal Ids ignore_unknown_ids (bool) – Ignore IDs that are not found rather than throw an exception.
Returns:	Requested transformation jobs.
Return type:	TransformationJobList

Examples

Get jobs by id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.jobs.retrieve_multiple(ids=[1, 2, 3])

List transformation jobs¶

TransformationJobsAPI.list(limit: Optional[int] = 25, transformation_id: Optional[int] = None, transformation_external_id: Optional[str] = None) → cognite.client.data_classes.transformations.jobs.TransformationJobList¶

List all running transformation jobs.

Parameters:	limit (int) – Limits the number of results to be returned. To retrieve all results use limit=-1, default limit is 25. transformation_id (int) – Filters the results by the internal transformation id. transformation_external_id (str) – Filters the results by the external transformation id.
Returns:	List of transformation jobs
Return type:	TransformationJobList

Example

List transformation jobs:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> transformation_jobs_list = c.transformations.jobs.list()

List transformation jobs of a single transformation:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> transformation_jobs_list = c.transformations.jobs.list(transformation_id = 1)

Transformation Schema¶

Get transformation schema¶

TransformationSchemaAPI.retrieve(destination: cognite.client.data_classes.transformations.common.TransformationDestination, conflict_mode: Optional[str] = None) → cognite.client.data_classes.transformations.schema.TransformationSchemaColumnList¶

Get expected schema for a transformation destination.

Parameters:	destination (TransformationDestination) – destination for which the schema is requested. conflict_mode (Optional[str]) – conflict mode for which the schema is requested.
Returns:	List of column descriptions
Return type:	TransformationSchemaColumnList

Example

Get the schema for a transformation producing assets:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes import TransformationDestination
>>> c = CogniteClient()
>>> columns = c.transformations.schema.retrieve(destination = TransformationDestination.assets())

Data classes¶

class cognite.client.data_classes.transformations.ContainsAny(tags: Optional[List[str]] = None)¶

Bases: cognite.client.data_classes.transformations.TagsFilter

Return transformations that has one of the tags specified.

Parameters:	tags (List[str]) – The resource item contains at least one of the listed tags. The tags are defined by a list of external ids.

Examples

List only resources marked as a PUMP or as a VALVE:

>>> from cognite.client.data_classes import ContainsAny
>>> my_tag_filter = ContainsAny(tags=["PUMP", "VALVE"])

class cognite.client.data_classes.transformations.SessionDetails(session_id: Optional[int] = None, client_id: Optional[str] = None, project_name: Optional[str] = None)¶

Bases: object

Details of a source session which provid.

Parameters:	session_id (int) – CDF source session ID client_id (str) – Idp source client ID project_name (str) – CDF source project name

dump(camel_case: bool = False) → Dict[str, Any]¶

Dump the instance into a json serializable Python data type.

Parameters:	camel_case (bool) – Use camelCase for attribute names. Defaults to False.
Returns:	A dictionary representation of the instance.
Return type:	Dict[str, Any]

class cognite.client.data_classes.transformations.Transformation(id: int = None, external_id: str = None, name: str = None, query: str = None, destination: TransformationDestination = None, conflict_mode: str = None, is_public: bool = True, ignore_null_fields: bool = False, source_oidc_credentials: Optional[OidcCredentials] = None, destination_oidc_credentials: Optional[OidcCredentials] = None, created_time: Optional[int] = None, last_updated_time: Optional[int] = None, owner: str = None, owner_is_current_user: bool = True, has_source_oidc_credentials: Optional[bool] = None, has_destination_oidc_credentials: Optional[bool] = None, running_job: TransformationJob = None, last_finished_job: TransformationJob = None, blocked: TransformationBlockedInfo = None, schedule: TransformationSchedule = None, data_set_id: int = None, cognite_client: CogniteClient = None, source_nonce: Optional[NonceCredentials] = None, destination_nonce: Optional[NonceCredentials] = None, source_session: Optional[SessionDetails] = None, destination_session: Optional[SessionDetails] = None, tags: Optional[List[str]] = None)¶

Bases: cognite.client.data_classes._base.CogniteResource

The transformations resource allows transforming data in CDF.

Parameters:

id (int) – A server-generated ID for the object.
external_id (str) – The external ID provided by the client. Must be unique for the resource type.
name (str) – The name of the Transformation.
query (str) – SQL query of the transformation.
destination (TransformationDestination) – see TransformationDestination for options.
conflict_mode (str) – What to do in case of id collisions: either “abort”, “upsert”, “update” or “delete”
is_public (bool) – Indicates if the transformation is visible to all in project or only to the owner.
ignore_null_fields (bool) – Indicates how null values are handled on updates: ignore or set null.
source_oidc_credentials (Optional[OidcCredentials]) – Configures the transformation to authenticate with the given oidc credentials key on the destination.
destination_oidc_credentials (Optional[OidcCredentials]) – Configures the transformation to authenticate with the given oidc credentials on the destination.
created_time (int) – The number of milliseconds since 00:00:00 Thursday, 1 January 1970, Coordinated Universal Time (UTC), minus leap seconds.
last_updated_time (int) – The number of milliseconds since 00:00:00 Thursday, 1 January 1970, Coordinated Universal Time (UTC), minus leap seconds.
owner (str) – Owner of the transformation: requester’s identity.
owner_is_current_user (bool) – Indicates if the transformation belongs to the current user.
has_source_oidc_credentials (bool) – Indicates if the transformation is configured with a source oidc credentials set.
has_destination_oidc_credentials (bool) – Indicates if the transformation is configured with a destination oidc credentials set.
running_job (TransformationJob) – Details for the job of this transformation currently running.
last_finished_job (TransformationJob) – Details for the last finished job of this transformation.
blocked (TransformationBlockedInfo) – Provides reason and time if the transformation is blocked.
schedule (TransformationSchedule) – Details for the schedule if the transformation is scheduled.
cognite_client (CogniteClient) – The client to associate with this object.
source_nonce (NonceCredentials) – Single use credentials to bind to a CDF session for reading.
destination_nonce (NonceCredentials) – Single use credentials to bind to a CDF session for writing.
source_session (SessionDetails) – Details for the session used to read from the source project.
destination_session (SessionDetails) – Details for the session used to write to the destination project.

dump(camel_case: bool = False) → Dict[str, Any]¶

Dump the instance into a json serializable Python data type.

Parameters:	camel_case (bool) – Use camelCase for attribute names. Defaults to False.
Returns:	A dictionary representation of the instance.
Return type:	Dict[str, Any]

class cognite.client.data_classes.transformations.TransformationFilter(include_public: bool = True, name_regex: Optional[str] = None, query_regex: Optional[str] = None, destination_type: Optional[str] = None, conflict_mode: Optional[str] = None, cdf_project_name: Optional[str] = None, has_blocked_error: Optional[bool] = None, created_time: Union[Dict[str, Any], cognite.client.data_classes.shared.TimestampRange, None] = None, last_updated_time: Union[Dict[str, Any], cognite.client.data_classes.shared.TimestampRange, None] = None, data_set_ids: Optional[List[Dict[str, Any]]] = None, tags: Optional[cognite.client.data_classes.transformations.TagsFilter] = None)¶

Bases: cognite.client.data_classes._base.CogniteFilter

No description.

Parameters:

include_public (bool) – Whether public transformations should be included in the results. The default is true.
name_regex (str) – Regex expression to match the transformation name
query_regex (str) – Regex expression to match the transformation query
destination_type (str) – Transformation destination resource name to filter by.
conflict_mode (str) – Filters by a selected transformation action type: abort/create, upsert, update, delete
cdf_project_name (str) – Project name to filter by configured source and destination project
has_blocked_error (bool) – Whether only the blocked transformations should be included in the results.
created_time (Union[Dict[str, Any], TimestampRange]) – Range between two timestamps
last_updated_time (Union[Dict[str, Any], TimestampRange]) – Range between two timestamps
data_set_ids (List[Dict[str, Any]]) – Return only transformations in the specified data sets with these ids.
tags (TagsFilter) – Return only the resource matching the specified tags constraints. It only supports ContainsAny as of now.

dump(camel_case: bool = True) → Dict[str, Any]¶

Dump the instance into a json serializable Python data type.

Parameters:	camel_case (bool) – Use camelCase for attribute names. Defaults to False.
Returns:	A dictionary representation of the instance.
Return type:	Dict[str, Any]

class cognite.client.data_classes.transformations.TransformationList(resources: Collection[Any], cognite_client: CogniteClient = None)¶: Bases: cognite.client.data_classes._base.CogniteResourceList

class cognite.client.data_classes.transformations.TransformationPreviewResult(schema: TransformationSchemaColumnList = None, results: List[Dict] = None, cognite_client: CogniteClient = None)¶

Bases: cognite.client.data_classes._base.CogniteResource

Allows previewing the result of a sql transformation before executing it.

Parameters:	schema (TransformationSchemaColumnList) – List of column descriptions. results (List[Dict]) – List of resulting rows. Each row is a dictionary where the key is the column name and the value is the entrie.

dump(camel_case: bool = False) → Dict[str, Any]¶

Dump the instance into a json serializable Python data type.

Parameters:	camel_case (bool) – Use camelCase for attribute names. Defaults to False.
Returns:	A dictionary representation of the instance.
Return type:	Dict[str, Any]

class cognite.client.data_classes.transformations.TransformationUpdate(id: Optional[int] = None, external_id: Optional[str] = None)¶

Bases: cognite.client.data_classes._base.CogniteUpdate

Changes applied to transformation

Parameters:	id (int) – A server-generated ID for the object. external_id (str) – External Id provided by client. Should be unique within the project.

dump(camel_case: bool = True) → Dict[str, Any]¶

Dump the instance into a json serializable Python data type.

Returns:	A dictionary representation of the instance.
Return type:	Dict[str, Any]

class cognite.client.data_classes.transformations.schedules.TransformationSchedule(id: int = None, external_id: str = None, created_time: int = None, last_updated_time: int = None, interval: str = None, is_paused: bool = False, cognite_client: CogniteClient = None)¶

Bases: cognite.client.data_classes._base.CogniteResource

The transformation schedules resource allows running recurrent transformations.

Parameters:

id (int) – Transformation id.
external_id (str) – Transformation externalId.
created_time (int) – Time when the schedule was created.
last_updated_time (int) – Time when the schedule was last updated.
interval (str) – Cron expression describes when the function should be called. Use http://www.cronmaker.com to create a cron expression.
is_paused (bool) – If true, the transformation is not scheduled.
cognite_client (CogniteClient) – The client to associate with this object.

class cognite.client.data_classes.transformations.schedules.TransformationScheduleList(resources: Collection[Any], cognite_client: CogniteClient = None)¶: Bases: cognite.client.data_classes._base.CogniteResourceList

class cognite.client.data_classes.transformations.schedules.TransformationScheduleUpdate(id: Optional[int] = None, external_id: Optional[str] = None)¶

Bases: cognite.client.data_classes._base.CogniteUpdate

Changes applied to transformation schedule

Parameters:	id (int) – Transformation id. external_id (str) – Transformation externalId.

class cognite.client.data_classes.transformations.notifications.TransformationNotification(id: int = None, transformation_id: int = None, transformation_external_id: str = None, destination: str = None, created_time: int = None, last_updated_time: int = None, cognite_client: CogniteClient = None)¶

Bases: cognite.client.data_classes._base.CogniteResource

The transformation notification resource allows configuring email alerts on events related to a transformation run.

Parameters:

id (int) – A server-generated ID for the object.
transformation_id (int) – Transformation Id.
transformation_external_id (str) – Transformation external Id.
destination (str) – Email address where notifications should be sent.
created_time (int) – Time when the notification was created.
last_updated_time (int) – Time when the notification was last updated.
cognite_client (CogniteClient) – The client to associate with this object.

class cognite.client.data_classes.transformations.notifications.TransformationNotificationFilter(transformation_id: Optional[int] = None, transformation_external_id: Optional[str] = None, destination: Optional[str] = None)¶

Bases: cognite.client.data_classes._base.CogniteFilter

Parameters:	transformation_id (Optional[int]) – Filter by transformation internal numeric ID. transformation_external_id (str) – Filter by transformation externalId. destination (str) – Filter by notification destination.

class cognite.client.data_classes.transformations.notifications.TransformationNotificationList(resources: Collection[Any], cognite_client: CogniteClient = None)¶: Bases: cognite.client.data_classes._base.CogniteResourceList

class cognite.client.data_classes.transformations.jobs.TransformationJob(id: int = None, status: TransformationJobStatus = None, transformation_id: int = None, transformation_external_id: str = None, source_project: str = None, destination_project: str = None, destination: TransformationDestination = None, conflict_mode: str = None, query: str = None, error: str = None, ignore_null_fields: bool = False, created_time: int = None, started_time: int = None, finished_time: int = None, last_seen_time: int = None, cognite_client: CogniteClient = None)¶

Bases: cognite.client.data_classes._base.CogniteResource

The transformation job resource allows following the status of execution of a transformation run.

Parameters:

id (int) – A server-generated ID for the object.
status (TransformationJobStatus) – Status of the job.
transformation_id (int) – Server-generated ID of the transformation.
transformation_external_id (str) – external ID of the transformation.
source_project (str) – Name of the CDF project the data will be read from.
destination_project (str) – Name of the CDF project the data will be written to.
destination_type (str) – Target resource type of the transformation.
destination_database (str) – Target database if the destination type is raw.
destination_table (str) – Target table name if the destination type is RAW.
conflict_mode (str) – What to do in case of id collisions: either “abort”, “upsert”, “update” or “delete”.
query (str) – Query of the transformation that is being executed.
error (str) – Error message from the server.
ignore_null_fields (bool) – Indicates how null values are handled on updates: ignore or set null.
created_time (int) – Time when the job was created.
started_time (int) – Time when the job started running.
finished_time (int) – Time when the job finished running.
last_seen_time (int) – Time of the last status update from the job.
cognite_client (CogniteClient) – The client to associate with this object.

metrics() → cognite.client.data_classes.transformations.jobs.TransformationJobMetricList¶: Get job metrics.

update() → None¶: Get updated job status.

wait(polling_interval: float = 1, timeout: Optional[float] = None) → cognite.client.data_classes.transformations.jobs.TransformationJob¶

Waits for the job to finish.

Parameters:	polling_interval (float) – time (s) to wait between job status updates, default is one second. timeout (Optional[float]) – maximum time (s) to wait, default is None (infinite time). Once the timeout is reached, it returns with the current status.
Returns:	self.
Return type:	TransformationJob

Examples

run transformations 1 and 2 in parallel, and run 3 once they finish successfully:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> job1 = c.transformations.run(id = 1, wait = False)
>>> job2 = c.transformations.run(id = 2, wait = False)
>>> job1.wait()
>>> job2.wait()
>>> if TransformationJobStatus.FAILED not in [job1.status, job2.status]:
>>>     c.transformations.run(id = 3, wait = False)

wait transformation for 5 minutes and do something if still running:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> job = c.transformations.run(id = 1, wait = False)
>>> job.wait(timeout = 5.0*60)
>>> if job.status == TransformationJobStatus.FAILED:
>>>     # do something if job failed
>>> elif job.status == TransformationJobStatus.COMPLETED:
>>>     # do something if job completed successfully
>>> else:
>>>     # do something if job is still running

wait_async(polling_interval: float = 1, timeout: Optional[float] = None) → cognite.client.data_classes.transformations.jobs.TransformationJob¶

Asyncio coroutine, waits for the job to finish asynchronously.

Parameters:	polling_interval (float) – time (s) to wait between job status updates, default is one second. timeout (Optional[float]) – maximum time (s) to wait, default is None (infinite time). Once the timeout is reached, it returns with the current status.
Returns:	coroutine object that will finish when the job finishes and resolves to self.
Return type:	Awaitable[TransformationJob]

Examples

run transformations 1 and 2 in parallel, and run 3 once they finish successfully:

>>> from asyncio import ensure_future
>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> async def run_successive_transformations():
>>>     job1 = c.transformations.run(id = 1, wait = False)
>>>     job2 = c.transformations.run(id = 2, wait = False)
>>>     await job1.wait_async()
>>>     await job2.wait_async()
>>>     if TransformationJobStatus.FAILED not in [job1.status, job2.status]:
>>>         c.transformations.run(id = 3, wait = False)
>>>
>>> ensure_future(run_successive_transformations())

wait transformation for 5 minutes and do something if still running:

>>> from asyncio import ensure_future
>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> async def run_successive_transformations():
>>>     job = c.transformations.run(id = 1, wait = False)
>>>     await job.wait_async(timeout = 5.0*60)
>>>     if job.status == TransformationJobStatus.FAILED:
>>>         # do something if job failed
>>>     elif job.status == TransformationJobStatus.COMPLETED:
>>>         # do something if job completed successfully
>>>     else:
>>>         # do something if job is still running
>>>
>>> ensure_future(run_successive_transformations())

class cognite.client.data_classes.transformations.jobs.TransformationJobFilter(transformation_id: Optional[int] = None, transformation_external_id: Optional[str] = None)¶

Bases: cognite.client.data_classes._base.CogniteFilter

Parameters:	transformation_id (Optional[int]) – Filter jobs by transformation internal numeric ID. transformation_external_id (str) – Filter jobs by transformation external ID.

class cognite.client.data_classes.transformations.jobs.TransformationJobList(resources: Collection[Any], cognite_client: CogniteClient = None)¶: Bases: cognite.client.data_classes._base.CogniteResourceList

class cognite.client.data_classes.transformations.jobs.TransformationJobMetric(id: int = None, timestamp: int = None, name: str = None, count: int = None, cognite_client: CogniteClient = None)¶

Bases: cognite.client.data_classes._base.CogniteResource

The transformation job metric resource allows following details of execution of a transformation run.

Parameters:	timestamp (int) – Time of the last metric update. name (str) – Name of the metric. count (int) – Value of the metric. cognite_client (CogniteClient) – The client to associate with this object.

class cognite.client.data_classes.transformations.jobs.TransformationJobMetricList(resources: Collection[Any], cognite_client: CogniteClient = None)¶: Bases: cognite.client.data_classes._base.CogniteResourceList

class cognite.client.data_classes.transformations.jobs.TransformationJobStatus¶

Bases: str, enum.Enum

An enumeration.

class cognite.client.data_classes.transformations.schema.TransformationSchemaColumn(name: str = None, sql_type: str = None, type: TransformationSchemaType = None, nullable: bool = False, cognite_client: CogniteClient = None)¶

Bases: cognite.client.data_classes._base.CogniteResource

Represents a column of the expected sql structure for a destination type.

Parameters:	name (str) – Column name sql_type (str) – Type of the column in sql format. type (TransformationSchemaType) – Type of the column in json format. nullable (bool) – Values for the column can be null or not cognite_client (CogniteClient) – The client to associate with this object.

class cognite.client.data_classes.transformations.schema.TransformationSchemaColumnList(resources: Collection[Any], cognite_client: CogniteClient = None)¶: Bases: cognite.client.data_classes._base.CogniteResourceList

class cognite.client.data_classes.transformations.common.Edges(view: Optional[cognite.client.data_classes.transformations.common.ViewInfo] = None, instance_space: Optional[str] = None, edge_type: Optional[cognite.client.data_classes.transformations.common.EdgeType] = None)¶: Bases: cognite.client.data_classes.transformations.common.TransformationDestination

class cognite.client.data_classes.transformations.common.Instances(data_model: Optional[cognite.client.data_classes.transformations.common.DataModelInfo] = None, instance_space: Optional[str] = None)¶: Bases: cognite.client.data_classes.transformations.common.TransformationDestination

class cognite.client.data_classes.transformations.common.Nodes(view: Optional[cognite.client.data_classes.transformations.common.ViewInfo] = None, instance_space: Optional[str] = None)¶: Bases: cognite.client.data_classes.transformations.common.TransformationDestination

class cognite.client.data_classes.transformations.common.RawTable(database: Optional[str] = None, table: Optional[str] = None)¶: Bases: cognite.client.data_classes.transformations.common.TransformationDestination

class cognite.client.data_classes.transformations.common.SequenceRows(external_id: Optional[str] = None)¶: Bases: cognite.client.data_classes.transformations.common.TransformationDestination

class cognite.client.data_classes.transformations.common.TransformationBlockedInfo(reason: Optional[str] = None, created_time: Optional[int] = None)¶

Bases: object

Information about the reason why and when a transformation is blocked.

Parameters:	reason (str) – Reason why the transformation is blocked. created_time (Optional[int]) – Timestamp when the transformation was blocked.

class cognite.client.data_classes.transformations.common.TransformationDestination(type: Optional[str] = None)¶

Bases: object

TransformationDestination has static methods to define the target resource type of a transformation

Parameters:	type (str) – Used as data type identifier on transformation creation/retrieval.

static asset_hierarchy() → cognite.client.data_classes.transformations.common.TransformationDestination¶: To be used when the transformation is meant to produce asset hierarchies.

static assets() → cognite.client.data_classes.transformations.common.TransformationDestination¶: To be used when the transformation is meant to produce assets.

static data_sets() → cognite.client.data_classes.transformations.common.TransformationDestination¶: To be used when the transformation is meant to produce data sets.

static datapoints() → cognite.client.data_classes.transformations.common.TransformationDestination¶: To be used when the transformation is meant to produce numeric data points.

static edges(view: Optional[cognite.client.data_classes.transformations.common.ViewInfo] = None, instance_space: Optional[str] = None, edge_type: Optional[cognite.client.data_classes.transformations.common.EdgeType] = None) → cognite.client.data_classes.transformations.common.Edges¶

Parameters:	view (ViewInfo) – information of the view. instance_space (str) – space id of the instance. edge_type (EdgeType) – information about the type of the edge
Returns:	pointing to the target flexible data model.
Return type:	Edges

static events() → cognite.client.data_classes.transformations.common.TransformationDestination¶: To be used when the transformation is meant to produce events.

static files() → cognite.client.data_classes.transformations.common.TransformationDestination¶: To be used when the transformation is meant to produce files.

static instances(data_model: Optional[cognite.client.data_classes.transformations.common.DataModelInfo] = None, instance_space: Optional[str] = None) → cognite.client.data_classes.transformations.common.Instances¶

Parameters:	data_model (DataModelInfo) – information of the Data Model. instance_space (str) – space id of the instance.
Returns:	pointing to the target centric data model.
Return type:	Instances

static labels() → cognite.client.data_classes.transformations.common.TransformationDestination¶: To be used when the transformation is meant to produce labels.

static nodes(view: Optional[cognite.client.data_classes.transformations.common.ViewInfo] = None, instance_space: Optional[str] = None) → cognite.client.data_classes.transformations.common.Nodes¶

Parameters:	view (ViewInfo) – information of the view. instance_space (str) – space id of the instance.
Returns:	pointing to the target flexible data model.
Return type:	Nodes

static raw(database: str = '', table: str = '') → cognite.client.data_classes.transformations.common.RawTable¶

To be used when the transformation is meant to produce raw table rows.

Parameters:	database (str) – database name of the target raw table. table (str) – name of the target raw table
Returns:	TransformationDestination pointing to the target table

static relationships() → cognite.client.data_classes.transformations.common.TransformationDestination¶: To be used when the transformation is meant to produce relationships.

static sequence_rows(external_id: str = '') → cognite.client.data_classes.transformations.common.SequenceRows¶

To be used when the transformation is meant to produce sequence rows.

Parameters:	external_id (str) – Sequence external id.
Returns:	TransformationDestination pointing to the target sequence rows

static sequences() → cognite.client.data_classes.transformations.common.TransformationDestination¶: To be used when the transformation is meant to produce sequences.

static string_datapoints() → cognite.client.data_classes.transformations.common.TransformationDestination¶: To be used when the transformation is meant to produce string data points.

static timeseries() → cognite.client.data_classes.transformations.common.TransformationDestination¶: To be used when the transformation is meant to produce time series.