Transformations

TransformationsAPI

Create transformations

TransformationsAPI.create(transformation: Transformation | Sequence[Transformation]) → Transformation | TransformationList

Create one or more transformations.

Parameters: transformation (Transformation | Sequence[Transformation]) – Transformation or list of transformations to create.
Returns: Created transformation(s)
Return type: Transformation | TransformationList

Examples

Create new transformations:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes import Transformation, TransformationDestination
>>> from cognite.client.data_classes.transformations.common import ViewInfo, EdgeType, DataModelInfo
>>> c = CogniteClient()
>>> transformations = [
>>>     Transformation(
>>>         name="transformation1",
>>>         destination=TransformationDestination.assets()
>>>     ),
>>>     Transformation(
>>>         name="transformation2",
>>>         destination=TransformationDestination.raw("myDatabase", "myTable")
>>>     ),
>>>      Transformation(
>>>      name="transformation3",
>>>      view = ViewInfo(space="TypeSpace", external_id="TypeExtId", version="version"),
>>>      destination=TransformationDestination.nodes(view, "InstanceSpace")
>>>      ),
>>>      Transformation(
>>>      name="transformation4",
>>>      view = ViewInfo(space="TypeSpace", external_id="TypeExtId", version="version"),
>>>      destination=TransformationDestination.edges(view, "InstanceSpace")
>>>      ),
>>>      Transformation(
>>>      name="transformation5",
>>>      edge_type = EdgeType(space="TypeSpace", external_id="TypeExtId"),
>>>      destination=TransformationDestination.edges(edge_type,"InstanceSpace")
>>>      ),
>>>      Transformation(
>>>      name="transformation6",
>>>      data_model = DataModelInfo(space="modelSpace", external_id="modelExternalId",version="modelVersion",destination_type="viewExternalId"),
>>>      destination=TransformationDestination.instances(data_model,"InstanceSpace")
>>>      ),
>>>      Transformation(
>>>      name="transformation7",
>>>      data_model = DataModelInfo(space="modelSpace", external_id="modelExternalId",version="modelVersion",destination_type="viewExternalId", destination_relationship_from_type="connectionPropertyName"),
>>>      destination=TransformationDestination.instances(data_model,"InstanceSpace")
>>>      ),
>>> ]
>>> res = c.transformations.create(transformations)

Retrieve transformations by id

TransformationsAPI.retrieve(id: int | None = None, external_id: str | None = None) → Transformation | None

Retrieve a single transformation by id.

Parameters

id (int | None) – ID
external_id (str | None) – No description.

Returns

Requested transformation or None if it does not exist.

Return type

Transformation | None

Examples

Get transformation by id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.retrieve(id=1)

Get transformation by external id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.retrieve(external_id="1")

TransformationsAPI.retrieve_multiple(ids: Sequence[int] | None = None, external_ids: Sequence[str] | None = None, ignore_unknown_ids: bool = False) → TransformationList

Retrieve multiple transformations.

Parameters

ids (Sequence[int] | None) – List of ids to retrieve.
external_ids (Sequence[str] | None) – List of external ids to retrieve.
ignore_unknown_ids (bool) – Ignore IDs and external IDs that are not found rather than throw an exception.

Returns

Requested transformation or None if it does not exist.

Return type

TransformationList

Examples

Get multiple transformations:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.retrieve_multiple(ids=[1,2,3], external_ids=['transform-1','transform-2'])

Run transformations by id

TransformationsAPI.run(transformation_id: int | None = None, transformation_external_id: str | None = None, wait: bool = True, timeout: float | None = None) → TransformationJob

Run a transformation.

Parameters

transformation_id (int | None) – Transformation internal id
transformation_external_id (str | None) – Transformation external id
wait (bool) – Wait until the transformation run is finished. Defaults to True.
timeout (float | None) – maximum time (s) to wait, default is None (infinite time). Once the timeout is reached, it returns with the current status. Won’t have any effect if wait is False.

Returns

Created transformation job

Return type

TransformationJob

Examples

Run transformation to completion by id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> res = c.transformations.run(transformation_id = 1)

Start running transformation by id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> res = c.transformations.run(transformation_id = 1, wait = False)

async TransformationsAPI.run_async(transformation_id: int | None = None, transformation_external_id: str | None = None, timeout: float | None = None) → TransformationJob

Run a transformation to completion asynchronously.

Parameters

transformation_id (int | None) – internal Transformation id
transformation_external_id (str | None) – external Transformation id
timeout (float | None) – maximum time (s) to wait, default is None (infinite time). Once the timeout is reached, it returns with the current status.

Returns

Completed (if finished) or running (if timeout reached) transformation job.

Return type

TransformationJob

Examples

Run transformation asynchronously by id:

>>> import asyncio
>>> from cognite.client import CogniteClient
>>>
>>> c = CogniteClient()
>>>
>>> async def run_transformation():
>>>     res = await c.transformations.run_async(id = 1)
>>>
>>> loop = asyncio.get_event_loop()
>>> loop.run_until_complete(run_transformation())
>>> loop.close()

Preview transformations

TransformationsAPI.preview(query: str | None = None, convert_to_string: bool = False, limit: int = 100, source_limit: int | None = 100, infer_schema_limit: int | None = 1000) → TransformationPreviewResult

Preview the result of a query.

Parameters

query (str | None) – SQL query to run for preview.
convert_to_string (bool) – Stringify values in the query results, default is False.
limit (int) – Maximum number of rows to return in the final result, default is 100.
source_limit (int | None) – Maximum number of items to read from the data source or None to run without limit, default is 100.
infer_schema_limit (int | None) – Limit for how many rows that are used for inferring result schema, default is 1000.

Returns

Result of the executed query

Return type

TransformationPreviewResult

Examples

Preview transformation results as schema and list of rows:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> query_result = c.transformations.preview(query="select * from _cdf.assets")

Preview transformation results as pandas dataframe:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> df = c.transformations.preview(query="select * from _cdf.assets").to_pandas()

Cancel transformation run by id

TransformationsAPI.cancel(transformation_id: int | None = None, transformation_external_id: str | None = None) → None

Cancel a running transformation.

Parameters

transformation_id (int | None) – Transformation internal id
transformation_external_id (str | None) – Transformation external id

Examples

Wait transformation for 1 minute and cancel if still running:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes import TransformationJobStatus
>>> c = CogniteClient()
>>>
>>> res = c.transformations.run(id = 1, timeout = 60.0)
>>> if res.status == TransformationJobStatus.RUNNING:
>>>     res.cancel()

List transformations

List all transformations.

Parameters

include_public (bool) – Whether public transformations should be included in the results. (default true).
name_regex (str | None) – Regex expression to match the transformation name
query_regex (str | None) – Regex expression to match the transformation query
destination_type (str | None) – Transformation destination resource name to filter by.
conflict_mode (str | None) – Filters by a selected transformation action type: abort/create, upsert, update, delete
cdf_project_name (str | None) – Project name to filter by configured source and destination project
has_blocked_error (bool | None) – Whether only the blocked transformations should be included in the results.
created_time (dict[str, Any] | TimestampRange | None) – Range between two timestamps
last_updated_time (dict[str, Any] | TimestampRange | None) – Range between two timestamps
data_set_ids (list[int] | None) – Return only transformations in the specified data sets with these ids.
data_set_external_ids (list[str] | None) – Return only transformations in the specified data sets with these external ids.
tags (TagsFilter | None) – Return only the resource matching the specified tags constraints. It only supports ContainsAny as of now.
limit (int | None) – Limits the number of results to be returned. To retrieve all results use limit=-1, default limit is 25.

Returns

List of transformations

Return type

TransformationList

Example

List transformations:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> transformations_list = c.transformations.list()

Update transformations

TransformationsAPI.update(item: Transformation | TransformationUpdate | Sequence[Transformation | TransformationUpdate]) → Transformation | TransformationList

Update one or more transformations

Parameters: item (Transformation | TransformationUpdate | Sequence[Transformation | TransformationUpdate]) – Transformation(s) to update
Returns: Updated transformation(s)
Return type: Transformation | TransformationList

Examples

Update a transformation that you have fetched. This will perform a full update of the transformation:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> transformation = c.transformations.retrieve(id=1)
>>> transformation.query = "SELECT * FROM _cdf.assets"
>>> res = c.transformations.update(transformation)

Perform a partial update on a transformation, updating the query and making it private:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes import TransformationUpdate
>>> c = CogniteClient()
>>> my_update = TransformationUpdate(id=1).query.set("SELECT * FROM _cdf.assets").is_public.set(False)
>>> res = c.transformations.update(my_update)

Delete transformations

TransformationsAPI.delete(id: int | Sequence[int] | None = None, external_id: str | Sequence[str] | None = None, ignore_unknown_ids: bool = False) → None

Delete one or more transformations.

Parameters

id (int | Sequence[int] | None) – Id or list of ids.
external_id (str | Sequence[str] | None) – External ID or list of external ids.
ignore_unknown_ids (bool) – Ignore IDs and external IDs that are not found rather than throw an exception.

Example

Delete transformations by id or external id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> c.transformations.delete(id=[1,2,3], external_id="function3")

Transformation Schedules

Create transformation Schedules

TransformationSchedulesAPI.create(schedule: TransformationSchedule | Sequence[TransformationSchedule]) → TransformationSchedule | TransformationScheduleList

Schedule the specified transformation with the specified configuration(s).

Parameters: schedule (TransformationSchedule | Sequence[TransformationSchedule]) – Configuration or list of configurations of the schedules to create.
Returns: Created schedule(s)
Return type: TransformationSchedule | TransformationScheduleList

Examples

Create new schedules:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes import TransformationSchedule
>>> c = CogniteClient()
>>> schedules = [TransformationSchedule(id = 1, interval = "0 * * * *"), TransformationSchedule(external_id="transformation2", interval = "5 * * * *"))]
>>> res = c.transformations.schedules.create(schedules)

Retrieve transformation schedules

TransformationSchedulesAPI.retrieve(id: int | None = None, external_id: str | None = None) → TransformationSchedule | None

Retrieve a single transformation schedule by the id or external id of its transformation.

Parameters

id (int | None) – transformation ID
external_id (str | None) – transformation External ID

Returns

Requested transformation schedule or None if it does not exist.

Return type

TransformationSchedule | None

Examples

Get transformation schedule by transformation id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.schedules.retrieve(id=1)

Get transformation schedule by transformation external id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.schedules.retrieve(external_id="1")

Retrieve multiple transformation schedules

TransformationSchedulesAPI.retrieve_multiple(ids: Sequence[int] | None = None, external_ids: Sequence[str] | None = None, ignore_unknown_ids: bool = False) → TransformationScheduleList

Retrieve multiple transformation schedules by the ids or external ids of the corresponding transformations.

Parameters

ids (Sequence[int] | None) – transformation IDs
external_ids (Sequence[str] | None) – transformation External IDs
ignore_unknown_ids (bool) – Ignore IDs and external IDs that are not found rather than throw an exception.

Returns

Requested transformation schedules.

Return type

TransformationScheduleList

Examples

Get transformation schedules by transformation ids:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.schedules.retrieve_multiple(ids=[1, 2, 3])

Get transformation schedules by transformation external ids:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.schedules.retrieve_multiple(external_ids=["t1", "t2"])

List transformation schedules

TransformationSchedulesAPI.list(include_public: bool = True, limit: int | None = 25) → TransformationScheduleList

List all transformation schedules.

Parameters

include_public (bool) – Whether public transformations should be included in the results. (default true).
limit (int | None) – Limits the number of results to be returned. To retrieve all results use limit=-1, default limit is 25.

Returns

List of schedules

Return type

TransformationScheduleList

Example

List schedules:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> schedules_list = c.transformations.schedules.list()

Update transformation schedules

TransformationSchedulesAPI.update(item: TransformationSchedule | TransformationScheduleUpdate | Sequence[TransformationSchedule | TransformationScheduleUpdate]) → TransformationSchedule | TransformationScheduleList

Update one or more transformation schedules

Parameters: item (TransformationSchedule | TransformationScheduleUpdate | Sequence[TransformationSchedule | TransformationScheduleUpdate]) – Transformation schedule(s) to update
Returns: Updated transformation schedule(s)
Return type: TransformationSchedule | TransformationScheduleList

Examples

Update a transformation schedule that you have fetched. This will perform a full update of the schedule:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> transformation_schedule = c.transformations.schedules.retrieve(id=1)
>>> transformation_schedule.is_paused = True
>>> res = c.transformations.schedules.update(transformation_schedule)

Perform a partial update on a transformation schedule, updating the interval and unpausing it:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes import TransformationScheduleUpdate
>>> c = CogniteClient()
>>> my_update = TransformationScheduleUpdate(id=1).interval.set("0 * * * *").is_paused.set(False)
>>> res = c.transformations.schedules.update(my_update)

Delete transformation schedules

TransformationSchedulesAPI.delete(id: int | Sequence[int] | None = None, external_id: str | Sequence[str] | None = None, ignore_unknown_ids: bool = False) → None

Unschedule one or more transformations

Parameters

id (int | Sequence[int] | None) – Id or list of ids
external_id (str | Sequence[str] | None) – External ID or list of external ids
ignore_unknown_ids (bool) – Ignore IDs and external IDs that are not found rather than throw an exception.

Examples

Delete schedules by id or external id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> c.transformations.schedules.delete(id=[1,2,3], external_id="3")

Transformation Notifications

Create transformation notifications

TransformationNotificationsAPI.create(notification: TransformationNotification | Sequence[TransformationNotification]) → TransformationNotification | TransformationNotificationList

Subscribe for notifications on the transformation errors.

Parameters: notification (TransformationNotification | Sequence[TransformationNotification]) – Notification or list of notifications to create.
Returns: Created notification(s)
Return type: TransformationNotification | TransformationNotificationList

Examples

Create new notifications:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes import TransformationNotification
>>> c = CogniteClient()
>>> notifications = [TransformationNotification(transformation_id = 1, destination="[email protected]"), TransformationNotification(transformation_external_id="transformation2", destination="[email protected]"))]
>>> res = c.transformations.notifications.create(notifications)

List transformation notifications

TransformationNotificationsAPI.list(transformation_id: int | None = None, transformation_external_id: str | None = None, destination: str | None = None, limit: int | None = 25) → TransformationNotificationList

List notification subscriptions.

Parameters

transformation_id (int | None) – Filter by transformation internal numeric ID.
transformation_external_id (str | None) – Filter by transformation externalId.
destination (str | None) – Filter by notification destination.
limit (int | None) – Limits the number of results to be returned. To retrieve all results use limit=-1, default limit is 25.

Returns

List of transformation notifications

Return type

TransformationNotificationList

Example

List all notifications:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> notifications_list = c.transformations.notifications.list()

List all notifications by transformation id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> notifications_list = c.transformations.notifications.list(transformation_id = 1)

Delete transformation notifications

TransformationNotificationsAPI.delete(id: int | Sequence[int] | None = None) → None

Deletes the specified notification subscriptions on the transformation. Does nothing when the subscriptions already don’t exist

Parameters: id (int | Sequence[int] | None) – Id or list of transformation notification ids

Examples

Delete schedules by id or external id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> c.transformations.notifications.delete(id=[1,2,3])

Transformation Jobs

Retrieve transformation jobs

TransformationJobsAPI.retrieve(id: int) → TransformationJob | None

Retrieve a single transformation job by id.

Parameters: id (int) – Job internal Id
Returns: Requested transformation job or None if it does not exist.
Return type: TransformationJob | None

Examples

Get transformation job by id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.jobs.retrieve(id=1)

TransformationJobsAPI.retrieve_multiple(ids: Sequence[int], ignore_unknown_ids: bool = False) → TransformationJobList

Retrieve multiple transformation jobs by id.

Parameters

ids (Sequence[int]) – Job internal Ids
ignore_unknown_ids (bool) – Ignore IDs that are not found rather than throw an exception.

Returns

Requested transformation jobs.

Return type

TransformationJobList

Examples

Get jobs by id:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> res = c.transformations.jobs.retrieve_multiple(ids=[1, 2, 3])

List transformation jobs

TransformationJobsAPI.list(limit: int | None = 25, transformation_id: int | None = None, transformation_external_id: str | None = None) → TransformationJobList

List all running transformation jobs.

Parameters

limit (int | None) – Limits the number of results to be returned. To retrieve all results use limit=-1, default limit is 25.
transformation_id (int | None) – Filters the results by the internal transformation id.
transformation_external_id (str | None) – Filters the results by the external transformation id.

Returns

List of transformation jobs

Return type

TransformationJobList

Example

List transformation jobs:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> transformation_jobs_list = c.transformations.jobs.list()

List transformation jobs of a single transformation:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>> transformation_jobs_list = c.transformations.jobs.list(transformation_id = 1)

Transformation Schema

Get transformation schema

TransformationSchemaAPI.retrieve(destination: TransformationDestination, conflict_mode: str | None = None) → TransformationSchemaColumnList

Get expected schema for a transformation destination.

Parameters

destination (TransformationDestination) – destination for which the schema is requested.
conflict_mode (str | None) – conflict mode for which the schema is requested.

Returns

List of column descriptions

Return type

TransformationSchemaColumnList

Example

Get the schema for a transformation producing assets:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes import TransformationDestination
>>> c = CogniteClient()
>>> columns = c.transformations.schema.retrieve(destination = TransformationDestination.assets())

Data classes

class cognite.client.data_classes.transformations.ContainsAny(tags: list[str] | None = None)

Bases: TagsFilter

Return transformations that has one of the tags specified.

Parameters: tags (list[str] | None) – The resource item contains at least one of the listed tags. The tags are defined by a list of external ids.

Examples

List only resources marked as a PUMP or as a VALVE:

>>> from cognite.client.data_classes.transformations import ContainsAny
>>> my_tag_filter = ContainsAny(tags=["PUMP", "VALVE"])

class cognite.client.data_classes.transformations.SessionDetails(session_id: int | None = None, client_id: str | None = None, project_name: str | None = None)

Bases: object

Details of a source session.

Parameters

session_id (int | None) – CDF source session ID
client_id (str | None) – Idp source client ID
project_name (str | None) – CDF source project name

dump(camel_case: bool = False) → dict[str, Any]

Dump the instance into a json serializable Python data type.

Parameters: camel_case (bool) – Use camelCase for attribute names. Defaults to False.
Returns: A dictionary representation of the instance.
Return type: dict[str, Any]

class cognite.client.data_classes.transformations.Transformation(id: int | None = None, external_id: str | None = None, name: str | None = None, query: str | None = None, destination: TransformationDestination | None = None, conflict_mode: str | None = None, is_public: bool = True, ignore_null_fields: bool = False, source_oidc_credentials: OidcCredentials | None = None, destination_oidc_credentials: OidcCredentials | None = None, created_time: int | None = None, last_updated_time: int | None = None, owner: str | None = None, owner_is_current_user: bool = True, has_source_oidc_credentials: bool | None = None, has_destination_oidc_credentials: bool | None = None, running_job: TransformationJob | None = None, last_finished_job: TransformationJob | None = None, blocked: TransformationBlockedInfo | None = None, schedule: TransformationSchedule | None = None, data_set_id: int | None = None, cognite_client: CogniteClient | None = None, source_nonce: NonceCredentials | None = None, destination_nonce: NonceCredentials | None = None, source_session: SessionDetails | None = None, destination_session: SessionDetails | None = None, tags: list[str] | None = None)

Bases: CogniteResource

The transformations resource allows transforming data in CDF.

Parameters

id (int | None) – A server-generated ID for the object.
external_id (str | None) – The external ID provided by the client. Must be unique for the resource type.
name (str | None) – The name of the Transformation.
query (str | None) – SQL query of the transformation.
destination (TransformationDestination | None) – see TransformationDestination for options.
conflict_mode (str | None) – What to do in case of id collisions: either “abort”, “upsert”, “update” or “delete”
is_public (bool) – Indicates if the transformation is visible to all in project or only to the owner.
ignore_null_fields (bool) – Indicates how null values are handled on updates: ignore or set null.
source_oidc_credentials (OidcCredentials | None) – Configures the transformation to authenticate with the given oidc credentials key on the destination.
destination_oidc_credentials (OidcCredentials | None) – Configures the transformation to authenticate with the given oidc credentials on the destination.
created_time (int | None) – The number of milliseconds since 00:00:00 Thursday, 1 January 1970, Coordinated Universal Time (UTC), minus leap seconds.
last_updated_time (int | None) – The number of milliseconds since 00:00:00 Thursday, 1 January 1970, Coordinated Universal Time (UTC), minus leap seconds.
owner (str | None) – Owner of the transformation: requester’s identity.
owner_is_current_user (bool) – Indicates if the transformation belongs to the current user.
has_source_oidc_credentials (bool | None) – Indicates if the transformation is configured with a source oidc credentials set.
has_destination_oidc_credentials (bool | None) – Indicates if the transformation is configured with a destination oidc credentials set.
running_job (TransformationJob | None) – Details for the job of this transformation currently running.
last_finished_job (TransformationJob | None) – Details for the last finished job of this transformation.
blocked (TransformationBlockedInfo | None) – Provides reason and time if the transformation is blocked.
schedule (TransformationSchedule | None) – Details for the schedule if the transformation is scheduled.
data_set_id (int | None) – No description.
cognite_client (CogniteClient | None) – The client to associate with this object.
source_nonce (NonceCredentials | None) – Single use credentials to bind to a CDF session for reading.
destination_nonce (NonceCredentials | None) – Single use credentials to bind to a CDF session for writing.
source_session (SessionDetails | None) – Details for the session used to read from the source project.
destination_session (SessionDetails | None) – Details for the session used to write to the destination project.
tags (list[str] | None) – No description.

dump(camel_case: bool = False) → dict[str, Any]

Dump the instance into a json serializable Python data type.

Parameters: camel_case (bool) – Use camelCase for attribute names. Defaults to False.
Returns: A dictionary representation of the instance.
Return type: dict[str, Any]

Bases: CogniteFilter

No description.

Parameters

include_public (bool) – Whether public transformations should be included in the results. The default is true.
name_regex (str | None) – Regex expression to match the transformation name
query_regex (str | None) – Regex expression to match the transformation query
destination_type (str | None) – Transformation destination resource name to filter by.
conflict_mode (str | None) – Filters by a selected transformation action type: abort/create, upsert, update, delete
cdf_project_name (str | None) – Project name to filter by configured source and destination project
has_blocked_error (bool | None) – Whether only the blocked transformations should be included in the results.
created_time (dict[str, Any] | TimestampRange | None) – Range between two timestamps
last_updated_time (dict[str, Any] | TimestampRange | None) – Range between two timestamps
data_set_ids (list[dict[str, Any]] | None) – Return only transformations in the specified data sets with these ids.
tags (TagsFilter | None) – Return only the resource matching the specified tags constraints. It only supports ContainsAny as of now.

dump(camel_case: bool = True) → dict[str, Any]

Dump the instance into a json serializable Python data type.

Parameters: camel_case (bool) – Use camelCase for attribute names. Defaults to False.
Returns: A dictionary representation of the instance.
Return type: dict[str, Any]

class cognite.client.data_classes.transformations.TransformationList(resources: Collection[Any], cognite_client: CogniteClient | None = None): Bases: CogniteResourceList[Transformation], IdTransformerMixin

class cognite.client.data_classes.transformations.TransformationPreviewResult(schema: TransformationSchemaColumnList | None = None, results: list[dict] | None = None, cognite_client: CogniteClient | None = None)

Bases: CogniteResource

Allows previewing the result of a sql transformation before executing it.

Parameters

schema (TransformationSchemaColumnList | None) – List of column descriptions.
results (list[dict] | None) – List of resulting rows. Each row is a dictionary where the key is the column name and the value is the entry.
cognite_client (CogniteClient | None) – No description.

dump(camel_case: bool = False) → dict[str, Any]

Dump the instance into a json serializable Python data type.

Parameters: camel_case (bool) – Use camelCase for attribute names. Defaults to False.
Returns: A dictionary representation of the instance.
Return type: dict[str, Any]

class cognite.client.data_classes.transformations.TransformationUpdate(id: int | None = None, external_id: str | None = None)

Bases: CogniteUpdate

Changes applied to transformation

Parameters

id (int) – A server-generated ID for the object.
external_id (str) – External Id provided by client. Should be unique within the project.

dump(camel_case: bool = True) → dict[str, Any]

Dump the instance into a json serializable Python data type.

Parameters: camel_case (bool) – No description.
Returns: A dictionary representation of the instance.
Return type: dict[str, Any]

class cognite.client.data_classes.transformations.schedules.TransformationSchedule(id: int | None = None, external_id: str | None = None, created_time: int | None = None, last_updated_time: int | None = None, interval: str | None = None, is_paused: bool = False, cognite_client: CogniteClient | None = None)

Bases: CogniteResource

The transformation schedules resource allows running recurrent transformations.

Parameters

id (int | None) – Transformation id.
external_id (str | None) – Transformation externalId.
created_time (int | None) – Time when the schedule was created.
last_updated_time (int | None) – Time when the schedule was last updated.
interval (str | None) – Cron expression describes when the function should be called. Use http://www.cronmaker.com to create a cron expression.
is_paused (bool) – If true, the transformation is not scheduled.
cognite_client (CogniteClient | None) – The client to associate with this object.

class cognite.client.data_classes.transformations.schedules.TransformationScheduleList(resources: Collection[Any], cognite_client: CogniteClient | None = None): Bases: CogniteResourceList[TransformationSchedule]

class cognite.client.data_classes.transformations.schedules.TransformationScheduleUpdate(id: int | None = None, external_id: str | None = None)

Bases: CogniteUpdate

Changes applied to transformation schedule

Parameters

id (int) – Transformation id.
external_id (str) – Transformation externalId.

class cognite.client.data_classes.transformations.notifications.TransformationNotification(id: int | None = None, transformation_id: int | None = None, transformation_external_id: str | None = None, destination: str | None = None, created_time: int | None = None, last_updated_time: int | None = None, cognite_client: CogniteClient | None = None)

Bases: CogniteResource

The transformation notification resource allows configuring email alerts on events related to a transformation run.

Parameters

id (int | None) – A server-generated ID for the object.
transformation_id (int | None) – Transformation Id.
transformation_external_id (str | None) – Transformation external Id.
destination (str | None) – Email address where notifications should be sent.
created_time (int | None) – Time when the notification was created.
last_updated_time (int | None) – Time when the notification was last updated.
cognite_client (CogniteClient | None) – The client to associate with this object.

class cognite.client.data_classes.transformations.notifications.TransformationNotificationFilter(transformation_id: int | None = None, transformation_external_id: str | None = None, destination: str | None = None)

Bases: CogniteFilter

Parameters

transformation_id (int | None) – Filter by transformation internal numeric ID.
transformation_external_id (str | None) – Filter by transformation externalId.
destination (str | None) – Filter by notification destination.

class cognite.client.data_classes.transformations.notifications.TransformationNotificationList(resources: Collection[Any], cognite_client: CogniteClient | None = None): Bases: CogniteResourceList[TransformationNotification]

Bases: CogniteResource

The transformation job resource allows following the status of execution of a transformation run.

Parameters

id (int | None) – A server-generated ID for the object.
status (TransformationJobStatus | None) – Status of the job.
transformation_id (int | None) – Server-generated ID of the transformation.
transformation_external_id (str | None) – external ID of the transformation.
source_project (str | None) – Name of the CDF project the data will be read from.
destination_project (str | None) – Name of the CDF project the data will be written to.
destination (TransformationDestination | None) – No description.
conflict_mode (str | None) – What to do in case of id collisions: either “abort”, “upsert”, “update” or “delete”.
query (str | None) – Query of the transformation that is being executed.
error (str | None) – Error message from the server.
ignore_null_fields (bool) – Indicates how null values are handled on updates: ignore or set null.
created_time (int | None) – Time when the job was created.
started_time (int | None) – Time when the job started running.
finished_time (int | None) – Time when the job finished running.
last_seen_time (int | None) – Time of the last status update from the job.
cognite_client (CogniteClient | None) – The client to associate with this object.

metrics() → TransformationJobMetricList: Get job metrics.

update() → None: Get updated job status.

wait(polling_interval: float = 1, timeout: float | None = None) → TransformationJob

Waits for the job to finish.

Parameters

polling_interval (float) – time (s) to wait between job status updates, default is one second.
timeout (float | None) – maximum time (s) to wait, default is None (infinite time). Once the timeout is reached, it returns with the current status.

Returns

self.

Return type

TransformationJob

Examples

run transformations 1 and 2 in parallel, and run 3 once they finish successfully:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> job1 = c.transformations.run(id = 1, wait = False)
>>> job2 = c.transformations.run(id = 2, wait = False)
>>> job1.wait()
>>> job2.wait()
>>> if TransformationJobStatus.FAILED not in [job1.status, job2.status]:
>>>     c.transformations.run(id = 3, wait = False)

wait transformation for 5 minutes and do something if still running:

>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> job = c.transformations.run(id = 1, wait = False)
>>> job.wait(timeout = 5.0*60)
>>> if job.status == TransformationJobStatus.FAILED:
>>>     # do something if job failed
>>> elif job.status == TransformationJobStatus.COMPLETED:
>>>     # do something if job completed successfully
>>> else:
>>>     # do something if job is still running

async wait_async(polling_interval: float = 1, timeout: float | None = None) → TransformationJob

Asyncio coroutine, waits for the job to finish asynchronously.

Parameters

polling_interval (float) – time (s) to wait between job status updates, default is one second.
timeout (float | None) – maximum time (s) to wait, default is None (infinite time). Once the timeout is reached, it returns with the current status.

Returns

coroutine object that will finish when the job finishes and resolves to self.

Return type

TransformationJob

Examples

run transformations 1 and 2 in parallel, and run 3 once they finish successfully:

>>> from asyncio import ensure_future
>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> async def run_successive_transformations():
>>>     job1 = c.transformations.run(id = 1, wait = False)
>>>     job2 = c.transformations.run(id = 2, wait = False)
>>>     await job1.wait_async()
>>>     await job2.wait_async()
>>>     if TransformationJobStatus.FAILED not in [job1.status, job2.status]:
>>>         c.transformations.run(id = 3, wait = False)
>>>
>>> ensure_future(run_successive_transformations())

wait transformation for 5 minutes and do something if still running:

>>> from asyncio import ensure_future
>>> from cognite.client import CogniteClient
>>> c = CogniteClient()
>>>
>>> async def run_successive_transformations():
>>>     job = c.transformations.run(id = 1, wait = False)
>>>     await job.wait_async(timeout = 5.0*60)
>>>     if job.status == TransformationJobStatus.FAILED:
>>>         # do something if job failed
>>>     elif job.status == TransformationJobStatus.COMPLETED:
>>>         # do something if job completed successfully
>>>     else:
>>>         # do something if job is still running
>>>
>>> ensure_future(run_successive_transformations())

class cognite.client.data_classes.transformations.jobs.TransformationJobFilter(transformation_id: int | None = None, transformation_external_id: str | None = None)

Bases: CogniteFilter

Parameters

transformation_id (int | None) – Filter jobs by transformation internal numeric ID.
transformation_external_id (str | None) – Filter jobs by transformation external ID.

class cognite.client.data_classes.transformations.jobs.TransformationJobList(resources: Collection[Any], cognite_client: CogniteClient | None = None): Bases: CogniteResourceList[TransformationJob]

Bases: CogniteResource

The transformation job metric resource allows following details of execution of a transformation run.

Parameters

id (int | None) – No description.
timestamp (int | None) – Time of the last metric update.
name (str | None) – Name of the metric.
count (int | None) – Value of the metric.
cognite_client (CogniteClient | None) – The client to associate with this object.

class cognite.client.data_classes.transformations.jobs.TransformationJobMetricList(resources: Collection[Any], cognite_client: CogniteClient | None = None): Bases: CogniteResourceList[TransformationJobMetric]

class cognite.client.data_classes.transformations.jobs.TransformationJobStatus(value)

Bases: str, Enum

An enumeration.

class cognite.client.data_classes.transformations.schema.TransformationSchemaColumn(name: str | None = None, sql_type: str | None = None, type: TransformationSchemaType | None = None, nullable: bool = False, cognite_client: CogniteClient | None = None)

Bases: CogniteResource

Represents a column of the expected sql structure for a destination type.

Parameters

name (str | None) – Column name
sql_type (str | None) – Type of the column in sql format.
type (TransformationSchemaType | None) – Type of the column in json format.
nullable (bool) – Values for the column can be null or not
cognite_client (CogniteClient | None) – The client to associate with this object.

class cognite.client.data_classes.transformations.schema.TransformationSchemaColumnList(resources: Collection[Any], cognite_client: CogniteClient | None = None): Bases: CogniteResourceList[TransformationSchemaColumn]

class cognite.client.data_classes.transformations.common.Edges(view: ViewInfo | None = None, instance_space: str | None = None, edge_type: EdgeType | None = None): Bases: TransformationDestination

class cognite.client.data_classes.transformations.common.Instances(data_model: DataModelInfo | None = None, instance_space: str | None = None): Bases: TransformationDestination

class cognite.client.data_classes.transformations.common.Nodes(view: ViewInfo | None = None, instance_space: str | None = None): Bases: TransformationDestination

class cognite.client.data_classes.transformations.common.OidcCredentials(client_id: str, client_secret: str, scopes: str | list[str], token_uri: str, audience: str | None = None, cdf_project_name: str | None = None)

Bases: object

Class that represents OpenID Connect (OIDC) credentials used to authenticate towards Cognite Data Fusion (CDF).

Note

Is currently only used to specify inputs to TransformationsAPI like source_oidc_credentials and destination_oidc_credentials.

Parameters

client_id (str) – Your application’s client id.
client_secret (str) – Your application’s client secret
scopes (str | list[str]) – A list of scopes or a comma-separated string (for backwards compatibility).
token_uri (str) – OAuth token url
audience (str | None) – Audience (optional)
cdf_project_name (str | None) – Name of CDF project (optional)

dump(camel_case: bool = False) → dict[str, Any]

Dump the instance into a json serializable Python data type.

Parameters: camel_case (bool) – Use camelCase for attribute names. Defaults to False.
Returns: A dictionary representation of the instance.
Return type: dict[str, Any]

class cognite.client.data_classes.transformations.common.RawTable(database: str | None = None, table: str | None = None): Bases: TransformationDestination

class cognite.client.data_classes.transformations.common.SequenceRows(external_id: str | None = None): Bases: TransformationDestination

class cognite.client.data_classes.transformations.common.TransformationBlockedInfo(reason: str, created_time: int)

Bases: object

Information about the reason why and when a transformation is blocked.

Parameters

reason (str) – Reason why the transformation is blocked.
created_time (int) – Timestamp when the transformation was blocked.

class cognite.client.data_classes.transformations.common.TransformationDestination(type: str | None = None)

Bases: object

TransformationDestination has static methods to define the target resource type of a transformation

Parameters: type (str | None) – Used as data type identifier on transformation creation/retrieval.

static asset_hierarchy() → TransformationDestination: To be used when the transformation is meant to produce asset hierarchies.

static assets() → TransformationDestination: To be used when the transformation is meant to produce assets.

static data_sets() → TransformationDestination: To be used when the transformation is meant to produce data sets.

static datapoints() → TransformationDestination: To be used when the transformation is meant to produce numeric data points.

static edges(view: ViewInfo | None = None, instance_space: str | None = None, edge_type: EdgeType | None = None) → Edges

Parameters

view (ViewInfo | None) – information of the view.
instance_space (str | None) – space id of the instance.
edge_type (EdgeType | None) – information about the type of the edge

Returns

pointing to the target flexible data model.

Return type

Edges

static events() → TransformationDestination: To be used when the transformation is meant to produce events.

static files() → TransformationDestination: To be used when the transformation is meant to produce files.

static instances(data_model: DataModelInfo | None = None, instance_space: str | None = None) → Instances

Parameters

data_model (DataModelInfo | None) – information of the Data Model.
instance_space (str | None) – space id of the instance.

Returns

pointing to the target centric data model.

Return type

Instances

static labels() → TransformationDestination: To be used when the transformation is meant to produce labels.

static nodes(view: ViewInfo | None = None, instance_space: str | None = None) → Nodes

Parameters

view (ViewInfo | None) – information of the view.
instance_space (str | None) – space id of the instance.

Returns

pointing to the target flexible data model.

Return type

Nodes

static raw(database: str = '', table: str = '') → RawTable

To be used when the transformation is meant to produce raw table rows.

Parameters

database (str) – database name of the target raw table.
table (str) – name of the target raw table

Returns

TransformationDestination pointing to the target table

Return type

RawTable

static relationships() → TransformationDestination: To be used when the transformation is meant to produce relationships.

static sequence_rows(external_id: str = '') → SequenceRows

To be used when the transformation is meant to produce sequence rows.

Parameters: external_id (str) – Sequence external id.
Returns: TransformationDestination pointing to the target sequence rows
Return type: SequenceRows

static sequences() → TransformationDestination: To be used when the transformation is meant to produce sequences.

static string_datapoints() → TransformationDestination: To be used when the transformation is meant to produce string data points.

static timeseries() → TransformationDestination: To be used when the transformation is meant to produce time series.