Data Ingestion

Raw

Databases

`AsyncCogniteClient.raw.databases.create`(name)	Create one or more databases.
`AsyncCogniteClient.raw.databases.delete`(name)	Delete one or more databases.
`AsyncCogniteClient.raw.databases.list`([limit])	List databases.

Tables

`AsyncCogniteClient.raw.tables.create`(...)	Create one or more tables.
`AsyncCogniteClient.raw.tables.delete`(...)	Delete one or more tables.
`AsyncCogniteClient.raw.tables.list`(db_name)	List tables.

Rows

`AsyncCogniteClient.raw.rows.delete`(db_name, ...)	Delete rows from a table.
`AsyncCogniteClient.raw.rows.insert`(db_name, ...)	Insert one or more rows into a table.
`AsyncCogniteClient.raw.rows.insert_dataframe`(...)	Insert pandas dataframe into a table.
`AsyncCogniteClient.raw.rows.list`(db_name, ...)	List rows in a table.
`AsyncCogniteClient.raw.rows.retrieve`(...)	Retrieve a single row by key.
`AsyncCogniteClient.raw.rows.retrieve_dataframe`(...)	Retrieve rows in a table as a pandas dataframe.

RAW Data classes

class cognite.client.data_classes.raw.Database(name: str, created_time: int | None = None)

Bases: WriteableCogniteResourceWithClientRef[DatabaseWrite]

A NoSQL database to store customer data.

Parameters:

name (str) – Unique name of a database.
created_time (int | None) – Time the database was created.

as_write() → DatabaseWrite: Returns this Database as a DatabaseWrite

tables( limit: int | None = None, ) → TableList

Get the tables in this database.

Parameters:: limit (int | None) – The number of tables to return.
Returns:: List of tables in this database.
Return type:: TableList

async tables_async( limit: int | None = None, ) → TableList

Get the tables in this database.

Parameters:: limit (int | None) – The number of tables to return.
Returns:: List of tables in this database.
Return type:: TableList

class cognite.client.data_classes.raw.DatabaseList( resources: Sequence[T_CogniteResource], )

Bases: WriteableCogniteResourceList[DatabaseWrite, Database], NameTransformerMixin

as_write() → DatabaseWriteList: Returns this DatabaseList as a DatabaseWriteList

class cognite.client.data_classes.raw.DatabaseWrite(name: str)

Bases: WriteableCogniteResource[DatabaseWrite]

A NoSQL database to store customer data.

Parameters:: name (str) – Unique name of a database.

as_write() → DatabaseWrite: Returns this DatabaseWrite instance.

class cognite.client.data_classes.raw.DatabaseWriteList( resources: Sequence[T_CogniteResource], ): Bases: CogniteResourceList[DatabaseWrite], NameTransformerMixin

class cognite.client.data_classes.raw.Row(key: str, columns: dict[str, Any], last_updated_time: int)

Bases: RowCore

This represents a row in a NO-SQL table. This is the read version of the Row class, which is used when retrieving a row.

Parameters:

key (str) – Unique row key
columns (dict[str, Any]) – Row data stored as a JSON object.
last_updated_time (int) – The number of milliseconds since 00:00:00 Thursday, 1 January 1970, Coordinated Universal Time (UTC), minus leap seconds.

as_write() → RowWrite: Returns this Row as a RowWrite

class cognite.client.data_classes.raw.RowCore(key: str, columns: dict[str, Any])

Bases: WriteableCogniteResource[RowWrite], ABC

No description.

Parameters:

key (str) – Unique row key
columns (dict[str, Any]) – Row data stored as a JSON object.

to_pandas() → pandas.DataFrame

Convert the instance into a pandas DataFrame.

Returns:: The pandas DataFrame representing this instance.
Return type:: pandas.DataFrame

class cognite.client.data_classes.raw.RowList( resources: Sequence[T_CogniteResource], )

Bases: RowListCore[Row]

as_write() → RowWriteList: Returns this RowList as a RowWriteList

class cognite.client.data_classes.raw.RowListCore( resources: Sequence[T_CogniteResource], )

Bases: WriteableCogniteResourceList[RowWrite, T_Row], ABC

to_pandas() → pandas.DataFrame

Convert the instance into a pandas DataFrame.

Returns:: The pandas DataFrame representing this instance.
Return type:: pandas.DataFrame

class cognite.client.data_classes.raw.RowWrite(key: str, columns: dict[str, Any])

Bases: RowCore

This represents a row in a NO-SQL table. This is the write version of the Row class, which is used when creating a row.

Parameters:

key (str) – Unique row key
columns (dict[str, Any]) – Row data stored as a JSON object.

as_write() → RowWrite: Returns this RowWrite instance.

class cognite.client.data_classes.raw.RowWriteList( resources: Sequence[T_CogniteResource], ): Bases: RowListCore[RowWrite]

class cognite.client.data_classes.raw.Table(name: str, created_time: int | None = None)

Bases: WriteableCogniteResourceWithClientRef[TableWrite]

A NoSQL database table to store customer data. This is the read version of the Table class, which is used when retrieving a table.

Parameters:

name (str) – Unique name of the table
created_time (int | None) – Time the table was created.

as_write() → TableWrite: Returns this Table as a TableWrite

rows( key: str | None = None, limit: int | None = None, ) → Row | RowList | None

Get the rows in this table.

Parameters:

key (str | None) – Specify a key to return only that row.
limit (int | None) – The number of rows to return.

Returns:

List of tables in this database.

Return type:

Row | RowList | None

async rows_async( key: str | None = None, limit: int | None = None, ) → Row | RowList | None

Get the rows in this table.

Parameters:

key (str | None) – Specify a key to return only that row.
limit (int | None) – The number of rows to return.

Returns:

List of tables in this database.

Return type:

Row | RowList | None

class cognite.client.data_classes.raw.TableList( resources: Sequence[T_CogniteResource], )

Bases: WriteableCogniteResourceList[TableWrite, Table], NameTransformerMixin

as_write() → TableWriteList: Returns this TableList as a TableWriteList

class cognite.client.data_classes.raw.TableWrite(name: str)

Bases: WriteableCogniteResource[TableWrite]

A NoSQL database table to store customer data This is the write version of the Table class, which is used when creating a table.

Parameters:: name (str) – Unique name of the table

as_write() → TableWrite: Returns this TableWrite instance.

class cognite.client.data_classes.raw.TableWriteList( resources: Sequence[T_CogniteResource], ): Bases: CogniteResourceList[TableWrite], NameTransformerMixin

Extraction pipelines

`AsyncCogniteClient.extraction_pipelines.create`(...)	Create one or more extraction pipelines.
`AsyncCogniteClient.extraction_pipelines.delete`([...])	Delete one or more extraction pipelines.
`AsyncCogniteClient.extraction_pipelines.list`([limit])	List extraction pipelines.
`AsyncCogniteClient.extraction_pipelines.retrieve`([...])	Retrieve a single extraction pipeline by id.
`AsyncCogniteClient.extraction_pipelines.retrieve_multiple`([...])	Retrieve multiple extraction pipelines by ids and external ids.
`AsyncCogniteClient.extraction_pipelines.update`(item)	Update one or more extraction pipelines.

Extraction pipeline runs

`AsyncCogniteClient.extraction_pipelines.runs.create`(run)	Create one or more extraction pipeline runs.
`AsyncCogniteClient.extraction_pipelines.runs.list`(...)	List runs for an extraction pipeline with given external_id.

Extraction pipeline configs

`AsyncCogniteClient.extraction_pipelines.config.create`(config)	Create a new configuration revision.
`AsyncCogniteClient.extraction_pipelines.config.list`(...)	Retrieve all configuration revisions from an extraction pipeline.
`AsyncCogniteClient.extraction_pipelines.config.retrieve`(...)	Retrieve a specific configuration revision, or the latest by default.
`AsyncCogniteClient.extraction_pipelines.config.revert`(...)	Revert to a previous configuration revision.

Extractor Config Data classes

class cognite.client.data_classes.extractionpipelines.ExtractionPipeline( id: int, external_id: str, name: str, data_set_id: int, created_time: int, last_updated_time: int, description: str | None = None, raw_tables: list[dict[str, str]] | None = None, last_success: int | None = None, last_failure: int | None = None, last_message: str | None = None, last_seen: int | None = None, schedule: str | None = None, contacts: list[ExtractionPipelineContact] | None = None, metadata: dict[str, str] | None = None, source: str | None = None, documentation: str | None = None, notification_config: ExtractionPipelineNotificationConfiguration | None = None, created_by: str | None = None, )

Bases: ExtractionPipelineCore

An extraction pipeline is a representation of a process writing data to CDF, such as an extractor or an ETL tool. This is the read version of the ExtractionPipeline class, which is used when retrieving extraction pipelines.

Parameters:

id (int) – A server-generated ID for the object.
external_id (str) – The external ID provided by the client. Must be unique for the resource type.
name (str) – The name of the extraction pipeline.
data_set_id (int) – The id of the dataset this extraction pipeline related with.
created_time (int) – The number of milliseconds since 00:00:00 Thursday, 1 January 1970, Coordinated Universal Time (UTC), minus leap seconds.
last_updated_time (int) – The number of milliseconds since 00:00:00 Thursday, 1 January 1970, Coordinated Universal Time (UTC), minus leap seconds.
description (str | None) – The description of the extraction pipeline.
raw_tables (list[dict[str, str]] | None) – list of raw tables in list format: [{“dbName”: “value”, “tableName” : “value”}].
last_success (int | None) – Milliseconds value of last success status.
last_failure (int | None) – Milliseconds value of last failure status.
last_message (str | None) – Message of last failure.
last_seen (int | None) – Milliseconds value of last seen status.
schedule (str | None) – One of None/On trigger/Continuous/cron regex.
contacts (list[ExtractionPipelineContact] | None) – list of contacts
metadata (dict[str, str] | None) – Custom, application specific metadata. String key -> String value. Limits: Maximum length of key is 128 bytes, value 10240 bytes, up to 256 key-value pairs, of total size at most 10240.
source (str | None) – Source text value for extraction pipeline.
documentation (str | None) – Documentation text value for extraction pipeline.
notification_config (ExtractionPipelineNotificationConfiguration | None) – Notification configuration for the extraction pipeline.
created_by (str | None) – Extraction pipeline creator, usually an email.

as_write() → ExtractionPipelineWrite: Returns this ExtractionPipeline as a ExtractionPipelineWrite

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineConfig( external_id: str, revision: int, created_time: int, config: str | None = None, description: str | None = None, )

Bases: ExtractionPipelineConfigCore

An extraction pipeline config

Parameters:

external_id (str) – The external ID of the associated extraction pipeline.
revision (int) – The revision number of this config as a positive integer.
created_time (int) – The number of milliseconds since 00:00:00 Thursday, 1 January 1970, Coordinated Universal Time (UTC), minus leap seconds.
config (str | None) – Contents of this configuration revision.
description (str | None) – Short description of this configuration revision.

as_write() → ExtractionPipelineConfigWrite: Returns this ExtractionPipelineConfig as a ExtractionPipelineConfigWrite

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineConfigCore( external_id: str, config: str | None = None, description: str | None = None, )

Bases: WriteableCogniteResource[ExtractionPipelineConfigWrite], ABC

An extraction pipeline config

Parameters:

external_id (str) – The external ID of the associated extraction pipeline.
config (str | None) – Contents of this configuration revision.
description (str | None) – Short description of this configuration revision.

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineConfigList( resources: Sequence[T_CogniteResource], ): Bases: WriteableCogniteResourceList[ExtractionPipelineConfigWrite, ExtractionPipelineConfig], ExternalIDTransformerMixin

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineConfigRevision( external_id: str, revision: int, description: str | None, created_time: int, )

Bases: CogniteResource

An extraction pipeline config revision

Parameters:

external_id (str) – The external ID of the associated extraction pipeline.
revision (int) – The revision number of this config as a positive integer.
description (str | None) – Short description of this configuration revision.
created_time (int) – The number of milliseconds since 00:00:00 Thursday, 1 January 1970, Coordinated Universal Time (UTC), minus leap seconds.

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineConfigRevisionList( resources: Sequence[T_CogniteResource], ): Bases: CogniteResourceList[ExtractionPipelineConfigRevision], ExternalIDTransformerMixin

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineConfigWrite( external_id: str, config: str | None = None, description: str | None = None, )

Bases: ExtractionPipelineConfigCore

An extraction pipeline config

Parameters:

external_id (str) – The external ID of the associated extraction pipeline.
config (str | None) – Contents of this configuration revision.
description (str | None) – Short description of this configuration revision.

as_write() → ExtractionPipelineConfigWrite: Returns this ExtractionPipelineConfigWrite instance.

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineConfigWriteList( resources: Sequence[T_CogniteResource], ): Bases: CogniteResourceList[ExtractionPipelineConfigWrite], ExternalIDTransformerMixin

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineContact( name: str | None = None, email: str | None = None, role: str | None = None, send_notification: bool | None = None, )

Bases: CogniteResource

A contact for an extraction pipeline

Parameters:

name (str | None) – Name of contact
email (str | None) – Email address of contact
role (str | None) – Role of contact, such as Owner, Maintainer, etc.
send_notification (bool | None) – Whether to send notifications to this contact or not

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineCore( external_id: str, name: str, description: str | None, data_set_id: int, raw_tables: list[dict[str, str]] | None, schedule: str | None, contacts: list[ExtractionPipelineContact] | None, metadata: dict[str, str] | None, source: str | None, documentation: str | None, notification_config: ExtractionPipelineNotificationConfiguration | None, created_by: str | None, )

Bases: WriteableCogniteResource[ExtractionPipelineWrite], ABC

An extraction pipeline is a representation of a process writing data to CDF, such as an extractor or an ETL tool.

Parameters:

external_id (str) – The external ID provided by the client. Must be unique for the resource type.
name (str) – The name of the extraction pipeline.
description (str | None) – The description of the extraction pipeline.
data_set_id (int) – The id of the dataset this extraction pipeline related with.
raw_tables (list[dict[str, str]] | None) – list of raw tables in list format: [{“dbName”: “value”, “tableName” : “value”}].
schedule (str | None) – One of None/On trigger/Continuous/cron regex.
contacts (list[ExtractionPipelineContact] | None) – list of contacts
metadata (dict[str, str] | None) – Custom, application specific metadata. String key -> String value. Limits: Maximum length of key is 128 bytes, value 10240 bytes, up to 256 key-value pairs, of total size at most 10240.
source (str | None) – Source text value for extraction pipeline.
documentation (str | None) – Documentation text value for extraction pipeline.
notification_config (ExtractionPipelineNotificationConfiguration | None) – Notification configuration for the extraction pipeline.
created_by (str | None) – Extraction pipeline creator, usually an email.

dump(camel_case: bool = True) → dict[str, Any]

Dump the instance into a json serializable Python data type.

Parameters:: camel_case (bool) – Use camelCase for attribute names. Defaults to True.
Returns:: A dictionary representation of the instance.
Return type:: dict[str, Any]

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineList( resources: Sequence[T_CogniteResource], ): Bases: WriteableCogniteResourceList[ExtractionPipelineWrite, ExtractionPipeline], IdTransformerMixin

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineNotificationConfiguration( allowed_not_seen_range_in_minutes: int | None = None, )

Bases: CogniteResource

Extraction pipeline notification configuration

Parameters:: allowed_not_seen_range_in_minutes (int | None) – Time in minutes to pass without any Run. Null if extraction pipeline is not checked.

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineRun( id: int, status: Literal['success', 'failure', 'seen'], extpipe_external_id: str | None = None, message: str | None = None, created_time: int | None = None, )

Bases: ExtractionPipelineRunCore

A representation of an extraction pipeline run.

Parameters:

id (int) – A server-generated ID for the object.
status (Literal['success', 'failure', 'seen']) – success/failure/seen.
extpipe_external_id (str | None) – The external ID of the extraction pipeline.
message (str | None) – Optional status message.
created_time (int | None) – The number of milliseconds since 00:00:00 Thursday, 1 January 1970, Coordinated Universal Time (UTC), minus leap seconds.

as_write() → ExtractionPipelineRunWrite: Returns this ExtractionPipelineRun as a ExtractionPipelineRunWrite

dump(camel_case: bool = True) → dict[str, Any]

Dump the instance into a json serializable Python data type.

Parameters:: camel_case (bool) – Use camelCase for attribute names. Defaults to True.
Returns:: A dictionary representation of the instance.
Return type:: dict[str, Any]

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineRunCore( status: Literal['success', 'failure', 'seen'], message: str | None, created_time: int | None, )

Bases: WriteableCogniteResource[ExtractionPipelineRunWrite], ABC

A representation of an extraction pipeline run.

Parameters:

status (Literal['success', 'failure', 'seen']) – success/failure/seen.
message (str | None) – Optional status message.
created_time (int | None) – The number of milliseconds since 00:00:00 Thursday, 1 January 1970, Coordinated Universal Time (UTC), minus leap seconds.

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineRunFilter( external_id: str | None = None, statuses: SequenceNotStr[str] | None = None, message: StringFilter | None = None, created_time: dict[str, Any] | TimestampRange | None = None, )

Bases: CogniteFilter

Filter runs with exact matching

Parameters:

external_id (str | None) – The external ID of related ExtractionPipeline provided by the client. Must be unique for the resource type.
statuses (SequenceNotStr[str] | None) – success/failure/seen.
message (StringFilter | None) – message filter.
created_time (dict[str, Any] | TimestampRange | None) – Range between two timestamps.

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineRunList( resources: Sequence[T_CogniteResource], ): Bases: WriteableCogniteResourceList[ExtractionPipelineRunWrite, ExtractionPipelineRun], IdTransformerMixin

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineRunWrite( extpipe_external_id: str, status: Literal['success', 'failure', 'seen'], message: str | None = None, created_time: int | None = None, )

Bases: ExtractionPipelineRunCore

A representation of an extraction pipeline run. This is the write version of the ExtractionPipelineRun class, which is used when creating extraction pipeline runs.

Parameters:

extpipe_external_id (str) – The external ID of the extraction pipeline.
status (Literal['success', 'failure', 'seen']) – success/failure/seen.
message (str | None) – Optional status message.
created_time (int | None) – The number of milliseconds since 00:00:00 Thursday, 1 January 1970, Coordinated Universal Time (UTC), minus leap seconds.

as_write() → ExtractionPipelineRunWrite: Returns this ExtractionPipelineRunWrite instance.

dump( camel_case: bool = True, ) → dict[str, Any]

Dump the instance into a json serializable Python data type.

Parameters:: camel_case (bool) – Use camelCase for attribute names. Defaults to True.
Returns:: A dictionary representation of the instance.
Return type:: dict[str, Any]

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineRunWriteList( resources: Sequence[T_CogniteResource], ): Bases: CogniteResourceList[ExtractionPipelineRunWrite]

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineUpdate(id: int | None = None, external_id: str | None = None)

Bases: CogniteUpdate

Changes applied to an extraction pipeline

Parameters:

id (int) – A server-generated ID for the object.
external_id (str) – The external ID provided by the client. Must be unique for the resource type.

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineWrite( external_id: str, name: str, data_set_id: int, description: str | None = None, raw_tables: list[dict[str, str]] | None = None, schedule: str | None = None, contacts: list[ExtractionPipelineContact] | None = None, metadata: dict[str, str] | None = None, source: str | None = None, documentation: str | None = None, notification_config: ExtractionPipelineNotificationConfiguration | None = None, created_by: str | None = None, )

Bases: ExtractionPipelineCore

An extraction pipeline is a representation of a process writing data to CDF, such as an extractor or an ETL tool. This is the write version of the ExtractionPipeline class, which is used when creating extraction pipelines.

Parameters:

external_id (str) – The external ID provided by the client. Must be unique for the resource type.
name (str) – The name of the extraction pipeline.
data_set_id (int) – The id of the dataset this extraction pipeline related with.
description (str | None) – The description of the extraction pipeline.
raw_tables (list[dict[str, str]] | None) – list of raw tables in list format: [{“dbName”: “value”, “tableName” : “value”}].
schedule (str | None) – One of None/On trigger/Continuous/cron regex.
contacts (list[ExtractionPipelineContact] | None) – list of contacts
metadata (dict[str, str] | None) – Custom, application specific metadata. String key -> String value. Limits: Maximum length of key is 128 bytes, value 10240 bytes, up to 256 key-value pairs, of total size at most 10240.
source (str | None) – Source text value for extraction pipeline.
documentation (str | None) – Documentation text value for extraction pipeline.
notification_config (ExtractionPipelineNotificationConfiguration | None) – Notification configuration for the extraction pipeline.
created_by (str | None) – Extraction pipeline creator, usually an email.

as_write() → ExtractionPipelineWrite: Returns this ExtractionPipelineWrite instance.

class cognite.client.data_classes.extractionpipelines.ExtractionPipelineWriteList( resources: Sequence[T_CogniteResource], ): Bases: CogniteResourceList[ExtractionPipelineWrite], ExternalIDTransformerMixin

class cognite.client.data_classes.extractionpipelines.StringFilter(substring: str | None = None)

Bases: CogniteFilter

Filter runs on substrings of the message

Parameters:: substring (str | None) – Part of message