Hosted Extractors

Sources

Create new source

async SourcesAPI.create( items: SourceWrite | Sequence[SourceWrite], ) → Source | SourceList

Parameters:: items (SourceWrite | Sequence[SourceWrite]) – Source(s) to create.
Returns:: Created source(s)
Return type:: Source | SourceList

Examples

Create new source:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes.hosted_extractors import EventHubSourceWrite
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> source = EventHubSourceWrite('my_event_hub', 'http://myeventhub.com', "My EventHub", 'my_key', 'my_value')
>>> res = client.hosted_extractors.sources.create(source)

Delete source

async SourcesAPI.delete( external_ids: str | SequenceNotStr[str], ignore_unknown_ids: bool = False, force: bool = False, ) → None

Delete one or more sources

Parameters:

external_ids (str | SequenceNotStr[str]) – The external ID provided by the client. Must be unique for the resource type.
ignore_unknown_ids (bool) – Ignore external IDs that are not found rather than throw an exception.
force (bool) – Delete any jobs associated with each item.

Examples

Delete sources by id:

>>> from cognite.client import CogniteClient, AsyncCogniteClient
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> client.hosted_extractors.sources.delete(["myMQTTSource", "MyEventHubSource"])

List sources

async SourcesAPI.list( limit: int | None = 25, ) → SourceList

List sources

Parameters:: limit (int | None) – Maximum number of sources to return. Defaults to 25. Set to -1, float(“inf”) or None to return all items.
Returns:: List of requested sources
Return type:: SourceList

Examples

List sources:

>>> from cognite.client import CogniteClient, AsyncCogniteClient
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> source_list = client.hosted_extractors.sources.list(limit=5)

Iterate over sources, one-by-one:

>>> for source in client.hosted_extractors.sources():
...     source  # do something with the source

Iterate over chunks of sources to reduce memory load:

>>> for source_list in client.hosted_extractors.sources(chunk_size=25):
...     source_list # do something with the sources

Retrieve sources

async SourcesAPI.retrieve( external_ids: str | SequenceNotStr[str], ignore_unknown_ids: bool = False, ) → Source | SourceList

Retrieve one or more sources.

Parameters:

external_ids (str | SequenceNotStr[str]) – The external ID provided by the client. Must be unique for the resource type.
ignore_unknown_ids (bool) – Ignore external IDs that are not found rather than throw an exception.

Returns:

Requested sources

Return type:

Source | SourceList

Examples

>>> from cognite.client import CogniteClient, AsyncCogniteClient
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> res = client.hosted_extractors.sources.retrieve('myMQTTSource')

Get multiple sources by id:

>>> res = client.hosted_extractors.sources.retrieve(["myMQTTSource", "MyEventHubSource"], ignore_unknown_ids=True)

Update sources

async SourcesAPI.update( items: SourceWrite | SourceUpdate | Sequence[SourceWrite | SourceUpdate], mode: Literal['replace_ignore_null', 'patch', 'replace'] = 'replace_ignore_null', ) → Source | SourceList

Update one or more sources.

Parameters:

items (SourceWrite | SourceUpdate | Sequence[SourceWrite | SourceUpdate]) – Source(s) to update.
mode (Literal['replace_ignore_null', 'patch', 'replace']) – How to update data when a non-update object is given (SourceWrite). If you use ‘replace_ignore_null’, only the fields you have set will be used to replace existing (default). Using ‘replace’ will additionally clear all the fields that are not specified by you. Last option, ‘patch’, will update only the fields you have set and for container-like fields such as metadata or labels, add the values to the existing. For more details, see Update and Upsert Mode Parameter.

Returns:

Updated source(s)

Return type:

Source | SourceList

Examples

Update source:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes.hosted_extractors import EventHubSourceUpdate
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> source = EventHubSourceUpdate('my_event_hub').event_hub_name.set("My Updated EventHub")
>>> res = client.hosted_extractors.sources.update(source)

Jobs

Create new job

async JobsAPI.create( items: JobWrite | Sequence[JobWrite], ) → Job | JobList

Create one or more jobs.

Parameters:: items (JobWrite | Sequence[JobWrite]) – Job(s) to create.
Returns:: Created job(s)
Return type:: Job | JobList

Examples

Create new job:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes.hosted_extractors import EventHubSourceWrite
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> job_write = EventHubSourceWrite('my_event_hub', 'http://myeventhub.com', "My EventHub", 'my_key', 'my_value')
>>> job = client.hosted_extractors.jobs.create(job_write)

Delete job

async JobsAPI.delete( external_ids: str | SequenceNotStr[str], ignore_unknown_ids: bool = False, ) → None

Delete one or more jobs

Parameters:

external_ids (str | SequenceNotStr[str]) – The external ID provided by the client. Must be unique for the resource type.
ignore_unknown_ids (bool) – Ignore external IDs that are not found

Examples

Delete jobs by external id:

>>> from cognite.client import CogniteClient, AsyncCogniteClient
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> client.hosted_extractors.jobs.delete(["myMQTTJob", "MyEventHubJob"])

List jobs

async JobsAPI.list( limit: int | None = 25, ) → JobList

List jobs

Parameters:: limit (int | None) – Maximum number of jobs to return. Defaults to 25. Set to -1, float(“inf”) or None to return all items.
Returns:: List of requested jobs
Return type:: JobList

Examples

List jobs:

>>> from cognite.client import CogniteClient, AsyncCogniteClient
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> job_list = client.hosted_extractors.jobs.list(limit=5)

Iterate over jobs, one-by-one:

>>> for job in client.hosted_extractors.jobs():
...     job  # do something with the job

Iterate over chunks of jobs to reduce memory load:

>>> for job_list in client.hosted_extractors.jobs(chunk_size=25):
...     job_list # do something with the jobs

Retrieve jobs

async JobsAPI.retrieve( external_ids: str | SequenceNotStr[str], ignore_unknown_ids: bool = False, ) → Job | None | JobList

Retrieve one or more jobs.

Parameters:

external_ids (str | SequenceNotStr[str]) – The external ID provided by the client. Must be unique for the job type.
ignore_unknown_ids (bool) – Ignore external IDs that are not found

Returns:

Requested jobs

Return type:

Job | None | JobList

Examples

>>> from cognite.client import CogniteClient, AsyncCogniteClient
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> res = client.hosted_extractors.jobs.retrieve('myJob')

Get multiple jobs by id:

>>> res = client.hosted_extractors.jobs.retrieve(["myJob", "myOtherJob"], ignore_unknown_ids=True)

Update job

async JobsAPI.update( items: JobWrite | JobUpdate | Sequence[JobWrite | JobUpdate], mode: Literal['replace_ignore_null', 'patch', 'replace'] = 'replace_ignore_null', ) → Job | JobList

Update one or more jobs.

Parameters:

items (JobWrite | JobUpdate | Sequence[JobWrite | JobUpdate]) – Job(s) to update.
mode (Literal['replace_ignore_null', 'patch', 'replace']) – How to update data when a non-update object is given (JobWrite). If you use ‘replace_ignore_null’, only the fields you have set will be used to replace existing (default). Using ‘replace’ will additionally clear all the fields that are not specified by you. Last option, ‘patch’, will update only the fields you have set and for container-like fields such as metadata or labels, add the values to the existing. For more details, see Update and Upsert Mode Parameter.

Returns:

Updated job(s)

Return type:

Job | JobList

Examples

Update job:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes.hosted_extractors import EventHubSourceUpdate
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> job = EventHubSourceUpdate('my_event_hub').event_hub_name.set("My Updated EventHub")
>>> updated_job = client.hosted_extractors.jobs.update(job)

List job logs

async JobsAPI.list_logs( job: str | None = None, source: str | None = None, destination: str | None = None, limit: int | None = 25, ) → JobLogsList

List job logs.

Parameters:

job (str | None) – Require returned logs to belong to the job given by this external ID.
source (str | None) – Require returned logs to belong to the any job with source given by this external ID.
destination (str | None) – Require returned logs to belong to the any job with destination given by this external ID.
limit (int | None) – Maximum number of logs to return. Defaults to 25. Set to -1, float(“inf”) or None to return all items.

Returns:

List of requested job logs

Return type:

JobLogsList

Examples

Reqests logs for a specific job:

>>> from cognite.client import CogniteClient, AsyncCogniteClient
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> res = client.hosted_extractors.jobs.list_logs(job="myJob")

List job metrics

async JobsAPI.list_metrics( job: str | None = None, source: str | None = None, destination: str | None = None, limit: int | None = 25, ) → JobMetricsList

List job metrics.

Parameters:

job (str | None) – Require returned metrics to belong to the job given by this external ID.
source (str | None) – Require returned metrics to belong to the any job with source given by this external ID.
destination (str | None) – Require returned metrics to belong to the any job with destination given by this external ID.
limit (int | None) – Maximum number of metrics to return. Defaults to 25. Set to -1, float(“inf”) or None to return all items.

Returns:

List of requested job metrics

Return type:

JobMetricsList

Examples

Reqests metrics for a specific job:

>>> from cognite.client import CogniteClient, AsyncCogniteClient
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> res = client.hosted_extractors.jobs.list_metrics(job="myJob")

Destinations

Create new destinations

async DestinationsAPI.create( items: DestinationWrite | Sequence[DestinationWrite], ) → Destination | DestinationList

Create one or more destinations.

Parameters:: items (DestinationWrite | Sequence[DestinationWrite]) – Destination(s) to create.
Returns:: Created destination(s)
Return type:: Destination | DestinationList

Examples

Create new destination:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes.hosted_extractors import DestinationWrite, SessionWrite
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> destination = DestinationWrite(external_id='my_dest', credentials=SessionWrite("my_nonce"), target_data_set_id=123)
>>> res = client.hosted_extractors.destinations.create(destination)

Delete destinations

async DestinationsAPI.delete( external_ids: str | SequenceNotStr[str], ignore_unknown_ids: bool = False, force: bool = False, ) → None

Delete one or more destsinations

Parameters:

external_ids (str | SequenceNotStr[str]) – The external ID provided by the client. Must be unique for the resource type.
ignore_unknown_ids (bool) – Ignore external IDs that are not found
force (bool) – Delete any jobs associated with each item.

Examples

Delete destinations by id:

>>> from cognite.client import CogniteClient, AsyncCogniteClient
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> client.hosted_extractors.destinations.delete(["myDest", "MyDest2"])

List destinations

async DestinationsAPI.list( limit: int | None = 25, ) → DestinationList

List destinations

Parameters:: limit (int | None) – Maximum number of destinations to return. Defaults to 25. Set to -1, float(“inf”) or None to return all items.
Returns:: List of requested destinations
Return type:: DestinationList

Examples

List destinations:

>>> from cognite.client import CogniteClient, AsyncCogniteClient
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> destination_list = client.hosted_extractors.destinations.list(limit=5)

Iterate over destinations, one-by-one:

>>> for destination in client.hosted_extractors.destinations():
...     destination  # do something with the destination

Iterate over chunks of destinations to reduce memory load:

>>> for destination_list in client.hosted_extractors.destinations(chunk_size=25):
...     destination_list # do something with the destinationss

Retrieve destinations

async DestinationsAPI.retrieve( external_ids: str | SequenceNotStr[str], ignore_unknown_ids: bool = False, ) → Destination | DestinationList

Retrieve one or more destinations.

Parameters:

external_ids (str | SequenceNotStr[str]) – The external ID provided by the client. Must be unique for the resource type.
ignore_unknown_ids (bool) – Ignore external IDs that are not found

Returns:

Requested destinations

Return type:

Destination | DestinationList

Examples

>>> from cognite.client import CogniteClient, AsyncCogniteClient
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> res = client.hosted_extractors.destinations.retrieve('myDestination')

Get multiple destinations by id:

>>> res = client.hosted_extractors.destinations.retrieve(["myDestination", "myDestination2"], ignore_unknown_ids=True)

Update destinations

async DestinationsAPI.update( items: DestinationWrite | DestinationUpdate | Sequence[DestinationWrite | DestinationUpdate], mode: Literal['replace_ignore_null', 'patch', 'replace'] = 'replace_ignore_null', ) → Destination | DestinationList

Update one or more destinations.

Parameters:

items (DestinationWrite | DestinationUpdate | Sequence[DestinationWrite | DestinationUpdate]) – Destination(s) to update.
mode (Literal['replace_ignore_null', 'patch', 'replace']) – How to update data when a non-update object is given (DestinationWrite). If you use ‘replace_ignore_null’, only the fields you have set will be used to replace existing (default). Using ‘replace’ will additionally clear all the fields that are not specified by you. Last option, ‘patch’, will update only the fields you have set and for container-like fields such as metadata or labels, add the values to the existing. For more details, see Update and Upsert Mode Parameter.

Returns:

Updated destination(s)

Return type:

Destination | DestinationList

Examples

Update destination:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes.hosted_extractors import DestinationUpdate
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> destination = DestinationUpdate('my_dest').target_data_set_id.set(123)
>>> res = client.hosted_extractors.destinations.update(destination)

Mappings

Create new mappings

async MappingsAPI.create( items: MappingWrite | Sequence[MappingWrite], ) → Mapping | MappingList

Create one or more mappings.

Parameters:: items (MappingWrite | Sequence[MappingWrite]) – Mapping(s) to create.
Returns:: Created mapping(s)
Return type:: Mapping | MappingList

Examples

Create new mapping:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes.hosted_extractors import MappingWrite, CustomMapping
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> mapping = MappingWrite(external_id="my_mapping", mapping=CustomMapping("some expression"), published=True, input="json")
>>> res = client.hosted_extractors.mappings.create(mapping)

Delete mappings

async MappingsAPI.delete( external_ids: str | SequenceNotStr[str], ignore_unknown_ids: bool = False, force: bool = False, ) → None

Delete one or more mappings

Parameters:

external_ids (str | SequenceNotStr[str]) – The external ID provided by the client. Must be unique for the resource type.
ignore_unknown_ids (bool) – Ignore external IDs that are not found
force (bool) – Delete any jobs associated with each item.

Examples

Delete mappings by id:

>>> from cognite.client import CogniteClient, AsyncCogniteClient
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> client.hosted_extractors.mappings.delete(["myMapping", "MyMapping2"])

List mappings

async MappingsAPI.list( limit: int | None = 25, ) → MappingList

List mappings

Parameters:: limit (int | None) – Maximum number of mappings to return. Defaults to 25. Set to -1, float(“inf”) or None to return all items.
Returns:: List of requested mappings
Return type:: MappingList

Examples

List mappings:

>>> from cognite.client import CogniteClient, AsyncCogniteClient
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> mapping_list = client.hosted_extractors.mappings.list(limit=5)

Iterate over mappings, one-by-one:

>>> for mapping in client.hosted_extractors.mappings():
...     mapping  # do something with the mapping

Iterate over chunks of mappings to reduce memory load:

>>> for mapping_list in client.hosted_extractors.mappings(chunk_size=25):
...     mapping_list # do something with the mappings

Retrieve mappings

async MappingsAPI.retrieve( external_ids: str | SequenceNotStr[str], ignore_unknown_ids: bool = False, ) → Mapping | MappingList

Retrieve one or more mappings.

Parameters:

external_ids (str | SequenceNotStr[str]) – The external ID provided by the client. Must be unique for the resource type.
ignore_unknown_ids (bool) – Ignore external IDs that are not found

Returns:

Requested mappings

Return type:

Mapping | MappingList

Examples

>>> from cognite.client import CogniteClient, AsyncCogniteClient
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> res = client.hosted_extractors.mappings.retrieve('myMapping')

Get multiple mappings by id:

>>> res = client.hosted_extractors.mappings.retrieve(["myMapping", "myMapping2"], ignore_unknown_ids=True)

Update mappings

async MappingsAPI.update( items: MappingWrite | MappingUpdate | Sequence[MappingWrite | MappingUpdate], ) → Mapping | MappingList

Update one or more mappings.

Parameters:: items (MappingWrite | MappingUpdate | Sequence[MappingWrite | MappingUpdate]) – Mapping(s) to update.
Returns:: Updated mapping(s)
Return type:: Mapping | MappingList

Examples

Update mapping:

>>> from cognite.client import CogniteClient
>>> from cognite.client.data_classes.hosted_extractors import MappingUpdate
>>> client = CogniteClient()
>>> # async_client = AsyncCogniteClient()  # another option
>>> mapping = MappingUpdate('my_mapping').published.set(False)
>>> res = client.hosted_extractors.mappings.update(mapping)