1. Packages
  2. Airbyte Provider
  3. API Docs
  4. SourceHuggingFaceDatasets
airbyte 0.10.0 published on Monday, Apr 14, 2025 by airbytehq

airbyte.SourceHuggingFaceDatasets

Explore with Pulumi AI

SourceHuggingFaceDatasets Resource

Example Usage

Coming soon!
Coming soon!
Coming soon!
Coming soon!
package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.airbyte.SourceHuggingFaceDatasets;
import com.pulumi.airbyte.SourceHuggingFaceDatasetsArgs;
import com.pulumi.airbyte.inputs.SourceHuggingFaceDatasetsConfigurationArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
    public static void main(String[] args) {
        Pulumi.run(App::stack);
    }

    public static void stack(Context ctx) {
        var mySourceHuggingfacedatasets = new SourceHuggingFaceDatasets("mySourceHuggingfacedatasets", SourceHuggingFaceDatasetsArgs.builder()
            .configuration(SourceHuggingFaceDatasetsConfigurationArgs.builder()
                .dataset_name("...my_dataset_name...")
                .dataset_splits("{ \"see\": \"documentation\" }")
                .dataset_subsets("{ \"see\": \"documentation\" }")
                .build())
            .definitionId("f15ad351-9b44-4be5-9747-f4289e4b150e")
            .secretId("...my_secret_id...")
            .workspaceId("aca928e7-7bba-4a6a-9a12-c17667fab2c3")
            .build());

    }
}
Copy
resources:
  mySourceHuggingfacedatasets:
    type: airbyte:SourceHuggingFaceDatasets
    properties:
      configuration:
        dataset_name: '...my_dataset_name...'
        dataset_splits:
          - '{ "see": "documentation" }'
        dataset_subsets:
          - '{ "see": "documentation" }'
      definitionId: f15ad351-9b44-4be5-9747-f4289e4b150e
      secretId: '...my_secret_id...'
      workspaceId: aca928e7-7bba-4a6a-9a12-c17667fab2c3
Copy

Create SourceHuggingFaceDatasets Resource

Resources are created with functions called constructors. To learn more about declaring and configuring resources, see Resources.

Constructor syntax

new SourceHuggingFaceDatasets(name: string, args: SourceHuggingFaceDatasetsArgs, opts?: CustomResourceOptions);
@overload
def SourceHuggingFaceDatasets(resource_name: str,
                              args: SourceHuggingFaceDatasetsArgs,
                              opts: Optional[ResourceOptions] = None)

@overload
def SourceHuggingFaceDatasets(resource_name: str,
                              opts: Optional[ResourceOptions] = None,
                              configuration: Optional[SourceHuggingFaceDatasetsConfigurationArgs] = None,
                              workspace_id: Optional[str] = None,
                              definition_id: Optional[str] = None,
                              name: Optional[str] = None,
                              secret_id: Optional[str] = None)
func NewSourceHuggingFaceDatasets(ctx *Context, name string, args SourceHuggingFaceDatasetsArgs, opts ...ResourceOption) (*SourceHuggingFaceDatasets, error)
public SourceHuggingFaceDatasets(string name, SourceHuggingFaceDatasetsArgs args, CustomResourceOptions? opts = null)
public SourceHuggingFaceDatasets(String name, SourceHuggingFaceDatasetsArgs args)
public SourceHuggingFaceDatasets(String name, SourceHuggingFaceDatasetsArgs args, CustomResourceOptions options)
type: airbyte:SourceHuggingFaceDatasets
properties: # The arguments to resource properties.
options: # Bag of options to control resource's behavior.

Parameters

name This property is required. string
The unique name of the resource.
args This property is required. SourceHuggingFaceDatasetsArgs
The arguments to resource properties.
opts CustomResourceOptions
Bag of options to control resource's behavior.
resource_name This property is required. str
The unique name of the resource.
args This property is required. SourceHuggingFaceDatasetsArgs
The arguments to resource properties.
opts ResourceOptions
Bag of options to control resource's behavior.
ctx Context
Context object for the current deployment.
name This property is required. string
The unique name of the resource.
args This property is required. SourceHuggingFaceDatasetsArgs
The arguments to resource properties.
opts ResourceOption
Bag of options to control resource's behavior.
name This property is required. string
The unique name of the resource.
args This property is required. SourceHuggingFaceDatasetsArgs
The arguments to resource properties.
opts CustomResourceOptions
Bag of options to control resource's behavior.
name This property is required. String
The unique name of the resource.
args This property is required. SourceHuggingFaceDatasetsArgs
The arguments to resource properties.
options CustomResourceOptions
Bag of options to control resource's behavior.

Constructor example

The following reference example uses placeholder values for all input properties.

var sourceHuggingFaceDatasetsResource = new Airbyte.SourceHuggingFaceDatasets("sourceHuggingFaceDatasetsResource", new()
{
    Configuration = new Airbyte.Inputs.SourceHuggingFaceDatasetsConfigurationArgs
    {
        DatasetName = "string",
        DatasetSplits = new[]
        {
            "string",
        },
        DatasetSubsets = new[]
        {
            "string",
        },
    },
    WorkspaceId = "string",
    DefinitionId = "string",
    Name = "string",
    SecretId = "string",
});
Copy
example, err := airbyte.NewSourceHuggingFaceDatasets(ctx, "sourceHuggingFaceDatasetsResource", &airbyte.SourceHuggingFaceDatasetsArgs{
Configuration: &.SourceHuggingFaceDatasetsConfigurationArgs{
DatasetName: pulumi.String("string"),
DatasetSplits: pulumi.StringArray{
pulumi.String("string"),
},
DatasetSubsets: pulumi.StringArray{
pulumi.String("string"),
},
},
WorkspaceId: pulumi.String("string"),
DefinitionId: pulumi.String("string"),
Name: pulumi.String("string"),
SecretId: pulumi.String("string"),
})
Copy
var sourceHuggingFaceDatasetsResource = new SourceHuggingFaceDatasets("sourceHuggingFaceDatasetsResource", SourceHuggingFaceDatasetsArgs.builder()
    .configuration(SourceHuggingFaceDatasetsConfigurationArgs.builder()
        .datasetName("string")
        .datasetSplits("string")
        .datasetSubsets("string")
        .build())
    .workspaceId("string")
    .definitionId("string")
    .name("string")
    .secretId("string")
    .build());
Copy
source_hugging_face_datasets_resource = airbyte.SourceHuggingFaceDatasets("sourceHuggingFaceDatasetsResource",
    configuration={
        "dataset_name": "string",
        "dataset_splits": ["string"],
        "dataset_subsets": ["string"],
    },
    workspace_id="string",
    definition_id="string",
    name="string",
    secret_id="string")
Copy
const sourceHuggingFaceDatasetsResource = new airbyte.SourceHuggingFaceDatasets("sourceHuggingFaceDatasetsResource", {
    configuration: {
        datasetName: "string",
        datasetSplits: ["string"],
        datasetSubsets: ["string"],
    },
    workspaceId: "string",
    definitionId: "string",
    name: "string",
    secretId: "string",
});
Copy
type: airbyte:SourceHuggingFaceDatasets
properties:
    configuration:
        datasetName: string
        datasetSplits:
            - string
        datasetSubsets:
            - string
    definitionId: string
    name: string
    secretId: string
    workspaceId: string
Copy

SourceHuggingFaceDatasets Resource Properties

To learn more about resource properties and how to use them, see Inputs and Outputs in the Architecture and Concepts docs.

Inputs

In Python, inputs that are objects can be passed either as argument classes or as dictionary literals.

The SourceHuggingFaceDatasets resource accepts the following input properties:

Configuration This property is required. SourceHuggingFaceDatasetsConfiguration
WorkspaceId This property is required. string
DefinitionId string
The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
Name string
Name of the source e.g. dev-mysql-instance.
SecretId string
Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
Configuration This property is required. SourceHuggingFaceDatasetsConfigurationArgs
WorkspaceId This property is required. string
DefinitionId string
The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
Name string
Name of the source e.g. dev-mysql-instance.
SecretId string
Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
configuration This property is required. SourceHuggingFaceDatasetsConfiguration
workspaceId This property is required. String
definitionId String
The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name String
Name of the source e.g. dev-mysql-instance.
secretId String
Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
configuration This property is required. SourceHuggingFaceDatasetsConfiguration
workspaceId This property is required. string
definitionId string
The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name string
Name of the source e.g. dev-mysql-instance.
secretId string
Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
configuration This property is required. SourceHuggingFaceDatasetsConfigurationArgs
workspace_id This property is required. str
definition_id str
The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name str
Name of the source e.g. dev-mysql-instance.
secret_id str
Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
configuration This property is required. Property Map
workspaceId This property is required. String
definitionId String
The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name String
Name of the source e.g. dev-mysql-instance.
secretId String
Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.

Outputs

All input properties are implicitly available as output properties. Additionally, the SourceHuggingFaceDatasets resource produces the following output properties:

CreatedAt double
Id string
The provider-assigned unique ID for this managed resource.
ResourceAllocation SourceHuggingFaceDatasetsResourceAllocation
actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
SourceId string
SourceType string
CreatedAt float64
Id string
The provider-assigned unique ID for this managed resource.
ResourceAllocation SourceHuggingFaceDatasetsResourceAllocation
actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
SourceId string
SourceType string
createdAt Double
id String
The provider-assigned unique ID for this managed resource.
resourceAllocation SourceHuggingFaceDatasetsResourceAllocation
actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
sourceId String
sourceType String
createdAt number
id string
The provider-assigned unique ID for this managed resource.
resourceAllocation SourceHuggingFaceDatasetsResourceAllocation
actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
sourceId string
sourceType string
created_at float
id str
The provider-assigned unique ID for this managed resource.
resource_allocation SourceHuggingFaceDatasetsResourceAllocation
actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
source_id str
source_type str
createdAt Number
id String
The provider-assigned unique ID for this managed resource.
resourceAllocation Property Map
actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
sourceId String
sourceType String

Look up Existing SourceHuggingFaceDatasets Resource

Get an existing SourceHuggingFaceDatasets resource’s state with the given name, ID, and optional extra properties used to qualify the lookup.

public static get(name: string, id: Input<ID>, state?: SourceHuggingFaceDatasetsState, opts?: CustomResourceOptions): SourceHuggingFaceDatasets
@staticmethod
def get(resource_name: str,
        id: str,
        opts: Optional[ResourceOptions] = None,
        configuration: Optional[SourceHuggingFaceDatasetsConfigurationArgs] = None,
        created_at: Optional[float] = None,
        definition_id: Optional[str] = None,
        name: Optional[str] = None,
        resource_allocation: Optional[SourceHuggingFaceDatasetsResourceAllocationArgs] = None,
        secret_id: Optional[str] = None,
        source_id: Optional[str] = None,
        source_type: Optional[str] = None,
        workspace_id: Optional[str] = None) -> SourceHuggingFaceDatasets
func GetSourceHuggingFaceDatasets(ctx *Context, name string, id IDInput, state *SourceHuggingFaceDatasetsState, opts ...ResourceOption) (*SourceHuggingFaceDatasets, error)
public static SourceHuggingFaceDatasets Get(string name, Input<string> id, SourceHuggingFaceDatasetsState? state, CustomResourceOptions? opts = null)
public static SourceHuggingFaceDatasets get(String name, Output<String> id, SourceHuggingFaceDatasetsState state, CustomResourceOptions options)
resources:  _:    type: airbyte:SourceHuggingFaceDatasets    get:      id: ${id}
name This property is required.
The unique name of the resulting resource.
id This property is required.
The unique provider ID of the resource to lookup.
state
Any extra arguments used during the lookup.
opts
A bag of options that control this resource's behavior.
resource_name This property is required.
The unique name of the resulting resource.
id This property is required.
The unique provider ID of the resource to lookup.
name This property is required.
The unique name of the resulting resource.
id This property is required.
The unique provider ID of the resource to lookup.
state
Any extra arguments used during the lookup.
opts
A bag of options that control this resource's behavior.
name This property is required.
The unique name of the resulting resource.
id This property is required.
The unique provider ID of the resource to lookup.
state
Any extra arguments used during the lookup.
opts
A bag of options that control this resource's behavior.
name This property is required.
The unique name of the resulting resource.
id This property is required.
The unique provider ID of the resource to lookup.
state
Any extra arguments used during the lookup.
opts
A bag of options that control this resource's behavior.
The following state arguments are supported:
Configuration SourceHuggingFaceDatasetsConfiguration
CreatedAt double
DefinitionId string
The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
Name string
Name of the source e.g. dev-mysql-instance.
ResourceAllocation SourceHuggingFaceDatasetsResourceAllocation
actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
SecretId string
Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
SourceId string
SourceType string
WorkspaceId string
Configuration SourceHuggingFaceDatasetsConfigurationArgs
CreatedAt float64
DefinitionId string
The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
Name string
Name of the source e.g. dev-mysql-instance.
ResourceAllocation SourceHuggingFaceDatasetsResourceAllocationArgs
actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
SecretId string
Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
SourceId string
SourceType string
WorkspaceId string
configuration SourceHuggingFaceDatasetsConfiguration
createdAt Double
definitionId String
The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name String
Name of the source e.g. dev-mysql-instance.
resourceAllocation SourceHuggingFaceDatasetsResourceAllocation
actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
secretId String
Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
sourceId String
sourceType String
workspaceId String
configuration SourceHuggingFaceDatasetsConfiguration
createdAt number
definitionId string
The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name string
Name of the source e.g. dev-mysql-instance.
resourceAllocation SourceHuggingFaceDatasetsResourceAllocation
actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
secretId string
Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
sourceId string
sourceType string
workspaceId string
configuration SourceHuggingFaceDatasetsConfigurationArgs
created_at float
definition_id str
The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name str
Name of the source e.g. dev-mysql-instance.
resource_allocation SourceHuggingFaceDatasetsResourceAllocationArgs
actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
secret_id str
Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
source_id str
source_type str
workspace_id str
configuration Property Map
createdAt Number
definitionId String
The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name String
Name of the source e.g. dev-mysql-instance.
resourceAllocation Property Map
actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
secretId String
Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
sourceId String
sourceType String
workspaceId String

Supporting Types

SourceHuggingFaceDatasetsConfiguration
, SourceHuggingFaceDatasetsConfigurationArgs

DatasetName This property is required. string
DatasetSplits List<string>
Splits to import. Will import all of them if nothing is provided (see https://huggingface.co/docs/dataset-viewer/en/configsandsplits for more details)
DatasetSubsets List<string>
Dataset Subsets to import. Will import all of them if nothing is provided (see https://huggingface.co/docs/dataset-viewer/en/configsandsplits for more details)
DatasetName This property is required. string
DatasetSplits []string
Splits to import. Will import all of them if nothing is provided (see https://huggingface.co/docs/dataset-viewer/en/configsandsplits for more details)
DatasetSubsets []string
Dataset Subsets to import. Will import all of them if nothing is provided (see https://huggingface.co/docs/dataset-viewer/en/configsandsplits for more details)
datasetName This property is required. String
datasetSplits List<String>
Splits to import. Will import all of them if nothing is provided (see https://huggingface.co/docs/dataset-viewer/en/configsandsplits for more details)
datasetSubsets List<String>
Dataset Subsets to import. Will import all of them if nothing is provided (see https://huggingface.co/docs/dataset-viewer/en/configsandsplits for more details)
datasetName This property is required. string
datasetSplits string[]
Splits to import. Will import all of them if nothing is provided (see https://huggingface.co/docs/dataset-viewer/en/configsandsplits for more details)
datasetSubsets string[]
Dataset Subsets to import. Will import all of them if nothing is provided (see https://huggingface.co/docs/dataset-viewer/en/configsandsplits for more details)
dataset_name This property is required. str
dataset_splits Sequence[str]
Splits to import. Will import all of them if nothing is provided (see https://huggingface.co/docs/dataset-viewer/en/configsandsplits for more details)
dataset_subsets Sequence[str]
Dataset Subsets to import. Will import all of them if nothing is provided (see https://huggingface.co/docs/dataset-viewer/en/configsandsplits for more details)
datasetName This property is required. String
datasetSplits List<String>
Splits to import. Will import all of them if nothing is provided (see https://huggingface.co/docs/dataset-viewer/en/configsandsplits for more details)
datasetSubsets List<String>
Dataset Subsets to import. Will import all of them if nothing is provided (see https://huggingface.co/docs/dataset-viewer/en/configsandsplits for more details)

SourceHuggingFaceDatasetsResourceAllocation
, SourceHuggingFaceDatasetsResourceAllocationArgs

default Property Map
optional resource requirements to run workers (blank for unbounded allocations)
jobSpecifics List<Property Map>

SourceHuggingFaceDatasetsResourceAllocationDefault
, SourceHuggingFaceDatasetsResourceAllocationDefaultArgs

SourceHuggingFaceDatasetsResourceAllocationJobSpecific
, SourceHuggingFaceDatasetsResourceAllocationJobSpecificArgs

JobType string
enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
ResourceRequirements SourceHuggingFaceDatasetsResourceAllocationJobSpecificResourceRequirements
optional resource requirements to run workers (blank for unbounded allocations)
JobType string
enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
ResourceRequirements SourceHuggingFaceDatasetsResourceAllocationJobSpecificResourceRequirements
optional resource requirements to run workers (blank for unbounded allocations)
jobType String
enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
resourceRequirements SourceHuggingFaceDatasetsResourceAllocationJobSpecificResourceRequirements
optional resource requirements to run workers (blank for unbounded allocations)
jobType string
enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
resourceRequirements SourceHuggingFaceDatasetsResourceAllocationJobSpecificResourceRequirements
optional resource requirements to run workers (blank for unbounded allocations)
job_type str
enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
resource_requirements SourceHuggingFaceDatasetsResourceAllocationJobSpecificResourceRequirements
optional resource requirements to run workers (blank for unbounded allocations)
jobType String
enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
resourceRequirements Property Map
optional resource requirements to run workers (blank for unbounded allocations)

SourceHuggingFaceDatasetsResourceAllocationJobSpecificResourceRequirements
, SourceHuggingFaceDatasetsResourceAllocationJobSpecificResourceRequirementsArgs

Import

$ pulumi import airbyte:index/sourceHuggingFaceDatasets:SourceHuggingFaceDatasets my_airbyte_source_hugging_face_datasets ""
Copy

To learn more about importing existing cloud resources, see Importing resources.

Package Details

Repository
airbyte airbytehq/terraform-provider-airbyte
License
Notes
This Pulumi package is based on the airbyte Terraform Provider.