airbyte 0.8.0-beta2, Mar 27 25

airbyte 0.8.0-beta2 published on Thursday, Mar 27, 2025 by airbytehq

airbyte.DestinationS3Glue

Explore with Pulumi AI

airbyte 0.8.0-beta2 published on Thursday, Mar 27, 2025 by airbytehq

Example Usage

Coming soon!

Coming soon!

Coming soon!

Coming soon!

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.airbyte.DestinationS3Glue;
import com.pulumi.airbyte.DestinationS3GlueArgs;
import com.pulumi.airbyte.inputs.DestinationS3GlueConfigurationArgs;
import com.pulumi.airbyte.inputs.DestinationS3GlueConfigurationFormatArgs;
import com.pulumi.airbyte.inputs.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonArgs;
import com.pulumi.airbyte.inputs.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
    public static void main(String[] args) {
        Pulumi.run(App::stack);
    }

    public static void stack(Context ctx) {
        var myDestinationS3glue = new DestinationS3Glue("myDestinationS3glue", DestinationS3GlueArgs.builder()
            .configuration(DestinationS3GlueConfigurationArgs.builder()
                .access_key_id("A012345678910EXAMPLE")
                .file_name_pattern("{date}")
                .format(DestinationS3GlueConfigurationFormatArgs.builder()
                    .jsonLinesNewlineDelimitedJson(DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonArgs.builder()
                        .compression()
                        .flattening("No flattening")
                        .formatType("JSONL")
                        .build())
                    .build())
                .glue_database("airbyte_database")
                .glue_serialization_library("org.openx.data.jsonserde.JsonSerDe")
                .s3_bucket_name("airbyte_sync")
                .s3_bucket_path("data_sync/test")
                .s3_bucket_region("us-gov-west-1")
                .s3_endpoint("http://localhost:9000")
                .s3_path_format(String.format("%s/%s/%s_%s_%s_%s_", NAMESPACE,STREAM_NAME,YEAR,MONTH,DAY,EPOCH))
                .secret_access_key("a012345678910ABCDEFGH/AbCdEfGhEXAMPLEKEY")
                .build())
            .definitionId("7a9c505b-1990-4b38-b243-1c254ff78c36")
            .workspaceId("e7df4138-2176-42e6-aa11-ed72fe60b365")
            .build());

    }
}

resources:
  myDestinationS3glue:
    type: airbyte:DestinationS3Glue
    properties:
      configuration:
        access_key_id: A012345678910EXAMPLE
        file_name_pattern: '{date}'
        format:
          jsonLinesNewlineDelimitedJson:
            compression: {}
            flattening: No flattening
            formatType: JSONL
        glue_database: airbyte_database
        glue_serialization_library: org.openx.data.jsonserde.JsonSerDe
        s3_bucket_name: airbyte_sync
        s3_bucket_path: data_sync/test
        s3_bucket_region: us-gov-west-1
        s3_endpoint: http://localhost:9000
        s3_path_format: ${NAMESPACE}/${STREAM_NAME}/${YEAR}_${MONTH}_${DAY}_${EPOCH}_
        secret_access_key: a012345678910ABCDEFGH/AbCdEfGhEXAMPLEKEY
      definitionId: 7a9c505b-1990-4b38-b243-1c254ff78c36
      workspaceId: e7df4138-2176-42e6-aa11-ed72fe60b365

Create DestinationS3Glue Resource

Resources are created with functions called constructors. To learn more about declaring and configuring resources, see Resources.

Constructor syntax

new DestinationS3Glue(name: string, args: DestinationS3GlueArgs, opts?: CustomResourceOptions);

@overload
def DestinationS3Glue(resource_name: str,
                      args: DestinationS3GlueArgs,
                      opts: Optional[ResourceOptions] = None)

@overload
def DestinationS3Glue(resource_name: str,
                      opts: Optional[ResourceOptions] = None,
                      configuration: Optional[DestinationS3GlueConfigurationArgs] = None,
                      workspace_id: Optional[str] = None,
                      definition_id: Optional[str] = None,
                      name: Optional[str] = None)

func NewDestinationS3Glue(ctx *Context, name string, args DestinationS3GlueArgs, opts ...ResourceOption) (*DestinationS3Glue, error)

public DestinationS3Glue(string name, DestinationS3GlueArgs args, CustomResourceOptions? opts = null)

public DestinationS3Glue(String name, DestinationS3GlueArgs args)
public DestinationS3Glue(String name, DestinationS3GlueArgs args, CustomResourceOptions options)

type: airbyte:DestinationS3Glue
properties: # The arguments to resource properties.
options: # Bag of options to control resource's behavior.

Parameters

name string: The unique name of the resource.
args DestinationS3GlueArgs: The arguments to resource properties.
opts CustomResourceOptions: Bag of options to control resource's behavior.

resource_name str: The unique name of the resource.
args DestinationS3GlueArgs: The arguments to resource properties.
opts ResourceOptions: Bag of options to control resource's behavior.

ctx Context: Context object for the current deployment.
name string: The unique name of the resource.
args DestinationS3GlueArgs: The arguments to resource properties.
opts ResourceOption: Bag of options to control resource's behavior.

name string: The unique name of the resource.
args DestinationS3GlueArgs: The arguments to resource properties.
opts CustomResourceOptions: Bag of options to control resource's behavior.

name String: The unique name of the resource.
args DestinationS3GlueArgs: The arguments to resource properties.
options CustomResourceOptions: Bag of options to control resource's behavior.

Constructor example

The following reference example uses placeholder values for all input properties.

var destinationS3GlueResource = new Airbyte.DestinationS3Glue("destinationS3GlueResource", new()
{
    Configuration = new Airbyte.Inputs.DestinationS3GlueConfigurationArgs
    {
        Format = new Airbyte.Inputs.DestinationS3GlueConfigurationFormatArgs
        {
            JsonLinesNewlineDelimitedJson = new Airbyte.Inputs.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonArgs
            {
                Compression = new Airbyte.Inputs.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionArgs
                {
                    Gzip = new Airbyte.Inputs.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzipArgs
                    {
                        CompressionType = "string",
                    },
                    NoCompression = new Airbyte.Inputs.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompressionArgs
                    {
                        CompressionType = "string",
                    },
                },
                Flattening = "string",
                FormatType = "string",
            },
        },
        GlueDatabase = "string",
        S3BucketName = "string",
        S3BucketPath = "string",
        AccessKeyId = "string",
        FileNamePattern = "string",
        GlueSerializationLibrary = "string",
        S3BucketRegion = "string",
        S3Endpoint = "string",
        S3PathFormat = "string",
        SecretAccessKey = "string",
    },
    WorkspaceId = "string",
    DefinitionId = "string",
    Name = "string",
});

example, err := airbyte.NewDestinationS3Glue(ctx, "destinationS3GlueResource", &airbyte.DestinationS3GlueArgs{
Configuration: &.DestinationS3GlueConfigurationArgs{
Format: &.DestinationS3GlueConfigurationFormatArgs{
JsonLinesNewlineDelimitedJson: &.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonArgs{
Compression: &.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionArgs{
Gzip: &.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzipArgs{
CompressionType: pulumi.String("string"),
},
NoCompression: &.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompressionArgs{
CompressionType: pulumi.String("string"),
},
},
Flattening: pulumi.String("string"),
FormatType: pulumi.String("string"),
},
},
GlueDatabase: pulumi.String("string"),
S3BucketName: pulumi.String("string"),
S3BucketPath: pulumi.String("string"),
AccessKeyId: pulumi.String("string"),
FileNamePattern: pulumi.String("string"),
GlueSerializationLibrary: pulumi.String("string"),
S3BucketRegion: pulumi.String("string"),
S3Endpoint: pulumi.String("string"),
S3PathFormat: pulumi.String("string"),
SecretAccessKey: pulumi.String("string"),
},
WorkspaceId: pulumi.String("string"),
DefinitionId: pulumi.String("string"),
Name: pulumi.String("string"),
})

var destinationS3GlueResource = new DestinationS3Glue("destinationS3GlueResource", DestinationS3GlueArgs.builder()
    .configuration(DestinationS3GlueConfigurationArgs.builder()
        .format(DestinationS3GlueConfigurationFormatArgs.builder()
            .jsonLinesNewlineDelimitedJson(DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonArgs.builder()
                .compression(DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionArgs.builder()
                    .gzip(DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzipArgs.builder()
                        .compressionType("string")
                        .build())
                    .noCompression(DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompressionArgs.builder()
                        .compressionType("string")
                        .build())
                    .build())
                .flattening("string")
                .formatType("string")
                .build())
            .build())
        .glueDatabase("string")
        .s3BucketName("string")
        .s3BucketPath("string")
        .accessKeyId("string")
        .fileNamePattern("string")
        .glueSerializationLibrary("string")
        .s3BucketRegion("string")
        .s3Endpoint("string")
        .s3PathFormat("string")
        .secretAccessKey("string")
        .build())
    .workspaceId("string")
    .definitionId("string")
    .name("string")
    .build());

destination_s3_glue_resource = airbyte.DestinationS3Glue("destinationS3GlueResource",
    configuration={
        "format": {
            "json_lines_newline_delimited_json": {
                "compression": {
                    "gzip": {
                        "compression_type": "string",
                    },
                    "no_compression": {
                        "compression_type": "string",
                    },
                },
                "flattening": "string",
                "format_type": "string",
            },
        },
        "glue_database": "string",
        "s3_bucket_name": "string",
        "s3_bucket_path": "string",
        "access_key_id": "string",
        "file_name_pattern": "string",
        "glue_serialization_library": "string",
        "s3_bucket_region": "string",
        "s3_endpoint": "string",
        "s3_path_format": "string",
        "secret_access_key": "string",
    },
    workspace_id="string",
    definition_id="string",
    name="string")

const destinationS3GlueResource = new airbyte.DestinationS3Glue("destinationS3GlueResource", {
    configuration: {
        format: {
            jsonLinesNewlineDelimitedJson: {
                compression: {
                    gzip: {
                        compressionType: "string",
                    },
                    noCompression: {
                        compressionType: "string",
                    },
                },
                flattening: "string",
                formatType: "string",
            },
        },
        glueDatabase: "string",
        s3BucketName: "string",
        s3BucketPath: "string",
        accessKeyId: "string",
        fileNamePattern: "string",
        glueSerializationLibrary: "string",
        s3BucketRegion: "string",
        s3Endpoint: "string",
        s3PathFormat: "string",
        secretAccessKey: "string",
    },
    workspaceId: "string",
    definitionId: "string",
    name: "string",
});

type: airbyte:DestinationS3Glue
properties:
    configuration:
        accessKeyId: string
        fileNamePattern: string
        format:
            jsonLinesNewlineDelimitedJson:
                compression:
                    gzip:
                        compressionType: string
                    noCompression:
                        compressionType: string
                flattening: string
                formatType: string
        glueDatabase: string
        glueSerializationLibrary: string
        s3BucketName: string
        s3BucketPath: string
        s3BucketRegion: string
        s3Endpoint: string
        s3PathFormat: string
        secretAccessKey: string
    definitionId: string
    name: string
    workspaceId: string

DestinationS3Glue Resource Properties

To learn more about resource properties and how to use them, see Inputs and Outputs in the Architecture and Concepts docs.

Inputs

In Python, inputs that are objects can be passed either as argument classes or as dictionary literals.

The DestinationS3Glue resource accepts the following input properties:

Configuration DestinationS3GlueConfiguration
WorkspaceId string
DefinitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
Name string: Name of the destination e.g. dev-mysql-instance.

Configuration DestinationS3GlueConfigurationArgs
WorkspaceId string
DefinitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
Name string: Name of the destination e.g. dev-mysql-instance.

configuration DestinationS3GlueConfiguration
workspaceId String
definitionId String: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
name String: Name of the destination e.g. dev-mysql-instance.

configuration DestinationS3GlueConfiguration
workspaceId string
definitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
name string: Name of the destination e.g. dev-mysql-instance.

configuration DestinationS3GlueConfigurationArgs
workspace_id str
definition_id str: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
name str: Name of the destination e.g. dev-mysql-instance.

configuration Property Map
workspaceId String
definitionId String: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
name String: Name of the destination e.g. dev-mysql-instance.

Outputs

All input properties are implicitly available as output properties. Additionally, the DestinationS3Glue resource produces the following output properties:

CreatedAt double
DestinationId string
DestinationType string
Id string: The provider-assigned unique ID for this managed resource.
ResourceAllocation DestinationS3GlueResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.

CreatedAt float64
DestinationId string
DestinationType string
Id string: The provider-assigned unique ID for this managed resource.
ResourceAllocation DestinationS3GlueResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.

createdAt Double
destinationId String
destinationType String
id String: The provider-assigned unique ID for this managed resource.
resourceAllocation DestinationS3GlueResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.

createdAt number
destinationId string
destinationType string
id string: The provider-assigned unique ID for this managed resource.
resourceAllocation DestinationS3GlueResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.

created_at float
destination_id str
destination_type str
id str: The provider-assigned unique ID for this managed resource.
resource_allocation DestinationS3GlueResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.

createdAt Number
destinationId String
destinationType String
id String: The provider-assigned unique ID for this managed resource.
resourceAllocation Property Map: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.

Look up Existing DestinationS3Glue Resource

Get an existing DestinationS3Glue resource’s state with the given name, ID, and optional extra properties used to qualify the lookup.

public static get(name: string, id: Input<ID>, state?: DestinationS3GlueState, opts?: CustomResourceOptions): DestinationS3Glue

@staticmethod
def get(resource_name: str,
        id: str,
        opts: Optional[ResourceOptions] = None,
        configuration: Optional[DestinationS3GlueConfigurationArgs] = None,
        created_at: Optional[float] = None,
        definition_id: Optional[str] = None,
        destination_id: Optional[str] = None,
        destination_type: Optional[str] = None,
        name: Optional[str] = None,
        resource_allocation: Optional[DestinationS3GlueResourceAllocationArgs] = None,
        workspace_id: Optional[str] = None) -> DestinationS3Glue

func GetDestinationS3Glue(ctx *Context, name string, id IDInput, state *DestinationS3GlueState, opts ...ResourceOption) (*DestinationS3Glue, error)

public static DestinationS3Glue Get(string name, Input<string> id, DestinationS3GlueState? state, CustomResourceOptions? opts = null)

public static DestinationS3Glue get(String name, Output<String> id, DestinationS3GlueState state, CustomResourceOptions options)

resources:  _:    type: airbyte:DestinationS3Glue    get:      id: ${id}

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

resource_name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

The following state arguments are supported:

Configuration DestinationS3GlueConfiguration
CreatedAt double
DefinitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
DestinationId string
DestinationType string
Name string: Name of the destination e.g. dev-mysql-instance.
ResourceAllocation DestinationS3GlueResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
WorkspaceId string

Configuration DestinationS3GlueConfigurationArgs
CreatedAt float64
DefinitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
DestinationId string
DestinationType string
Name string: Name of the destination e.g. dev-mysql-instance.
ResourceAllocation DestinationS3GlueResourceAllocationArgs: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
WorkspaceId string

configuration DestinationS3GlueConfiguration
createdAt Double
definitionId String: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
destinationId String
destinationType String
name String: Name of the destination e.g. dev-mysql-instance.
resourceAllocation DestinationS3GlueResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
workspaceId String

configuration DestinationS3GlueConfiguration
createdAt number
definitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
destinationId string
destinationType string
name string: Name of the destination e.g. dev-mysql-instance.
resourceAllocation DestinationS3GlueResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
workspaceId string

configuration DestinationS3GlueConfigurationArgs
created_at float
definition_id str: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
destination_id str
destination_type str
name str: Name of the destination e.g. dev-mysql-instance.
resource_allocation DestinationS3GlueResourceAllocationArgs: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
workspace_id str

configuration Property Map
createdAt Number
definitionId String: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
destinationId String
destinationType String
name String: Name of the destination e.g. dev-mysql-instance.
resourceAllocation Property Map: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
workspaceId String

Supporting Types

DestinationS3GlueConfiguration, DestinationS3GlueConfigurationArgs

Format DestinationS3GlueConfigurationFormat: Format of the data output. See \n\nhere\n\n for more details
GlueDatabase string: Name of the glue database for creating the tables, leave blank if no integration
S3BucketName string: The name of the S3 bucket. Read more \n\nhere\n\n.
S3BucketPath string: Directory under the S3 bucket where data will be written. Read more \n\nhere\n\n
AccessKeyId string: The access key ID to access the S3 bucket. Airbyte requires Read and Write permissions to the given bucket. Read more \n\nhere\n\n.
FileNamePattern string: The pattern allows you to set the file-name format for the S3 staging file(s)
GlueSerializationLibrary string: The library that your query engine will use for reading and writing data in your lake. Default: "org.openx.data.jsonserde.JsonSerDe"; must be one of ["org.openx.data.jsonserde.JsonSerDe", "org.apache.hive.hcatalog.data.JsonSerDe"]
S3BucketRegion string: The region of the S3 bucket. See \n\nhere\n\n for all region codes. Default: ""; must be one of ["", "af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-south-2", "ap-southeast-1", "ap-southeast-2", "ap-southeast-3", "ap-southeast-4", "ca-central-1", "ca-west-1", "cn-north-1", "cn-northwest-1", "eu-central-1", "eu-central-2", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "il-central-1", "me-central-1", "me-south-1", "sa-east-1", "us-east-1", "us-east-2", "us-gov-east-1", "us-gov-west-1", "us-west-1", "us-west-2"]
S3Endpoint string: Your S3 endpoint url. Read more \n\nhere\n\n. Default: ""
S3PathFormat string: Format string on how data will be organized inside the S3 bucket directory. Read more \n\nhere\n\n
SecretAccessKey string: The corresponding secret to the access key ID. Read more \n\nhere\n\n

Format DestinationS3GlueConfigurationFormat: Format of the data output. See \n\nhere\n\n for more details
GlueDatabase string: Name of the glue database for creating the tables, leave blank if no integration
S3BucketName string: The name of the S3 bucket. Read more \n\nhere\n\n.
S3BucketPath string: Directory under the S3 bucket where data will be written. Read more \n\nhere\n\n
AccessKeyId string: The access key ID to access the S3 bucket. Airbyte requires Read and Write permissions to the given bucket. Read more \n\nhere\n\n.
FileNamePattern string: The pattern allows you to set the file-name format for the S3 staging file(s)
GlueSerializationLibrary string: The library that your query engine will use for reading and writing data in your lake. Default: "org.openx.data.jsonserde.JsonSerDe"; must be one of ["org.openx.data.jsonserde.JsonSerDe", "org.apache.hive.hcatalog.data.JsonSerDe"]
S3BucketRegion string: The region of the S3 bucket. See \n\nhere\n\n for all region codes. Default: ""; must be one of ["", "af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-south-2", "ap-southeast-1", "ap-southeast-2", "ap-southeast-3", "ap-southeast-4", "ca-central-1", "ca-west-1", "cn-north-1", "cn-northwest-1", "eu-central-1", "eu-central-2", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "il-central-1", "me-central-1", "me-south-1", "sa-east-1", "us-east-1", "us-east-2", "us-gov-east-1", "us-gov-west-1", "us-west-1", "us-west-2"]
S3Endpoint string: Your S3 endpoint url. Read more \n\nhere\n\n. Default: ""
S3PathFormat string: Format string on how data will be organized inside the S3 bucket directory. Read more \n\nhere\n\n
SecretAccessKey string: The corresponding secret to the access key ID. Read more \n\nhere\n\n

format DestinationS3GlueConfigurationFormat: Format of the data output. See \n\nhere\n\n for more details
glueDatabase String: Name of the glue database for creating the tables, leave blank if no integration
s3BucketName String: The name of the S3 bucket. Read more \n\nhere\n\n.
s3BucketPath String: Directory under the S3 bucket where data will be written. Read more \n\nhere\n\n
accessKeyId String: The access key ID to access the S3 bucket. Airbyte requires Read and Write permissions to the given bucket. Read more \n\nhere\n\n.
fileNamePattern String: The pattern allows you to set the file-name format for the S3 staging file(s)
glueSerializationLibrary String: The library that your query engine will use for reading and writing data in your lake. Default: "org.openx.data.jsonserde.JsonSerDe"; must be one of ["org.openx.data.jsonserde.JsonSerDe", "org.apache.hive.hcatalog.data.JsonSerDe"]
s3BucketRegion String: The region of the S3 bucket. See \n\nhere\n\n for all region codes. Default: ""; must be one of ["", "af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-south-2", "ap-southeast-1", "ap-southeast-2", "ap-southeast-3", "ap-southeast-4", "ca-central-1", "ca-west-1", "cn-north-1", "cn-northwest-1", "eu-central-1", "eu-central-2", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "il-central-1", "me-central-1", "me-south-1", "sa-east-1", "us-east-1", "us-east-2", "us-gov-east-1", "us-gov-west-1", "us-west-1", "us-west-2"]
s3Endpoint String: Your S3 endpoint url. Read more \n\nhere\n\n. Default: ""
s3PathFormat String: Format string on how data will be organized inside the S3 bucket directory. Read more \n\nhere\n\n
secretAccessKey String: The corresponding secret to the access key ID. Read more \n\nhere\n\n

format DestinationS3GlueConfigurationFormat: Format of the data output. See \n\nhere\n\n for more details
glueDatabase string: Name of the glue database for creating the tables, leave blank if no integration
s3BucketName string: The name of the S3 bucket. Read more \n\nhere\n\n.
s3BucketPath string: Directory under the S3 bucket where data will be written. Read more \n\nhere\n\n
accessKeyId string: The access key ID to access the S3 bucket. Airbyte requires Read and Write permissions to the given bucket. Read more \n\nhere\n\n.
fileNamePattern string: The pattern allows you to set the file-name format for the S3 staging file(s)
glueSerializationLibrary string: The library that your query engine will use for reading and writing data in your lake. Default: "org.openx.data.jsonserde.JsonSerDe"; must be one of ["org.openx.data.jsonserde.JsonSerDe", "org.apache.hive.hcatalog.data.JsonSerDe"]
s3BucketRegion string: The region of the S3 bucket. See \n\nhere\n\n for all region codes. Default: ""; must be one of ["", "af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-south-2", "ap-southeast-1", "ap-southeast-2", "ap-southeast-3", "ap-southeast-4", "ca-central-1", "ca-west-1", "cn-north-1", "cn-northwest-1", "eu-central-1", "eu-central-2", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "il-central-1", "me-central-1", "me-south-1", "sa-east-1", "us-east-1", "us-east-2", "us-gov-east-1", "us-gov-west-1", "us-west-1", "us-west-2"]
s3Endpoint string: Your S3 endpoint url. Read more \n\nhere\n\n. Default: ""
s3PathFormat string: Format string on how data will be organized inside the S3 bucket directory. Read more \n\nhere\n\n
secretAccessKey string: The corresponding secret to the access key ID. Read more \n\nhere\n\n

format DestinationS3GlueConfigurationFormat: Format of the data output. See \n\nhere\n\n for more details
glue_database str: Name of the glue database for creating the tables, leave blank if no integration
s3_bucket_name str: The name of the S3 bucket. Read more \n\nhere\n\n.
s3_bucket_path str: Directory under the S3 bucket where data will be written. Read more \n\nhere\n\n
access_key_id str: The access key ID to access the S3 bucket. Airbyte requires Read and Write permissions to the given bucket. Read more \n\nhere\n\n.
file_name_pattern str: The pattern allows you to set the file-name format for the S3 staging file(s)
glue_serialization_library str: The library that your query engine will use for reading and writing data in your lake. Default: "org.openx.data.jsonserde.JsonSerDe"; must be one of ["org.openx.data.jsonserde.JsonSerDe", "org.apache.hive.hcatalog.data.JsonSerDe"]
s3_bucket_region str: The region of the S3 bucket. See \n\nhere\n\n for all region codes. Default: ""; must be one of ["", "af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-south-2", "ap-southeast-1", "ap-southeast-2", "ap-southeast-3", "ap-southeast-4", "ca-central-1", "ca-west-1", "cn-north-1", "cn-northwest-1", "eu-central-1", "eu-central-2", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "il-central-1", "me-central-1", "me-south-1", "sa-east-1", "us-east-1", "us-east-2", "us-gov-east-1", "us-gov-west-1", "us-west-1", "us-west-2"]
s3_endpoint str: Your S3 endpoint url. Read more \n\nhere\n\n. Default: ""
s3_path_format str: Format string on how data will be organized inside the S3 bucket directory. Read more \n\nhere\n\n
secret_access_key str: The corresponding secret to the access key ID. Read more \n\nhere\n\n

format Property Map: Format of the data output. See \n\nhere\n\n for more details
glueDatabase String: Name of the glue database for creating the tables, leave blank if no integration
s3BucketName String: The name of the S3 bucket. Read more \n\nhere\n\n.
s3BucketPath String: Directory under the S3 bucket where data will be written. Read more \n\nhere\n\n
accessKeyId String: The access key ID to access the S3 bucket. Airbyte requires Read and Write permissions to the given bucket. Read more \n\nhere\n\n.
fileNamePattern String: The pattern allows you to set the file-name format for the S3 staging file(s)
glueSerializationLibrary String: The library that your query engine will use for reading and writing data in your lake. Default: "org.openx.data.jsonserde.JsonSerDe"; must be one of ["org.openx.data.jsonserde.JsonSerDe", "org.apache.hive.hcatalog.data.JsonSerDe"]
s3BucketRegion String: The region of the S3 bucket. See \n\nhere\n\n for all region codes. Default: ""; must be one of ["", "af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-south-2", "ap-southeast-1", "ap-southeast-2", "ap-southeast-3", "ap-southeast-4", "ca-central-1", "ca-west-1", "cn-north-1", "cn-northwest-1", "eu-central-1", "eu-central-2", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "il-central-1", "me-central-1", "me-south-1", "sa-east-1", "us-east-1", "us-east-2", "us-gov-east-1", "us-gov-west-1", "us-west-1", "us-west-2"]
s3Endpoint String: Your S3 endpoint url. Read more \n\nhere\n\n. Default: ""
s3PathFormat String: Format string on how data will be organized inside the S3 bucket directory. Read more \n\nhere\n\n
secretAccessKey String: The corresponding secret to the access key ID. Read more \n\nhere\n\n

DestinationS3GlueConfigurationFormat, DestinationS3GlueConfigurationFormatArgs

JsonLinesNewlineDelimitedJson DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJson

JsonLinesNewlineDelimitedJson DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJson

jsonLinesNewlineDelimitedJson DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJson

jsonLinesNewlineDelimitedJson DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJson

json_lines_newline_delimited_json DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJson

jsonLinesNewlineDelimitedJson Property Map

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJson, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonArgs

Compression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompression: Whether the output files should be compressed. If compression is selected, the output filename will have an extra extension (GZIP: ".jsonl.gz").
Flattening string: Whether the input json data should be normalized (flattened) in the output JSON Lines. Please refer to docs for details. Default: "Root level flattening"; must be one of ["No flattening", "Root level flattening"]
FormatType string: Default: "JSONL"; must be "JSONL"

Compression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompression: Whether the output files should be compressed. If compression is selected, the output filename will have an extra extension (GZIP: ".jsonl.gz").
Flattening string: Whether the input json data should be normalized (flattened) in the output JSON Lines. Please refer to docs for details. Default: "Root level flattening"; must be one of ["No flattening", "Root level flattening"]
FormatType string: Default: "JSONL"; must be "JSONL"

compression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompression: Whether the output files should be compressed. If compression is selected, the output filename will have an extra extension (GZIP: ".jsonl.gz").
flattening String: Whether the input json data should be normalized (flattened) in the output JSON Lines. Please refer to docs for details. Default: "Root level flattening"; must be one of ["No flattening", "Root level flattening"]
formatType String: Default: "JSONL"; must be "JSONL"

compression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompression: Whether the output files should be compressed. If compression is selected, the output filename will have an extra extension (GZIP: ".jsonl.gz").
flattening string: Whether the input json data should be normalized (flattened) in the output JSON Lines. Please refer to docs for details. Default: "Root level flattening"; must be one of ["No flattening", "Root level flattening"]
formatType string: Default: "JSONL"; must be "JSONL"

compression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompression: Whether the output files should be compressed. If compression is selected, the output filename will have an extra extension (GZIP: ".jsonl.gz").
flattening str: Whether the input json data should be normalized (flattened) in the output JSON Lines. Please refer to docs for details. Default: "Root level flattening"; must be one of ["No flattening", "Root level flattening"]
format_type str: Default: "JSONL"; must be "JSONL"

compression Property Map: Whether the output files should be compressed. If compression is selected, the output filename will have an extra extension (GZIP: ".jsonl.gz").
flattening String: Whether the input json data should be normalized (flattened) in the output JSON Lines. Please refer to docs for details. Default: "Root level flattening"; must be one of ["No flattening", "Root level flattening"]
formatType String: Default: "JSONL"; must be "JSONL"

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompression, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionArgs

Gzip DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzip
NoCompression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompression

Gzip DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzip
NoCompression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompression

gzip DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzip
noCompression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompression

gzip DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzip
noCompression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompression

gzip DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzip
no_compression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompression

gzip Property Map
noCompression Property Map

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzip, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzipArgs

CompressionType string: Default: "GZIP"; must be "GZIP"

CompressionType string: Default: "GZIP"; must be "GZIP"

compressionType String: Default: "GZIP"; must be "GZIP"

compressionType string: Default: "GZIP"; must be "GZIP"

compression_type str: Default: "GZIP"; must be "GZIP"

compressionType String: Default: "GZIP"; must be "GZIP"

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompression, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompressionArgs

CompressionType string: Default: "No Compression"; must be "No Compression"

CompressionType string: Default: "No Compression"; must be "No Compression"

compressionType String: Default: "No Compression"; must be "No Compression"

compressionType string: Default: "No Compression"; must be "No Compression"

compression_type str: Default: "No Compression"; must be "No Compression"

compressionType String: Default: "No Compression"; must be "No Compression"

DestinationS3GlueResourceAllocation, DestinationS3GlueResourceAllocationArgs

Default DestinationS3GlueResourceAllocationDefault: optional resource requirements to run workers (blank for unbounded allocations)
JobSpecifics List<DestinationS3GlueResourceAllocationJobSpecific>

Default DestinationS3GlueResourceAllocationDefault: optional resource requirements to run workers (blank for unbounded allocations)
JobSpecifics []DestinationS3GlueResourceAllocationJobSpecific

default_ DestinationS3GlueResourceAllocationDefault: optional resource requirements to run workers (blank for unbounded allocations)
jobSpecifics List<DestinationS3GlueResourceAllocationJobSpecific>

default DestinationS3GlueResourceAllocationDefault: optional resource requirements to run workers (blank for unbounded allocations)
jobSpecifics DestinationS3GlueResourceAllocationJobSpecific[]

default DestinationS3GlueResourceAllocationDefault: optional resource requirements to run workers (blank for unbounded allocations)
job_specifics Sequence[DestinationS3GlueResourceAllocationJobSpecific]

default Property Map: optional resource requirements to run workers (blank for unbounded allocations)
jobSpecifics List<Property Map>

DestinationS3GlueResourceAllocationDefault, DestinationS3GlueResourceAllocationDefaultArgs

CpuLimit string
CpuRequest string
EphemeralStorageLimit string
EphemeralStorageRequest string
MemoryLimit string
MemoryRequest string

CpuLimit string
CpuRequest string
EphemeralStorageLimit string
EphemeralStorageRequest string
MemoryLimit string
MemoryRequest string

cpuLimit String
cpuRequest String
ephemeralStorageLimit String
ephemeralStorageRequest String
memoryLimit String
memoryRequest String

cpuLimit string
cpuRequest string
ephemeralStorageLimit string
ephemeralStorageRequest string
memoryLimit string
memoryRequest string

cpu_limit str
cpu_request str
ephemeral_storage_limit str
ephemeral_storage_request str
memory_limit str
memory_request str

cpuLimit String
cpuRequest String
ephemeralStorageLimit String
ephemeralStorageRequest String
memoryLimit String
memoryRequest String

DestinationS3GlueResourceAllocationJobSpecific, DestinationS3GlueResourceAllocationJobSpecificArgs

JobType string: enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
ResourceRequirements DestinationS3GlueResourceAllocationJobSpecificResourceRequirements: optional resource requirements to run workers (blank for unbounded allocations)

JobType string: enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
ResourceRequirements DestinationS3GlueResourceAllocationJobSpecificResourceRequirements: optional resource requirements to run workers (blank for unbounded allocations)

jobType String: enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
resourceRequirements DestinationS3GlueResourceAllocationJobSpecificResourceRequirements: optional resource requirements to run workers (blank for unbounded allocations)

jobType string: enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
resourceRequirements DestinationS3GlueResourceAllocationJobSpecificResourceRequirements: optional resource requirements to run workers (blank for unbounded allocations)

job_type str: enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
resource_requirements DestinationS3GlueResourceAllocationJobSpecificResourceRequirements: optional resource requirements to run workers (blank for unbounded allocations)

jobType String: enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
resourceRequirements Property Map: optional resource requirements to run workers (blank for unbounded allocations)

DestinationS3GlueResourceAllocationJobSpecificResourceRequirements, DestinationS3GlueResourceAllocationJobSpecificResourceRequirementsArgs

CpuLimit string
CpuRequest string
EphemeralStorageLimit string
EphemeralStorageRequest string
MemoryLimit string
MemoryRequest string

CpuLimit string
CpuRequest string
EphemeralStorageLimit string
EphemeralStorageRequest string
MemoryLimit string
MemoryRequest string

cpuLimit String
cpuRequest String
ephemeralStorageLimit String
ephemeralStorageRequest String
memoryLimit String
memoryRequest String

cpuLimit string
cpuRequest string
ephemeralStorageLimit string
ephemeralStorageRequest string
memoryLimit string
memoryRequest string

cpu_limit str
cpu_request str
ephemeral_storage_limit str
ephemeral_storage_request str
memory_limit str
memory_request str

cpuLimit String
cpuRequest String
ephemeralStorageLimit String
ephemeralStorageRequest String
memoryLimit String
memoryRequest String

Import

$ pulumi import airbyte:index/destinationS3Glue:DestinationS3Glue my_airbyte_destination_s3_glue ""

To learn more about importing existing cloud resources, see Importing resources.

Package Details

Repository: airbyte airbytehq/terraform-provider-airbyte
License
Notes: This Pulumi package is based on the airbyte Terraform Provider.

airbyte 0.8.0-beta2 published on Thursday, Mar 27, 2025 by airbytehq

airbytehq/terraform-provider-airbyte

airbyte.DestinationS3Glue

On this page

On this page

Example Usage

Create DestinationS3Glue Resource

Constructor syntax

Parameters

Constructor example

DestinationS3Glue Resource Properties

Inputs

Outputs

Look up Existing DestinationS3Glue Resource

Supporting Types

DestinationS3GlueConfiguration, DestinationS3GlueConfigurationArgs

DestinationS3GlueConfigurationFormat, DestinationS3GlueConfigurationFormatArgs

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJson, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonArgs

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompression, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionArgs

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzip, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzipArgs

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompression, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompressionArgs

DestinationS3GlueResourceAllocation, DestinationS3GlueResourceAllocationArgs

DestinationS3GlueResourceAllocationDefault, DestinationS3GlueResourceAllocationDefaultArgs

DestinationS3GlueResourceAllocationJobSpecific, DestinationS3GlueResourceAllocationJobSpecificArgs

DestinationS3GlueResourceAllocationJobSpecificResourceRequirements, DestinationS3GlueResourceAllocationJobSpecificResourceRequirementsArgs

Import

Package Details

On this page

On this page