feat(glue-table): support storage descriptor, partition keys and table parameters

[posquit0]

Description

Complete the glue-table module, which previously only created a bare table (name / description / type). This revives the work from the old glue-table branch (a0ef899) and extends it:

• storage_descriptor support: location, input_format / output_format (with validation), compressed, columns, and ser_de (SerDe name / serialization library / parameters)
• partition_keys support
• parameters support — enables Athena partition projection (projection.*, storage.location.template) and other table properties
• Outputs for all new attributes

Usage

module "table" {
  source = "tedilabs/data/aws//modules/glue-table"

  database = "dns"
  name     = "route53_resolver_query_logs"

  location      = "s3://my-log-bucket/AWSLogs/123456789012/vpcdnsquerylogs/"
  input_format  = "org.apache.hadoop.mapred.TextInputFormat"
  output_format = "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"
  compressed    = true
  ser_de = {
    serialization_library = "org.openx.data.jsonserde.JsonSerDe"
  }

  columns = [
    { name = "query_timestamp", type = "string" },
    { name = "query_name", type = "string" },
    # ...
  ]
  partition_keys = [
    { name = "vpc_id", type = "string" },
    { name = "date", type = "string" },
  ]

  parameters = {
    "projection.enabled" = "true"
    # ...
  }
}

Notes

• Remaining unimplemented arguments (partition_index, retention, skewed_info, target_table, etc.) are listed in the comment above the resource.

[gemini-code-assist]

Code Review

This pull request expands the glue-table module to support additional AWS Glue Catalog Table configurations, including columns, partition keys, SerDe parameters, and storage formats. The review feedback focuses on making the Terraform code more idiomatic and robust. Specifically, it is recommended to default optional string variables like input_format and output_format to null instead of empty strings to simplify validation and resource block assignments. Additionally, the ser_de output should be updated to use splat syntax (*) and flatten for consistency and to prevent potential out-of-bounds errors.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

By defaulting input_format and output_format to null instead of "", we can simplify these assignments and avoid ternary checks.

    input_format  = var.input_format
    output_format = var.output_format

[gemini-code-assist]

Using null as the default value for optional string variables is more idiomatic in Terraform than using an empty string "". This also allows us to simplify the validation block by checking if the variable is null first.

  default     = null
  nullable    = true

  validation {
    condition = var.input_format == null ? true : contains([
      "org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat",
      "com.amazon.emr.cloudtrail.CloudTrailInputFormat",
      "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat",
      "org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat",
      "org.apache.hadoop.mapred.TextInputFormat",
    ], var.input_format)
    error_message = "Supported values for `input_format` are `org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat`, `com.amazon.emr.cloudtrail.CloudTrailInputFormat`, `org.apache.hadoop.hive.ql.io.orc.OrcInputFormat`, `org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat`, `org.apache.hadoop.mapred.TextInputFormat`."
  }

[gemini-code-assist]

Using null as the default value for optional string variables is more idiomatic in Terraform than using an empty string "". This also allows us to simplify the validation block by checking if the variable is null first.

  default     = null
  nullable    = true

  validation {
    condition = var.output_format == null ? true : contains([
      "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat",
      "org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat",
      "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat",
      "org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat",
    ], var.output_format)
    error_message = "Supported values for `output_format` are `org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat`, `org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat`, `org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat`, `org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat`."
  }

[gemini-code-assist]

Using storage_descriptor[0] is inconsistent with the other outputs in this file which use storage_descriptor[*]. It is also safer to use storage_descriptor[*] combined with flatten to avoid potential index out of bounds errors.

output "ser_de" {
  description = "The configuration of the SerDe (Serializer/Deserializer) of the table."
  value       = one(flatten(aws_glue_catalog_table.this.storage_descriptor[*].ser_de_info))
}