Is there a way to add column descriptions to a declaration in dataform? - Stack Overflow

I am building a pipeline to combine the order data from a variety of ERPs. Currently the repos are setup as such:

erp1_repo
└── definitions
    ├── sources
    │   ├── order_table_1.sqlx (declaration)
    │   └── customer_table_1.sqlx (declaration)
    └── staging
        └── erp1_order_staging.sqlx (view)

erp2_repo
└── definitions
    ├── sources
    │   ├── order_table_2.sqlx (declaration)
    │   └── customer_table_2.sqlx (declaration)
    └── staging
        └── erp2_order_staging.sqlx (view)

order_repo
└── definitions
    ├── sources
    │   ├── erp1_order_staging.sqlx (declaration)
    │   └── erp2_order_staging.sqlx (declaration)
    └── fact_order.sqlx (incremental table)

I have tried adding the column descriptions to the order_table_1.sqlx declaration as such and executed the upstream erp1_order_staging.sqxl view with all three execution options checked, but the descriptions in order_table_1 did not get updated in BigQuery.

config {
  type: "declaration",
  database: "database_name",
  schema: "schema",
  name: "order_table_1",
  description: "order_table_1 description",
  columns: {
    ORDERNO: "Order Number",
    CUSTNO: "Customer Number",
    AMOUNT: "Order Amount",
    QTYORD: "Order Quantity"
  }
}

I am building a pipeline to combine the order data from a variety of ERPs. Currently the repos are setup as such:

erp1_repo
└── definitions
    ├── sources
    │   ├── order_table_1.sqlx (declaration)
    │   └── customer_table_1.sqlx (declaration)
    └── staging
        └── erp1_order_staging.sqlx (view)

erp2_repo
└── definitions
    ├── sources
    │   ├── order_table_2.sqlx (declaration)
    │   └── customer_table_2.sqlx (declaration)
    └── staging
        └── erp2_order_staging.sqlx (view)

order_repo
└── definitions
    ├── sources
    │   ├── erp1_order_staging.sqlx (declaration)
    │   └── erp2_order_staging.sqlx (declaration)
    └── fact_order.sqlx (incremental table)

I have tried adding the column descriptions to the order_table_1.sqlx declaration as such and executed the upstream erp1_order_staging.sqxl view with all three execution options checked, but the descriptions in order_table_1 did not get updated in BigQuery.

config {
  type: "declaration",
  database: "database_name",
  schema: "schema",
  name: "order_table_1",
  description: "order_table_1 description",
  columns: {
    ORDERNO: "Order Number",
    CUSTNO: "Customer Number",
    AMOUNT: "Order Amount",
    QTYORD: "Order Quantity"
  }
}

• dataform

Share Improve this question Follow edited Jan 31 at 16:27 Samuel Frison asked Jan 31 at 16:24 Samuel FrisonSamuel Frison 111 bronze badge

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

A declaration is just a pointer to a table that exists outside of your dataform code. Adding a "columns" attribute is apparently allowed in declarations, which is odd to me, but I can't imagine it doing anything.

Adding the columns attribute as you've done to one of the .sqlx files that produce a table or view will add the column descriptions in BigQuery as you'd expect.