Databricks Authentication Error with Terraform provider upgrade (azurerm > v2.8.1)

-
Problem

While trying to upgrade terraform provider azurerm from v2.6.4 to v3.10.0, I faced the following error when I ran terraform plan. 
cannot configure azure-client-secret auth: cannot get workspace: please set `azure_workspace_resource_id` provider argument. Attributes used: auth_type. Environment variables used: ARM_CLIENT_SECRET, ARM_CLIENT_ID, ARM_TENANT_ID. Please check https://registry.terraform.io/providers/databrickslabs/databricks/latest/docs#authentication for details


Databricks authentication, which used to work fine, suddenly stopped working with the version change, saying that azure_workspace_resource_id is not set while it is there.

provider "databricks" {
    azure_workspace_resource_id = <databricks_workspace_id>
}


The terraform version I used was v1.2.3, and the databricks provider was v1.0.0.
The authentication failed not only for azure-client-secret but also with azure-cli.


Try 1 (x)
provider "databricks" {
    azure_workspace_resource_id = <databricks_workspace_id>
    host = <databricks_workspace_url>
}

I tried to follow every possible advice from the reported issues or the latest documentation. 

One of those was to add host in the provider, however, the error was still there. Other tries include upgrading azure-cli, upgrading terraform version to the latest one and so on, but the issue was still the same.
Considering that it works with certain version of azurerm (>= 2.73.0 and <=2.81.0), I concluded that it should be a problem of azurerm.


Fix

After deploying with azurerm v2.81.0 first and then retry with v3.10.0, the issue suddenly disappeared.
My assumption is that v2.81.0 is updating the resource (e.g. databricks_workspace) with the new schema and is backward compatible, while v3.10.0 is retrieving only with the new schema.

Comments

Post a Comment