Databricks table ACLs let data engineers programmatically grant and revoke access to tables. With table ACLs enabled, users have no access to data stored in the cluster’s managed tables by default until granted access, thereby providing improved security for sensitive data. With Databricks runtime 3.5 or above, table ACLs allow authorized users to run SQL and Python commands against tables to which they have been granted access. But due to security vulnerabilities, table ACLs do not support R or Scala commands so you may see messages like this returned in your notebook:
Your administrator has only allowed python and sql commands on this cluster. This execution contained at least one disallowed language.
In partnership with Databricks, Immuta has engineered a data governance solution for fine-grained access (table-, row-, column- and cell-level) controls and advanced privacy techniques that work in tandem with table ACLs for Python and SQL today. The access and privacy controls are enforced natively in the Databricks platform so users do not have to change their workflows.
Many of our customers are now asking for R and Scala support for fine-grained access controls, so we’re excited to announce an early access program through which you can test the solution and provide input as soon as it’s available.
Immuta for Databricks Architecture
The big data era required separation of compute/storage to scale, which brought the rise of Apache Spark and Databricks. Now, the “personal data” era – where organizations analyze large volumes of sensitive data for analytics and data science – requires separation of platform/policy to scale. Organizations using Databricks and Immuta are adopting this architectural best practice, as it enables scaling access and privacy controls when working with personal or other sensitive data. Today, SQL and Python are supported with table ACLs; the same native architecture will extend to R and Scala while completely removing the need for table ACLs and high concurrency clusters.
Get Started Now
This article was originally written to announce early access for this capability, but is now generally available in the latest release of Immuta to support fine-grained access control from Python, SQL, R and Scala notebooks in Databricks.