Fine-grained Access Control With AWS LakeFormation
Last updated
Was this helpful?
Last updated
Was this helpful?
are complex systems. They not only get data from many source systems but also many systems and users consume data from them. For large-scale implementations it becomes complicated for customers to manage thousands of IAM roles and policy that controls access to their datalake. simplifies datalake access management by providing on the data lake catalog. Please refer to various AWS LakeFormation terminologies .
In this model, a set of initial represented by IAM roles and users known as can grant Lake Formation permissions on data locations and Data Catalog resources to any principal (including self). Once the permissions on various data lake objects are created, users (IAM Users, Roles) can access to the datalake objects through their preferred compute engines like , , that's integrated with . Lake formation authorizes access to datalake resources when the execute workloads on data catalog.
Metadata access control – Permissions on Data Catalog resources (Data Catalog permissions).
These permissions enable principals to create, read, update, and delete metadata databases and tables in the Data Catalog.
Data access permissions enable principals to read and write data to underlying Amazon S3 locations. Data location permissions enable principals to create metadata databases and tables that point to specific Amazon S3 locations.
As of 03/31/2020, AWS Lake formation provides prermissions at various levels.
Catalog level permissions
CREATE_DATABASE
Database level permissions
CREATE
ALTER
DROP
Tables level permissions
ALTER
DROP
SELECT - Applicable to underlying data
DELETE - Applicable to underlying data
INSERT - Applicable to underlying data
Column level permissions
SELECT - Applicable to underlying data
When an user runs a workload on Lakeformation catalog using an integrated compute service, the compute service requests for access to Lakeformation. Based on the access level defined on the catalog objects, AWS Lake formation vends short-term credentials to the compute service. The compute service then uses the temporary credentials to directly access S3 objects and execute the workload. For column level access control, the compute engine filters out attributes that the user/role don't have access to after the objects have been downloaded from S3 as part processing(as of 03/31/2020).
is divided into the following two areas:
Underlying data access control – Permissions on locations in (data access permissions and data location permissions).
For both areas, Lake Formation uses a combination of Lake Formation permissions and permissions. The IAM permissions model consists of IAM policies. The Lake Formation permissions model is implemented as DBMS-style GRANT/REVOKE commands, such as Grant SELECT on tableName to userName.
