How to secure and get in compliance with your Elasticsearch data – using the Elastic Stack to meet GDPR requirements?
Data flow diagram:
GDPR Compliance initiative process
- Prepare;
- Protect;
- Privacy processes.
In the preparation stage, an organization decides on what data they might have:
- Data mapping: identifying and documenting all the data flows within the organization that control process personal data.
- Privacy impact statement: you classify the severity of data i.e in case of a data breach or data leak.
- Data Retention planning: determine how long the personal data is stored and how to delete those data?
In the Protection stage: implement an appropriate level of protection/security. Checking the data flow and ensuring the security and distribution of personal data are minimized.
Using the Elastic stack to meet GDPR requirements
In the below diagram, the Elastic logo indicates the GDPR implementation steps which a user can deploy the elastic stack features to handle GDPR.
Mapping data and ingesting it into the Elastic is the crucial step in regards to GDPR and if the organization is unable to identify the relevant data flow then the GDPR initiative may be incomplete/ineffective.
Access Control
To prevent unauthorized access to personal data i.e data stored in Elasticsearch. Elastic’s Xpack security features provide a standalone authentication mechanism that enables quick password protection of a cluster and also Xpack can be integrated with the external authentication mechanism to manage users in an organization, such as LDAP, AD, or PKI. It also includes IP address filtering to whitelist and blacklist specific IP addresses and subnets to control network layer access to the elastic search cluster.
However, Just authenticating the users is not enough for GDPR compliance, we need extra control over what data a user can access and what tasks can they perform when interacting with the data. Xpack security features also enable control to authorize users by assigning them access privileges and roles and then assigning these roles to users. This role-based access control (RBAC) provides the ability to specify which users can perform read or write operations on the Elasticsearch indices or documents or fields.
Logging and Auditing
When Elasticsearch is used as data storage, Elastic Xpack features enable you to maintain an audit trail by auditing security events. The audit logs produced by Elasticsearch when Xpack security is enabled, let you see who is accessing your data and what they are doing, and by analyzing access patterns and things like “Failed attempts to access cluster ” insight can be gained by security teams to understand attempted attacks and data breaches.
When Elasticsearch is not used as primary data storage, it can be used as a centralized logging platform for managing security-related logs from throughout an organization’s infrastructure, it can be any machines, servers, network.
Monitoring and Detection
Xpack security feature helps admins to monitor the infra as well as the health of the Elastic cluster. Here Xpack alerting features enables automated monitoring of log, notifications of failure, or interruptions. Along with Xpack alerting features, Xpack machine learning jobs (anomaly detection), and Kibana dashboards can be used for threat detection platforms. There are a lot of features that comes with Xpack. Check it here.
More information
Xpack security control allows you to implement the access controls as required by GDPR. It also enables end-to-end security either internal or external such as internal communication between nodes and external connection to Elasticsearch via an application. Click on the link for more information about Security minimal setup, configuring Stack security or Elasticsearch Beats and Logstash security. If you want more information about Elastic Stack, click here. If you want more information about X-pack, click here.