Background

OpenSearch users have adopted the security plugin to manage the data permissions of users across various teams within their organization. In addition to providing fine grained access control for actions on the data and the cluster, various authentication mechanisms including basic and SAML, the security plugin also provides the ability to isolate OpenSearch Dashboards objects using a concept of tenants. An OpenSearch cluster is often shared across different teams or customers using services of a common vendor. In such scenarios we need to provide admins with the necessary tools to easily isolate data sets as well as any derived Dashboards objects including visualizations, dashboards, reports etc.

Problem statement

OpenSearch users want to share and collaborate on Dashboard objects with one or more users/teams in a simple and easy workflow, while having the ability to see and filter through private or shared objects in the same view. In addition, users want a simple way to isolate the data and Dashboard objects in a scoped view so only users of that role can access the data and associated Dashboards objects (eg: visualizations, saved queries, reports, dashboards etc) to generate valuable insights and support in making business decisions.

Existing solution for sharing Dashboards objects

Tenants in Dashboards is a popular feature that provides the ability to end users to create shared collaboration spaces for saving Dashboards objects such as visualizations, dashboards, reports, saved searches, default index patterns. This is the only solution that helps admins to configure their clusters to provide a clear and isolated view to multiple teams in their organization or to multiple clients that may be hosted on the same cluster. This customized view helps make business decisions faster by providing relevant insights to specific sets of users.

Tenants are configured by the admin who is responsible for all the security configuration and permissions mapping for users in the organization. Tenants are most commonly used in the following ways

1. to share or collaborate with users or similar role/team/project
2. to create a clear data separation of indices and visuals on the data in such indices. Roles and tenants have a many to many relationship, making it much harder to manage these mappings.

Shortcomings of the existing solution

While tenants help in separating Dashboards objects, they are often misunderstood to also provide a traditional multi-tenant architecture that provides data separation and/or infrastructure isolation (hardware, power etc). This is simply untrue as tenancy is designed as a Dashboards concept and should not be mixed with the separate role to permissions mapping needed for the data indices stored in OpenSearch.

Following is a comprehensive list of shortcomings of tenants as a mechanism to share and collaborate

• Tenants are misinterpreted to provide data separation, but they are really only designed for Dashboards objects separation
• Users cannot share objects with other users who are not part of the same tenant (Eg: index patterns, visualizations etc)
• Users cannot change the access scope of an object (Eg: moving index patterns, visualizations etc from one tenant to another)
• Users don’t have a simple way to list all objects that they have the permissions to. They are required to switch between tenants to see a list of objects belonging to that tenant (which takes several seconds as Dashboards is reloaded every-time there is a change in tenant). Also users need to remember the tenant where they created each object
• Complete Dashboards experience is anchored around the current tenant that the user has selected instead of being anchored around the user and their permissions
• Currently on first session login, Dashboards defaults to “private” tenant. This is confusing for customers as those who are just assigned one tenant (eg: “global") and dont need to know the concept of tenants, are now required to understand the concept of tenants and make a selection at login
• Roles need to be carefully mapped to individual tenants with the right level of permissions (read or read and write). This is a many to many relationship
• Many roles can be assigned to the same tenant. One role can have access to many tenants. Different users logging in with different data permissions may need different default index patterns, which is not possible with current implementation, leading to a degraded experience where users may see missing or empty objects (eg: visualizations)
• Users with inadequate permissions to certain objects (eg: visualizations) do not know what permissions they are missing and there is a significant back and forth that needs to happen with the admin, who needs to re-verify their data permissions
• In order to share objects, users need a way to define the scope of the object and permissions for other users or roles
• There are multiple different workflows for sharing objects using tenants, plugin permissions, cluster and data permissions resulting in different experiences and adding confusion
• Number of private tenant indices can become a performance bottleneck for organizations with hundreds or thousands of users

This has resulted in a sub-par experience for OpenSearch users who often ask to disable tenancy entirely.

Proposed solution

OpenSearch users have made us aware of many of the shortcomings of the current solution that is difficult to use, while lacking some critical features. This is an opportunity to rethink the approach to enabling sharing with OpenSearch Dashboards.

The shortcomings highlighted in the above section have led us to rethinking the approach to sharing Dashboards objects.

As we have seen above, the current mechanism doesnt meet users requirements of sharing and collaborating on Dashboards objects in a simple and easy way. This is an opportunity to redesign and simplify the core workflows and functionality that is currently partially fulfilled by tenancy, to create a simpler user sharing experience. Following is a list of recommended features

• Users can share individual Dashboards objects with other users, roles or external identities (also knows as backend roles), given they have the permissions to shareRole mapping workflow should allow turning on default sharing with users of the same role
• Users can see a list of all Dashboards objects in one place. Eg: If the user is viewing a list of visualizations, they must be able to view all the visualizations that were created by them or shared with them, in one place. Similarly for other objects including dashboards, index patterns etc
• By default, new user created objects are shared with the users of the same role. They can be easily marked as private or shared with additional users, before saving the object
• New user created objects are shared with the role by default and can be easily changed to be shared with one or more users or made private
• Index patterns should be tied to a user. Index patterns are the primary anchor for all searches including discover, visualizations, dashboards etc. This configuration should be specific to a user as every user may have a unique combination of roles and data permissions. Every user may want to search on a different index pattern, that should be remembered across logins
• Advanced settings (such as settings for search, visualization, timeline, notifications, allowing leading wildcards, dark mode etc) that are currently in scope of a tenant, will instead be in scope of a user so they can customize their views and preferences
• The security sub-section in Dashboards will now also include a pending/approved permission requests tab. This will show user requests with permissions to specific data sets. Admins will have the option to approve or deny such requests
• Users should be able to “Request access” with one click on any data being visualized across Dashboards . This request automatically shows up in the admin or security manager dashboard, for approval or rejection
• Simplify the role mapping workflows to unify data, plugin and sharing permissions in a unified workflow that is anchored around a user or role

New concepts

As we work on rethinking how we enable OpenSearch users to share and collaborate, we can preserve some core concepts, while introducing new ones. Following are concepts that need to be introduced for a seamless sharing experience.

User profile

Currently we support very limited user specific settings that are remembered on login, but only from that specific browser (eg: last used tenant). We don't have any mechanisms in place that remember user preferences across logins. Also some experiences need to evolve from being tied to tenants, instead to be part of a user profile. For eg: index pattern is currently tied to a tenant or advanced settings (such as settings for search, visualization, timeline, notifications, allowing leading wildcards, dark mode etc) are per tenant. Every user can have a unique data permissions, preferences and should be able to select an index pattern or Dashboards settings that are relevant to the queries they run. This will also eliminate the issue of users with different roles logging into the same tenant and at least one of them not being able to see any data visualized due to the default index pattern not matching their data permissions or being forced to use the advanced settings of the specific tenant. User profile will also be helpful if we decouple Dashboards from OpenSearch, as it will help narrow down the specific query in the context of that user.

Sharing and permissions

Sharing will replace the core functionality that tenancy provides today, by allowing a mechanism to collaborate with other users and roles. Each Dashboard object can be individually shared with different permissions (read, read/write) with other users or roles. Each user will have the ability to share the Dashboard object that they create, but admin can limit users from sharing altogether (eg: for read only roles).

Dashboards objects ownership

Currently Dashboards objects are owned by the tenant that creates them. One or more users may be assigned to that role and direct ownership of the object isn't saved. As we introduce the concept of sharing, it is critical to have ownership of individual Dashboard objects. By default any newly created object will be owned by the user creating it.

Aggregate object views

Currently if a user has access to multiple tenants, they need to switch between tenants, in order to see the relevant Dashboard objects of that tenant. We will introduce a new concept of aggregate views wherein the user can see the full list of all Dashboard object categories including objects that they created and objects that are shared with them. We will provide powerful filtering mechanisms that would allow users to easily find the objects they are looking for. Eg: On the dashboards page, users can see a list of all the dashboards that they created as well as dashboards that were shared with them by other users. Similarly for different categories of Dashboard objects including visualizations, saved searches etc, each of these objects will provide a simplified listing of all objects of that type. The user will not have to leave the page and jump between tenants to find the objects they are looking for. Aggregate object views will provide a single pane of glass for Dashboards objects.

As a result of the sharing mechanism, we will also do away with the inability of moving dashboard objects from one tenant to other. Now there is no need to do that as the user can see all the objects in a single unified pane.

Administrator or security manager dashboard

One of the other problematic areas that goes hand in hand with tenancy is data isolation. While tenancy wasn't designed for data permissions, we are proposing a simpler design that would allow to make this experience easier. When a user views a shared dashboard object that they don't have data permissions to visualize, there would be a mechanism for the user to “Request permission”. Such requests will show up in a admin/security manager panel where they can be approved or denied with the click of a button. We will also provide mechanisms to group actions for easier management. Once approved, the requesting user can see the data that they need to see. In the background, the system will add the data permissions (with explicit approval from the admin) needed by the user to view the requested data.

Tagging

As clusters scale, there will be several users and continuous stream of Dashboards objects creation activity. In order to simplify viewing, filtering, sharing and grouping, we will introduce the mechanism of tagging objects. This will allow users to share various categories of Dashboards objects in one flow. Tagging is also useful to take bulk actions such as deleting objects, group tagging, sharing etc. Eg: If a user wants to share all relevant objects related to “incident-april-5-2022”, they can do so by simply filtering for objects with a specific tag and share them in one workflow. Note that since tags are relevant to the specific user, when objects are shared, the associated tags are not. Every user can tag and group objects how they see fit for their usecase without being overwhelmed with tags associated by object creators.

Simplifying role permissions

Currently roles are assigned access to tenants with either a read or read/write permissions. This creates confusion between boundaries of data permissions and dashboard object permissions, leaving admins often struggling to find the perfect combination of data, plugin, cluster and tenant permissions. In addition some roles are exclusively defined for plugins making it time consuming and difficult to arrive at the perfect combination of permissions for a given user or role. We will aggregate all permissions within one workflow so admins can decide the right cluster, index, plugin and sharing permissions in the same role mapping workflow. Admins can turn on “Sharing by default” in any role, so any user who has assumed that role, can automatically share their newly created Dashboard objects with other users of the same role meeting the needs of users who want data isolation but also share Dashboard objects within the same role. This will give admins a comprehensive view of the role permissions across data, cluster, plugins and sharing.

Migration of existing tenants and dependent plugins

While we introduce new, powerful and simplified concepts, we need to find a proper migration path for clusters currently enabled with tenancy. We need to do this without changing the scope of the permissions and without requiring significant overhead on the administrator for any reconfiguration. There are some plugins that also use the concept of tenancy (eg: Reporting). We need to provide a clean migration path for such plugins.

Index patterns are being migrated from being associated with a role to being associated with a user. This is a significant shift in usage behavior and we need to ensure that post migration the scope of permissions stays the same while each user also has access to index patterns previously associated with the tenants that their roles had access to.

Requirements

Attached requirements highlight how we wish to integrate the core concepts of sharing into existing workflows while carefully introducing new concepts to meet the users requirements (See attached spreadsheet)

Sharing with Dashboards GitHub v1.xlsx

Ask

We are requesting the community to evaluate this RFC, share their feedback, comments and proposals.