Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Workspace][Feature] Add ACL related functions #5084

Merged
merged 38 commits into from
Mar 5, 2024
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
46323e7
[Workspace] Add ACL related functions for workspace (#146)
gaobinlong Sep 14, 2023
20d12cf
Modify changelog
gaobinlong Sep 21, 2023
b025788
Add more unit test cases
gaobinlong Sep 21, 2023
ab08ab9
Modify test case
gaobinlong Sep 21, 2023
c180e87
Some minor change
gaobinlong Sep 22, 2023
cb0e6b0
Add more test cases
gaobinlong Sep 22, 2023
9a0586d
Merge branch 'main' into acl
joshuarrrr Sep 23, 2023
369e7ec
Optimize some code and the comments of the functions
gaobinlong Oct 2, 2023
3022b5d
Merge remote-tracking branch 'upstream/main' into acl
gaobinlong Oct 2, 2023
1f7ce1a
Add more comments for some basic functions
gaobinlong Oct 2, 2023
c91c061
Merge remote-tracking branch 'upstream/main' into acl
gaobinlong Oct 3, 2023
cad9cdf
Export more interfaces
gaobinlong Oct 3, 2023
c866b80
Merge branch 'main' into acl
joshuarrrr Oct 3, 2023
01da891
merge upstream main
gaobinlong Oct 7, 2023
6b3e3a1
consume permissions in repository
SuZhou-Joe Oct 8, 2023
fb47a13
feat: consume permissions in serializer
SuZhou-Joe Oct 8, 2023
ffedcb4
Add unit tests for consuming permissions in repository
gaobinlong Oct 9, 2023
b9de211
Merge remote-tracking branch 'upstream/main' into acl
gaobinlong Oct 17, 2023
9b31d17
Remove double exclamation
gaobinlong Oct 17, 2023
0347e62
Rename some variables
gaobinlong Oct 17, 2023
3db8adc
merge main
gaobinlong Feb 26, 2024
f042dd1
Merge remote-tracking branch 'upstream/main' into acl
gaobinlong Feb 27, 2024
d52ebaa
Merge branch 'main' into acl
gaobinlong Feb 27, 2024
9787ff1
Remove duplicated semicolon
gaobinlong Feb 29, 2024
db34803
Merge remote-tracking branch 'upstream/main' into acl
gaobinlong Feb 29, 2024
04fcc8f
Add permissions field to the mapping only if the permission control i…
gaobinlong Feb 29, 2024
8b72ba1
Fix test failure
gaobinlong Feb 29, 2024
bc18907
Add feature flag config to the yml file
gaobinlong Mar 1, 2024
a0b062f
Merge remote-tracking branch 'upstream/main' into acl
gaobinlong Mar 1, 2024
9eab96b
Make the comment of feature flag more clear
gaobinlong Mar 1, 2024
281f3a3
Make comment more clear
gaobinlong Mar 1, 2024
5488d12
Remove management permission type
gaobinlong Mar 4, 2024
a29f248
Merge main
gaobinlong Mar 4, 2024
c9056de
Fix test failure
gaobinlong Mar 4, 2024
a951c48
Merge remote-tracking branch 'upstream/main' into acl
gaobinlong Mar 4, 2024
2ed6365
Merge remote-tracking branch 'upstream/main' into acl
gaobinlong Mar 4, 2024
8b014ff
Merge remote-tracking branch 'upstream/main' into acl
gaobinlong Mar 5, 2024
b80101a
Merge remote-tracking branch 'upstream/main' into acl
gaobinlong Mar 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- Replace OuiSelect component with OuiSuperSelect in data-source plugin ([#5315](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/5315))
- [Workspace] Add core workspace service module to enable the implementation of workspace features within OSD plugins ([#5092](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/5092))
- [Workspace] Setup workspace skeleton and implement basic CRUD API ([#5075](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/5075))
- [Workspace] Add ACL related functions ([#5084](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/5084/))
gaobinlong marked this conversation as resolved.
Show resolved Hide resolved
- [Decouple] Add new cross compatibility check core service which export functionality for plugins to verify if their OpenSearch plugin counterpart is installed on the cluster or has incompatible version to configure the plugin behavior([#4710](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/4710))
- [Discover] Add long numerals support [#5592](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/5592)
- [Discover] Display inner properties in the left navigation bar [#5429](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/5429)
Expand Down
6 changes: 5 additions & 1 deletion config/opensearch_dashboards.yml
Original file line number Diff line number Diff line change
Expand Up @@ -285,4 +285,8 @@
# opensearchDashboards.survey.url: "https://survey.opensearch.org"

# Set the value of this setting to true to enable plugin augmentation on Dashboard
# vis_augmenter.pluginAugmentationEnabled: true
# vis_augmenter.pluginAugmentationEnabled: true

# Set the value to true to enable permission control for saved objects
gaobinlong marked this conversation as resolved.
Show resolved Hide resolved
# Permission control depends on OpenSearch Dashboards has authentication enabled, set it to false if no authentication is configured
# savedObjects.permission.enabled: true
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@

import { IndexMapping, SavedObjectsTypeMappingDefinitions } from './../../mappings';
import { buildActiveMappings, diffMappings } from './build_active_mappings';
import { configMock } from '../../../config/mocks';

describe('buildActiveMappings', () => {
test('creates a strict mapping', () => {
Expand Down Expand Up @@ -91,6 +92,12 @@ describe('buildActiveMappings', () => {
expect(hashes.aaa).toEqual(hashes.bbb);
expect(hashes.aaa).not.toEqual(hashes.ccc);
});

test('permissions field is added when permission control flag is enabled', () => {
const rawConfig = configMock.create();
rawConfig.get.mockReturnValue(true);
expect(buildActiveMappings({}, rawConfig)).toHaveProperty('properties.permissions');
});
});

describe('diffMappings', () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,10 @@

import crypto from 'crypto';
import { cloneDeep, mapValues } from 'lodash';
import { Config } from 'packages/osd-config/target';
import {
IndexMapping,
SavedObjectsFieldMapping,
SavedObjectsMappingProperties,
SavedObjectsTypeMappingDefinitions,
} from './../../mappings';
Expand All @@ -47,11 +49,36 @@ import {
* @param typeDefinitions - the type definitions to build mapping from.
*/
export function buildActiveMappings(
typeDefinitions: SavedObjectsTypeMappingDefinitions | SavedObjectsMappingProperties
typeDefinitions: SavedObjectsTypeMappingDefinitions | SavedObjectsMappingProperties,
opensearchDashboardsRawConfig?: Config
): IndexMapping {
const mapping = defaultMapping();

const mergedProperties = validateAndMerge(mapping.properties, typeDefinitions);
let mergedProperties = validateAndMerge(mapping.properties, typeDefinitions);
// if permission control for saved objects is enabled, the permissions field should be added to the mapping
if (opensearchDashboardsRawConfig?.get('savedObjects.permission.enabled')) {
const principals: SavedObjectsFieldMapping = {
properties: {
users: {
type: 'keyword',
},
groups: {
type: 'keyword',
},
},
};
mergedProperties = validateAndMerge(mapping.properties, {
permissions: {
properties: {
read: principals,
write: principals,
management: principals,
gaobinlong marked this conversation as resolved.
Show resolved Hide resolved
library_read: principals,
library_write: principals,
},
},
});
}

return cloneDeep({
...mapping,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ import { SavedObjectTypeRegistry } from '../../saved_objects_type_registry';
import { IndexMigrator } from './index_migrator';
import { MigrationOpts } from './migration_context';
import { loggingSystemMock } from '../../../logging/logging_system.mock';
import { configMock } from '../../../config/mocks';

describe('IndexMigrator', () => {
let testOpts: jest.Mocked<MigrationOpts> & {
Expand All @@ -59,6 +60,93 @@ describe('IndexMigrator', () => {
};
});

test('creates the index when permission control for saved objects is enabled', async () => {
const { client } = testOpts;

testOpts.mappingProperties = { foo: { type: 'long' } as any };
const rawConfig = configMock.create();
rawConfig.get.mockReturnValue(true);
testOpts.opensearchDashboardsRawConfig = rawConfig;

withIndex(client, { index: { statusCode: 404 }, alias: { statusCode: 404 } });

await new IndexMigrator(testOpts).migrate();

expect(client.indices.create).toHaveBeenCalledWith({
body: {
mappings: {
dynamic: 'strict',
_meta: {
migrationMappingPropertyHashes: {
foo: '18c78c995965207ed3f6e7fc5c6e55fe',
migrationVersion: '4a1746014a75ade3a714e1db5763276f',
namespace: '2f4316de49999235636386fe51dc06c1',
namespaces: '2f4316de49999235636386fe51dc06c1',
originId: '2f4316de49999235636386fe51dc06c1',
permissions: '07c04cdd060494956fdddaa7ef86e8ac',
references: '7997cf5a56cc02bdc9c93361bde732b0',
type: '2f4316de49999235636386fe51dc06c1',
updated_at: '00da57df13e94e9d98437d13ace4bfe0',
},
},
properties: {
foo: { type: 'long' },
migrationVersion: { dynamic: 'true', type: 'object' },
namespace: { type: 'keyword' },
namespaces: { type: 'keyword' },
originId: { type: 'keyword' },
type: { type: 'keyword' },
updated_at: { type: 'date' },
permissions: {
properties: {
library_read: {
properties: {
users: { type: 'keyword' },
groups: { type: 'keyword' },
},
},
library_write: {
properties: {
users: { type: 'keyword' },
groups: { type: 'keyword' },
},
},
management: {
properties: {
users: { type: 'keyword' },
groups: { type: 'keyword' },
},
},
read: {
properties: {
users: { type: 'keyword' },
groups: { type: 'keyword' },
},
},
write: {
properties: {
users: { type: 'keyword' },
groups: { type: 'keyword' },
},
},
},
},
references: {
type: 'nested',
properties: {
name: { type: 'keyword' },
type: { type: 'keyword' },
id: { type: 'keyword' },
},
},
},
},
settings: { number_of_shards: 1, auto_expand_replicas: '0-1' },
},
index: '.kibana_1',
});
});

test('creates the index if it does not exist', async () => {
const { client } = testOpts;

Expand Down
16 changes: 12 additions & 4 deletions src/core/server/saved_objects/migrations/core/migration_context.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
*/

import { Logger } from 'src/core/server/logging';
import { Config } from 'packages/osd-config/target';
import { MigrationOpenSearchClient } from './migration_opensearch_client';
import { SavedObjectsSerializer } from '../../serialization';
import {
Expand Down Expand Up @@ -65,6 +66,7 @@ export interface MigrationOpts {
* prior to running migrations. For example: 'opensearch_dashboards_index_template*'
*/
obsoleteIndexTemplatePattern?: string;
opensearchDashboardsRawConfig?: Config;
}

/**
Expand All @@ -90,10 +92,15 @@ export interface Context {
* and various info needed to migrate the source index.
*/
export async function migrationContext(opts: MigrationOpts): Promise<Context> {
const { log, client } = opts;
const { log, client, opensearchDashboardsRawConfig } = opts;
const alias = opts.index;
const source = createSourceContext(await Index.fetchInfo(client, alias), alias);
const dest = createDestContext(source, alias, opts.mappingProperties);
const dest = createDestContext(
source,
alias,
opts.mappingProperties,
opensearchDashboardsRawConfig
);

return {
client,
Expand Down Expand Up @@ -125,10 +132,11 @@ function createSourceContext(source: Index.FullIndexInfo, alias: string) {
function createDestContext(
source: Index.FullIndexInfo,
alias: string,
typeMappingDefinitions: SavedObjectsTypeMappingDefinitions
typeMappingDefinitions: SavedObjectsTypeMappingDefinitions,
opensearchDashboardsRawConfig?: Config
): Index.FullIndexInfo {
const targetMappings = disableUnknownTypeMappingFields(
buildActiveMappings(typeMappingDefinitions),
buildActiveMappings(typeMappingDefinitions, opensearchDashboardsRawConfig),
source.mappings
);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ import {
import { loggingSystemMock } from '../../../logging/logging_system.mock';
import { SavedObjectTypeRegistry } from '../../saved_objects_type_registry';
import { SavedObjectsType } from '../../types';
import { configMock } from '../../../config/mocks';

const createRegistry = (types: Array<Partial<SavedObjectsType>>) => {
const registry = new SavedObjectTypeRegistry();
Expand Down Expand Up @@ -76,6 +77,12 @@ describe('OpenSearchDashboardsMigrator', () => {
const mappings = new OpenSearchDashboardsMigrator(options).getActiveMappings();
expect(mappings).toMatchSnapshot();
});

it('permissions field exists in the mappings when the feature is enabled', () => {
gaobinlong marked this conversation as resolved.
Show resolved Hide resolved
const options = mockOptions(true);
const mappings = new OpenSearchDashboardsMigrator(options).getActiveMappings();
expect(mappings).toHaveProperty('properties.permissions');
});
});

describe('runMigrations', () => {
Expand Down Expand Up @@ -146,7 +153,12 @@ type MockedOptions = OpenSearchDashboardsMigratorOptions & {
client: ReturnType<typeof opensearchClientMock.createOpenSearchClient>;
};

const mockOptions = () => {
const mockOptions = (isPermissionControlEnabled?: boolean) => {
const rawConfig = configMock.create();
rawConfig.get.mockReturnValue(false);
if (isPermissionControlEnabled) {
rawConfig.get.mockReturnValue(true);
}
const options: MockedOptions = {
logger: loggingSystemMock.create().get(),
opensearchDashboardsVersion: '8.2.3',
Expand Down Expand Up @@ -186,6 +198,7 @@ const mockOptions = () => {
skip: false,
},
client: opensearchClientMock.createOpenSearchClient(),
opensearchDashboardsRawConfig: rawConfig,
};
return options;
};
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
import { OpenSearchDashboardsConfigType } from 'src/core/server/opensearch_dashboards_config';
import { BehaviorSubject } from 'rxjs';

import { Config } from 'packages/osd-config/target';
import { Logger } from '../../../logging';
import { IndexMapping, SavedObjectsTypeMappingDefinitions } from '../../mappings';
import { SavedObjectUnsanitizedDoc, SavedObjectsSerializer } from '../../serialization';
Expand All @@ -54,6 +55,7 @@ export interface OpenSearchDashboardsMigratorOptions {
opensearchDashboardsConfig: OpenSearchDashboardsConfigType;
opensearchDashboardsVersion: string;
logger: Logger;
opensearchDashboardsRawConfig: Config;
}

export type IOpenSearchDashboardsMigrator = Pick<
Expand Down Expand Up @@ -83,6 +85,7 @@ export class OpenSearchDashboardsMigrator {
status: 'waiting',
});
private readonly activeMappings: IndexMapping;
private readonly opensearchDashboardsRawConfig: Config;

/**
* Creates an instance of OpenSearchDashboardsMigrator.
Expand All @@ -94,6 +97,7 @@ export class OpenSearchDashboardsMigrator {
savedObjectsConfig,
opensearchDashboardsVersion,
logger,
opensearchDashboardsRawConfig,
}: OpenSearchDashboardsMigratorOptions) {
this.client = client;
this.opensearchDashboardsConfig = opensearchDashboardsConfig;
Expand All @@ -107,9 +111,13 @@ export class OpenSearchDashboardsMigrator {
typeRegistry,
log: this.log,
});
this.opensearchDashboardsRawConfig = opensearchDashboardsRawConfig;
// Building the active mappings (and associated md5sums) is an expensive
// operation so we cache the result
this.activeMappings = buildActiveMappings(this.mappingProperties);
this.activeMappings = buildActiveMappings(
this.mappingProperties,
this.opensearchDashboardsRawConfig
);
}

/**
Expand Down Expand Up @@ -181,6 +189,7 @@ export class OpenSearchDashboardsMigrator {
? 'opensearch_dashboards_index_template*'
: undefined,
convertToAliasScript: indexMap[index].script,
opensearchDashboardsRawConfig: this.opensearchDashboardsRawConfig,
});
});

Expand Down
Loading
Loading