Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog

The types of changes are:

Added for new features.
Changed for changes in existing functionality.
Developer Experience for changes in developer workflow or tooling.
Deprecated for soon-to-be removed features.
Docs for documentation only changes.
Removed for now removed features.
Fixed for any bug fixes.
Security in case of vulnerabilities.

Unreleased

Added

Access support for Shippo #2484

2.10.0

Added

Allow users to configure their username and password via the config file #2884
Add authentication to the masking endpoints as well as accompanying scopes #2909
Add an Organization Management page (beta) #2908
Adds assigned systems to user management table #2922
APIs to support Privacy Notice management (create, read, update) #2928

Changed

Improved standard layout for large width screens and polished misc. pages #2869
Changed UI paths in the admin-ui #2869
- /add-systems/new --> /add-systems/manual
- /system --> /systems
Added individual ID routes for systems #2902
Deprecated adding scopes to users directly; you can only add roles. #2848
Changed About Fides page to say "Fides Core Version:" over "Version". #2899
Polish Admin UI header & navigation #2897
Give new users a "viewer" role by default #2900
Tie together save states for user permissions and systems #2913
Removing payment types from Stripe connector params #2915
Viewer role can now access a restricted version of the user management page #2933
Change Privacy Center email placeholder text #2935
Restricted setting Approvers as System Managers #2891
Adds confirmation modal when downgrading user to "approver" role via Admin UI #2924
Changed the toast message for new users to include access control info #2939

Fixed

Restricted Contributors from being able to create Owners #2888
Allow for dynamic aspect ratio for logo on Privacy Center 404 #2895
Allow for dynamic aspect ratio for logo on consent page #2895
Align role dscription drawer of Admin UI with top nav: #2932
Fixed error message when a user is assigned to be an approver without any systems #2953

Developer Experience

Update frontend npm packages (admin-ui, privacy-center, cypress-e2e) #2921

2.9.2

Fixed

Allow multiple data uses as long as their processing activity name is different #2905
use HTML property, not text, when dispatching Mailchimp Transactional emails #2901
Remove policy key from Privacy Center submission modal #2912

2.9.1

Added

Added Attentive erasure email connector #2782

Changed

Removed dataset based email connectors #2782
Changed Auth0's authentication strategy from bearer to oauth2_client_credentials #2820
renamed the privacy declarations field "Privacy declaration name (deprecated)" to "Processing Activity" #711

Fixed

Fixed issue where the scopes list passed into FidesUserPermission could get mutated with the total_scopes call #2883

Removed

removed the privacyDeclarationDeprecatedFields flag #711

2.9.0

Added

The ability to assign users as system managers for a specific system #2714
New endpoints to add and remove users as system managers #2726
Warning about access control migration to the UI #2842
Adds Role Assignment UI #2739
Add an automated migration to give users a viewer role #2821

Changed

Removed "progressive" navigation that would hide Admin UI tabs until Systems / Connections were configured #2762
Added system.privacy_declaration.name to datamap response #2831

Developer Experience

Retired legacy navV2 feature flag #2762
Update Admin UI Layout to fill viewport height #2812

Fixed

Fixed issue where unsaved changes warning would always show up when running fidesplus #2788
Fixed problem in datamap export with datasets that had been updated via SaaS instantiation #2841
Fixed problem in datamap export with inconsistent custom field ordering #2859

2.8.3

Added

Serialise bson.ObjectId types in SAR data packages #2785

Fixed

Fixed issue where more than 1 populated custom fields removed a system from the datamap export #2825

2.8.2

Fixed

Resolved a bug that stopped custom fields populating the visual datamap #2775
Patch appconfig migration to handle existing db record #2780

2.8.1

Fixed

Disabled hiding Admin UI based on user scopes #2771

2.8.0

Added

Add API support for messaging config properties #2551
Access and erasure support for Kustomer #2520
Added the erase_after field on collections to be able to set the order for erasures #2619
Add a toggle to filter the system classification to only return those with classification data #2700
Added backend role-based permissions #2671
Access and erasure for Vend SaaS Connector #1869
Added endpoints for storage and messaging config setup status #2690
Access and erasure for Jira SaaS Connector #1871
Access and erasure support for Delighted #2244
Improve "Upload a new dataset YAML" #1531
Input validation and sanitization for Privacy Request fields #2655
Access and erasure support for Yotpo #2708
Custom Field Library Tab #527
Allow SendGrid template usage #2728
Added ConnectorRunner to simplify SaaS connector testing #1795
Adds support for Mailchimp Transactional as a messaging config #2742

Changed

Admin UI
- Add flow for selecting system types when manually creating a system #2530
- Updated forms for privacy declarations #2648
- Delete flow for privacy declarations #2664
- Add framework to have UI elements respect the user's scopes #2682
- "Manual Webhook" has been renamed to "Manual Process". #2717
Convert all config values to Pydantic Field objects #2613
Add warning to 'fides deploy' when installed outside of a virtual environment #2641
Redesigned the default/init config file to be auto-documented. Also updates the fides init logic and analytics consent logic #2694
Change how config creation/import is handled across the application #2622
Update the CLI aesthetics & docstrings #2703
Updates Roles->Scopes Mapping #2744
Return user scopes as an enum, as well as total scopes #2741
Update MessagingServiceType enum to be lowercased throughout #2746

Developer Experience

Set the security environment of the fides dev setup to prod instead of dev #2588
Removed unexpected default Redis password #2666
Privacy Center
- Typechecking and validation of the config.json will be checked for backwards-compatibility. #2661
Combined conftest.py files #2669

Fixed

Fix support for "redis.user" setting when authenticating to the Redis cache #2666
Fix error with the classify dataset feature flag not writing the dataset to the server #2675
Allow string dates to stay strings in cache decoding #2695
Admin UI
- Remove Identifiability (Data Qualifier) from taxonomy editor 2684
FE: Custom field selections binding issue on Taxonomy tabs #2659
Fix Privacy Request Status when submitting a consent request when identity verification is required #2736

2.7.1

Fix error with the classify dataset feature flag not writing the dataset to the server #2675

2.7.0

Fides API
- Access and erasure support for Braintree #2223
- Added route to send a test message #2585
- Add default storage configuration functionality and associated APIs #2438
Admin UI
- Custom Metadata #2536
  - Create Custom Lists
  - Create Custom Field Definition
  - Create custom fields from a the taxonomy editor
  - Provide a custom field value in a resource
  - Bulk edit custom field values #2612
  - Custom metadata UI Polish #2624
Privacy Center
- The consent config default value can depend on whether Global Privacy Control is enabled. #2341
- When GPC is enabled, the UI indicates which data uses are opted out by default. #2596
- inspectForBrowserIdentities now also looks for ljt_readerID. #2543

Added

Added new Wunderkind Consent Saas Connector #2600
Added new Sovrn Email Consent Connector #2543
Log Fides version at startup #2566

Changed

Update Admin UI to show all action types (access, erasure, consent, update) #2523
Removes legacy verify_oauth_client function #2527
Updated the UI for adding systems to a new design #2490
Minor logging improvements #2566
Various form components now take a stacked or inline variant #2542
UX fixes for user management #2537
Updating Firebase Auth connector to mask the user with a delete instead of an update #2602

Fixed

Fixed bug where refreshing a page in the UI would result in a 404 #2502
Usernames are case insensitive now and prevent all duplicates #2487
- This PR contains a migration that deletes duplicate users and keeps the oldest original account.
Update Logos for shipped connectors #2464
Search field on privacy request page isn't working #2270

Developer Experience

Added new Cypress E2E smoke tests #2241
New command nox -s e2e_test which will spin up the test environment and run true E2E Cypress tests against it #2417
Cypress E2E tests now run in CI and are reported to Cypress Cloud #2417
Change from randomint to uuid in mongodb tests to reduce flakiness. #2591

Removed

Remove feature flagged config wizard stepper from Admin UI #2553

2.6.6

Changed

Improve Readability for Custom Masking Override Exceptions #2593

2.6.5

Added

Added config properties to override database Engine parameters #2511
Increased default pool_size and max_overflow to 50 #2560

2.6.4

Fixed

Fixed bug for SMS completion notification not being sent #2526
Fixed bug where refreshing a page in the UI would result in a 404 #2502

2.6.3

Fixed

Handle case where legacy dataset has meta: null #2524

2.6.2

Fixed

Issue addressing missing field in dataset migration #2510

2.6.1

Fixed

Fix errors when privacy requests execute concurrently without workers #2489
Enable saas request overrides to run in worker runtime #2489

2.6.0

Added

Added the env option to the security configuration options to allow for users to completely secure the API endpoints #2267
Unified Fides Resources
- Added a dataset dropdown selector when configuring a connector to link an existing dataset to the connector configuration. #2162
- Added new datasetconfig.ctl_dataset_id field to unify fides dataset resources #2046
Add new connection config routes that couple them with systems #2249
Add new select/deselect all permissions buttons #2437
Endpoints to allow a user with the user:password-reset scope to reset users' passwords. In addition, users no longer require a scope to edit their own passwords. #2373
New form to reset a user's password without knowing an old password #2390
Approve & deny buttons on the "Request details" page. #2473
Consent Propagation
- Add the ability to execute Consent Requests via the Privacy Request Execution layer #2125
- Add a Mailchimp Transactional Consent Connector #2194
- Allow defining a list of opt-in and/or opt-out requests in consent connectors #2315
- Add a Google Analytics Consent Connector for GA4 properties #2302
- Pass the GA Cookie from the Privacy Center #2337
- Rename "user_id" to more specific "ga_client_id" #2356
- Patch Google Analytics Consent Connector to delete by client_id #2355
- Add a "skip_param_values option" to optionally skip when we are missing param values in the body #2384
- Adds a new Universal Analytics Connector that works with the UA Tracking Id
Adds intake and storage of Global Privacy Control Signal props for Consent #2599

Changed

Unified Fides Resources
- Removed several fidesops schemas for DSR's in favor of updated Fideslang schemas #2009
- Removed DatasetConfig.dataset field #2096
- Updated UI dataset config routes to use new unified routes #2113
- Validate request body on crud endpoints on upsert. Validate dataset data categories before save. #2134
- Updated test env setup and quickstart to use new endpoints #2225
Consent Propagation
- Privacy Center consent options can now be marked as executable in order to propagate consent requests #2193
- Add support for passing browser identities to consent request patches #2304
Update fideslang to 1.3.3 #2343
Display the request type instead of the policy name on the request table #2382
Make denial reasons required #2400
Display the policy key on the request details page #2395
Updated CSV export #2452
Privacy Request approval now uses a modal #2443

Developer Experience

nox -s test_env has been replaced with nox -s "fides_env(dev)"
New command nox -s "fides_env(test)" creates a complete test environment with seed data (similar to fides_env(dev)) but with the production fides image so the built UI can be accessed at localhost:8080 #2399
Change from code climate to codecov for coverage reporting #2402

Fixed

Home screen header scaling and responsiveness issues #2200
Privacy Center identity inputs validate even when they are optional. #2308
The PII toggle defaults to false and PII will be hidden on page load #2388
Fixed a CI bug caused by git security upgrades #2441
Privacy Center
- Identity inputs validate even when they are optional. #2308
- Submit buttons show loading state and disable while submitting. #2401
- Phone inputs no longer request country SVGs from external domain. #2378
- Input validation errors no longer change the height of modals. #2379
Patch masking strategies to better handle null and non-string inputs #2307
Renamed prod pushes tag to be latest for privacy center and sample app #2401
Update firebase connector to better handle non-existent users #2439

2.5.1

Developer Experience

Allow db resets only if config.dev_mode is True #2321

Fixed

Added a feature flag for the recent dataset classification UX changes #2335

Security

Add a check to the catchall path to prevent returning paths outside of the UI directory #2330

Developer Experience

Reduce size of local Docker images by fixing .dockerignore patterns #2360

2.5.0

Docs

Update the docs landing page and remove redundant docs #2184

Added

Added the user command group to the CLI. #2153
Added Code Climate test coverage uploads. #2198
Added the connection key to the execution log #2100
Added endpoints to retrieve DSR Rules and Rule Targets #2116
Added Fides version number to account dropdown in the UI #2140
Add link to Classify Systems page in nav side bar #2128
Dataset classification UI now polls for results #2123
Update Privacy Center Icons #1800
Privacy Center fides-consent.js:
- Fides.shopify integration function. #2152
- Dedicated folder for integrations.
- Fides.meta integration function (fbq). #2217
Adds support for Twilio email service (Sendgrid) #2154
Access and erasure support for Recharge #1709
Access and erasure support for Friendbuy Nextgen #2085

Changed

Admin UI Feature Flags - #2101
- Overrides can be saved in the browser.
- Use NEXT_PUBLIC_APP_ENV for app-specific environment config.
- No longer use react-feature-flags library.
- Can have descriptions. #2243
Made privacy declarations optional when adding systems manually - #2173
Removed an unclear logging message. #2266
Allow any user with user:delete scope to delete other users #2148
Dynamic imports of custom overrides and SaaS test fixtures #2169
Added AuthenticatedClient to custom request override interface #2171
Only approve the specific collection instead of the entire dataset, display only top 1 classification by default #2226
Update sample project resources for fides evaluate usage in fides deploy #2253

Removed

Removed unused object_name field on s3 storage config #2133

Fixed

Remove next-auth from privacy center to fix JS console error #2090
Admin UI - Added Missing ability to assign user:delete in the permissions checkboxes #2148
Nav bug: clicking on Privacy Request breadcrumb takes me to Home instead of /privacy-requests #497
Side nav disappears when viewing request details #2129
Remove usage of load dataset button and other dataset UI modifications #2149
Improve readability for exceptions raised from custom request overrides #2157
Importing custom request overrides on server startup #2186
Remove warning when env vars default to blank strings in docker-compose #2188
Fix Cookie House purchase modal flashing 'Error' in title #2274
Stop dependency from upgrading packaging to version with known issue #2273
Privacy center config no longer requires identity_inputs and will use email as a default #2263
No longer display remaining days for privacy requests in terminal states #2292

Removed

Remove "Create New System" button when viewing systems. All systems can now be created via the "Add systems" button on the home page. #2132

2.4.0

Developer Experience

Include a pre-check workflow that collects the pytest suite #2098
Write to the application db when running the app locally. Write to the test db when running pytest #1731

Changed

Move the fides.ctl.core. and fides.ctl.connectors modules into fides.core and fides.connectors respectively #2097
Fides: Skip cypress tests due to nav bar 2.0 #2102

Added

Adds new erasure policy for complete user data masking #1839
New Fides Home page #1864
Nav 2.0 - Replace form flow side navs with top tabs #2037
Adds new erasure policy for complete user data masking #1839
Added ability to use Mailgun templates when sending emails. #2039
Adds SMS id verification for consent #2094

Fixed

Store fides_consent cookie on the root domain of the Privacy Center #2071
Properly set the expire-time for verification codes #2105

2.3.1

Fixed

Resolved an issue where the root_user was not being created #2082

Added

Nav redesign with sidebar groups. Feature flagged to only be visible in dev mode until release. #2030
Improved error handling for incorrect app encryption key #2089
Access and erasure support for Friendbuy API #2019

2.3.0

Added

Common Subscriptions for app-wide data and feature checks. #2030
Send email alerts on privacy request failures once the specified threshold is reached. #1793
DSR Notifications (toast) #1895
DSR configure alerts btn #1895
DSR configure alters (FE) #1895
Add a usage session to Nox to print full session docstrings. #2022

Added

Adds notifications section to toml files #2026

Changed

Updated to use loguru logging library throughout codebase #2031
Do not always create a fides.toml by default #2023
The fideslib module has been merged into fides, code redundancies have been removed #1859
Replace 'ingress' and 'egress' with 'sources' and 'destinations' across UI #2044
Update the functionality of fides pull -a <filename> to include all resource types. #2083

Fixed

Timing issues with bulk DSR reprocessing, specifically when analytics are enabled #2015
Error caused by running erasure requests with disabled connectors #2045
Changes the SlowAPI ratelimiter's backend to use memory instead of Redis #2054

2.2.2

Docs

Updated the readme to use new new docs site #2020

Deprecated

The documentation site hosted in the /docs directory has been deprecated. All documentation updates will be hosted at the new docs site #2020

Fixed

Fixed mypy and pylint errors #2013
Update connection test endpoint to be effectively non-blocking #2000
Update Fides connector to better handle children with no access results #2012

2.2.1

Added

Add health check indicator for data flow scanning option #1973

Changed

The celery.toml is no longer used, instead it is a subsection of the fides.toml file #1990
Update sample project landing page copy to be version-agnostic #1958
get and ls CLI commands now return valid fides object YAML #1991

Developer Experience

Remove duplicate fastapi-caching and pin version. #1765

2.2.0

Added

Send email alerts on privacy request failures once the specified threshold is reached. #1793
Add authenticated privacy request route. #1819
Enable the onboarding flow #1836
Access and erasure support for Fullstory API #1821
Add function to poll privacy request for completion #1860
Added rescan flow for the data flow scanner #1844
Add rescan flow for the data flow scanner #1844
Add Fides connector to support parent-child Fides deployments #1861
Classification UI now polls for updates to classifications #1908

Changed

The organization info form step is now skipped if the server already has organization info. #1840
Removed the description column from the classify systems page. #1867
Retrieve child results during fides connector execution #1967

Fixed

Fix error in parent user creation seeding. #1832
Fix DSR error due to unfiltered empty identities #1901

Docs

Remove documentation about no-longer used connection string override #1824
Fix typo in headings #1824
Update documentation to reflect configs necessary for mailgun, twilio_sms and twilio_email service types #1846

...

2.1.0

Added

Classification flow for system data flows
Classification is now triggered as part of data flow scanning
Include ingress and egress fields on system export and datamap/ endpoint #1740
Repeatable unique identifier for dataset fides_keys and metadata #1786
Adds SMS support for identity verification notifications #1726
Added phone number validation in back-end and react phone number form in Privacy Center #1745
Adds SMS message template for all subject notifications #1743
Privacy-Center-Cypress workflow for CI checks of the Privacy Center. #1722
Privacy Center fides-consent.js script for accessing consent on external pages. Details
Erasure support for Twilio Conversations API #1673
Webserver port can now be configured via the CLI command #1858

Changed

Optional dependencies are no longer used for 3rd-party connectivity. Instead they are used to isolate dangerous dependencies. #1679
All Next pages now automatically require login. #1670
Running the webserver command no longer prompts the user to opt out/in to analytics#1724

Developer Experience

Admin-UI-Cypress tests that fail in CI will now upload screen recordings for debugging. #1728
Enable remote debugging from VSCode of live dev app #1780

Removed

Removed the Privacy Center cookieName config introduced in 2.0.0. #1756

Fixed

Exceptions are no longer raised when sending analytics on Windows #1666
Fixed wording on identity verification modal in the Privacy Center #1674
Update system fides_key tooltip text #1533
Removed local storage parsing that is redundant with redux-persist. #1678
Show a helpful error message if Docker daemon is not running during "fides deploy" #1694
Allow users to query their own permissions, including root user. #1698
Single-select taxonomy fields legal basis and special category can be cleared. #1712
Fixes the issue where the security config is not properly loading from environment variables. #1718
Fixes the issue where the CLI can't run without the config values required by the webserver. #1811
Correctly handle response from adobe jwt auth endpoint as milliseconds, rather than seconds. #1754
Fixed styling issues with the EditDrawer component. #1803

Security

Bumped versions of packages that use OpenSSL #1683

2.0.0

Added

Allow delete-only SaaS connector endpoints #1200
Privacy center consent choices store a browser cookie. #1364
- The format is generic. A reasonable set of defaults will be added later: #1444
- The cookie name defaults to fides_consent but can be configured under config.json > consent > cookieName.
- Each consent option can provide an array of cookieKeys.
Individually select and reprocess DSRs that have errored #1203
Bulk select and reprocess DSRs that have errored #1205
Config Wizard: AWS scan results populate in system review forms. #1454
Integrate rate limiter with Saas Connectors. #1433
Config Wizard: Added a column selector to the scan results page of the config wizard #1590
Config Wizard: Flow for runtime scanner option #1640
Access support for Twilio Conversations API #1520
Message Config: Adds Twilio Email/SMS support #1519

Changed

Updated mypy to version 0.981 and Python to version 3.10.7 #1448

Developer Experience

Repository dispatch events are sent to fidesctl-plus and fidesops-plus #1263
Only the docs-authors team members are specified as CODEOWNERS #1446
Updates the default local configuration to not defer tasks to a worker node #1552
Updates the healthcheck to return health status of connected Celery workers #1588

Docs

Remove the tutorial to prepare for new update #1543
Add system management via UI documentation #1541
Added DSR quickstart docs, restructured docs navigation #1651
Update privacy request execution overview docs #1258

Fixed

Fixed system dependencies appearing as "N/A" in the datamap endpoint when there are no privacy declarations #1649

1.9.6

Fixed

Include systems without a privacy declaration on data map #1603
Handle malformed tokens #1523
Remove thrown exception from getAllPrivacyRequests method #1592
Include systems without a privacy declaration on data map #1603
After editing a dataset, the table will stay on the previously selected collection instead of resetting to the first one. #1511
Fix redis db_index config issue #1647

Docs

Add unlinked docs and fix any remaining broken links #1266
Update privacy center docs to include consent information #1537
Update UI docs to include DSR countdown information and additional descriptions/filtering #1545

Changed

Allow multiple masking strategies to be specified when using fides as a masking engine #1647

1.9.5

Added

The database includes a plus_system_scans relation, to track the status and results of System Scanner executions in fidesctl-plus #1554

1.9.4

Fixed

After editing a dataset, the table will stay on the previously selected collection instead of resetting to the first one. #1511

1.9.2

Deprecated

Added a deprecation warning for the entire package #1244

Added

Dataset generation enhancements using Fides Classify for Plus users:
- Integrate Fides Plus API into placeholder features introduced in 1.9.0. #1194
Fides Admin UI:
- Configure Connector after creation #1204

Fixed

Privacy Center:
- Handle error on startup if server isn't running #1239
- Fix styling issue with cards #1240
- Redirect to index on consent save #1238

1.9.1

Changed

Update fideslang to v1.3.1 #1136

Changed

Update fideslang to v1.3.1 #1136

1.9.0 - 2022-09-29

Added

Dataset generation enhancements using Fides Classify for Plus users:
- Added toggle for enabling classify during generation. #1057
- Initial implementation of API request to kick off classify, with confirmation modal. #1069
- Initial Classification & Review status for generated datasets. #1074
- Component for choosing data categories based on classification results. #1110
- The dataset fields table shows data categories from the classifier (if available). #1088
- The "Approve" button can be used to update the dataset with the classifier's suggestions. #1129
System management UI:
- New page to add a system via yaml #1062
- Skeleton of page to add a system manually #1068
- Refactor config wizard system forms to be reused for system management #1072
- Add additional optional fields to system management forms #1082
- Delete a system through the UI #1085
- Edit a system through the UI #1096
Cypress component testing #1106

Changed

Changed behavior of load_default_taxonomy to append instead of upsert #1040
Changed behavior of adding privacy declarations to decouple the actions of the "add" and "next" buttons #1086
Moved system related UI components from the config-wizard directory to the system directory #1097
Updated "type" on SaaS config to be a simple string type, not an enum #1197

Developer Experience

Optional dependencies may have their version defined only once, in optional-requirements.txt #1171

Docs

Updated the footer links #1130

Fixed

Fixed the "help" link in the UI header #1078
Fixed a bug in Data Category Dropdowns where checking i.e. user.biometric would also check user.biometric_health #1126

Security

Upgraded pymysql to version 1.0.2 #1094

1.8.6 - 2022-09-28

Added

Added classification tables for Plus users #1060

Fixed

Fixed a bug where rows were being excluded from a data map #1124

1.8.5 - 2022-09-21

Changed

Update fideslang to v1.3.0 #1103

1.8.4 - 2022-09-09

Added

Initial system management page #1054

Changed

Deleting a taxonomy field with children will now cascade delete all of its children as well. #1042

Fixed

Fixed navigating directly to frontend routes loading index page instead of the correct static page for the route.
Fix truncated evaluation error messages #1053

1.8.3 - 2022-09-06

Added

Added more taxonomy fields that can be edited via the UI #1000 #1028
Added the ability to add taxonomy fields via the UI #1019
Added the ability to delete taxonomy fields via the UI #1006
- Only non-default taxonomy entities can be deleted #1023
Prevent deleting taxonomy is_default fields and from adding is_default=True fields via the API #990.
Added a "Custom" tag to distinguish user defined taxonomy fields from default taxonomy fields in the UI #1027
Added initial support for enabling Fides Plus #1037
- The useFeatures hook can be used to check if plus is enabled.
- Navigating to/from the Data Map page is gated behind this feature.
- Plus endpoints are served from the private Plus image.

Fixed

Fixed failing mypy tests #1030
Fixed an issue where fides push --diff would return a false positive diff #1026
Pinned pydantic version to < 1.10.0 to fix an error in finding referenced fides keys #1045

Fixed

Fixed failing mypy tests #1030
Fixed an issue where fides push --diff would return a false positive diff #1026

Docs

Minor formatting updates to Policy Webhooks documentation #1114

Removed

Removed create superuser #1116

1.8.2 - 2022-08-18

Added

Added the ability to edit taxonomy fields via the UI #977 #1028
New column is_default added to DataCategory, DataUse, DataSubject, and DataQualifier tables #976
Added the ability to add taxonomy fields via the UI #1019
Added the ability to delete taxonomy fields via the UI #1006
- Only non-default taxonomy entities can be deleted #1023
Prevent deleting taxonomy is_default fields and from adding is_default=True fields via the API #990.
Added a "Custom" tag to distinguish user defined taxonomy fields from default taxonomy fields in the UI #1027

Changed

Upgraded base Docker version to Python 3.9 and updated all other references from 3.8 -> 3.9 #974
Prepend all database tables with ctl_ #979
Moved the admin-ui code down one level into a ctl subdir #970
Extended the /datamap endpoint to include extra metadata #992

1.8.1 - 2022-08-08

Deprecated

The following environment variables have been deprecated, and replaced with the new environment variable names indicated below. To avoid breaking existing workflows, the deprecated variables are still respected in v1.8.1. They will be removed in a future release.
- FIDESCTL__API__DATABASE_HOST --> FIDESCTL__DATABASE__SERVER
- FIDESCTL__API__DATABASE_NAME --> FIDESCTL__DATABASE__DB
- FIDESCTL__API__DATABASE_PASSWORD --> FIDESCTL__DATABASE__PASSWORD
- FIDESCTL__API__DATABASE_PORT --> FIDESCTL__DATABASE__PORT
- FIDESCTL__API__DATABASE_TEST_DATABASE_NAME --> FIDESCTL__DATABASE__TEST_DB
- FIDESCTL__API__DATABASE_USER --> FIDESCTL__DATABASE__USER

Developer Experience

The included docker-compose.yml no longer references outdated ENV variables #964

Docs

Minor release documentation now reflects the desired patch release process #955
Updated references to ENV variables #964

Fixed

Deprecated config options will continue to be respected when set via environment variables #965
The git cache is rebuilt within the Docker container #962
The wheel pypi build no longer has a dirty version tag #962
Add setuptools to dev-requirements to fix versioneer error #983

1.8.0 - 2022-08-04

Added

Initial configuration wizard UI view
- System scanning step: AWS credentials form and initial generate API usage.
- System scanning results: AWS systems are stored and can be selected for review
CustomInput type "password" with show/hide icon.
Pull CLI command now checks for untracked/unstaged files in the manifests dir #869
Pull CLI command has a flag to pull missing files from the server #895
Add BigQuery support for the generate command and /generate endpoint #814 & #917
Added user auth tables 915
Standardized API error parsing under ~/types/errors
Added taxonomy page to UI #902
- Added a nested accordion component for displaying taxonomy data #910
Add lru cache to get_config 927
Add support for deprecated API config values #959
fides is now an alias for fidesctl as a CLI entrypoint #926
Add user auth routes 929
Bump fideslib to 3.0.1 and remove patch code931
Update the fidesctl python package to automatically serve the UI #941
Add push cli command alias for apply and deprecate apply 943
Add resource groups tagging api as a source of system generation 939
Add GitHub Action to publish the fidesctl package to testpypi on pushes to main #951
Added configWizardFlag to ui to hide the config wizard when false [#1453

Changed

Updated the datamap endpoint to return human-readable column names as the first response item #779
Remove the obscure requirement from the generate endpoint #819
Moved all files from fidesapi to fidesctl/api #885
Moved scan and generate to the list of commands that can be run in local mode #841
Upgraded the base docker images from Debian Buster to Bullseye #958
Removed ipython as a dev-requirement #958
Webserver dependencies now come as a standard part of the package #881
Initial configuration wizard UI view
- Refactored step & form results management to use Redux Toolkit slice.
Change id field in tables from an integer to a string 915
Update fideslang to 1.1.0, simplifying the default taxonomy and adding tags for resources #865
Merge existing configurations with fideslib library #913
Moved frontend static files to src/fidesctl/ui-build/static #934
Replicated the error response handling from the /validate endpoint to the /generate endpoint #911

Developer Experience

Remove API_PREFIX from fidesctl/core/utils.py and change references to API_PREFIX in fidesctl/api/reoutes/util.py 922

Fixed

Dataset field columns show all columns by default in the UI #898
Fixed the missing .fides./ directory when locating the default config #933

1.7.1 - 2022-07-28

Added

Add datasets via YAML in the UI #813
Add datasets via database connection #834 #889
Add delete confirmation when deleting a field or collection from a dataset #809
Add ability to delete datasets from the UI #827
Add Cypress for testing 713
Add datasets via database connection (UI only) #834
Add Okta support to the /generate endpoint #842
Add db support to /generate endpoint 849
Added OpenAPI TypeScript client generation for the UI app. See the README for more details.

Changed

Remove the obscure requirement from the generate endpoint #819

Developer Experience

When releases are published, dispatch a repository webhook event to ethyca/fidesctl-plus #938

Docs

recommend/replace pip installs with pipx #874

Fixed

CustomSelect input tooltips appear next to selector instead of wrapping to a new row.
Datasets without the third_country_transfer will not cause the editing dataset form to not render.
Fixed a build issue causing an unknown version of fidesctl to be installed in published Docker images #836
Fixed an M1-related SQLAlchemy bug #816
Endpoints now work with or without a trailing slash. #886
Dataset field columns show all columns by default in the UI #898
Fixed the tag specific GitHub Action workflows for Docker and publishing docs. #901

1.7.0 - 2022-06-23

Added

Added dependabot to keep dependencies updated
A warning now issues for any orphan datasets as part of the apply command 543
Initial scaffolding of management UI #561
A new audit command for system and organization resources, checking data map attribute compliance #548
Static UI assets are now built with the docker container #663
Host static files via fidesapi #621
A new generate endpoint to enable capturing systems from infrastructure from the UI #642
A new datamap endpoint to enable visualizing a data map from the UI #721
Management UI navigation bar #679
Management UI integration #736
- Datasets
- Systems
- Taxonomy (data categories)
Initial dataset UI view #768
- Add interaction for viewing a dataset collection
- Add column picker
- Add a data category checklist tree
- Edit/delete dataset fields
- Edit/delete dataset collections
- Edit datasets
- Add a component for Identifiability tags
- Add tooltips for help on forms
- Add geographic location (third_country_transfers) country selection. Supported by new dependency i18n-iso-countries.
Okta, aws and database credentials can now come from fidesctl.toml config #694
New validate endpoint to test aws and okta credentials #722
Initial configuration wizard UI view
- Manual entry steps added (name and describe organization, pick entry route, and describe system manually including privacy declarations)
A new image tagged ethyca/fidesctl:dev is published on each push to main 781
A new cli command (fidesctl sync) #765

Changed

Comparing server and CLI versions ignores .dirty only differences, and is quiet on success when running general CLI commands 621
All endpoints now prefixed by /api/v1 #623
Allow AWS credentials to be passed to generate system via the API #645
Update the export of a datamap to load resources from the server instead of a manifest directory #662
Refactor export to remove CLI specific uses from the core modules and load resources#725
Bump version of FastAPI in setup.py to 0.77.1 to match optional-requirements.txt #734
Docker images are now only built and pushed on tags to match when released to pypi #740
Okta resource scanning and generation now works with systems instead of datasets #751

Developer Experience

Replaced make with nox #547
Removed usage of fideslang module in favor of new external package shared across projects #619
Added a UI service to the docker-compose deployment #757
TestClient defined in and shared across test modules via conftest.py #759

Docs

Replaced all references to make with nox #547
Removed config/schemas page #613
Dataset UI and config wizard docs added (ethyca#697)
The fides README now walks through generating a datamap #746

Fixed

Updated fideslog to v1.1.5, resolving an issue where some exceptions thrown by the SDK were not handled as expected #609
Updated the webserver so that it won't fail if the database is inaccessible #649
Updated external tests to handle complex characters #661
Evaluations now properly merge the default taxonomy into the user-defined taxonomy #684
The CLI can now be run without installing the webserver components #715

1.6.1 - 2022-06-15

Docs

Updated Release Steps

Fixed

Resolved a failure with populating applicable data subject rights to a data map
Handle invalid characters when generating a fides_key #761

1.6.0 - 2022-05-02

Added

ESLint configuration changes #514
User creation, update and permissions in the Admin UI #511
Yaml support for dataset upload #284

Breaking Changes

Update masking API to take multiple input values #443

Docs

DRP feature documentation #520

1.4.2 - 2022-05-12

Added

GET routes for users #405
Username based search on GET route #444
FIDESOPS__DEV_MODE for Easier SaaS Request Debugging #363
Track user privileges across sessions #425
Add first_name and last_name fields. Also add them along with created_at to FidesUser response #465
Denial reasons for DSR and user AuditLog #463
DRP action to Policy #453
CHANGELOG.md file#484
DRP status endpoint #485
DRP exerise endpoint #496
Frontend for privacy request denial reaons #480
Publish Fidesops to Pypi #491
DRP data rights endpoint #526

Changed

Converted HTTP Status Codes to Starlette constant values #438
SaasConnector.send behavior on ignore_errors now returns raw response #462
Seed user permissions in create_superuser.py script #468
User API Endpoints (update fields and reset user passwords) #471
Format tests with black #466
Extract privacy request endpoint logic into separate service for DRP #470
Fixing inconsistent SaaS connector integration tests #473
Add user data to login response #501

Breaking Changes

Update masking API to take multiple input values #443

Docs

Added issue template for documentation updates #442
Clarify masking updates #464
Added dark mode #476

Fixed

Removed miradb test warning #436
Added missing import #448
Removed pypi badge pointing to wrong package #452
Audit imports and references #479
Switch to using update method on PUT permission endpoint #500

Developer Experience

added isort as a CI check
Include tests/ in all static code checks (e.g. mypy, pylint)

Changed

Published Docker image does a clean install of Fidesctl
with_analytics is now a decorator

Fixed

Third-Country formatting on Data Map
Potential Duplication on Data Map
Exceptions are no longer raised when sending AnalyticsEvents on Windows
Running fidesctl init now generates a server_host and server_protocol rather than server_url

Files

CHANGELOG.md

Latest commit

History

CHANGELOG.md

File metadata and controls

Changelog

Added

Added

Changed

Fixed

Developer Experience

Fixed

Added

Changed

Fixed

Removed

Added

Changed

Developer Experience

Fixed

Added

Fixed

Fixed

Fixed

Added

Changed

Developer Experience

Fixed

Added

Changed

Fixed

Developer Experience

Removed

Changed

Added

Fixed

Fixed

Fixed

Fixed

Added

Changed

Developer Experience

Fixed

Developer Experience

Fixed

Security

Developer Experience

Docs

Added

Changed

Removed

Fixed

Removed

Developer Experience

Changed

Added

Fixed

Fixed

Added

Added

Added

Changed

Fixed

Docs

Deprecated

Fixed

Added

Changed

Developer Experience

Added

Changed

Fixed

Docs

Added

Changed

Developer Experience

Removed

Fixed

Security