v0.9.3

• Fixed issue when Databricks SDK config objects were overridden for installation config files (#170). This commit addresses an issue where Databricks SDK config objects were being overridden during installation config files creation, which has been resolved by modifying the _marshal method in the installation class to handle databricks.sdk.core.Config instances more carefully, and by introducing a new helper function get_databricks_sdk_config in the paths.py file, which retrieves the Databricks SDK configuration and improves the reliability and robustness of the SDK configuration. This fixes bug #169 and ensures that the SDK configuration is not accidentally modified during the installation process, preventing unexpected behavior and errors. The changes are isolated to the paths.py file and do not affect other parts of the codebase.

Contributors: @FastLee

v0.9.2

• Bump actions/checkout from 4.2.1 to 4.2.2 (#160). In this release, the 'actions/checkout' dependency has been updated from version 4.2.1 to 4.2.2. This update includes changes to the 'url-helper.ts' file, which now utilizes well-known environment variables for improved reliability and maintainability. Additionally, unit test coverage for the isGhes function has been expanded. These changes are recommended for adoption to take advantage of the enhancements. The pull request includes a detailed changelog, commit history, and instructions for managing the update using Dependabot commands and options.
• Bump databrickslabs/sandbox from acceptance/v0.3.1 to 0.4.2 (#166). In the latest release, the databrickslabs/sandbox Python package has been updated from version acceptance/v0.3.1 to 0.4.2. This update includes new features such as installation instructions, additional go-git libraries, and modifications to the README file. Dependency updates include a bump in the version of golang.org/x/crypto used. The pull request for this update was created by a GitHub bot, Dependabot, which will manage any conflicts and respond to comments containing specific commands. It is essential to thoroughly review and test this updated version to ensure that the new methods and modifications to existing functionality do not introduce any issues or regressions, and that the changes are well-documented and justified.
• Don't draft automated releases (#159). In this release, the draft release feature in the GitHub Actions workflow has been disabled, enhancing the release process for software engineers. The 'draft: true' parameter has been removed from the Draft release job, which means that automated releases will now be published immediately upon creation instead of being created as drafts. This modification simplifies and streamlines the release process, making it more efficient for engineers who adopt the project. The change is aimed at reducing the time and effort required in manually publishing draft releases, thereby improving the overall experience for project contributors and users.
• Updated custom Path support for python 3.13 (#161). In this revision, the project's continuous integration (CI) workflow has been updated to include Python 3.13, enhancing compatibility and enabling early identification of platform-specific issues. The paths module has been refactored into several submodules for better organization, and a new submodule, databrickspath_posixpath, has been added to distinguish PosixPath from DBFSPath and WorkspacePath. The comparison and equality behavior of _DatabricksPath objects has been modified to include parser property identity checks in Python 3.13, ensuring consistent behavior and eliminating exceptions when built-in paths are compared with custom paths. These updates promote confidence in the project's long-term viability and adaptability in response to evolving language standards.

Dependency updates:

• Bump actions/checkout from 4.2.1 to 4.2.2 (#160).
• Bump databrickslabs/sandbox from acceptance/v0.3.1 to 0.4.2 (#166).

Contributors: @asnare, @dependabot[bot], @nfx

v0.9.1

• Bump actions/checkout from 4.1.7 to 4.2.0 (#149). In this pull request, the actions/checkout dependency is upgraded from version 4.1.7 to 4.2.0 in the acceptance.yml and downstreams.yml workflow files. The new version provides additional Ref and Commit outputs, as well as updated dependencies, which aim to improve the functionality and security of the checkout process. The Ref output is a string representing the reference that was checked out, and the Commit output is the SHA-1 hash of the checked-out commit. Dependency updates include bumping the braces package from 3.0.2 to 3.0.3 and updating the minor-npm-dependencies group across one directory with four updates. These changes contribute to a more reliable and efficient checkout process and enhance the overall functionality and maintainability of the Action. Software engineers are recommended to review the changes and ensure they do not introduce conflicts with their current setup before adopting the new version.
• Bump actions/checkout from 4.2.0 to 4.2.1 (#152). In this update, the version of the actions/checkout GitHub Action is bumped from 4.2.0 to 4.2.1 in a project's GitHub workflow files. This new version includes a modification to check out other refs/* by commit if provided, falling back to the ref. This change enhances the flexibility of the checkout action in handling different types of references, which could be useful for users working with multiple branches or references in their workflows. The update also adds a workflow file for publishing releases to an immutable action package. This release was contributed by the new project collaborator, @orhantoy, who made the change in pull request 1924.
• Bump databrickslabs/sandbox from acceptance/v0.3.0 to 0.3.1 (#155). In this update, the dependency for databrickslabs/sandbox has been bumped from version acceptance/v0.3.0 to 0.3.1. This change includes bug fixes, upgrades to go-git libraries, and dependency updates. The golang.org/x/crypto library was specifically bumped from version 0.16.0 to 0.17.0 in both /go-libs and /runtime-packages. Additionally, the cac167b commit expanded acceptance test logs and introduced experimental OIDC refresh token rotation. The acceptance test job in the workflow was also updated to use the new version of databrickslabs/sandbox. Ignore conditions were added for previous versions of databrickslabs/sandbox in this release. The README was also modified, and install instructions were added to the changelog.
• Catch all errors when checking Databricks path, notably BadRequest ones (#156). This commit introduces improvements to the error handling of the exists method in the paths.py file when checking Databricks path. Previously, only NotFound errors were caught, but now BadRequest errors are also handled, addressing issue #2882. The exists method has been updated to catch and manage DatabricksError exceptions, which now encompass BadRequest errors, ensuring comprehensive error handling for Databricks path-related operations. Additionally, the _cached_file_info and _cached_object_info attributes are now initialized when a DatabricksError exception occurs, returning False accordingly. This enhancement maintains consistency and accuracy in the exists method while broadening the range of errors captured, resulting in a more robust and reliable codebase with enhanced error reporting for users.
• Normalize databricks paths as part of resolving them (#157). In this release, the resolve method in the paths.py file of the databricks/labs/blueprint project has been enhanced to handle parent directory references ("..") consistently with Python's built-in Path object. Previously, Path("/a/b/../c").resolve() would return Path("/a/b/c"), while Databricks paths were not behaving consistently. This modification introduces a new _normalize() method, which processes the path parts and ensures that ".." segments are handled correctly. The commit also includes a new test function, 'test_resolve_is_consistent', which checks the consistent resolution of Databricks paths with various input formats, such as relative paths, ".." or "." components, and absolute paths. This change ensures that the resolved path will be normalized according to the expected behavior, regardless of the input format, contributing to the resolution of issue #2882. By normalizing Databricks paths in the same fashion as Python's built-in Path object, the code should become more robust and predictable, providing a more reliable and predictable experience for software engineers utilizing the project.
• Updated databrickslabs/sandbox requirement to acceptance/v0.3.0 (#153). In this pull request, the databrickslabs/sandbox package requirement in the downstreams GitHub Actions workflow is updated to version 0.3.0, which is the latest version available. This package provides a sandbox environment for development and testing, and the new version includes bug fixes and dependency updates that may enhance its reliability and performance. Dependabot has been used to ensure a smooth update process, with any conflicts being resolved automatically. However, it is recommended to review the changelog and test the updated version before merging this pull request to ensure compatibility and functionality in your specific use case. Additionally, Dependabot commands are available to manage ignore conditions for this dependency.

Dependency updates:

• Bump actions/checkout from 4.1.7 to 4.2.0 (#149).
• Bump actions/checkout from 4.2.0 to 4.2.1 (#152).
• Updated databrickslabs/sandbox requirement to acceptance/v0.3.0 (#153).
• Bump databrickslabs/sandbox from acceptance/v0.3.0 to 0.3.1 (#155).

Contributors: @dependabot[bot], @ericvergnaud

v0.9.0

• Added Databricks CLI version as part of routed command telemetry (#147). A new environment variable, "DATABRICKS_CLI_VERSION", has been introduced in the Databricks CLI version for routed command telemetry. This variable is incorporated into the with_user_agent_extra method, which adds it to the user agent for outgoing requests, thereby enhancing detailed tracking and version identification in telemetry data. The with_user_agent_extra method is invoked twice, with the blueprint prefix and the version variable, followed by the cli prefix and the DATABRICKS_CLI_VERSION environment variable, ensuring that both the blueprint and CLI versions are transmitted in the user agent for all requests.

Contributors: @nfx

v0.8.3

• add missing stat() methods to DBFSPath and WorkspacePath (#144). The stat() method has been added to both DBFSPath and WorkspacePath classes, addressing issues #142 and #143. This method, which adheres to the Posix standard, returns file status in the os.stat_result format, providing access to various metadata attributes such as file size, last modification time, and creation time. By incorporating this method, developers can now obtain essential file information for Databricks File System (DBFS) and Databricks Workspace paths when working with these classes. The change includes a new test case for stat() in the test_paths.py file to ensure the correctness of the method for both classes.

Contributors: @ericvergnaud

v0.8.2

• Make hatch a prerequisite (#137). In version 1.9.4, hatch has become a prerequisite for installation in the GitHub workflow for the project's main branch, due to occasional failures in pip install hatch that depend on the local environment. This change, which includes defining the hatch version as an environment variable and adding a new step for installing hatch with a specific version, aims to enhance the reliability of the build and testing process by eliminating potential installation issues with hatch. Users should install hatch manually before executing the Makefile, as the line pip install hatch has been removed from the Makefile. This change aligns with the approach taken for ucx, and users are expected to understand the requirement to install prerequisites before executing the Makefile. To contribute to this project, please install hatch using pip install hatch, clone the GitHub repository, and run make dev to start the development environment and install necessary dependencies.
• support files with unicode BOM (#138). The recent change to the open-source library introduces support for handling files with a Unicode Byte Order Mark (BOM) during file upload and download operations in Databricks Workspace. This new functionality, added to the WorkspacePath class, allows for easier reading of text from files with the addition of a read_text method. When downloading a file, if it starts with a BOM, it will be detected and used for decoding, regardless of the preferred encoding based on the system's locale. The change includes a new test function that verifies the accurate encoding and decoding of files with different types of BOM using the appropriate encoding. Despite the inability to test Databrick notebooks with a BOM due to the Databricks platform modifying the uploaded data, this change enhances support for handling files with various encodings and BOM, improving compatibility with a broader range of file formats, and ensuring more accurate handling of files with BOM.

Contributors: @ericvergnaud

v0.8.1

• Fixed py3.10 compatibility for _parts in pathlike (#135). The recent update to our open-source library addresses the compatibility issue with Python 3.10 in the _parts property of a certain type. Prior to this change, there was also a _cparts property that returned the same value as _parts, which has been removed and replaced with a direct reference to _parts. The _parts property can now be accessed via reverse equality comparison, and this change has been implemented in the joinpath and __truediv__ methods as well. This enhancement improves the library's compatibility with Python 3.10 and beyond, ensuring continued functionality and stability for software engineers working with the latest Python versions.

Contributors: @nfx

v0.8.0

• Added DBFSPath as os.PathLike implementation (#131). The open-source library has been updated with a new class DBFSPath, an implementation of os.PathLike for Databricks File System (DBFS) paths. This new class extends the existing WorkspacePath support and provides pathlib-like functionality for DBFS paths, including methods for creating directories, renaming and deleting files and directories, and reading and writing files. The addition of DBFSPath includes type-hinting for improved code linting and is integrated in the test suite with new and updated tests for path-like objects. The behavior of the exists and unlink methods have been updated for WorkspacePath to improve performance and raise appropriate errors.
• Fixed .as_uri() and .absolute() implementations for WorkspacePath (#127). In this release, the WorkspacePath class in the paths.py module has been updated with several improvements to the .as_uri() and .absolute() methods. These methods now utilize PathLib internals, providing better cross-version compatibility. The .as_uri() method now uses an f-string for concatenation and returns the UTF-8 encoded string representation of the WorkspacePath object via a new __bytes__() dunder method. Additionally, the .absolute() method has been implemented for the trivial (no-op) case and now supports returning the absolute path of files or directories in Databricks Workspace. Furthermore, the glob() and rglob() methods have been enhanced to support case-sensitive pattern matching based on a new case_sensitive parameter. To ensure the integrity of these changes, two new test cases, test_as_uri() and test_absolute(), have been added, thoroughly testing the functionality of these methods.
• Fixed WorkspacePath support for python 3.11 (#121). The WorkspacePath class in our open-source library has been updated to improve compatibility with Python 3.11. The .expanduser() and .glob() methods have been modified to address internal changes in Python 3.11. The is_dir() and is_file() methods now include a follow_symlinks parameter, although it is not currently used. A new method, _scandir(), has been added for compatibility with Python 3.11. The expanduser() method has also been updated to expand ~ (but not ~user) constructs. Additionally, a new method is_notebook() has been introduced to check if the path points to a notebook in Databricks Workspace. These changes aim to ensure that the library functions smoothly with the latest version of Python and provides additional functionality for users working with Databricks Workspace.
• Properly verify versions of python (#118). In this release, we have made significant updates to the pyproject.toml file to enhance project dependency and development environment management. We have added several new packages to the dependencies section to expand the library's functionality and compatibility. Additionally, we have removed the python field, as it is no longer necessary. We have also updated the path field to specify the location of the virtual environment, which can improve integration with popular development tools such as Visual Studio Code and PyCharm. These changes are intended to streamline the development process and make it easier to manage dependencies and set up the development environment.
• Type annotations on path-related unit tests (#128). In this open-source library update, type annotations have been added to path-related unit tests to enhance code clarity and maintainability. The tests encompass various scenarios, including verifying if a path exists, creating, removing, and checking directories, and testing file attributes such as distinguishing directories, notebooks, and regular files. The additions also cover functionality for opening and manipulating files in different modes like read binary, write binary, read text, and write text. Furthermore, tests for checking file permissions, handling errors, and globbing (pattern-based file path matching) have been incorporated. The tests interact with a WorkspaceClient mock object, simulating file system interactions. This enhancement bolsters the library's reliability and assists developers in creating robust, well-documented code when working with file system paths.
• Updated WorkspacePath to support Python 3.12 (#122). In this release, the WorkspacePath implementation has been updated to ensure compatibility with Python 3.12, in addition to Python 3.10 and 3.11. The class was modified to replace most of the internal implementation and add extensive tests for public interfaces, ensuring that the superclass implementations are not used unless they are known to be safe. This change is in response to the significant changes in the superclass implementations between Python 3.11 and 3.12, which were found to be incompatible with each other. The WorkspacePath class now includes several new methods and tests to ensure that it functions seamlessly with different versions of Python. These changes include testing for initialization, equality, hash, comparison, path components, and various path manipulations. This update enhances the library's adaptability and ensures it functions correctly with different versions of Python. Classifiers have also been updated to include support for Python 3.12.
• WorkspacePath fixes for the .resolve() implementation (#129). The .resolve() method for WorkspacePath has been updated to improve its handling of relative paths and the strict argument. Previously, relative paths were not properly validated and would be returned as-is. Now, relative paths will cause the method to fail. The strict argument is now checked, and if set to True and the path does not exist, a FileNotFoundError will be raised. The method .absolute() is used to obtain the absolute path of the file or directory in Databricks Workspace and is used in the implementation of .resolve(). A new test, test_resolve(), has been added to verify these changes, covering scenarios where the path is absolute, the path exists, the path does not exist, and the path is relative. In the case of relative paths, a NotImplementedError is raised, as .resolve() is not supported for them.
• WorkspacePath: Fix the .rename() and .replace() implementations to return the target path (#130). The .rename() and .replace() methods of the WorkspacePath class have been updated to return the target path as part of the public API, with .rename() no longer accepting the overwrite keyword argument and always failing if the target path already exists. A new private method, ._rename(), has been added to include the overwrite argument and is used by both .rename() and .replace(). This update is a preparatory step for factoring out common code to support DBFS paths. The tests have been updated accordingly, combining and adding functions to test the new and updated methods. The .unlink() method's behavior remains unchanged. Please note that the exact error raised when .rename() fails due to an existing target path is yet to be defined.

Dependency updates:

• Bump sigstore/gh-action-sigstore-python from 2.1.1 to 3.0.0 (#133).

Contributors: @asnare, @nfx, @dependabot[bot]

v0.7.0

• Added databricks.labs.blueprint.paths.WorkspacePath as pathlib.Path equivalent (#115). This commit introduces the databricks.labs.blueprint.paths.WorkspacePath library, providing Python-native pathlib.Path-like interfaces to simplify working with Databricks Workspace paths. The library includes WorkspacePath and WorkspacePathDuringTest classes offering advanced functionality for handling user home folders, relative file paths, browser URLs, and file manipulation methods such as read/write_text(), read/write_bytes(), and glob(). This addition brings enhanced, Pythonic ways to interact with Databricks Workspace paths, including creating and moving files, managing directories, and generating browser-accessible URIs. Additionally, the commit includes updates to existing methods and introduces new fixtures for creating notebooks, accompanied by extensive unit tests to ensure reliability and functionality.
• Added propagation of blueprint version into User-Agent header when it is used as library (#114). A new feature has been introduced in the library that allows for the propagation of the blueprint version and the name of the command line interface (CLI) command used in the User-Agent header when the library is utilized as a library. This feature includes the addition of two new pairs of OtherInfo: blueprint/X.Y.Z to indicate that the request is made using the blueprint library and cmd/<name> to store the name of the CLI command used for making the request. The implementation involves using the with_user_agent_extra function from databricks.sdk.config to set the user agent consistently with the Databricks CLI. Several changes have been made to the test file for test_useragent.py to include a new test case, test_user_agent_is_propagated, which checks if the blueprint version and the name of the command are correctly propagated to the User-Agent header. A context manager http_fixture_server has been added that creates an HTTP server with a custom handler, which extracts the blueprint version and the command name from the User-Agent header and stores them in the user_agent dictionary. The test case calls the foo command with a mocked WorkspaceClient instance and sets the DATABRICKS_HOST and DATABRICKS_TOKEN environment variables to test the propagation of the blueprint version and the command name in the User-Agent header. The test case then asserts that the blueprint version and the name of the command are present and correctly set in the user_agent dictionary.
• Bump actions/checkout from 4.1.6 to 4.1.7 (#112). In this release, the version of the "actions/checkout" action used in the Checkout Code step of the acceptance workflow has been updated from 4.1.6 to 4.1.7. This update may include bug fixes, performance improvements, and new features, although specific changes are not mentioned in the commit message. The Unshallow step remains unchanged, continuing to fetch and clean up the repository's history. This update ensures that the latest enhancements from the "actions/checkout" action are utilized, aiming to improve the reliability and performance of the code checkout process in the GitHub Actions workflow. Software engineers should be aware of this update and its potential impact on their workflows.

Dependency updates:

• Bump actions/checkout from 4.1.6 to 4.1.7 (#112).

Contributors: @nfx, @dependabot[bot]

v0.6.3

• fixed Command.get_argument_type bug with UnionType (#110). In this release, the Command.get_argument_type method has been updated to include special handling for UnionType, resolving a bug that caused the function to crash when encountering this type. The method now returns the string representation of the annotation if the argument is a UnionType, providing more accurate and reliable results. To facilitate this, modifications were made using the types module. Additionally, the foo function has a new optional argument optional_arg of type str, with a default value of None. This argument is passed to the some function in the assertion. The Prompts type has been added to the foo function signature, and an assertion has been added to verify if prompts is an instance of Prompts. Lastly, the default value of the address argument has been changed from an empty string to "default", and the same changes have been applied to the test_injects_prompts test function.

Contributors: @nkvuong