Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] stdout logs not appearing during databricks launch runs #1325

Open
rwpurvis opened this issue Aug 22, 2024 · 1 comment
Open

[BUG] stdout logs not appearing during databricks launch runs #1325

rwpurvis opened this issue Aug 22, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@rwpurvis
Copy link

Describe the bug

When running tests via the vscode debug functionality per the tutorial, the logs do not appear in the debug console. I am able to see the logs on the Driver logs page in the UI.

To Reproduce
Steps to reproduce the behavior:

  1. Add a pytest runner script and launch.json configuration
  2. Run Unit Tests (on Databricks)
  3. Observe output with missing logs in Debug Console

Example output (anonymized):

8/22/2024, 3:24:37 PM - Creating execution context on cluster <cluster ID> ...
8/22/2024, 3:24:42 PM - Synchronizing code to <directory> ...
8/22/2024, 3:24:42 PM - Running <directory>\pytest_databricks.py ...


8/22/2024, 3:26:56 PM - Done (took 138949ms)

Previously, pytest logs appeared here.

Screenshots

See above output

System information:

  1. Paste the output ot the Help: About command (CMD-Shift-P).
Version: 1.85.2 (user setup)
Commit: 8b3775030ed1a69b13e4f4c628c612102e30a681
Date: 2024-01-18T06:40:10.514Z
Electron: 25.9.7
ElectronBuildId: 26354273
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: 11.4.183.29-electron.0
OS: Windows_NT x64 10.0.19045
  1. Databricks Extension Version
    Tried with v2.3.2 and v1.3.1

Databricks Extension Logs
Please attach the databricks extension logs

Additional context
Add any other context about the problem here.

@rwpurvis rwpurvis added the bug Something isn't working label Aug 22, 2024
@rwpurvis
Copy link
Author

rwpurvis commented Sep 6, 2024

Some more info here...I mark all pyspark tests with slow, so that non-pyspark tests can be run locally via pytest -m "not slow". When I select these tests via the launch.json, the stdout logs appear. Here's the launch.json:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "type": "databricks",
            "request": "launch",
            "name": "Unit Tests (on Databricks)",
            "program": "${workspaceFolder}/<sub_directory>/pytest_databricks.py",
            "args": [
                ".", "-vv", "-m", "slow"
            ],
            "env": {}
        },
    ]
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant