Artificial intelligence
Beyond web interfaces, developers often appreciate working in a local development environment. That’s why Databricks has launched an extension for Visual Studio Code (VS Code). This extension makes it possible to operate the Lakehouse from within a popular IDE, writing code locally, debugging it and deploying it to remote clusters. Such a tool improves developer productivity and brings software engineering best practices to analytics.
According to the official documentation, the VS Code extension for Databricks allows you to :
Define, deploy and run Databricks Asset Bundles to apply CI/CD schemes to Lakeflow and Delta Live Tables jobs. These bundles make it easy to manage sets of notebooks, configurations and scripts in a single, reusable package.
Run local Python code on Databricks clusters. You can develop and test your scripts on your own machine, then send them to a cluster to benefit from the computing power of the Lakehouse.
Launch notebooks or Python, R, Scala or SQL files as Databricks jobs from VS Code.
Configure and debug the environment using Databricks Connect to run and debug notebook cells directly from the IDE.
Synchronize local code with code stored in Databricks workspaces, making it easy to version and share projects.
This extension is available free of charge on the VS Code marketplace, and has received positive reviews for the productivity it brings to developers.
The VS Code extension reduces the context for change: developers no longer need to switch from the IDE to the Databricks web interface. They can apply software development best practices (Git version control, code reviews, continuous integration) directly in their familiar environment. The ability to debug notebooks cell by cell simplifies problem diagnosis. Finally, the extension facilitates collaboration between teams by synchronizing local code and workspaces.
Getting started:
Install the extension from the VS Code marketplace.
Open a folder or create a new project using the Create a new Databricks project option available via the Databricks icon in the sidebar.
Select a host (Azure, AWS or GCP) and an authentication mode (Databricks profile, PAT, etc.).
Choose a project template and configure the bundle files.
Write your code, run it on a remote cluster via the context menu and synchronize changes.
Although the extension supports the execution of notebooks in Python, R, Scala and SQL as jobs, it doesn’t yet offer advanced autocompletion for these languages; development features focus mainly on Python. In addition, the extension collects anonymized telemetric data to improve the product, but this can be disabled in the settings. Finally, some users note that the extension is still young and that improvements are expected, including deeper integration with Unity Catalog and support for other IDE environments.
Why use the VS Code extension for Databricks? It enables developers to write, debug and deploy code locally by connecting to Databricks clusters via their preferred IDE, thus integrating software engineering best practices.
What tasks can I perform with the extension? You can create Databricks projects, define asset bundles, run Python scripts or notebooks, transform them into jobs and debug cells with Databricks Connect.
Is this extension free? Yes. The extension is available free of charge from the VS Code marketplace and installs in just a few clicks.
Are there any limitations? It focuses primarily on Python, while full support for R, Scala and SQL remains limited for advanced functionality. The extension collects anonymized telemetry data by default, but you can disable this option in the settings.