– by Elizabeth on July 28, 2022
Snowflake, amongst other products, offers a cloud data warehousing platform with out-of-the-box features like scalable compute, data sharing, or data cloning. Products within Snowflake’s ecosystem, like Snowsight, allow analysts to explore and visualize their data for ad-hoc use cases. But what if you’d like to turn your ad-hoc queries into compelling narratives and easily shareable assets? With Deepnote—now a Snowflake Select Technology Partner—your team can do just that.
In this article, we will walk you through the key pillars that make the Deepnote and Snowflake experience delightful, and show how you can bring both into your toolkit to make your querying and exploration easier, more secure, and collaborative.
For a quick start, we build out a template project to show you the ropes. Use our playground warehouse or connect to your own Snowflake instance using the Integrations menu.
Whether you're a seasoned analyst or new to the whole notebooks thing, you’ve likely encountered SQL Alchemy API
, enabling connection to data sources from within your notebook environment. This is a very common way of extracting data and preparing it for manipulation within a dataframe. However, whilst generaly accepted, it also comes with a couple of pain-points we've listed below:
\
characters to be appended at the end of each line to help the notebook environment recognise that this is all one string to be queried.Here’s what we did to alleviate these concerns. In Deepnote, there's no need to directly use a connector to extract data from your Snowflake warehouse. Through the power of SQL blocks, in Deepnote you can run queries directly on the top of your Snowflake warehouse, from the comfort of your notebook. All you have to do is add your Snowflake credentials and voilà - your notebook is now an interface for exploring your Snowflake data (in a collaborative way too, should you choose to do so!).
In addition to a first-class SQL experience, Deepnote helps you eliminate the memory constraints. You can simply switch over to a more powerful VM instance or spin down dynamically based off your compute needs.
Having access to the Snowflake warehouse coupled with the exploratory powers of notebook means you can write queries, save the results into pandas DataFrames, and visualize them all in one go. You can even inject Python variables into your SQL queries with jinjasql to build out more complex queries and move seamlessly between SQL and Python.
With the built-in DataFrame viewer, we can examine the dataset for missing values, the most common categorical values, distributions of numeric columns, and more. The built-in filters and sorting make it easy to gain a deeper understanding of the data and relationships that might impact our future model's predictions. Lastly, built-in no-code charts allow us to examine our target variable as a function of other features without having to write any additional code. We can seamlessly switch from code to visualizations, and go right back into querying as needed.
As data teams move to using hosted notebooks, it can present security risks with respect to meta-data and authentication credentials. Ongoing git-commits and pushes to shared repositories highlight the chance of leaked credentials and potential security breaches.
At Deepnote, we set out to build the best notebooks experience that lets you collaborate effectively and build better data products faster, and do so in a secure way. All files in Deepnote are encrypted at rest, whether they are files you create or anything you upload. For sensitive information (such as database integrations or environment variables), we apply a layer of AES-256-CBC encryption before storing them in our database where the decryption keys are stored separately.
Better yet, you can configure the connection at the Workspace level and decide whether to make it available to all members and shared projects or just a specific project. As soon as the Snowflake connection is set up, your data is secure, and we don’t have to worry about re-configuring things again. Say goodbye to your multiple JSON files for sharing credentials and simply share access to your project.
At Deepnote, we believe collaboration is key to the notebooks experience. At the end of the day, data work is a team sport. We put a lot of thought into improving in-team collaboration for data teams, but equally, we want to make sure their work is accessible to others - subject matter experts, leaders and data consumers. The emphasis on collaboration links to different steps in your exploratory journey:
Share this post
No credit card required. Run your first notebook in seconds.