Deepnote
IntegrationsPricing
Join usDocsSign in

Getting started

Deepnote documentation

Environment

Long-running jobs

Long-running jobs

Deepnote is designed to support most data analysis workflows, many of which include long-running tasks. This page explains when the hardware for your projects can be turned off.

What it means for Deepnote to turn off the hardware for a project

Simplifying a little bit, Deepnote has many machines (computers) running in the cloud (pods in a Kubernetes cluster if you want the real jargon). When you start the hardware in your project, we turn on one of these machines for you and run all your computations there. Once you're done with your computations, we turn off this machine for you just like you would turn off your own laptop at the end of your workday. Unlike with your laptop though, you can view the files in your project even while the hardware for your project is turned off – you only need running hardware to execute computations. We automatically start this hardware whenever you start any execution in your project (e.g. when you run a cell).

After the hardware notebook is turned off, we store the outputs of cells, but the values of variables are gone (just like when you turn off your computer).

To start your hardware back up after we turn off the hardware for your project, run any cell or click "Start machine" in the environment sidebar.

How Deepnote decides when to turn off hardware of a project

All projects get hardware turned off when no one is using them. We do this to help you avoid using up your free hardware hours unnecessarily or pay for the unused paid machines. The following two conditions of inactivity have to be met:

  1. The project has no running cells in notebooks. Code running in terminals does not prevent hardware from turning off.
  2. There were no changes to the code or any execution for at least 15 minutes. This maximum idle time is customizable for projects in teams with a Pro plan or higher.

📖 Example: Bob is running a data analysis that takes 12 hours to complete. He starts the analysis and then closes Deepnote for the night. He comes back in the morning just before the analysis is complete and finds his notebook with all his variables is still live. Bob does some ad-hoc exploration and then goes to lunch with the notebook idle. After he comes back, he can still see all his outputs, but the variables and all other runtime objects are gone as the above conditions were met and the hardware for his project was turned off.

Furthermore, hardware of projects running on free hardware gets turned off automatically after running continuously for 24 hours. Any running computation is interrupted. In extraordinary circumstances, we also reserve the right to turn off the hardware for the project earlier. In this case, you will see a message "Your hardware was shut down" next time you visit the project. Please note this does not apply to paid hardwareprojects running on paid hardware can run as long as the owner wants.

Finally, hardware of projects running on free hardware is turned off if their owner runs out of free quota and they don't have a paying method set up. In this case, an info message will also appear next time you visit the notebook, notifying you about running out of the free quota.

Notebook size: Keep in mind that the maximum size of a notebook, together with outputs generated by long running cell executions, is 30MB.

Custom inactivity grace period

Users using our pay-per-use hardware can customize the behavior above and prevent the hardware being automatically turned off after 15 minutes of inactivity. This can be useful if you're unsure when your long-running job finishes and want to keep its state preserved (such as python variables or terminal outputs). You can currently set the grace period to be either 15 minutes or 24 hours.

Deepnote
Product
© 2022 Deepnote. All rights reserved.