Reactive notebooks in Deepnote
Traditional computational notebooks like Jupyter receive a lot of criticism. The primary issues usually have to do with out-of-order execution, reasoning about hidden state, and as a result, poor reproducibility of results. We believe that reactive notebooks address some of these issues head on. This article outlines how reactivity works in Deepnote.
Reactive notebooks are like Excel spreadsheets – you just input the data and code, and the whole document updates itself accordingly. No need to click to manually run cells. This makes reactive notebooks easier to reason about and more reproducible.
If you want to try out a reactive notebook for yourself, just open the demo project
Key challenges of notebooks today
Traditional notebooks like Jupyter allow you to execute cells in arbitrary order, potentially repeating the execution of any cell. This has the potential to create hidden state that is extremely difficult to reason about.
Let's take a look at an example:
In the above example, the cells were executed in the order: 1st, 2nd, 3rd, 2nd. This created an out-of-order computational narrative that is very unintuitive. Someone who just reads the notebook—e.g. a supervisor or a colleague—usually does it in a top-to-bottom fashion, so notebook executions are usually interpreted to happen in this order, too.
More hidden state
Every time a notebook is edited but not executed, it becomes stale, as executing it again would create different outputs than the ones already present. To build on the above example, let's say I delete the last cell, and the notebook now becomes this:
This notebook obviously isn't reproducible anymore, even if we account for out-of-order execution.
Iterating on notebooks in this manner is extremely common when doing exploratory programming, which is what notebooks are intended for. So these kinds of issues are very common.
A notebook is reproducible when someone else can take all of its code, run it on a different computer than the author's, and reliably get the same results – in this case, the same cell outputs. Reproducibility is very important in science, where it's part of a standard peer review process. Since data science is also a science (as the name suggests), it's very important for any data science work to also be reproducible.
A recent paper found that only 25% of jupyter notebooks on Github were reproducible.
It is not easy to create a notebook that is reproducible – it's not just about executing your notebook from top to bottom. A reproducible notebook also includes a reproducible environment, and code that is deterministic. These are all issues we're solving at Deepnote.
A reactive notebook is a notebook that is always kept up-to-date. So whenever its code is changed or a cell is deleted or moved, the notebook's outputs are automatically updated as if the notebook was executed fresh, from top to bottom.
This is what reactive notebooks look like in Deepnote:
Reactive notebooks aide exploratory programming by making iteration loops tighter. When building e.g. charts, updating the code automatically updates the visualizations so there's an instant feedback loop on the changes in the code and data. This truly makes reactive notebooks a powerful tool for exploration.
The ceiling of reactivity
There are situations in which a reactive notebook doesn't shine: e.g. when training a model on large data. Often, training a non-trivial model can take up to a few minutes or even hours, so a reactive notebook wouldn't have enough time to update itself between edits. Once your use case becomes complex and your bottleneck is not the notebook but the actual execution of code, you can simply opt out of reactivity and go back to a conventional notebook.
Reactive notebooks are the future
Everyone who works with a computer is familiar with spreadsheet tools like Excel. This makes it the most popular computational document. It contains cells with data, cells with code, and it's a reactive document – everything is kept up-to-date after edits. Data science notebooks deserve the same level of convenience and ease-of-use. That's why we at Deepnote believe reactive notebooks are the future.
You can give Deepnote reactive notebooks a spin today! We've recently opened up our public beta so sign up and try it for free.