Working with colab

There’s a cool free google account feature that tends to go somewhat under the radar. It’s called Google Colab and allows you to author and execute Jupyter notebooks entirely online. I use them extensively for a botanical survey workflow for Glorious Grasslands. The data is captured using Epicollect 5 and downloaded in their standard CSV format. There’s an extensive set of scripts that take the raw data, perform all kinds of validation and the output spreadsheets in the format needed by the ecologists. There are two major shortcomings with Jupyter in general and Colab in particular which make is very tricky to control development: Jupyter notebooks store code and text and output in a single JSON file. This means every time you execute it the file has changed, regardless of whether you have changed any core code. The jupyter notebooks are stored in google drive which only has limited per-file versioning. When you execute a notebook, it creates a virtual python machine with the google…