Note: Since posting this I’ve turned it into the Heroku deployment guide in the official fast.ai course notes, so you really only need to read one or the other.

So you’ve created a deep learning model and it can understand the difference between different breeds of dog or different Hemsworth brothers. Awesome.

Now what? There are various options for deploying some kind of working demo.

Some people want to put a lot of effort into very beautifully presented standards compliant ajax enabled websites and mobile apps. That’s fine, I’m not knocking it, but life is short and personally I just want somewhere I can upload my notebook so I can go “LOOK I MADE A THING AND IT’S REALLY COOL!!”

Heroku combined with Voila is perfect for me. Voila renders Jupyter notebooks as web pages and Heroku is a lovely free service which lets me just throw my notebooks at it. Heroku also scales as you grow so you can start free and work your way up if you want to host something serious on it. (You can also use Heroku for proper websites if you prefer, google away).

If you take a look at the repo for my demo site on Heroku, you’ll see:

  • Procfile – what to do when the heroku application starts
  • requirements.txt – modules you need
  • Some .ipynb notebooks

Procfile – mine looks like this:

web: voila –port=$PORT –no-browser –enable_nbextensions=True

If you add notebook.ipynb to the end of that string then Voila will display that notebook when the app starts but you can only display that notebook and if you try to link to any others you’ll get an http error. I want to use this for multiple demos so I don’t specify a notebook, but if you go to the root of the app at joedockrill.herokuapp.com then you’ll be greeted with a rather unattractive list of files.

Hopefully you can tell from the rather unattractive list of files where I’m headed with this. I use default.ipynb as a homepage to list all the demos on the website; It just means having to explicitly link to joedockrill.herokuapp.com/voila/render/default.ipynb which isn’t the biggest pain in the world.

requirements.txt is for all the modules you need. do not do pip installs in your notebook. Heroku builds an image with everything you need once when you deploy, then just copies it onto a server when someone runs the app.

Once you have your repo in place you just need to

  • Create an account on Heroku
  • Connect it to your github
  • Choose the repo
  • Choose between automatic deploys when the repo changes or manual deploys when you press the button

Two things to keep in mind. First, there is a maximum compiled “slug size” for your app image and it’s 500MB. If you intend to deploy multiple demos with large model files then keep the pickles on Google Drive or something similar and load them from there. You also need to make sure that you use the CPU versions of Pytorch because the GPU ones are massive by comparison. (See my requirements.txt file).

The other issue (related to the first) is that Voila runs all the code in your notebook before it renders that part of the UI. That’s an issue when you have to load a model export across the intertubes. You’ll see in the clown classifier that the markdown cell at the top displays right away so I waffle and talk about the demo and make excuses and hope that it’s rendered the rest of the UI by the time you finish reading.

I haven’t yet found a way around this and believe me I’ve tried. I even tried downloading the model from drive on a worker thread so my foreground would complete and render the UI before the download started. It works fine in Colab but Voila didn’t like it, and somehow still managed to wait until the background thread had completed before rendering anything. It was like trying to make VB5 behave itself with multi-threading APIs. I gave up.

Don’t get me wrong though, Voila and Heroku are both lovely and I highly recommend giving them a go. If there's a better option for a quick demo project, I haven't found it yet.