Lessons Learned from Building an AI Writing App [Guide]

It's (mostly) powered by OpenAI's GPT-2 and has additional fine-tuned models: The main technical challenges were creating an app that could deliver OpenAI's GPT-2 Medium (a ml model that generates text) quickly and simultaneously support 10-20 heavy users. Above steps had to be fully automated or scaling failed. It feels dirty (in 2019) to write bash scripts as part of an automated deployment, but it's necessary when using autoscaling on Google startup-scripts. Kubernetes is another option, but I'm not smart enough to use K8. Google startup-scripts runs a shell script when the machine starts.