Visible Thoughts Project and Bounty Announcement - Machine Intelligence Research Institute

We at MIRI are soliciting help with an AI-alignment project centered around building a dataset, described below. We have $200,000 in prizes for building the first fragments of the dataset, plus an additional $1M prize/budget for anyone who demonstrates the ability to build a larger dataset at scale. If this project goes well, then it may be the first of a series of prizes we offer for various projects. Below, I'll say more about the project, and about the payouts and interim support we're offering. Hypothesis: Language models can be made more understandable (and perhaps also more capable, though this is not the goal) by training them to produce visible thoughts.