The other day, my good friend TY showed me an article on 'Existential Risk Prevention as Global Priority' upon learning about my new project (which is simply what you are reading). Through that, I was introduced to the (unofficially-named) rationality community, a collection of groups concerned with improving individual rational decision-making skills. What follows is my tentative answer (can it ever be anything else?) to TY's question: "so what do you think?"
The Facts
From what I have gathered, the rationality community is roughly 10,000+ people (estimate based on the number of newsletter subscribers) united around several non-profit or academic organisations:
Machine Intelligence Research Institute (MIRI), founded in 2000
Mission: To ensure that the creation of smarter-than-human intelligence has a positive impact
- Owner of the domain: intelligence.org
- Formerly known as the Singularity Institute
- Concerned with averting existential risk that could result from 'non-friendly' superintelligence upon singularity
- Non-profit receiving important donations from the Thiel Foundation
Center for Applied Rationality (CFAR), founded in 2012
Mission: Using decision science to help yourself and the world
- Owner of the domain: rationality.org
- Spin-off of MIRI
- Lead workshops on:
- How to make accurate predictions
- How to avoid self-deception
- How to get your motivation where your arithmetic says it should be
- ...
- Concerned with eliminating cognitive biases and improving human rationality using recent research in cognitive science
- Not exactly sure how they are funded, but I believe that they are paid for the immersive workshops that they conduct for SF and Boston based companies
Future of Humanity Institute (FHI), founded in 2005
Mission: Look at big-picture questions for human civilization and use the tools of mathematics, philosophy, and science to explore the risks and opportunities that arise from technological change, weigh ethical dilemmas and evaluate global priorities
- Part of the Faculty of Philosophy and the Oxford Martin School at the University of Oxford
- Areas of research
- Global catastrophic risks that pose serious threats to human well-being on a global scale
- Improvement in human ability to form accurate beliefs about the future based on evidence and to make smart decisions based upon those beliefs
- Use of medicine, technology and techniques to improve the capacities of people beyond what would consider normal or healthy
The three organisations are all concerned with the long-term development of humanity, so to speak, and the method that they have embraced for bring on a brighter future for all of us is rationality. On the other hand, they come with slightly different flavours: MIRI is trying to pre-emptively avoid the evolution of a non-human rationality that might kill us all, literally; CFAR has a more bottom-up approach, trying to improve the use of rationality in our day-to-day lives; FHI has a wider scope than MIRI, researching existential risks other than the rise of non-human superintelligence, and does not shy away from possible redefinition of what is 'human'.
The Impressions
Upon a first glance, you might say that the mission of this blog, as defined previously, is not that different from theirs. Yes, I am also very interested in the decision making process, in which our bounded rationality definitely plays a major role, and yes, I am also concerned with 'the future of humanity' and where existing and future technology is taking us. However, the approaches that we have chosen to explore such topics are quite different.
The Rational Community
Rationale
We are about to make some very important decisions. Those decisions have dire consequences. The best way to make better decisions with human optimal consequences is to improve our rationality. We need to improve human rationality by training ourselves to form accurate beliefs based on evidence and to make smart decisions based upon those beliefs.
Actions
Perform hardcore quantitative research. Start a community of like-minded people. Get funding from like-minded people for more research. Publish stuff on the Internet.
Connections
Rationale
We are about to make some very important decisions. Those decisions have dire consequences. I have no idea what consequences are the best, this depends on who you are asking. I intuitively feel that sole focus on any single method is dangerous, therefore I try to rely upon my (bounded) rationality, understanding of different cultures and languages, sense of beauty, and everything else I can muster as a confused human being. I constant absorb and drop new theories, conjectures, assumptions as I judge needed.
Actions
Think and dream really hard about it without systematic plan. Talk to people without systematic plan. Publish stuff on the Internet.
Facing the same realisation (humans, we better shape up, challenging times ahead! ), through different mechanisms, we arrive at the same endpoint, co-existing as part of the great web that is the Internet.
I have to admit that, because of my unrepresentative personal history, I am (most probably irrationally) allergic to any particular group that claims to represent others (read 'me') and make decisions on their (read 'my') behalf. Therefore I was naturally skeptical of their undertakings when I first heard about their ambitious scope (Future of Humanity?! You'd better not be telling me what I should do...). After closer examination, I have to say that they are earnest (they do care about the stuff they do, future of humanity and all, enormously) and worthy of my respect despite of our different valuations of rationality and expectations for it as a tool. One question I still hold is that, judging by the profiles (MIRI, CFAR, and FHI) of the key organizers, the rationality community does not seem to be a very diverse crowd in terms of occupational background: they are mostly educated in STEM disciplines at elite universities in the US and the UK; I wonder what effect that has over the collective paradigm that is being formed within the community? But again, echo chambers abound online these days, they are not more or less exclusive than any other.
So what?
Everything here is just a very long prelude to answer TY's second question: "what do you think about 'Existential Risk Prevention as Global Priority'?", which is forthcoming in a few days.