Decision Making: how to make better decisions and predictions

December 29, 2019

This is the second of two posts on Decision Making and Team Organisation and Behaviours.

Here's I'm interested in one-off complex decisions like: should we invest in building product X? Or, Is it a good idea to merge team A with team B? So I've been reading evidence based research on the tools and techniques that lead to better predictions and decision making. I found that it is possible to get better, here's how:

General Rules from research literature:

Crucially, every research on individual training that I've seen is clear about two things: (1) people who test higher for cognitive ability and open-mindedness make better predictive judgements, and also (2) everyone can successfully learn and train in the bullet points above to make better decisions and have more accurate forecasts.

Recommendations from Intelligence Analysts

The USA's intelligence community has put a lot of money into research for improving processes for aggregating information and making decisions. I summarise some of the main points they make:

Similar to the academic research. Two different emphasis are the "Devil's Advocate" role and the "Scenario planning". Perhaps because of the high stakes.

In Conclusion:

We can improve the quality of our decision making if we use the techniques and tools above and train ourselves. Think probabilistically, be explicit about alternatives, question assumptions, aggregate information from multiple people and sources.

As I wrote before: it's hard to generalise in this field, but the alternative is to do things that have no evidence. So in my teams and organisations I'll advocate for the points made above.

Refferences:

The US government's Intelligence Advanced Research Projects Activity (IARPA) programmes on decision making: ACE, FUSE, CREATE, ForeST and SHARP.

https://www.cebma.org/wp-content/uploads/Rousseau-2018-Organizational-Dynamics-Evidence-Based-Decisions.pdf

https://www.sas.upenn.edu/~baron/journal/16/16511/jdm16511.pdf

https://www.aaai.org/ocs/index.php/FSS/FSS12/paper/view/5623/5867

http://journal.sjdm.org/19/190925a/jdm190925a.pdf

http://journal.sjdm.org/18/18919/jdm18919.pdf

http://journal.sjdm.org/14/14411/jdm14411.pdf

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.720.6264&rep=rep1&type=pdf

https://www.edge.org/conversation/philip_tetlock-edge-master-class-2015-a-short-course-in-superforecasting-class-i

http://d1c25a6gwz7q5e.cloudfront.net/papers/download/122011_american_psychologist.pdf

https://www.pnas.org/content/pnas/112/50/15343.full.pdf

http://www.erim.eur.nl/fileadmin/erim_content/documents/CorporatePredictionMarkets-Oct1-FirmX.pdf

https://www.apa.org/pubs/journals/releases/xap-0000040.pdf

https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/Tradecraft%20Primer-apr09.pdf

Daniel Kahneman's book "Thinking, Fast and Slow"