Systematic Climate Company Ratings
I’ve been interested in working for a climate company awhile. The last time I spent concerted effort trying to find a good match was for a few months in the spring of 2021. I learned quite a bit and documented it all in Workflowy. Among other things, this involved checkins with 7 people weekly for 6 weeks, calls with several people at companies I admired, and lots of reading/listening.
But I felt like I failed by the end of it, because I hadn’t found a good match.
I’m toying around with giving this another try, 18 months later. But this time I had a new cute idea on how I might go about doing this: implement a rating system. This idea was inspired by Daniel Kahneman where he studies how effective simple rating systems can be. I found a good summary of this in this random blog post.
Here’s the relevant part:
“Several studies have shown that human decision makers are inferior to a prediction formula even when they are given the score suggested by the formula! They feel that they can overrule the formula because they have additional information about the case, but they are wrong more often than not. According to Meehl, there are few circumstances under which it is a good idea to substitute judgment for a formula.”
(Kahneman, Daniel. Thinking, Fast and Slow (p. 224). Macmillan. Kindle Edition.)
Kahneman then explicates the remarkable contribution by Robin Dawes who showed that scores derived from an equal weighting of factors outperformed regression-based formulas in many prediction applications. (“The Robust Beauty of Improper Linear Models in Decision Making” American Psychologist, July 1979, Vol. 34, No. 7,571-582, http://www.niaoren.info/pdf/Beauty/9.pdf )
“The surprising success of equal-weighting schemes has an important practical implication: it is possible to develop useful algorithms without any prior statistical research. Simple equally weighted formulas based on existing statistics or on common sense are often very good predictors of significant outcomes.”
(Kahneman, Daniel. Thinking, Fast and Slow (p. 226). Macmillan. Kindle Edition.)
Kahneman illustrates this point with the story of Virginia Apgar, who used her expert knowledge to draft and refine the Apgar score that enables a physician or nurse to rate the health of newborn babies. Low scores call for intervention by the care team.
An algorithm for decision-making is even more useful when the decision-maker faces psychological challenges. Stress, for example, appears to reduce at least some aspects of cognitive function. (See for example C.S. Mackenzie et al. (2007), “Cognitive Functioning Under Stress: Evidence from Informal Caregivers of Palliative Patients”, Journal of Palliative Medicine, 10(3): 749–758. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1905830/ )
I have a spreadsheet of over 100 climate companies, and I want to get a sense of which companies to call/learn more about. Or even to decide to work for. I previously would hope that there was some obvious excitement that might jump out at me. This worked pretty well in the past for the tech companies I’ve worked for, but I’ve got a sense that the strategy works best when many of the options are generally good matches.
But for these climate companies, I think the vast majority of them aren’t obviously good fits. So far, the way that I’ve handled this is tried to open my mind up to the possibility of working at companies even if they don’t excite me. It’s quite possible I’m being overly critical of the options. But I think the biggest downside to this strategy is that I would agonize too much. I’d see a new company, notice my gut instinct that it wasn’t a good match, and then try to contradict those feelings. Was I sure it was a bad fit? My gut was sure, but my logical self was not. And this would soon result in existential questions. Did I really care about climate work? Or did I just want to say that I did?
The net result of this process was two fold:
- Slow progress
The GIDE rating system
In coming back to evaluating climate companies, I have a new idea to try out: use some quick formula like the Apgar score where I rate companies. This way, I can quickly compare a larger swath of companies without actually making any tough decisions about them (like if I should reach out).
Here’s the factors I settled on:
- Does the company have the capacity to prevent or remove large amounts of Greenhouse Gasses in the atmosphere?
- Do I feel inspired by this company?
- e.g. they have cool team members
- e.g. their idea is really innovative/unique
- e.g. they have a cool story
- e.g. their work seems impressive
- Would I enjoy my day to day work?
- Will I get to work on challenging problems?
- Will I have the opportunity to be highly collaborative (vs working individually)?
- Will there be opportunities for growth and/or learning?
- Will I be involved in directly building things (vs say, sending emails to politicians)?
- fwiw I previously followed a work kit that illuminated these factors as important to me, at least with respect to my enjoyment of day to day work.
- Do I think this company will be effective in achieving its goals?
- Typically this means established companies with a proven track record score well here, while projects that are still in the research phase won’t.
These are of course highly tuned to my own preferences. And I also don’t necessarily need everything to be met. In fact, the initial goal is simply to compare companies, not to decide on if something is a sufficiently good fit.
To speed up the rating process, I need a quick way to remember these factors. I picked the acronym GIDE: GHGs, Inspired, Day to day work, Effectiveness. For each factor, I rate it out of 5.
Before digging into some examples, I want to explicitly point out their limitations:
- They’re highly personalized to me.
- They’re quick, gut guesses.
- I don’t explain all the details of how I came to that guess.
The reason I write these examples down is simply for illustrative puposes, and not to say anything meaningful outside of my own head. You might even notice that the company that’s ostensibly far more successful (way more funding, more established team, has actually made an impact, etc, etc) has a lower rating than the other. This definitely doesn’t mean that I think it’s less worthy of a company to exist, or to work for. But it does illuminate something surprising: the underdog company has more positives than I think I would have otherwise realized.
https://dandelionenergy.com/ — GIDE=4,4,3,2=13/20
- GHGs: Moving households from propane to electric heating overall has a large potential effect, and it’s direct. That is, every house that gets converted has a measurable amount of GHGs that are reduced. I rate this 4/5.
- Inspired: Drilling big holes is cool. Geology, building hardware is cool. 4/5
- Day to day work: After talking to someone there, I don’t forsee the technical problems being particularly challenging. It seems more operation-heavy software, which involves shipping lots of small, effective, changes quickly. I think I’ll have some interesting opportunities to learn about regulations, contractor businesses, govt contracts, etc. In general, I don’t know much about this field, while it seems the the company has lots of deep expertises.
- Effectiveness: 2/5. This one was interesting. The size of the company is large, which would often mean good things. But from what I can glean, the company has had and will continue to have many significant hurdles. I won’t go into detail, but my confidence that they’ll actually succeed is unfortunately low.
https://ambri.com/ — GIDE=5,5,4?,1 = 15/20
- GHGs: battery storage is a huge and expensive problem for transitioning the grid away from fossil fuels. 5/5
- Inspired: chemistry is cool (I know little about). Big shipping-container-sized batteries probably have all manner of cool things about them. 5/5
- Day to day work: I’m not sure. I bet since the company is small there’d be lots of things I could contribute to that I’d feel proud of. The work is probably not very challenging, but I could imagine I might learn some cool things about chemistry and working with past researchers. 4?/5 (the notation here means a 4, but with higher than usual variance)
- Effectiveness: 1/5 :(. This company has been running for 13 years, and still hasn’t shipped anything. They say they’re now planning on 2023. I have some weak guess 4 years ago they were also saying things were right around the corner. Failing to ship something for that long is really worrying. I have never heard of a company that’s existed that long without shipping that turned out to be successful. Please enlighten me with counter examples :).
I’ll see if I can talk to someone at Ambri to double check my suspicions. If they flip my effectiveness rating, this could be really compelling company to work for.
I have one guess about the outcome of this exercise: I’ll find that every company will rank quite low in one of these metrics. And I may ultimately feel that the climate industry isn’t the right fit for me. In a sense, this rating system may just give me an excuse to not work on climate. I’ll have a cute list of dozens of companies, where I’ve diligently rated them one by one, and found that none of them are a good fit. And then I’ll conversely rate traditional tech companies and find that my ratings will be higher. And with these pairs of ratings, will concede that the most effective thing I can do is donate money, instead of working directly in the field. In some sense, this is a “risk” involved in this exercise. But I’m generally open minded enough that I only expect this will be my conclusion if it has a significant amount of truth to it.