Why do you care about AI Existential Safety?

I have for a long time believed that the alignment problem would pose an existential risk if AGI was developed, but have been sceptical of non-trivial probabilities of that occuring. However, recently I have significantly increased my credence in short-term timelines due to AGI. This is partly because of the recent progress in LLMs. More significantly, it is because I have in my research found (to me) surprising overlap between the computational models we can use to explain human cognition, and those used to design artificial agents. This makes me think that we are approximating general principles of intelligence, and that AGI is realistic. Consequently, I want to do what I can to contribute to the mitigation of the alignment problem.

Please give at least one example of your research interests related to AI existential safety:

I am interested in understanding the structure of human values from an empirical standpoint so that an AGI could use this structure to generatively predict our preferences, even when we have failed to specify them, or have specified them incorrectly. A part of my current research project is to model this structure using reinforcement learning models. If that would work, then I believe it could provide a solution to a significant component of the alignment problem.