Why do you care about AI Existential Safety?

Progress in AI existential safety would enable us to deploy beneficial AI systems responsibly.

For better or worse, AI technologies impact the causes that you and I care about, from animal welfare to the simple desire not to die in an AI-exacerbated nuclear war or pandemic. Advanced AI systems could improve lives in countless ways: curing diseases, alleviating climate change, and so on. But I can all too easily imagine futures in which AI systems make money for a reckless few at the risk of civilization-scale catastrophes for everyone else. With safety guarantees in place, responsible actors, too, could benefit from advanced systems.

I’m always looking for ways to contribute to the interdisciplinary project of aligning AI to benefit the entire world, in all its complexity and diversity. I believe that it is crucial to draw upon the insights of many types of experts–AI researchers, philosophers, scientists, policymakers, religious thinkers, engineers, and ordinary people–to robustly and holistically mitigate risks.

Please give at least one example of your research interests related to AI existential safety:

As of March 2023, I study inverse reinforcement learning and deontic logic. I’m working toward ensuring that a learned reward function will not incentivize forbidden patterns of behavior, even in adversarially generated scenarios.