The rise of artificial intelligence (AI) has enabled transformative new technologies with a wide range of potential applications ranging from cancer diagnostics to natural language processing. Today, these algorithms are typically confined to cloud datacenters, where heavy computing workloads necessitate the use of expensive, power-hungry hardware. Such reliance on cloud infrastructure poses potential privacy and security concerns to the end user, in addition to environmental sustainability concerns, especially as worldwide computing energy consumption rapidly approaches global energy production capacity. To alleviate such concerns, we envision a future hardware ecosystem where the locus of machine intelligence is focused on mobile computing systems where interactions with cloud datacenters are intentionally restricted by the users or system designers. We explore two key technology enablers which could pave a path towards energy-efficient, personalized AI on mobile devices, including high-performance emerging non-volatile memory technologies (e.g., magnetoresistive random access memories) and edge AI algorithms which naturally enable on-device personalization.