Reinforcement learning (RL) has emerged as a fundamental approach to decision-making in machine learning, with applications across a wide range of real-world domains, and several practical extensions, such as multi-agent RL (MARL) and RL from human feedback (RLHF). Despite the growing successful applications of these systems, there exists an inherent threat when it comes to applying them in the real world, where ill-intentioned third parties may intervene in both their training process and deployment. This typically has catastrophic consequences, where even small and inexpensive perturbations to the environment may cause the system to substantially diverge from the desired behavior. The purpose of this thesis is to provide a thorough investigation of various adversarial attacks and robustness against such attacks to (MA)RL and (MA)RLHF systems. In particular, we study training-time and test-time attacks in MARL and propose algorithms that are shown to be provably robust against such attacks. Beyond MARL, we establish fundamental statistical results for RLHF, provide a rigorous characterization of poisoning attacks to RLHF, and develop robust extensions for MARLHF. Our work is centered around robust algorithmic approaches with provable guarantees under common assumptions.

Anomaly detection is one of the fundamental topics in machine learning and artificial intelligence. The aim is to identify instances that deviate from the norm, so-called anomalies. Such anomalies arise in many scenarios, ranging from attacks on computer or energy networks to critical faults in industrial systems or rare tumors in medical imaging data. In this talk, I will first provide an overview of the field of anomaly detection, with an emphasis on approaches based on deep neural networks. I will then present recent algorithms and theoretical advances in deep anomaly detection. In particular, I will discuss why certain methods work and how their behavior can be better understood using principled analysis. Furthermore, I will highlight recent developments in anomaly detection for time series data, including new benchmark datasets and theoretically-grounded evaluation metrics. Finally, I will conclude with an outlook on promising future research directions in anomaly detection.

We build secure software on a seemingly obvious assumption: the processor executes instructions correctly and enforces the boundaries between programs, operating systems, and virtual machines. But what if the CPU itself breaks these rules? Over the past decade, research has found increasingly powerful ways in which processors violate the abstractions on which software relies. What started with tiny timing differences revealing secrets led to attacks such as Spectre and Meltdown, and more recently to CPU bugs that can directly corrupt data, bypass isolation, or expose information across security boundaries.

In this talk, I will show how seemingly harmless implementation details and useful optimizations turn into practical security vulnerabilities. Using examples from our research across Intel, AMD, and RISC-V processors, I will talk about how we discover and understand CPU vulnerabilities and ultimately turn them into security problems. The broader lesson goes beyond processor security: abstractions are only as strong as the layers implementing them. If we want trustworthy computer systems, hardware must increasingly be tested, analyzed, and secured with the same rigor that we already apply to software.