Publications and Preprints

2024

ROIL: Robust Offline Imitation Learning, Gersi Doko, Guang Yang, Daniel S. Brown, Marek Petrik, RL Conference, 2024.

Non-adaptive Online Finetuning for Offline Reinforcement Learning, Audrey Huang, Mohammad Ghavamzadeh, Nan Jiang, Marek Petrik, RL Conference, 2024.

A Convex Relaxation Approach to Bayesian Regret Minimization in Offline Bandits, Mohammad Ghavamzadeh, Marek Petrik, Guy Tennenholtz, International Conference on Machine Learning, 2024.

On the convex formulations of robust Markov decision processes, Julien Grand-Clément, Marek Petrik, To appear in Mathematics of Operations Research, 2024.

Beyond discounted returns: Robust Markov decision processes with average and Blackwell optimality, Julien Grand-Clement, Marek Petrik, Nicolas Vielle, Arxiv, 2024.

2023

On Dynamic Programming Decompositions of Static Risk Measures in Markov Decision Processes, Jia Lin Hau, Erick Delage, Mohammad Ghavamzadeh, Marek Petrik, Neural Information Processing Systems (Neurips), 2023.

Percentile Criterion Optimization in Offline Reinforcement Learning, Cyrus Cousins, Elita Lobo, Marek Petrik, Yair Zick, Neural Information Processing Systems (Neurips), 2023.

Reducing Blackwell and Average Optimality to Discounted MDPs via the Blackwell Discount Factor, Julien Grand-Clement, Marek Petrik, Neural Information Processing Systems (Neurips), 2023.

Solving Multi-Model MDPs by Coordinate Ascent and Dynamic Programming, Xihong Su, Marek Petrik, Uncertainty in Artificial Intelligence (UAI), 2023.

Policy Gradient in Robust MDPs with Global Convergence Guarantee, Qiuhao Wang, Chin Pang Ho, Marek Petrik, International Conference on Machine Learning (ICML), 2023.

Entropic Risk Optimization in Discounted MDPs, Jia Lin Hau, Marek Petrik, Mohammad Ghavamzadeh, Artificial Intelligence and Statistics, 2023.

2022

Robust Phi-Divergence MDPs, Chin Pang Ho, Marek Petrik, Wolfram Wiesemann, Neural Information Processing Systems (NeurIPS), 2022.

RASR: Risk-Averse Soft-Robust MDPs with EVaR and Entropic Risk, Jia Lin Hau, Marek Petrik, Mohammad Ghavamzadeh, Reazul Russel, 2022.

Data Poisoning Attacks on Off-Policy Policy Evaluation Methods, Elita Lobo, Harvineet Singh, Marek Petrik, Cynthia Rudin, Himabindu Lakkaraju, Uncertainty in Artificial Intelligence, 2022. Proofs

2021

Fast Algorithms for L-infinity constrained S-rectangular Robust MDPs, Bahram Behzadian, Marek Petrik, Chin Pang Ho, Neural Information Processing Systems (NeurIPS), 2021.

Partial Policy Iteration for L1-Robust Markov Decision Processes, Chin Pang Ho, Marek Petrik, Wolfram Wiesemann, Journal of Machine Learning Research (JMLR), 2021. Preprint

Optimizing Percentile Criterion using Robust MDPs, Bahram Behzadian, Reazul Hasan Russel, Marek Petrik, Chin Pang Ho, Artificial Intelligence and Statistics (AISTATS), 2021.

Robust Behavior Cloning with Adversarial Demonstration Detection, Mostafa Hussein, Brendan Crowe, et al., International Conference on Intelligent Robots and Systems (IROS), 2021.

Policy Gradient Bayesian Robust Optimization for Imitation Learning, Zaynah Javed, Daniel Brown, et al., International Conference on Machine Learning (ICML), 2021.

Spatially optimized distribution of household rainwater harvesting and greywater recycling systems, Shannon Stang, Masoumeh Khalkhalia, Marek Petrik, Michael Palace, Zhongming Lu, Weiwei Mo, Journal of Cleaner Production 312:20, 2021.

Soft-Robust Algorithms for Batch Reinforcement Learning, Elita A. Lobo, Mohammad Ghavamzadeh, Marek Petrik, 2021.

2020

Bayesian Robust Optimization for Imitation Learning, Daniel S. Brown, Scott Niekum, Marek Petrik, Advances in Neural Information Processing Systems (NeurIPS), 2020. Arxiv

MMS SITL Ground Loop: Automating the burst data selection process, Matthew R. Argall, Colin Small, Samantha Piatt, Liam Breen, Marek Petrik, and others, Frontiers in Astronomy and Space Sciences, 2020. Arxiv

Comparison between a Linear Regression and an Artificial Neural Network Model to Detect and Localize Damage in the Powder Mill Bridge, Kathryn Kaspar, Erin Santini-Bell, Marek Petrik, Masoud Sanayei, Transportation Research Record: Journal of the Transportation Research Board, 2020.

Beliefs We Can Believe in: Replacing Assumptions with Data in Real-Time Search, Maximilian Fickert, Tianyi Gu, Leonhard Staut, Wheeler Ruml, Joerg Hoffmann, Marek Petrik, AAAI Conference on Artificial Intelligence (AAAI), 2020.

2019

Beyond Confidence Regions: Tight Bayesian Ambiguity Sets for Robust MDPs, Reazul Hasan Russel, Marek Petrik, Advances in Neural Information Processing Systems (NeurIPS), 2019.

Fast Feature Selection for Linear Value Function Approximation, Bahram Behzadian, Soheil Gharatappeh, Marek Petrik, International Conference on Automated Planning and Scheduling (ICAPS), 2019.

Inverse Reinforcement Learning of Interaction Dynamics from Demonstrations, Mostafa Hussein, Momotaz Begum, and Marek Petrik, International Conference on Robotics and Automation (ICRA), 2019.

Real-time Planning as Decision-making Under Uncertainty, Andrew Mitchell, Wheeler Ruml, Fabian Spaniol, Joerg Hoffmann, Marek Petrik, AAAI Conference on Artificial Intelligence (AAAI), 2019.

2018

Proximal Gradient Temporal Difference Learning: Stable Reinforcement Learning with Polynomial Sample Complexity, Bo Liu, Mohammad Ghavamzadeh, Ian Gemp, Mohammad Ghavamzadeh, Ji Liu, Sridhar Mahadevan, Marek Petrik, Journal of Artificial Intelligence Research (63):462-493, 2018.

Policy Conditioned Uncertainty Sets for Robust Markov Decision Processes, Andrea Tirinzoni, Xiangli Chen, Marek Petrik, Brian Ziebart, Advances in Neural Information Processing Systems (NIPS) 2018.

Fast Bellman Updates for Robust MDPs, Chin Pang Ho, Marek Petrik, Wolfram Wiesemann, International Conference on Machine Learning (ICML), 2018. [Full Paper], [Slides], [Code].

Low-rank Feature Selection for Reinforcement Learning, Bahram Behzadian, Marek Petrik, International Symposium on Artificial Intelligence and Mathematics, 2018.

2017

A Practical Method for Solving Contextual Bandit Problems Using Decision Trees, Adam N. Elmachtoub, Ryan McNellis, Marek Petrik, Sechan Oh, Uncertainty in Artificial Intelligence (UAI), 2017.

Value Directed Exploration in Multi-Armed Bandits with Structured Priors, Bence Cserna, Marek Petrik, Reazul Hasan Russel, Wheeler Ruml, Uncertainty in Artificial Intelligence (UAI), 2017.

Robust Partially-Compressed Least-Squares, Stephen Becker, Ban Kawas, Marek Petrik, AAAI Conference, 2017.

2016

Safe Policy Improvement by Minimizing Robust Baseline Regret, Marek Petrik, Yinlam Chow, Mohammad Ghavamzadeh, Conference on Neural Information Processing Systems (NIPS), 2016.

Interpretable Policies for Dynamic Product Recommendations, Marek Petrik, Ronny Luss, Uncertainty in Artificial Intelligence (UAI), 2016.

2015

Robust Policy Optimization with Baseline Guarantees, Yinlam Chow, Marek Petrik, Mohammad Ghavamzadeh, arXiv:1506.04514.

Robust Partially-Compressed Least-Squares, Stephen Becker, Ban Kawas, Marek Petrik, Karthikeyan N. Ramamurthy, arXiv:1510.04905.

Tight Approximations of Dynamic Risk Measures, Dan Iancu, Marek Petrik, Dharmashankar Subramanian, Mathematics of Operations Research, 40(3), 2015.

Finite-Sample Analysis of Proximal Gradient TD Algorithms, Bo Liu, Ji Liu, Mohammad Ghavamzadeh, Sridhar Mahadevan, Marek Petrik, Uncertainty in Artificial Intelligence (UAI), 2015, (Best Student Paper Award). [Appendix]

Optimal Threshold Control for Energy Arbitrage with Degradable Battery Storage, Marek Petrik, Xiaojian Wu, Uncertainty in Artificial Intelligence (UAI), 2015. [Appendix]

2014

RAAM: The Benefits of Robustness in Approximating Aggregated MDPs in Reinforcement Learning, Marek Petrik, Dharmashankar Subramanian, Conference on Neural Information Processing Systems (NIPS), (spotlight), 2014. [Full Paper].

Efficient and Accurate Methods for Updating Generalized Linear Models with Multiple Feature Additions, Amit Dhurandhar, Marek Petrik, Journal of Machine Learning Research 15:2607-2627, 2014. [bib]

Combining Social Media and Customer Behavior Analytics for Personalized Customer Engagements, Markus Ettl, Prateek Jain, Ronny Luss, Marek Petrik, Rajesh Ravi, Chitra Venkatramani, IBM Journal of Research Development, 58 (5/6) 7:1-7:12, 2014.

2013

Optimizing Deliveries in Agile Supply Chains with Demand Shocks, Francisco Barahona, Markus Ettl, Marek Petrik, Peter Rimshnick, Winter Simulation Conference, 2013.

Solution Methods for Constrained Markov Decision Process with Continuous Probability Modulation, Janusz Marecki, Marek Petrik, Dharmashankar Subramanian, Conference on Uncertainty in Artificial Intelligence (UAI), 2013.

2012

An Approximate Solution Method for Large Risk-Averse Markov Decision Processes, Marek Petrik and Dharmashankar Subramanian. Conference on Uncertainty in Artificial Intelligence (UAI), 2012.

Distributionally Robust Approach to Approximate Dynamic Programming, Marek Petrik, International Conference on Machine Learning (ICML), 2012. Also presented at European Workshop on Reinforcement Learning, 2012. Extended Technical Report (includes proofs).

Optimizing the end-to-end value chain through demand shaping and advanced customer analytics, Brenda Dietrich, Markus Ettl, Roger D. Lederman, Marek Petrik, 11th International Symposium on Process Systems Engineering, 2012.

2011

The Price of Dynamic Inconsistency for Distortion Risk Measures, Pu Huang, Dan Iancu, Marek Petrik, Dharmashankar Subramanian. Technical Report, 2011.

Linear Dynamic Programs for Resource Management, Marek Petrik and Shlomo Zilberstein, Conference on Artificial Intelligence (AAAI) [Computational Sustainability Track], 2011.

Robust Approximate Bilinear Programming for Value Function Approximation, Marek Petrik and Shlomo Zilberstein, Journal of Machine Learning Research 12(Oct):3027-3063, 2011.

2010

Optimization-based Approximate Dynamic Programming, Marek Petrik, Ph.D. Dissertation, 2010. Also, the original double-spaced version, and the defense presentation.

Feature Selection Using Regularization in Approximate Linear Program for Markov Decision Processes, Marek Petrik, Gavin Taylor, Ron Parr, Shlomo Zilberstein. International Conference on Machine Learning (ICML) 27, 2010. Technical Report (includes proofs and algorithms): arXiv 1005.1860.

2009

Robust Value Function Approximation Using Bilinear Programming, Marek Petrik and Shlomo Zilberstein, Conference on Neural Information Processing Systems (NIPS) 22 (spotlight), 2009. Technical Report (includes proofs) UM-CS-2009-052.

A Bilinear Programming Approach for Multiagent Planning, Marek Petrik and Shlomo Zilberstein, Journal of Artificial Intelligence Research 35:235-274, 2009.

Hybrid Least-Squares Algorithms for Approximate Policy Evaluation, Jeff Johns, Marek Petrik, Sridhar Mahadevan, European Conference on Machine Learning, and Machine Learning journal, 2009.

Constraint Relaxation in Approximate Linear Programs, Marek Petrik and Shlomo Zilberstein, International Conference on Machine Learning (ICML), 2009.

Robust Approximate Optimization for Large Scale Planning Problems, Marek Petrik, AAAI Doctoral Consortium, 2009.

Blood Management Using Approximate Linear Programming, Marek Petrik and Shlomo Zilberstein, Presented at INFORMS Computing Society Meeting, Charleston, SC, 2009.

2008

A Successive Approximation Algorithm for Coordination Problems, Marek Petrik and Shlomo Zilberstein, 9th International Symposium on Artificial Intelligence and Mathematics, Fort Lauderdale, Florida, 2008.

Biasing Approximate Dynamic Programming with a Lower Discount Factor,Marek Petrik and Bruno Scherrer, Conference on Neural Information Processing Systems (NIPS), 2008.

Learning Heuristic Functions Through Approximate Linear Programming, Marek Petrik and Shlomo Zilberstein, International Conference on Automated Planning and Scheduling (ICAPS), 2008.

Interaction Structure and Dimensionality Reduction in Decentralized MDPs Martin Allen, Marek Petrik, Shlomo Zilberstein, The National Conference on Artificial Intelligence (AAAI), 2008. Extented technical report #UM-CS-2008-11.

2007

Anytime Coordination Using Separable Bilinear Programs, Marek Petrik, Shlomo Zilberstein, National Conference on Artificial Intelligence (AAAI), 2007.

An Analysis of Laplacian Methods for Value Function Approximation in MDPs, Marek Petrik, International Joint Conference on Artificial Intelligence (IJCAI), 2007.

Average-Reward Decentralized Markov Decision Processes, Marek Petrik, Shlomo Zilberstein, International Joint Conference on Artificial Intelligence (IJCAI), 2007.

2006

Learning Parallel Portfolios of Algorithms, Marek Petrik, Shlomo Zilberstein, Annals of Mathematics and Artificial Intelligence, 48(1-2):85-106, 2006

Learning Static Parallel Portfolios of Algorithms, Marek Petrik, Shlomo Zilberstein, International Symposium on Artificial Intelligence and Mathematics, Fort Lauderdale, 2006.

Learning Parallel Portfolios of Algorithms, Marek Petrik, Diploma Thesis at Univerzita Komenskeho, June 7th 2005. The code, and presentation are also available.

Statistically Optimal Combination of Algorithms, Marek Petrik, SOFSEM, 2005. (Best Student Poster).

Workshops and Reports

Soft-Robust Algorithms for Batch Reinforcement Learning, Elita A. Lobo, Mohammad Ghavamzadeh, Marek Petrik, IJCAI Workshop: R2AW 2021.

Robust Maximum Entropy Behavior Cloning, Mostafa Hussein, Brendan Crowe, Marek Petrik, Momotaz Begum, 3rd Robot Learning Workshop: Grounding Machine Learning Development in the Real World, NeurIPS 2020 workshop, 2020.

Robust Risk-Averse Sequential Decision Making, Jason Carter, Marek Petrik, NeurIPS 2019 Safety and Robustness in Decision Making Workshop 2019

Beyond Confidence Regions: Tight Bayesian Ambiguity Sets for Robust MDPs, Marek Petrik, Reazul Hasan Russell, arXiv:1902.07605, 2019.

Optimizing Norm-bounded Weighted Ambiguity Sets for Robust MDPs Bahram Behzadian, Reazul Hasan Russel, Marek Petrik, NeurIPS 2019 Safety and Robustness in Decision Making Workshop 2019

Robust Exploration with Tight Bayesian Plausibility Sets, Reazul Hasan Russel, Tianyi Gu, Marek Petrik, RLDM 2019

Robust Pest Management Using Reinforcement Learning, Talha Siddique, Jia Lin Hau, Shadi Atallah, Marek Petrik, RLDM 2019

Tight Bayesian Ambiguity Sets for Robust MDPs, Reazul Hasan Russel, Marek Petrik, Infer2Control NIPS Workshop, 2018.

Interpretable Reinforcement Learning with Ensemble Methods, Alexander Brown, Marek Petrik, arXiv:1809.06995.

Feature Selection by Singular Value Decomposition for Reinforcement Learning, Bahram Behzadian, Marek Petrik, Prediction and Generative Modeling Workshop at IJCAI/ICML, 2018.

Policy Conditioned Uncertainty Sets for Robust MDPs, Andrea Tirinzoni, Xiangli Chen, Marek Petrik, Brian Ziebart, Planning and Learning Workshop at IJCAI/ICML 2018.

Computing Robust Strategies for Managing Invasive Plants, Andreas Lydakis, Jenica Allen, Marek Petrik, Tim Szewczyk, AI for Wildlife Conservation Workshop at IJCAI/ICML, 2018.

Building an Interpretable Recommender via Loss-Preserving Transformation, Amit Dhurandhar, Sechan Oh, Marek Petrik, 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016).

Safe Policy Improvement by Minimizing Robust Baseline Regret Marek Petrik, Yinlam Chow, Mohammad Ghavamzadeh, 2016 ICML Workshop on Reliable Machine Learning in the Wild.