S. Duran and I. M. Verloop, Asymptotic optimal control of Markov-modulated restless bandits, Proceedings of ACM SIGMETRICS, vol.2, pp.1-25, 2018.
URL : https://hal.archives-ouvertes.fr/hal-01696329

S. Duran, U. Ayesta, and I. M. Verloop, On the Whittle's Index of Markov-modulated restless bandits, 2020.

S. Duran and P. Robert, Analysis of a stochastic model of replication with linear breakdown rates, 2020.

. Bibliography,

S. Aalto, P. Lassila, and P. Osti, Whittle index approach to size-aware scheduling with time-varying channels, Proceedings of the 2015 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, pp.57-69, 2015.

S. H. Ahmad, M. Liu, T. Javidi, Q. Zhao, and B. Krishnamachari, Optimality of myopic sensing in multichannel opportunistic access, IEEE Transactions on Information Theory, vol.55, pp.4040-4050, 2009.

E. Altman, K. E. Avrachenkov, and R. Núnez-queija, Perturbation analysis for denumerable Markov chains with application to queueing models, Advances in Applied Probability, vol.36, issue.3, p.53, 2004.
URL : https://hal.archives-ouvertes.fr/inria-00071873

A. Anand and G. De-veciana, A Whittle's index based approach for QoE optimization in wireless networks, Proceedings of ACM SIGMETRICS, 2018.

P. S. Ansell, K. D. Glazebrook, J. Niño-mora, and M. O'keeffe, Whittle's index policy for a multi-class queueing system with convex holding costs, Mathematical Methods of Operations Research, vol.57, pp.21-39, 2003.

J. Anselmi, Asymptotically optimal open-loop load balancing, Queueing Systems, vol.87, pp.245-267, 2017.
URL : https://hal.archives-ouvertes.fr/hal-01614892

A. Asanjarani and Y. Nazarathy, The role of information in system stability with partially observable servers, 2016.

K. J. Åström, Optimal control of Markov processes with incomplete state information, In: Journal of Mathematical Analysis and Applications, vol.10, p.105, 1965.

U. Ayesta, M. Erausquin, and P. Jacko, A Modeling Framework for Optimizing the Flow-Level Scheduling with Time-Varying Channels, Performance Evaluation, vol.67, pp.1014-1029, 2010.

U. Ayesta, P. Jacko, and V. Novak, A Nearly-Optimal Index Rule for Scheduling of Users with Abandonment, Proceedings of IEEE INFOCOM. Hong Kong, vol.8, 2011.

R. Bellman, A Markovian decision process, Journal of mathematics and mechanics, pp.679-684, 1957.

M. Benaïm and J. Boudec, A class of mean field interaction models for computer and communication systems, Performance Evaluation, vol.65, pp.823-838, 2008.

D. P. Bertsekas, Dynamic programming and optimal control, vol.1, 1995.

S. Bhulai, A. C. Brooms, and F. M. Spieksma, On structural properties of the value function for an unbounded jump Markov process with an application to a processor sharing retrial queue, Queueing Systems, vol.76, pp.425-446, 2014.

P. Billingsley, Convergence of probability measures, vol.31, p.17, 1968.

A. Bobbio, M. Gribaudo, and M. Telek, Analysis of large scale interacting systems by mean field method, 2008 Fifth International Conference on Quantitative Evaluation of Systems, vol.10, p.9, 2008.

C. Bordenave, D. Mcdonald, and A. Proutiére, A particle system in interaction with a rapidly varying environment: Mean field limits and applications, Networks and heterogeneous media, vol.5, p.29, 2010.
URL : https://hal.archives-ouvertes.fr/hal-00629339

D. Borthakur, HDFS architecture guide, Hadoop Apache Project, vol.53, pp.1-13, 2008.

R. J. Boucherie and N. M. Van-dijk, Queueing networks: a fundamental approach, vol.154, 2010.

P. H. Brill, Level crossing methods in stochastic models, vol.13, p.47, 2008.

A. Budhiraja, A. Ghosh, and X. Liu, Scheduling control for Markov-modulated single-server multiclass queueing systems in heavy traffic, Queueing Systems, vol.78, pp.57-97, 2014.

C. Buyukkoc, P. Varaiya, and J. Walrand, The cµ rule revisited, Advances in applied probability, vol.17, pp.237-238, 1985.

E. Çinlar, Introduction to Stochastic Processes, p.32, 1975.

B. D. Coleman, A stochastic process model for mechanical breakdown, Transactions of the Society of Rheology, vol.1, p.86, 1957.

J. G. Dai and S. He, Many-server queues with customer abandonment: A survey of diffusion and fluid approximations, Journal of Systems Science and Systems Engineering, vol.21, p.38, 2012.

N. M. Van-dijk, Approximate uniformization for continuous-time Markov chains with an application to performability analysis, Stochastic processes and their applications, vol.40, p.69, 1992.

N. Ehsan and M. Liu, On the optimality of an index policy for bandwidth allocation with delayed state observation and differentiated services, Proceedings of IEEE INFOCOM. Hong Kong, vol.8, 2004.

J. Fu, B. Moran, and P. G. Taylor, Restless bandits in action: Resource allocation, competition and reservation, p.105, 2018.

J. Fu, Y. Nazarathy, S. Moka, and P. G. Taylor, Towards Q-learning the Whittle Index for Restless Bandits, 2019 Australian & New Zealand Control Conference (ANZCC), p.106, 2019.

N. Gast and B. Gaujal, A mean field approach for optimization in discrete time, Discrete Event Dynamic Systems, vol.21, p.3, 2011.
URL : https://hal.archives-ouvertes.fr/hal-00788770

J. M. George and J. M. Harrison, Dynamic control of a queue with adjustable service rate, Operations research, vol.49, issue.2, pp.720-731, 2001.

S. Ghemawat, H. Gobioff, and S. Leung, The google file system, Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, 2003.

J. Gittins, K. Glazebrook, and R. Weber, Multi-Armed Bandit Allocation Indices, p.39, 1989.

J. C. Gittins, Bandit processes and dynamic allocation indices, Journal of the Royal Statistical Society: Series B, vol.41, p.7, 1979.

K. D. Glazebrook, C. Kirkbride, and J. Ouenniche, Index policies for the admission control and routing of impatient customers to heterogeneous service stations, Operations Research, vol.57, pp.975-989, 2009.

K. D. Glazebrook and H. M. Mitchell, An index policy for a stochastic scheduling model with improving/ deteriorating jobs, Naval Research Logistics, vol.49, pp.706-721, 2002.

, Special Issue on Queueing Systems with Abandonments. 2013 (cit, p.38

O. Hernández-lerma and J. B. Lasserre, Discrete-time Markov control processes: basic optimality criteria, vol.30, 2012.

D. J. Hodge and K. D. Glazebrook, On the asymptotic optimality of greedy index heuristics for multi-action restless bandits, Advances in Applied Probability, vol.47, p.22, 2015.

B. Ji, G. G. Gupta, M. Sharma, X. Lin, and N. B. Shroff, Achieving optimal throughput and near-optimal asymptotic delay performance in multichannel wireless networks with low complexity: a practical greedy scheduling policy, IEEE/ACM Transactions on Networking, vol.23, pp.880-893, 2014.

A. Karthik, A. Mukhopadhyay, and R. R. Mazumdar, Choosing among heterogeneous server clouds, Queueing Systems, vol.85, p.86, 2017.

A. Lakshman and P. Malik, Cassandra: a decentralized structured storage system, In: ACM SIGOPS Operating Systems Review, vol.44, pp.35-40, 2010.

M. Larrañaga, U. Ayesta, and I. M. Verloop, Index Policies for multi-class queues with convex holding cost and abandonments, Proceedings of ACM SIGMETRICS, 2014.

M. Larrañaga, U. Ayesta, and I. M. Verloop, Asymptotically optimal index policies for an abandonment queue with convex holding cost, Queueing Systems, vol.81, pp.99-169, 2015.

M. Larrañaga, U. Ayesta, and I. M. Verloop, Dynamic control of birth-and-death restless bandits: application to resource-allocation problems, IEEE/ACM Transactions on Networking, vol.24, pp.3812-3825, 2016.

J. Boudec, D. Mcdonald, and J. Mundinger, A generic mean field convergence result for systems of interacting objects, Fourth international conference on the quantitative evaluation of systems, vol.10, p.9, 2007.

K. Liu and Q. Zhao, Indexability of restless bandit problems and optimality of Whittle index for dynamic multichannel access, IEEE Transactions on Information Theory, vol.56, pp.5547-5567, 2010.

S. T. Maguluri, R. Srikant, and L. Ying, Stochastic models of load balancing and scheduling in cloud computing clusters, 2012 Proceedings IEEE Infocom, p.86, 2012.

A. Mahajan and D. Teneketzis, Multi-Armed Bandit Problems". In: Foundations and Application of Sensor Management, p.39, 2007.

Y. Nazarathy, T. Taimre, A. Asanjarani, J. Kuhn, B. Patch et al., The challenge of stabilizing control for queueing systems with unobservable server states, IEEE Proceedings of the 5th Australian Control Conference, 2015.

J. Niño-mora, Dynamic priority allocation via restless bandit marginal productivity indices, vol.8, pp.161-198, 2007.

J. Niño-mora, Marginal productivity index policies for admission control and routing to parallel multi-server loss queues with reneging, Lecture Notes in Computer Science, vol.4465, pp.138-149, 2007.

J. Niño-mora and S. S. Villar, Sensor scheduling for hunting elusive hiding targets via Whittle's restless bandit index policy, International Conference on NETwork Games, Control and Optimization, 2011.

. Ieee, , p.9, 2011.

J. R. Norris, Markov chains, Cambridge Series in Statistical and Probabilistic Mathematics. Reprint of 1997 original, vol.2, p.237, 1998.

W. Ouyang, A. Eryilmaz, and N. B. Shroff, Asymptotically optimal downlink scheduling over Markovian fading channels, 2012 Proceedings IEEE INFOCOM, pp.1224-1232, 2012.

M. T. Özsu and P. Valduriez, Principles of distributed database systems, vol.85, p.4, 2011.

M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, vol.67, p.46, 1994.

V. Raghunathan, V. Borkar, M. Cao, and P. R. Kumar, Index policies for real-time multicast scheduling for wireless broadcast systems, Proceedings of IEEE INFOCOM, vol.8, 2008.

A. W. Richa, M. Mitzenmacher, and R. Sitaraman, The power of two random choices: A survey of techniques and results, Combinatorial Optimization, vol.9, p.86, 2001.

S. M. Ross, Introduction to stochastic dynamic programming, 2014.

Z. Schechner, A load-sharing model: The linear breakdown rule, Naval research logistics quarterly, vol.31, p.86, 1984.

A. L. Stolyar, Maxweight scheduling in a generalized switch: State space collapse and workload minimization in heavy traffic, The Annals of Applied Probability, vol.14, p.52, 2004.

W. Sun, V. Simon, S. Monnet, P. Robert, and P. Sens, Analysis of a stochastic model of replication in large distributed storage systems: A mean-field approach, Proceedings of the ACM on Measurement and Analysis of Computing Systems, vol.1, p.24, 2017.
URL : https://hal.archives-ouvertes.fr/hal-01494235

W. Sun and P. Robert, Analysis of large urn models with local mean-field interactions, In: Electronic Journal of Probability, vol.24, p.86, 2019.
URL : https://hal.archives-ouvertes.fr/hal-01710964

R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction, vol.106, p.14, 2018.

A. Sznitman, Topics in propagation of chaos, Ecole d'été de probabilités de Saint-Flour XIX-1989

A. S. Tanenbaum and M. Van-steen, Distributed systems: principles and paradigms, 2007.

H. C. Tijms, Stochastic modelling and analysis: a computational approach, vol.88, p.68, 1986.

I. M. Verloop, Asymptotically optimal priority policies for indexable and nonindexable restless bandits, The Annals of Applied Probability, vol.26, pp.1947-1995, 2016.
URL : https://hal.archives-ouvertes.fr/hal-00743781

J. and V. Neumann, Probabilistic logics and the synthesis of reliable organisms from unreliable components, Automata studies, vol.34, p.85, 1956.

R. R. Weber and G. Weiss, On an index policy for restless bandits, Journal of Applied Probability, vol.27, p.45, 1990.

P. Whittle, Restless bandits: Activity allocation in a changing world, Journal of applied probability, vol.25, pp.287-298, 1988.

P. Whittle, Optimal Control, Basics and Beyond, p.39, 1996.