L. L. Andrew, A. Wierman, and A. Tang, Power aware speed scaling in processor sharing systems, Proceedings of IEEE INFOCOM, 2009.

S. H. Ahmad, M. Liu, T. Javidi, Q. Zhao, and B. Krishnamachari, Optimality of Myopic Sensing in Multichannel Opportunistic Access, IEEE Transactions on Information Theory, vol.55, issue.9, pp.4040-4050, 2009.
DOI : 10.1109/TIT.2009.2025561

A. Anand and G. De-veciana, A Whittle's index based approach for QoE optimization in wireless networks, Proceedings of ACM SIGMETRICS, 2018.

P. S. Ansell, K. D. Glazebrook, J. Niño-mora, and M. O. Keeffe, Whittle's index policy for a multi-class queueing system with convex holding costs, Mathematical Methods of Operations Research, vol.57, pp.21-39, 2003.

J. Anselmi, Asymptotically optimal open-loop load balancing, Queueing Systems, vol.30, issue.1, pp.245-267, 2017.
DOI : 10.1007/s11134-017-9547-9
URL : https://hal.archives-ouvertes.fr/hal-01614892

A. Asanjarani and Y. Nazarathy, The role of information in system stability with partially observable servers, 1610.

U. Ayesta, M. Erausquin, and P. Jacko, A modeling framework for optimizing the flow-level scheduling with time-varying channels, Performance Evaluation, vol.67, issue.11, pp.1014-1029, 2010.
DOI : 10.1016/j.peva.2010.08.015

U. Ayesta, M. Erausquin, and P. Jacko, Resource-sharing in a single server with time-varying capacity, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2011.
DOI : 10.1109/Allerton.2011.6120192

U. Ayesta, P. Jacko, and V. Novak, A nearly-optimal index rule for scheduling of users with abandonment, 2011 Proceedings IEEE INFOCOM, 2011.
DOI : 10.1109/INFCOM.2011.5935122

M. Bena¨?mbena¨?m and J. Boudec, A class of mean field interaction models for computer and communication systems, Performance Evaluation, vol.65, issue.11-12, pp.823-838, 2008.
DOI : 10.1016/j.peva.2008.03.005

P. Billingsley, Convergence of probability measures, 1968.
DOI : 10.1002/9780470316962

C. Bordenave, D. Mcdonald, and A. Proutiére, A particle system in interaction with a rapidly varying environment: Mean field limits and applications. Networks and heterogeneous media, pp.31-62, 2010.
URL : https://hal.archives-ouvertes.fr/hal-00629339

S. C. Borst, User-level performance of channel-aware scheduling algorithms in wireless data networks, IEEE/ACM Transactions on Networking, vol.13, issue.3, pp.636-647, 2005.
DOI : 10.1109/TNET.2005.850215

S. Bubeck and N. Cesa-bianchi, Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and trends in machine learning, pp.1-122, 2012.

A. Budhiraja, A. Ghosh, and X. Liu, Scheduling control for Markov-modulated single-server multiclass queueing systems in heavy traffic, Queueing Systems, vol.15, issue.1, pp.57-97, 2014.
DOI : 10.1016/0167-6377(94)90009-4

C. Buyukkoc, P. Varaya, and J. Walrand, The cµ rule revisited Advances of Applied Probability, pp.237-238, 1985.

F. Cecchi and P. Jacko, Nearly-optimal scheduling of users with Markovian time-varying transmission rates. Performance Evaluation, pp.99-10016, 2016.

N. Ehsan and M. Liu, On the optimality of an index policy for bandwidth allocation with delayed state observation and differentiated services, IEEE INFOCOM 2004, 2004.
DOI : 10.1109/INFCOM.2004.1354606

N. Gast and B. Gaujal, A mean field approach for optimization in discrete time, Discrete Event Dynamic Systems, vol.27, issue.10, pp.63-101, 2011.
DOI : 10.2307/3214547
URL : https://hal.archives-ouvertes.fr/hal-00788770

J. C. Gittins, K. D. Glazebrook, and R. R. Weber, Multi-Armed Bandit Allocation Indices, 2011.
DOI : 10.1002/9780470980033

K. D. Glazebrook, C. Kirkbride, and J. Ouenniche, Index Policies for the Admission Control and Routing of Impatient Customers to Heterogeneous Service Stations, Operations Research, vol.57, issue.4, pp.975-989, 2009.
DOI : 10.1287/opre.1080.0632

K. D. Glazebrook and H. M. Mitchell, An index policy for a stochastic scheduling model with improving/deteriorating jobs, Naval Research Logistics, vol.25, issue.7, pp.706-721, 2002.
DOI : 10.1017/S0021900200040420

D. J. Hodge and K. D. Glazebrook, On the asymptotic optimality of greedy index heuristics for multi-action restless bandits, Advances in Applied Probability, vol.23, issue.03, pp.652-667, 2015.
DOI : 10.1017/S0021900200014030

M. Larrañaga, U. Ayesta, and I. M. Verloop, Index policies for multi-class queues with convex holding cost and abandonments, Proceedings of ACM SIGMETRICS, 2014.

M. Larrañaga, I. M. Ayesta, and . Verloop, Dynamic Control of Birth-and-Death Restless Bandits: Application to Resource-Allocation Problems, IEEE/ACM Transactions on Networking, vol.24, issue.6, pp.3812-3825, 2016.
DOI : 10.1109/TNET.2016.2562564

K. Liu and Q. Zhao, Indexability of Restless Bandit Problems and Optimality of Whittle Index for Dynamic Multichannel Access, IEEE Transactions on Information Theory, vol.56, issue.11, pp.5547-5567, 2010.
DOI : 10.1109/TIT.2010.2068950

A. Mahajan and D. Teneketzis, Multi-Armed Bandit Problems, Foundations and Application of Sensor Management, pp.121-308, 2007.
DOI : 10.1007/978-0-387-49819-5_6
URL : http://www.eecs.umich.edu/~adityam/publications/books/MAB-Chapter.pdf

Y. Nazarathy, T. Taimre, A. Asanjarani, J. Kuhn, B. Patch et al., The challenge of stabilizing control for queueing systems with unobservable server states, IEEE Proceedings of the 5th Australian Control Conference, 2015.

J. Niño-mora, Dynamic priority allocation via restless bandit marginal productivity indices, TOP, vol.14, issue.2, pp.161-198, 2007.
DOI : 10.1109/NGI.2007.371218

J. Niño-mora, Marginal Productivity Index Policies for Admission Control and Routing to Parallel Multi-server Loss Queues with Reneging, Lecture Notes in Computer Science, vol.4465, pp.138-149, 2007.
DOI : 10.1007/978-3-540-72709-5_15

J. R. Norris, Markov chains, volume 2 of Cambridge Series in Statistical and Probabilistic Mathematics, 1998.

W. Ouyang, A. Eryilmaz, and N. B. Shroff, Asymptotically optimal downlink scheduling over Markovian fading channels, 2012 Proceedings IEEE INFOCOM, 2012.
DOI : 10.1109/INFCOM.2012.6195483

M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, 1994.
DOI : 10.1002/9780470316887

V. Raghunathan, V. Borkar, M. Cao, and P. R. Kumar, Index Policies for Real-Time Multicast Scheduling for Wireless Broadcast Systems, IEEE INFOCOM 2008, The 27th Conference on Computer Communications, 2008.
DOI : 10.1109/INFOCOM.2008.217

A. Slivkins and E. Upfal, Adapting to a changing environment: The Brownian restless bandits, Proceedings of 21st Annual Conference on Learning Theory, pp.343-354, 2008.

I. M. Verloop, Asymptotic optimal control of multi-class restless bandits, Annals of Applied Probability, vol.26, issue.4, 1947.

R. R. Weber and G. Weiss, On an index policy for restless bandits, Journal of Applied Probability, vol.25, issue.03, pp.637-648, 1990.
DOI : 10.2307/3214547

P. Whittle, Restless bandits: activity allocation in a changing world, Journal of Applied Probability, vol.1, issue.A, pp.287-298, 1988.
DOI : 10.1214/aop/1176994469

P. Whittle, Optimal Control, Basics and Beyond, 1996.