These are exercises I have done, as a part of reading Reinforcement Learning, An Introduction, 2nd edition by Andrew Barto and Richard Sutton