Core Components of Reinforcement Learning

Environment

Can be thought of as the “world” that the context is taking place within.

Agent

The Agent is the entity that is acting upon the environment. The Agent can take actions based on decisions, which influence the state that the environment is in.

Action Space

The actions in which are of disposal to the agent. If the agent can only move in a 2-directional plane, than the action space would be {Left, Right, Down, Up}. The agent can choose actions based on the state of the environment.

State Space

All possible states that an environment can take shape in. For example in a 2x2 box, in which the agent can only be in one box, the state space is the 2x2 box.

State

A individual state that the environment can be in. Going back to the example in a 2x2 box, any one of the quadrants is a state that the agent can be in.

Probabilistic Transitions

$$ p(s', r | a,s), \\ \sum_{s',r} p(s', r | a,s) = 1 $$

p: probability, s': next state, r: reward, a: action taken, s: current state

In Plain English:

• The probability of s’ and getting reward r given the agent currently being in state s and taking action a.
• The Sum of all possible states and rewards given you take action a in state s is one. This ensures that all possible states are considered.