Our latest TMLR paper on contextual RL discusses how cRL allows principled study on generalization in RL and proposes a flexible benchmark suite for cRL. Check out the paper here and the benchmark here.