NovelGym and evaluation suites for hybrid planning + learning agents.
We design domains and metrics that evaluate novelty handling across open-world scenarios with hybrid planning+RL agents. We develop openAI gym based domains like NovelGym and NovelGridworlds that provide a flexible framework for benchmarking. We also provide clear metrics and protocols for evaluating novelty handling.