• IceSlider: the agent needs to reach and stop on the pink square. Each action propels him to the closest obstacle in that direction.
• DigitJump: the agent needs to reach the bottom right corner. Each action moves the agent by a number of steps equal to the number at the current position.

These environments are released as supplementary material for the anonymous NeurIPS submission "Planning from Pixels in Environments with Combinatorially Hard Search Spaces".

The environments can be tested interactively in Google COLAB here.

Simply download and unzip the package, then

pip install -e https://github.com/martius-lab/puzzlegen

from puzzlegen import DigitJump, IceSlider

env = DigitJump(seed=42)
env.reset()

for _ in range(20):
  obs, rew, done, info = env.step(env.action_space.sample())
  print(f'Reward: {rew}')