The current pre-release of the client ("pioneer" in the version) is only compatible to itself. So you can only play with other testers. Please be aware!
For anyone seeing in this in the future -
PLAY a custom game FIRST. It can just be a 1v1 versus an AI. The client does not create all necessary files and directories until you play your first game.
Replays should then work.