I think the idea has merit, but it also means that you can have 3K ladder rating because you are really good at 5x5 maps, but if you take it on with someone on a 20x20 map you maybe couldn't beat a 1k ladder player that is really good at those (but don't get to play that often - apparently)
In a similar situation: custom games have true skill rating too. Someone can be 2K at setons, but if they play on any other map then they'd be 1.4K or lower instead. But they still have 2k rating because they only play setons. This entails that because of having the choice on what you want to play, the rating doesn't actually mean that much anymore and you need to 'know the person'. This is why there are game names with 'unless you're a known setoner' - or similar for any other common map.
That is why ladder is ladder - you take a random distribution of the pool and players (that are roughly equal) and by doing so it averages out: therefore the true skill rating means a lot more. Introducing this concept would negate that, which you acknowledge in your post at the end .