I wasn't aware that C++ was particularly slow to compile, although I could be wrong.
It may be that Cocos2d-x contains a lot more code than the iOS only version due to being cross platform. I can imagine that the code base is larger due to a lot of platform specific code, and because there are many things that the project includes that would be part of the iOS sdk in the Obj-C version.
I think the Swift compiler is a little on the slow side at the moment. I've heard other people complain about it a bit, but I think it's because the language isn't mature enough to be at a point where the compiler is being optimised for speed.
Personally I haven't really had any issues, seems fast enough to me, even when I'm on the 12inch macbook. Although I'm not working on anything with large dependancies at the moment.
Either way, Swift is the future. Obj-C is going away.