What's holding motion controls back?
The obvious answer is that for most games there are simply other options that work better.
There is also the issue that if a control scheme isn't common there is less incentive for developers to take proper use of it but even games that are exclusively made for one system typically fail to take proper use of the motion control, this has been the case for the PS3 and PS4. In the case of the PS4 touchpad the witcher 3 was one of the few titles that used it well and it wasn't even exclusive to the system.
For the typical fps mouse and keyboard is more effective and it's also immersive since the movement of the weapon will correspond to the movement of your hand. One advantage with advanced motion controls (like valve index) is that then you can two guns and two different directions but then you need to come up with some gameplay mechanic that take good use of that.
For the typical racing game a physical
racing wheel is better, it's just a bit expensive.
Motion controls still help the switch a lot since then you can get portability and still enjoy precise and effective controls, in the case of PS4 and xbox one the xim apex probably offer more precision than gyro aiming so and with stationary consoles being able to use mouse and keyboard would probably be far more valuable than gyro aiming support.
Sure you might still prefer gyro aiming but it then only work as a mouse and keyboard replacement when you are unwilling/unable to use a mouse.
It's just difficult to come up with any good gameplay element that you need the additional degrees of freedom high end VR controls offer.