I have my minecraft mouse the opposite so right click to break blocks and left click to place, when i try to attach the trailer it just destroys the vehicle.

Did you right click to put it on. It could also be you crouched and you are just carrying the Vehicle