I read an article that said fertility is a low heritabile trait. The article said environment, ie groceries were more important.
I know some semen traits can be heritible, from sire to son only. I have one sire group the bulls all had huge scrotals and abnormally high proximal droplets. The daughters of the bull have not passed any defects on to their sons. Maybe this was environmental as the large scrotals might have been prone to being knocked around.
I have a cow that breeds very easilly but has had 2 heifers come in dry. Her bulls look good and test well.
Cattle with low fertility don't get many chances to produce offspring as they get culled. Everyone keeps using the cattle that breed, but we all still wind up with some drys. Makes you wonder.