I can jump in a bit and add to the confusion.
The HE EPD are calculated using member data from Canada, the US and now South America. Animal records are adjusted for things such as age at weaning, age of dam, twin, etc. An animal's individual performance records are only ever directly compared to animals in the same contemporary group (same herd, sex, management from birth, etc).
The evaluation then looks at differences (example, a calf that weans 100 pounds heavier than group average) and the proportion of that difference that can be explained by genetics (heritability). The difference is now strictly a genetic difference between animals in the contemporary group. The system also uses related information to help improve the prediction on traits of interest. For example, we know some of the genes that make bigger weaning weights, can also result in increased birth weight, so that relationship is used in calculating the EPD.
Because the comparison is now on a genetic level (through the use of contemporary groups) the animals can now be connected using the pedigrees as a giant spider web connecting herds and cattle over time.
I will have to check, but I believe the Hereford base is established so that the animals that come out as the foundation of the pedigree tree average around zero and then genetic trend (genetic change) takes care of where we are at today in terms of breed average.
As we gather more information, the accuracy increases. This could include progeny records, grandprogeny, added weaning and yearling weights, or even gene markers moving forward.
An EPD on a yearling bull is no better or worse than his actual weight, but an EPD is better for creating genetic change (we can argue about progress) because it has most of the environmental influences removed and shows more of the true genetic component. In other words we are more likely to pick cattle that move in the direction we want because most of the environmental confusion is removed.
As a side note, milk is a stupidly named EPD since it is actually more like maternal influence and contains the genetic components of enhanced weaning weight that are unexplained by growth genetics. Think of it as added pounds at weaning because of the mom being a good mother (although it is highly related to milk production).
Most breeds try to maintain consistency in appearance whenever they make an evaluation change, to prevent confusion in the commercial user of the EPD.
Last - GOALS and BALANCE. We know EPDs work to create change because we all bench about the changes that are being made.