Quality declined after WW2 - not by violating codes, just moving away from oak trim, hardwood strip flooring, high ceilings, basements, etc. And cheap land for suburban subdivisions. Not necessarily bad houses, just cheaply made compared to classics like the American Four Square or the Craftsman bungalow. Later, more plastic materials - laminates, PEX plumbing, paneling, etc.
As for union pay scales, surely you didn't miss the Reagan era, the PATCO strikebreaking and his general attack on unions,, the ensuing passage of Right to Work laws in many states (especially states with the fastest growth), the increases in hiring off-the-book, undocumented labor, etc.
One sardine, that's it.