A return duct with gaps in a humid attic, crawlspace, or garage does not just leak air — it actively draws air from that space into the system whenever the blower runs, ambient odour included. If that space smells musty, a leaky return can carry that smell straight into every room the duct system serves, even if nothing inside the ducts themselves is damp.
Separately, duct insulation that has stayed wet from condensation (see ducts sweating in the attic) can hold moisture long enough to develop its own odour, which then gets pushed through the supply side into occupied rooms.