Louisville (5-0) trailed 17-6 midway through the second quarter, but scored 15 unanswered points for the win, even though star quarterback Teddy Bridgewater was rendered nearly useless because of a driving storm that soaked the Roberts Stadium turf.
Perry scored both touchdowns during the rally. His 1-yard score pulled Louisville within 17-12 just before halftime and his 14-yard touchdown midway through the fourth quarter proved to be the game winner.
Louisville is off to its best start since 2006. Southern Miss (0-4) is in the midst of its worst start since 1976.
Southern Miss had a chance to win on the final drive, but a halfback pass by Desmond Johnson fell incomplete on fourth down.
Bridgewater came into the game averaging 262.2 yards passing, but was held to 9-of-13 passing for 85 yards, one touchdown and one interception.
Louisville looked like it would cruise early in the first quarter. Bridgewater turned a beautiful play-action fake into a 29-yard touchdown pass to DeVante Parker and a 6-0 lead.
But then the light showers turned into a full-blown downpour, making it all but impossible to effectively pass. The field turf turned into a gigantic lake, with an inch of water covering nearly the entire playing surface.
That's when things started going south for the Cardinals - whose short-passing offense has been so effective. Bridgewater tried a short pass to Parker later in the first quarter, but it slipped through his hands and right to Southern Miss cornerback Reggie Hunt, who returned it 23 yards for a touchdown.
Then the Golden Eagles drove the field twice in a row, scoring on a 21-yard field goal by Corey Acosta and a 1-yard run by Jalen Richard to push their lead to 17-6. Big runs by Johnson (50 yards) and Kendrick Hardy (44 yards) were instrumental on each of those drives.
The running backs for both teams had a huge advantage because the defense couldn't change directions on the waterlogged field, but only if snaps and handoffs were executed properly. Both teams fumbled several exchanges.
Louisville scored a touchdown on Perry's 1-yard run late in the second quarter as the 198-pounder sloshed forward into the end zone, but the two-point conversion failed, and the Cardinals trailed 17-12 at halftime.
Southern Miss went the entire first half without completing a pass.
Louisville's John Wallace made a 38-yard field goal late in the third quarter to pull the Cardinals within 17-15. The Golden Eagles tried to answer, but Acosta's 48-yard field goal attempt banged off the left upright and fell into the end zone.
That's when Southern Miss' mistakes started to add up.
Peter Boehme nearly whiffed on a punt deep in Southern Miss territory, and it skipped just 6 yards. Louisville took over at the Golden Eagles' 22 and six plays later scored on Perry's 14-yard touchdown run.
Southern Miss freshman Ricky Lloyd - the third-string quarterback who made his first career start after injuries to Chris Campbell and Anthony Alford - completed just 2 of 8 passes for 25 yards. Johnson rushed for a team-high 99 yards.