That changed on Sunday in a 24-6 loss to the Houston Texans that extended the Saints' skid to three games.
Payton was certainly disappointed in the defeat, but wouldn't say it was the most frustrated he's been after a loss.
"So let's not look for the 'most frustrated ever' headline," Payton said.
New Orleans (4-7) didn't score a touchdown for the first time since 2005, snapping a streak of 155 games and Drew Brees' streak of games with a touchdown pass ended at 45.
The last time the Saints didn't score a touchdown was Dec. 24, 2005 against Detroit and Brees' last game without a touchdown pass came on Nov. 29, 2012 against Atlanta.
"It's disappointing to lose a game like that, and it's definitely disappointing not to score a touchdown," Payton said.
But the problems weren't limited to offense as the defense gave up several big plays in the first game since defensive coordinator Rob Ryan was fired and replaced by Dennis Allen.
"There will be some things that we've got to clean up, but overall it was organized and efficient," Payton said of Allen's first game.
Brian Hoyer, who threw two touchdown passes, went 7 for 7 on the opening drive, capped by a 10-yard touchdown pass to Ryan Griffin, to make it 7-0 and give Houston a score on the opening possession for the first time this season.
The Texans (6-5) have won four games in a row for the first time since winning six straight in 2012.
Brees was 25 of 44 for 228 yards. The Saints converted just 3 of 12 third down attempts.
"You get in third and long against these guys, you are asking for trouble because obviously they dial up the pass rush," Brees said.
Hoyer was 21 of 27 with an interception in his return after missing last week's game with a concussion.
An 8-yard touchdown run by Alfred Blue extended Houston's lead to 21-6 in the third. Griffin grabbed a short pass in the middle of the field and dashed for a 37-yard gain on that drive.
J.J. Watt had two sacks to push his NFL lead to 13 1/2 and would have had another, but Brees threw the ball into the turf as Watt wrapped him up and was penalized for intentional grounding in the second quarter.
"We just got completely out-executed and outplayed," New Orleans tackle Zach Strief said. "That's on us. That's on the players. The schemes are good. We have to win battles."
Kareem Jackson, who returned after missing four games with an ankle injury, intercepted Brees at the goal line in the third quarter and returned it 50 yards. He might have gone all the way, but Jadeveon Clowney, who was downfield blocking, got in his way, allowing the defense to catch up and bring him down.
Griffin knocked down the pylon on his way out of bounds on his touchdown on the first drive and the play was reviewed but upheld because the ball crossed the goal line.
The Texans pushed the lead to 14-0 when Hoyer threw a shovel pass to Cecil Shorts for a touchdown on the next drive.
Hoyer completed his first 11 passes before he was intercepted by Jairus Byrd. But the Saints couldn't move the ball and had to punt.
The Saints cut the lead to 14-3 on a 30-yard field goal in the second quarter. That score was set up by a 29-yard run by Mark Ingram.
New Orleans trailed 14-6 at halftime after a career-long 57-yard field goal by Kai Forbath.
NOTES: New Orleans WR Marques Colston finished with three receptions for 38 yards to give him 702 receptions in his career. ... Houston WR DeAndre Hopkins caught a pass in his 43rd straight game and finished with five receptions for 36 yards.