Fitts grew up in Helena, Alabama, and attended Helena High School. As a senior, he had a 6–5 win–loss record on the mound with a 1.68 earned run average (ERA) and 102 strikeouts in 74 2⁄3 innings pitched.
Fitts enrolled at Auburn University to play college baseball for the Auburn Tigers. He began seeing more consistent playing time towards the end of his 2019 freshman season and made two appearances in the College World Series, finishing the year with a 5.31 ERA with 47 strikeouts in 62 2⁄3 innings pitched. Fitts made six appearances as a sophomore in 2020, with a 1–0 record and a 2.77 ERA with 16 strikeouts before the season was cut short due to the coronavirus pandemic. Fitts entered his junior season on the watchlist for the Golden Spikes Award. He finished the season with a 1–3 record, a 5.88 ERA, and 41 strikeouts over 41 1⁄3 innings. That summer, he played collegiate summer baseball with the Wareham Gatemen of the Cape Cod Baseball League
On December 5, 2023, the Yankees traded Fitts, Greg Weissert, and Nicholas Judice to the Boston Red Sox in exchange for Alex Verdugo. Entering the 2024 season, he was ranked as the Red Sox' number 11 minor-league prospect by Baseball America.