Baker played at Alabama in 2020 and 2021. Over 21 games in two seasons he had nine receptions for 116 yards and one touchdown. After the 2021 season, Baker entered the transfer portal. He originally committed to transfer to the University of Kentucky before switching to the University of Central Florida (UCF). In his first season at UCF in 2022, Baker started 13 of 14 games and had 56 receptions for 796 yards and five touchdowns. As a senior in 2023, he was named first team All-Big 12 after recording 52 receptions for 1,139 yards and seven touchdowns. After the season, Baker entered the 2024 NFL draft.