Epidemiological population studies highlight the presence of substantial individual variability in reading skill, with approximately 5–10% of individuals characterized as having specific reading disability (SRD). Despite reported substantial heritability, typical for a complex trait, the specifics of the connections between reading and the genome are not understood. Recently, the SETBP1 gene has been implicated in several complex neurodevelopmental syndromes and disorders that impact language. Here, we examined the relationship between common polymorphisms in this gene, reading, and reading associated behaviors using data from an ongoing project on the genetic basis of SRD (n = 135). In addition, an exploratory analysis was conducted to examine the relationship between SETBP1 and brain activation using functional magnetic resonance imaging (fMRI; n = 73). Gene-based analyses revealed a significant association between SETBP1 and phonological working memory, with rs7230525 as the strongest associated single nucleotide polymorphism (SNP). fMRI analysis revealed that the rs7230525-T allele is associated with functional neural activation during reading and listening to words and pseudowords in the right inferior parietal lobule (IPL). These findings suggest that common genetic variation within SETBP1 is associated with reading behavior and reading-related brain activation patterns in the general population.