Provo has a bit of a tight rental market, so you do need to pay attention to when you're searching. If you look around February or March, you might find that you're in a much less competitive marketplace. You might have to look a bit longer to find the perfect apartment, but once you do, you at least won't be competing against several other renters to get it.
The presence of the University makes Provo one of the most strictly religious cities in Utah. You will find that a much higher percentage of people who live here are LDS adherents as compared to Salt Lake City. Just be aware of that going in, because you don't want to say something that could accidentally be taken as a slight towards the church, or it could hurt your chances of getting an apartment.
The Utah Transit Authority runs the local bus service in Provo. You can get through most of the local area using the bus, but if you need to travel elsewhere in the county, you will find the connections limited. A car is the easiest way to get around, although you will find it difficult to park if you need to travel to the campus.
There are several museums on campus, such as the Monte L. Bean Life Science Museum and the BYU Museum of Palaeontology that are open to the public. The Provo Pioneer Village is a recreated Pioneer display that is a fun place to visit with the family. There are some spectacular views of Bridal Veil Falls from a variety of walking and biking trails. The Coffee Pod is the popular coffee house near BYU.