For an unparalleled view of the Bagan plain, you can take a hot air balloon ride at sunrise through a company called Balloons Over Bagan, for US$ 295 per person. These balloons are British made and have a perfect safety record. They do not operate during the summer.
You can rent a horse cart with a driver for around 12 000 - 15 000 Kyats for a full day. Shine Shine was our driver and we spent three days with him; we found him through Innwa guest house on main road. He is a very interesting man with large family which he was more than happy to introduce us to; we had dinner and spent two evenings after sight seeing at his village. Great fun.
Travelling around on a rented bicycle is quite easy (you do not have to compete with much vehicular traffic on the roads) and economical (as little as US$1 per day). In the morning, before it gets hot, is a particularly pleasant time to do this. People tend to rise late around Bagan, so travelling forth at 8 AM or so really emphasizes the sense of Bagan as "abandoned." Later in the day, particularly during the warm season, it may be uncomfortable to do this. In the dry season, bicycling through the sandy paths connecting the more remote temples can be harrowing exercise, but this is still the best way to get to where you want to go cheaply.