The buttons on a forum listing (list of topics) page are in postlist.tpl, the buttons on a post listing (the thread itself) are in showflat.tpl.
As the pagination box uses a pagination class that's unique to the pagination area you can style the pagination box via "extra css" in the style editor with:
.pagination .tdheader {
}
.pagination .alt-1 {
}
.pagination .alt-1 a {
}
The first being the "Page # of #" bit, the second being the page number, and the third being how you'd like the link to display.
A similar approach should work (read: untested) for the next/previous buttons as:
#prev-next-links .tdheader {
}