It's been a couple of days since my firmware update, so it's time for a small update on the subject.
The biggest advantage is that the mail client is very stable. I use (secure) IMAP and I have 3 mailboxes configured on the device. The mail client hasn't crashed yet.
The biggest downside (in my opinion) is that the 3G connection drops automatically, so you need to re-establish the connection every time. Sure, it saves bandwidth (and costs), but it's annoying that you actively have to monitor your e-mail.
Thankfully, the advantages outweigh the disadvantages, so I'm still happy with the Nokia E61i.