I have the HTC G2 Andriod and this thread helped me get on the right track, but I never found a complete answer to turning off the email sound notifications. Not sure why HTC hides this so well. I finally figured it out, so here it is:
Click on your Gmail icon.
Press "Menu"
Select "More"
Select "Settings"
Select the email address you'd like to change (eg yourname@gmail.com)
Select "Lables to Notify"
Hit the dropdown arrow next to "Inbox"
Select "Ringtone" and then select "Silent".
REFERENCE
http://androidforums.com/droid-support-troubleshooting/16521-how-disable-gmail-notification-sound.html