Next to Saigon's Cathedral sits the Central Post Office. Even if you do not need to send anything, do get in, and admire its huge hall.
By the way, if you are visiting Vietnam, I have to tell you that communications have improved a lot recently.
Internet is very pervasive. You will find internet access everywhere in Vietnam. Even in the remotest and smallest towns, I found internet access. Internet connections was, most of the times, very fast. In some cases I had to avoid "rush hours", that is, afternoons; when kids leave school and go to the internet cafes in mass. Most of the cafes have web cams and microphones, a much cheaper alternative than telephones.
If you are staying in Vietnam for a long time, you can bring your mobile phone and buy a local prepaid card.
♥ Visit Tagulandang Island ♥