I lived in Japan for a little over a year. I was in a fairly rural area and when i arrived i spoke no Japanese, but got on fine. I went to classes over there and learned a bit, but some people i know never learned any. They could manage no problem.
If you go to somewhere like Tokyo people will speak to you in English. They will be impressed you are trying, but will still speak to you in English.