My husband, who is not Hungarian, does pretty well saying: Nem beszelek magyarul. Which means: I do not speak Hungarian. Unfortunately, as he is saying that, he is indeed speaking Hungarian and that actually gets him in trouble. I guess he needs to change his answer to: Csak egy kicsit beszélek magyarul. Which means: I only speak a little Hungarian.
My caution to you is, if you are going to perfect a phrase, be prepared to hear the next thing in Hungarian. Those who speak fluently or link to practice will be so excited to hear that you speak Hungarian that they will definitely want to continue the conversation.