People who think the Confederate flag should be banned?

I am wondering why the people who freak out when they see a Confederate flag are usually the same people who freak out and call you 'racist' if you were to say that you think the Japanese national flag should have been banned after World War II, and you don't think they should fly it today. Two flags, two nations. What's the difference?