Changing class doesn't change their past or their knowledge. Think of it less as "I forgot how to use magic" and "now is not the time to use magic, I need to put some armor on".
Except they wouldn't be able to use magic anymore if they were given a tome. I don't like the system because, unlike split promotions, class switching completely alters the characters skills and physical abilities rather than just improve on what they already know. Promotions allow variety while staying "true" to the character's original concept.
But then what happens if there's a character whose from an army that is primarily armors, but the character wanted to be a mage, and shunned it? The character could explain why it is he refuses to wear armor--maybe he resents the army, but he was forced into it, and chose to go into a magic brigade that rarely gets used.
And then you put him in an armor and OH LOOK WHAT THE FUCK DUDE
I would be against it even if the case wasn't so extreme. I understand where IS is going with the class change system because it is fun to play around with the characters, but I personally will not use it until at least my 3rd playthrough because its just not Fire Emblem-y to me.