The man, surnamed Lu, allegedly stole the sword from a museum.
Police said he was carrying a Chinese flag and told police he was expressing his political views.
The injured guard has been taken to hospital and is in a stable condition.
Taiwan is self-ruling but is considered by Beijing to be a breakaway province, which will one day rejoin the mainland.