Can you bring an umbrella to an outdoor concert venue in Korea?

Usually not. Goyang Sports Complex and Incheon Main Stadium are open-air stadiums with no roof, and umbrellas are usually refused at the door. If it rains, you get rained on.
Bring a poncho. The thin ones sold at convenience stores are enough. Umbrellas are stopped because they block the view of the people behind you, so they are not allowed in even when it pours.
If rain is forecast, think about your shoes too. Outdoor stands collect water.
Incheon Main Stadium is served by Munhak Stadium station on Incheon Line 1, with last trains at 24:18 and 24:52, the same on weekdays and weekends.
This site marks which venues are outdoor on each concert page. Indoor venues do not have this rule.