I'm more expressing my anger that you do not have the option to react in that way, that's terrible.
I don't know how cold it is where you live, but the other option is carry a backpack with you to store a jacket. When I'm out late at night, that's what I prefer to do. During the day, obviously there's more cars out so I have more power to react negatively.
Wear the jacket when you are walking place to place. You can store the jacket in your backpack when you don't need it. It's a bit to carry, but worth it in an unsafe area. I hate down for good reason, but I own a down jacket that was gifted. It's great to scrunch up and fit in a backpack. Otherwise, a lightweight plain color hoodie could be used.