Yesterday, I was getting ready for worship services as fast as I possibly could, when I realized that the only dressy shirts I had were very low, and had to be layered. So I started looking for an undershirt. The only one I found, was one that I got specifically for when shirts aren't long enough, (because most of the time when I find an undershirt that in't too low, it's barely high enough) so it was a little low. And my mom didn't have any that would work either. So, being me and panicking when I realize that I have NOTHING to wear, I realized that I had SAFETY PINS! :D They're not just for crafting. What I did was I had my mom pin the straps of the camisole to my shirt so it wouldn't sink anymore. So, I suggest that for just such occasions, it might be smart to carry some around, just to make sure you can be modest at all times, and save yourself from embarrassment. BUT, what if you don't have any safety pins on hand? :)
Yesterday, I was at a friend's house, and her older sister, very time I go over, has a bunch of clothes that don't fit her anymore, and sh so lovingly gives them to me, because I am about a size or two smaller than her. So, I was changing into another shirt that I had to have an undershirt with, but in doing so, lost the safety pins. So, what I suggest is, at the back, tie the straps/sleeves together. You can find string almost anywhere you look! The higher you tie them, the higher your shirt will go. :)
I hope I helped you in your quest to be modest. I know I have these emergencies all the time, and I thought I'd share a couple of my fixes, because I'm sure I'm not the only one who has these problems. :)