Is it ok to warm up milk that was at room temp??

One hour or less should be fine. The only reason you should not use milk that has been previously offered to your baby is b/c there may have been bacteria transfer b/t your baby's mouth and the milk in the bottle...bacteria can spoil the milk faster.