All of my drinking water gets put through a Berkey filterYeah that's a good tip. The water from a plastic water bottle will taste like plastic after sitting a hot, but in a glass bottle it still tastes like clean water even after sitting for several days.
I always keep one of these handy
Spicy Abarth 500 Glass Water Bottle – Book Autowerks
18 oz. (550 mL) Glass water bottle with textured Silicone sleeve. Features screw-on polypropylene plastic lid with band for easy grabbing. Designed for use with hot and cold beverages. Hand-wash only. Do not place in freezer. Do not microwave. 2.75″ Dia x 8.5″ H DanBookfounder Dan is a DIY...bookautowerks.com
You should look into getting an RO or distiller. After drinking actual clean water for awhile you'll start to easily tell the difference between it and tap water.