The short answer is no. Do not drink tap water in Peru, not in Lima and not in Cusco. It is treated, but not to a standard your stomach is used to, and the pipes in older buildings are their own problem.
That said, the usual advice of buying bottled water everywhere is expensive, wasteful and unnecessary. There are better options.
Where the risk actually is
The tap is the obvious one. So is the ice in a drink, if the place is not one where you would trust the kitchen.
Salads and raw vegetables washed in tap water carry the same risk as the water itself, which is why experienced travellers are warier of a side salad than of a cooked dish.
Fruit you peel yourself is fine. Fruit already cut up and sitting on a stall is a different proposition.
As for brushing your teeth, most people use tap water and are fine. If your stomach is sensitive, use bottled for the first few days.
The options, in order of sense
A filter bottle
A bottle with a built-in filter is the best solution for a trip like this. You fill it anywhere, including on a trek, and you stop buying plastic.
Look for one rated for bacteria and protozoa, which covers what matters here. It pays for itself in about a week and solves the trek problem at the same time.
Purification tablets
Cheap, weightless and reliable. The drawbacks are the taste and the waiting time, usually around thirty minutes.
Worth carrying as a backup even if you have a filter, because filters clog and tablets never do.
Boiling
Boiling works, with one important caveat at altitude: water boils at a lower temperature the higher you are, so a rolling boil in Cusco is less effective than at sea level. Boil for around three minutes rather than just until it bubbles.
Most hotels have a kettle or will fill a bottle with boiled water if you ask. Many hostels in Cusco have a dispenser of treated water and will refill you for free or for a token amount.
Bottled
Fine, and sometimes the only option, but buy large bottles for your room rather than a stream of small ones. Check the seal is intact.
On the trek
Your operator boils or treats water for the group and refills your bottle at meals. That is standard and included.
Streams on the trail look clean and are not reliably so. There are animals upstream in almost every valley in the Andes.
Carry at least two litres of capacity. At altitude you need to drink considerably more than you would at home, and underdrinking makes altitude symptoms worse.
Mate de coca, and why it helps
Coca tea is offered everywhere and it is made with boiled water, which makes it one of the safest drinks in the country and a useful habit.
The same goes for any hot tea or coffee. If you are somewhere small and unsure about the water, ordering something hot is the sensible move.
If something does go wrong
Mild stomach upset is common in the first days and usually passes. Rehydration salts, which every pharmacy sells, do more than any other single thing.
See a doctor if there is fever, blood, or symptoms lasting more than two or three days. Cusco has clinics used to treating travellers, and your insurance should cover the visit.
One note for trekkers: an upset stomach at 4,000 metres is considerably worse than one in a hotel. If you are unwell the day before a trek starts, say so rather than pushing through.