No, it probably won't damage anything, as the PC's USB port is protected from overvoltage by the D1 schottky device on your arduino board.
Update, my previous answer contained the following paragraph, which is not related to your problem
But you should take care of some things.
- Make sure, the voltage applied to your +5V-Net is regulated properly elsewhere. I.e. choose a laboratory PSU or another good
source.
- Avoid having both your PC and your 5V-supply connected to line voltage. Ground levels may differ due to Y-capacitors present in the
switching PSUs. Those provide a AC voltage of half your line voltage
to DC ground (only with 0.3 to 0.7 mA, but enough to cause
disturbances on your signals at 55 volts in the US or 115 V in the EU)
- If you have to use two switching PSU for both PC and arduino, provide an equipotential bonding between your PC's mass (e.g.
VGA-port-screw) to your arduino's PSU's ground directly, to prevent
leakage currents from the Y-capacitors wandering around in your
circuitry.
Update according to your enhanced question
You are using a battery powered DC-DC supply for your Arduino. So, my previous notes do not apply to your problem.
- 5.1 V seem exact enough to drive your arduino safely. No need for additional stabilisation measures
- D1 (see genuine arduino schematic) separates USB-Power from +5V and prevents current flowing in reverse direction over your USB connection.
- Arduino circuitry and USB port have common GND so there are always well defined voltage levels.
In my opinion, there's nothing with your setup, which may harm your circuitry or your PC.
Depending on the maximum power of your DC-DC converter you may consider adding a fuse to protect converter and components in case of short circuit.
