Aggressive undervolting is a promising technique to reduce the energy consumption of FPGAs. This energy-saving is achieved by eliminating the large voltage guardband level that is added by manufacturers to ensure the worst-case process and environmental scenarios. Further energy-savings gain is also possible; however, at the cost of performance or reliability cost. We extensively demonstrate this power, performance, energy, and reliability trade-off for off-the-shelf FPGA under different conditions like temperature, for different workloads like neural networks, and with different FPGA technologies like embedded- or HPC-based systems.