Temperature has more of an effect than just slowing down fish. It can be an important factor in digestion and other functions. Sure the fish won't die suddenly in colder water but that doesn't mean it's a good idea. Your tank should mirror the environment the fish evolved to thrive in...not the temperature they can technically survive in. If someone locked me in a cave at 50 degrees year round with no clothing or way to compensate for the temp I wouldn't necessarily die but I would be be happy about it.