According to Edward Noga, a professor at NC State that is considered an expert in the disease & health of fish, chlorine levels as low as 0.10 mg/l (which is common in many tap water levels) can be accutely fatal in aquaria that has low levels of organics. Others have found residual chlorine levels as low as 0.05 mg/l to be toxic to certain species of fish.
There has been a TON of work done in this area over the past few decades, by a number of researchers, such as Tompkins & Tsai (1976) for anyone that is interested in understanding just how toxic both chlorine & chloramine can be in a closed system that contains fish.
Unfortunately the LC50 (survival time) of most tropical species of fish placed under various levels of chlorine/chloramine has yet to be established, but the bottom line is both substances are highly toxic to most aquatic organisms.
From the EPA ....
http://www.epa.gov/chemfact/s_chlori.txt
V. ENVIRONMENTAL EFFECTS
A. Toxicity to Aquatic Organisms
Chlorine has high acute toxicity to aquatic organisms; many toxicity
values are less than or equal to 1 mg/L. Twenty-four-hour LC50
values range from 0.076 to 0.16 mg/L for Daphnia magna (water flea)
and from 0.005 to 0.1 mg/L for Daphnia pulex (cladocern)
(AQUIRE 1994); 48-hour LC50 values range from 5.3 to 12.8 mg/L for
Nitocra spinipes (snail); and 96-hour LC50 values range from 0.13
to 0.29 mg/L for Oncorhynchus mykiss (rainbow trout), from 0.1 to
0.18 mg/L for Salvelinus fontinalis (brook trout), and from
0.71-0.82 mg/L for Lepomis cyanellus (green sunfish) (AQUIRE 1994).
Papillomas of the oral cavity in fish have been associated
with exposure to chlorinated water supplies (NTP 1992).
Low level chlorination (0.05 to 0.15 mg/L) results in significant
shifts in the species composition of marine phytoplankton
communities (HSDB 1994).