I tried this once with a solenoid, a small float switch and a twelve volt power supply. It was total overkill, and difficult to maintain. I have never had much luck with solenoids, and would not recommend.
I simpler thing might be a simple stopcock, the mechanism that used to be in toilets. Very simple, the stopcock ball rises up and closes the valve. They are however quite big. I do not know where you would get a small discrete one.