What is ambient? One can think of it as a global color and brightness of parts of your object that are in shadow. But by changing this color (or changing the amount of ambient light in your scene), you are globally removing the ability of your material to go completely black when no light is shining on it. This feature was added to try and simulate the more complex phenomenon of light bouncing off of diffuse surfaces, illuminating other surfaces (Global Illumination), but fails since it adds light to the entire shadow surface, not just parts that are actually receiving light from secondary bounces. It's generally advisable to make your ambient color black, and have no ambient light in your scene.
