[MNT] Bump NumPy to 1.20

[oscargus]

PR Summary

Related to #24884

(Will add release note.)

PR Checklist

Documentation and Tests

• Has pytest style unit tests (and pytest passes)
• Documentation is sphinx and numpydoc compliant (the docs should build without error).
• New plotting related features are documented with examples.

Release Notes

• New features are marked with a .. versionadded:: directive in the docstring and documented in doc/users/next_whats_new/
• API changes are marked with a .. versionchanged:: directive in the docstring and documented in doc/api/next_api_changes/
• Release notes conform with instructions in next_whats_new/README.rst or next_api_changes/README.rst

[tacaswell]

I have a concern that this is higher than our minimum we tell the packaging system as I am not 100% sure that if we build a wheel with numpy 1.20.3 it will work with numpy 1.20.0. I suspect it will be OK, but we should be sure before we merge.

[oscargus]

Yeah, I didn't really understand why it was 1.19.2 earlier and not, say, 1.19.0. (Anyway, since oldest_supported_numpy will be installed in the isolated build environment, I think that is the one being used, so I'll try removing this completely and see what happens...)

Edit: I skipped this as I realized that it is built with --no-deps. But changed to 1.20.0.