Lint blib2to3

[clavedeluna]

Description

The goal of this PR is to get files under blib2to3 to be linted during pre-commit. Unfortunately, many of these files have difficult to manage code so to make this PR manageable to review, I've decided to fix some of the flake8 errors while adding noqa to others.

As an example of why I added noqa to some, there are some flake8 linting errors for src/blib2to3/pgen2/tokenize.py: generate_tokens, which indicate that a rehaul of this function is really necessary.

Checklist - did you ...

• Add an entry in CHANGES.md if necessary? is this necessary for internal change?
• Add / update tests if necessary?
• Add new / update outdated documentation?

References #3355

[github-actions]

diff-shades reports zero changes comparing this PR (7dee7bf) to main (c23a5c1).

---

What is this? | Workflow run | diff-shades documentation

[JelleZijlstra]

Not a full review (and I know this is still a draft), but please try to keep unnecessary changes to a minimum to reduce churn.

[clavedeluna]

Not a full review (and I know this is still a draft), but please try to keep unnecessary changes to a minimum to reduce churn.

For sure, small PRs are always my goal. So far, everything is needed to make things pass. Is there something you think is so far out of bounds?

[clavedeluna]

I have to say that adding linting to blib2to3 has raised some pretty egregious code. I think it's worth splitting some of the changes to another PR. So this PR would be adding linting to that dir and maybe fixing just some minor flake8 failures, but anything that requires serious fixing I can add noqa to that line and open separate PRs to remove the noqas so we can discuss the code changes.

[ichard26]

The PR seems fine, but I'm not a big fan of the token -> token_mod changes, it doesn't seem worth it.

[clavedeluna]

The PR seems fine, but I'm not a big fan of the token -> token_mod changes, it doesn't seem worth it.

Let me give you an example that hopefully convinces you. The original import is
from blib2to3.pgen2 import token

any code such as this

def printtoken(type, token, xxx_todo_changeme, xxx_todo_changeme1, line):  # for testing
    (srow, scol) = xxx_todo_changeme
    (erow, ecol) = xxx_todo_changeme1
    print(
        "%d,%d-%d,%d:\t%s\t%s"
        % (srow, scol, erow, ecol, token.tok_name[type], repr(token))
    )

has a token argument which is clearly overriding the import token name. In fact, when token.tok_name is called, attempting to index the tok_name dict in token.py, it thinks we're indexing the token argument. This is pretty egregious code (no offense to original authors....)

So modifying the import is absolutely necessary, though other options could be renaming the variable names instead of the import, changing token.py module name to something else, etc. Happy to hear preferred alternatives, but for now an import alias was the easiest to do.