Skip to content
Discussion options

You must be logged in to vote

You were right: punct_default was never imported or defined. It was a latent bug in the add_unicode_punct=True path, not a symbol supplied elsewhere.

The same block had two more defects:

  1. sys was also not imported, so execution failed there first.
  2. list.extend() returns None, but its return value was assigned back to punct_marks.

All three problems are fixed in merged PR #3238, commit 1d8080af. The implementation now inspects only unique characters present in the input, selects Unicode category P characters that are not already covered by string.punctuation, and extends the punctuation list in place. This also avoids scanning the entire Unicode range for every call.

The fix includes regr…

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by LauraGPT
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants