Non-English spam: non-Latin scripts, Latin-script diacritics, and common stopwords.

[u/DEAD1nsane]

⚠️ Regex cannot identify every unaccented Latin-script language, so this is broad but not exhaustive.

⚠️ REMOVE --- FROM TOP OR BOTTOM DEPENDING ON WHERE IN YOUR YAML IT IS GOING

---

COPY FROM THIS GIST so you don't mess up spacing

---
# Non-English spam: non-Latin scripts, Latin-script diacritics, and common stopwords.
# Regex cannot identify every unaccented Latin-script language, so this is broad but not exhaustive.
type: submission
title+body (regex):
  - '(?i)[\U00000370-\U000003FF\U00000400-\U000004FF\U00000500-\U0000052F\U00000530-\U0000058F\U00000590-\U000005FF\U00000600-\U000006FF\U00000700-\U0000074F\U00000750-\U0000077F\U00000780-\U000007BF\U000008A0-\U000008FF]'
  - '(?i)[\U00000900-\U00000DFF\U00000E00-\U00000EFF\U00000F00-\U00000FFF\U00001000-\U0000109F\U000010A0-\U000010FF\U00001200-\U0000137F\U000013A0-\U000013FF\U00001400-\U0000167F\U00001780-\U000017FF\U00001800-\U000018AF]'
  - '(?i)[\U00001100-\U000011FF\U00002E80-\U00002EFF\U00003000-\U0000303F\U00003040-\U0000309F\U000030A0-\U000030FF\U00003100-\U0000312F\U00003130-\U0000318F\U00003400-\U00004DBF\U00004E00-\U00009FFF\U0000AC00-\U0000D7AF\U0000F900-\U0000FAFF\U00020000-\U0002A6DF]'
  - '[\U000000C0-\U0000012F\U00000132-\U0000017E\U00000180-\U0000024F\U00001E00-\U00001EFF]'
  - '(^|[^A-Za-zÀ-ÿ])(algo|casa|como|esta|estamos|estar|este|lugar|nada|nos|nunca|parece|por|porque|sobre|todo|todos|vamos|ver|vez|vida|ahora|alguien|bueno|cosa|cosas|creo|cuando|decir|desde|después|dije|dijo|dios|donde|ellos|entonces|eres|esa|ese|eso|espera|estaba|estas|esto|estoy|fue|fuera|gente|gracias|hablar|hace|hacer|hecho|hijo|hola|hombre|los|mejor|mierda|mis|mismo|momento|mucho|mundo|muy|nadie|noche|nosotros|otra|otro|pasa|pero|podemos|puede|puedes|puedo|quiere|quieres|quiero|quién|qué|sabes|seguro|siempre|siento|también|tenemos|tengo|tiempo|tiene|tienes|tipo|trabajo|tus|uno|usted|verdad|voy|acha|acho|ainda|alguém|anos|apenas|aqui|assim|até|bem|certo|coisa|coisas|depois|deus|deve|dia|disse|dizer|dois|ela|ele|eles|essa|esse|estava|estou|falar|faz|fazendo|fazer|ficar|foi|homem|isso|isto|lhe|mais|melhor|mesmo|meu|minha|muito|nem|noite|obrigado|onde|pai|pelo|pessoas|pode|posso|pouco|pra|preciso|qual|quando|quem|quer|quero|sei|sem|sempre|senhor|seu|seus|sua|talvez|também|tem|temos|tenho|ter|tinha|tudo|uma|verdade|vou)([^A-Za-zÀ-ÿ]|$)'
  - '(^|[^A-Za-zÀ-ÿ])(avec|merci|bonjour|bonsoir|vous|voici|très|être|où|parce|pourquoi|quand|combien|toujours|jamais|nicht|danke|bitte|warum|welche|welcher|welches|diese|dieser|dieses|auch|noch|schon|immer|keine|kein|keinen|meine|deine|seine|ihre|unsere|grazie|questo|questa|questi|queste|molto|anche|della|dello|nella|nello|siamo|abbiamo|hanno|puoi|bedankt|alleen|omdat|wanneer|waarom|welke|deze|heeft|hebben|worden|wordt|kunnen|moeten|graag)([^A-Za-zÀ-ÿ]|$)'
action: remove
action_reason: "Non-English spam"
---

[u/MustaKotka]

I allowlisted English characters only. That's negating known sets of English characters => report/filter.

[u/DEAD1nsane → u/MustaKotka]

Just to clarify — this rule doesn't allowlist English, it blocklists non-English. No ^[A-Za-z]+$ logic.

From automoderator.yaml:1050-1061, title+body (regex):

• Pattern 1 - Mediterranean / Middle East scripts

  • 0370-03FF - Greek and Coptic
  • 0400-04FF - Cyrillic
  • 0500-052F - Cyrillic Supplement
  • 0530-058F - Armenian
  • 0590-05FF - Hebrew
  • 0600-06FF - Arabic
  • 0700-074F - Syriac
  • 0750-077F - Arabic Supplement
  • 0780-07BF - Thaana
  • 08A0-08FF - Arabic Extended-A

• Pattern 2 - South / Southeast Asia, Caucasus, Africa

  • 0900-0DFF - Indic block
  • Devanagari / Bengali / Gurmukhi / Gujarati / Oriya / Tamil / Telugu / Kannada / Malayalam / Sinhala
  • 0E00-0EFF - Thai + Lao
  • 0F00-0FFF - Tibetan
  • 1000-109F - Myanmar
  • 10A0-10FF - Georgian
  • 1200-137F - Ethiopic
  • 13A0-13FF - Cherokee
  • 1400-167F - Unified Canadian Aboriginal Syllabics
  • 1780-17FF - Khmer
  • 1800-18AF - Mongolian

• Pattern 3 - East Asia

  • 1100-11FF - Hangul Jamo
  • 2E80-2EFF - CJK Radicals Supplement
  • 3000-303F - CJK Symbols and Punctuation
  • 3040-309F - Hiragana
  • 30A0-30FF - Katakana
  • 3100-312F - Bopomofo
  • 3130-318F - Hangul Compatibility Jamo
  • 3400-4DBF - CJK Extension A
  • 4E00-9FFF - CJK Unified Ideographs
  • AC00-D7AF - Hangul Syllables
  • F900-FAFF - CJK Compatibility Ideographs
  • 20000-2A6DF - CJK Extension B

• Pattern 4 - Latin diacritics

  • 00C0-012F - Latin-1 Supplement letters + Latin Extended-A start
  • 0132-017E - Latin Extended-A remainder
  • 0180-024F - Latin Extended-B
  • 1E00-1EFF - Latin Extended Additional

• Patterns 5-6 - stopwords with (^|[^A-Za-zÀ-ÿ])...([^A-Za-zÀ-ÿ]|$)

  • Spanish / Portuguese list
  • French / German / Italian / Dutch list

So it's not negating an English allowlist to report/filter. It's explicitly matching known non-Latin blocks + diacritic Latin + common non-English stopwords to remove. Unaccented Latin-script spam in e.g. Spanish without diacritics can still slip through — noted in the comment on line 1051 — but that's a recall gap, not an English-allowlist false-positive.

[u/MustaKotka → u/DEAD1nsane]

Maintaining this must be a nightmare.

What's the benefit of blocking non-English vs. allowing English? Set aside the word filters, they're not really a part of this.

[u/DEAD1nsane → u/MustaKotka]

Fair — if you include the stopwords, yes. I dropped those from this discussion, I agree they're high-maintenance.

For just patterns 1-4, blocklisting is lower-maintenance than allowlisting English, and that's why I did it that way:

• Allowlisting English means: remove if any char is NOT in your allowed set. You have to define English as A-Za-z + numbers + all legit punctuation + smart quotes / em-dash + emoji + URL chars + fancy-text bypasses, etc. Miss one and you nuke legit English posts. That's where the false-positives come from, and why I haven't had them with the current rule.

• Blocklisting non-English means: only fire when a known foreign-script block actually appears (Greek, Cyrillic, Arabic, Thai, Tibetan, CJK, Hangul, etc. + 00C0-024F / 1E00-1EFF diacritic Latin). Normal English with emoji, links, punctuation just passes through and never triggers.

Unicode blocks almost never change, so patterns 1-4 are set-and-forget. An English allowlist needs constant exception-tuning.

Other reason: mixed spam. Most non-English spam on my sub is English hook + non-English payload to evade. A pure allowlist that passes if any English exists misses it. A blocklist that fires if any foreign script exists catches it.

So yeah, agree the wordlists are a nightmare to maintain — I'm setting those aside. The script blocks alone give me almost all the value with almost none of the false-positives an English-only [^A-Za-z] filter would cause.

[u/MustaKotka → u/DEAD1nsane]

---

# Karma farm (repost) bot prevention, reports posts with non-Latin characters in their title

# \u0000-\u02AF = Basic Latin, Latin-1 Supplement, Latin Extended-A, Latin Extended-B, IPA Extensions
# \u1E00-\u1EFF = Latin Extended Additional
# \u2000-\u20CF = General Punctuation, Superscripts and Subscripts, Currency Symbols
# \u2150-\u215F = Number Forms (partial, common fractions only)

title (includes, regex): ['[^\u0000-\u02AF\u1E00-\u1EFF\u2000-\u20CF\u2150-\u215F]']
action: report
action_reason: Non-Latin character found in title, potential bot

---

Our filters seem to do a little different things, it seems. Our goal is not to nuke. We want to flag these, because for example a lot of bots and LLMs use emoji which are also filtered in our config. The use of non-Latin characters is especially common in post titles and we've found that to be a sufficient solution.

"Foreign" languages (=typeset other than Latin) are most often caught by this and there isn't enough pure non-English, Latin-based spam to warrant investigating those. Our members catch non-English posts and comments very often which usually means other failsafes, such as report limits, catch those.

So yes, I think I misunderstood your use case at first. Ours is just philosophically different from yours. Sorry.

[u/DEAD1nsane → u/MustaKotka]

I have automations set for blocking emojis in title

[u/DEAD1nsane → u/MustaKotka]

All good, thanks for clarifying — yeah, philosophically different use case.

Yours is report-only triage for bot/LLM titles with [^0000-02AF1E00-1EFF2000-20CF2150-215F] — makes sense, you don't want to nuke, you want a flag, and your members + report limits catch the rest.

Mine is removal for r/CasinoFreebies submissions because we get straight non-English spam posts that need to go, not just get flagged. That's why I'm matching known foreign-script blocks directly instead of negating a Latin allowlist — lower false-positive rate on normal English with emoji/links/punctuation, which is what I was seeing with zero false catches.

Agree on dropping the stopwords from this — that's the nightmare part. Keeping just the script blocks.