r/ethdev • u/Upper-Bid-9222 • 22h ago
Question How to diagnose EVM events exhausting retries before notification delivery (missing valuation/verified token amounts)?
I'm maintaining a Python asyncio EVM monitoring pipeline with a PostgreSQL event journal and a durable notification outbox.
During one production session, a read-only reconciliation identified 49 candidate semantic event identities across 11 underlying transactions. Each identity reached five processing attempts without creating delivery or outbox records. Most occurred on Base.
The findings were:
- 26 identities had
VALUATION_UNVERIFIED. - 23 had both valuation issues and
TOKEN_AMOUNT_UNVERIFIED. - Those 23 identities came from only 3 transactions.
These are not 49 independent failed transactions, and this does not establish that 49 eligible alerts were lost.
We haven't determined whether each failure was caused by missing token prices, missing native-asset prices, unverified token decimals, provider/indexing delays, or an application logic bug.
The final NO_DURABLE_DELIVERY diagnostic tells us that no durable notification was created, but it doesn't fully explain which upstream evidence was missing at each attempt.
My questions:
- What minimal per-attempt diagnostics would you capture to distinguish missing data, invalid metadata, provider/indexing lag, and application logic failures?
- How would you preserve evidence provenance and freshness when retrying incomplete valuation or token-amount verification?
- How would you distinguish temporary evidence failures from permanently unsupported assets without infinite retries or incorrect notification thresholds?
- Since these 49 identities correspond to only 11 transactions, how would you correlate failures across attempts without double-counting them? Would you recommend structured reason codes, provider-evidence snapshots, or both?
I can share a small, redacted trace and relevant code excerpts if needed.
My priority is correctness. I don't want to assume 18 decimals, invent prices, weaken verification, incorrectly filter unresolved events, or blindly resend notifications.