Google OAuth Testing Mode: A Silent Token Expiry Bomb

On 2026-07-12, we encountered a peculiar issue in production with a user’s Google integrations in the Lima app. Both their google_tasks and google_calendar integrations were marked EXPIRED in the database. What made this particularly tricky was the state of the associated data: refresh_token and access_token were still present, but last_sync_error was null. The integration had simply died without recording why, leaving us in the dark.

The Silent Problem

Initial investigation pointed to some potentially misleading clues. The rows’ optimistic-lock version counters were suspiciously high: 16,662 and 14,743 respectively. This immediately raised a red flag, suggesting a possible retry storm hammering these rows. Perhaps a bug in our GoogleSyncScheduler was constantly trying to refresh a bad token and saving the row on each failure, driving up the version count.

However, deeper inspection disproved the retry-storm hypothesis. The version counters were actually frozen, showing no updates for approximately 10 hours and 6 hours. This meant no active process was repeatedly modifying these rows. Furthermore, our GoogleSyncScheduler, which runs every 5 minutes, is explicitly designed to only select CONNECTED or ERROR integrations via findAllActiveByProvider. EXPIRED integrations are deliberately excluded from this query. The high version numbers were simply historical accumulation: roughly 3 row saves per sync cycle, multiplied by 288 cycles per day, over about 20 days, averaging 833 versions per day. This was normal operational noise, not a symptom of the current problem.

The Technical Cause: Google OAuth Testing Mode

The real root cause was far more insidious and subtle. The refresh token for these integrations had been issued around 2026-06-22, a period when our Google OAuth consent screen was still in ‘Testing’ mode. Google has a policy for applications in ‘Testing’ mode: it kills refresh tokens after 7 days. Even though MoWave One published the app ‘In production’ on 2026-07-05, the token in question was already past its expiration date by then. Every subsequent attempt to refresh the token failed with an invalid_grant error, and because our system wasn’t recording specific error messages at the time of expiry, the integration simply transitioned to EXPIRED without any indication of why.

The Fix: Logging the Failure Reason

The immediate fix involved enhancing our error logging. Previously, Integration.markExpired() solely set status=EXPIRED. We modified it to markExpired(reason), which now writes human-readable causes into the last_sync_error field. For example, a failed refresh token will now log ‘Refresh token rejected by Google (invalid_grant), reconnect the integration’ from our doRefresh method. Similarly, a 401 variant from PullSyncService will also populate this field.

This design deliberately preserves the existing behavior where tokens are never wiped on expiry. The status transitions to EXPIRED (not DISCONNECTED), and an IntegrationDisconnectedEvent fires only on the first transition to EXPIRED to prevent notification spam. The user is then presented with a ‘Reconnect’ button in the Lima app, prompting them to re-authenticate and get a fresh token. This prevents data loss while clearly communicating the need for user action.

We’ve thoroughly covered this new behavior in our integrations test suite (194 tests), specifically asserting that last_sync_error is correctly populated on transition. Our IntegrationService now boasts 97% line and 95% branch coverage, ensuring this critical logging is robust.

-- Example of the kind of query we were running to debug
SELECT
    id,
    user_id,
    provider_type,
    status,
    last_sync_error,
    last_sync_at,
    version
FROM
    integrations
WHERE
    status = 'EXPIRED'
AND
    last_sync_error IS NULL;

Takeaways

This incident provided a stark reminder: when a third-party dependency fails, it is absolutely essential to persist the reason at the failure site. A simple status flag without a cause can turn a 5-minute diagnosis into a multi-hour database archaeology session. Furthermore, Google OAuth ‘Testing’ mode is a time bomb for any token minted before you officially publish your application. Be vigilant about when tokens are issued relative to your app’s publishing status. Always record explicit errors, not just status changes.