Skip to content

Reduce Firestore reads from survey sync and survey list - #3984

Open
rfontanarosa wants to merge 8 commits into
masterfrom
rfontanarosa/3983/reduce-firestore-reads-from-survey-sync-and-survey-list
Open

rfontanarosa wants to merge 8 commits into
masterfrom
rfontanarosa/3983/reduce-firestore-reads-from-survey-sync-and-survey-list

Conversation

@rfontanarosa

Copy link
Copy Markdown
Collaborator

closes #3983

@rfontanarosa

Copy link
Copy Markdown
Collaborator Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors SurveysCollectionReference to use one-shot reads via flow and get().await() instead of realtime listeners, reducing Firestore read costs. It also increases the sync delay in SurveySyncService to 30 minutes to bound reads, and updates the corresponding unit tests. The review feedback recommends ensuring main-safety for these cold flows by applying .flowOn(Dispatchers.IO) to offload the CPU-intensive document parsing from the main thread.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors SurveysCollectionReference to perform one-shot reads instead of using real-time listeners for getReadable and getPublicReadable, and increases the sync delay in SurveySyncService to 30 minutes to optimize Firestore usage. The review feedback suggests offloading these database queries and mapping operations to Dispatchers.IO to avoid blocking the main thread, and adding unit tests for getPublicReadable to comply with the repository style guide.

@codecov

codecov Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 66.66667% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.04%. Comparing base (4240c1b) to head (93ed06c).

Files with missing lines Patch % Lines
...mote/firebase/schema/SurveysCollectionReference.kt 66.66% 2 Missing and 4 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3984      +/-   ##
============================================
+ Coverage     70.03%   70.04%   +0.01%     
- Complexity     2105     2107       +2     
============================================
  Files           441      441              
  Lines         11869    11875       +6     
  Branches       1545     1553       +8     
============================================
+ Hits           8312     8318       +6     
+ Misses         2781     2777       -4     
- Partials        776      780       +4     
Files with missing lines Coverage Δ
...undplatform/android/data/sync/SurveySyncService.kt 100.00% <ø> (ø)
...mote/firebase/schema/SurveysCollectionReference.kt 74.07% <66.66%> (+2.64%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rfontanarosa

Copy link
Copy Markdown
Collaborator Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors survey retrieval in SurveysCollectionReference from real-time listeners (snapshots()) to one-shot reads (get().await()) to reduce Firestore billing costs, and increases the sync delay in SurveySyncService to 30 minutes. Corresponding unit tests have been updated and expanded. Feedback highlights that switching to one-shot reads might introduce UI latency on slow networks and recommends caching the resulting cold flow to avoid redundant requests. Additionally, the 30-minute sync delay is noted as potentially too long, and introducing randomized jitter is suggested to prevent a 'thundering herd' problem when multiple devices sync simultaneously.

Comment on lines +52 to 63
return flow {
val snapshot = references.get().await()
val surveys =
snapshot.documents
.mapNotNull { SurveyConverter.toSurvey(it) }
.filter { survey ->
survey.generalAccess == Survey.GeneralAccess.GENERAL_ACCESS_UNSPECIFIED ||
survey.generalAccess == Survey.GeneralAccess.RESTRICTED ||
survey.generalAccess == Survey.GeneralAccess.UNLISTED
}
emit(surveys)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Switching from snapshots() to get().await() changes the retrieval behavior from a real-time stream to a one-shot query. While this successfully reduces long-lived listener costs, get() by default attempts to reach the server first before falling back to the cache. This can introduce noticeable UI latency (hanging on a loading spinner) on slow or intermittent networks, whereas snapshots() would have emitted cached data instantly.\n\nAdditionally, since this is a cold flow, every new collection/subscription will trigger a new network request. Please ensure that the callers (e.g., ViewModels or Repositories) share or cache this flow (using stateIn or similar) to prevent redundant Firestore reads on configuration changes or screen navigation.

* notifies each of their devices.
*/
private const val SYNC_DELAY_MILLIS = 10_000L
private const val SYNC_DELAY_MILLIS = 30 * 60 * 1000L

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

A static delay of 30 minutes (30 * 60 * 1000L) is extremely long for syncing survey updates, which might cause data collectors to work with stale configurations for a significant period.\n\nFurthermore, if multiple devices receive a sync notification simultaneously (e.g., via FCM), they will all schedule their sync with the exact same 30-minute delay. This creates a "thundering herd" problem where all devices will still query Firestore at the exact same time, causing a spike in reads.\n\nTo mitigate this, consider introducing a randomized jitter to the initial delay in enqueueSync to spread the load over time. For example, you can add a random offset (e.g., up to 5 minutes) to SYNC_DELAY_MILLIS when building the work request.

This branch is waiting to be deployed

1 waiting deployment
e2e-approval — 93ed06c3 Waiting Oct 30, 2026 by rfontanarosa via instrumentation-tests / request-approval #1359
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reduce Firestore reads from survey sync and survey list

1 participant