Returns on a study-log app I built with Rork had stalled, so I turned up the push notifications. What had been one a day became three — morning, noon, night. The logic was plain: more reminders, more comebacks.
For the first few days, push-driven opens did climb. Looking at that one number on the dashboard, I was satisfied for a while. Then two weeks later, when I lined up D7 retention (the share still returning seven days after install) week by week, the line was pointing down. The cohort from the week I raised frequency had slipped from 14.2% to 9.8% on D7.
I added notifications because I wanted people back — and those notifications were speeding their exit. These are my notes on unwinding that assumption with numbers. It's a small app I run as an indie developer, but deciding frequency by cohort instead of by feel should hold at any scale.
Push-driven opens only looked like they were rising
My first stumble was watching the wrong metric. "Times someone tapped a notification to open" naturally rises somewhat when you send three times as many. More sent means more taps in absolute terms — that isn't success.
What I actually wanted was whether a user was still coming back a week later, notification or not. I should have recorded push-driven opens and organic, self-initiated opens separately from the start.
// lib/analytics.js — record opens by their trigger
// Without this split, you overrate the notifications.
export function trackAppOpen({ source, userId }) {
// source: 'push' | 'organic' | 'deeplink'
analytics.capture('app_open', {
userId,
source, // ← the lifeline for later analysis
ts: Date.now(),
});
}
Notifications.addNotificationResponseReceivedListener(() => {
trackAppOpen({ source: 'push', userId: currentUser.id });
});
if (!launchedFromNotification) {
trackAppOpen({ source: 'organic', userId: currentUser.id });
}Once I split by source, the picture sharpened. Push-driven opens rose about 1.9x per week, while organic opens actually dropped 12%. Users I dragged back with notifications were losing the habit of opening on their own. Summed together it wasn't a gain — it was eroding D7.
Cut cohorts by install week and line up the metrics
To claim "more push made it worse," you have to compare before and after on the same ruler. I grouped users by their install week and, per cohort, lined up D1, D7, D30, and opt-out rate.
| Install week | Push frequency | D1 | D7 | D30 | Opt-out rate |
|---|---|---|---|---|---|
| Week 1 (baseline) | 1/day | 31.4% | 14.2% | 6.1% | 4.3% |
| Week 2 | 3/day | 30.9% | 11.6% | — | 9.7% |
| Week 3 | 3/day | 31.1% | 9.8% | — | 13.1% |
D1 barely moved. Frequency doesn't touch the first-day experience, which makes sense. What moved was D7 and opt-out rate. The two heavy weeks kept sliding on D7, and the share of users who revoked notification permission jumped from 4.3% to 13.1%. Once a user opts out, they never get a notification again. I was trading immediate returns for the future return channel itself.
The cohort SQL isn't hard. As long as the event log carries source, the rest is aggregation.
-- D7 retention by install week
-- installs: user_id, install_date
-- opens: user_id, ts, source
SELECT
date_trunc('week', i.install_date) AS cohort_week,
count(distinct i.user_id) AS installs,
count(distinct case
when o.ts::date = i.install_date + interval '7 day'
then o.user_id end) AS d7_returned,
round(100.0 * count(distinct case
when o.ts::date = i.install_date + interval '7 day'
then o.user_id end) / count(distinct i.user_id), 1) AS d7_rate
FROM installs i
LEFT JOIN opens o ON o.user_id = i.user_id
GROUP BY 1
ORDER BY 1;