Ilias Trichopoulos's Blog

How to Combine TanStack Query Data in React

Ilias Trichopoulos — Sat, 01 Aug 2026 00:00:00 GMT

Who this is for. You already use TanStack Query and are comfortable with useQuery and query keys. If you are not there yet, read TkDodo’s React Query series first. It is the best writing there is on this library, and everything below assumes it.

One common TanStack Query pattern is that you can create your own wrapper hooks around the primitive hooks the library provides, especially if you want to reuse them across several components or other hooks, or just want to keep your components’ code clean:

// src/components/Asset.tsx
const Asset = ({ assetId }) => {
  const { data } = useAssetPrice(assetId)

  return (/* ... */)
}

// src/api/queries.ts
const useAssetPrice = (assetId) => useQuery(priceQuery(assetId))

const priceQuery = (assetId) =>
  queryOptions({
    queryKey: ['prices'],
    queryFn: () => fetchPrices(),
    select: (prices) => prices[assetId]
  })

TkDodo recommends exactly this in Practical React Query, and it is good advice. The query key and the fetcher live in one place. The component doesn’t need to know how the data arrives, it just uses the data it needs.

An interesting question then is: What goes inside these wrapper hooks?

For an example like the one above where the data the component needs comes directly from a single API endpoint, the answer is simple: just the useQuery call. But what if the shape of the data the component needs has to be derived by combining data from multiple API endpoints?

To illustrate what I mean: Take the example of a portfolio app that supports multiple accounts and displays the total value. You create usePortfolioValue().

The endpoints you have available for deriving the value are:

// get assets list of 1 account
GET /accounts/:id/assets -> [{ assetId, amount }]

// get prices of all assets from an external market-data service
GET /prices              -> { assetId: price }

How would you do it?

A detailed catalogue of options for transforming data already exists. Combining is a different problem, and only partially addressed online. If your backend can do the JOIN for you, take it. But if it can’t, keep reading.

You will see this called different things depending on where you read about it. Derived queries. Query composition. Combining queries. Same question.

In this article I am exploring different ways the community and myself have thought of to solve this problem, each with its own trade-offs.

Spoiler alert: There is no one best solution, but a solution listed here might help you in your particular situation.

The problem

Back to the portfolio example. The user connects several accounts, let’s say 3 accounts:

stocks account
precious metals account
crypto account

Each account holds several assets. The overview page looks something like this:

Total value
$48,150.00                                 

Asset   Allocation              %     Value
───────────────────────────────────────────
AAPL    ██████░░░░░░░░░░░░░░   32%  $15,408
VWRL    █████░░░░░░░░░░░░░░░   24%  $11,556
BTC     ████░░░░░░░░░░░░░░░░   18%   $8,667
MSFT    ███░░░░░░░░░░░░░░░░░   16%   $7,704
GOLD    ██░░░░░░░░░░░░░░░░░░   10%   $4,815
───────────────────────────────────────────
                              100%  $48,150

Given the 2 available GET endpoints mentioned above, let’s try to derive the total portfolio value.

1. Combine in the component

The most obvious one is to combine the data in the render function of the component, the body of our wrapper hook, by calling useQuery/useQueries for each piece of data and stitching them together:

const usePortfolioValue = () => {
  const accountIds = useAccountIds()

  const { data: prices } = useQuery(pricesQuery())
  const { data: assets } = useQueries({
    queries: accountIds.map(assetsQuery),
    combine: mergeAssets
  })

  return useMemo(() => 
    reduceTotalValue(assets, prices),
    [assets, prices]
  )
}

// module scope, so the reference stays stable
const mergeAssets = (assetsPerAccounts) => 
  assetsPerAccounts.flatMap((account) => account.data ?? [])

This is a totally valid approach. It is important, however, to understand what happens under the hood. Let’s do some arithmetic. Our usePortfolioValue() hook will create several Observers once it’s called, one for each Query. One for the prices data, and one for each account data. In our example of 3 accounts, that is 4 Observers per consumer, one for prices and one for each account, for every use of usePortfolioValue() across our app.

Another thing worth noticing is what happens when our cached data changes.

If prices change, the prices Observer notifies the component and it rerenders. The mergeAssets() function, however, won’t run. It lives in a different Observer, the one watching the accounts, and none of the queries that Observer watches has changed. combine is memoized per Observer, so it hands back the previous result untouched. The useMemo() deps do change, of course, so we get a new portfolio value.
If an account’s data change, the corresponding Observer will run the combine method (our mergeAssets()) which will return new results. useMemo() runs again and our components rerender.

2. Combine in `combine` of `useQueries`

Did you notice how useful that combine memoization is? This next approach hands all the queries to one useQueries call, so the whole derivation gets it:

const usePortfolioValue = () => {
  const accountIds = useAccountIds()

  const { data } = useQueries({
    queries: [
      ...accountIds.map(assetsQuery),
      pricesQuery()
    ],
    combine: combineResults
  })

  return data
}

// prices was listed last, so everything before it is an account
const combineResults = (results) => {
  const assetsPerAccounts = results.slice(0, -1)
  const prices = results.at(-1)?.data ?? {}

  return reduceTotalValue(mergeAssets(assetsPerAccounts), prices)
}

reduceTotalValue() and mergeAssets() stay the same as before, nothing new here about them. What is new is that reduceTotalValue() is no longer called at the component/hook render function level but rather in the Observer level, just like our mergeAssets() from before. That means the result is memoized, and the output of combine runs through structural sharing before it reaches the consumer, so the components rerender only when the portfolio value actually changes. If, for example, prices of two assets change together and they cancel each other out, the components won’t rerender. I wouldn’t judge you if you think this is a stupid example. But at least it demonstrates my point.

Did you notice something, though? What happens now if only 1 price changes? The Observer will run combineResults() which itself will run mergeAssets() and the new portfolio value is returned. In our previous approach, mergeAssets() didn’t need to run. The assets didn’t change, only a price changed.

You traded one optimization for another. If your mergeAssets() is the expensive one, prefer the first approach. If your reduceTotalValue() is the expensive one, prefer the latter one.

(I’d add that you’re also trading some code readability.)

As for the number of Observers, nothing changed. Still 4 Observers per consumer. Same queries, just combined in a different place.

You are happy with your usePortfolioValue() hook, you pat yourself on the back, and you start using it around in your codebase. In fact, you realize you need to calculate the allocation percentage for each asset row, so you decide to use your new hook on each asset row. Now, usePortfolioValue() has 6 consumers, the header value, and each row.

(If you think this example is not realistic and that the developer should instead prop-drill the value or put it in a Context, you have a point. That is a discussion of its own and I cover it in a follow-up. Here it is just a way to illustrate the other approaches.)

Total value
$48,150.00                                   <- usePortfolioValue()

Asset   Allocation              %     Value
───────────────────────────────────────────
AAPL    ██████░░░░░░░░░░░░░░   32%  $15,408  <- usePortfolioValue()
VWRL    █████░░░░░░░░░░░░░░░   24%  $11,556  <- usePortfolioValue()
BTC     ████░░░░░░░░░░░░░░░░   18%   $8,667  <- usePortfolioValue()
MSFT    ███░░░░░░░░░░░░░░░░░   16%   $7,704  <- usePortfolioValue()
GOLD    ██░░░░░░░░░░░░░░░░░░   10%   $4,815  <- usePortfolioValue()
───────────────────────────────────────────
                              100%  $48,150

You think to yourself, “hmm… so much wasted computation. Every time I use usePortfolioValue(), the results of reduceTotalValue() and mergeAssets() need to be recalculated.” And you would be correct. No matter which of the above approaches you use, the results are cached either at the component level (in useMemo()) or in the Observer level (in combine). The results are cached per consumer.

There are several ideas on how to share a cache between consumers. One idea is to move the memoization outside of the components and the Observers completely by using a separate library like fast-memoize.js. There are, however, a few more ideas that don’t require an additional package, which you might find useful.

3. Combine in `queryFn` with `Promise.all`

One of these ideas is to move the derived value cache one level deeper. In the QueryCache itself. Who controls what goes in the QueryCache? The queryFn function of a Query. So here’s a take on it, using Promise.all():

const usePortfolioValue = () => {
  const accountIds = useAccountIds()
  
  const { data } = useQuery(portfolioQuery(accountIds))

  return data
}

const portfolioQuery = (accountIds) =>
  queryOptions({
    queryKey: ['portfolio', accountIds],
    queryFn: async () => {
      const assetsPromises = Promise.all(accountIds.map(fetchAccountAssets))
      const [assetsPerAccounts, prices] = await Promise.all([
        assetsPromises,
        fetchPrices()
      ])

      return reduceTotalValue(assetsPerAccounts.flat(), prices)
    }
  })

The derivation of the portfolio value now happens at the Query level and stored in the QueryCache. It is calculated on every refetch. It is not calculated per consumer anymore. When all 6 of the above components mount, the calculation will run only once. Your components now have a cache they can share. But at what cost?

Since the QueryCache now stores only the derived value, any component that simply needs the price of an asset needs to fetch it again:

const { data: prices } = useQuery(pricesQuery())

So one cost of a shared derived cache is additional network requests. You’ve lost granularity.

Another cost is error handling. I purposefully left loading and error states out of scope to keep the article content simple. But it’s worth talking about it here. With the above Promise.all() implementation, if any of the API calls returns an error, the whole Query throws and will be retried again, depending on your configuration. Imagine an extreme scenario where your app has 100 accounts, not 3. Fire 101 requests at the same time and you will most probably hit some rate limiting and 429s. Even if only 1 out of those 101 returns a 429, all 101 requests will then be retried.

This again, is a trade-off against additional network requests, and an even more dangerous one that you need to watch out for. You could, of course, reach for Promise.allSettled() but that wouldn’t solve your retry problem, unless you re-implement retries within the queryFn but let’s not get there.

Last thing worth mentioning: you are now down to 1 Observer per consumer, from 4. The whole fan-out moved inside the queryFn.

4. Combine in `queryFn` with `fetchQuery`

This loss of granularity really bugs you. But then you remember: The queryFn doesn’t care what’s inside as long as it’s a Promise. API calls return promises, of course, but so do queryClient.fetchQuery calls. What if we replaced our raw API calls from the example above with calls to the QueryCache?

const usePortfolioValue = () => {
  const accountIds = useAccountIds()
  
  const { data } = useQuery(portfolioQuery(accountIds))

  return data
}

const portfolioQuery = (accountIds) =>
  queryOptions({
    queryKey: ['portfolio', accountIds],
    queryFn: async () => {
      const assetsPromises = Promise.all(
        accountIds.map((id) => queryClient.fetchQuery(assetsQuery(id)))
      )
      const [assets, prices] = await Promise.all([
        assetsPromises,
        queryClient.fetchQuery(pricesQuery())
      ])

      return reduceTotalValue(assets.flat(), prices)
    }
  })

Now, look at that. With only a two-line change, each account keeps its own cache. Prices too. So we can still make the below calls anywhere in our app and we will hit the same cache:

useQuery(assetsQuery(accountId))
useQuery(pricesQuery())

Depending on staleTime, fetchQuery returns the cached value for everything still fresh and only fetches what was invalidated. And the composed result gets an entry too, computed once and read by everyone. I felt very proud when I came up with this idea, and very happy when I later discovered that TkDodo pointed at this solution as well, in a discussion titled, aptly, Derived Queries.

You must realize though what you have just traded, once again. A derived entry in the QueryCache is a snapshot. Not a subscription. If the prices query is invalidated, no consumer of usePortfolioValue() will re-render. The portfolioQuery has not been invalidated. Query invalidation for the portfolio value is now on your hands. I’ll save you the Uncle Ben quote here.

Nevertheless, it can be useful. If your app is built around manual invalidations, this can be a totally valid approach. In an app I’ve worked on I use a “change detection” query that asks the server “are there new data?” and invalidates several queries only when the answer is yes.

And like the previous approach, you are down to 1 Observer per consumer. Unlike it, each source keeps its own cache entry.

Which one to reach for

	Observers per consumer	Derivation runs	What you give up
1. Combine in the component	4	once per consumer	nothing shared between consumers
2. Combine in `combine`	4	once per consumer	any query data change re-runs the merge too
3. `Promise.all` in `queryFn`	1	once, shared	granular caches, and one failure retries everything
4. `fetchQuery` in `queryFn`	1	once, shared	you own invalidation of the derived entry

Reading it as a decision:

Few consumers? Stop at approach 1 or 2. The sharing the others buy you is not worth the trade. Pick between them by which function is the expensive one, mergeAssets() or reduceTotalValue().
Many consumers, and nothing else in the app needs the raw sources? Approach 3 is the simplest thing that shares the work. Watch the error handling.
Many consumers, and other screens do need the sources? Approach 4. Pay for it by invalidating the derived entry yourself. Or use an external library, which comes with its own trade-offs.

None of these is the right answer in general. They are four different answers to “where should this value be computed and cached”. Each of your use-cases can have a different answer. You are not restricted to using only one of them.

Conclusion

All above approaches are different ideas of how to implement your useQuery/useQueries hook wrappers. We started simple, by deriving and memoizing the data in the render functions, moving to memoizing them in the Observer level, and finally bringing them inside the QueryCache so that all consumers can have access to the same derived data. There are some things though that I alluded to but haven’t talked about.

I keep counting Observers and I never said why it matters. That turns out to be an article of its own, because it is really a question about where you call these hooks rather than how you write them.

TanStack Query encourages you to call them wherever you need the data. That advice is good, and it is not free. I built a slight variation of this portfolio page in nine different ways and measured each one. I’ll present my research in a follow-up article.

What do you think about these approaches? Have you thought of other ones? Did I miss a hidden trade-off? I’d be very happy to hear about it!

Reference of community discussions

Originally published at nop33.com.

Instant Cold Starts with a Persisted TanStack Query Cache (Part 5)

Ilias Trichopoulos — Fri, 10 Jul 2026 00:00:00 GMT

Part 4 improved performance: requests gated, derivations cached and shared. But one path none of that touches is the cold start. When the app launches with an empty cache, a big wallet still has to fetch everything before it can paint a single balance. This final part is about persisting the cache.

Persisting the cache for instant cold starts

Even with the gate and the compute-once layer, a fresh launch is brutal: a big wallet has to refetch everything before it can render. The fix is to persist the TanStack cache to disk and restore it on launch, so the app paints from last session’s data instantly and only refetches what the gate says is stale.

Measured on today’s build with a 42-address wallet: a launch with an empty cache fires 165 backend requests before the dashboard is fully live. A launch with a restored cache fires seven: four latest-transaction gate polls and three price refreshes. The dashboard paints from last session’s data immediately, and the gate revalidates in the background. (The full before-and-after table, including the pre-TanStack baseline, closes this post and brings this series to a conclusion.)

The interesting part is when to write. The obvious approach, persist on every cache change, felt wrong, and the stock persisters throttle the write to once per second. But the throttle is also a trap: it defers the write, so when the app is backgrounded or the wallet is switched, the write loses the race against process suspension and you restore nothing. So I hand-rolled the persisters to write synchronously, only at lifecycle edges. Here is the mobile one, on top of MMKV:

import { MMKV } from 'react-native-mmkv'
import { stringify } from '@alephium/web3'

// We deliberately DON'T use createSyncStoragePersister: its 1000ms throttle defers the write,
// so the actual MMKV write is lost when the app is backgrounded or the wallet is switched.
export const createTanstackAsyncStoragePersister = (key) => {
  const storage = new MMKV({ id: key })
  return {
    persistClient: (client) => storage.set(CACHE_KEY, stringify(client)),
    restoreClient: () => { const s = storage.getString(CACHE_KEY); return s ? JSON.parse(s) : undefined },
    removeClient: () => storage.delete(CACHE_KEY)
  }
}

Source: tanstackAsyncStoragePersister.ts

The trigger is a single AppState listener that fires the write on the transition to background:

useEffect(() => {
  const sub = AppState.addEventListener('change', (next) => {
    if (next.match(/inactive|background/) && appState.current === 'active' && walletId) {
      persistQueryCache(walletId)
    }
    appState.current = next
  })
  return () => sub.remove()
}, [persistQueryCache, walletId])

Source: usePersistQueryCacheOnBackground.ts

The desktop equivalent is an Electron before-quit dance: the main process intercepts the quit, sends an IPC message to the renderer, the renderer awaits the (async, IndexedDB) write, and only then lets the app actually exit.

One honest correction worth stating plainly: I used to justify this design with performance, but the real justification is correctness, avoiding data loss on suspend. When I profiled the pre-optimization build sitting idle on the dashboard, the throttled persister did turn out to be the single largest CPU consumer of a wallet at rest, re-serializing the entire cache on every poll tick, roughly 4% CPU spent persisting data in which nothing had visibly changed. Wasteful, then. But never the jank I used to blame on it: the main thread was 92% idle throughout. The data-loss reasoning is airtight; the performance reasoning was mostly a phantom.

Design choices

await persistQueryClientSave({
  queryClient,
  persister: createPersister(getPersisterKey(walletId)), // one cache file per wallet
  dehydrateOptions: {
    // Never write testnet/devnet data to disk.
    shouldDehydrateQuery: (query) =>
      query.meta?.isMainnet === false ? false : defaultShouldDehydrateQuery(query)
  }
})

Source: persistQueryClientContext.tsx

Per-wallet namespacing. Each wallet gets its own cache blob, restored on unlock, so switching wallets never bleeds data across.
Amounts as strings. Every balance is stored as a string, not a bigint, because bigint is not JSON-serializable. This keeps the persisted payload plain JSON and is the invariant the whole persistence layer quietly depends on.
maxAge: Infinity, and a deliberate asymmetry. Desktop wipes its cache on a version mismatch; mobile does not, because mobile updates ship silently and clearing the cache on every update would force a full refetch and a visibly slow launch for infrequent users. The right fix is not a time-based maxAge (which punishes the once-a-month user) but a hand-bumped buster schema version that only changes when the shape of cached data changes, keeping the cache across normal updates and invalidating it precisely when it would otherwise be unsafe.

The cold-start payoff

Here is the scenario that justifies persisting computed data, not just raw responses. A user connects to a dApp in their browser, which launches the desktop wallet. The wallet rehydrates and immediately opens a “select an address to connect” modal, which needs the per-address search strings so the user can filter. If those search strings are not already in the restored cache, they recompute on the main thread the instant the modal mounts, and the UI janks.

This is why “persist sources, derive on read”, the tidy principle jotai would nudge me toward, is wrong for this path. Cold-start-critical derived data must be persisted already computed, so it is there the microsecond the modal opens. My architecture already does the right thing here, and it is the most concrete argument against the lazier alternative.

Persistence trade-offs

No size guard. Because the balance and token queries are gcTime: Infinity, they are never garbage-collected, so a heavily-used whale wallet’s persisted blob grows unbounded, and on mobile it is stringify-d synchronously on the JS thread at the exact moment the OS is suspending the app. If this ever bites, the surgical fix is to trim persisted infinite-transaction history to its first page on dehydrate (the biggest contributor, and not cold-start-critical), not to drop the derived queries the connect modal depends on.
Serialization fidelity differs across platforms. Mobile round-trips through JSON; desktop’s IndexedDB uses structured clone, which natively handles bigint, Map, and Set. Today this is moot because everything is a string, but it is an undocumented invariant one refactor away from a hard-to-reproduce corruption bug.

Summary

Batching cuts the request count, throttling caps the rate of what is left, retrying recovers the residue, persistence eliminates most cold-start requests entirely, and the gate (Part 3) kills steady-state requests. Each layer covers the failure mode of the one below it.

The series, measured

One table to close the series. Same 42-address wallet (203 tokens, 360 NFTs), same machine, same four actions (unlock, wait for it to settle, open the Addresses page, return to the Overview), development builds throughout. Three eras: the Redux architecture whose problems started this series in Part 1, the naive TanStack migration that Part 4 profiled, and today’s architecture with the gate, the compute-once layer, and persistence.

	Redux (v2.3.6)	Naive TanStack	Today, cold start	Today, warm start
Backend requests	1,595	210	165	7
`429` responses	981 (61%)	0	0	0
Total blocking time	2.4s	207.9s	3.8s	1.9s
Worst single main-thread task	0.5s	65.4s	0.7s	0.8s
Main thread busy	38%	86%	25%	8%
Balances visible	after a ~40s trickle	after the freezes	~5s to first, ~15s to all	instantly, from the restored cache

The arc is honest: the Redux era was never CPU-bound. It drowned the network and starved the user of data. The naive TanStack migration fixed the network in one shot, request deduplication and batching cut the flood to 210 requests and killed the 429s outright, but it melted the CPU instead. The architecture of Parts 3, 4, and 5 holds both at once: a main thread quieter than the Redux era ever managed (25% busy on a cold start versus Redux’s 38%, with balances arriving in seconds instead of a 40-second trickle), warm-relaunch requests down 228x (1,595 to 7), blocking time down 107x against the naive migration (207.9s to 1.9s), and not a single 429.

Final words

If you are deep in TanStack Query and any of this makes you wince, especially the side-effect-in-queryFn gate or the manual coherence model, I would genuinely like to hear it. The best architectures get sharper under exactly that kind of scrutiny.

This wraps up the series. It started with three problems, a tangled flow, a request flood, and slow updates, and closed with a layered TanStack Query architecture that answers all three. Thanks for reading.

Useful resources:

Originally published at nop33.com.

Derived Wallet State Without Melting the CPU (TanStack Query, Part 4)

Ilias Trichopoulos — Fri, 03 Jul 2026 00:00:00 GMT

Part 3 got the request count under control by gating everything behind a single cheap poll. But the migration that cut the requests also created a bottleneck of its own: the CPU. Almost no component wants a raw API response. It wants data derived from several of them, and deriving that data inside every component is what made the app janky. The old Redux architecture never had this problem, because it computed each derivation once, into the store (the exact hesitation from Part 2). The naive TanStack version multiplied it by every component that asked.

The problem

Wallet worth calculation

To calculate the worth of the wallet, we need to:

find out the balances of each token for every address
find out the price of each token
multiply token price with its balance and sum the results

To get the token balances of every address, we need to:

call the address ALPH balance endpoint
call the token balances endpoint

The above will return a list of token IDs and their balances. To get prices we need the token symbol (like USDT, ALPH, etc.).

get token symbols by fetching the latest token list from GitHub. This is a JSON file mapping token IDs to metadata (including the token’s symbol, like WETH).
get token prices by calling the token prices endpoint and passing the list of symbols

We can now multiply the balance with the price and sum the results for every address to finally calculate the wallet’s worth.

The symptom

The dashboard janked. Scrolling the token list stuttered, and every background refresh blocked the UI. It was not a network problem (Part 3 had already tamed the requests); it was the CPU.

I loaded my wallet, and profiled the Electron renderer through four minutes of completely ordinary usage: unlock the wallet, wait for it to settle, click to the Addresses page, click back to the Overview. (Development build, so the absolute numbers are inflated. The shape is what matters.)

The trace reads like a crime scene:

Total blocking time: 207,861 ms. Three and a half minutes of blocked main thread in a four-minute recording.
Unlocking ran back-to-back main-thread tasks of 47.7s, 43.0s, and 17.9s: about 109 seconds of continuous freeze.
Navigating to the Addresses page blocked for about 7 seconds. Navigating back to the Overview produced a single 65-second task followed by a 27-second follow-up: about 93 seconds, from one click.
The Frames track shows stretches of 30 to 40 seconds where the app shipped zero frames.

For calibration: the same four actions on the pre-TanStack build cost 2.4 seconds of total blocking time. This jank was not inherited. It was new, and it was mine.

Two details in that trace taught me the most. First, the 65-second block is one task: with React’s legacy sync rendering, the entire notify-and-re-render avalanche completes in a single turn of the event loop, which is why the app cannot even repaint while it happens. Second, the asymmetry: leaving the dashboard cost 7 seconds, returning to it cost 93. The page being mounted pays the observer bill, and the dashboard, with its wallet worth total and token lists fanning out across every address, has by far the biggest bill.

The profiler pointed the finger at long-running functions inside the TanStack Query library itself, not my own code. But which functions surprised me, and it is worth being precise about, because I had misdiagnosed it until I measured.

The problem, in code

Almost no component wants a raw API response. They want derived data: the wallet’s total balance across all addresses, the list of tokens separated into listed fungibles, unlisted fungibles, and NFTs, the fiat value of a holding. Early on, every component that needed this computed it itself, with a hook shaped like this:

// The naive version: every component that needed the wallet's tokens called a hook like this.
const useWalletTokensByType = () => {
  const addressHashes = useUnsortedAddressesHashes()

  return useQueries({
    queries: addressHashes.map((hash) => addressTokensBalancesQuery({ hash, networkId, isNodeOnline })),
    // This combine re-derives the same result on every render, in every component that calls the hook.
    combine: (results) => separateTokensByType(results) // sort, group by standard, merge metadata, fold balances…
  })
}

Twenty components calling a hook like that created twenty sets of observers and ran separateTokensByType twenty times on every render.

For a long time, my diagnosis was structural sharing: TanStack’s replaceEqualDeep walking large token arrays once per observer. The profiler says otherwise. replaceEqualDeep accounts for well under a tenth of a second, and my combine callbacks for even less, in traces where the main thread was pegged for minutes. What actually dominated was the bookkeeping TanStack performs for each observer: defaultQueryOptions (re-defaulting the options of every query, of every hook, on every render), trackResult’s property-tracking wrappers, createResult, query-key hashing, and a garbage collector spending 13 to 24 percent of the entire trace sweeping up after all of it.

The mechanic that makes this multiply is the one TkDodo documents for select: it runs once per QueryObserver, and so does all the machinery around it. Nothing about an observer’s work is deduplicated across components: thirty components calling the same hook means thirty observers per underlying query, each paying the full bookkeeping bill. With a wallet of dozens of addresses, hooks like the one above multiplied into thousands of observers. While optimizing, we counted a single hook being called 1,398 times during one wallet unlock. Stabilizing the selector with useCallback, the usual first advice, does nothing here, because the cost is per-observer, not per-render. The fix had to do two things: run the expensive composition once, and drastically cut the number of observers.

Fix 1: compute once, cache the result

The first move was to push the composition into a queryFn, and let TanStack cache the computed result under its own key. A query’s queryFn can pull its dependencies straight from the cache with queryClient.fetchQuery (which deduplicates and shares the cache with any useQuery on the same key). This is exactly the pattern Dominik Dorfmeister endorses:

“you can use queryClient.fetchQuery inside a queryFn of another query.”

That is what the level:N keys are about: they are a derived-data dependency graph. Level 1 composes the two level-0 balance queries:

export const addressBalancesQuery = ({ addressHash, networkId, isNodeOnline, skip }) =>
  queryOptions({
    queryKey: ['address', addressHash, 'level:1', 'balances-all', { networkId }],
    ...getQueryConfig({ staleTime: Infinity, gcTime: Infinity, networkId }),
    queryFn: shouldSkip(isNodeOnline, skip)
      ? skipToken
      : async () => {
          // Compose from the cached level-0 queries (fetched once, shared everywhere).
          const { balances: alph } = await queryClient.fetchQuery(addressAlphBalancesQuery({ addressHash, networkId, isNodeOnline, skip }))
          const { balances: tokens } = await queryClient.fetchQuery(addressTokensBalancesQuery({ addressHash, networkId, isNodeOnline, skip }))

          return {
            addressHash,
            balances: alph.totalBalance !== '0' ? [{ id: ALPH.id, ...alph }, ...tokens] : tokens
          }
        }
  })

Source: addressQueries.ts

The consumer is now trivial, useQuery(addressBalancesQuery(...)), with zero computation in the component. The expensive composition happens once, is stored once, and is structurally shared once. The same idea powers tokenQuery, which resolves any token ID by chaining fetchQuery calls: check the token list, then the token type, then the appropriate metadata, caching the resolved token under its own key.

Fix 2: hoist the fan-out into a context

A “fan-out” is one logical need that expands into many parallel requests: “show the wallet’s tokens” becomes one query per address. fetchQuery-in-queryFn solves cross-query composition, but a hook that does useQueries across all 50 addresses still creates 50 observers per component that calls it. So I hoist those wallet-wide fan-outs into a React Context that runs the useQueries once for the whole tree. A small factory builds these:

export const createDataContext = ({ useDataHook, combineFn, defaultValue }) => {
  const DataContext = createContext({ data: defaultValue, isLoading: false, isFetching: false, error: false })

  const DataContextProvider = ({ children }) => {
    // The expensive useQueries fan-out + combine runs ONCE here, not in every consumer.
    const { data, isLoading, isFetching, error } = useDataHook(combineFn)
    const value = useMemo(() => ({ data, isLoading, isFetching, error }), [data, isLoading, isFetching, error])
    return <DataContext.Provider value={value}>{children}</DataContext.Provider>
  }

  return { useData: () => useContext(DataContext), DataContextProvider }
}

Source: createDataContext.tsx

This is React Context used the way TkDodo recommends, as dependency injection, not as a second state store. The single source of truth is still the query cache; the context just hoists the subscription and the combine to one place so consumers read a ready-made value.

The payoff was measured twice. During the optimization itself, we counted hook invocations on a single wallet unlock: useFetchWalletBalancesAlphByAddress went from 1,398 calls to 3, and the refactor cut combine executions by 99.7% overall. And profiling today’s build through the same four actions as the trace above: total blocking time fell from 207.9 seconds to 1.9, a 107x reduction, and the click back to the Overview went from 93 seconds of freeze to about a quarter of a second.

Why two tools, not one

The split is deliberate, and it maps onto where each tool can reach:

Single-source derivation (slice one query) uses select. It is the lighter tool, it recomputes reactively from live data, and crucially it carries no manual-invalidation burden. I use it for things like “this one token’s balance out of the address’s balance list.”
Cross-source composition (combine multiple queries) uses fetchQuery-in-queryFn, because select only ever sees one query’s data. The price is that the derived entry caches a snapshot and does not auto-recompute when its sources change, so I own its invalidation. Concretely, when the Part 3 gate detects a change it does not just refetch the raw balances; it invalidates the derived queries in dependency order, level 0 before the level 1+ entries that read it, so each one recomputes on fresh inputs. That ordering is exactly what the level:N key names encode.

An alternative I weighed: aggregates as cached queries

I could have gone further and discarded React Context entirely, turning each wallet-wide aggregate (the total balance, the tokens-by-type list) into its own fetchQuery-in-queryFn query keyed on the address set. It is worth saying why I mostly did not.

What that buys you: one mechanism instead of two, and, more interestingly, the aggregate becomes a cached entry that persists to disk. On cold start the wallet total would be there instantly, with no recompute, which is exactly the philosophy of Part 5. Context-derived values are recomputed on every mount.

What it costs you:

Reactivity. useQueries + combine re-runs automatically when any underlying address query changes. A cached aggregate is a snapshot you must fold into the invalidation cascade by hand, more of the coherence-ownership tax from Part 3.
Partial loading state. combine can report “some addresses are still loading” and let the UI fill in incrementally as each one resolves. A single aggregate query is all-or-nothing: nothing renders until the whole queryFn finishes.
Dynamic address sets. With useQueries the address list is just the queries array. As a query, the set goes into the key, so adding an address spawns a fresh entry and leaves the old one lingering until garbage collection.

The sharp version of the trade: move the all-or-nothing aggregates that benefit from persistence (the wallet total) to cached queries, and keep Context only where consumers need partial, incremental loading across a changing address set. I kept Context for the dashboard aggregates precisely because the per-section loading states matter there.

A direction I want to explore: jotai

Full disclosure: I did not weigh jotai-tanstack-query against this design, for the unglamorous reason that I did not know it existed at the time. I want to look into it, because on paper it addresses things I solved by hand. Jotai derived atoms compute once, are shared across all subscribers, recompute automatically when their sources change, and re-render only the components reading them. That would, in principle, collapse my hand-written level cascade: derived atoms would recompute themselves when a source changes, instead of me invalidating six levels in order.

Two things make me cautious for these particular apps. The wallets already run Redux Toolkit, so adding Jotai means a third state paradigm and a “which state lives where” tax. And jotai’s derive-on-read recomputes derived data lazily, which is the opposite of what the cold-start path needs (the subject of Part 5). So it sits firmly on my “investigate later” list rather than being a regret. If you have shipped jotai-tanstack-query at this kind of scale, I would genuinely like to compare notes.

Trade-offs and what I would improve

I own invalidation for every derived query. Forget to invalidate a level and it serves a stale snapshot forever. The level:N convention makes this systematic, but it is a convention, not a guarantee, a refactor away from a subtle bug.
Context still fans out re-renders. Any single address’s balance change produces a new memoized context value and re-renders every consumer. I measured this and it is fine today, but at whale-wallet scale use-context-selector or finer-grained contexts would localize it.
I predicted the invalidation cascade would be the next hotspot, and measurement proved me wrong. For a while the largest remaining TanStack cost in my profiles was partialMatchKey/matchQuery walking the cache, and I assumed the level:N cascade was to blame, since invalidateQueries is O(cache size) and the cascade calls it once per level, per address. Attributed properly (by caller chain, not raw self time), the cascade was ~20ms. The real cost, 3.2 seconds on a cold start, was a single useIsFetching({ predicate }) in the always-mounted header refresh button: TanStack re-runs that predicate against every cache entry on every cache event, and mine did an O(addresses) includes on top. Swapping it for a small incremental cache subscription (O(1) per event) took the scans to zero, and collapsed the notifyManager timer churn with it, since each of those events had also been scheduling a setTimeout(0) flush. The lesson:

useIsFetching with a predicate is a full-cache scan per cache event and does not belong in an always-mounted component.

Learnings

Own your coherence when you leave the library’s guarantees

The single most important thing I internalized is that the moment you step outside staleTime + invalidateQueries, the moment you build a change detector, or cache a derived snapshot, you have taken ownership of cache coherence from the library. Every one of this architecture’s real bugs (the locked-balance staleness from Part 3) lives exactly at the seam where my hand-rolled invariant leaks. That is not an argument against doing it; the request savings are enormous and real. It is an argument for knowing, precisely, which invariant you are now responsible for upholding, and auditing where it can break.

Compute once, cache many, invalidate precisely

The through-line of the whole design is the same shape repeated at three scales: do not recompute what you can cache (this part’s derived queries), do not refetch what cannot have changed (Part 3’s gate), do not re-derive on a cold path what you can persist already computed (Part 5’s connect modal). TanStack’s cache is the natural home for all three, as long as you respect that select and combine are per-observer and that a cached derivation is a snapshot, not a subscription.

My constants are calibrated by vibes

If there is one honest weakness that cuts across the whole series, it is this: every numeric boundary in the system, the 60-second and 5-minute poll tiers, the 30-day active/dormant split, the 10-req/s throttle, the retry count, was chosen by feel and validated by the absence of complaints. The architecture is sound; its calibration is intuition. The cheapest, highest-leverage improvement I could make is to replace those magic numbers with measured, server-header-driven values so the system tunes itself instead of relying on my guesses.

What comes next

Even with requests gated and derivations cached, one path stays slow: the cold start, when there is nothing in the cache yet and a big wallet has to fetch everything before it can paint a single balance. Part 5 is about persisting the cache to disk so the app restores last session’s data instantly, and the batch, throttle, and retry layers that keep the whole system under the rate limit.

Originally published at nop33.com.

Cut Wallet API Calls with Change Detection in TanStack Query (Part 3)

Ilias Trichopoulos — Fri, 26 Jun 2026 00:00:00 GMT

Part 1 laid out the three problems that made the old architecture untenable: a request flow that was hard to reason about, a self-inflicted request flood, and painfully slow data updates. Part 2 moved the whole async layer off Redux thunks and into the TanStack Query cache, which untangled that flow. This part addresses the root cause of the request flood. The trick is to stop polling the expensive endpoints altogether and gate every one of them behind a single cheap query per address.

⚡ TL;DR

A wallet with N addresses fans out into hundreds of requests just to render: measured on the last pre-TanStack release, my 42-address wallet fired 1,709 in one 43-second session. Polling all of them on a short staleTime would keep hammering the backend.
I gate every expensive query behind one cheap “latest transaction” poll per address. If the latest-transaction hash has not changed, balances cannot have changed, so nothing downstream refetches.
A new transaction hash invalidates that address’s downstream queries. Every balance and token query uses staleTime: Infinity, so freshness is event-driven, not time-driven.
This is hand-built change detection and not the docs’ dependent-query pattern.
The end state, measured: relaunching that same wallet today costs seven backend requests and zero 429s.

As we covered in Part 1, the fan-out was brutal: measured on the last pre-TanStack release, my 42-address wallet fired 1,709 requests in a single 43-second session, six endpoints per address plus retries, and 61% of the 1,595 that reached our backend were rejected as 429. The naive approach is to put a short staleTime on every query and let TanStack refetch whenever a component mounts. That, however, would not bring us any closer to the goal of reducing the number of requests.

Change detection

The key insight that unlocks everything is that an address’s balances cannot change unless a new transaction touches that address. So instead of polling the expensive endpoints, I poll one cheap thing, the latest transaction hash for each address, and treat it as a gate. If the hash has not changed, nothing downstream can have changed, so I do not fetch any of it. It essentially asks:

Has this address sent or received any transactions since the last time I checked?

The gate is a single, small query per address. Its queryFn fetches the latest transaction, compares the new hash against the previously cached one, and, only on a change, imperatively invalidates everything that depends on it:

export const addressLatestTransactionQuery = ({ addressHash, networkId, isExplorerOnline, skip }) =>
  queryOptions({
    queryKey: ['address', addressHash, 'transaction', 'latest', { networkId }],
    ...getQueryConfig({ staleTime: ONE_MINUTE_MS, gcTime: FIVE_MINUTES_MS, networkId }),
    queryFn: shouldSkip(isExplorerOnline, skip)
      ? skipToken
      : async ({ queryKey }) => {
          const latestTx = await throttledClient.explorer.addresses.getAddressesAddressLatestTransaction(addressHash)
          const cached = queryClient.getQueryData(queryKey)

          if (latestTx !== undefined && latestTx.hash !== cached?.latestTx?.hash) {
            await invalidateAddressQueries(addressHash)
            await invalidateWalletQueries()
            await invalidateTokenPrices()
          }

          return {
            addressHash,
            latestTx: latestTx ? {
              hash: latestTx.hash,
              timestamp: latestTx.timestamp,
            } : undefined,
          }
        }
  })

Source: transactionQueries.ts

The gate is polled per address by a single hook mounted at the root of the authenticated app, with notifyOnChangeProps: [] so the polling instances never trigger a re-render. They exist purely to keep the cache warm and run that side effect:

useQueries({
  queries: frequentlyUsedAddressHashes.map((addressHash) => ({
    ...addressLatestTransactionQuery({ addressHash, networkId, isExplorerOnline }),
    refetchInterval: FREQUENT_ADDRESSES_TRANSACTIONS_REFRESH_INTERVAL, // 60s
    notifyOnChangeProps: []
  }))
})

Source: useAddressesDataPolling.ts

Side note: A polling strategy per address

Polling every address on one fixed interval is wasteful: most addresses in a wallet are dormant, and a dormant address does not need to be checked every minute. So rather than a single hard-coded interval, the polling rate is chosen per address by a small strategy pattern: each address is assigned a polling strategy based on how recently it transacted. I considered an address that transacted in the last 30 days as “frequent” and polls every 60 seconds. A dormant one polls every 5 minutes. This is about to go away, however, with the introduction of the explorer API’s new “latest activity for many addresses” endpoint.

Freshness without staleTime

The consequence of all this is that every balance, token, and NFT query is configured with staleTime: Infinity and gcTime: Infinity. They are fetched once and then never refetch on their own. The only thing that makes them refetch is the gate detecting a new transaction. Freshness is event-driven, not time-driven.

Here is what that buys, measured on today’s build with the same 42-address wallet: relaunching the app with a warm cache (how the cache survives restarts is Part 5’s subject) costs seven backend requests: four latest-transaction gate polls, because only four of the 42 addresses are in the frequent tier, plus three price refreshes. Zero 429s. The same wallet on the pre-TanStack build fired 1,595.

Where this diverges from the docs (and why that matters)

It is easy to wave at the TanStack “request waterfalls” guide and claim this is just dependent queries. It is not.

The docs’ dependent-query pattern gates on data availability, do not fetch a user’s projects until you know the user’s ID, and they are blunt that even this “by definition constitutes a form of request waterfall, which hurts performance.” My gate is different: it gates on change detection, do not refetch B unless A changed. TanStack Query has no native concept for “refetch B only if A changed”; the idiomatic tools are staleTime and invalidateQueries, and I have hand-built a change detector on top of them.

That is a perfectly legitimate thing to do, but it means cache coherence is now my responsibility, not the library’s. The whole model rests on one invariant, balances change if and only if the latest transaction hash changes, and the interesting bugs all live where that invariant leaks.

Trade-offs and what I would improve

The gate is O(addresses) with no batching. A 50-address hot wallet emits roughly 50 latest-transaction polls per minute, forever, even when nothing changes. The single highest-leverage change would be a batched endpoint that collapses those N polls into one request. Which is something I am working on together with my colleague.
The locked-balance coherence hole. Alephium supports time-locked outputs. When a lockup expires, the balance moves from locked to available at a block height, with no new transaction. The gate keys on transaction hashes, so it never fires, and the locked/available split silently goes stale. The clean fix is event-driven (#1672): when a balance carries a lock, read the UTXO’s lockTime and schedule a one-shot invalidation at exactly that moment.
The side effect lives inside the queryFn. Reading the previous value via getQueryData to diff against the new one is neat. It gets the old hash at exactly the right moment without a side map. But it couples the gate’s loading state to the entire downstream cascade. A queryCache.subscribe observer would decouple detection from fetching, at the cost of tracking the previous hash myself.

What comes next

With the flood metered down to one cheap poll per address, the next bottleneck stopped being the network and started being the CPU, a bottleneck the migration itself had introduced. Part 4 is about composing all this cached data into the shapes components actually render, the wallet’s worth, the tokens split by type, without re-deriving it in every component and melting the main thread.

Originally published at nop33.com.

You Can't Wipe a Secret in JavaScript, but You Should Try

Ilias Trichopoulos — Sun, 21 Jun 2026 00:00:00 GMT

A self-custody wallet has exactly one job it cannot get wrong: keep the user’s secrets secret. For the Alephium desktop wallet (Electron) and the mobile wallet (React Native), those secrets come in three forms:

the mnemonic, the 24-word BIP39 recovery phrase the whole wallet derives from,
the seed / HD root key computed from the mnemonic, and
the per-address private keys derived from that root.

Encrypting them at rest is the easy part, and the part everyone remembers to do. The harder, quieter problem is what happens while the wallet is unlocked, when those secrets necessarily exist in plain text somewhere in the process’s memory. This is the story of discovering just how exposed that memory was, and the work it took to clear it up.

A spoiler on the honesty front, because it matters: in a JavaScript runtime you cannot guarantee a secret is wiped from memory. What you can do is shrink its lifetime, change its form, and reduce its blast radius. Everything below is one of those three. It’s defense in depth, not a magic wand, and I’ll be precise about the difference at the end.

The threat model

It helps to name what we’re defending against, because it changes which work is worth doing.

The realistic adversary here is another process reading this process’s memory: a memory-scraping infostealer, someone poking around with a tool like System Informer (formerly Process Hacker), a crash dump that ends up in a bug report, secrets paged out to a swap file, or a secret accidentally shipped off-device inside a crash report or analytics event.

What this work does not defend against is an attacker already running code as the user. If they own the machine, they can keylog the password and read the seed at the moment you legitimately use it. No amount of buffer-wiping fixes a compromised host.

Why JavaScript makes this genuinely hard

Three properties of the runtime fight you the whole way:

Strings are immutable. You cannot overwrite a string’s contents. The engine may keep the old bytes around until garbage collection, and may have copied them when the string was interned or concatenated. The only real fix is to never put a secret in a string.
Garbage collection is non-deterministic. Even a buffer you zero out may have been copied by the runtime, and the original copy lingers until the GC decides to reclaim it. Wiping is therefore best-effort, and has to happen as early as possible to keep the window short.

Step 1: get the secrets out of Redux

The first move was structural: introduce a single Keyring (a module modeled closely on MetaMask’s eth-hd-keyring) that owns every secret in memory and exposes operations, not material.

The public surface is things like signTransaction, signMessageHash, and exportPublicKeyOfAddress. Private keys are derived lazily and cached inside the keyring; the mnemonic is used to initialize and then dropped. Nothing secret ever touches Redux, and nothing secret is ever serialized.

The cleanup lifecycle is explicit:

public clear = () => {
  this.addresses.forEach((address) => {
    if (address.privateKey) {
      resetArray(address.privateKey) // zero the bytes in place
      address.privateKey = null
    }
  })

  this.hdWallet = null
  this.addresses = []
}

Where resetArray is exactly as unglamorous as it should be:

export const resetArray = (array: Uint8Array) => {
  for (let i = 0; i < array.length; i++) {
    array[i] = 0
  }
}

Step 2: stop storing the mnemonic as a string

Notice that resetArray takes a Uint8Array, not a string. That’s the whole point. Because strings can’t be wiped, the mnemonic had to stop being one.

Instead of persisting "absurd swing tornado ...", the wallet stores the mnemonic as a compact Uint8Array of the indices of each word in the BIP39 word list. It’s smaller, and (most importantly) it’s mutable, so it can be zeroed in place the moment it’s no longer needed.

The format is versioned at rest (version: 1 = the legacy word-string, version: 2 = the Uint8Array), so older wallets keep working through a migration. And the one place a string mnemonic is still legitimately needed is behind a function whose name is the warning label:

// It will convert the mnemonic from Uint8Array to string, leaking it to the
// memory. Use only when absolutely needed, ie: displaying the mnemonic for backup.
export const dangerouslyConvertUint8ArrayMnemonicToString = (mnemonic: Uint8Array) => /* ... */

Step 3: the part where I found the mnemonic in RAM anyway

After all of the above, I did the thing you should always do after claiming a security improvement: I tried to break it. On a Windows VM I unlocked the wallet, attached Process Hacker to the process, and searched its memory for a couple of words from my recovery phrase.

There it was. The complete 24-word mnemonic, in plain text, freshly present in memory every single time the wallet unlocked.

My keyring wasn’t the culprit. The leak was below my code, inside the BIP39 library doing the seed derivation. Deep in mnemonicToSeed, the library builds an intermediate Uint8Array (the actual input to PBKDF2) and, having derived the seed, simply returns and lets that array fall out of scope. “Out of scope” is not “erased.” Until the GC got around to it, the decoded mnemonic sat there in full.

You can’t fix a dependency’s internals from your own code, but you can vendor a patch. Using pnpm patch, I wrapped the derivation so the intermediate buffer is zeroed the instant the seed exists:

function mnemonicToSeedSync(mnemonic, wordlist, passphrase = '') {
  const encodedMnemonicUint8Array = encodeMnemonicForSeedDerivation(mnemonic, wordlist)
  const seed = pbkdf2(sha512, encodedMnemonicUint8Array, salt(passphrase), { c: 2048, dkLen: 64 })

  resetUint8Array(encodedMnemonicUint8Array) // ← the fix

  return seed
}

After the patch, the same Process Hacker search came up empty. The string form of the mnemonic no longer survived an unlock.

This find is also what pushed my to use MetaMask’s hardened crypto stack, swapping bip39/bip32 for @metamask/scure-bip39 and @scure/bip32, on the principle that the most-attacked wallet in the ecosystem has already paid for a lot of this scrutiny.

Step 4: wipe on every path, especially the failing ones

The happy path is easy to wipe. The path that throws halfway through derivation is the one that quietly leaves secrets lying around, and it’s the one attackers are happy to trigger on purpose.

So every keyring init clears first, then blanks its inputs the moment it’s done with them:

public initFromEncryptedMnemonic = async (encryptedMnemonic, password, passphrase) => {
  const { version, decryptedMnemonic } = await decryptMnemonic(encryptedMnemonic, password)

  this.clear()
  this._initFromMnemonic(decryptedMnemonic, passphrase)

  encryptedMnemonic = ''
  password = ''
  passphrase = ''
  resetArray(decryptedMnemonic)

  return version
}

and a follow-up pass made sure the keyring’s secrets are cleared on errors as well, not just on success.

One detail in that snippet is worth calling out, because it’s the honest exception to the “never put a secret in a string” rule. The password and the optional BIP39 passphrase arrive from a UI text field, and a text field only ever hands you a string. You can’t choose Uint8Array for input the platform delivers as text, and you can’t zero a string once you hold it. So the best available move is the one you see here: drop the reference the instant you’re done (password = '', passphrase = '') so the original becomes garbage-collectable as fast as possible. It’s weaker than wiping a buffer in place, and it’s worth being upfront that these two user-entered values are exactly where the rule bends.

Step 5: don’t forget the exits

A secret sitting in RAM is one risk. A secret leaving the device inside telemetry is a worse one, because now it’s on someone else’s server. A stack trace can carry a variable; an analytics event can carry a property; either can carry a key.

So the last piece was making sure none of this leaves the device by accident: sanitizing exceptions before they’re reported, stripping potentially sensitive fields from analytics events, and scrubbing error payloads so that the act of diagnosing a problem can never become the act of leaking a secret.

What this actually buys you (and what it doesn’t)

Let me keep the promise I made at the top and be exact:

The seed and the active private keys must exist in plain text in memory while the wallet is unlocked. You cannot sign a transaction without them. This work shrinks how long and in how many forms they exist. It does not, and cannot, make them not exist.
Wiping is best-effort. A zeroed buffer may have been copied by the runtime; GC timing is not under your control.
The library patch removed the string leak of the mnemonic at unlock. The derived seed still lives in the keyring until you clear() it, which is exactly why step 4 exists.
None of this saves a machine that’s already running attacker code as the user.

What you get for the effort is a real, measurable reduction in exposure: secrets out of persisted storage, out of long-lived strings, out of crash reports, wiped on every code path, and cleared from the keyring on lock. The bar an attacker has to clear goes from “read the process memory once” to “own the machine at the exact moment of use.” That’s the whole game with this class of defense: not invincibility, but moving the cost.

Takeaways

If you handle secrets in a JS/TS app, the checklist that fell out of this:

Never put a secret in a string when you control its representation. Use Uint8Array so you can zero it. Credentials typed into a UI field are the exception you can’t dodge: drop the reference as early as possible.
Keep secrets out of global stores and anything that persists (Redux, redux-persist, async storage).
Wipe early, and wipe on the error path too. Derivation that throws is where secrets leak.
Verify with a memory inspector. Process Hacker / System Informer takes ten minutes and will humble you.
Audit your dependencies’ internals. The leak was below my code, in a well-regarded library.
Don’t leak through the exits. Scrub crash reports and analytics before they leave the device.

None of this is novel cryptography. It’s disciplined plumbing, and for a wallet, the plumbing is the product.

Originally published at nop33.com.

Migrating Redux Server State to TanStack Query (Part 2)

Ilias Trichopoulos — Fri, 19 Jun 2026 00:00:00 GMT

Part 1 of the series focused on explaining the problems:

tangled request flow
self-inflicted DDoS
slow app launch and data updates.

We were using Redux as an async-state manager, a job it was never designed for. This part is the migration to a tool that was designed for that job: TanStack Query. The migration was not an one-shotted rewrite, but a months-long effort, with incremental shipments.

⚡ TL;DR

The old architecture stored server state (balances, tokens, prices, transactions) in Redux and hand-managed loading, error, and refetching for every slice. That is why the request flow in Part 1 was so hard to follow.
The mental shift was to realize that server state is not client state. TanStack Query owns the server cache and Redux keeps what the user and the device own (theme, open modals, settings, wallet metadata).
Redux’s own RTK Query got a real trial first. It shipped to production, then lost a same-branch, head-to-head migration attempt to TanStack Query.
I migrated incrementally, one domain at a time, with TanStack Query running next to Redux for the better part of a year.
Components stopped selecting from a global store and started asking for exactly what they need through thin useFetch* hooks. One hesitation almost stopped me: those hooks re-derive computed data per observer, not once. That is the cliffhanger into Part 4.

Redux was doing a job it was never built for

Redux is an excellent client-state manager. But we were using it as a network cache. Every resource the wallet read from the backend, ALPH balances, token balances, token metadata, prices, transactions, had a slice, a handful of async thunks, a status flag or two, a set of selectors, and a useEffect somewhere that decided when to fire the thunk.

Here is what that looked like for address data. A thunk fanned out into more thunks and toggled a “syncing” flag so it could not re-enter:

// before: server state modeled as Redux actions. One thunk orchestrates three more,
// and a "syncing" flag guards against overlapping runs.
export const syncAddressesData = createAsyncThunk(
  'addresses/syncAddressesData',
  async (payload, { getState, dispatch }) => {
    dispatch(syncingAddressDataStarted())
    const addresses = payload ?? (getState() as RootState).addresses.ids

    await dispatch(syncAddressesBalances(addresses))
    await dispatch(syncAddressesTokens(addresses))
    return await dispatch(syncAddressesTransactions(addresses)).unwrap()
  }
)

The trigger was pulled by a useEffect in App.tsx, reading a pile of selectors, nested a few conditionals deep, deciding on behalf of the whole app when it was time to sync:

// before: App.tsx is the fetch orchestrator. A pile of selectors feeds one nested effect.
const addressHashes = useAppSelector(selectAddressIds)
const addressesStatus = useAppSelector((s) => s.addresses.status)
const isSyncingAddressData = useAppSelector((s) => s.addresses.syncingAddressData)
const isLoadingTokensMetadata = useAppSelector((s) => s.assetsInfo.loading)
// ...a dozen more selectors...

useEffect(() => {
  if (network.status !== 'online') return

  if (assetsInfo.status === 'uninitialized' && !isLoadingTokensMetadata) {
    dispatch(syncNetworkTokensInfo())
  }
  if (addressesStatus === 'uninitialized' && !isSyncingAddressData && addressHashes.length > 0) {
    dispatch(syncAddressesData())
  }
}, [addressHashes.length, addressesStatus, isSyncingAddressData, assetsInfo.status, isLoadingTokensMetadata])

The component that renders an address balance is nowhere near this code.

The decision to fetch lives in App.tsx.
The fetch lives in a thunk.
The result lands in a slice.
The component reads it through a selector.

To trace one balance from screen to network you hop through four files and an effect dependency array. This is the “hard to reason about” flow from Part 1. That’s what you get when you model server state as client state. You end up re-implementing caching, deduplication, background refetching, and loading and error tracking by hand, on top of a library that does none of it for you.

Server state is not client state

The idea that unlocked the whole migration is one I first saw articulated in TkDodo’s “React Query as a State Manager”: most apps do not have one kind of state, they have two, and those two want very different things.

Client state is synchronous, you own it, and it is the single source of truth. Which modal is open, the current theme, the user’s settings. Redux is superb at this.
Server state is asynchronous, you do not own it, and you only ever hold a snapshot of a truth that lives somewhere else. The balances, token metadata, prices, transactions, etc, live on the Alephium explorer backend and node and can change at any time.

TanStack Query offers caching, request deduplication, background refetching, stale tracking, loading and error states, retries, all of it out of the box. So the migration was really one decision applied everywhere:

if the source of truth is the blockchain, it belongs in the query cache
if the source of truth is the user or the device, it stays in Redux.

“Move everything to React Query” is not the right approach. To make the above lines more concrete:

Moved to TanStack Query (server state)	Stayed in Redux (client state)
ALPH and token balances	Which modals are open
Token metadata and token type	Settings: theme, language, region
Token prices	Network settings
Transactions	Address labels and contacts (user-authored metadata)

The detour: we tried RTK Query first

Accepting that server state needs its own tool does not automatically point at TanStack Query. We were already deep in the Redux ecosystem, so the obvious first candidate was RTK Query, Redux Toolkit’s own answer to server state.

We did not just evaluate it on paper: the desktop wallet had been fetching the ALPH price through RTK Query since late 2022, NFT collection data moved to it in April 2024, both shipped to production, and we were planning to adopt its retry machinery next.

The full migration attempt is where it fell apart. We tried to move the wallet’s whole fan-out onto RTK Query’s endpoint-centric createApi model and ran straight into its gap: there is no useQueries, no first-class way to fire a dynamic list of queries, which is the one thing a wallet with N addresses does all day. We caught ourselves hand-rolling a useLoopedQueries hook, rebuilding the exact machinery we were trying to stop building. We pivoted to TanStack Query. Our explorer app had been running TanStack Query in production since mid-2023, useQueries fit the fan-out natively, and the cache persistence we had already tried there became a pillar of the final architecture. We didn’t reject RTK Query in the abstract. We tried it, shipped it, but it was out-competed on our hardest use case.

The after: the component asks for what it needs

On the TanStack side, a resource is one queryOptions object: its key, its fetcher, and its config, together, with no slice, no thunk, and no status flag to maintain.

// after: the entire definition of a resource, key + fetch + config in one object.
export const addressBalancesQuery = ({ addressHash, networkId }) =>
  queryOptions({
    queryKey: ['address', addressHash, 'balance', { networkId }],
    queryFn: () => throttledClient.explorer.addresses.getAddressesAddressBalance(addressHash)
  })

The real queries carry more than this, cache lifetimes, skip conditions, and the derived-data keys that Part 4 leans on, but the shape is always this: one object per resource. Components never touch it directly. They call a thin hook, and the hook hands back exactly the async state Redux made us track by hand:

// after: no selector, no dispatch, no effect. Ask for the token, get back { data, isLoading }.
export const useFetchToken = (id: TokenId) => {
  const networkId = useNetworkId()

  const { data, isLoading } = useQuery(tokenQuery({ id, networkId }))

  return { data, isLoading }
}

Source: useFetchToken.ts

Two things changed at once here:

The trigger moved down the tree. Instead of App.tsx deciding when to fetch on behalf of the whole app, each component asks for its own data where it renders it, and TanStack deduplicates so a hundred components asking for the same token still make one request. This is state colocation applied to server state: keep the request next to the thing that needs it, and lift it up only when you must.
App.tsx was massively simplified. The fetch-orchestration hub, the dozen selectors and the nested effect, are now gone leaving App.tsx with simple lifecycle wiring.

Migrating without stopping the world

TanStack Query and Redux ran side by side for the better part of a year. The migration was tracked as one epic with three explicit goals, quoted here because they are a good summary of why you would take this on at all:

Reduce load on explorer backend (less requests). Improve devX by having to manage less async operations through side-effects. Improve app performance.

The path across was deliberately gradual:

Start at the leaves. The first queries were isolated reads with no Redux entanglement.
Convert domain by domain. Slice by slice, the Redux state moved to Tanstack Query.
Do the hardest domain last. Transactions, with infinite pagination and pending mempool state, are the hardest thing to model as a cache, so they went last. I opened the pull request that finished them with the line:

“This PR is the final boss of the migration from Redux data fetching to TanStack Query.”

Extract, then reuse. Once the desktop patterns were proven, they moved into a shared package so the React Native wallet could adopt the same query factories and hooks, which is how the mobile migration wrapped up in mid-2025.

The hesitation that almost kept me on Redux

Truth be told, I was not a big fan of migrating to Tanstack Query in the beginning. After the initial investigation through the docs and some prototyping, I worried that it will be hard to combine data from multiple sources into a single data model.

With everything in Redux, derived state was computed once, at the request flow. As raw data arrived from the API, the Redux thunks calculated the derived values and wrote them into the store for all components to consume. That calculation happened once, and the result was cached.

The TanStack model inverts that. There is no store to write the derived value into, so each component’s hook re-derives it from the cached raw API responses, and TanStack’s select and combine run once per observer, not once per app. A dashboard where thirty components need the same computed shape is thirty recomputations on every refresh. For a while this looked like a genuine reason not to migrate. I would be trading a tangled-but-compute-efficient store for a clean-but-wasteful one.

The solution to this comes in Part 4, where I push composition into cached queries and hoist the expensive fan-outs so the derivation runs once again, this time without giving up the clean model.

What comes next

The async layer now lives in a cache instead of a store. That untangles the flow from Part 1, but it does nothing yet for the flood. In fact, a naive cache, as the defaults of TanStack Query, will cheerfully refetch all the time. Part 3 is where the flood finally gets solved, by gating every expensive query behind a single cheap “latest transaction” poll per address, so balances refetch only when a transaction actually touches them.

Originally published at nop33.com.

Why Async State Broke Our Crypto Wallets (TanStack Query Case Study, Part 1)

Ilias Trichopoulos — Fri, 12 Jun 2026 00:00:00 GMT

A pretty common need on the client side of web and mobile applications is managing asynchronous state, data that lives on the server. The Alephium mobile and desktop wallets are no exception. This series is a case study of using TanStack Query to manage async state in real production apps. In it, I describe the problems I faced building them, and the solutions I architected and implemented to address them.

This first article sets the scene. Before any solution makes sense, I want you to feel the three problems that made the old architecture untenable. The solutions come in the articles that follow. I would recommend not skipping ahead so that it all makes sense!

The apps

Here is a quick overview of what these apps are and what they run on.

The Alephium desktop wallet is an Electron app for macOS, Windows, and Linux. It lets the user create and securely store cryptocurrency wallets. A wallet, to its core, is simply a list of 24 words called a “mnemonic”. Multiple addresses (think IBAN) can be derived from this mnemonic. The user can send funds to and from these addresses. The user can create as many addresses as they want. Funds can be ALPH (the native token), tokens (USDT, wETH, and so on), and NFTs. The Alephium mobile wallet is a React Native app for Android and iOS, in feature parity with the desktop wallet.

Both trigger a massive fan-out architecture from the frontend’s perspective: one wallet holds many addresses, and each address needs its balances, its tokens, its NFTs, and their metadata, each from a different API endpoint. The apps must manage, trigger, and resolve all these concurrent, sometimes independent and sometimes interdependent requests.

On launch the wallet fires a burst of requests for its balances, tokens, and prices. While they're in flight the UI shows a skeleton loader; as the responses land, the real data (the wallet's worth) is calculated and replaces it. View the animation on nop33.com

The rest of this post is about what happens when that simple picture meets a real wallet with dozens of addresses and hundreds of tokens.

Problem 1: the request flow is hard to reason about

As I explain on my portfolio page for the desktop wallet, the very first implementation kept both app state and async state in a plain React.Context, with network requests fired by fetch inside useEffect hooks. That was the state of things when I joined the team in 2021. Soon after I joined, and once we started hitting some performance bottlenecks, I proposed migrating to more sophisticated state management with Redux Toolkit. This was the first major project that I owned. Network requests moved into async thunks, triggered from useEffect hooks, and the resulting (computed) async state lived in Redux. This made a lot of sense at the time, considering that the blockchain did not yet support smart contracts. There was no need for something more complex. A few years down the line, however, the Alephium blockchain was supercharged with smart contract support, which led to a Cambrian explosion of tokens and NFTs.

On wallet unlock: localStorage seeds the addresses, a useEffect dispatches the thunk, the thunk talks to the server, calculates the computed state and writes the result into the Redux slice, and that state update re-renders the App (which re-runs the effect). View the animation on nop33.com

The migration to Redux bought us an efficient rendering strategy: when a slice of state changed, only the components consuming that exact slice re-rendered. The cost was that as the project evolved the logic triggering those network requests became genuinely hard to follow.

Here is the happy path on app unlock:

The app initializes Redux with the list of addresses found in localStorage.
That list sits in the dependency array of a useEffect in App.tsx, which calls the fetchAddressesBalances thunk.
The thunk makes its network requests, writes the balances into Redux, and flips isBalancesInitialized to true.
Another useEffect has that flag in its dependency array. When it becomes true, it fires fetchTokensDetails to fetch metadata (name, logo, symbol) for the token IDs discovered in the previous step.

So a single unlock moves back and forth between Redux state, useEffect dependency arrays, and async thunks. To trace one user action the developer has to hold the whole zig-zag in their head.

Problem 2: too many requests

The desktop wallet’s user base grew, and the server grew overloaded with requests. The wallets were essentially DDoS-ing our own backend. The backend team had to add rate-limiting, which fixed the server side but created a new problem on the client side: the server now frequently responded with 429 ("Too Many Requests") and the apps displayed inaccurate data.

The patch was obvious: throttle the number of requests the clients send, and add a retry policy for 429 responses. A combination of p-throttle and fetch-retry replaced the bare fetch calls:

import pThrottle from 'p-throttle'
import fetchRetry from 'fetch-retry'

const throttle = pThrottle({
  limit: 10,
  interval: 1000
})

const throttledFetch = throttle((url, options = {}) => {
  fetch(url, options)
})

const MAX_API_RETRIES = 3

const exponentialBackoffFetchRetry = fetchRetry(throttledFetch, {
  retryOn: (_, __, response: Response | null) => {
    return !!response && response.status === 429
  },
  retryDelay: (attempt: number) => {
    return Math.pow(2, attempt) * 1000
  },
  retries: MAX_API_RETRIES
})

This saved the day. The server could keep up, and the client slowly received all the data it needed, eventually showing up-to-date information. But, as you may already suspect, this created a new problem.

Problem 3: painfully slow data updates on app launch

Throttling solved the symptom but not the cause. The client still fired way too many requests on app launch. It just fired them out slowly. This meant that the user would stare at skeleton loaders for a considerable amount of time each time they launched their app until they were able to see their balances.

Here’s a breakdown of a profiling session of a power user’s wallet of 42 addresses, 203 tokens, and 360 NFTs. The actions performed were to unlock the wallet, wait for it to settle, click to the Addresses page, click back to the Overview. It revealed some interesting insights:

1,709 requests fired in 43 seconds. The real fan-out was six endpoints per address.
700 of them left in the first ten seconds. The server rejected 981 with 429, and the exponential-backoff retries stretched the data trickle past the 40-second mark.
With no request deduplication, the same price-chart URL was fetched 17 times within the recording.
The CPU, meanwhile, was mostly idle (total blocking time: 2.4 seconds). The app was not slow because it was working hard. It was slow because it was waiting.

What’s worse is that every async state update even after the initial launch would also fire the same number of requests, taking the same amount of time.

We had to attack the real problem: fire dramatically fewer requests in the first place.

Network requests batching

A low-hanging fruit was to batch the network requests for token metadata. Token metadata is the worst fan-out: resolving 200 tokens used to be 200 requests. The idea to reach for @yornaath/batshit came from my colleague Mika, and it turned out to be a very effective lever against the 429s. It collects individual lookups inside a 10ms window and fires one batched POST of up to 80:

import { create, windowedFiniteBatchScheduler } from '@yornaath/batshit'

// Up to 80 individual token-metadata lookups made within a 10ms window collapse into one POST.
const createFTMetadataBatcher = () =>
  create({
    fetcher: throttledClient.explorer.tokens.postTokensFungibleMetadata,
    resolver: (results, queryTokenId) => results.find(({ id }) => id === queryTokenId),
    scheduler: windowedFiniteBatchScheduler({ maxBatchSize: 80, windowMs: 10 })
  })

Source: queryBatchers.ts

The biggest gains in reducing the number of requests, however, comes next.

What comes next

I worked closely with the backend team, and together we landed on a multi-layer solution that addressed all three problems at once: the tangled flow, the request flood, and the slow updates. The foundation of that solution is TanStack Query.

In the next parts of this series I’ll walk through that architecture: how I migrated the whole async layer off Redux and onto TanStack Query, how I gate expensive queries behind a single cheap poll so balances only refetch when something actually changes, how I compose derived state without melting the CPU, how I persist the cache for instant cold starts, and the layered defense that keeps all of it under the rate limit. Each layer is a direct answer to one of the three problems above.

Onward to Part 2, where I migrate the async layer off Redux and onto TanStack Query.

Originally published at nop33.com.

Speeding Up Cold Start of a React Native app

Ilias Trichopoulos — Mon, 01 Jun 2026 00:00:00 GMT

When building a production-hardened crypto wallet in React Native, your dependency tree can quickly become an absolute galaxy of cryptographic primitives, polyfills, Web3 clients, and heavy wallet connection frameworks. Don’t take my word for it, have a look at several good open-source React Native wallets such as:

Recently, I noticed that the Alephium mobile wallet hit a wall: cold-boot startup times crept up to a grueling 7+ seconds. Some users reported waiting up to 15s. I did not manage to reproduce such long times on my devices. It would still take 2s on my Pixel 8 Pro and 4s on my old OnePlus 6T. Nevertheless, I could see through profiling that the native UI thread sat frozen on the splash screen while the single-threaded JavaScript engine spent its precious initialization cycles parsing, evaluating, and spinning up heavy background modules.

By restructuring the compilation pipeline, decoupling native build profiles, and leveraging deferred runtime lifecycles, I managed to drop visual launch times down to a clean 2 seconds with fluid native transitions.

1. The Gatekeeper: Global `inlineRequires`

On the web, code-splitting (through React.lazy) physically chunks the code to save network bandwidth. On mobile, 100% of the JS bundle is already compiled and sitting locally on the device’s flash storage. The bottleneck isn’t network download size but CPU execution time instead.

In all Expo projects by default, standard top-level static imports (import X from 'Y') force the Hermes engine to synchronously execute those modules the exact millisecond the app boots. The breakthrough came from flipping a single global transformer switch in metro.config.js:

config.transformer.getTransformOptions = async () => ({
  transform: {
    inlineRequires: true
  },
});

What this actually does

Metro automatically traverses the source files during bundling and rewrites static imports into lazy getters behind the scenes. The heavy screens and services remain completely un-evaluated in memory until the exact microsecond the app explicitly mounts or calls them.

Profiling in Android Studio

To shed some light into the problem, I opened the /android folder in Android Studio to use the profiling tools that the React Native ecosystem is lacking. In the system trace below it can be seen that the massive, pitch-black void on the Native UI thread (alephium.wallet) is doing absolutely nothing. The native side of the app is completely functional, but it is idling in an empty loop, stubbornly displaying the splash screen because it has received zero instructions from the JavaScript engine on what layout to draw.

In contrast, at the bottom is the JS thread (mqt_v_js). That solid, unbroken turquoise bar running continuously past the 5-second mark is the Hermes JavaScript Engine. It is running a single, massive, synchronous CPU task: parsing the entire codebase and executing all top-level module code. The UI thread cannot drop the splash screen until this turquoise block finishes its initial pass and sends the layout instructions over the bridge.

The solution was to let the Metro bundler lazily evaluate the dependencies automatically. By enabling inline requires, Metro automatically rewrites the top-level static imports (import { Web3Provider } from 'walletconnect') into localized requests (require('walletconnect')) behind the scenes. This means that heavy depedencies such as WalletConnect are completely ignored during the initial startup script (bringing the boot time down) and they are loaded incrememtally only when an explicit function call executes them later on.

The visual profile has completely shifted:

The Native UI thread (alephium.wallet) awakens: Instead of a long black void of idling, you see a dense cluster of vibrant, multi-colored execution blocks firing early in the timeline (around the 1-second mark). These blocks represent native Android view layout measurements, MainActivity lifecycle passes, and paint calls.
The JavaScript thread (mqt_v_js): While the turquoise bar still runs to process code, its initial execution payload is split. It processes the root entry logic rapidly, unblocks the bridge early, and allows the native system to start drawing UI elements and managing transitions while other deferred modules are processed in the background.

Why does changing the Metro bundler configuration suddenly wake up the native Android thread?

Without inlineRequires, the bundle acts like a giant synchronous waterfall. The app cannot display the first screen until every single file, even code for deep sub-navigation screens the user has’t opened yet, is fully evaluated by the CPU. The native UI thread is held hostage by the JavaScript thread.

When we flip inlineRequires: true, Metro rewrites our top-level static imports into lazy function wrappers (require()). This completely changes the low-level thread execution:

The initial JS payload shrinks: Only the absolute bare-minimum framework code is executed on boot.
The bridge fires early: The JS thread finishes its initial pass in under a second and immediately sends the root view structure to the native side.
True parallel execution: As seen in the optimized trace, the native UI thread goes to work immediately drawing the initial views and preparing transitions, while the Hermes engine processes deferred modules and heavy crypto packages in parallel when they are explicitly called.

Step-by-step anatomy of a RN cold start

Let’s look under the hood on what’s happening when we write and build our project and when we launch the app.

Phase 1: Compile

When we write code, we use startard ES6 static imports at the top of the files for readability and type safety:

// Inside SendScreen.tsx
import WalletConnectClient from '~/services/walletConnectService';

export const SendScreen = () => {
  // Uses WalletConnectClient...
};

When we build the project, the Metro bundler traverses the project files. Because inlineRequires is set to true, Metro modifies the source code as it compresses it into the final index.android.bundle file. It strips away the top-level import and wraps it in a lazy JS getter property (hence the “inline” naming):

// What Metro outputs
Object.defineProperty(exports, "WalletConnectClient", {
  get: function() {
    return require('~/services/walletConnectService'); 
  }
});

const SendScreen = () => {
  // Every time you call 'WalletConnectClient', it triggers the getter function above
};

Phase 2: App launch (cold start)

When the user taps the app icon on their phone, a strict sequence of events plays out across the native OS and the JS engine:

Step 1: Native process forking (0.0s)

The Android OS forks a native Linux process for the app. The Android Runtime (ART) initializes the app’s container, reads the AndroidManifest.xml, creates the main native thread, and calls bindApplication.

Step 2: Native splash screen mounts (~0.2s)

The native UI thread (alephium.wallet) loads the theme configurations and instantly draws the native splash screen layout. The phone is now frozen displaying this layout, waiting for instructions on what to replace it with.

Step 3: The JS thread spawns (~0.4s)

The native side boots up the Hermes JS engine on a dedicated background thread (mqt_v_js). Hermes allocates its memory heap and reads the production bundle file (index.android.bundle) from the phone’s internal storage into its RAM.

Step 4: Grobal script evaluation

This is where the big performance difference happens. Hermes must execute the entire global scope of the bundle to discover the registered application components.

Without inline requires (the old way): Hermes runs the entry file. It sees a top-level static import for SendScreen. To resolve it, it must immediately jump to SendScreen.tsx, parse it, and execute it. Inside SendScreen, it hits the static import for WalletConnect. It stops, jumps to WalletConnect, parses its dependencies, instantiates the cryptographic polyfills, and runs heavy initialization setups. This reates a big domino effect. Thousands of files are synchronously read, parsed, and executed in one continuous, blocking CPU loop. The JS thread pins at 100% (the long turquoise bar), and the native UI thread sits in total darkness, unable to drop the splash screen.

With inline requires (the optimized way): Hermes runs the entry file. It encounters the lazy properties created by Metro. It skips the contents, sinice its code is wrapped in a getter closure. Instead of evaluating thousands of files, Hermes merely maps out the lightweight getter references in memory. The entire initial bundle execution pass finishes in a fraction of a second.

Phase 3: Handover and lazy runtime evaluation

Step 5: Unlocking the bridge (~1.5s)

Because Hermes finished its initial script evaluation pass quickly, it immediately reaches the root AppRegistry.registerComponent call. It executes App.tsx and sends a layout tree over the native bridge to the native UI thread.

Step 6: Splash screen drops (~1.8s)

The native UI thread receives its first set of design instructions from JS. It tears down the native splash screen layout, mounts the root SafeAreaProvider container, and displays the compiled view structure. The visual boot process is complete.

Step 7: On-demand lazy execution at runtime

Now the app is open and fully interactive. Eventually, a deferred layout hook forces the code to read an object belonging to the WalletConnect service:

const client = await WalletConnectService.init();

The exact microsecond theat variable name is evaluated, the JS engine triggeres that hidden get() function Metro generated back in Phase 1. Hermes synchronously pauses for a few microseconds, reads the local file bytes for the WalletConnect client, evaluates the cryptographic libraries for the first time, caches the instance in memory so it never has to do it again, and completes the function call.

React Native cold start thread timelines, before and after enabling global inline requires: the UI thread sits frozen on the splash screen for over 7 seconds while the JS thread runs one synchronous evaluation loop, and afterwards the UI thread is interactive at 1.8 seconds with the heavy frameworks loaded lazily on demand. View the diagram on nop33.com

The app structure went from a big upfront waterfall to an incremental on-demand execution system. The binary footprint stays exactly the same but the hardware treats it with efficiency.

2. Reducing bundle size

The bundle size affects startup time. Expo provides a useful dev package called expo-atlas. It can analyze the JS bundle and give useful information to regarding depedencies and code. The initial bundle size of the mobile wallet was 37.8MB with 6622 modules. After several rounds of optimizations, I managed to reduce it down to 27.2MB with 4351 modules:

Replaced lucide-react-native with @react-native-vector-icons/lucide: The initial icons library took ~9% of the total bundle size. This was a low-hanging fruit.
Got rid of viem: I was surprised to see that we needed this library. Using pnpm why viem revealed that the only package using it is @walletconnect/utils. With an investigation on GitHub I realized that this dependency does not exist in a following minor release of the depedency. So I edited package.json and updated the overrides section to load the new version.
Commented out devtools: I was even more surprised to see that the devtools contributed to production release bundle size.
Optimized Lottie animations: A particular lottie file was almost 2% of the bundle size. Simply replacing it with a more optimized one shaved off several KB.

Learnings

The `React.lazy` trap: Why web optimizations fail on mobile

Coming from a React web background, my immediate instinct to fix a heavy screen is to reach for React.lazy(() => import('./SendScreen')) and wrap it in a <Suspense> boundary. On a desktop browser, this is a gold standard. On a native mobile device, it is an anti-pattern that actively degrades user experience. The friction comes down to a fundamental misunderstanding of what we are optimizing for on different platforms:

On the web: The primary constraint is network bandwidth. React.lazy instructs the bundler to split a screen into a separate chunk (sendScreen.chunk.js) so the user doesn’t waste data downloading code they might never look at over cellular networks.
On mobile: The primary constraint is CPU execution. 100% of the JavaScript bundle is already compiled and sitting locally on the physical storage of the phone. There is zero network latency when moving between screens.

When wrapping a mobile layout component in React.lazy, you force React to treat a local, instantaneous file read as a slow, asynchronous Promise. When a user taps a navigation button, the native stack navigator immediately tells the GPU to kick off a fluid, hardware-accelerated slide or cross-fade transition. But because of React.lazy, the destination screen hits an unexpected <Suspense fallback={null}> wall. The native transition drops frames, jitters, or flashes a blank background while the JavaScript engine waits for the micro-task queue to click over and resolve the promise.

Instead of fighting the architecture with manual web code-splitting, keep the imports standard and static:

import SendScreen from '~/navigation/SendScreen';

By leaning entirely on global inlineRequires in the Metro configuration, the bundler defers parsing the file until it’s called, but handles the resolution synchronously from local flash storage. This gives you total protection against cold-start bloat without sacrificing buttery-smooth, native hardware transitions.

Troubleshooting

To setup Android Studio, these are some things I had to do:

A problem occurred starting process ‘command ‘node’

Launch app from the /android directory with

open -a "Android Studio"

Inconsistent JVM-target compatibility detected for tasks ‘compileDebugJavaWithJavac’ (17) and ‘compileDebugKotlin’ (21).

Even though I am using Expo 54 and React Native 0.81, the docs still recommend using Java 17. Enforce a matching target version globally.

npx expo install expo-build-properties

Open your root app.config.js and add the plugin configuration to explicitly pin the Java version to 17 across both toolchains:

{
  "expo": {
    "plugins": [
      [
        "expo-build-properties",
        {
          "android": {
            "javaVersion": "17"
          }
        }
      ]
    ]
  }
}

Regenerate native directories with:

npx expo prebuild --clean

Invalid Gradle JDK configuration found.

Change Gradle JDK location and select JAVA_HOME.

White screen when starting profiling process

It helps to first build the release variant with:

npx expo android:run --variant release

Making app profilable

Update your app.config.js file with:

// Injects the <profileable android:shell="true" /> tag into your production manifest
const withProfileableManifest = (config) => {
  return withAndroidManifest(config, (modConfig) => {
    const mainApplication = modConfig.modResults.manifest.application[0];
    if (!mainApplication['profileable']) {
      mainApplication['profileable'] = [{
        $: { 'android:shell': 'true' }
      }];
    }
    return modConfig;
  });
};

module.exports = {
  expo: {
    // ... your other existing configurations
    plugins: [
      withProfileableManifest, // 👈 Add the new manifest plugin here
      // ... your other plugins
    ]
  }
};

Build variants panel

Make sure to enable the Build variants panel (through the View menu item) and set the :app module’s active build variant to release and Active ABI to arm64-v8a (or whatever arch your Android device has). Then click Profiler: Run 'app' as profileable (low overhead) from the top bar play icon.

Useful resources:

Originally published at nop33.com.

Giving OpenClaw Safe Access to Your Obsidian Vault

Ilias Trichopoulos — Thu, 16 Apr 2026 00:00:00 GMT

I have an AI assistant (OpenClaw) running on my home server. I also have an Obsidian vault with all my notes. I wanted my private AI to read my notes to understand me better, and to write daily journal entries on my behalf.

But I didn’t want it to be able to delete or modify my existing notes if something went wrong.

The problem

Giving an AI agent filesystem access is a trust problem. If the agent can read and write freely, a single bug or hallucination could corrupt or delete files. With a sync system like Syncthing propagating changes across devices, the damage would spread everywhere within seconds.

I needed two things:

Read access to the entire vault, so the AI can learn about me and reference my notes
Write access to a single folder (Journal/) - so it can create daily notes, monthly reviews, and annual reviews

And critically: the AI should have no way to escalate its access, even if it tries.

The architecture

My setup runs on Proxmox VE with two VMs:

VM 101 (production-docker) - stores the Obsidian vault on disk, runs Syncthing
VM 102 (openclaw) - runs the AI assistant

The vault lives at /dionysus/obsidian/vault on VM 101. The key insight is to use two separate channels for read and write, each with different permissions:

      OpenClaw VM (102)                      VM 101 (production-docker)
┌────────────────────────────┐             ┌───────────────────────────┐
│                            │             │                           │
│  NFS mount (read-only)    ──────────────>│  /dionysus/obsidian/vault │
│  /mnt/obsidian-vault       │             │                           │
│                            │             │                           │
│  SSH (forced command)     ──────────────>│  write script             │
│  (only writes to Journal/) │             │  (validates path + ext)   │
│                            │             │                           │
└────────────────────────────┘             └───────────────────────────┘

The read path: NFS

NFS (Network File System) lets one machine share a directory over the network. On VM 101, I exported the vault as read-only:

/dionysus/obsidian/vault 192.168.21.60(ro,sync,no_subtree_check)

On the OpenClaw VM, it’s mounted at /mnt/obsidian-vault. The AI can read every note, but the operating system enforces that it cannot write, delete, or modify anything through this mount. No application-level workaround can bypass this since it’s enforced by the NFS server.

The write path: SSH with a forced command

For writing journal entries, the AI uses SSH. The key is configured with a forced command, which means it can only execute one specific script, no matter what command the AI tries to run.

The SSH key in authorized_keys on VM 101 looks like this:

command="/usr/local/bin/obsidian-write-journal-for-openclaw",no-port-forwarding,no-agent-forwarding,no-pty,no-X11-forwarding ssh-ed25519 AAAA... openclaw-obsidian-write

Even if the AI tries to ssh user@server "rm -rf /", the SSH server ignores the requested command and runs the forced script instead. The no-pty flag prevents getting an interactive shell. The no-port-forwarding and no-agent-forwarding flags close other escape routes.

The write script

The script on VM 101 validates every write request:

#!/bin/bash
VAULT="/dionysus/obsidian/vault"
ALLOWED_DIR="Journal"

# SSH passes the original command via this variable
if [[ -n "$SSH_ORIGINAL_COMMAND" ]]; then
  eval set -- $SSH_ORIGINAL_COMMAND
fi

MODE="write"

while [[ "$1" == --* ]]; do
  case "$1" in
    --append) MODE="append"; shift ;;
    *) echo "ERROR: Unknown flag $1" >&2; exit 1 ;;
  esac
done

FILENAME="$1"

# Must provide a filename
if [[ -z "$FILENAME" ]]; then
  echo "ERROR: No filename provided" >&2
  exit 1
fi

# Must be inside Journal/
if [[ "$FILENAME" != "$ALLOWED_DIR/"* ]] || [[ "$FILENAME" == *".."* ]]; then
  echo "ERROR: Can only write to $ALLOWED_DIR/" >&2
  exit 1
fi

# Must be a markdown file
if [[ "$FILENAME" != *.md ]]; then
  echo "ERROR: Only .md files are allowed" >&2
  exit 1
fi

mkdir -p "$VAULT/$(dirname "$FILENAME")"

if [[ "$MODE" == "append" ]]; then
  cat >> "$VAULT/$FILENAME"
else
  cat > "$VAULT/$FILENAME"
fi

Three checks:

Path restriction - filename must start with Journal/ and cannot contain .. (no path traversal)
Extension restriction - only .md files
No delete capability - the script can only create or update files, there’s no delete operation

The --append flag is important. If I’ve already written something in my daily note manually, the AI appends its summary below without replacing my content.

How the AI uses it

From the OpenClaw VM:

# Create a daily note
echo "# 2026-04-16

## What I did today
- Set up Syncthing for Obsidian sync
- Configured NFS and SSH access for OpenClaw
" | ssh -i ~/.ssh/obsidian_write ilias@192.168.21.52 "Journal/2026-04-16.md"

# Append to an existing note
echo "
## Evening reflection
Productive day focused on infrastructure.
" | ssh -i ~/.ssh/obsidian_write ilias@192.168.21.52 "--append Journal/2026-04-16.md"

Once written, Syncthing propagates the changes to my MacBook and Android phone automatically.

What happens if the AI goes rogue?

Threat	Protection
Delete all notes	Can’t - NFS mount is read-only, write script has no delete operation
Overwrite notes outside Journal/	Can’t - script rejects any path not starting with `Journal/`
Modify the write script itself	Can’t - script lives on VM 101, outside the AI’s reach
Get a shell on VM 101	Can’t - SSH key has forced command and `no-pty`
Bypass SSH and write directly	Can’t - NFS mount is read-only
Corrupt a journal entry	Recoverable - Syncthing keeps old versions in `.stversions/`

The layered approach means there’s no single point of failure. The AI would need to compromise both the NFS protocol and the SSH forced command mechanism to do real damage. Both are enforced at the OS/protocol level, not the application level.

The journal structure

All AI-written entries go into Journal/ with date-based naming:

Type	Filename	Trigger
Daily note	`2026-04-16.md`	Every evening - AI asks what I did, writes a summary
Monthly review	`2026-04.md`	End of month - AI reads that month’s daily notes and synthesizes
Annual review	`2026.md`	End of year - AI reads the 12 monthly reviews and reflects

Daily notes follow the same format as my Obsidian Daily Notes plugin template, so they integrate seamlessly with the rest of my vault.

Why not just use an API?

A REST API with authentication would work too, but the SSH forced command approach has advantages for a home server:

No additional service to run - SSH is already there
No authentication code to write - SSH keys handle it
Battle-tested security - SSH forced commands have been used for decades (think git over SSH, rsync backup scripts)
Simple to audit - one script, one key, one authorized_keys line

Takeaway

The core pattern is: separate your read and write channels, and enforce restrictions at the infrastructure level, not the application level. An AI agent promising to only write to one folder is not the same as an AI agent that can only write to one folder. The difference matters when things go wrong.

Originally published at nop33.com.

From Callbacks to Promises: Same Patterns, Less Pain (part 2)

Ilias Trichopoulos — Sun, 12 Apr 2026 14:00:00 GMT

In Part 1, I built four exercises to internalize callback-based async control flow: basic callbacks, sequential iteration, unlimited parallel, and limited parallel. Each exercise taught a pattern but also exposed a category of bug that callbacks force you to handle manually.

Here’s the table I ended with: five rules of callback discipline that you, the programmer, must implement correctly:

Rule	What goes wrong if you break it
Both code paths must be async (no Zalgo)	Callers can’t predict whether their code runs before or after the callback
`return` after calling `finalCb(err)`	The success path also fires, which leads to a double result
`hasError` flag at the top of every callback	`finalCb` fires multiple times on concurrent errors
`nextIndex` (not `completed`) for scheduling	Phantom tasks scheduled beyond the end of the array
`finalCb` called exactly once per invocation	Consumers see duplicate responses, corrupted state

In this post, I rebuild all four exercises with promises and async/await. Every rule in that table becomes a language-enforced guarantee. The code gets shorter. The bugs become impossible. And the patterns become almost trivially simple to express.

The starting point: promisify

Before I could use promises, I needed to bridge the callback-based fetchUser into the promise world. There are two ways to do this:

Option A - rewrite from scratch:

const fetchUserP = (id) => {
  return new Promise((resolve, reject) => {
    if (id < 0) return reject(new Error('Invalid id'))
    setTimeout(
      () => {
        resolve({ id, name: 'User ' + id })
      },
      100 + Math.random() * 300,
    )
  })
}

Option B - wrap the existing function:

const promisify = (fn) => {
  return (...args) => {
    return new Promise((resolve, reject) => {
      fn(...args, (err, result) => {
        if (err) return reject(err)
        resolve(result)
      })
    })
  }
}

const fetchUserP = promisify(fetchUser)

Both produce the same observable behavior. But Option B is more useful in the real world. You’ll frequently inherit callback-based APIs from Node’s older stdlib, legacy npm packages, and C++ bindings. The promisify wrapper is how you bridge them without rewriting. It’s also what Node’s built-in util.promisify does under the hood. The pattern is literally: if (err) reject(err); else resolve(result). That’s the entire bridge between the two worlds.

The accidental discovery: promises are Zalgo-safe

Look at Option A’s error path:

if (id < 0) return reject(new Error('Invalid id'))

I’m calling reject synchronously, inside the executor, on the same tick as fetchUserP(-5) itself. In Part 1, this was exactly the “releasing Zalgo” sin - a function that sometimes calls its continuation synchronously and sometimes asynchronously. I had to wrap the error path in process.nextTick to fix it.

Here, I didn’t wrap it. And it works fine. Why?

Because the ECMAScript spec guarantees that every .then, .catch, and await resumption runs as a microtask, regardless of whether the promise was settled synchronously or asynchronously. Even if you call reject inside the executor before it returns, the .catch handler doesn’t fire until after the current synchronous call stack empties.

You can verify this yourself:

fetchUserP(-5).catch((err) => console.error('catch:', err.message))
console.log('after sync code')

Output:

after sync code
catch: Invalid id

The catch handler runs after console.log('after sync code'), even though the promise was already rejected when .catch was attached. That’s not luck - it’s a language guarantee.

You cannot release Zalgo from a promise, even if you try. This is one of the most important reasons promises exist.

Here’s how the five callback-discipline rules from Part 1 map to promise guarantees:

Callback discipline (manual)	Promise equivalent (automatic)
`process.nextTick` to avoid sync/async inconsistency	Handlers always run as microtasks, even if the executor settled synchronously
`return` after `finalCb(err)` to prevent double-fire	A promise settles exactly once - extra `resolve`/`reject` calls are silently ignored
`hasError` flag + guard at top of callback	`Promise.all` / `Promise.race` handle this internally
`nextIndex` bookkeeping for scheduling	Built into `Promise.all`; manual only for limited concurrency
`finalCb` fires exactly once	A promise can only resolve or reject once, by specification

Exercise 5: Sequential with async/await

The callback version of sequential iteration (Part 1, Exercise 2) required a recursive iterate(index) helper - 20 lines of code managing a results array, a termination check, error propagation, and the recursive call.

The async/await version:

const fetchUsersSequentiallyAsync = async (ids) => {
  const results = []
  for (const id of ids) results.push(await fetchUserP(id))
  return results
}

Three lines.

That’s not a compressed version of something bigger. That is the function. The for...of loop replaces the recursive iterator. results.push() replaces the closure variable and index tracking. return results replaces finalCb(null, results). And error handling? There is none - if await fetchUserP(id) rejects, the await throws, the throw exits the loop, the throw exits the function, and the returned promise rejects. The caller catches it with try/catch.

console.time('sequential')
try {
  const users = await fetchUsersSequentiallyAsync([1, 2, 3, 4, 5])
  console.timeEnd('sequential')
  for (const u of users) console.log(`Got user: ${u.id} - ${u.name}`)
} catch (err) {
  console.timeEnd('sequential')
  console.error('Caught:', err.message)
}

Output:

Got user: 1 - User 1
Got user: 2 - User 2
Got user: 3 - User 3
Got user: 4 - User 4
Got user: 5 - User 5
sequential: 1.466s

1.466s - identical to the callback version’s 1.472s. Async/await is not a speedup. It’s a clarity improvement. The underlying work happens at exactly the same pace.

Short-circuit on error - for free

With [1, 2, -3, 4, 5]:

  → start fetch 1
  ← done   fetch 1
  → start fetch 2
  ← done   fetch 2
  → start fetch -3
  ← error  fetch -3
sequential: 492ms
Caught: Invalid id

Fetches 4 and 5 are never attempted. The moment await fetchUserP(-3) rejects, the await expression throws, the throw exits the for...of loop (no more iterations), and the promise returned by the async function rejects. No flags, no counters, no return finalCb(err). Just a thrown exception unwinding the stack the way exceptions unwind stacks in synchronous code.

Compare to the callback version: there, I needed if (err) { finalCb(err); return } inside the inner callback, a return after calling finalCb to stop the iteration, careful attention that finalCb was only called once, and process.nextTick on the error path to avoid Zalgo. All of that is now done by the language.

The `.forEach` trap

One common mistake: using .forEach instead of for...of:

// DOES NOT work - all fetches run in parallel
ids.forEach(async (id) => {
  const user = await fetchUserP(id)
  results.push(user)
})

The async callback returns a promise that .forEach ignores. The loop doesn’t wait. All five fetches get kicked off concurrently. Use for...of or a plain for loop if you want await to sequence the iterations.

Exercise 6: Parallel with `Promise.all`

The callback version of unlimited parallel (Part 1, Exercise 3) was 25 lines - new Array(ids.length), a completed counter, a hasError flag with a guard at the top of every callback, and results[i] = result for order-preserving writes.

The Promise.all version:

const fetchUsersInParallelAsync = (ids) => Promise.all(ids.map((id) => fetchUserP(id)))

One line. Promise.all takes an array of promises and returns a single promise that resolves with an array of results in the same order as the input - regardless of which one settled first. On the first rejection, it rejects immediately.

Everything I hand-coded in the callback version is built in:

new Array(ids.length) + results[i] = result → order-preserving is built in
completed counter + completed === ids.length → “all done” detection is built in
hasError flag + guard at top of callback → first-rejection-wins is built in

Output (success):

parallel-success: 321ms
Got user: 1 - User 1
Got user: 2 - User 2
Got user: 3 - User 3
Got user: 4 - User 4
Got user: 5 - User 5

Output (error with [1, 2, -3, 4, 5]):

  → start fetch 1
  → start fetch 2
  → start fetch -3
  → start fetch 4
  → start fetch 5
  ← error  fetch -3
parallel-error: 0.439ms
Caught: Invalid id
  ← done   fetch 4
  ← done   fetch 2
  ← done   fetch 5
  ← done   fetch 1

Notice the four ← done lines after the Caught: line. Those setTimeout callbacks still fired - the timers were not cancelled. Promise.all short-circuits on the first rejection, but it cannot cancel in-flight work. The other promises keep running, consuming resources, until they naturally complete. Their results are silently discarded.

JavaScript promises have no built-in cancellation. In this toy example with 300ms timers, that’s harmless. In a real system where each in-flight operation holds a database connection or an HTTP socket, this is why limited concurrency matters even more with promises.

The `.map(fn)` gotcha

My first version passed fetchUserP directly to .map:

Promise.all(ids.map(fetchUserP)) // crashed!

This blew up with TypeError: cb is not a function. Why?

Array.prototype.map passes three arguments to its callback: (element, index, array). So ids.map(fetchUserP) is equivalent to:

ids.map((element, index, array) => fetchUserP(element, index, array))

fetchUserP is promisify(fetchUser), which spreads all arguments and appends the callback:

fetchUser(1, 0, [1,2,3,4,5], (err, result) => { ... })

fetchUser takes (id, cb) - so id = 1, cb = 0 (the index!). When the setTimeout fires and tries to call cb(null, user), it calls 0(null, user). A number, not a function. Crash.

The fix: ids.map(id => fetchUserP(id)) - wrap it so only the element is forwarded.

This is the same family of bugs as the classic ['1', '2', '3'].map(parseInt) returning [1, NaN, NaN] - parseInt receives the index as its radix parameter. The rule: never pass a multi-argument function directly to .map unless you’re certain the extra arguments are harmless.

Exercise 7: Limited concurrency with promises

This is the capstone exercise - the hardest pattern from chapter 4, rebuilt with promises. I implemented it two ways.

Approach 1: Port the callback version

The most direct translation: wrap the tryNext / while loop / counter logic from Part 1 inside a new Promise:

const fetchUsersWithConcurrencyAsync = (ids, concurrency) => {
  return new Promise((resolve, reject) => {
    let running = 0
    let completed = 0
    let nextIndex = 0
    const results = new Array(ids.length)

    const tryNext = () => {
      while (running < concurrency && nextIndex < ids.length) {
        const i = nextIndex
        running++
        nextIndex++
        fetchUserP(ids[i])
          .then((result) => {
            results[i] = result
            completed++
            running--
            if (completed === ids.length) resolve(results)
            else tryNext()
          })
          .catch(reject)
      }
    }

    tryNext()
  })
}

This works, but I hit an interesting bug on my first attempt. I put running-- in a .finally() handler instead of inside .then():

fetchUserP(ids[i])
  .then((result) => {
    results[i] = result
    if (++completed === ids.length) resolve(results)
    else tryNext() // tryNext runs HERE, with stale running count
  })
  .catch(reject)
  .finally(() => running--) // decrement runs AFTER tryNext

The problem: .then(), .catch(), and .finally() handlers chain as sequential microtasks. .then() runs first, and .finally() runs after it. So when tryNext() runs inside .then(), running hasn’t been decremented yet - it’s still at the concurrency cap. The while (running < concurrency) check fails, no new task starts, and the system effectively runs at concurrency - 1.

My output confirmed it: with concurrency 3, the timing was 2.458s (matching concurrency 2’s theoretical ceil(20/2) × 250ms ≈ 2.5s), not the expected ~1.7s. After moving running-- to the top of .then() - before tryNext() - the timing dropped to 1.765s.

The lesson: in promise chains, the order of handlers matters. State mutations that affect scheduling logic must happen before the scheduling call, not in a later handler in the chain.

Approach 2: Promise.race pool

A cleaner approach that leverages Promise.race to naturally manage the concurrency window:

const fetchUsersWithConcurrencyAsync = async (ids, concurrency) => {
  const results = new Array(ids.length)
  const pool = new Set()

  for (let i = 0; i < ids.length; i++) {
    if (pool.size === concurrency) {
      await Promise.race(pool)
    }
    const promise = fetchUserP(ids[i])
      .then((result) => (results[i] = result))
      .finally(() => pool.delete(promise))
    pool.add(promise)
  }

  await Promise.all(pool)
  return results
}

The idea: maintain a Set of currently-running promises. The for loop spins through ids, adding promises to the pool. When the pool hits capacity, await Promise.race(pool) suspends the loop until one promise settles, freeing a slot. After the loop, await Promise.all(pool) drains any remaining in-flight work.

Notice that .finally() is used correctly here - for cleanup (removing a promise from the pool set), not for control-flow state. The scheduling isn’t driven by a counter; it’s driven by await Promise.race, which naturally resumes the for loop after the .then() and .finally() microtasks have both run. By the time the loop continues, the settled promise has already been removed from the pool.

Comparing the two approaches:

	Approach 1 (port)	Approach 2 (pool)
State variables	`running`, `completed`, `nextIndex`, `results`	`results`, `pool`
Scheduling	Manual `tryNext()` with `while` loop	`await Promise.race(pool)`
”All done” detection	`completed === ids.length` then `resolve(results)`	`await Promise.all(pool)` then `return results`
Error propagation	`.catch(reject)`	Automatic - `Promise.race` throws, `await` propagates
Lines of function body	~20	~10

Both produce the same output:

  → start fetch 1
  → start fetch 2
  → start fetch 3
  ← done   fetch 2
  → start fetch 4
  ← done   fetch 1
  → start fetch 5
  ...
  ← done   fetch 20
limited-parallel: 1.587s
Fetched 20 users

Exactly 3 in flight at all times, results in input order, ~1.6s total.

The big picture

Here are all eight exercises across both posts, mapped side by side:

#	Pattern	Concurrency	Tools	What you handle manually
1	Basic callback	-	`fetchUser(id, cb)`	Zalgo prevention, error-first convention
2	Sequential	1	Recursive `iterate()`	Recursion, termination check, error short-circuit
3	Unlimited parallel	N	Counter + `hasError` flag	Flag placement, order-preserving, exactly-once callback
4	Limited parallel	k	`tryNext()` + `while` loop	4 state variables, scheduling guard, phantom-task prevention
5	Promisify	-	`new Promise` wrapper	Nothing - the bridge is mechanical
6	Sequential	1	`for...of` + `await`	Nothing - loop + await + throw handle everything
7	Unlimited parallel	N	`Promise.all`	Nothing - order, counting, and error handling are built in
8	Limited parallel	k	`Promise.race` pool	Pool management (10 lines vs 30)

The arc is clear. As you move down the table, the “What you handle manually” column empties out. The patterns don’t change - sequential, parallel, limited parallel are the same problems in both halves. What changes is how much of the bookkeeping the language does for you.

Async/await doesn’t make your code faster. The timings are identical: ~1.5s sequential, ~300ms parallel, ~1.7s limited parallel, regardless of whether you use callbacks or promises. What it does is make the mental model tractable. You stop having to reason about “which set of things are true at the exact moment each callback fires” and start writing code that reads like synchronous logic - with for, if, try/catch, and return - where the only new concept is that await suspends until a promise settles.

That’s the whole arc of chapters 4 and 5 of the book. If you’ve worked through these exercises, you’ve internalized it. Happy coding.

Originally published at nop33.com.

Mastering Async Callbacks in Node.js, Without the Complexity (part 1)

Ilias Trichopoulos — Sun, 12 Apr 2026 10:00:00 GMT

I’ve been reading Node.js Design Patterns (4th Edition) by Mario Casciaro and Luciano Mammino. It’s an excellent book. I’ve even read the 3rd Edition on my Kobo, but I realized I am having a tough time comprehending technical subjects on an e-book reader, so I decided to order the physical book.

As I was reading in (in parallel with my favorite LLMs to test my comprehension), chapters 4 and 5 hit me like a wall. Both chapters teach async control flow patterns (callbacks in ch. 4, promises/async-await in ch. 5) through a progressively complex “web spider” example that crawls URLs, parses HTML, writes files to disk, and handles recursive link traversal.

The patterns themselves aren’t that hard. But the spider mixes four concerns at once: HTTP requests, filesystem I/O, URL parsing, and the control-flow pattern you’re actually trying to learn. I kept re-reading the same code and feeling like I understood it, but when I tried to write something from scratch, nothing stuck.

So I took a different approach: strip away everything except the control-flow pattern and build it myself, step by step, with Claude Code as my guide. Instead of a web spider, I used a trivially simple fake async function fetchUser(id, cb) that simulates a database call with setTimeout. No HTTP. No filesystem. No URL parsing. Just the pattern.

It worked. The patterns finally clicked. In this post (Part 1 of 2), I’ll walk you through the four callback exercises I used to internalize chapter 4. In Part 2, I’ll rebuild the same exercises with promises and async/await and show how much simpler they become.

The setup: a fake async function

Every exercise in this series uses the same tiny function:

const fetchUser = (id, cb) => {
  if (id < 0) {
    const err = new Error(`Invalid id: ${id}`)
    return process.nextTick(() => cb(err))
  }

  setTimeout(
    () => {
      cb(null, { id, name: 'User ' + id })
    },
    100 + Math.random() * 300,
  )
}

It takes an id and a callback, waits 100-400ms to simulate network latency, and returns a fake user object. If the id is negative, it errors. That’s it.

But even this 10-line function has a subtlety worth understanding before we move on.

The Zalgo problem

Notice that the error branch uses process.nextTick(() => cb(err)) instead of just cb(err). Why not call the callback immediately?

Because the success branch calls the callback asynchronously (inside a setTimeout). If the error branch called it synchronously, then fetchUser would be sometimes sync, sometimes async depending on its input. This is known as “releasing Zalgo”: a function that unpredictably switches between sync and async behavior is nightmarish to reason about.

Here’s the problem it causes. Consider this caller:

console.log('before')
fetchUser(-1, (err) => console.log('callback:', err.message))
console.log('after')

With a synchronous error branch, the output would be:

before
callback: Invalid id
after

The callback fires before console.log('after') runs. But with a positive id (async branch), the output would be:

before
after
callback: User 1

Same function, different ordering depending on the input. Code that works after the function call might run before or after the callback, unpredictably. That’s Zalgo.

The rule is simple: an async function must be async on every code path, always. Wrapping the error callback in process.nextTick guarantees that the callback is always deferred to the next iteration of the event loop, regardless of which branch executes.

Exercise 1: The error-first callback

Before tackling any control-flow patterns, I made sure I could write and consume an error-first callback correctly. The exercise: call fetchUser three times. Once with a valid id, once with an invalid id, and once more with a valid id, and handle the results.

const cb = (err, result) => {
  if (err) {
    console.error(err)
  } else {
    console.log(`Got user: ${result.id} - ${result.name}`)
  }
}

fetchUser(1, cb)
fetchUser(-5, cb)
fetchUser(2, cb)

Output:

Error: Invalid id
Got user: 1 - User 1
Got user: 2 - User 2

Two things to notice:

1. The error-first convention. When there’s no error, we pass null as the first argument: cb(null, user). When there is an error, we pass only the error: cb(err). The callback always checks err first. This convention is baked into Node.js and every library that uses callbacks. Ugly af.

2. The output order. The error prints first even though fetchUser(-5) is the second call. Why? Because process.nextTick callbacks run before setTimeout callbacks in the event loop. The error (deferred via nextTick) fires before either success callback (deferred via setTimeout). The order isn’t “order of invocation” - it’s “order of event-loop scheduling.”

Simple exercise, but the foundation for everything that follows. If this is solid, the rest builds on top.

Exercise 2: Sequential execution

Now the real work begins. The goal: fetch a list of users one at a time, each call waiting for the previous one to finish. This is the “sequential iteration pattern” from chapter 4 of the book.

The challenge

Write a function fetchUsersSequentially(ids, finalCb) that:

Takes an array of ids (e.g., [1, 2, 3, 4, 5]) and a final callback
Calls fetchUser on each id one after another. Each call must wait for the previous one to finish.
When all are done, calls finalCb(null, results) with an array of user objects in the same order as ids
If any call fails, stops immediately and calls finalCb(err)

Hints (try it yourself first)

The canonical way to do sequential iteration with callbacks is recursion via an iterator function:

Define an inner function iterate(index) that checks if index === ids.length. If yes, we’re done, so call finalCb(null, results).
Otherwise, call fetchUser(ids[index], (err, user) => { ... }). In that inner callback: on error, call finalCb(err) and return. Otherwise, push user into results and call iterate(index + 1).
Kick it off with iterate(0).

Do not use a for loop! With callbacks, a for loop would fire all the requests at once (parallel), not sequentially. That’s Exercise 3.

My solution

const fetchUsersSequentially = (ids, finalCb) => {
  const results = []

  const iterate = (index) => {
    if (index === ids.length) {
      return finalCb(null, results)
    }

    fetchUser(ids[index], (err, result) => {
      if (err) {
        return finalCb(err)
      }
      results.push(result)
      iterate(index + 1)
    })
  }

  iterate(0)
}

The driver code:

console.time('sequential')
fetchUsersSequentially([1, 2, 3, 4, 5], (err, results) => {
  console.timeEnd('sequential')
  if (err) return console.error(err)
  for (const user of results) console.log(`Got user: ${user.id} - ${user.name}`)
})

Output:

Got user: 1 - User 1
Got user: 2 - User 2
Got user: 3 - User 3
Got user: 4 - User 4
Got user: 5 - User 5
sequential: 1.472s

Five sequential fetches, each averaging ~250ms. Total: ~1.5s. Remember this number, we’ll compare it against parallel execution.

Style note: “done” check at the top

Putting the if (index === ids.length) check at the top of iterate (rather than checking “is this the last one?” after fetching) handles the edge case of an empty array naturally. If ids is [], iterate(0) immediately sees 0 === 0 and calls finalCb(null, []). With the check at the bottom, you’d call fetchUser(undefined, ...) - a latent bug.

Exercise 3: Unlimited parallel execution

Now the opposite: fetch all users at the same time and gather the results when everything’s done.

The challenge

Write fetchUsersInParallel(ids, finalCb) with the same contract finalCb(err, results), results in input order, fires exactly once. But kick off all fetches immediately.

Hints

Use a for loop to kick off all fetches at once.
Track completions with a completed counter. When it equals ids.length, call finalCb(null, results).
Use results[i] = user (not .push()) to preserve input order despite out-of-order completions.
Use a hasError flag to ensure finalCb is called at most once after an error.

My solution

const fetchUsersInParallel = (ids, finalCb) => {
  const results = new Array(ids.length)
  let completed = 0
  let hasError = false

  for (let i = 0; i < ids.length; i++) {
    fetchUser(ids[i], (err, result) => {
      if (hasError) return
      if (err) {
        hasError = true
        return finalCb(err)
      }
      results[i] = result
      if (++completed === ids.length) finalCb(null, results)
    })
  }
}

Output (success case):

parallel-success: 309ms
Got user: 1 - User 1
Got user: 2 - User 2
Got user: 3 - User 3
Got user: 4 - User 4
Got user: 5 - User 5

309ms for five fetches, versus ~1.5s sequential. The speedup is almost 5x! All five ran concurrently, bottlenecked only by the slowest one. That’s the whole point of parallel.

What I learned: why `hasError` must be checked at the top

My first attempt had the hasError check in the wrong place. I only checked !hasError on the success path:

// Buggy version
fetchUser(ids[i], (err, result) => {
  if (err) {
    hasError = true
    return finalCb(err)
  }
  results[i] = result
  if (++completed === ids.length && !hasError) finalCb(null, results)
  //                                ^^^^^^^^^ only checked here
})

This seemed reasonable. If an error occurred, completed would never reach ids.length (error’d callbacks don’t increment it), so the success callback would never fire. And that’s true. But what about multiple errors?

With [1, -2, -3, 4]:

fetchUser(-2) errors → hasError = true, finalCb(err). First call. Good.
fetchUser(-3) errors → hasError = true (already was), finalCb(err). Second call. Bad!

finalCb fired twice. The flag didn’t protect the error path from other errors.

The fix: check hasError at the very top of the callback, before anything else. This drops all late-arriving callbacks, no matter if they’re late errors or late successes:

fetchUser(ids[i], (err, result) => {
  if (hasError) return // drop everything after first error
  if (err) {
    hasError = true
    return finalCb(err)
  }
  results[i] = result
  if (++completed === ids.length) finalCb(null, results)
})

This is the kind of bug that only manifests with specific input patterns (multiple errors) and specific timing. In a real system, it would show up as intermittent double-responses, corrupted state, or, in the worst case, charging a credit card twice. The lesson: with callbacks, every single code path through the callback must be explicitly guarded. There’s no built-in safety net.

Why `let` matters in the for loop

A subtle but critical detail: the loop uses let i, not var i. With let, each iteration gets its own binding of i captured by the closure. With var, every callback would close over the same i variable (which would be 5 by the time any callback fires), and you’d write every result to results[5]. Use let.

Exercise 4: Limited parallel execution

This is the hardest exercise and the most important one. It’s where the TaskQueue class in the book comes from, and it’s the pattern behind database connection pools, http.Agent maxSockets, worker pools, and every other concurrency limiter you’ll encounter in production.

Why concurrency limits exist

In Exercise 3, we ran 5 fetches in parallel. That’s fine. But what if you had 1000?

Every concurrent operation holds real resources:

File descriptors - each open socket, file, or pipe is a file descriptor in Unix. Your process has a hard cap (ulimit -n, commonly 1024, for my MacBook it’s 2048). Opening one past the limit crashes with EMFILE.
Database connections - each connection is a live TCP socket plus a backend process/thread on the DB server. Postgres defaults to max_connections = 100. Hit that limit and your next connection attempt fails (and you’re not the only client).
Memory - each in-flight operation has buffers, closures, and intermediate state sitting in RAM.
Remote rate limits - the server you’re talking to has its own caps. Hammer it hard enough and you get 429s, 503s, or bans.
Bandwidth - 1000 concurrent downloads through a shared pipe each get 1/1000th of the bandwidth, so they all finish slowly.

There’s also a less obvious reason: throughput is not a monotonic function of concurrency. For most real workloads, throughput rises as you add concurrency up to some optimum (where your bottleneck resource is fully utilized), then falls as contention overhead outpaces the gains. A hundred workers fighting over one lock get less done than ten workers doing the same job.

The limited-parallel pattern isn’t a Node quirk. It’s the right shape for talking to any finite resource, in any language.

The challenge

Write fetchUsersWithConcurrency(ids, concurrency, finalCb), same contract as before, but at most concurrency fetches may be in flight at any time.

Hints

You need four state variables:

results - pre-allocated new Array(ids.length)
nextIndex - the index of the next id to fetch (starts at 0)
running - how many fetches are currently in flight (starts at 0)
completed - how many have finished successfully (starts at 0)

And one internal helper, tryNext(), with a while loop:

while (running < concurrency AND nextIndex < ids.length AND not hasError):
    capture nextIndex into a local variable
    increment nextIndex and running
    call fetchUser, and in the callback:
        decrement running
        handle errors (with hasError guard)
        write result to results[i]
        check if completed === ids.length
        call tryNext() to fill the freed slot

The while loop handles both “first call launches a burst” and “subsequent calls top up one at a time.”

My solution

const fetchUsersWithConcurrency = (ids, concurrency, finalCb) => {
  const results = new Array(ids.length)
  let nextIndex = 0
  let running = 0
  let completed = 0
  let hasError = false

  const tryNext = () => {
    while (running < concurrency && nextIndex < ids.length && !hasError) {
      const i = nextIndex
      nextIndex++
      running++

      fetchUser(ids[i], (err, result) => {
        running--
        if (hasError) return
        if (err) {
          hasError = true
          return finalCb(err)
        }
        results[i] = result
        if (++completed === ids.length) return finalCb(null, results)
        tryNext()
      })
    }
  }

  tryNext()
}

With logging added to fetchUser, the output with 20 ids and concurrency 3:

  → start fetch 1
  → start fetch 2
  → start fetch 3
  ← done   fetch 1
  → start fetch 4
  ← done   fetch 3
  → start fetch 5
  ← done   fetch 2
  → start fetch 6
  ...
  ← done   fetch 19
  ← done   fetch 20
limited-parallel: 1.765s
Fetched 20 users

You can see the rhythm: exactly 3 → start lines before any ← done, then each completion immediately triggers a new start. The pipeline stays full.

Timing: 1.765s for 20 tasks with concurrency 3. Compare:

Fully parallel (concurrency = 20): ~400ms
Fully sequential (concurrency = 1): ~5s
Limited parallel (concurrency = 3): ~1.8s

Limited parallel sits between the two - trading throughput for resource bounds. Real systems use this every day.

What I learned: `nextIndex` vs `completed`

My first version used completed < ids.length as the scheduling guard instead of nextIndex < ids.length. It seemed equivalent: “keep going until we’re done.” It wasn’t.

The difference:

nextIndex tracks how many tasks have been kicked off (scheduled)
completed tracks how many tasks have finished

When you’ve scheduled all 20 tasks but only 17 have completed, completed < ids.length is still true, so the while loop enters and tries to schedule task #21, which is ids[20], which is undefined.

Here’s the trace from my buggy version:

  ← done   fetch 17
  → start fetch undefined    ← phantom task!
  ← done   fetch 19
  → start fetch undefined    ← another one!
  ← done   fetch undefined
limited-parallel: 1.791s
Fetched 21 users               ← wrong count!

The → start fetch undefined lines are phantom tasks because the scheduler tried to fetch beyond the end of the array. The Fetched 21 users is because results[20] was written by a phantom fetch, extending the array past its original length.

The fix: use nextIndex < ids.length in the scheduling guard. “Should I schedule more?” is strictly about the input queue, not about the state of the world.

Every task flows through three states:

       unscheduled         →    in flight    →    completed
(ids.length - nextIndex)        (running)        (completed)

The while loop moves tasks from unscheduled to in flight. Its guard must ask “is there anything left to schedule?” (nextIndex < ids.length), not “is everything done?” (completed < ids.length).

Wrapping up Part 1

Here’s what I had to manually juggle with callbacks across these four exercises:

Rule	What goes wrong if you break it
Both code paths must be async (no Zalgo)	Callers can’t predict whether their code runs before or after the callback
`return` after calling `finalCb(err)`	The success path also fires, leading to double result
`hasError` flag at the top of every callback	`finalCb` fires multiple times on concurrent errors
`nextIndex` (not `completed`) for scheduling	Phantom tasks scheduled beyond the end of the array
`finalCb` called exactly once per invocation	Consumers see duplicate responses, corrupted state

Every one of these is a rule you, the programmer, must remember and implement correctly. The language doesn’t enforce any of them. A missing return, a misplaced flag check, or the wrong counter in a guard, any one of these can cause a bug that only surfaces under specific input patterns and specific timing.

In Part 2, I rebuild all four exercises with promises and async/await. Every rule in the table above becomes a language-enforced guarantee that you couldn’t violate if you tried. The code gets shorter, the bugs get impossible, and the same patterns become almost trivially simple to express.

Originally published at nop33.com.

Syncing Obsidian Across Devices with Syncthing and a Home Server

Ilias Trichopoulos — Fri, 10 Apr 2026 00:00:00 GMT

I use Obsidian for all my notes: projects, recipes, finances, travel plans, everything. OK, that’s a lie. I use Notion as well. I’ve been trying to migrate over to Obsidian for years. The main reason why I haven’t yet done it is a seamless sync between my MacBook and my Android phone without relying on a cloud service. Obsidian Sync exists, but I already have a home server running 24/7. So, why not use it?

Syncthing is an open-source, peer-to-peer file synchronization tool. No central server, no account, end-to-end encrypted. Exactly what I needed.

My setup

My home server is an HP EliteDesk running Proxmox VE. Inside it:

LXC 100 - Caddy reverse proxy (192.168.21.51)
VM 101 - production-docker (192.168.21.52) running Docker with a UFW firewall

I already use this setup to host Calibre Web for my ebook library and Umami for website analytics. A 40 GB virtual drive called “Dionysus” (thanks Yann), mounted at /dionysus, stores all persistent data.

The plan: run Syncthing as a Docker container on VM 101, store the vault on Dionysus alongside my Calibre library, and sync to my MacBook and phone. The server acts as an always-on hub. Even if my laptop is asleep when I edit a note on my phone, the changes flow through the server and sync to the MacBook when it wakes up.

Step 1: Syncthing on the server

I created directories for the Syncthing config and the vault data:

mkdir -p /home/ilias/docker/syncthing/config
mkdir -p /dionysus/obsidian/vault

The config lives alongside my other Docker services (/home/ilias/docker/syncthing/), while the actual vault data goes on the Dionysus drive.

Here’s the docker-compose.yml:

services:
  syncthing:
    image: syncthing/syncthing:latest
    container_name: syncthing
    hostname: hp-elitedesk-syncthing
    environment:
      - PUID=1000
      - PGID=1000
    volumes:
      - ./config:/var/syncthing/config
      - /dionysus/obsidian/vault:/var/syncthing/data
    network_mode: host
    restart: unless-stopped
    healthcheck:
      test: curl -fkLsS -m 2 127.0.0.1:8384/rest/noauth/health | grep -o --color=never OK || exit 1
      interval: 1m
      timeout: 10s
      retries: 3

network_mode: host is important: Docker’s default bridge networking breaks Syncthing’s local device discovery. Without it, devices on the same LAN won’t find each other automatically.

cd /home/ilias/docker/syncthing
docker compose up -d

Step 2: Firewall rules

Syncthing needs a few ports open on the VM:

sudo ufw allow 22000/tcp comment 'Syncthing sync'
sudo ufw allow 22000/udp comment 'Syncthing QUIC sync'
sudo ufw allow 21027/udp comment 'Syncthing local discovery'
sudo ufw allow from 192.168.21.0/24 to any port 8384 proto tcp comment 'Syncthing GUI (LAN only)'

Port	Protocol	Purpose
22000	TCP+UDP	Sync protocol + QUIC
21027	UDP	Local discovery
8384	TCP	Web GUI (LAN only)

The GUI is restricted to the local network. For remote access, I use Caddy (more on that later).

Step 3: Secure the GUI

The Syncthing web GUI is available at http://192.168.21.52:8384. First thing: set a username and password under Actions > Settings > GUI. Then note the Device ID. You’ll need it to connect other devices.

Step 4: macOS setup

On my MacBook:

brew install syncthing
brew services start syncthing

The GUI opens at http://127.0.0.1:8384. I added the server’s Device ID via + Add Remote Device, then accepted the connection on the server side.

Step 5: Share the vault

On the server GUI, I created a shared folder:

Folder Label: Obsidian Vault
Folder ID: obsidian-vault (must match exactly on all devices)
Folder Path: /var/syncthing/data (maps to /dionysus/obsidian/vault on the host)
Folder Type: Send & Receive
File Versioning: Staggered (keeps recent versions frequently, older versions at increasing intervals. This is a safety net against accidental deletions propagating everywhere)

On the MacBook I accepted the folder and pointed it to ~/ObsidianVault. Since the vault already existed on my MacBook, Syncthing detected the matching files and skipped re-transferring them.

Step 6: Android setup

The original Syncthing Android app is deprecated. Install Syncthing-Fork (by catfriend1) from F-Droid.

Critical Android settings:

Disable battery optimization: Android Settings > Apps > Syncthing-Fork > Battery > Unrestricted. Without this, Android kills the app in the background and sync stops.
Grant All Files Access: needed on Android 11+ to sync files outside the app’s own directory.

Then add the server’s Device ID, accept on the server side, and accept the shared folder pointing to /storage/emulated/0/ObsidianVault. Install Obsidian from the Play Store and open the vault from that path.

Step 7: The .stignore file

Syncthing’s .stignore file works like .gitignore. It tells Syncthing which files to skip. This file is not synced between devices, so you need to create it on each one separately, in the root of the vault.

// Obsidian workspace state (device-specific, changes constantly)
.obsidian/workspace.json
.obsidian/workspace-mobile.json

// OS junk
.DS_Store
Thumbs.db
._*

// Temp/lock files
*.tmp
~*

The workspace.json ignore is the most important one. This file changes every time you click anything in Obsidian. Without ignoring it, you’d get constant sync activity and frequent conflicts.

Step 8: Remote access

Direct sync from anywhere

On my local network, devices connect directly. But away from home, Syncthing falls back to relay servers. They are encrypted, but slower. To enable direct connections from anywhere, I added a port forwarding rule on my Unifi UDR7 router:

Port: 22000 (TCP + UDP)
Forward to: 192.168.21.52:22000

Web GUI via Caddy

I already have Caddy reverse-proxying other self-hosted services. Adding the Syncthing GUI was one more block in the Caddyfile:

syncthing.example.com {
        header {
                X-Content-Type-Options nosniff
                X-Frame-Options DENY
                X-XSS-Protection "1; mode=block"
        }
        reverse_proxy 192.168.21.52:8384 {
                header_up Host {upstream_hostport}
                header_up X-Scheme https
        }
}

After adding a DNS record in Cloudflare and reloading Caddy, the GUI was accessible from anywhere.

The result

I now have my Obsidian vault syncing across three devices with no cloud dependency:

Home server - always-on hub, stores the vault on a dedicated virtual drive
MacBook - full bidirectional sync
Android phone - full bidirectional sync

Editing a note on my phone while on the go, then picking up right where I left off on my laptop at home. It just works! And if anything goes wrong, staggered file versioning keeps old versions around for recovery.

Total cost: a bit of time setting it up, and zero monthly fees.

I wonder, however, how much battery the Syncthing-Fork app will consume on my phone. To be continued…

Originally published at nop33.com.

How I Shrunk a Blockchain SDK by 88%: Modernizing @alephium/web3

Ilias Trichopoulos — Thu, 09 Apr 2026 00:00:00 GMT

The @alephium/web3 TypeScript SDK is how developers connect to and interact with the Alephium blockchain. It’s similar to what viem is for Ethereum. It provides utilities like address validation, transaction signing, smart contract interaction, and API communication.

But it had a problem. A big, 742 kB problem.

The starting point

A developer who imported a single function from our SDK, say, isValidAddress, paid for the entire library. A plain website with one text input and one SDK call produced a 742 kB JavaScript bundle. A React app? 928 kB. Getting it to work in React Native required 12 workarounds including custom Metro resolvers, native crypto modules, and a dev build (Expo Go wouldn’t work).

The root causes were typical of TypeScript libraries built a few years ago:

No ESM output - strictly CommonJS, defeating tree-shaking
Monolithic UMD browser bundle - webpack squashed everything into one opaque file
Heavy Node.js polyfills - crypto-browserify, stream-browserify, buffer shipped as dependencies
Redundant crypto libraries - both elliptic and @noble/secp256k1
A global side effect - BigInt.prototype.toJSON was monkey-patched on import, making "sideEffects": false impossible

Before I touched any code: benchmarking

Before changing anything, I built four benchmark apps. The idea was to have the same code running in Node.js, a Vite website, a Vite + React webapp, and an Expo (React Native) app. Each one imported isValidAddress, validated an address, and fetched a balance. This gave me concrete “before” numbers to measure against.

The Expo app was the most revealing. Getting it to work required: a custom entry point, a Buffer polyfill, react-native-quick-crypto (because crypto-browserify crashed at module evaluation time), readable-stream, path-browserify, events, process, an empty fs shim, a custom Metro resolver to bypass the UMD bundle (which referenced self, undefined in React Native), extraNodeModules for 6 builtins, node-linker=hoisted for pnpm, and expo-dev-client because Expo Go couldn’t handle the native crypto module.

That’s 12 workarounds just to call one function.

Phase 1: The diet

The first phase was about removing what we no longer need, without changing the build system.

Bumping Node.js to >= 20

Node 14 was the minimum. Node 14 is EOL. Bumping to 20 unlocked native fetch (goodbye cross-fetch) and globalThis.crypto.subtle (goodbye Node crypto imports). I originally tried Node >= 18 but discovered that crypto.subtle is undefined on Node 18 in non-secure contexts. But since Node 18 is also EOL, Node 20 was a better choice.

Consolidating on @noble

The SDK used both elliptic (legacy, huge, pulls in bn.js) and @noble/secp256k1 (modern, audited, zero-dependency). I replaced elliptic with @noble/secp256k1 for secp256k1, added @noble/curves for P-256 and Ed25519, and replaced blakejs with @noble/hashes/blake2b.

For the wallet package, I replaced bip39 with @scure/bip39 and bip32 with @scure/bip32, both from the same @noble/@scure ecosystem that viem uses. This eliminated the entire noble-wrapper.ts adapter file that existed solely to bridge bip32’s API with @noble.

Removing the BigInt monkey-patch

The BigInt.prototype.toJSON mutation was a global side effect. Any consumer importing anything from the SDK got this prototype modification. It made declaring "sideEffects": false impossible, which in turn made tree-shaking impossible.

I replaced it with a stringify utility (inspired by viem’s approach), which is a drop-in JSON.stringify replacement that converts BigInt to strings via a replacer function. The swagger-typescript-api template was updated to use stringify automatically in the generated TypeScript definition API files.

Dependencies went from 11 to 6 for @alephium/web3, and from 8 to 4 for @alephium/web3-wallet. Huge win already 💪

Phase 2: The build system journey

This is where things got interesting and where I learnt the most.

Attempt 1: tsup

My first idea was to use tsup. tsup uses esbuild under the hood and promises dead-simple dual CJS/ESM output. I got it working, but hit problems:

bundle: true inlined all dependencies into a single file, which is the opposite of what I wanted for tree-shaking
bundle: false with .cjs/.mjs extensions broke Node’s CJS directory resolution (require('./api') doesn’t check .cjs files)
bundle: false with .js/.mjs extensions broke Vite’s dev server because bare import paths like "./api" in .mjs files resolved to .js (CJS) instead of .mjs

Attempt 2: tsup with workarounds

I tried various combinations: explicit file paths in barrel exports, resolve.conditions in Vite config, optimizeDeps.include, bundled ESM with externalized deps. Each fix introduced a new problem.

The solution: tsc dual-build (like viem)

I looked at how viem does it. They don’t use tsup or any build tool wrapper. They simply use plain tsc twice:

CJS pass: tsc --module commonjs --outDir dist/_cjs + {"type":"commonjs"} in dist/_cjs/package.json
ESM pass: tsc --outDir dist/_esm + {"type":"module","sideEffects":false} in dist/_esm/package.json

That’s quite a smart a clean approach 🧠

All files use .js extension. The nested package.json determines interpretation. No extension confusion, no resolver hacks.

But I hit one more issue: strict ESM resolvers (Vitest, Node.js with "type": "module") require explicit .js extensions in import paths. tsc doesn’t rewrite import specifiers - from './api' stays as from './api' in the output, which fails in strict ESM because there’s no ./api.js file (it’s ./api/index.js). The solution to this woudl require a massive refactoring of all import statements around the code base. It didn’t feel right to me to have a TS codebase full of .js imports (even though that’s what viem does).

My solution: I decided to use tsc-alias as a post-build tool that rewrites the ESM output to add .js extensions (./api → ./api/index.js). Source TS files stay clean. CJS output doesn’t need it. One line added to the build script.

Phase 3: Package configuration

Getting the package.json right is surprisingly nuanced:

{
  "type": "commonjs",
  "sideEffects": false,
  "main": "dist/_cjs/index.js",
  "module": "dist/_esm/index.js",
  "types": "dist/_cjs/index.d.ts",
  "exports": {
    ".": {
      "import": {
        "default": "./dist/_esm/index.js",
        "types": "./dist/_esm/index.d.ts"
      },
      "require": {
        "default": "./dist/_cjs/index.js",
        "types": "./dist/_cjs/index.d.ts"
      }
    },
    "./api/explorer": { "..." },
    "./api/node": { "..." }
  }
}

Each field serves a different consumer: main for legacy CJS, module for legacy bundlers, types for TypeScript with moduleResolution: "node", and exports for everything modern. The import/require conditions each have their own types entry: CJS declarations in _cjs/, ESM declarations in _esm/. This avoids the FalseCJS type issue.

I added publint and @arethetypeswrong/cli to validate the configuration. Not even viem passes attw for strict node16 ESM resolution. It’s a known limitation of dual CJS/ESM packages when tsc doesn’t rewrite import paths in .d.ts files. I decided to ignore node16 and move forward.

TypeScript 5.9

Upgrading from TypeScript 4.9 to 5.9 across the monorepo was straightforward but gave me moduleResolution: "bundler" which properly resolves exports fields in package.json. I also migrated from Jest to Vitest, which handles ESM natively. No more transformIgnorePatterns hacks for @noble and @scure packages. However, upgrading @noble/hashes and @noble/curves to v2 (ESM-only) is still blocked by the dual CJS/ESM build: the CJS build uses --moduleResolution node which can’t resolve subpath exports from ESM-only packages. That upgrade waits for either dropping CJS or bumping to Node 22+ with --module nodenext. Node 20 reaches EOL in April 2026 (current month as of writing). So, soon I can upgrade those libraries to v2.

The results

I tested with the actual benchmark apps, using packages published to a local Verdaccio registry (exactly as consumers experience from npm):

App	v2.0.10	v3.0.0	Reduction
Website (vanilla JS)	1,697 kB	207 kB	-88%
Webapp (React)	1,904 kB	407 kB	-79%

And in the real production apps from the alephium-frontend monorepo:

App	Before	After	Reduction
Explorer: `index.js` chunk	2,480 kB	730 kB	-71%
Desktop wallet	7,740 kB	4,944 kB	-36%
Mobile wallet (iOS)	22.5 MB	21.3 MB	-5.3%

The explorer also removed rollup-plugin-node-polyfills since it’s no longer needed. The mobile wallet’s improvement is more modest because the SDK is a smaller fraction of the total React Native bundle.

For Expo/React Native, the setup went from 12 workarounds to 2: react-native-get-random-values (for @noble/secp256k1) and an empty fs shim (Metro resolves dynamic imports statically). Expo Go works and no dev build required.

A colleague rightfully questioned the need for an empty fs shim. The reason it is needed is that the SDK uses the Node.js builtin fs module for 2 functions. As long as these 2 functions are not consumed, the Vite project or the vanilla JS website project do not complain. The Metro bundler of the Expo project, however, imports everything at ones before doing the static analysis, and it throws an exception, since fs is not available. Creating an empty shim solved this problem. After some more investigation, I decided to follow the isomorphic-git example, by simply providing the required method of the fs module (in my case that was readFile) as a parameter to these 2 functions that required the fs module. That way, any consumer of those 2 functions will simply have to pass fs.readFile as an argument to them. This completely eliviated the need for a shim, leading to smoother devX when integrating the SDK.

What I’d do differently

Start with tsc, not tsup. I spent significant time debugging tsup’s ESM resolution quirks before realizing that viem’s plain-tsc approach avoids them entirely. Build tool wrappers add convenience but also abstraction. When something breaks, you’re debugging the wrapper’s behavior, not your code.

Benchmark with published packages from day one. I initially tested with pnpm link and tarball installs, which behave differently from real npm installs. Setting up Verdaccio early would have caught issues sooner. Frontend development with multiple shared local packages is still pain.

Don’t underestimate package.json complexity. Getting exports, types, main, module, type, and sideEffects right for dual CJS/ESM is genuinely hard. Tools like publint and attw help, but even the reference libraries (viem) don’t pass every check.

What’s next

Drop CJS or bump to Node 22+. This will unlock @noble v2 / @scure v2 (ESM-only packages), currently blocked by the CJS build’s --moduleResolution node
Sub-path exports - @alephium/web3/codec, @alephium/web3/address, etc. for even more granular tree-shaking

The full roadmap, benchmark data, and reference apps are available in the web3-rewrite repo. Lastly, I wrote a migration guide that explains how to go from v2 to v3.

Originally published at nop33.com.

Ilias Trichopoulos's Blog

How to Combine TanStack Query Data in React

The problem

1. Combine in the component

2. Combine in combine of useQueries

3. Combine in queryFn with Promise.all

4. Combine in queryFn with fetchQuery

Which one to reach for

Conclusion

Reference of community discussions

Instant Cold Starts with a Persisted TanStack Query Cache (Part 5)

Persisting the cache for instant cold starts

Design choices

The cold-start payoff

Persistence trade-offs

Summary

The series, measured

Final words

Derived Wallet State Without Melting the CPU (TanStack Query, Part 4)

The problem

Wallet worth calculation

The symptom

The problem, in code

Fix 1: compute once, cache the result

Fix 2: hoist the fan-out into a context

Why two tools, not one

An alternative I weighed: aggregates as cached queries

A direction I want to explore: jotai

Trade-offs and what I would improve

Learnings

Own your coherence when you leave the library’s guarantees

Compute once, cache many, invalidate precisely

My constants are calibrated by vibes

What comes next

Cut Wallet API Calls with Change Detection in TanStack Query (Part 3)

Change detection

Side note: A polling strategy per address

Freshness without staleTime

Where this diverges from the docs (and why that matters)

Trade-offs and what I would improve

What comes next

You Can't Wipe a Secret in JavaScript, but You Should Try

The threat model

Why JavaScript makes this genuinely hard

Step 1: get the secrets out of Redux

Step 2: stop storing the mnemonic as a string

Step 3: the part where I found the mnemonic in RAM anyway

Step 4: wipe on every path, especially the failing ones

Step 5: don’t forget the exits

What this actually buys you (and what it doesn’t)

Takeaways

Migrating Redux Server State to TanStack Query (Part 2)

Redux was doing a job it was never built for

Server state is not client state

The detour: we tried RTK Query first

The after: the component asks for what it needs

Migrating without stopping the world

The hesitation that almost kept me on Redux

What comes next

Why Async State Broke Our Crypto Wallets (TanStack Query Case Study, Part 1)

The apps

Problem 1: the request flow is hard to reason about

Problem 2: too many requests

Problem 3: painfully slow data updates on app launch

Network requests batching

What comes next

Speeding Up Cold Start of a React Native app

1. The Gatekeeper: Global inlineRequires

What this actually does

Profiling in Android Studio

Why does changing the Metro bundler configuration suddenly wake up the native Android thread?

Step-by-step anatomy of a RN cold start

Phase 1: Compile

Phase 2: App launch (cold start)

Step 1: Native process forking (0.0s)

Step 2: Native splash screen mounts (~0.2s)

Step 3: The JS thread spawns (~0.4s)

Step 4: Grobal script evaluation

Phase 3: Handover and lazy runtime evaluation

Step 5: Unlocking the bridge (~1.5s)

2. Combine in `combine` of `useQueries`

3. Combine in `queryFn` with `Promise.all`

4. Combine in `queryFn` with `fetchQuery`

1. The Gatekeeper: Global `inlineRequires`

The `React.lazy` trap: Why web optimizations fail on mobile

The `.forEach` trap

Exercise 6: Parallel with `Promise.all`

The `.map(fn)` gotcha

What I learned: why `hasError` must be checked at the top

Why `let` matters in the for loop

What I learned: `nextIndex` vs `completed`