@@W-21951736@@ - Integrate MRT Data Access Layer - #3787
Conversation
|
Git2Gus App is installed but the |
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
| expect(typeof data.__ERROR__.stack).toBe('string') | ||
| }) | ||
|
|
||
| test('includes serialized custom site and global preferences in #mobify-data', async () => { |
There was a problem hiding this comment.
Should we test the opposite that the MRT data store is disabled by default?
We can keep this test but I think we want to make sure to override the default value to make sure the flag is enabled before running the test:
process.env.PWAKIT_MRT_DATA_STORE_ENABLED = 'true'
There was a problem hiding this comment.
I assume we don't want to commit these plan md files.
There was a problem hiding this comment.
Correct. I'll be cleaning up a few files before merging.
There was a problem hiding this comment.
I assume we don't want to commit these plan md files.
| // MRT Data Store (opt-in): when disabled, skip preference resolution and omit `__MRT_DATA_STORE__` from | ||
| // `#mobify-data`. Enable via `app.mrtDataStore.enabled` or `PWAKIT_MRT_DATA_STORE_ENABLED=true`. | ||
| // When enabled, `initializeDataStore` from runtime mirrors the storefront-next flow (provider once, then keys). | ||
| const mrtDataStoreEnabled = isMrtDataStoreEnabled(config) | ||
| let customSitePreferences = {} | ||
| let customGlobalPreferences = {} | ||
| if (mrtDataStoreEnabled) { | ||
| await initializeDataStore() | ||
| ;[customSitePreferences, customGlobalPreferences] = await Promise.all([ | ||
| getCustomSitePreferences({ | ||
| siteId: res.locals.site?.id | ||
| }), | ||
| getCustomGlobalPreferences() | ||
| ]) | ||
| } | ||
|
|
There was a problem hiding this comment.
Is this the best pattern to follow to integrate the data-store feature directly in the react-render pipeline gated using a flag?
A cleaner solution could be to follow the HoC model we use with withReactQuery and withLegacyGetProps. We could create a withMrtDataStore HoC, and use it like withReactQuery(withMrtDataStore(AppConfig), options).
There was a problem hiding this comment.
I wouldn't necessarily say that is a cleaner solution, per say. Using an HoC like data fetching was done for a different reason, which we don't have in the data store requirement. (withReactQuery/withLegacyGetProps was reacting an abstraction to support multiple data providers, we don't have that here, and probably won't ever have that.)
If we wanted to implement this similar to how we are implementing the data fetching we would still have to modify the react-rending pipeline anyway. As mentioned previously the hoc + doInitAppState is an abstraction that allows use to have multiple data providers for state in the application. That isn't what we are trying to achieve here with the MRT data store implementation, e.g. we aren't going to have another provider/source of data store data.
I'm not convinced that what you are suggesting is something we should look into, primarily because it would be extra effort and we don't have intentions of having a second means of applying data store data, so it would probably be overkill.
There was a problem hiding this comment.
The main idea is to avoid introducing the pattern of making direct changes to the react rendering but I see the HoC could be overkill. Should we at least extract the data store logic into a helper to make the performRender easier to read?
| * DAL / Data Store key suffix; full key is `<siteId>${CUSTOM_SITE_PREFERENCES_KEY_SUFFIX}`. | ||
| * Confirm with MRT. | ||
| */ | ||
| export const CUSTOM_SITE_PREFERENCES_KEY_SUFFIX = '-custom-site-preferences' | ||
|
|
||
| /** | ||
| * Full DAL / Data Store key for org/global custom preferences (no site id prefix). | ||
| * Confirm with MRT. | ||
| */ | ||
| export const CUSTOM_GLOBAL_PREFERENCES_DATA_STORE_KEY = 'custom-global-preferences' |
There was a problem hiding this comment.
Do we still need to confirm these constants with MRT? Can we clean the comments?
There was a problem hiding this comment.
I didn't even know those "* Confirm with MRT." comments where in there. No need to confirm, we know those are the values.
| "@aws-sdk/client-dynamodb": "^3.989.0", | ||
| "@aws-sdk/lib-dynamodb": "^3.989.0", | ||
| "@h4ad/serverless-adapter": "4.4.0", | ||
| "@salesforce/mrt-utilities": "0.1.3", |
There was a problem hiding this comment.
nit: Should we use here the caret for the version?
There was a problem hiding this comment.
I think that was pinned because that is the current convention in sf-next and that is the implementation AI was referenced to. I can update that to be more in align with the pwa-kit convensions.
| * @returns {Promise<DataStoreProvider>} | ||
| */ | ||
| export function getDataStore() { | ||
| if (providerPromise) { |
There was a problem hiding this comment.
Should we add a conditional to make sure we have a value before caching? if the first request runs into a race condition where runs befroe the env setup we could end with and empty cached provider.
| if (providerPromise) { | |
| if (providerPromise && hasMrtEnvironment()) { |
There was a problem hiding this comment.
That change would stop us from reusing the cached local/no-op provider when the MRT env trio isn’t set—which is exactly when we still need that cache. In practice Lambda already has env before the handler runs, and tests can reset the cache with resetDataStoreProviderCacheForTests() when they flip env.
| return { | ||
| kind: 'noop', | ||
| async getEntry() { | ||
| return {value: {}} |
There was a problem hiding this comment.
Should we print a warning log for the key that we are returning an empty value for debugging?
There was a problem hiding this comment.
Sure. I can add some logging.
| // MRT Data Store (opt-in): when disabled, skip preference resolution and omit `__MRT_DATA_STORE__` from | ||
| // `#mobify-data`. Enable via `app.mrtDataStore.enabled` or `PWAKIT_MRT_DATA_STORE_ENABLED=true`. | ||
| // When enabled, `initializeDataStore` from runtime mirrors the storefront-next flow (provider once, then keys). | ||
| const mrtDataStoreEnabled = isMrtDataStoreEnabled(config) | ||
| let customSitePreferences = {} | ||
| let customGlobalPreferences = {} | ||
| if (mrtDataStoreEnabled) { | ||
| await initializeDataStore() | ||
| ;[customSitePreferences, customGlobalPreferences] = await Promise.all([ | ||
| getCustomSitePreferences({ | ||
| siteId: res.locals.site?.id | ||
| }), | ||
| getCustomGlobalPreferences() | ||
| ]) | ||
| } | ||
|
|
There was a problem hiding this comment.
The main idea is to avoid introducing the pattern of making direct changes to the react rendering but I see the HoC could be overkill. Should we at least extract the data store logic into a helper to make the performRender easier to read?
| '<rootDir>/node_modules/@h4ad/serverless-adapter/lib/$1/index.cjs' | ||
| '<rootDir>/node_modules/@h4ad/serverless-adapter/lib/$1/index.cjs', | ||
| '^@salesforce/mrt-utilities/data-store$': | ||
| '<rootDir>/node_modules/@salesforce/mrt-utilities/dist/esm/middleware/data-store.js', |
There was a problem hiding this comment.
Does using require.resolve here works instead of reaching into ../node_modules/@salesforce/mrt-utilities/..?
E.g.:
const mrtMiddlewareDataStore = require.resolve(
'@salesforce/mrt-utilities/dist/esm/middleware/data-store.js'
)
There was a problem hiding this comment.
I don't think there would be much of a gain in using that that method. This is what the change would look like as per AI:
## Files affected
| File | Role |
|------|------|
| `packages/pwa-kit-runtime/jest.config.js` | **Only** file that needs an edit. |
Nothing else has to change: imports, tests, and other packages keep the same public specifiers; only how Jest resolves the mapper target changes.
---
## What the change looks like
Introduce one resolved path and reuse it for both entries that currently point at the same file:
```javascript
// eslint-disable-next-line @typescript-eslint/no-var-requires
const base = require('internal-lib-build/configs/jest/jest.config')
// eslint-disable-next-line @typescript-eslint/no-var-requires
const mrtMiddlewareDataStore = require.resolve(
'@salesforce/mrt-utilities/dist/esm/middleware/data-store.js'
)
module.exports = {
...base,
moduleNameMapper: {
...base.moduleNameMapper,
'^@h4ad/serverless-adapter/lib/(.*)$':
'<rootDir>/node_modules/@h4ad/serverless-adapter/lib/$1/index.cjs',
'^@salesforce/mrt-utilities/data-store$': mrtMiddlewareDataStore,
'^@salesforce/mrt-utilities/middleware$': mrtMiddlewareDataStore,
// ... rest unchanged
},
// ...
}
require.resolve returns an absolute path; Jest’s moduleNameMapper accepts that.
If ESLint complains about require.resolve, mirror the same no-var-requires disable you already use for require('internal-lib-build/...') (as in the snippet).
| const StoreLocator = loadable(() => import('./pages/store-locator'), { | ||
| fallback | ||
| }) | ||
| const DemoMrtDataStore = loadable(() => import('./pages/demo-mrt-data-store'), { |
There was a problem hiding this comment.
Are we leaving this demo page in the final template intentionally or was just for the review?
There was a problem hiding this comment.
That was only for the review. It ill be removed when I get the +1
| await initializeDataStore() | ||
| ;[customSitePreferences, customGlobalPreferences] = await Promise.all([ | ||
| getCustomSitePreferences({ | ||
| siteId: res.locals.site?.id | ||
| }), | ||
| getCustomGlobalPreferences() | ||
| ]) |
There was a problem hiding this comment.
It would be nice to know how much performance tax are we paying...
Is it possible to add server timing entries for this?
There was a problem hiding this comment.
Good call out. I've update the code to have performance marks for initialization of the store and its fetching.
Here is what I have found:
- Cold start fetch is about 65ms, with subsequent fetches being 7ms
Please note that this is the reason this feature is opt-in and not opt-out. I expect customers site and global preferences to be a lot larger than our out of the box sites'.
I've update the doc ticket to include an AC covering this topic.
| // Tests import @salesforce/pwa-kit-runtime dist, which requires @salesforce/mrt-utilities/data-store | ||
| // (or legacy middleware). Published CJS entries can contain ESM syntax; map to the compiled ESM | ||
| // DataStore slice (same as pwa-kit-runtime / react-sdk Jest). | ||
| const mrtMiddlewareDataStore = path.join( | ||
| __dirname, | ||
| '..', | ||
| 'pwa-kit-runtime', | ||
| 'node_modules', | ||
| '@salesforce', | ||
| 'mrt-utilities', | ||
| 'dist', | ||
| 'esm', | ||
| 'middleware', | ||
| 'data-store.js' | ||
| ) |
There was a problem hiding this comment.
I feel like something is wrong with our monorepo build pipeline, we shouldn't need to do import via relative paths, but that may require some work inside the monorepo and not a blocking comment for your PR
PR description
What
PWA Kit apps can read custom site preferences and custom global preferences through a single universal API that behaves correctly on the server during SSR and in the browser after hydration—without calling the Data Store from the client.
How it works (high level)
req/resthrough every call—important when many requests run concurrently.#mobify-databootstrap payload under a singlewindowobject (__MRT_DATA_STORE__), with nested fields for site and global preferences—so the client path mirrors what was just rendered on the server.windowobject, so behavior stays consistent after load and navigation for data that is effectively static until the next full document load (unless the app introduces its own refresh pattern later).Why it matters
Notes for reviewers
Testing (Production)
Runtime and React SDK tests cover resolution, async context, serialization shape, and client-side reads; run the full CI suite before merge.
Demo have bee deployed here --> https://zysg-004-sbx-7341468-pwa-kit-da-bdabe0f7.sfdc-ckzqgc-ecom1.exp-delivery-soak.com/us/en-US/demo/mrt-data-store
The BM backing this demo is here --> https://zysg-004.unified.demandware.net/on/demandware.store/Sites-Site/default%3bapp%3d__bm_merchant%3bsite%3dRefArch/ViewBM-Home?csrf_token=4fHRMkph7JkiayN2Y9xHuN6DIS5MFJF5n61IqhZihEXl855RlFBV4jqA9Qd8wmkr-Ki2QgGBBwibNiCANEFVZuLOFw-E0KTQms4EwOuFN1kMuQrf98nS8HkO9VkcOQLHkOSios0nMQHtyoDC5Fuf5Wm-jXhAcgG1vJ_AwJK82HdLdb_Er-4=
NOTE: You can use the above to change site preferences for RefArch and note their effect on the demo site.
Testing (Local)
Start your dev server using the following command:
Next validate that the local data store in properly initialized and the application displays it's data be viewing this page --> http://localhost:3000/global/en-GB/demo/mrt-data-store
Deploying to MRT
Tail Logs