re-frame

https://github.com/Day8/re-frame/blob/master/docs/README.md https://github.com/Day8/re-frame/blob/master/docs/External-Resources.md
Mark Wardle 2021-05-14T14:46:25.461500Z

Hi all. Is there a recommended way to manage an expiring or max-size-based cache in the app-db? I am fetching against a server and it would be nice to cache some results as they are bound to be used again and again, but I could see that a long-running session could accidentally cache too much data? Or am I better using a separate caching mechanism outside of the app-db, but then I lose the benefit of subscriptions.

p-himik 2021-05-14T14:50:46.461600Z

How do you plan to check whether a cached value is still in use or not?

Mark Wardle 2021-05-14T14:53:55.461800Z

I’d simply send an event to refetch in the event of a cache miss. So it could be time-based or LRU etc.

p-himik 2021-05-14T14:56:07.462Z

I don't think that can work. Suppose you have a cached value. You use subscribe to get that value in a view. That subscribe is called at time T. You have TTL set to 10 minutes. At T+10 minutes some periodic process or whatever removes that entry, because there's no way to check that some subscribe uses it. That reaction returned by that call to subscribe gets re-evaluated and now returns nil. There were 0 user actions, but the UI has changed because some cache entry has expired.

Mark Wardle 2021-05-14T15:00:49.462300Z

Thank you - yes that makes sense. I guess I was planning to write a component that knows that if the result is nil to show a busy spinner until it isn’t nil anymore. These would be parameterised subscriptions with a unique identifier for the ‘thing’.

p-himik 2021-05-14T15:11:05.462500Z

But that wouldn't solve the problem. What will re-fetch the data? Otherwise, the spinner will be there forever. You can't solve it without knowing exactly what data is used. Regular caches are plain data storages. app-db is, on the other hand, a data storage wrapped in a Reagent atom, which makes any change to it to automatically propagate everywhere. That's not the case for regular caches, so regular caching approaches simply won't work.

p-himik 2021-05-14T15:13:17.462700Z

You can create some soft of a countdown latch for each and every entry, where the number shows how many clients that particular entry has. When the counter reaches 0, the entry is removed. But it will require quite some work, and will require you to make subscriptions that alter app-db, which is essentially discouraged.

p-himik 2021-05-14T15:16:01.462900Z

I did something similar a few years ago. But it deals with normalized data with a well-defined schema with an additional constraint that IDs of the items that are actually used are stored in a predefined location. With that, I was able to periodically remove everything from the storage whose ID is not in the list. But such an approach won't work in situations where you need to subscribe to random data that's not defined exactly by an ID an that list.

Mark Wardle 2021-05-14T15:16:06.463100Z

Hmmm thank you - that’s really helpful and I appreciate it. Perhaps I’ll just track the size and wipe out the app-db if it exceeds a pre-defined maximum - and the way I already have this configured is to automatically trigger a fetch event if a defined entry is not available anyway. The view components are designed to cope with an entry not being available and can trigger a refetch.

Mark Wardle 2021-05-14T15:16:50.463300Z

Or maybe not bother and see if it is genuinely a real problem!

Mark Wardle 2021-05-14T15:17:04.463500Z

Thank you. Really helpful to talk it through.

p-himik 2021-05-14T15:22:06.463700Z

No problem. Another suggestion - to prevent the view from being in the "loading" state periodically, instead of wiping, re-fetching, and placing data in app-db just fetch and swap the data.

1👍
Mark Wardle 2021-05-14T15:25:34.464Z

Yes. Actually bandwidth being what it is, that is an even better idea. Thank you.

lilactown 2021-05-14T15:31:57.464200Z

you could tie the data to the lifetime of the subscription

lilactown 2021-05-14T15:32:20.464400Z

well, actually, re-frame caches those so maybe not.

lilactown 2021-05-14T15:32:50.464600Z

if you used a normal ratom you definitely could

Mark Wardle 2021-05-14T15:52:46.464800Z

Yes - I had originally wondered whether I could hook into the subscription lifecycle or just do it independently of the app-db. Thank you

lilactown 2021-05-14T16:06:24.465Z

I would try it out

lilactown 2021-05-14T16:07:33.465200Z

Reagent reactions have a notion of "disposal" where once reagent's runtime detects that no one is listening to it, it will remove itself from listening to other reactions/ratoms and optionally run an on-dispose function if present

lilactown 2021-05-14T16:08:23.465400Z

it's not super well documented, and I'm not sure how it would work with with re-frame's subscriptions due to what I mentioned above about how re-frame caches reactions created by subscriptions

lilactown 2021-05-14T16:10:17.465600Z

see make-reaction here: http://reagent-project.github.io/docs/master/reagent.ratom.html#var-make-reaction

Mark Wardle 2021-05-14T16:11:06.465800Z

Ahhh! That looks very helpful! Thank you very much - I’ll have a look!

1
2021-05-14T19:55:08.466900Z

question about async-flow. I'm seeing a situation where two async flows - concurrent "instances" of the same async flow - are interfering with each other. We perform the initial event twice, but then perform the secondary event just once, since the loser of the race sees that the winner already did it. (And the rule says to halt on that event.) One solution could be to thread in a uuid and match on that - to make the :seen? events distinct and only halt if we've seen "our" event. Another approach could be to unwind the async flow rules into callbacks. Any one have an opinion on this? Or see an option I've missed? I'm wondering in particular if async-flow lib already has a way to distinguish one instance of a flow from another.

2021-05-14T19:57:40.467900Z

i do see [:async-flow/id-n ] events coming through, where n identifies one instance of the flow from another. can i tap into that?