superstructor 2021-02-28T06:44:04.030200Z

Alternatively if you don't need legacy browser support -

thheller 2021-02-28T09:49:31.030900Z

has anyone ever implemented this benchmark for re-frame?

thheller 2021-02-28T09:50:08.031500Z

I'm kinda interested to see how the subscribe stuff performs in benchmark conditions

thheller 2021-02-28T09:51:55.032800Z

not interested in the rendering/react aspects at all but I'm interested to see what the performance of mounting/unmounting queries looks like


@thheller Not the same, but there was this (out of date) analysis:


@jacek.schae updates A RealWorld Comparison of Front-End Frameworks every year, and may have done other benchmarks, or would like to.

thheller 2021-02-28T09:56:53.035500Z

yeah I saw that one but I was looking specifically for this benchmark

thheller 2021-02-28T09:57:16.036300Z

mostly because its an easy setup to see some common situations in webapps in benchmark conditions


same thing only newer

thheller 2021-02-28T09:57:38.037100Z

(eg. no app should have 1000 items rendered in a list but in a benchmark its useful to see)


@huxley thanks

thheller 2021-02-28T09:59:33.038600Z

the real world bench is useful for other stuff and probably more useful overall but I'm specifically looking for examples that stress the subscribe mechanisms

thheller 2021-02-28T09:59:52.039200Z

I can set write it myself, just wondering if someone else did before me


shouldn't the results be essentially identical to the reagent?

thheller 2021-02-28T10:03:21.040200Z

the rendering yes but thats not what I'm interested in

thheller 2021-02-28T10:19:05.043100Z

FWIW I did this benchmark for the stuff I'm working on and have two variants. one that would be similar to re-frame (full) and the other pure reagent (light). both perform more or less the same overall but full performs better in updates (eg. select row) but light performs better in mount/unmount (create/clear rows). I would expect re-frame to be somewhat similar in results when compared to reagent.


At some point, I'll free re-frame subscriptions from using reagent reactions

thheller 2021-02-28T10:21:37.045400Z

that would be neato. then I could just plug it into my stuff 🙂


I'll be moving to using one map for all the subscriptions. The keys in this map will be the subscription vectors And the values in the map will be a map representing the subscription, including a dirty? flag, a set of dependent (subscriptions, identified by their subscription vectors) s, etc


So simple

thheller 2021-02-28T10:23:54.047400Z

yeah. as far as I understand one "common" bottleneck for re-frame apps is when one modification results in too many subscription updates

thheller 2021-02-28T10:24:28.048200Z

eg. in a map of a thousand items you update one and 1000 subscriptions run to figure out if they are dirty. which is fast as long as its just a (get db thing) and compares identical? for 999 items

thheller 2021-02-28T10:25:19.048800Z

but if you actually compute something that becomes a bottleneck




We've never found it to be much of an issue


Because the identical? test shortcuts early


And because we don't display 1000 things (elements of a vector?)


My proposed rework of subscriptions is more about clean up and doing it a much simpler way


Hey, while you are here ... I know you have been working on a framework ...


... Zach Oaks has convinced me that using a map for app-db is ultimaitely a bad idea ... flat data is better

thheller 2021-02-28T10:29:34.053600Z

maybe the stuff I tried to describe here with transacted observed would also benefit the re-frame model (it should)

thheller 2021-02-28T10:30:30.054300Z

I assume you are referring to his o'doyle rules thing?



thheller 2021-02-28T10:31:01.055500Z

yeah that looks very interesting but seems impossible to scale this to what an webapp would need (adding/removing rules seems rather expensive)


That was the reason I didn't use DataScript originally. We had too much data. And it took too long to import. But ... The problem with using a map is that it is very "placeful"


I didn't realise quite how placeful until I started writing a tutorial on writing reusable components for re-frame


It struck me a little hard. The difficult part of creating a reusable component in re-frame is that the component's subscriptions and event handlers have to know "where" in pp-db they need to access




That means it is hard to take components from one app to another


There are things you can do to mitigate this


You can load data into well known places within app-db

thheller 2021-02-28T10:34:45.060600Z

well you can always make things reusable by separating that out and just having a render function that takes the data and renders it

thheller 2021-02-28T10:35:07.061200Z

and the other one that gets it from wherever and calls it


Yes, correct


These are all suggestions I came up with. BUT ... I was quite struck by this placefulness thing It means the subscriptions and event handlers had to get parameterised (by place)

thheller 2021-02-28T10:36:32.063300Z

but I agree that flat is better. my normalized db thingy is one big map with one level of nesting.

thheller 2021-02-28T10:36:49.063700Z

yeah but how else would you solve that?


Anyway ... just a thought


Datascript is flat

thheller 2021-02-28T10:37:27.064800Z

but then you have to pay everwhere for turning it into maps so you can actually work with the data again

thheller 2021-02-28T10:38:29.065300Z

(update thing :foo inc) becomes rather difficult too




There were some libraries which stored "entities" in app-db in a very regular way. I can't find them offhand.


They might yet be a middle ground


Anyway, I don't have an answer yet, but it is on my mind.

thheller 2021-02-28T10:43:55.069300Z

yeah I do think so. normalizing the data to one level to avoid duplication of entities seems to be enough. making it flatter to EAV tuples means you need to reconstruct maps all the time where you really just want to do (get db id) (which datascript has a wrapper for to emulate but thats not the same thing)


Ah, one of them was


Hmm. I'll have to read up again.



thheller 2021-02-28T10:45:58.071200Z

ah that looks good

thheller 2021-02-28T10:46:00.071400Z



currently it looks like everyone is trying to solve the same problem in a similar way


thheller 2021-02-28T10:52:24.072300Z

oh thanks. didn't see that one yet.


I am currently trying to solve the same problem myself as a hobby ; )


in production we use rum + patched datascript, but it is very slow and tedious


I think I'm struggling with a similar problem atm, where one thing in a collection changes and that causes recalculations in multiple subscription chains


Data is preprocessed first, then indexed in various ways and postprocessed at leafs. We have something that implements incremental indexing at the root, but further change propagation requires calculations from scratch in every subscription. My current idea is to have "streaming subscriptions" that would propagate changes only, similar to the idea behind transducers


The normalized flat db is great, but there is a problem similar to datascript, all subscriptions are recalculated with every change in the db. It is possible to do something like in posh, check each transaction and see if it matches the query. I have no idea how efficient this would be though, but I'm going to test and find out 😉


I'm currently reading the penpot code from cover to cover, they have a slightly different approach to state management there.


exactly based on streaming


Penpot is UXbox right? I think posh's change inference wasn't always correct at the time, curious if that's changed


yes, uxbox is now a penpot


and yes, posh sometimes returned incorrect results, especially when retracting from db


by the way, datascript is not fast at any time, in fact performance is poor


with and without posh


That's my experience with datascript as well


simple flat db in the form of a {?id ?map} searched with meander is just as fast for query and many times faster when it comes to transactions or pull

p-himik 2021-02-28T12:16:27.085300Z

A small correction - all level 2 subscriptions are recalculated on each app-db change.

p-himik 2021-02-28T12:20:44.085500Z

Just some potentially useful musings. I ended up rewriting subgraph to accommodate my needs and I'm still quite happy with the outcome. Although, I use it only for the domain data that comes from an RDBMS in an already normalized form. One thing I have found quite helpful is having multiple indices for every foreign key attribute and being able to query based on that. "Give me all items by a coll of IDs" is useful, but sometimes "give me all items that reference that other item" is even better.

p-himik 2021-02-28T12:24:20.085700Z

Another feature that's very useful for my use cases is to be able to store user-caused changes separately and query the changed state and the original state separately. Sort of like a transaction. It facilitation creation of complex and reusable components where you can enter a bunch of data, see what exactly you've changed, check validation results, and only then save the data.

p-himik 2021-02-28T13:43:28.085900Z

Just in case - there are also and

lilactown 2021-02-28T17:14:39.086800Z

if it's interesting to y'all, I was inspired by trying to use datascript + re-frame to author this lib:

lilactown 2021-02-28T17:16:03.088200Z

it normalizes your data into a relatively flat, simple map. and gives an API to easily pull data out using EQL; though of course you can just get-in to get specific data out too

lilactown 2021-02-28T17:16:58.089200Z

it's not the power of datalog but I found that datalog wasn't what I wanted most of the time anyway when relying on datascript to store my app's entities; it wasn't performant enough


somewhere here I posted autonormal as an example 😉

lilactown 2021-02-28T17:18:45.090Z

I see that now 😄 skimmed too fast

lilactown 2021-02-28T17:21:09.091200Z

i'm hoping to eventually base a "pathom-client" lib on it but in the meantime, I find it moderately useful just for getting out of the "place-oriented" ness that y'all have been talking about

lilactown 2021-02-28T17:22:13.092400Z

it still relies on parameterizing your subs/events to talk about specific entities tho. which is why i typically only use app-db for domain data and let UI state live in local component state

lilactown 2021-02-28T17:24:29.092700Z

btw I'm basically just waiting for someone to ask for custom schemas in autonormal. happy to riff on what a good API for that would look like if anyone is interested

phronmophobic 2021-02-28T20:53:14.099500Z

I recently wrote my thoughts on pretty much the same topic. Also, pretty much everyone here wrote something that influenced how I think about user interfaces! Thanks! (Tony Kay from fulcro and the minds behind hoplon were also big influences). > An entity can use three main techniques to refer to another entity: nesting, identifiers, and stateful references. > - Clojure Applied Chapter 1 The main difference between using datascript and subscriptions isn't flat vs nested, it's the type of reference that is used. Applications that use datascript as a data model tend to use identifiers as references whereas re-frame uses stateful references. What makes a component "reusable" isn't the type of reference that is used, it's whether or not the information needed to produce a reference is passed as an argument to the component or is hard coded in the component. When using identifiers, passing all the necessary information to build a reference is trivial (it's just the identifier). For data models that use nesting or stateful references, it's less straightforward. The tricky part with users interfaces is that there tends to be a lot of incidental state. Given the choice between implicit state handling that's less reusable and explicit state handling that's more reusable, developers tend to prefer implicit state handling. Ideally, incidental state should be handled implicitly and essential state should be handled explicitly which would make it easier to write reusable components by default. Here's the long version:


Creating components is one thing


Using re-frame, always at some point I have problems with data denormalization and map fatigue in general. On the other hand, when e.g. I use datascript, or other ideas for flat normalized db, the problem is performance and synchronization with backend and/or local-storage. Some data should be excluded from storage, what is trivial in case of nested maps, but more complicated in case of flat structure.


every solution has its weaknesses


@smith.adriane paths are identities in re-frame (paths within app-db)

phronmophobic 2021-02-28T22:04:23.102Z

so is it more correct to say re-frame uses both nesting and stateful references or just paths/nesting?


I mention this only because I'm not sure what "stateful references" are. I know, I know, I should read your page.

phronmophobic 2021-02-28T22:04:46.102400Z

I don't actually explain "stateful references"




Then I'm off the hook :-)

phronmophobic 2021-02-28T22:05:22.103300Z

I just kind of ignore them, but I give re-frame as an example of a library that uses stateful references

phronmophobic 2021-02-28T22:05:38.103500Z

so I can update it if I'm wrong


See the link provided above

phronmophobic 2021-02-28T22:06:03.103900Z

I actually cite that page in my post,


I have actually rewritten that page at some time, but never published the rewrite. I'm stuck on the final part of how best to get around placefulness

phronmophobic 2021-02-28T22:07:41.104400Z

it's a tough problem.


Tradeoffs all the way down.

phronmophobic 2021-02-28T22:08:45.104800Z

I've been using macros to "track" usage of data derived from props to automatically produce references based on paths/nesting, but I'm not sure it's the right solution


I can remember once spending an entire weekend trying to make Datascript faster, by turning it into a column store database and using the GPU. (GPUs are very fast with vectors) I failed. :-)


And when I say I failed, I mean I failed at the very first hurdle: using the GPU.

phronmophobic 2021-02-28T22:11:40.106100Z

the other approaches I've looked at: • Om (uses proxies). After using it, it was ok, but had too many caveats for me to want to try again • macros (my current approach) • fulcro has ui components define queries as part of the component definition. I think that approach could also work, but I'd rather the queries were automatically written for me.


Sorry, it was the section above the link I supplied

phronmophobic 2021-02-28T22:13:30.106900Z

I like Clojure Applied's definition: > An entity can use three main techniques to refer to another entity: nesting, identifiers, and stateful references.


I'm not reading that and feeling wiser :-)

phronmophobic 2021-02-28T22:14:44.107400Z

path's within an app-db being the same as "nesting"

phronmophobic 2021-02-28T22:15:51.107600Z

It's been tough finding good resources on the subject. Re-frame's docs are one of the best resources I could

phronmophobic 2021-02-28T22:16:44.107800Z

It has been interesting comparing different approaches. Re-frame mainly focuses on path/nesting references and fulcro focuses on identifers as references


An identity is something which identifies an entity. And it is very context specific.


In C we used pointers


In a database, it is a unique key


In Clojure it is typically a keyword


Or a path of keywords and integers (a path)

phronmophobic 2021-02-28T22:20:36.109100Z

I think it's possible to write UI components in a way such that they're independent of which reference type is required, but it's a challenge to design it in a way that it's not an abstract mess.

phronmophobic 2021-02-28T22:20:57.109300Z

The identity depends on the application's data model, not the language (although languages will usually have a lot to say about what kind of data model is idiomatic).


Yeah, that's a distinction without a difference for me. Practically speaking. But maybe I'm missing something.


I have to go but I present this challenge ... What is the identity of a certain collection of customers (the big ones)


So just to be clear, I'm not talking about individual customers


I'm talking about the collection of these customers

phronmophobic 2021-02-28T22:26:11.110400Z

is it a subset or the full collection?


Because I might need to distinguish good customers from bad ones


Yeah, so how do I identify the two collections.


Now I want to show the user this collection, now I want to show the other collection

p-himik 2021-02-28T22:51:49.111700Z

It depends on the implementation. A single customer can be ID'ed by [:customer 7], for example. A collection of customers by [:customer [1 2 3]], for example. Of course, this precludes having a single customer having an ID that's a collection, but I think there can be reasonable limitations. Another way would be something like [[:customer 1] [:customer 2] [:customer 3]], similar limitations apply. And there are many other ways - again, depends on the implementation. And perhaps the particular set of limitations/idiosyncrasies you're willing to deal with.

phronmophobic 2021-02-28T22:52:03.111900Z

;; uses nesting and identifiers
(def data
  {:customers {0 {:name "Bob"}
               1 {:name "Mary"}
               2 {:name "Sue"}}
   :lists {0 {:name "good"
              :customers [[:customers 0]
                          [:customers 2]]}
           1 {:name "bad"
              :customers [[:customers 1]]}}})

;; reference is [:lists 0] or [:lists 1]

;; alternative data model
(def data
  {"b4931d86-efca-4e86-b397-e54315673d32" {:name "Bob"}
   "133e9085-9481-4a64-befd-ce3b071a4c8a" {:name "Mary"}
   "f3c630f5-35e2-4600-8bf6-dc6fdbe76dae" {:name "Sue"}
   "b97c51b5-1078-45a9-a325-b8cb71f0959d" {:name "good"
                                           :customers ["b4931d86-efca-4e86-b397-e54315673d32"
   "51bf61bf-5163-4fd9-a52a-060de35ab7f7" {:name "bad"
                                           :customers ["133e9085-9481-4a64-befd-ce3b071a4c8a"]}})

;; references are "b97c51b5-1078-45a9-a325-b8cb71f0959d" and "51bf61bf-5163-4fd9-a52a-060de35ab7f7"

;; stateful references
;; not encouraged!
(let [bob (ref {:name "bob"})
      mary (ref {:name "Mary"})
      sue (ref {:name "Sue"})]
  (def customers [bob mary sue])
  (def good-customers (ref [bob sue]))
  (def bad-customers (ref [mary])))

;; references are good-customers and bad-customers


@p-himik yeah I was imagining that there might be two collections at different points within app-db. I was attempting to draw out that path is identity in re-frame, collections have identity. So too do the customer entities with them.