Clojurians Log v2

not exactly cljs dev question: Curious if there were any attempts at improving transit-js performance? Looking at the source it seems like the algorithm recursively walks parsed JSON tree, which suggests that it could be more performant w/o recursion, since recursion is typically slower in JS engines (?).

@roman01la I don't think many people actually use transit-js

which makes sense, transit is only really popular in clojure and clojure is a relatively small community

so the pie slice of people who use transit and use JS is even smaller

which is to say it has likely been fast enough for the people who do use it

transit-cljs depends on transit-js

^shove a foot in my mouth

yeah, I’m referring to transit-js specifically, because it’s where the core of it is, transit-cljs is more of a wrapping library

the reason why I’m asking is because while parsing is fast, it’s still not fast enough in envs such as React Native or for real-time/streaming type of web apps, where updates are being consumed frequently and transit parsing periodically blocks the main thread

what other tech is fast enough in this scenario? (just looking to learn here)

it is likely a significant effort to implement the transit design in a streaming fashion, so I'm kind of wondering out loud if even that would be fast enough for the scenario you're asking about and whether there are known alternatives that meet the criteria (like, is streaming json sufficient?)

@alexmiller I don’t have an answer for this in a form of a working library, but in our case we are not sending custom data types over the wire except of UUID. @mfikes I’m wondering if a modified version of cljs-bean (that would recognize UUIDs) could make a difference?

@roman01la See https://cljdoc.org/d/cljs-bean/cljs-bean/1.7.0/doc/transit

@mfikes nice! but that’s only writing to transit, right?

Oh, right.

@alexmiller I don’t think streaming parsing will be faster or even comparable to the current implementation. However would be interesting to see how performance would change with a non-recursive implementation, afaik JS engines are not that good at recursion, I might be wrong though.

that seems like an overly generic conjecture with an absence of data :)

this is the sort of thing that may be true at a point in time, then becomes conventional wisdom, but the enormous performance pressures on js engines completely invalidate over time. but I don't have any clue either way

@roman01la strawman - try and find a json parser that is fast enough for your use case

like parse the data from your server as json, then insert mock data into the app

that would give a jumping off point to figure out where transit-js can improve / there is room for a different impl of the spec

@roman01la it would be interesting to figure out if that it would be faster but I think it would be a significant amount of work

the amount of tuning that went into transit-js is non-trivial

the other thing is that transit-js is currently unsuitable for streaming - so dropping the recursion seems a bit dubious to me vs. just doing something better (suited)

back when I worked on none of the JS streaming parsers seemed practical - that assumption may not be true anymore

it also of course leads to a different API which isn't necessary in many circumstances and a big pain

i.e. the streaming parsers - one big problem is that you need a sensible event loop thing but a lot of JS environment aren't going to give you that out of the box

https://cognitect.github.io/transit-tour/

it's an old version of transit but there haven't been any performance related changes since the first version

at the bottom are timings, it's interesting to check these between browser

the gap has only decreased over time on all engines

Firefox used to be 2X slower

but using the latest FireFox converting 53K of JS is only 0.10 ms more than the equivalent payload of JSON (i.e. duplicated keys)

so key caching cuts the JSON parse time in half

and then you need another full walk

but the end result is only slightly slower than non-cached key JSON

it's true that in React Native you don't get native compilation so numbers there would be interesting - but I'm pretty skeptical about the streaming web apps

unless there's good benchmarks to show the benefits of a JS streaming parser over just paginating the result or whatever