Clojurians Log v2

@lilactown why not just use core.async channels?

How to convert PersistenceList to vector in clojure?

What package does PersistenceList come from?

(vec my-persistent-list)

@p-himik I assume Aditi means the clojure.lang

oh wait, that's PersistentList

(vec the-list)or`(into [] the-list)`

because I want to do something else 😛

in all serious, core.async and what I'm fiddling with are different. I used the term "stream" above because I wanted to skip over explaining my exact use case, but what I'm actually building is an incremental computation graph

examples of this in other languages are Adapton, Jane Street's Incremental, KnockoutJS/MobX

the difference between a channel and what I want is that a channel has no "current" state, while an incremental computation does. with a channel, you block (pull) until you get a message; with an incremental computation, you use the current value to compute your answer, and it will tell you (push) when it has a new value so that you may recompute

Hmm, ok 🙂 If I’m understanding correctly, it sounds like you can achieve something similar by combining core.async channels with transducers and one atom per channel which holds the latest state.

you could probably create an incremental computation graph built on top of channels and atoms, sure

that's sort of unrelated to my question

Right… 🙂 I just always try to re-invent the wheel less (if possible):

(defonce ch+state
  (let [*chan-state (atom nil)
        ch          (cljs.core.async/chan 1 (comp
                                              (map inc)
                                              (map (fn [x]
                                                     (reset! *chan-state x)
                                                     x))))]
    {:chan-state *chan-state
     :chan       ch}))

(comment
  (cljs.core.async/put! (:chan ch+state) 42)
  @(:chan-state ch+state))

I can get deeper into this but just FYI there is a plethora of rich content on incremental computations out there

Are you looking into this for front-end or for back-end purposes? (or both)

what I'm working on is intended to be CLJC. there are already a few passable solutions for CLJS

Nice, I like CLJC 🙂

taking your ch+state and assuming you would use add-watch to compose them, you'll run into "glitches" where some atoms will be updated while others are not. there's also problems to solve with computing the graph in an efficient order.

I see what you mean… you could “buffer” multiple updates and update a shared atom every N ms, let’s say

Or update one shared atom naively (to start) every time there’s a change

it also implies or assumes a particular solution to the problem I posted in the channel, which is that if you deref the state before a "valid" value has been received then it returns nil. I don't know if that's a good solution

That’ll ensure consistency

Yeah… until something has been “done”, I think it’s ok-ish to be nil… I think that’s a fair thing in Clojure, nil signifies something that’s just not there yet

Or not there, in general

consider the following graph:

(def counter (source 0))

(def a (compute #(deref counter) (remove even?))

(def b (compute #(inc @a))

if I initialize a as nil until #(deref counter) returns a not even? value, then when b gets initialized it will throw a NPE

I would have to code defensively around this:

(def b (compute #(when-let [n @a] (inc n))))

and this would cascade throughout the rest of my computations

> Or update one shared atom naively (to start) every time there’s a change this still runs into the problem where between propagation of values through channels, you will see one ch+state update before another. you need a way to update the entire graph and commit it at once 🙂

So just so I understand, (compute …) returns something that is

(instance? clojure.lang.IDeref (compute ...)) => true

yeah

Same for (source… ) ok…

What I’m suggesting is having one atom shared across the whole graph.

yes

So you’re saying that the whole graph needs to act as “one” unit…. so all computation of all nodes run together, and complete “at once” ?

yes exactly. I want it (and the way I've already implemented it) to work is that you make changes to inputs, and the graph recomputes, then updates the state of all nodes at once and then notifies external listeners

I'm considering using core.async for the scheduling of computation execution, but using them as the sort of connection between nodes feels like a red herring. it doesn't actually solve any of my problems

I understand now 🙂 Cool, I still think that it’s possible to achieve the same results with channels + one global atom shared across nodes… you’d just need to schedule/buffer all the change/update functions from all nodes on that global atom and run them all in one (swap! …) call … that would ensure a consistency (I think)

I guess I'll say, I've already implemented it with half of what you say - the state of the graph is stored in one global atom shared across all nodes - without core.async. I can send changes to inputs, and this schedules a recomputation of the graph. Each node knows how to identify itself in the graph and recompute its value based on its dependencies.

why would I want to add core.async into this mix?

For sure, sounds like the same idea.

I recently learned about something called differential dataflow

https://timelydataflow.github.io/differential-dataflow/

Might be of interest to you, if you’re digging deep into this.

thanks for the pointer. differential dataflow definitely falls under the umbrella of incremental computation. I've been reading the literature 🙂

Hello, is out there any library to visualize clojure code? Not namespaces, function/variable usages. I found some, thought about trying to build one myself, but wanted to ask around first 🙂 something like this: https://200ok.ch/posts/using-clojure-to-visualize-dependencies-in-clojure-code.html

Some things I know of that should be able to handle vars: - https://github.com/jpmonettas/clograms - https://github.com/gfredericks/clj-usage-graph - https://github.com/greglook/lein-hiera - https://github.com/testedminds/lein-topology - https://github.com/stuartsierra/class-diagram (useful for interop with Java) - https://github.com/benedekfazekas/morpheus - https://github.com/clojure/tools.analyzer (might be useful if you decide to write your own)

nice list 🙂

"This list is incomplete. You can help by expanding it." :D

yes, thank you very much! I'll take my time reading the source codes. 😄

morpheus and https://github.com/SevereOverfl0w/vizns are both based on the analysis output from clj-kondo, it's pretty easy to get going with that data and write your own tools on top. More info here: https://github.com/clj-kondo/clj-kondo/blob/master/analysis/README.md

if I understand correctly, we are currently looking for a similar solution. for now, we are testing something like this. https://github.com/ribelo/rum/blob/2a41d176f5f62fc96ca40af0d00cf7fa523d5c09/src/rum/atom.cljs

lazy atoms that recalculate according to the viewing hierarchy. we want to get something similar to subscribe with re-frame, but stay as close as possible to pure react

FYI this was answered here https://ask.clojure.org/index.php/10347/spec-instrumentation-does-not-work-with-extend-with-metadata

yeah that looks nice

just reading the source, it looks like it may still have "glitches" occur where updates are triggered before the entire graph has completed; or if an error occurs, it will throw in the middle of updating the graph and you'll end in an inconsistent state

you will also end up doing more computations than needed if you find yourself in a diamond dependency graph:

src
  /   \
a       b
  \   /
    c

you can still do a lot of things with those caveats, though. reagent suffers from both of those problems as well and plenty of teams are successful building apps with it