Clojurians Log v2

@wilkerlucio The new combo lib is what I can (and plan to) maintain. It has the desired version bumps, but does have some porting things you’ll have to do. I’ve left NAVIS and they have not formally given me permission to do use their github accounts to maintain Untangled. The new “official” version is at my github account: http://www.github.com/awkay/untangled There is a 1.0.0-SNAPSHOT on clojars. (Untangled spec has a 1.0.0-alpha4-SNAPSHOT as well)

though they do not, themselves, plan to do anything with it. I’m working with them to either get them to remove the now-stale resources from the net, or to get permission to take over the untangled-web org…the latter, of course, is my preference.

@bbktsk So, client headers. You’ll have to program that into your own networking implementation on the client side. The model is primarily aimed at situations where you log in (and can therefore use cookies). As far as where to store that info. Up to you. If it is just data, then you could put it in the app database if you want. As long as it is serializable data you won’t hurt anything (support viewer needs to be able to serialize the app database).

BTW: I’m going to be on the road (as in car road trip) for the next couple of weeks. I plan to be working on Untangled and other things during this time, but I may be in BFE without inet.

@tony.kay will om-css also be moving to a different repo ?

also is there a roadmap for untangled, so that we can fallow and maybe pitch in ? 🙂

@tony.kay thank you!

I may have spoken too soon when I said it is working, my problem is back. I was trying to simplify it, with only a partial success. I have a working code that behaves strangely (imho, maybe I’m just doing something wrong), but it is different “strange” from what my real code does. Anyway, here’s the gist:

https://gist.github.com/214e981c72df70f1f2e965575aaa4469

Short summary: Three components, People, Person (with ::id, ::name and ::car) and Car (with ::type and ::color). Relation between Person and Car is 1:1 . The fake remote api is setup with setTimeout to return response after half a sec and has two endpoints: initial load returns two persons, however, with ::car of each set to nil. Second endpoint returns Car for specified Person. The idea is “Load all persons first, then, on demand, load car individually when requested”. There’s a mutation inital-load (no problem here) and load-car that calls (df/load-field-action state Person [::person-by-id id] ::car :remote :remote :marker false :refresh [::people]). The latter one is invoked by buttons on individual Persons . This is where strangeness begins: first, if I click the button, the request is sent to the fake remote api, then immediately render is called but with empty list of people. Then remote api returns response, render is called again, this time with correct props. In effect, I click the “load car for this person” button, list of persons disappears and is replaced by “no data loaded” placeholder, then, after half a sec, it reappears with correct data, including newly loaded car. Second odd thing is that if I do not include the :refresh key in df/load-field-action, only the first render (with empty list) happens and no render is executed after fake api returns data.

I guess that the main question here is: Why is render first called with empty data?

Oh, and an important note: the above behaviour manifests in Chrome and Firefox. Safari, on the other hand, works as expected (render is called only once, after remote request finishes and with correct data). All on current macOS.

@bbktsk loads replace the data with a load marker. See the docs. You can turn that off with an option to the load call

@claudiu Depends on if I get control of the untangled-web group on github. As far as roadmap: I have not written a formal one yet. If there’s something you’d like to see appear, that is probably the best thing to try to contribute 🙂

sorry @bbktsk didn’t see your comment on load marker…was reading too quickly

so, render is called on any mutation to display local optimistic updates. Render getting an empty list of ppl means the query found no people

which means something wiped them in your db

@tony.kay It seems so, but on the next render (after remote returns) the data are back 😯

have you tried putting in a large enough delay that you can look at the db in the in-between time?

Also, why there’s no render after remote returns unless I specify :refresh […] ? df/load-field-action should refresh target component automatically, right?

good idea. will try now.

so, refresh happens on the component that triggered the mutation. Nothing is automatic beyond that. Om has no idea what data you’re mutating, so it assumes the sub-tree only. The follow-on reads (e.g. refresh) are needed if you update something outside of that tree.

My guess is you’re triggering the mutation far into the tree?

I’m looking at your gist…just a min

OK, so I see several things.

1. Does more than one person refer to the same type of car?

If so, you need to load the car from the list level, or do the follow-on read

2. You do not need to write an initial-load mutation. Just use load.

3. You do not need to write a load-car mutation, just use load-field

But note: load-field MUST be done in the context of something that joins it as a field in the query, which will necessitate a follow-on read if that thing might be visible elsewhere in the tree

e.g. it goes in Person…`(load-field this ::car {:refresh [::people]})`

Tried to longer delay. Initial load, state is good. Hit “load car”, people disappeared, checked state, it is still good (original data + some untangled stuff). It seems the state atom is not wiped.

1. Nope, strictly one to one.

2. + 3. yup, tried both (or at least for 3.), same result.

oh…line 55

don’t do that

or at least, line 52 and 55 are not compatible

one says the data is relative to the component, the other says the data lives at root

Where is your root, BTW?

Root component? It’s People. It’s missing ui/react-key, but I fixed that already.

Ah, so root is special. It doesn’t get an Ident, because it IS root

if you give it one you get an extra edge in your graph that can just be confusing

And the query of the root component is against the root of the db automatically, so your query should not use a link query

is should join on the {::people (om/get-query People)}

I don’t necessarily think that should fix your problem, though 😕

I think line 113 might be a problem

load-field writes a query of the form: [{[:person-by-id id] [::id {::car [::type ::color]}]}]

your response has an extra level in it

the response should be:

{ [:person-by-id id]  { ::id n  ::car { ::type x ::color blue } } }

aaaaaand it works now 😎

113 is ok , the extra ::person-by-id is just value for condp a few lines above.

Oh, I see

mis-read

so the link query was breaking it???

oh…probably the ident was causing a problem with added indirection

Yes, just tried all combinations, it started to work only after I removed ident on People and changed the query to relative.

Yeah, an ident on root means you want it to be normalized (put in it’s own table)…but its root

Hmmm. I wonder if that’s what I am doing wrong in my real app.

I’ve never tried it but I’d suspect it would put the initial tables into a non-root node, making your tables not work like tables.

well, we know one thing: it breaks &^$

perhaps worth a warning note in the Dev Guide

Just a final quick question: I assume that if suddenly :om.next/tables in my app state is an empty set, something went wrong, right?

No, I think that is an old om next thing that isn’t actually used

I think D.N. originally thought it was going to be needed, and wrote some code to make it, but as far as I can tell it isn’t used nor does it maintain proper track of the tables.

I expect it will disappear as they clean up the code for a 1.0 release there

the only place you can even find ref to it is in the tree->db function (which normalizes things). Nothing in Om Next ever tries to read it.

(except in two old tests)

the merge behavior will overwrite it every time a normalization happens (which is after every network interaction)

Thank you. There goes my last clue 😎 Anyway, thanks a lot for you help, again 😎 Although I’m having some problems, I really like Untangled so far.

You’re welcome. I’m glad you’re liking it.

I like how most of the problems end up being something really pretty simple. If you get the ident, query, and initial state right, you’ve got most of it…and those are so simple once you get it.

then you have mutations: pure functions from one state to another. It’s unfortunate you need REST…because when you don’t it just stays nice and simple 😉

not that REST is all that hard to add, as you’re seeing

Yup. It’s just a few days and my project is shaping pretty well. Now I just need to figure out why the reconciler throws that exception 😎

@bbktsk So, there is a chance you’ve found a bug. Do you have a full stack trace?

I assume you’re trying to load extra stuff on one thing (perhaps in a list of things), and after the first load you try another one, and that’s when you get the error?

@tony.kay That’s almost exactly what I am doing. One stacktrace coming up…

core.cljs:5487 Uncaught Error: Index out of bounds
    at Object.cljs$core$build_subvec [as build_subvec] (core.cljs:5487)
    at Function.cljs.core.subvec.cljs$core$IFn$_invoke$arity$3 (core.cljs:5499)
    at Function.cljs.core.subvec.cljs$core$IFn$_invoke$arity$2 (core.cljs:5497)
    at cljs$core$subvec (core.cljs:5490)
    at <http://om.next.Reconciler.om|om.next.Reconciler.om>$next$protocols$IReconciler$reconcile_BANG_$arity$2 (next.cljc?rel=1497896997432:2534)
    at <http://om.next.Reconciler.om|om.next.Reconciler.om>$next$protocols$IReconciler$reconcile_BANG_$arity$1 (next.cljc?rel=1497896997432:2485)
    at Function.om.next.protocols.reconcile_BANG_.cljs$core$IFn$_invoke$arity$1 (protocols.cljc?rel=1497896971054:20)
    at om$next$protocols$reconcile_BANG_ (protocols.cljc?rel=1497896971054:11)
    at next.cljc?rel=1497896997432:1470

so, couple things to check:

Loading data using (df/load-field-action state Bean [::bean-by-id ident] ::bean-attrs :remote :remote :marker false). There is some massaging of data being done in send, but the result is just a new value for a field.

So, in your UI, when you pick apart props and pass them to children, are there any cases where you’re not being careful to pass (unmodified) the props for a child to that child?

or where you’re rendering something outside of where it is queried?

Here’s why I ask: When Om does a UI refresh, it tries to do a targeted one (so that only the subtree causes overhead). The place where that exception is thrown: it is using the path (in the tree) of components, which is added to the data tree before being handed in to render. E.g. the database is queried, and a tree of data is returned. The UI query and this tree are then used to annotate that data tree with metadata at each component data node. The two must match, or strange things happen

Then the tree is either passed to root (and picked apart as you render) or the targeted component for partial refresh is given the partial result of the focused query

I’m not doing a great job of explaining it 😕

By “not being careful” do you mean “is it possible that some component is receiving data it is not declaring in its query”?

Transact happens ---> Om pulls the query, and eliminates all the bits that don’t have to do with the subtree that triggred the transact (this is called focusing the query) ---> Om runs parser on query and pulls data from db ---> Om runs path-meta to add metadata to this data

Yes, any mismatch on stateful components is an error

if something queries for data, it better be composed into parent query, and the props must be picked apart in a similar fashion from the props it receives

I’m 90% sure that with every subcomponent that has something like {:subc (om/query Subc)} in component's query, I just do (subc (:subc props))` somewhere in render. But I’ll check that.

Yep. Anything else that flows through the data tree must use om/computed

I’ve had one recent case of my own (that I have yet to track down) where I think the indexer got confused. Moving the transaction to the parent worked as a workaround, but you cannot do that with load-field

It wasn’t the same error, though.

@tony.kay Now I am 99% sure I am passing only what’s requested to each component. Although I have found interesting thing. I have hierarchy Doman -> Bean -> Attrs . By default, y˘ou see just a list of domain names. To see Domain’s Beans you have to click “show details”. Same for Bean and its Attrs. “show details” is a local mutation that takes ident and does just (swap! state update-in (conj ident :ui/open?) not (Domain and Bean have :ui/open? in their query and data). Also, when Domain is opened, a remote mutation is started (together with the “show details” one), to load extra data for Beans’s Attrs using df/load-field-action. Or that’s the plan, at least. When I removed the remote mutation (so that only local “show detail” mutation that toggles :ui/open? remains), everything worked just fine. I could open (and close) the hierarchy all the way down, no problem. Then I modified code so entire hierarchy is displayed all the time, regardless of :ui/open? on any level. Also, I added button to just start the remote mutation on individual Beans. And again, everything works. I click the “load data” button on each Domain and bam, data show up in domain’s Attrs without problem. So far, if I do only “show detail” mutation or only “load attr data for domain”, it works. But, if I load data to Beans in two different Domains and then do “show detail” on one Domain and then “show detail” on one of its Beans, I get that Index out of bounds exception 😮 . Most other combination do work: load data to one Bean in one Domain, then “show detail” everywhere, ok. load data to all Beans, “show detail” only on “Bean” (instead of going from top level Domain), works. And so on. The sequence to trigger the error is pretty specific, unfortunately exactly matches what I need to do. I guess the way to go now is to look into state and db->tree dumps “before” and “after” and look for something odd?

Hm.

It won’t hurt to verify that you graph is sane.

but it is very odd that you’d be able to do it most ways except one

I kind of suspect it has to do with the incremental render in Om (which prevents more costly from-root rendering)

The error you’re getting is in that exact algorithm…it walks from the component towards the root updating props

If I was sure the code was essentially right, I’d probably set the project up with Om in checkouts so I could debug the reconciler and see what’s going on.

well, for that matter, if you have devtools installed you can set breakpoints in Chrome and look at values

I’ll try to throw together a minimum case of what you just described in a devcard and see if I can reproduce it.

Thanks, that’d be great. I just compared state and (om/db->tree (om/get-query Root) @state @state) just before and after the exception, and there is no problem, the only difference is that ui/open? changes from false to true, as expected. btw, is that the right way to use om/db->tree? Docs says that the second arg is “some data in the default database format” and I am not quite sure what “some data” means here.

yes

it is recursive, so one of them has to be the root node (where the tables are at), while the other could be some data (which in turn contains idents)

when starting from root, they are the same

if you were starting, say, from Domain query, you’d give the value of a Domain as the data

but you’d still give the root for ident lookups

Aaaah! Now it makes sense! Thanks.

@bbktsk Well, I think I have something similar, and it works fine for me

I can throw it up on a branch and you can have a look

https://github.com/awkay/untangled/blob/load-sample/src/demos/recipes/load_samples_client.cljs

You can clone the repo, and switch to the load-sample branch

Then run figwheel with the -Ddemos build

Then run a standard REPL and run (run-demo-server)

Then use http://localhost:8081/demos.html#!/cards.load_samples_cards

ignore the comments. They’re for the unmodified code…which this is not

I did some debugging and I know what the problem is, although I do not know who to blame 😎

There’s a Bean component inside a Domain component. The code that throws exception is here, from next.cljc, function reconcile!, starting at line 2531:

(when-let [update-path (path c)]
  (loop [p (parent c)]
    (when (some? p)
      (let [update-path' (subvec update-path (count (path p)))]

When this executes, c is the Bean component, its path is [[:wmcng.beans/domain-by-id "java.nio"] :wmcng.beans/domain-beans 0]

The parent is Domain and its path is [:root-router :current-route :wmcng.beans/domains 1]

count of Domain’s path is 4, however, Bean’s path has only three elements, so it fails.

The code apparently assumes that parent’s path is a prefix of child’s path?

So, is this assumption valid and I messed up? Or is it not valid and I can blame someone else? 😎

um. try this: There is an option on untangled-client which is passed through to om

:path-opt I think it is…try setting it to false

No change. Are you sure about the name? There’s no mention of path-opt in entire om repo.

hold on

no hypen

pathopt

If that fixes it, I may just set it to false by default in Untangled…I’m not sure the regular Om community even knows about it. Early on it was meant to be a way to optimize re-renders…but it is possible it fell through the cracks

It has certain requirements that Untangled fulfills for you, so it defaults to off for regular Om Next

@tony.kay Nope, still the same error (same place & reason)

hm. you’re using 1.0.0 snapshot of Untangled, right?

well, it certainly looks like a bug in Om Next to me.

the assumption that an om-path will be longer in a child, when a child’s path can use an ident to shortcut, seems like a problem to me