Clojurians Log v2

I seem to have gotten the travis CI build of U.C. working better. I reduced the memory footprint of the JVM instances and made sure all of the versions of things used to run the automated tests were up to date, and that hopefully has solved the erratic failures on running the browser that runs the tests.

It’s looking to me like load-data is deleting fields on subsequent queries with fewer keywords. e.g. if I first call (load-data [:several :keys {:with [:some :joins]}]), and later I call (load-data [:several :keys]), then the data under the :with key is gone. has anyone else seen the same behavior?

doesn’t seem appropriate to me since I might want to update the data at :several and :keys but not another key

mmmm...I would be very surprised if that is the case

there is no memory of the prior

I have a feeling you've got some other bug...perhaps in a post mutation?

@tony.kay thought it might be a post-mutation but I’m sure it isn't

it definitely could be some other bug

but the data for all fields not specified in the second load is gone

do they have some similar root?

are you using get-query so the normalization will work?

what else are you not including in your description?

yes to using get-query

not sure what you mean by similar root

is an ident involved?

I don’t think I’m excluding anything relevant

yes, normalization is involved

not the same

I have a survey that is already normalized in app-state

then i’m pulling down a survey request, which itself pulls down certain details about that survey

the already normalized survey data is a superset of the survey data pulled down with the request

but all of that survey data uses the same ident, because both refer to the same survey

by the time the post-mutation runs, the data is already incorrect

ah, so you're saying the merge is stomping on the object

seems that way

instead of recursively merging into it

yes

that’s my running hypothesis

having a hard time tracking it down

merge-idents is a separate step in the low level Om merge. I'm trying to remember if we override that or just the general data merge

oh, but you're using load-data, so there isn't a rooted ident in the query

correct

it is possible it is a bug in the sweep merge

which , of course, is the most complicated bit of code in untangled

aaah, that would make a lot of sense

it has a set of tests...you might beef them up and see if you uncover badness

if it marks keys as missing

do you remember the basics of the algorithm?

I’m not sure I ever bothered to learn it 😬

I probably should

well, it is a bit involved...here is the basic idea: 1. It uses the query AND the result 2. It follows both through the recursive logic or the query and structure of the data 3. If something is asked for in Q and not in R, then it is marked as missing 4. It if is asked for in Q and IS in R, it should pick the proper merge (or recursive step)

Then after the merge runs, a sweep step removes everything marked missing

so if you ask [:a {:b [:c]}] with response {:a 1 :b { :c 2} }

it will look at key :a in query while looking for a value in result (1).

then it will look at join on :b while looking at the value in R at :b

of course, T (the target) is involved too

shouldn’t this all happen before looking at the app-state?

and then just do a deep merge?

@adambros wrote the algorithm, so if you find a problem, it might be best to get his help

yeah...I may not be remembering exactly how it works 😜

well I think you’re right

it has to use the query and the result

it is definitely deleting stuff from the app state

but it shouldn’t be

only from the remote result

I designed the algorithm on a white board and Anthony implemented it

haha ok

i’ll take the plunge and see what I can figure out

thanks

welcome. I'd start with tests using your scenario

sounds good to me

im running into an issue where when i get a 401 i have a network-error-callback that redirects to the logout page but for some reason it gets into an infinite loop when i go to a page that does a load-field BUT i can get out of the loop when i switch to a different tab and it does the logout correctly any ideas?