Clojurians Log v2

I am trying to run todomvc against a :dev datomic database rather than how it ships (:free or :mem), but getting this error:

user=> 16-07-12 06:34:36 chris-XPS-L521X ERROR [untangled.server.impl.components.handler:72] - Parser error:
 {:status 500,
 :body
 {:type "class java.lang.Exception",
  :message
  "processing rule: (q__26671 ?e), message: Unable to find data source: $__in__2 in: ($ $__in__2)"}}

The transactor is running, and "datomic:<dev://localhost:4334/todo>" is under /usr/local/etc/todomvc.edn's :url key.

@tony.kay @currentoor @therabidbanana pushed the change we were talking about yesterday to untangled-datomic. see https://github.com/untangled-web/untangled-datomic/commit/d6aac56485e09559a556e15b2c57964801bfe297. basically a wrapper for a transaction vector where you can specify a function to call after the schema migration completes successfully. that function takes the db connection so you can migrate the data

didn’t want to put the connection directly in the transaction function since it would be bad to call d/transact during the migration, which itself would be calling d/transact...

@ethangracer: awesome!

Should we expect these functions to be executed multiple times? Seems like it's possible they could be?

@therabidbanana: they have side effects right? So they shouldn't be called multiple times right?

no, in production the function should be called exactly once

when the migration is performed

if you’re in a dev environment on an in-memory database it’ll be called every time, of course

but the idea is when a migration happens you might want to port data just that one time, to match the new migration

e.g. renaming an attribute on all existing entities

@ethangracer: if I have only data to migrate, so no schema changes will an empty transaction vector work?

seems like it should but just making sure

wouldn’t that just be a standard transaction?

Well, I guess my point is that I was trying to read through this code before and I couldn't work out how they're prevented from running multiple times, other than conformity just being idempotent.

We don't use in-memory in dev mode much and I still see the logs for every single migration conforming

Every time I boot

Maybe that's just a misconfiguration on our side?

interesting. that would be a question for @adambros or @tony.kay — I think the system keeps track of which migrations have applied, but I’m not sure of the internals

it may log even though it does nothing

what I do know is that we have an app on production servers now and when we apply the migrations it’s not botching the data

🙂

Each migration is checked. We're using conformity, and it stores the applied migrations in the db so it knows what to re-apply.

I think our code does output which ones are being looked at so you can debug ordering issues and such

and it always looks at all of them...conformity decides if they get applied

we could have better output, I guess (e.g. ALREADY APPLIED)

I guess I don't see the part where conformity gets asked if it should run the after function for this new feature, if it is always checking all migrations, unless maybe ensure-conforms raises an error? https://github.com/untangled-web/untangled-datomic/commit/d6aac56485e09559a556e15b2c57964801bfe297#diff-e89eff854d20e40f1cd602547aafc92fR252

But if it was raising an error it'd be showing migration failures in our logs, I'd think

ah, good question

@therabidbanana: I agree

so, I think there needs to be a check if the database already conforms before running the migration

right now it’s happening after

we want the check after as a simple sanity check

but checking before so we know when to run data migration is needed

right, so both

right

ok, working on a patch

for that matter, you can then skip the ensure-conforms, since it probably starts with a short-circuit using conforms-to 🙂

and run the transactions ourselves? ensure-conforms is currently executing the migrations

right, but no need to call it if we're already calling conforms-to? and that returns false, correct?

yes, no need to call ensure-conforms, but we do have to call d/transact on each tx in the migration

oh right. thx

I trust you've got it 🙂

@therabidbanana great catch on the conformity, thanks for the code review!

patch is pushed

confidence? Tested well?

hard to test well because it’s such an integration test with datomic

oh, you guys are prob using snapshots...work with it for a few days and let me know. I'll push a release once we know it is solid

you want me to push a snapshot to clojars?

let me play with it for the rest of the day, and try applying a migration to QA first

I do think it’s solid though

oh, well, I pushed a snapshot to clojars anyway...no guarantees on snapshots 🙂

@ethangracer: glad I could help - especially since I know it won't be very long before we need this ourselves.

Hello. I’m trying out untangled-websocket and I hit a little bit of an issue. In https://github.com/untangled-web/untangled-websockets/blob/master/src/untangled/websockets/networking.cljs at line 98. The callback function utilizes the deref global-error-callback but this function is passed in at client creation. Here is an error message when server returns an exception to the client. `No protocol method IDeref.-deref defined for type function: function (status,error){ return null; }`. For the client initialization. :networking (wn/make-channel-client "/chsk" :global-error-callback (fn [status error] nil)) Thank you.

how do I run untangled-client tests after checking out the repo?

I did

npm install; lein doo chrome test