Clojurians Log v2

regarding "defentity" and similar: too gae-specific, imho. ideally we want a nosql api that is not tied to gae. we can do that with closure's meta-programming facilities (or so I jmagine). since a gae "entity" is nothing more than a map, and a gae key is just a vector of namespaced syms, we can in principle come up with closure abstractions that will work for many nosqls.

so e.g. in migae/datastore I treat entities as entity-maps, not essentially different from any other implementation of the map interface, except for the save and retrieve operations. otherwise than that, no different than hash-map.

so the pgmr need not be exposed to gae hidiosity.

I guess I’m not terribly interested in making a query DSL that works with any NoSQL database. I just wanted something better than the JavaAPI for Datastore.

Having said that, one of the things I hate about the Cassandra NoSQL database is that it has a SQL interface that doesn’t behave like any SQL database in earth behaves. It was a huge piece of misdirection to anyone coming to Cassandra.

This is one of the reasons why nobody I know ever uses GQL with Datastore.

The Python NDB library is much better than using GQL, for example.

I would much prefer a Clojure DSL that works with Datastore and Cassandra, if that’s part of the puzzle, mobileink!

So I support you in this effort!

I also support datastore specific over generic libraries for hte following two reasons: (1) iirc, datastore transactions/entities have different semantcis from simpledb / dynamodb (2) I want to be able ot read a datastore query and estimate how many read ops it takes (and how much goog is charging me) <-- a 'generic sql' layer would make this very hard

@qqq in case it wasn’t clear I wasn’t advocating for a SQL layer, either. I might not have made that clear.

@nickbaum: I think the only disagreement we have is "only datastore" vs "datastore + cassandra" -- I've never used Casandra; can you explain how it relates to datastore?

Cassandra is a FOSS NoSQL database written from the same whitepaper as Datastore.

It has only a SQL interface. But the underlying implementation makes your knowledge of SQL in any other database all but useless.

@nickbaum: interesting! what is this paper? I have always wanter to understand the theor behind datastore but never found it

I think it might be the Dynamo paper, I forget.

It hits all the checkboxes that Datastore does. Every feature is more or less there. It has been known to work at a cross-continental level, too.

But I’ve used it. It’s no where near as well designed as Datastore.

OK so there’s the BigTable whitepaper. https://static.googleusercontent.com/media/research.google.com/en//archive/bigtable-osdi06.pdf But Datastore is a combination of two technologies: BigTable + Megastore.

BigTable is the actual storage. Megastore is the indexes.

Megastore is eventual consistent. BigTable is strongly consistent.

(IOW, writing the indexes happen asynchronously to writes to BigTable)

@nickbauman : gae doc examples has code that auto generates login/logout urls; does your library have calls for that? ( I couldn't find any)

Nope! I'll write an issue in the template if you don't.

BTW I looked into the effort it would take to support save and delete hooks, @qqq ... Not going to be very easy because those features rely on annotations in subclasses of entity 😜

So the Clojure code will have to genclass for each save or delete hook you want.

Blech!

https://cloud.google.com/appengine/docs/java/datastore/callbacks

I imagine the syntax looking like:

(with-pre-put #((.setProperty (.getCurrentElement %) (name field) (Date.)) 
    (save! (…)))

Maybe we also need a (with-pre-puts … that takes a vector of pre-put functions?

Anyway, I have it here: https://github.com/nickbauman/cljgae-template/issues/15

@qqq I added login / logout to my issue list. https://github.com/nickbauman/cljgae-template/issues/16