Clojurians Log v2

genuine question: why do I want to use GAE instead of (1) firing up Digital Ocean servers, which offer 1TB traffic for $5.00 / month (2) having my DO servers make remote API calls to google datastore / storage / etc ... ?

This also gets around JVM limitations and lets me use websockets.

If your usecase requires websockets, than you probably would want to be using your own VM

That being said, using a compute engine instance will have very low latency with the rest of the GAE API's

Agree with Peter. Your calls to datastore will be long and unreliable with DO

We went around and around with websockets at work and we’re on AWS. We ended up using Google Realtime anyway. Websockets do not play well with the rest of our stack.

Turned out to be very finicky too

Google Realtime takes the problem one step further. Operational Transform is the algorithm you’re going to be hovering around with shared state. You’ll have to roll that yourself if you don’t use Google Realtime.

I haven't been keeping up with Google's cloud offerings as much of late: thanks for the Google Realtime pointer 🙂

Yeah we use it. It’s amazing. Basically came out of Google Wave. It’s where you can have many people interacting with one resource in realtime.

Many-to-many too

How well does it deal with offline/catch-up cases?

That is a use case where we don’t use realtime directly, peter.

Well, now that you mention it, we do use it.

I know a little bit about your use-case, Peter. You have a lot of offline needs. In our situation it’s a news editor’s SPA. He or she is looking at a bunch of stories and they update in realtime across multiple screens. They can edit a story and other members of the news team will see the updates in realtime as well. It does not have offline support. I’m a little surprised our reporters haven’t asked for that yet, seems important.

So I can tell you it works great but no offline support.

However, I would suggest that it would work fine with offline support except when two people are updating the same record. They you’ll need something like a 3-way merge tool in your app or something.

In our case we lock a story while a reporter, writer or editor is focused in the story (we consider them typing even if they’re only focused: made people less mad at each other, lol). It becomes read-only for everyone else.

In your case where people are working offline, this doesn’t fly.

Is that helpful?

Makes sense

The ideas can still work offline (i.e. Operational Transforms), but looks like the google api isn't really built around that use case.

@nickbauman : are OTs and state-based CRDTs the same thing, or two different approaches ?

OT?

Can you expand that acronym?

operational transform

Oh Operational Transform, I see, sorry I’m lame

I thought that' what google realtime used

Operational Transform is an algorithm. I’m not sure if calls to GRT comprise Conflict-free replicated data types though

(sorry about the edit)

I suspect they’re different, though.

Fundamentally.

okay, so google real time and phoenix presence (which I beliece usses state based crdts) are likely different things

this is an area I wish I knew more about

CRDTs are a much lower-level concept than OT. With OT you bind your system to a single resource.

with CRDTs, we're looking for ops that are indempotent, assocaitives, and transisitive

But underneath that resource may be a highly-available cluster of systems.

I don't see how can I get around that issue

Google Realtime relies on Google Drive, which is highly available.

yeah, but google can't solve network latency between client nd goog servers, or random disconnects

Maybe underneath you have something akin to a CRDT, but the drive API that Realtime uses doesn’t know about that.

is ther a recommended whit epaper on google real time OT ?

I feel like I'm misunderstanding the fundamentals.

I think you’ll find it in Google Wave whitepapers.

Realtime is the core thing in Wave.

Core concept of OT is super simple

https://github.com/Operational-Transformation/ot.py/tree/master/ot

You can read that code in a few minutes and understand it on an abstract level.

Then this is a specific implementation in text

https://github.com/Operational-Transformation/ot.py/blob/master/ot/text_operation.py

(Python is a decent language to communicate this stuff, too)