Clojurians Log v2

@nickbauman : I'm very optimistic about this becoming a very powerful web framework. GCP/Datsstore seems to have made all the right tradeoffs, and with the same language on client + server side, things can be really clean.

I really like appengine also. Running a personal project that has about 10k pageviews per day and paying about 3$ a month. Right now it's running on python, working on rebuilding it with clojure & om.next.

Does anyone have a production app on appengine ? From what I understand it should work well, except for core.async (since appengine does not allow you to create threads).

@claudiu : how does 10k page views / day only result in 3$ / month? shouldn't the bandiwdth alone push you over that?

@nickbauman: that looks interesting. Do you plan to split the different parts into libraries? We are running on GKE, but are using Datastore and have been struggling with defining a query language.

@qqq This is probably the result of the GAE free quota. https://cloud.google.com/appengine/docs/quotas 1GB of free daily traffic is included

@qqq On appengine you have free quotas. I don't pay for server instances, datastore usage, and the 1GB limit for bandwidth. I also have all the images through http://cloudflare.com & amazon s3.

https://www.infocaini.ro/ this is the website (python for now). But would cost a lot to have it on mysql + and with custom instances.

Not to mention that it would not scale automatically that well... had 1 week spike in trafic 170k pageview per day.... Only noticed because I got a billing alert. The website scaled perfectly, cost were really low except for bandwith.

solved the problem quickly by adding cloudflare on top of the google storage (the charge per month not per GB) and added lazy loading for images.

the only thing that currently anger me about GAE are: (1) lack of websockets (2) jvm warmstart time (10 seconds???) I need a clj -> go compiled for the server side

@qqq haven't looked into it much, but know they have something called channel api

https://cloud.google.com/appengine/docs/java/channel/javascript

@qqq why is it a big problem with the 10 seconds ? Free quota allows you to keep 1 instance running all the time, and another will be started only if trafic increases

@claudiu : channel api is deprecated; only available one now is firebase

was nota ware free alowed 1 instance running at all times

I tried it oncd, and got an unpleasant $30 bill for just 2 weeks

oh yep, firebase seems very nice but way to pricy

https://cloud.google.com/appengine/docs/java/sockets/ -> seems to be in beta now

https://clojurians.slack.com/archives/google-cloud/p1484290722000148 cloud datastore has the problem of being regional though.

You can currently have one per project and it's set to that one region.

I figure when I'm done with US and have to conquer other countires, I can hire professionals to scale. 🙂

fair, early optimisation is the devil.

@stijn That is exactly the plan. The query DSL will eventually become its own library and will get pulled in by the template. I just don’t think it’s quite “cooked” yet.

I want to add projection query support to it, for one, and I want to add so-called “kindless" ancestor queries to it, which turns out to be not that easy based on the current design. I also want the query API to nominally work without the defentity macro for people who don’t want to use that if they want. Those last two are related to each other intrinsically.

@qqq you can work with the channel API fairly straightforwardly. You just have to use the platform’s managed sockets, which are socket-like things that implement a subset of java.io.Socket.

I might be able to come up with a code snippet that will help you get started.

@poooogles funny I have friends who live overseas who want to watch things on American Nextflix and want to buy goods on American Amazon. Because their own versions of Netflix and Amazon don’t have them. I suspect the desire to have a “global” database is a specious one. Your laws and regulations and user requirements are not as portable as you think. You will need to have a different system of record anyway.

@nickbauman https://static.googleusercontent.com/media/research.google.com/en//archive/spanner-osdi2012.pdf is pretty handy when you have access to it 🙂

But yes, normally the limits reside outside of technology. It's just a shame you can't have multiple cloud datastores in the same project.

@poooogles As a mitigating factor, I’ve used post-put hooks to write JSON data to a GCS file in a format suitable for BigQuery and restore into Datastore. This could be used in a formal process to do and ETL into other region’s system of record. Some kind of reconciliation process that respects the laws, regulations and expectations of the other part of your global audience.

Snap! Now I realize that http://github.com/nickbauman/cljgae-template doesn’t support hooks. I will have to add that.

@nickbauman : (re websockets / channels on GCP) -- I don't understand. Channels API is being depreciated. We can't run websockets on GAE due to 30-sec timeout; but yes, I agree it's eash to run websockets on GCE.

Oh I dropped the 'web' in my mind and just paid attention to the 'socket' part 😊

My sense is that websockets fly in the face of stateless services. A server that can hold onto many open sockets at once doesn't also mean it can move data through all those sockets. It's a design disconcert. If you want to do that kind of thing it would be better to go with an approach where the browser is the System of Record (using local storage) and have background threads upload all state changed to the server AS A BACKUP that can be replayed downstream if the user logs in on a different machine.

The fastest way to expedite a job is not to have to do it in the first place.

Turn the problem around.

@nickbauman : how would you implement something that required real time communication, like (1) slack or (2) http://slither.io ?

Don't use the web. XMPP comes to mind in your specific case.

https://cloud.google.com/appengine/articles/using_xmpp GOGO must really hate real time

Oh man that's really weird.

Because GCM uses XMPP heavily!!

to the best of my knowledge, real time is impossible on GAE

so we have ot use GCE, at which polints, anything is possible

I did a fairly large integration last year using XMPP with GCM.

Right.

My point remains though. The web isn't great for real-time.

https://firebase.google.com/docs/cloud-messaging/

The XMPP support we benched as much faster than using HTTP.