Clojurians Log v2

@macropolo: hello! do u live in hong kong or just curious about whats going on here?

@onetom: hi unfortunately not, however I do visit every once in a while, so I was interested in seeing what is going on wrt clojure in hk

@onetom: great city btw, first came in 2011 and fell in love :simple_smile:

most people u see here are from https://exiconglobal.com and we are developing our platform in clojure http://hoplon.io and datomic

@onetom: interesting

datomic is great, I have the pleasure of using it in one of my projects

hey everyone, it would be great if you could introduce yourself

we should have a meetup too. my ex-colleague has created this months ago: http://www.meetup.com/clojure-hk/ but we never had a meeting yet

macropolo: what kind of data do u store in datomic? we just started using it actively recently and we store mobile app and app store data in it, besides our user and org data

@onetom: the project is concerned with geocoding and address processing, so, naturally, most of the data are hierarchical address information plus some request logging/job control/etc.

datasets are smallish, tens to hundreds of millions of datoms

that sounds like a gigabyte range db already

also it's an interesting type of data. why did u choose datomic? r u using its time travelling functionality? are u annotating your transactions with the user's or software subsystem's id for audit purposes?

gigabytes don't seem so big these days :simple_smile:

well... i was just dealing with a datomic db which occupied 4.6GB on dynamodb (by accident really, because we left of some uniqueness constraints and over a few months it has accumlated a lot of duplicate datoms :simple_smile:

it deleted that db the other day and ran bin/datomic gc-deleted-dbs on it and it took 70minutes to complete it

and even after that there was ~1.8GB garbage left over

there are no reqirements for time travel/strong audit capabilities per se, however datomic is quite suited for hierarchical data (vs say sql which will work but would look rather ugly) and read/write asymmetry

then i tried to rename the latest backup of it on s3 by aws s3 mv <s3://backup/orig-name> <s3://backup/archived-name> and that took 2.5hours and ~700'000 aws api operations.

we don't use dynamo, so disk space/iops are almost free

i should have run that from an EC2 machine probably, but still...

what storage do u use then?

local one since we currently can get away with it

yeah, hierarchical data is definitely a sweet spot for datoimc

u mean the built-in h2 datomic:dev:// protocol?

yes

on an unrelated note, bulk-load performance was somewhat of a problem (required index tuning), however I think it improved in recent releases

i was entertaining that idea too for a while but since we run on aws im not sure how much can i trust EBS to store it and even than the uptime might be not so nice

which version are u using?

i agree, if using aws dynamodb is a better idea

we just bumped up to the latest 0.9.5327 because the .5132 one had some disconnect error when we were creating and dropping in-memory dbs too quickly or something like that

0.9.5302 currently

otherwise ror/nodejs/angular/meteor is still the mainstream requirement in most jobs

there are quite a few companies who are experimenting with go and react too

the hesitation about developer availability is quite understandable, however i think currently there are more clojure developers than clojure jobs and also at least some of them are prepared to take a pay cut or compromise on other concerns just to use it in their workplace instead of whatever they are allowed now

as for go among other recently popular languages it is pretty much the anti-clojure :simple_smile:

some speculate it was google's elaborate attempt to ensure none of its competitors ever get ahead sorry if that offended anyone :simple_smile:

"anti-clojure" :simple_smile: my 1st association was anti-christ

so if u say there is a clojure dev surplus, what do u think how can i find them?

hmmm... i see there are some job channels on clojurians...

regarding paycut, i also made that compromise, but i dont really regret it (yet :simple_smile: it's really rewarding to work in clojure after node, ruby, etc...

i think the usual suspects (channels here, other communities and various job sites) would produce some results my statement is mostly based on "i like/learn/use clojure for my projects but my job doesn't let me do the same" i heard more than once :simple_smile: not tried to hire anyone yet however if you want clojure devs in a specific location and don't allow remote people, this it of course is more complicated since mean clojure developer density is obviously much lower

anyway, being able to give people cool toys (clojure in this case) may be an useful advantage when competing for developers with other organizations

that's what i thought too, but failed twice already in the past 9 months... in those cases ppl valued salary more 😞

did you look in hk only or for remote devs too?