Clojurians Log v2

@eyelidlessness: re the bdd stuff - you know about the testing macro in core.test?

@lfn3 i do, but i found (along with my team) that the core test framework didn't tend to encourage expressing behavior. honestly it may be a naming preference more than anything, but the rspec-ish semantics of describe / it / should encourages good testing

Anyone used Cascalog on a very recent version of Hadoop? I’m using Hadoop 2.7.1 and my Map-Reduce jobs runs fine on a local hadoop instance for that version but not on the cluster. I get a Class Not Found Exception for

cascading.tap.hadoop.io.MultiInputSplit

Hey @agile_geek it’s months since I looked at Cascading in general, but when I did it was not Hadoop 2.x compatible...

@maleghast: might explain the issue then! I strongly suspected this. I’ve asked the Cascalog user Google mail list so may get confirmation.

@agile_geek: Yeah, I was going to try it out a few months back ‘cos I’d seen some good talks and people like @otfrom had said very nice things(tm) but then I went and looked into it and it was all still tied to Hadoop 1.x and I was really turned off.

Hopefully this is changing / has changed, but if it hasn’t you might want to look at Onyx as an alternative..? (I’m still finding my way and it might not be an analog / synonym after all, but maybe)

This http://www.cascading.org/support/compatibility/ suggests Cascading 3.0.2 does support 2.7.1 but I’m not clear whether Cascalog is using that version of Cascading’s API.

Strange thing is that I can run this on my local hadoop distribution of 2.7.1 but it fails in the cluster

I’m wondering if the distro of Hadoop I’m using has Cascading installed.

In the future I will try to make a sort of financial web app with clojure. I could store the amounts on a agents like this (def cash_on_hand (agent 0 :validator number?)) so I can change the amount of cash-on-hands

Now I wonder what I the best way to "store" the transactions so I can make a report on all the transaction of a account. Store it in a list of {} ?

@roelof: a vector of maps is probably a good place to start. How do you envisage accessing the transactions?

If you always have an id for the account and want to go straight to all the transactions for a specific account then a map with the key being the account id and the value the vector of transaction maps might be the model you are looking for.

Of course you would probably want to persist the data so in reality you would probably not use an atom and just persist to some data store.

Think about what your data should look like and how you would most frequently access it then model it accordingly.

@agile_geek: thanks . persistent is something that I would be adding later . I think I need a database for it

I have to figure out which one. pqsql, mysql or maybe datomic

Hi, is anyone here familiar with postal ?

@roelof: sure. In general, use maps for key based access, vectors for random access and lists when you only ever access sequentially from the beginning. Lists are the least used data structure in Clojure.

agile_geek: then maps are my first choice

At this moment, my project is a toy project so I can experiment with things

I'm in need of some clarification about something, since send-message doesn't appear to be actually delivering any mail if I use :host "localhost"

On an Amazon Linux AMI instance

@agile_geek: can you recommend a database for my project ?

I've fiddled around with using smtp specifically gmail, but apparently I'm thrown an error more specifically javax.mail.AuthenticationFailedException "Please log in via your web browser and try again" even after using correct credentials to authenticate to the SMTP

@trancehime: I remember this being fiddly as hell when we were setting that up, I'll see if I can remember what made it work. Using postal (or clojurewerkz mailer, which uses it under the hood)?

I'm using postal yeah

I think setting "use less secure apps" as described here is what did it - https://support.google.com/accounts/answer/6009563?hl=en - but that didn't work immediately I think.

Finicky as all hell. I use :ssl true for SMTP authentication to test email delivery through GMAIL, but that didn't work

I've done that too

Actually doing that is what threw me the authentication error in my logs

I had to "use less secure apps" otherwise I'd receive an email telling me an insecure app was trying to access my email.

So after fixing that little hiccup, I tried again, this time from my Amazon ec2 instance (because I develop on a Windows machine and I don't have enough time to get send-mail or whatever working on here) and that's how I got the AuthenticationFailedException error. z.z

The config that was working looks like so:

:mailer-delivery-mode :smtp
:mailer-user "<mailto:whatever@gmail.com|whatever@gmail.com>"
:mailer-password "password"
:mailer-host "<http://smtp.gmail.com|smtp.gmail.com>"
:mailer-port 587

I imagine you have it the same though

I'll try port 587 ;o

but for now I gotta go home

@roelof: start slowly so pick something you are familiar with already (if you are familiar with anything). I buck the trend towards NoSQL databases by not choosing them by default so I tend to start with the assumption that I need a Relational database. Then it’s up to you. Postgres or MySQL are good places to start. Once you’ve installed them and create the tables you need I would read up on how to use just clojure/java.jdbc to start with. Then you might want to consider a Clojure library that abstracts away from the low level jdbc api like Korma but personally I like yesql (and @krisajenkins didn’t pay me to say that!)

@trancehime: FYI, it says the settings need to be different if you use 2-step verification; I don't (no phone) so maybe that's why it was enough to set less secure apps - https://galleryserverpro.com/use-gmail-as-your-smtp-server-even-when-using-2-factor-authentication-2-step-verification/

Re: databases - if you don't want to use plain JDBC, then I think clojure.jdbc is nicer to use than clojure.java.jdbc and certainly has nicer documentation. Otheriwse yeah, yesql seems pretty cool for beginners (and if you are doing a web application you can choose to use the Luminus "framework" which has that set up for you). Also, I suggest using PostgreSQL over MySQL, mainly because PostgreSQL cares about quality of their database, unlike MySQL : V #flamebait

Yeah, Ive been using yesql

the database i hve to connect to is mysql, I don't have control over that

oh wait

I'm stupid, mobile app

Ha, yeah, I was adding to what @agile_geek replied to @roelof xD

Still, yesql seems.better.rhan korma

I say this as a clojure noob

But since you have a mobile app, you have a mobile phone and you might have 2factor auth turned on

So check out what that link suggest doing, may help.

Haha no I mean Im using the slack mobile app

Well yeah, korma is too magical.

But I guess your point still stands

i think i may have fprgotten about 2 step auth

@roelof sure use

clojure.jdbc

@roelof: here's link to clojure.jdbc docs for convenience - http://funcool.github.io/clojure.jdbc/latest/

@agile_geek: yeah, that is nice, but I sometimes feel it's lacking some conditional templating; as it is if I want to filter by say, a combination of three attributes, then I would have to write nine queries to cover all the possibilities, wouldn't I?

@jaen: yes. Although you could fall back to raw-sql and construct a query string dynamically. I think if your use case makes this common then it falls under Kris’ advice in ‘When Should I Not Use Yesql’ (maybe he needs to extend the paragraph to include this use case?)

Hmm, I don't think so

The paragraph only talks about translating into different dialects

For example the Ruby counterpart uses ERB for query substitution which gives more leeway in how to structure queries

I'm fairly convinced such an addition would make sense for yesql.

I think for a financial app nosql cannot be used. Can Atomic be a solution ?

what makes you think that?

I just googled and see a lot of pages that said Atomic is one of the best DB which can work with clojure

@agile_geek: Im not familiar with any of them. I come from the ruby world where all is things is hidden in I think thet call it a orm

@roelof: not used AtomicDB but I wouldn’t jump to a non-relational database unless you need massive scaling capability. Stick with Postgres. In the Rails would you will have had a relational database behind Rails.

@agile_geek: thanks. then I will read the docs to decide which middleware I can use the best

@roelof: I’m afraid that if you’re not familiar with relational databases this may be a steep learning curve as none of the Clojure abstractions hide the base SQL like feel from you…deliberately. An ORM is not sensible abstraction in Clojure as there are no Objects (the O of ORM)

oke, another steep learning code 😞 I think I will then firs make a app which work in memory with agents and maps

and go on learning about clojure more with 4clojure

@agile_geek: can you help me then with my question on the beginners channel