portkey

Portkey: from REPL to Serverless in one call
tatut 2017-09-04T12:30:28.000175Z

yes

tatut 2017-09-04T12:38:45.000103Z

Error loading class portkey.LambdaStub: class java.lang.ExceptionInInitializerError
java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
Caused by: com.esotericsoftware.kryo.KryoException: Unable to find class: clojure.core.specs.alpha$fn__24
Serialization trace:
pred (clojure.spec.alpha$spec_impl$reify__751)
val (clojure.lang.Delay)
specs (clojure.spec.alpha$and_spec_impl$fn__869)
cform (clojure.spec.alpha$and_spec_impl$reify__875)
value (java.util.concurrent.atomic.AtomicReference)
state (clojure.lang.Atom)
at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:160) ...

tatut 2017-09-04T12:39:07.000193Z

trying to run a lambda that does a specql query

cgrand 2017-09-04T12:45:19.000153Z

whitelisting...

cgrand 2017-09-04T12:46:04.000258Z

we whitelisted clojure.core.* which is... brutal

cgrand 2017-09-04T12:46:47.000114Z

that’s why clojure.core.specs.alpha$fn__24 didn’t made it to the other side of the net

cgrand 2017-09-04T12:49:01.000119Z

@tatut btw really good job on SpecQL, when/if I can revisit DataFrames support in Powderkeg I will look at it in closer details

tatut 2017-09-04T12:59:29.000494Z

thanks

cgrand 2017-09-04T13:00:17.000366Z

I’m going to push a tentative fix in a couple of minutes

tatut 2017-09-04T13:00:23.000140Z

oh cool

cgrand 2017-09-04T13:05:48.000363Z

@tatut ^^that’s for you

tatut 2017-09-04T13:10:26.000238Z

I’ll try it out

tatut 2017-09-04T13:18:25.000076Z

that seems to work

tatut 2017-09-04T13:18:56.000119Z

I tried by simply returning an (s/explain-str ::my-spec some-data) from the lambda and it worked fine

tatut 2017-09-04T13:28:38.000412Z

great success! I now have a lambda that does a specql query and outputs an HTML page with the results..

🐲 2
cgrand 2017-09-04T13:29:54.000369Z

against RDS?

tatut 2017-09-04T13:30:02.000302Z

yes

tatut 2017-09-04T13:30:57.000271Z

probably wouldn’t do this in production, I’m just doing a simple polling app for a dev meetup we are hosting next week

cgrand 2017-09-04T13:33:05.000279Z

re: production, because of portkey being so alpha?

tatut 2017-09-04T13:33:42.000248Z

both that and using RDS from lambda, I don’t know if it’s a good idea

cgrand 2017-09-04T13:36:20.000150Z

even with a connection pool?

tatut 2017-09-04T13:36:51.000282Z

I’m simply using a connection map without any connection pooling, that’ll do for this example

tatut 2017-09-04T13:37:31.000029Z

It seems it is a common use-case, so much so that AWS has a tutorial for it http://docs.aws.amazon.com/lambda/latest/dg/vpc-rds.html

tatut 2017-09-04T13:37:42.000209Z

so maybe that is ok for production use

tatut 2017-09-04T13:43:18.000389Z

Issue: it seems portkey can’t handle arguments with dashes in their names

tatut 2017-09-04T13:44:00.000273Z

had (defn answer [poll-id option-id]) and tried to mount with path “/answer?poll={poll-id}&option={option-id}” and got an exception about unmapped arguments

tatut 2017-09-04T13:44:08.000361Z

removing the dashes worked

viesti 2017-09-04T13:57:02.000269Z

in the python of example, connection is created on top level, so pooling/keeping connection alive for a while is ok idea

viesti 2017-09-04T13:57:46.000190Z

there was someone in audience that pointed me to Zappa (a python serverless library, haven't looked at it)

viesti 2017-09-04T13:58:27.000157Z

but neat to find out a reference for pg connection pool server such as pgbouncer in that post

tatut 2017-09-04T14:05:24.000140Z

my specql using lambda: https://3ekj3mh91k.execute-api.eu-central-1.amazonaws.com/repl/results?poll=1

🦜 1
viesti 2017-09-04T14:08:27.000086Z

Achievement unlocked! :)

tatut 2017-09-04T14:08:50.000079Z

should I report to github issues that dash problem with argument names?

cgrand 2017-09-04T14:09:00.000254Z

musing: thinking about a story for automatic stage switch for XCD (extreme continuous deployment)

cgrand 2017-09-04T14:09:12.000169Z

@tatut yes please

tatut 2017-09-04T14:09:19.000090Z

will do

cgrand 2017-09-04T14:09:39.000396Z

and didn’t you also had an issue with an anon fn (a little birdie told me)

tatut 2017-09-04T14:10:51.000307Z

I didn’t try it, just asked if it was possible

cgrand 2017-09-04T14:19:33.000163Z

@tatut thanks

tatut 2017-09-04T14:21:15.000497Z

that was quick

tatut 2017-09-04T15:11:23.000340Z

It looks like portkey doesn’t support POST, i’m getting {"message":"Missing Authentication Token"} when trying to POST to a portkey URL

tatut 2017-09-04T15:12:52.000243Z

doing side effects with GET requests is suboptimal, any thoughts on how easily it could be added?

cgrand 2017-09-04T15:15:58.000244Z

this should be more configurable: methods, input content-types etc.

cgrand 2017-09-04T15:18:06.000187Z

tangentially related but: wouldn’t it be possible to mount a ring app? @viesti?

cgrand 2017-09-04T15:21:07.000016Z

for a start having a :method option to mount!

viesti 2017-09-04T15:22:38.000201Z

I've seen some ring handler wrappers but I think they didn't get far

cgrand 2017-09-04T15:22:50.000196Z

Hey, we have 2^7 commits! 🍾

🦜 2
cgrand 2017-09-04T15:23:26.000262Z

why? 🐲s?

viesti 2017-09-04T15:31:07.000199Z

possibly not enough aws synergy

cgrand 2017-09-04T15:54:13.000118Z

please elaborate

viesti 2017-09-04T16:02:39.000042Z

on post, have to see if request template needs change https://github.com/portkey-cloud/portkey/blob/a93e3d012a0aaa6a3d8e1998fc8a02dbcd21b08d/src/main/clojure/portkey/aws.clj#L47

viesti 2017-09-04T16:05:14.000074Z

aws has SAML and authorization lambda, existing ring auth mechanism might not fit, would have to see what ring middleware is actually useful

viesti 2017-09-04T16:07:11.000079Z

to figure out what should belong to mappings in the api gateway, what done by a lambda

viesti 2017-09-04T16:39:13.000074Z

hmm, where's the tweet on RDS lambda with Portkey? :)

viesti 2017-09-04T16:50:53.000151Z

ah, found it

tatut 2017-09-04T16:59:48.000141Z

could you put the :method and :content-type into the function metadata, so that it is colocated with the definition

cgrand 2017-09-04T17:00:43.000038Z

I'm not opposed to. However in var or fn metadata it should be namespacesd.

tatut 2017-09-04T17:01:13.000145Z

yeah, something like (defn ^{::pk/content-type "text/html" ::pk/method :POST} foo [bar] ...)

cgrand 2017-09-04T17:02:04.000183Z

That's it

viesti 2017-09-04T20:10:19.000113Z

started hacking post support, nothing working yet 😕