Clojurians Log v2

@kwladyka https://gist.github.com/stijnopheide/af90400b26a686e1145b9a5e2f0a2874 should get you started with lacinia + yada + authentication

Thank you, I found good article about auth for GraphQL https://www.apollographql.com/docs/apollo-server/features/authentication.html

How do you solve download files?

just to be sure, that's on older versions of yada / lacinia, but I think they are compatible with the current release

Sessions?

the only file upload / download we have in this app is images and we have separate resources for that (outside of graphql)

and yes we use the same cookie as we use for the authentication with the graphql endpoint

In other words I don’t see option to do auth only in GraphQL, because there is no way to not use REST, because there is always needs for download files

and upload

the example gist includes a bit more of the machinery of graphql query parsing than the edge example (https://github.com/juxt/edge/blob/master/phonebook-graphql/src/edge/phonebook/graphql.clj) because according to the spec you can also put your query in a GET request and choose whether you put it in the body or query parameter

it matched (at some point) the implementation from lacinia-pedestal, but I didn't bother keeping it up to date

yeah, I don't think that auth in GraphQL is the best option, UNLESS you only use that one endpoint

the advantage of relying on auth outside of graphql is that you can e.g. use browser cookies

Do you know how to use Redis with yada for sessions?

Is any middleware like for ring for that purpose?

Can I use ring middleware with yada?

https://github.com/clojusc/ring-redis-session

you could do ring middleware with yada, but you have to translate it into interceptors (i.e. the pre-processing and post-processing need to be split because yada is asynchronous)

@kwladyka we save sessions in an atom, but we could easily swap it out with Redis. It’s something we do in our auth scheme.

and we do sessions in our datomic database, where we stick it next to the user

sure, just I would like to glue Redis with yada, instead make +100 lines of code with my own solution

with ring it is 1 line 😉

(defonce user-profiles
  (atom {}))

(defmethod yada.security/verify "...." ...) 

There is no info in doc how to do interceptors in yada

*doc in progress

oh it could be quite easy hmm

you should read the part about security, because that’s the place where you should probably hook into (although you could do it with your own interceptor, just more work): https://juxt.pro/yada/manual/index.html#security

thanks, I have to learn how to manage sessions in yada before make this auth

yada even has support for roles and authorization, which we also use

here's cookie authentication for yada (read side): https://gist.github.com/stijnopheide/af90400b26a686e1145b9a5e2f0a2874#file-api-clj-L1

it's not going to take 100 lines

thanks, I have to read all this things now 😉

cookies are parsed by yada and put in the context, you have to implement a verify function for your auth and check the db if the cookie is valid

and like @borkdude suggests, start off with an atom and swap that out with a database once you have the need to have more than 1 instance of your webserver running

(or configure your load balancer to have sticky sessions :-D)

we have to deal with SSO and cookies from another subdomain. it wasn’t that hard to make it work

yes it is good idea. On the other hand I assume I will do often update on the beginning and people will lost they session everyday 😉

we verify our session every hour against some third party SSO service

@stijn is right, but things are improving in 1.3.0. In 1.2.x the cookies were parsed by the swagger parameters code so it was a useful side-effect they were available for response functions to use. 1.3.x formalises cookies - see https://github.com/juxt/yada/blob/master/doc/cookies.adoc - they are now declarative (because it's useful to have cookie parameters as data for further introspection, rather than having to hand-code them - of course you can still hand-code the traditional way if you need that).

cookie declarations can also specify a :consumer function - that means you can register a callback which will get called, and only called, with the value of the incoming cookie. Yada will work out which cookies belong to which consumers. Your cookie consumer gets the yada context, and can return it augmented with anything you like, including credentials which you can use later on in your authorization functions.

in future, it might be possible to specify a signing or encryption key

RFC 6265 Section 8.3:

> Servers SHOULD encrypt and sign the contents of cookies (using whatever format the server desires) when transmitting them to the user agent (even when sending the cookies over a secure channel).

yada will be able to do that for you (signing, encrypting, and checking incoming) once the declarations are in place

I've tried very carefully to preserve compatibility with previous yada versions. If, for any reason, your existing code doesn't work with 1.3.x, let me know because that's a bug.

alright, I’ll try out 1.3.0 and see if our scheme still works

we haven’t upgraded in a while:

;; Yada
     [yada "1.2.15"] ;; includes Ring 1.6.0-Beta
     [aleph "0.4.6"]

great - thanks @borkdude - I'll keep the alpha label for a while until things stabilizes. 1.2.15 is the most recent 1.2.x, there hasn't been many releases recently. I've pulled 1.2.16 due to a compatibility issue

pulled back from clojars?

@kwladyka I've been thinking about your feedback about the manual. I think it would be good to create a cheatsheet explaining all the keys you can put in a yada resource, for quick reference.

oh yes, it will be very good

there will be great to add to tutorial example of how to write yada POST (form / body )+ maybe GraphQL + how to write tests for that

I took me about 1 week to figure out

Most of people can give up 🙂

*but probably I lost most of the time for my magic bug 🙂

@malcolmsparks so far it still seems to work. I’ll keep you posted. I’m still using aleph 0.4.6.

Edge has a phonebook example for form/body posts and some GraphQL integration. yada does integrate with lacinia, including subscriptions, very well

https://clojurians.slack.com/archives/C0702A7SB/p1547920037184600

@borkdude me too - I can't get more recent versions of aleph to work - I'm getting a weird stack trace whenever I try

Yes there is an example, but without tests

@kwladyka I've fixed an issue this morning that caused tests to fail when upgrading from ring 1.6.0 to ring 1.7.1 - that might have been your problem

At that moment I do tests like that:

(defmacro with-server [handler build-url & body]
  `(let [listener# (yada/listener ~handler)
         close# (:close listener#)
         port# (:port listener#)
         ~build-url (fn [path#]
                      (str "<http://localhost:>" port# path#))]
     (try
       ~@body
       (finally
         (close#)))))

(deftest graphql-test
  (with-server core/handler build-url
    (testing "Graphql"
      (is (= 405 (-&gt; @(http/request
                        {:url (build-url "/graphql")
                         :method :get})
                     :status))
          "GET is not allowed.")
      (is (= 401 (-&gt; @(http/request
                        {:url (build-url "/graphql")
                         :headers {"Content-Type" "application/json"}
                         :method :post
                         :body "{\"email\":\"<mailto:foo@example.com|foo@example.com>\",\"password\":\"qwaszx\"}"})
                     :status))
          "Not authorized")
      #_(is (= 200 (-&gt; @(http/request
                          {:url (build-url "/graphql")
                           :headers {"Content-Type" "application/json"}
                           :method :post
                           :params {:query "{ game_by_id(id: \"1237\") { name designers { name }}}"}})
                       :status))
            "Authentication by session")
      (is (= 200 (-&gt; @(http/request
                        {:url (build-url "/graphql")
                         :headers {"Content-Type" "application/json"
                                   "Authorization" "bearer 89abddfb-2cff-4fda-83e6-13221f0c3d4f"}
                         :method :post
                         :body "{\"email\":\"<mailto:foo@example.com|foo@example.com>\",\"password\":\"qwaszx\"}"})
                     :status))
          "Authentication by token"))))

@malcolmsparks I will try the alpha too. Any other changes that we need to be aware of?

no, the issue is it is not obviously yada/response-for doesn’t work with :body

I mean, there is no way to send JSON body for example

mostly the changes are confined to a new security design (using new entries in the resource). The old design and old code is still supported, so as long as you don't mix old/new in the same resource, everything should work fine

(if you do mix, the schema validation will protect you)

@malcolmsparks is it recommended to upgrade to any new things? migration release notes?

I don't think there's any migration issues with 1.3.x - it should work fine with existing yada code -

So on the end I have to run listener normally and normally use HTTP client. But it is not obviously on the beginning.

Bidi question: is there a way to do request logging at the bidi level? my use case is, I want to log ALL requests in a bidi tree, also the ones that generate a 404 because there is no matching route. I know about the catch-all routes and could insert a resource at the end, but 1/ if I understand bidi matching correctly I would need to do it for each subtree? 2/ a swaggered endpoint doesn't like a 'true' route.

@kwladyka yeah, there could be more help and support around how to test

Especially when I have habit to use tools like https://github.com/xeqi/peridot

@stijn you only need one catch-all at the end of the structure. if a sub-tree 'fails' to match, matching continues

oh OK, I didn't know that

but some way of logging would be nice - a design feature of bidi is that it separates the data structure containing the routing with the algos that does the matching. So it should be possible to provide a slower algorithm that performs logging which could be enabled on a per-request basis.

it is sometimes a real pain to debug bidi routes

yeah, that’s a pain

especially the compact vector notation

There are also more questions like: :consumes #{"application/x-www-form-urlencoded" "application/edn" "application/json"} So how to write :parameters {:body s/Any} On different content type it should be validate different probably. Sometimes it is a String, sometimes EDN, sometimes normal form

so while resource require :body when it is JSON and :form :query when it is application/x-www-form-urlencoded, no idea how to code it at that moment

yada will coerce the request body to the right 'shape'. If the content differs on a per-content-type basis, then you need to validate in the response function. The response function can call yada/content-type on the context to discover which type was negotiated

@kwladyka I think this is a general problem with how Swagger views the world. In HTTP, there are only request bodies. There is no difference between a form and a body.

Mainly I want to show you issues for person how start with yada to let you make better tutorial 🙂

Swagger makes assumptions about how HTTP applications should be written which is somewhat narrower than HTTP.

Yes, I appreciate that @kwladyka, thanks

The nice thing about IETF RFCs is that the authors make a real effort to keep everything consistent with other RFCs. However, with Swagger, the authors are not so constrained and I feel they complicate things. One of my regrets with yada's design is making parameters a core feature - I'd prefer to deprecate existing parameter declarations and find another way, via extensions, to support Swagger. The Swagger support is so old in yada I'm not sure if new users use it any more. But I've heard good arguments that Swagger should be retained in yada.

So in future versions of yada I think there'll be alternative approaches to dealing with parameters, form validation, etc. Perhaps using libraries that more naturally fit those domains.

We’re actively using the swagger support in yada

Good to know. There's obviously a lot that's happened in the Swagger world since the first yada releases, for example, renaming to OpenAPI

I know a lot of teams find swagger useful to communicate their APIs with downstream consumers

I’m open to upgrading our codebase to something newer. Migration guide would be helpful

yes, that’s what we are doing

@malcolmsparks from my point of view about form validator everything can be good, unless it force me to have validation code in 2 different forms. I mean if I will have to have spec and special yada code for doing the same it is bad for me 🙂

@borkdude do you use the built-in swagger console? or just the swagger decls that yada resources produce

let me check

@kwladyka yes, I think it's better to allow users to do their own parameter handling and validation - but provide the callbacks in the right place so that code can indicate failure and error messages for 400 responses.

I think we’re using the builtin thing:

(defn new-swagger-resource
  [system routes]
  (resource
   system
   (let [spec (yada/swagger-spec
               routes
               {:info {:title "DOC Search API"
                       :version "1.0.0"
                       :description (slurp
                                     (io/resource
                                      "swagger/description.md"))}
                :tags api-tags})]
     {:response (fn [ctx] spec)})))

["/swagger-ui" (->
                    (yada/new-webjar-resource "swagger-ui" {:index-files ["index.html"]})
                    ;; Tag it so we can create an href to the Swagger UI
                    (tag :dre.resources/swagger))]

right - thanks

this was actually one of the things that made me choose yada over pedestal. it was hard to get swagger working with pedestal I found (2 years ago)

that swagger-ui is quite old now, i'm considering a refresh

the lib I found for that was very macro-heavy, I decided not to go ahead with it after giving it a try

I am not suggesting for a moment that yada will remove swagger, but it may be necessary for you to add an extra library dependency at some point in the future

(I believe it was also made by someone from juxt)

yes, we've done a lot with pedestal over the years, on various projects. I know what you mean about the macros.

no problem. I was referring to an alternative to parameters. I wouldn’t mind upgrading, but a guide would be most helpful

this was the lib in question: https://github.com/frankiesardo/route-swagger

we use swagger for a lack of alternative, but I'm not at all a fan

The existing parameters design will be kept. Any new designs will require new yada context keys. Things will grow by accretion, I don't feel like going back and rewriting old working code.

cool

it feels indeed like yada is 'opinionated' towards json that way. we do XML and edn/transit too and it doesn't really have any support for that (I mean in terms of documentation, not the processing)

I have to go, see you later! 🙂

but it's hard to support several formats I guess, I wouldn't know how a tool could communicate XML structure / json schema and transit/edn

@malcolmsparks I remember upgrading the swagger UI once, but our team didn’t like the change. big responses made the UI unresponsive. I don’t remember the details, but we decided to revert it.

Is https://juxt.pro/yada/manual/index.html#cookie-authentication is outdate and doesn’t work?

when use :cookie > Cannot turn resource-model into resource, because it doesn’t conform to a resource-model schema

{:realms {"session" {:authentication-schemes [{:scheme :cookie
                                                                    ;:cookie  "session"
                                                                    :verify (fn [cookie]
                                                                              (println "auth cookie" cookie)
                                                                              {:a 1})}]
                                          :authorization {:validate (fn [ctx creds]
                                                                      (println "creds" creds)
                                                                      ctx)}}}}

I don’t see in code defmethod verify :cookie. Maybe it is in the doc, but in code?

https://github.com/juxt/yada/pull/248

^ok it is confirmed

FWIW, I have:

(defmethod yada.security/verify "my_auth_scheme"
  [{:keys [resource cookies] :as ctx}
   {:keys [_verify _scheme]}]
 ...

I’m looking up the cookie in the context like this: (get-in ctx [:cookies ".MYCOOKIE"])

I am thinking if I can use this one https://github.com/juxt/yada/blob/master/ext/oauth2/src/yada/oauth.clj#L242 but I am to tired today. It is time to go sleep.

Good night

@kwladyka cookie auth is out-of-date, a replacement is coming. Basically the replacement is explained here: https://juxt.pro/yada/manual/index.html#cookies

The summary is that there is no such thing as 'cookie auth' - there is HTTP auth schemes which use the Authorization header, and there is state management via cookies.

The new 1.3.x version of yada allows you to do your own auth with cookies in that cookie consumers can add info the the context for the authorize interceptor to work with.