Hi I trying to stream the result from aws s3 and process it. What I am trying is something like
(def s3-result (aws/invoke client {:op :GetObject
:request {:Bucket "my-bucket"
:Key "path/to/some/big-file.csv"
:Range "bytes=0-30000" ;; just an example
}))
;; I want to process the file as it come in line-by-line without holding the things in memory
(-> s3-result
:Body)
;;=> this is of the type <http://java.io|java.io>.BufferedInputStream
I know that the result is available in the :Body
after the request.
Now does anyone know how to do this lazily?aws-api doesn't support streaming yet. https://github.com/cognitect-labs/aws-api/issues/14
We do plan to support "bring your own http-client" in the future, but there is no timeline for that.
@dchelimsky Thanks for the clarification.
You're welcome.
[ANN] Cognitect Labs' aws-api 0.8.437 is now available! https://groups.google.com/forum/#!topic/clojure/AIQzOkhUDBU
!Ref "#off-topic"