aws

http://status.aws.amazon.com/ https://www.expeditedssl.com/aws-in-plain-english
defa 2020-02-06T12:24:43.069900Z

Is cognitect.aws.client supposed to work with other S3 providers than amazon? I’m trying to connect to a NetApp StorageGRID S3 endpoint but don’t know how.

(aws/client {:api :s3
             :endpoint-override {:hostname "<http://s3.example.com|s3.example.com>"}})
gives me an error:
Execution error (ExceptionInfo) at cognitect.aws.client.api/client (api.clj:94).
No known endpoint.

2020-02-07T15:05:08.072700Z

aws-api's interactions with aws are based on data descriptions of their apis: (https://github.com/aws/aws-sdk-js/tree/master/apis)

2020-02-07T15:07:22.072900Z

AWS does not make any promises about those descriptions aligning with specs that other vendors use to make s3-compatible services, so neither can we.

2020-02-07T15:09:34.073100Z

Actually, looking at the error you posted above, "no known endpoint", that comes from aws-api even before trying to send a request out.

2020-02-07T15:44:37.073300Z

OK, so the answer to your original question is that no, aws api does not promise to work with endpoints outside of aws. Please feel free to submit an issue https://github.com/cognitect-labs/aws-api/issues and we can continue the convo there.

defa 2020-02-07T16:18:22.073500Z

When supplying a valid :region I manged to connect to a NetApp StorageGRID S3 endpoint.

(def s3 (aws/client {:api                  :s3
                     :region               "us-west-1"
                     :endpoint-override    {:protocol :https
                                            :hostname "<http://s3.example.com|s3.example.com>"}
                     :credentials-provider (credentials/basic-credentials-provider
                                             {:access-key-id     ACCESS_KEY
                                              :secret-access-key SECRET_KEY})}))
(aws/invoke s3 {:op :ListBuckets})

defa 2020-02-07T16:21:16.073700Z

withou :region the aws/client runs for a while an than failes with an error:

Execution error (ExceptionInfo) at cognitect.aws.region$chain_region_provider$reify__13062/fetch (region.clj:37).
No region found by any region provider.

defa 2020-02-07T16:22:23.073900Z

with :region "foo" the error is a little misleading:

Execution error (ExceptionInfo) at cognitect.aws.client.api/client (api.clj:94).
No known endpoint. 

defa 2020-02-07T16:23:58.074100Z

… but with :region "eu-central-1" the client is created and (aws/invoke s3 {:op :ListBuckets}) successfully delivers the expected results

jsyrjala 2020-02-06T13:29:13.070200Z

;; Mock S3 Service ;; https://github.com/adobe/S3Mock ;; https://github.com/adobe/S3Mock/blob/master/server/src/main/java/com/adobe/testing/s3mock/S3MockApplication.java (def props {S3MockApplication/PROP_HTTPS_PORT S3MockApplication/RANDOM_PORT       S3MockApplication/PROP_HTTP_PORT S3MockApplication/RANDOM_PORT       S3MockApplication/PROP_INITIAL_BUCKETS bucket       S3MockApplication/PROP_SILENT “true”       }) (def s3mock (S3MockApplication/start (HashMap. props) (into-array String []))) (def s3 (aws-api/client {:api :s3               :endpoint-override {:protocol :http                        :hostname “localhost”                        :port  (.getHttpPort s3mock)}}))

jsyrjala 2020-02-06T13:29:51.070600Z

I think I got it working against locally running mock server

kenny 2020-02-06T16:31:42.071Z

Can you create a presigned url with aws-api?

kenny 2020-02-06T16:35:19.071100Z

Ah https://github.com/cognitect-labs/aws-api/issues/5. Anyone have a workaround for now?

kenny 2020-02-06T16:36:48.071400Z

It sounds like the pieces are there and just need to be assembled.

1✅
ghadi 2020-02-06T17:51:42.071700Z

the s3 v4 signing api is really straightforward

ghadi 2020-02-06T17:51:48.071900Z

(the non-streaming variant)