datomic

Ask questions on the official Q&A site at https://ask.datomic.com!
twashing 2020-09-22T00:13:41.058700Z

I’m trying to launch a “Solo” Datomic Cloud CloudFormation stack. But it consistently fails with

CREATE_FAILED: Embedded stack arn:aws:cloudformation:blah:blah:stack/appname-StorageF7F305E7-S5WZ9NKP2OOE/ebc51350-fc61-11ea-9f8a-12709ad3d671 was not successfully created: The following resource(s) failed to create: [GetSystemName].
Is there anything basic I’m missing? Details are in this SO post. https://stackoverflow.com/questions/64001410/stack-resource-fails-datomic-cloud-cloudformation-launch

steveb8n 2020-09-22T04:58:42.059100Z

you could do what I have done for cloud. proxy the db/conn values and inject middleware at that layer. then you can implement your own filtering before or after the reads occur

steveb8n 2020-09-22T04:58:50.059300Z

it’s complex but works well

2020-09-22T11:48:49.059900Z

We are running HAProxy to load balance peer requests. What do you recommend to stick our sessions to? Is SSL session ID a good idea?

2020-09-22T11:49:11.060200Z

cc: @lennart.buit @jeroen.dejong

xceno 2020-09-22T11:54:09.064Z

I'm currently trying to follow the datomic ions tutorial, but after installing the datomic cli tools I'm stuck: Executing datomic system list-instances <system> fails with AWS Error: Unable to fetch region.. The alternative command using the aws cli directly, succeeds though. I've installed the aws cli via docker and made an alias. Could this be my problem? Edit: Installed AWS CLI tools via docker and tried again with a normal installation of the V1 and V2 tools. But the problem remains. Edit2: Okay nervermind. The old docker aws process was locking the .aws/credential file. With a standard aws cli V1 version everything runs fine now.

2020-09-22T12:10:33.064200Z

We were looking for something along the lines of https://www.haproxy.com/blog/maintain-affinity-based-on-ssl-session-id/

tvaughan 2020-09-22T13:04:17.064600Z

This was many years ago, but I worked on a project that tried to use SSL session ids as a session id. We abandoned it pretty quickly. There was no guarantee that clients would even try to resume an SSL session. Some had built-in timers to reinitiate SSL sessions every n minutes where n was very small. I have no idea if things have changed, but I certainly wouldn't just do this without a lot of research first

ghadi 2020-09-22T13:16:36.065300Z

@rob703 is the region set in your AWS profile?

xceno 2020-09-22T13:40:19.066700Z

Yes it's all running now, thanks! My problem was the aws docker image blocking my entire ~/.aws directory. So when i finally installed a regular V1 CLI it couldn't access the config files.

xceno 2020-09-22T13:49:08.071900Z

I have another question though, that got buried in #beginners. I want to deploy a fulcro3 project with a dependency to libpython-clj to datomic ions. It includes http-kit as server. 1. Can I just ssh into the ION EC2 instance and set up the necessary python environment? 2. When deploying a :web ion: Do I just ignore http-kit and route the incoming request straight through my ring middleware stack, like so?

(def get-items-by-type-lambda-proxy
  (apigw/ionize my-middleware-stack))

(def -main []
   (start-dev-server-normally)

Yuriy Zaytsev 2020-09-23T09:19:10.002100Z

@rob703 consider putting your python code to separate environment and interact via lambdas

xceno 2020-09-23T12:35:48.002600Z

I did consider this yesterday evening, it just looses the charm of being able to work with the code through clojure. Setting up a completely separate python environment and accessing it via remote calls just isn't the same as simply using it directly. But yes, if all things fail that will be our workaround. It won't be too bad, but I still hope someone comes up with another solution

joshkh 2020-09-22T15:15:41.072300Z

yes to 2.

2020-09-22T15:17:58.072500Z

But that wasn't an project on Datomic I presume?

joshkh 2020-09-22T15:18:19.072700Z

your web ion points to the ionizeed handler, where as an http-direct connection points to your standard, unionized handler

xceno 2020-09-22T15:20:28.072900Z

Perfect thanks! If I can figure out the python part now, I'm all set

tvaughan 2020-09-22T16:21:32.073100Z

Correct. It was not Datomic related

cjsauer 2020-09-22T17:00:13.081100Z

@steveb8n interesting. With what data does your middleware stack work with? Are the filters queries themselves, the results of which are then used as input to the next query, and so on? It seems with the client api you could end up performing large scans of the database if your filter was relatively wide...

cjsauer 2020-09-22T17:01:04.081700Z

(I think this may be part of the reason why the client api doesn’t support filter)

marshall 2020-09-22T18:10:13.082Z

the ec2 instance(s) that run datomic are intended to be ephemeral and replaceable, i would definitely recommend against modifying their environments directly

xceno 2020-09-22T18:17:20.082200Z

I see, any other ideas than on how to achieve my requirements? Further down the line we might even have the need for EC2 instances with GPU's for some machine learning and rendering tasks. I'd love to have it all neatly packaged in an ion, so we don't have to mess with all the AWS setups ourselves

marshall 2020-09-22T18:18:02.082400Z

interesting; I don’t have any ideas offhand, but I’ll bring it up w the team

xceno 2020-09-22T18:18:31.082600Z

awesome, thank you!

steveb8n 2020-09-22T22:52:18.083200Z

I went all out and added Pedestal interceptors in the proxy. The enter fns decorate the queries before execution with extra where clauses. In that way you can 1/ limit access 2/ maintain good performance

steveb8n 2020-09-22T22:52:53.083400Z

doesn’t work for d/pull so, in that case, I filter the data in the leave fn instead

steveb8n 2020-09-22T22:53:25.083600Z

for writes, the enter fns check that all references can be read using the same filters as the reads