Clojurians Log v2

I am having the most bizarre issue on my machine. I am on a Mac with the clojure cli installed via homebrew. If I launch an app or repl from cli, all network calls fail immediately with a socket error “Operation Not Permitted” and “Connection Refused”. Doing the exact same thing using leiningen works fine. Everything else is exactly the same. I’m not able to connect to databases or make http requests. I have the same issue when launching a server from figwheel (both legacy and figwheel-main). Occasionally I can get it to work for a single connection usually after uninstalling and re-installing Clojure in homebrew. I suspect this could be due to security software as my company has forced a few new scanners on our computers lately, but I’m just wondering what could possibly be different about the calls being made when launching via cli vs lein. Here is the simplest example of what fails for me. The equivalent with a project.clj and using lein repl works just fine. Any ideas what could possibly be different?

well if you want to get clj out of the debugging, you could clj -Spath and then java -cp <that-path> clojure.main

I'd wonder about Java version, and whether you have granted that Java version access to do network things

it will pop up a prompt for that - possible that's getting buried somehow through clj. just guessing

well i have tried multiple java versions. currently on adoptopenjdk 1.8. and using same version whether cli or lein

Perhaps you could also try -Srepro which ignores config in your user folder

I'd rather get clj completely out of the way first and see if it still fails

so yes it does work when doing the java -cp and still fails when doing the -Srepro

does clj / clojure differ?

they both fail

the clojure bash script ultimately basically does java -cp cp clojure.main

what if you just wrap the call to java in a simple bash script?

i’ll give it a try

#!/usr/bin/env bash

exec java -cp ... clojure.main

hmm. that fails. seems surprising

the plot thickens :)

bash?

lein is also bash that is

maybe look at perms for the clojure and lein scripts and their ownership / group?

ok. so yeah tried each method a few times and consistently calling java -cp … directly works and running the bash script with the same thing doesn’t

what if you remove the exec in the script?

that's a difference between lein script and clj script although seems hard to believe that's the difference

hmm. yes removing exec works

interesting

heh - i have some other words for it but yeah that one fits too 🙂

maybe bash is replaced by recent security tools update?

as we say at Nubank... fascinating

i am using zsh but have also tried running it via bash w/ same result. wouldn’t think that would make a difference. course i wouldn’t think exec and no exec would make a difference

bash is used by clj/clojure scripts

looks like i’m using bash from homebrew not the system bash

mac os X is no longer shipped with system bash )

ah. true. not sure if it helps at all but i’m on Catalina just upgraded last week from Mojave in an attempt to fix this issue

I don't mean to derail but i had a colleague that would get different jvms when using lein and clj. Quite subtle and due to homebrew bringing in jdk15 for lein. You mentioned you were using the same 1.8 jvm but how sure are you on this? How did you check it at runtime. (sorry if this is irrelevant but just something i saw recently)

yeah i’ve been wondering about that too. is it possible to get the jvm from repl? i do have JAVA_HOME set and the clojure script seems to use that if set rather than the openjdk installed by homebrew

clj -M -e '(System/getProperty "java.version")'

“1.8.0_292”

I don't think the java is relevant here

same as when i do java -version

smells to me like there is some kind of prohibition re fork

I can imagine something like that from a security pov

I don't know what it is or how to check for it

what’s really baffling to me is when it will work 1 time. yesterday i uninstalled clojure, did brew cleanup, brew doctor, re-installed and it worked for several attempts. but then by the end of the day, it stopped working

also when i do have a working repl or application instance, it seems to work endlessly until that is broken. so definitely seems to be some sort of permissions thing on when it starts

do higher ports work? ports < 1024 require root privileges to open I believe

network calls to higher ports?

privilege for low ports is on listening sockets, not destinations

ah, nvm

does it work with curl when it become broken with clj?

> I suspect this could be due to security software as my company has forced a few new scanners on our computers lately what's that about?

(reading through scrollback)

it's always the virus scanners

have Carbon Black and Microsoft Defender. wouldn’t shock me if there is other than I haven’t noticed

are you on windows?

but none of my co-workers are having this issue

no. mac Catalina

I didn't know Microsoft had a virus scanner for mac

my company likes to install as much Microsoft as they can though lol

the works for a while and then stops seems like it would match that theory

works very briefly then stops. almost like the first attempt gets through but then it gets blocked. but what could possibly be different about these network requests?

can always check the diagnostic logs for rejected connection attempts

will say they fail immediately. don’t think they are actually making it to the network

(AV tools, surveillance ware or macOS Console)

from the description, it doesn't seem like any network activity is happening

exec is going to replace the process so I can imagine a virus scanner seeing a process, replaced with another process space, then making network calls, as a possible virus

most solid hypothesis so far ^

lein doesn't do the exec and works

so then clojure would work, but not clj?

well they both exec at the end

oh wait, clojure also does exec

presumably a virus scanner would have some way to see and make this ok

Well, if that theory holds, then @dwhite44 could try https://github.com/borkdude/deps.clj and see if that works, since it doesn't exec

(just to confirm the theory more)

@dwhite44 you could try this binary: https://github.com/borkdude/deps.clj/releases/download/v0.0.15/deps.clj-0.0.15-macos-amd64.zip Instead of clojure you invoke deps. deps -Spath or whatever

i can give it a try. i think the sh file that works w/ no exec and fails with it is pretty good proof

But it could be that the virus scanner also marks this binary as a virus of course

is anybody familiar with how figwheel works? assume it also creates a new process and is likely why that’s not working? ultimately, I can get around the clojure cli not working by using leiningen but I can’t really do what i need to do if I can’t get around the figwheel problem

you could also try the uberjar if that is the case: https://github.com/borkdude/deps.clj/releases/download/v0.0.15/deps.clj-0.0.15-standalone.jar

java -jar <uberjar>.jar -Spath

let's take it one step at a time

right. just wanted to add the fighwheel as an additional hint. seems like that probably confirms the issue is the extra process created by exec

well maybe not confirms but more evidence

yes it does work using borkdude/deps

I mean, you can hack your clojure to remove the exec :)

yeah. i’ll give my project a try using that. i suspect though that figwheel will make it fail for the same reason

but at least knowing that difference is causing the problem gives me something to tell the IT people so they can look at the anti-virus software settings or something. I really appreciate everybody’s help on this!

@dwhite44 @alexmiller possibly this also works as a workaround:

alias exec=''

I would write a wrapper script called clojure2 / clj2 where I would apply this only locally and then invoke the normal tool (to not mess with other calls of exec in your system)

good idea!

that's a built-in - can you really subvert it like that?

I tested it and it seems to work

at least in zsh

seems terrible :)

that you can do that :)

it's a bit like clojure in this respect, I like it :P

hmm, forget it, it doesn't seem to work

This seems to work: clojure2:

#!/usr/bin/env bash

exec() {
 echo "exec replacement"
 eval $@
}

source clojure $@

(I don't give up)

hmm. using that script actually doesn’t work for me even though i do see the “exec replacement” being output

@dwhite44 to be 100% certain it's exec triggering your error, make a copy of clojure and remove "exec" from it

if the error persists, bisect the script

or maybe even easier: change echo "exec replacement" into echo "exec replacement" $@

perhaps that will tell you something, not sure

just copying the script and playing around with, inserting echo bla etc will probably help you most to discover what triggers it

copying the original script and removing the execs does work

ok, well, maybe keep doing that then :)

hmm. so interesting. even running the application with figwheel seems to be working using that. unless it’s just the brief temporary reprieve i sometimes get from making a change. guess i’ll know soon enough

this definitely seems to be at least a good solution for now. thanks everybody!