ALL NPPS Members

["Galgoczi, Gabor (PO)" <ggalgoczi1 AT bnl.gov>]

Dear Shuwei,

That is great, thanks! If you need more memory we can think about preemption, but pausing GPU jobs unfortunately doesn't free GPU RAM as far as I know, the VRAM stays allocated even when a process is suspended, so we can't just temporarily pause Ollama jobs to make room.

Best,

Gabor

---

From: Ye, Shuwei <yesw AT bnl.gov>
Sent: Friday, March 13, 2026 6:12 PM
To: Galgoczi, Gabor (PO) <ggalgoczi1 AT bnl.gov>; Torre Wenaus <wenaus AT gmail.com>
Cc: NPPS members <phys-npps-members-l AT lists.bnl.gov>
Subject: Re: Instructions for Offsite Access to Ollama Server on npps0

 

Dear Gabor,

I am responsible for managing the Ollama server on npps0. I can look into configuring the server to use only one GPU and limiting the VRAM to 24 GB.

Best regards,

--Shuwei

---

From: Galgoczi, Gabor (PO) <ggalgoczi1 AT bnl.gov>
Sent: Friday, March 13, 2026 4:18 PM
To: Torre Wenaus <wenaus AT gmail.com>; Ye, Shuwei <yesw AT bnl.gov>
Cc: NPPS members <phys-npps-members-l AT lists.bnl.gov>
Subject: Re: Instructions for Offsite Access to Ollama Server on npps0

 

Dear All,

Who is the owner of the Ollama process? Could you restrict it to GPU0 using CUDA_VISIBLE_DEVICES=0? Could also limit the utilized RAM by OLLAMA_MAX_VRAM setting.

When it is running it uses most of the RAM of both GPUs. We can not do eic-opticks work and our GitHub CI tests also fail due to insufficient free memory.

Thank you,

Gabor

---

From: phys-npps-members-l-request AT lists.bnl.gov <phys-npps-members-l-request AT lists.bnl.gov> on behalf of Ye, Shuwei <yesw AT bnl.gov>
Sent: Wednesday, March 11, 2026 11:11 AM
To: Torre Wenaus <wenaus AT gmail.com>
Cc: NPPS members <phys-npps-members-l AT lists.bnl.gov>
Subject: [[Phys-npps-members-l] ] Instructions for Offsite Access to Ollama Server on npps0

 

Dear Torre,

You can find the detailed instructions for offsite access to the Ollama server on our group machine npps0 in the following document:

https://docs.google.com/document/d/1eIixG_AIpc9-dgpMns5nTf2ocgQ4nHHPtd8JJSkaJt4/edit?tab=t.0#heading=h.j98hvgbbyt7w

For example, to use the model qwen3.5:35b via claude CLI on your laptop, follow these steps:

1. Establish an SSH tunnel to forward the local port 1080 to the remote Ollama server:
   Code

   ssh -f -N -L 1080:130.199.21.114:11434 ssh.bnl.gov
2. Set the required environment variables:
   Code

   export ANTHROPIC_AUTH_TOKEN=ollama export ANTHROPIC_API_KEY="" export ANTHROPIC_BASE_URL=http://localhost:1080
3. Launch Claude with the specified model:
   Code

   claude --model qwen3.5:35b

Please let me know if you encounter any issues or need further assistance.

Best regards,

--Shuwei