r/HPC • • Apr 08 '26

Is hpc related to AI datacenters?

4 Upvotes

I am really worried about the ai pandemic that is going on in tech jobs. It's gonna worsen.

Can HPC jobs be replaced easily? Since HPC involves supercomputers and gpu, i am assuming the demand for HPC will only increase. (Pardon me if I'm wrong, I don't know too much about this area). The big tech datacenters, do HPC people work on it? If it's true then you have the most job security.

I am considering to do a master to escape my country, and i have just started reading about HPC.


r/HPC • • Jan 04 '22

Guardant Health is Hiring a Senior HPC Engineer

1 Upvotes

Job link: https://smrtr.io/7CncQ Job Salary Range: $153k to $210k Annual Bonus: Yes Stock: RSU grants

A few things to note:

  • there is an on-call rotation with a strict response time and triage.

  • despite the position being mostly remote, while on rotation, you do have to be onsite the week of rotation.

  • Rotation will be every 5 weeks. GH covers expenses of the travel.

  • There is a component of potential travel aside from the main site. we have systems in San Diego, Barcelona and Tokyo beyond the main sites in the SF Bay Area. More will be forthcoming.

  • You do need to be a US resident.

  • You MUST be vaccinated to work at GH.


Senior HPC Infrastructure Engineer 505 Penobscot Dr, Redwood City, CA 94063, USA Employees can work remotely Full-time

Company Description

Guardant Health is a pioneer in non-invasive cancer diagnostics and the first company to commercialize a comprehensive genomic liquid biopsy. Our proprietary digital sequencing technology is transforming cancer treatment by providing an accurate and precise picture of the individual genomic alterations that cause tumors to grow, change, and develop resistance to treatment. We have combined decades of scientific research, advances in laboratory technology, and our breakthrough innovation in liquid biopsy to create new tests that have already handled tens of thousands of samples. We believe our tests can accelerate new drug development and improve the lives of all patients fighting cancer. Our current products are just the beginning of what we hope to accomplish, and new uses of our platform are emerging.

We succeed best by coordinating our creative talents and energies, working as a team to achieve results far beyond what any single individual could accomplish. We seek very talented people who want to be part of our fantastic team.

Job Description:

Guardant’s HPC team builds and operates the computational technology backbone of the company.

This includes scalable data storage that holds petabytes of genomics data, high performance compute clusters running a custom bioinformatics pipeline in production and R&D environments, and the software infrastructure that hosts an ecosystem of services for internal data processing and external data integration. To facilitate Guardant Health’s fast growth in the next few years, the HPC team is looking for a strong technical engineer who can help maintain and help grow the HPC infrastructure during its aggressive expansion, while working with corporate IT, SQA and DevOps/SRE teams.

While preferred to have someone local to the San Francisco Bay and on premises in Redwood City and Palo Alto, this role can be mostly worked remotely. While on rotation, during maintenances and during cluster deployments, being present at the location of the work is required.

** In this role, you will primarily:**

Help manage the HPC interconnect

Help integrate the HPC systems with the bandwidth on-demand system

Work with the networking infrastructure team to manage and optimize the connectivity to and from the HPC systems and locales

Help manage multiple HPC clusters and cluster file systems. 

Help research, develop and implement the next generation HPC solution

Troubleshoot the production system stack down to source code level e.g. shell scripts, python and others.

Maintain, monitor, and support the infrastructure environment and/or facilities.

Use and maintain enhanced production monitoring and additional capability.

Support improvements for increased system reliability and performance.

Support multiple systems or applications of medium to high complex (complexity defined by size, technology used, and system feeds and interfaces) with multiple concurrent users, ensuring control, integrity, and accessibility.

Support systems at remote locations, including internationally

Work with offsite consultants to maintain the infrastructure

Work with vendors to troubleshoot, upgrade and repair systems as needed

Participate in a 24/7 on-call rotation

About You

You enjoy an agile, very fast paced and highly technical environment. You are a self-driven accomplished technologist who strives to be ever improving your skills, value to the company and improve the computational infrastructure. You are dedicated to engineering excellence yet pragmatic and flexible. You have the ability to maintain the day-to-day support SLA while running various key projects that move the business forward.

4+ years of networking experience with certification of CCNA or better

4+ years of Linux/Unix administration, knowledge of Unix network protocols, TCP/IP network fundamentals, core infrastructure technologies and virtualization 

2+ years of large-scale data storage and compute clusters (HPC) infrastructure  

2+ years working in and with on-premise and cloud-based (AWS, Google, IBM and Azure) data-centers 

2+ years of building software release and ops processes and automation toolset 

2+ years providing documentation of system administration

Qualifications

Cisco Certified Network Professional certification

Experience with Arista and compatible networking, up to and including 400 gb/s links

Experience with Mellanox infiniband fabric

Experience administering IBM’s General Parallel File System

Experience administering Grid Engine scheduler

Experience with using Bright Cluster Manager

Experience with cloud bursting technologies

Experience with wide area file systems

Experience with docker and container technologies

Experience with Kubernetes

Operating infrastructure compliant with HIPAA and SOX standards

Education

B.S. in Computer Science or related field


r/HPC • • Feb 05 '20

I've created a career-centered website specific to the HPC / Big Data Industries

3 Upvotes

Hey everyone this is Alex.

About a month ago I asked you here if you would be interested in a site to help people like you in the HPC | Big Data field with how to get a job, a raise, a promotion, etc. How to craft your resume to get interviews, then pass them and get the job... (https://www.reddit.com/r/HPC/comments/ejudv6/how_to_get_a_jobraisepromotion_working_in_hpc_big/)

I'm happy today to announce HPC Careers | hpccareers.com

I’ve worked in the HPC / Big Data industry for over 10 years, and I LOVE IT!!!

I love working with cutting-edge technology at scale – big, fast, powerful tech…mixed with the most cutting-edge, world-impacting, and badass science / applications.

But what I love even more than that is helping people like you get a job in the field.

I'd be happy to answer any questions you have. Just PM me.


r/HPC • • Aug 28 '26

I built a cross-platform desktop client for SSH, SFTP and Slurm workflows — feedback welcome

2 Upvotes

Hi r/HPC,

I have been developing HPC Client GUI, an independent cross-platform desktop application for researchers working with SSH-accessible Slurm clusters.

It combines:

- SSH connection profiles

- Remote file browsing and SFTP transfers

- Slurm job submission, monitoring and cancellation

- Job output inspection

- Terminal and CLI workflows

- Optional X11 forwarding

- Declarative cluster profiles and job-template plugins

- ANSYS journal linting

It runs on Windows, macOS and Linux. Nothing needs to be installed on the cluster; it works through standard SSH, SFTP and Slurm commands.

The original use case was Türkiye’s TRUBA infrastructure, but the application is designed around generic SSH + Slurm behavior. It is an independent project and is not an official TRUBA tool.

Repository and downloads:

https://github.com/mskomek/hpc-client-gui

I would especially appreciate feedback on:

  1. Compatibility with different institutional Slurm configurations

  2. MFA, jump-host and SSH authentication workflows

  3. File-transfer and scheduler features researchers commonly struggle with

  4. Whether cluster administrators would be comfortable recommending a local client like this

  5. Documentation or packaging problems that could prevent adoption

The macOS packages are currently unsigned and this is disclosed in the release notes. The project is source-available under the PolyForm Noncommercial License 1.0.0.

If anyone is willing to test it against another cluster, the repository includes a read-only compatibility validation guide. Please do not share cluster addresses, credentials or private configuration.

Thank you.


r/HPC • • Apr 19 '25

I need to hire an expert to implement Lustree BeeGFS. Can anyone recommend freelancers to me?

0 Upvotes

r/HPC • • Jul 16 '26

Top 10 Data Center and AI Infrastructure Security Risks

0 Upvotes

We spent the past few months researching security risks across multi-tenant data centers and AI infrastructure.
The main concern we found is shared infrastructure: multiple customers running on the same data center infrastructure, GPU clusters, storage, and high-speed networks. Many neoclouds and AI data centers have also scaled faster than their security teams and practices, especially compared with more established cloud providers.
The research covers GPU clusters, RDMA and high-speed interconnects, tenant isolation, BMCs, firmware, shared storage, orchestration, and supply-chain risks.
We organized the findings into a practical framework called FORGE.
Would really appreciate feedback. Link in the comment


r/HPC • • Aug 16 '25

Tutorials/guide for HPC

2 Upvotes

hello guys , i am new to AI , i want to extends my knowledge to HPC. i am looking for a beginner guide from zero . i welcome all guidance available. thank you.


r/HPC • • Mar 19 '26

Will HPC benefit or be hurt by AI hype?

2 Upvotes

I discuss the oportunities and challenges for HPC in this AI era in this article.


r/HPC • • Apr 18 '25

Postgrad recommoendations

0 Upvotes

Not sure if this is the right subreddit for this but I'm currently a 3rd year CSE student from India with a decent GPA, I'm looking to get into graphics/GPU Software development/ ML Compilers /accelerators. I'm not sure which one yet but I read that the skillset for all these is very similar so I'm looking for a masters programme in which I can figure out what I want to do and continue my career in. I'm looking for programmer in Europe and US, any help would be appreciated. Thank you

EDIT: for starters I thought MSc in HPC at University of Edinburgh would be a good start where after graduating I could work in any of the above mentioned industries


r/HPC • • Sep 28 '22

Job Opening

1 Upvotes

We are looking for an HPC engineer with security clearance TS/SCI in the DC beltway area. If you have those qualifications, or you need further information, please send your resume to [email protected].


r/HPC • • Nov 11 '19

HPC MPI code

1 Upvotes

Hello, Im looking for some code to serve as a example of a parallelized program that would be run on a HPC in a low level language. Could anyone share some code? Can be any problem as long as it would be run on a HPC and is parallelized. This is for school project.


r/HPC • • Aug 04 '24

HPC etiquette - What justifies using the login node of a cluster

5 Upvotes

Hey, I have a job that I urgently need to compute. And I've been waiting 2 days to get a GPU and got none. There's a dude who's litterally using the whole cluster, while I need 1 gpu for 2h.


r/HPC • • Jun 15 '20

Need help with the visualization of HPC!

0 Upvotes

hi,

i need help visualizing HPC - Or better - results i can show that were made possible with HPC. I cannot find anything useful and the field is full of technology. I was hoping that maybe someone here has seen something lately that is easy to understand for laymen.

Many greetings and stay healthy.


r/HPC • • Sep 07 '20

Would you try a programming language that automatically and optimally threaded your code, chooses the optimal data structures for you and optimized your programs, and used high-speed yet easy memory management?

0 Upvotes

I discovered a new and much faster malloc implementation and initial testing is showing that programs that heavily rely on mallocing can run as much as 15x faster. Problem is is kind of needs to be baked into the language, or else requires a good deal of work to implement yourself. So I'd like to create a language around it that also guarantees memory safety.

Also, code can be written in a single-threaded manner with users simply indicating that order is important for some variables, not for others. A couple hints like that and we can automatically and just about optimally thread your code for you to maximize performance.

Finally, I want to be able to run the program in an optimize mode and track various things, such as how many times are users inserting vs reading vs resizing, etc data structures and then optimize the data structure being used for collections so coders don't have to think about it and guess.

Have started building a prototype, would love help building it if anyone would be interested. Initially testing is showing that a 2x to 20x improve over a 'naive' C++ implementation is easy to achieve and much faster to write.

Anything else you can think of? Would you be willing to try it out if I can throw up a demo and prove these performance gains across every program on benchmark games?


r/HPC • • Dec 03 '19

HPC focus of keynote at AWS re:Invent

Thumbnail
youtube.com
0 Upvotes

r/HPC • • Apr 11 '15

Re-architecting HPC: Intel's HPC Scalable System Framework

Thumbnail
intel.com
0 Upvotes

r/HPC • • Feb 08 '25

Building a tesla t4 workstation

0 Upvotes

I want to build a workstation for running tensorflow using a tesla t4 gpu (I am currently using google colab but run times were increased by 10x a week and a half ago probably due to what I am guessing is a driver update)

How do I build it and set up the software? Any pointing in the right direction will be appreciated


r/HPC • • May 18 '25

HELP! Trying to Land My Dream HPC Internship—Is My Resume Good Enough?

0 Upvotes

UPDATE: I made a few suggested changes on the resume. Feel free to provide your views and suggestions!

I’m currently a master's student in Information Technology and Management, and I'm chasing an internship as a Jr HPC (High-Performance Computing) Engineer at my university. I'm super passionate about landing this opportunity, but I'm worried my resume might not fully reflect how suitable I am for this role.

My background mainly involves working with ASP. NET, C#, Angular, MySQL.

Here's the deal: The internship focuses heavily on Linux, Git, automation with Ansible, HPC cluster deployment (OpenHPC, SLURM), and scientific software packaging/containerization. I've tried to subtly align my past experience with these aspects, but I'm not sure if it hits the mark perfectly.

Could you take a quick look at my resume (attached) and let me know if there’s anything glaringly off or any improvements I could make? Any advice on how to better align my experience with HPC or general tips to enhance my chances of landing this role would be incredibly appreciated!


r/HPC • • Apr 10 '15

HPC is dying, and MPI is killing it

Thumbnail news.ycombinator.com
5 Upvotes

r/HPC • • Oct 01 '24

How do I get a Job at HPC?

5 Upvotes

I was wondering how I can get a job. I have 10+ years of C++ experience.

The job sites seem automated or just delete my application.

I’m interested in applying my AI skills to simulation.


r/HPC • • 1d ago

Looking for academic/free HPC access for undergraduate research from Pakistan.

1 Upvotes

We are trying to develop a surrogate model for SITVC (Secondary Injection Thrust Vector Control). We currently have around 48 initial sampling points using a maximin Latin hypercube design, with all six input factors varying simultaneously. All the data is supposed to come from CFD simulations.

The two papers we are primarily using as references are Zmijanovich et al. and Abdelwahab et al. Zmijanovich used around 1.9 million cells, with a first cell height near the nozzle wall of approximately (10^{-6}) m, while Abdelwahab used around 5 million cells for their medium mesh.

Now, the problem is computational resources. We are a group of three, and even our best available system has only around 6 CPU cores and 16 GB of RAM. As a result, a single simulation can take several days to complete.

We haven't managed to complete even a single 3D simulation so far because of these limitations, and have only been able to work with 2D steady-state simulations. The problem is that the actual flow physics is inherently unsteady, so we eventually need to move towards 3D transient simulations.

Could you guys please suggest some affordable, preferably free, options for accessing decent computational resources? Based on the mesh sizes and simulations we are aiming for, we have been considering systems with 32–64 CPU cores and sufficient RAM, but we aren't sure whether this is actually necessary or what would be a realistic configuration for our work.

Also, we're based in Pakistan, not the US, which makes things a bit more complicated when it comes to accessing university HPC facilities, research grants, or cloud computing credits.

Any suggestions, especially from people who have dealt with similar computational constraints during their research, would be greatly appreciated!


r/HPC • • Jun 29 '25

HPC System design

1 Upvotes

I am looking to study about HPC System design . AAre there any good resources for that.


r/HPC • • May 22 '25

NFS to run software on nodes?

1 Upvotes

Does anyone know if I want to run software on a computer node if I have my software placed in an nfs directory if this is the right way to go? My gut tells me I should install software directly on each node to prevent communication slowdown, but I honestly do not know enough about networking to know if this is true.


r/HPC • • Nov 19 '13

My blog: Today's joint announcement at SC13 by IBM and Nvidia is a game changer and underreported in the media -- "End of the server in HPC"

Thumbnail
datascienceassn.org
0 Upvotes