Software Engineer - Hosted Model Infrastructure

Palo Alto, CA

Dev /

Full-time /

Hybrid

A World-Changing Company

Palantir builds the world’s leading software for data-driven decisions and operations. By bringing the right data to the people who need it, our platforms empower our partners to develop lifesaving drugs, forecast supply chain disruptions, locate missing children, and more.

The Role

We are a software engineering team with expertise in enabling ML models in production. We deploy AI models to run in variety of environments: air-gapped government networks, forward-deployed defense environments, edge nodes, and enterprises with strict data sovereignty requirements. Our customers rely on us for frontier AI capabilities running on hardware they control, often with constrained GPU resources and limited direct access. Rising to that challenge and meeting those expectations is what Palantir's excels at.

We treat models like any other software: continuously tested, continually delivered, packaged for reproducible deployment, and built for long-term maintainability. You will own services end-to-end, and work across the full stack, from inference engines, GPU scheduling to deployment pipelines, observability, and integration with Palantir's platform. The goal is to deliver new models and capabilities quickly and continuously.

Join us if you want to solve problems at the intersection of infrastructure and machine learning that directly enable critical customers.

Technologies We Use

Different backend languages, including Java, Rust, Python and Go
Model serving engines for GPU-accelerated inference
Docker and Kubernetes for containerization and orchestration
Industry-standard build tooling, including Gradle and GitHub

Core Responsibilities

Building high-performance model serving infrastructure that integrates with security models, hardware constraints, and different inference engines
Designing intelligent request handling including authentication, rate limiting, concurrency control, and audit logging for multi-tenant model access
Building and maintaining packaging and deployment pipelines enabling fast, secure, and reliable model rollouts across on-premises and air-gapped environments
Developing observability for production AI systems to enable easy service monitoring and fast incident triage and resolution
Debugging complex issues and performance problems throughout the stack, including open source inference engines, container runtimes, and GPU drivers, in environments you cannot always access directly
Designing and running testing and benchmarking infrastructure that validates model deployments across varying GPU hardware before they reach production
Working with product teams and customers to understand requirements, debug production issues, and deliver the models and capabilities they need
Integrating hosted model infrastructure with Palantir's deployment, configuration, and identity systems

What We Value

Ownership mindset and bias toward quality. Our software runs in environments where direct access for debugging is limited or unavailable.
High empathy for customer needs and drive to deliver reliable, easy-to-use models
Ability to work effectively across multiple languages and layers of the stack, from backend services and ML tooling to container orchestration and deployment configuration
Strong debugging skills and motivation to trace problems from application code through containers, orchestration, and hardware
Curiosity about emerging AI capabilities and the ability to quickly evaluate and integrate new models and technologies as the landscape evolves
Active US Security clearance, or eligibility and willingness to obtain a US Security clearance is beneficial, but not necessary

What We Require

4+ years of professional software engineering experience building and operating production systems
Engineering background in Computer Science, Mathematics, Software Engineering, Physics, or similar field
Strong coding skills with demonstrated proficiency in programming languages, such as Java, C++, Python, Rust, or similar languages. Familiarity with the Python ML ecosystem is valuable.
Experience with containers, Kubernetes, and deploying backend services in production environments
Strong written and verbal communication skills and ability to iterate quickly with teammates, incorporating feedback and holding a high bar for quality

Salary

The salary range for this position is estimated to be $145,000 - $200,000/year. Total compensation for this position may also include Restricted Stock units, sign-on bonus and other potential future incentives. Further note that total compensation for this position will be determined by each individual's relevant qualifications, work experience, skills, and other factors. This estimate excludes the value of any potential sign-on bonus; the value of any benefits offered; and the potential future value of any long-term incentives.

Our benefits aim to promote health and wellbeing across all areas of Palantirians’ lives. We work to continuously improve our offerings and listen to our community as we design and update them. The list below details our available benefits and some of the perks that can be enjoyed as an employee of Palantir Technologies.

Benefits

• Employees (and their eligible dependents) can enroll in medical, dental, and vision insurance as well as voluntary life insurance

• Employees are automatically covered by Palantir’s basic life, AD&D and disability insurance

• Commuter benefits

• Relocation assistance

• Take what you need paid time off, not accrual based

• 2 weeks paid time off built into the end of each year (subject to team and business needs)

• 10 paid holidays throughout the calendar year

• Supportive leave of absence program including time off for military service and medical events

• Paid leave for new parents and subsidized back-up care for all parents

• Fertility and family building benefits including but not limited to adoption, surrogacy, and preservation

• Stipend to help with expenses that come with a new child

• Employees can enroll in Palantir’s 401k plan

Life at Palantir

We want every Palantirian to achieve their best outcomes, that’s why we celebrate individuals’ strengths, skills, and interests, from your first interview to your longterm growth, rather than rely on traditional career ladders. Paying attention to the needs of our community enables us to optimize our opportunities to grow and helps ensure many pathways to success at Palantir. Promoting health and well-being across all areas of Palantirians’ lives is just one of the ways we’re investing in our community. Learn more at Life at Palantir and note that our offerings may vary by region.

In keeping consistent with Palantir’s values and culture, we believe employees are “better together” and in-person work affords the opportunity for more creative outcomes. Therefore, we encourage employees to work from our offices to foster connectivity and innovation. Many teams do offer hybrid options (WFH a day or two a week), allowing our employees to strike the right trade-off for their personal productivity. Based on business need, there are a few roles that allow for “Remote” work on an exceptional basis. If you are applying for one of these roles, you must work from the state in which you are employed. If the posting is specified as Onsite, you are required to work from an office.

If you want to empower the world's most important institutions, you belong here. Palantir values excellence regardless of background. We are proud to be an Equal Opportunity Employer for all, including but not limited to Veterans and those with disabilities. Palantir is committed to making the application and hiring process accessible to everyone and will provide a reasonable accommodation for those living with a disability. If you need an accommodation for the application or hiring process, please reach out and let us know how we can help.

Please note that you will never be asked to submit a payment or share financial information to participate in our interview process. If you suspect that you've been contacted by a scammer, we recommend you cease all communication with the individual and consider reporting them to the relevant authorities, such as the US FBI Internet Crime Complaint Center (IC3).

If you would like to understand more about how your personal data will be processed by Palantir, please see our Privacy Policy.

Apply

Different backend languages, including Java, Rust, Python and Go
Model serving engines for GPU-accelerated inference
Docker and Kubernetes for containerization and orchestration
Industry-standard build tooling, including Gradle and GitHub

Building high-performance model serving infrastructure that integrates with security models, hardware constraints, and different inference engines
Designing intelligent request handling including authentication, rate limiting, concurrency control, and audit logging for multi-tenant model access
Building and maintaining packaging and deployment pipelines enabling fast, secure, and reliable model rollouts across on-premises and air-gapped environments
Developing observability for production AI systems to enable easy service monitoring and fast incident triage and resolution
Debugging complex issues and performance problems throughout the stack, including open source inference engines, container runtimes, and GPU drivers, in environments you cannot always access directly
Designing and running testing and benchmarking infrastructure that validates model deployments across varying GPU hardware before they reach production
Working with product teams and customers to understand requirements, debug production issues, and deliver the models and capabilities they need
Integrating hosted model infrastructure with Palantir's deployment, configuration, and identity systems

Ownership mindset and bias toward quality. Our software runs in environments where direct access for debugging is limited or unavailable.
High empathy for customer needs and drive to deliver reliable, easy-to-use models
Ability to work effectively across multiple languages and layers of the stack, from backend services and ML tooling to container orchestration and deployment configuration
Strong debugging skills and motivation to trace problems from application code through containers, orchestration, and hardware
Curiosity about emerging AI capabilities and the ability to quickly evaluate and integrate new models and technologies as the landscape evolves
Active US Security clearance, or eligibility and willingness to obtain a US Security clearance is beneficial, but not necessary

4+ years of professional software engineering experience building and operating production systems
Engineering background in Computer Science, Mathematics, Software Engineering, Physics, or similar field
Strong coding skills with demonstrated proficiency in programming languages, such as Java, C++, Python, Rust, or similar languages. Familiarity with the Python ML ecosystem is valuable.
Experience with containers, Kubernetes, and deploying backend services in production environments
Strong written and verbal communication skills and ability to iterate quickly with teammates, incorporating feedback and holding a high bar for quality

Software Engineer - Hosted Model Infrastructure

Job Description