Back to jobs
Senior Backend Engineer
LiteLLM is the world's most popular AI Gateway, trusted by top companies like Adobe, Netflix, and NASA. Our platform empowers developers by providing secure, reliable access to LLMs and adjacent services, and we're looking for a Backend Engineer (New Grad) to help us build rock-solid guardrails and observability tooling at scale.
About the role
Come join our team as our first Security Engineer.
Responsibilities
Conduct deep-dive security reviews of the LiteLLM proxy codebase, identifying potential supply chain risks and dependency vulnerabilities.
Build and maintain automated security scanning for the Docker image, PyPI package, and CI/CD workflows (e.g., dependency scanning, secrets detection).
Design and enforce secure-by-default configurations for cloud deployments and self-hosted environments (e.g., API authentication, IAM least privilege, key rotation).
Implement and monitor intrusion detection or anomaly alerts specific to model and API usage patterns.
Lead incident response and post-mortem efforts — including vulnerability assessment, remediation roadmap, and communication with stakeholders.
Establish a formal CVE triage and disclosure protocol in coordination with the engineering team.
Perform internal red teaming or adversarial testing to simulate real-world attacks and strengthen defenses.
Partner with engineering to harden release pipelines (e.g., signed builds, provenance checks, reproducible builds).
Create secure coding guidelines and run regular training for developers on supply chain and dependency management.
Maintain threat models and continuously update them as LiteLLM’s product and architecture evolve.
What We’re Looking For
Bachelor’s or Master’s in Computer Science or related field
Software Supply Chain Security: Deep knowledge of SBOMs (Software Bill of Materials), dependency analysis, and tools like Snyk, Grype, or Trivy. You'll need to understand "poisoned" packages and dependency confusion attacks.
Container & CI/CD Security: Experience hardening Docker images and securing pipelines (GitHub Actions, GitLab CI) using signed builds (Sigstore/Cosign) and provenance checks.
Code Review & Static/Dynamic Analysis (SAST/DAST): Ability to manually audit Python code (since LiteLLM is Python-based) and configure automated scanners (e.g., Bandit, Semgrep).
Network & API Security: Understanding of OAuth2, JWT, and mTLS. Since it’s a proxy, you need to know how to prevent injection attacks and manage high-throughput API authentication.
Adversarial Testing: Knowledge of Prompt Injection, data exfiltration via LLMs, and the OWASP Top 10 for LLMs.
Anomaly Detection: Ability to build monitors for weird API patterns (e.g., token exhaustion attacks or unusual model routing).
Incident Response (IR): Experience leading post-mortems and managing the lifecycle of a breach, from discovery to stakeholder communication.
Vulnerability Management: Familiarity with CVSS scoring and the process of filing/triaging CVEs (Common Vulnerabilities and Exposures).
Threat Modeling: Using frameworks like STRIDE to map out risks as the LiteLLM architecture changes.
Why Join LiteLLM?
High-impact, mission-critical work on the core of compliance and reliability
Contribute directly to features used by enterprise customers at global scale
Fast-paced growth environment with room for technical ownership
Competitive salary, health, dental, and vision benefits
About LiteLLM
LiteLLM (https://github.com/BerriAI/litellm) is a Python SDK and Proxy Server enabling seamless calls to 100+ LLM APIs in the OpenAI format, trusted by industry leaders worldwide.
Ready to shape the future of secure, observable AI infrastructure? Apply now!