Crusoe's mission is to accelerate the abundance of energy and intelligence. We’re crafting the engine that powers a world where people can create ambitiously with AI — without sacrificing scale, speed, or sustainability. We are looking for a highly skilled engineer with deep expertise in building and operating observability platforms at scale. You will design, develop, and run Crusoe’s next-generation observability stack, enabling engineers to understand the internal state of distributed systems through metrics, logs, and traces. Your work will ensure reliability, performance, and actionable insights across Crusoe’s global infrastructure and cloud platform.
Requirements
- Deep expertise with metrics systems (Prometheus, Thanos, Mimir, Cortex), logging pipelines (Fluent Bit, Vector, Loki, ELK/Opensearch), and tracing platforms (Jaeger, Tempo, OpenTelemetry)
- Strong programming skills in Go or Python for automation, operators, and custom integrations
- Experience running observability platforms on Kubernetes and operating them at scale across multi-datacenter environments
- Proven ability to design, optimize, and scale telemetry pipelines handling high cardinality and high throughput data
- Solid understanding of distributed systems, performance engineering, and debugging complex workloads
- Familiarity with service meshes, networking, and workload instrumentation (Envoy, Istio, OpenTelemetry SDKs)
- Contributions to open source observability projects (Prometheus, OpenTelemetry, Grafana, Loki, etc.)
Responsibilities
- Designing and operating scalable observability systems (metrics, logging, tracing) across multi-datacenter Kubernetes environments
- Architecting end-to-end telemetry pipelines, including ingestion, storage, querying, and visualization
- Extending monitoring and alerting with Prometheus, Alertmanager, Thanos/Cortex, Grafana, and OpenTelemetry
- Building scalable log collection and processing pipelines with Fluent Bit, Vector, Loki, or ELK/Opensearch stacks
- Implementing distributed tracing platforms (Tempo, Jaeger, OpenTelemetry) and integrating with service meshes, load balancers, and APIs
- Defining and driving adoption of SLOs, SLIs, and error budgets across services and teams
- Automating provisioning and scaling of observability infrastructure with Kubernetes, Terraform, and custom tooling (Go, Python)
Other
- 7+ years of experience in infrastructure or platform engineering, with a focus on observability and monitoring systems
- Strong collaboration skills and the ability to influence engineering teams to adopt observability best practices
- Experience supporting AI/ML or GPU-heavy environments with high observability demands
- Knowledge of event-driven or streaming systems (Kafka, NATS, Pulsar) used in telemetry pipelines
- Experience implementing cost optimization strategies for large-scale observability platforms
