Senior Staff Applied AI Engineer - Context Retrieval

10+ years of software engineering experience, with significant time spent building production retrieval, search, or RAG systems at scale.

: lexical retrieval (BM25, Lucene/Elasticsearch/OpenSearch), dense retrieval (embeddings, ANN indexes — FAISS, ScaNN, HNSW), hybrid retrieval, and learning-to-rank.

Hands-on experience with

: RAG architectures, query rewriting, re-ranking with cross-encoders, long-context strategies, and grounding techniques that reduce hallucination.

Experience designing

agentic systems on top of retrieval

Own the end-to-end system: query understanding, content understanding and indexing, hybrid retrieval, ranking, and evaluation.

Make the architectural calls that will define how Databricks agents access context for years to come.

Retrieve across heterogeneous data — structured and unstructured.

Index and rank across structured assets (tables, columns, SQL queries, dashboards, code, notebooks, jobs) and unstructured content (docs, wikis, tickets, chat, images, video, audio).

Each modality has its own signals — design retrieval that exploits them rather than flattens them.

Connect to the SaaS surface area customers actually use.

Databricks agents are only as good as the context they can retrieve. Whether an agent is answering a question about last quarter's revenue, debugging a failing job, generating SQL against a 10,000-table lakehouse, or summarizing a Wiki page, its quality is bounded by what it can find — and how well it understands what it finds.