OpenSearch Full-Text Search: Index Design, Aggregations, Relevance Tuning, and HA

OpenSearch is the leading open-source distributed search and analytics engine for full-text search, log analytics, and observability. This guide covers index design, mapping strategies, aggregation queries, relevance tuning, ingest pipelines, and deploying a highly available cluster for production workloads.

OpenSearch Architecture

An OpenSearch cluster consists of nodes (JVM processes), primary shards (the main data shards, fixed at index creation), and replica shards (copies for redundancy and read throughput). Key node roles are: master (manages cluster state; run 3 dedicated masters for HA), data (stores and indexes data), ingest (pre-processes documents), and coordinating (routes requests without storing data).

Index Design and Mapping

Explicit mappings prevent mapping explosions and improve indexing throughput.

PUT /products
{
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1,
    "refresh_interval": "5s",
    "analysis": {
      "analyzer": {
        "product_analyzer": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": ["lowercase", "asciifolding"]
        }
      }
    }
  },
  "mappings": {
    "dynamic": "strict",
    "properties": {
      "product_id":  {"type": "keyword"},
      "name":        {"type": "text", "analyzer": "product_analyzer",
                      "fields": {"keyword": {"type": "keyword"}}},
      "description": {"type": "text", "analyzer": "product_analyzer"},
      "category":    {"type": "keyword"},
      "brand":       {"type": "keyword"},
      "price":       {"type": "double"},
      "rating":      {"type": "float"},
      "in_stock":    {"type": "boolean"},
      "tags":        {"type": "keyword"},
      "created_at":  {"type": "date"}
    }
  }
}

Field type selection: keyword for exact-match filtering, sorting, and aggregations; text for full-text search; numeric types for range queries. Dual-field mapping (text + keyword sub-field) enables both full-text search and aggregations on the same field.

Bool Query

GET /products/_search
{
  "query": {
    "bool": {
      "must":   [{"multi_match": {"query": "headphones", "fields": ["name^3", "description"]}}],
      "filter": [
        {"term":  {"category": "electronics"}},
        {"term":  {"in_stock": true}},
        {"range": {"price":  {"gte": 20, "lte": 300}}},
        {"range": {"rating": {"gte": 4.0}}}
      ],
      "should": [{"term": {"brand": "Sony"}}, {"term": {"tags": "noise-cancelling"}}]
    }
  }
}

Conditions in filter context do not affect the relevance score and are cached, making them significantly faster than must for non-scoring conditions.

Aggregations

Aggregations enable powerful analytics without a separate database.

GET /products/_search
{
  "size": 0,
  "aggs": {
    "by_category": {
      "terms": {"field": "category", "size": 20},
      "aggs": {
        "avg_price":  {"avg":   {"field": "price"}},
        "avg_rating": {"avg":   {"field": "rating"}}
      }
    },
    "price_histogram": {
      "histogram": {"field": "price", "interval": 50}
    },
    "top_brands": {
      "terms": {"field": "brand", "size": 10, "order": {"avg_rating": "desc"}},
      "aggs": {"avg_rating": {"avg": {"field": "rating"}}}
    }
  }
}

Date histogram for time-series analytics:

GET /orders/_search
{
  "size": 0,
  "aggs": {
    "orders_per_day": {
      "date_histogram": {"field": "created_at", "calendar_interval": "day"},
      "aggs": {
        "total_revenue":      {"sum":        {"field": "amount"}},
        "unique_customers":   {"cardinality": {"field": "customer_id"}}
      }
    }
  }
}

Relevance Tuning

Function score boosts by recency and popularity:

GET /products/_search
{
  "query": {
    "function_score": {
      "query": {"match": {"name": "headphones"}},
      "functions": [
        {"gauss": {"created_at": {"origin": "now", "scale": "30d", "decay": 0.5}}, "weight": 1.5},
        {"field_value_factor": {"field": "rating", "factor": 1.2, "modifier": "sqrt", "missing": 3.0}}
      ],
      "boost_mode": "multiply",
      "score_mode": "sum"
    }
  }
}

Pinned queries ensure sponsored or featured products always appear first:

{"query": {"pinned": {"ids": ["featured-001"], "organic": {"match": {"name": "headphones"}}}}}

Ingest Pipelines

Ingest pipelines transform documents before indexing without external ETL:

PUT /_ingest/pipeline/enrich-products
{
  "processors": [
    {"lowercase": {"field": "brand"}},
    {"trim":      {"field": "name"}},
    {"set":       {"field": "indexed_at", "value": "{{_ingest.timestamp}}"}},
    {"remove":    {"field": ["raw_data"], "ignore_missing": true}}
  ]
}

High Availability Deployment

Shard allocation awareness across availability zones:

PUT /products/_settings
{
  "index.routing.allocation.awareness.attributes": "zone",
  "index.routing.allocation.awareness.force.zone.values": "us-east-1a,us-east-1b,us-east-1c"
}

Node config (opensearch.yml) for dedicated master nodes:

node.attr.zone: us-east-1a
cluster.routing.allocation.awareness.attributes: zone
node.master: true
node.data: false
indices.breaker.total.limit: 70%

Index lifecycle management (ISM) for log indices:

PUT /_plugins/_ism/policies/log-policy
{"policy": {"default_state": "hot", "states": [
  {"name": "hot",    "actions": [{"rollover": {"min_size": "50gb"}}],
   "transitions": [{"state_name": "warm", "conditions": {"min_index_age": "3d"}}]},
  {"name": "warm",   "actions": [{"replica_count": {"number_of_replicas": 0}},
                                  {"force_merge": {"max_num_segments": 1}}],
   "transitions": [{"state_name": "delete", "conditions": {"min_index_age": "30d"}}]},
  {"name": "delete", "actions": [{"delete": {}}], "transitions": []}
]}}

Bulk Indexing with Python

Always use the bulk API for high-throughput writes:

from opensearchpy import OpenSearch, helpers

client = OpenSearch(hosts=[{"host": "opensearch", "port": 9200}])

def generate_actions(records):
    for r in records:
        yield {"_index": "products", "_id": r["product_id"], "_source": r}

helpers.bulk(client, generate_actions(records), chunk_size=500, request_timeout=30)

Search Performance Tips

• Use filter context (not must) for non-scoring conditions — results are cached
• Avoid deep pagination with from+size; use search_after with a sort key
• Profile slow queries with "profile": true in the request body
• Disable replicas during bulk loads, then re-enable
• Run OPTIMIZE (force merge to 1 segment) on read-only historical indices

Conclusion

OpenSearch delivers powerful full-text search and analytics when deployed thoughtfully. Explicit mappings prevent schema chaos. Bool queries with filter clauses leverage the query cache. Aggregations power faceted navigation without a separate analytics database. Function score and pinned queries give teams control over relevance. HA configuration with shard awareness and ISM policies ensures resilient, cost-efficient clusters at scale.