OpenSearch 全文搜索：索引设计、聚合、相关性调优与高可用

OpenSearch 是领先的开源分布式搜索与分析引擎，适用于全文搜索、日志分析和可观测性。本指南涵盖索引设计、映射策略、聚合查询、相关性调优、摄取管道以及为生产工作负载部署高可用集群。

OpenSearch 架构

OpenSearch 集群由节点（JVM 进程）、主分片（主要数据分片，创建索引时固定）和副本分片（用于冗余和读取吞吐量的副本）组成。关键节点角色包括：master（管理集群状态；为高可用运行 3 个专用主节点）、data（存储和索引数据）、ingest（预处理文档）和 coordinating（路由请求但不存储数据）。

索引设计与映射

显式映射可防止映射爆炸并提高索引吞吐量。

PUT /products
{
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1,
    "refresh_interval": "5s",
    "analysis": {
      "analyzer": {
        "product_analyzer": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": ["lowercase", "asciifolding"]
        }
      }
    }
  },
  "mappings": {
    "dynamic": "strict",
    "properties": {
      "product_id":  {"type": "keyword"},
      "name":        {"type": "text", "analyzer": "product_analyzer",
                      "fields": {"keyword": {"type": "keyword"}}},
      "description": {"type": "text", "analyzer": "product_analyzer"},
      "category":    {"type": "keyword"},
      "brand":       {"type": "keyword"},
      "price":       {"type": "double"},
      "rating":      {"type": "float"},
      "in_stock":    {"type": "boolean"},
      "tags":        {"type": "keyword"},
      "created_at":  {"type": "date"}
    }
  }
}

字段类型选择：keyword 用于精确匹配过滤、排序和聚合；text 用于全文搜索；数值类型用于范围查询。双字段映射（text + keyword 子字段）可在同一字段上同时实现全文搜索和聚合。

Bool 查询

GET /products/_search
{
  "query": {
    "bool": {
      "must":   [{"multi_match": {"query": "headphones", "fields": ["name^3", "description"]}}],
      "filter": [
        {"term":  {"category": "electronics"}},
        {"term":  {"in_stock": true}},
        {"range": {"price":  {"gte": 20, "lte": 300}}},
        {"range": {"rating": {"gte": 4.0}}}
      ],
      "should": [{"term": {"brand": "Sony"}}, {"term": {"tags": "noise-cancelling"}}]
    }
  }
}

filter 上下文中的条件不影响相关性评分且会被缓存，因此对于非评分条件，其速度显著快于 must。

聚合

聚合无需单独的数据库即可实现强大的分析功能。

GET /products/_search
{
  "size": 0,
  "aggs": {
    "by_category": {
      "terms": {"field": "category", "size": 20},
      "aggs": {
        "avg_price":  {"avg":   {"field": "price"}},
        "avg_rating": {"avg":   {"field": "rating"}}
      }
    },
    "price_histogram": {
      "histogram": {"field": "price", "interval": 50}
    },
    "top_brands": {
      "terms": {"field": "brand", "size": 10, "order": {"avg_rating": "desc"}},
      "aggs": {"avg_rating": {"avg": {"field": "rating"}}}
    }
  }
}

用于时间序列分析的日期直方图：

GET /orders/_search
{
  "size": 0,
  "aggs": {
    "orders_per_day": {
      "date_histogram": {"field": "created_at", "calendar_interval": "day"},
      "aggs": {
        "total_revenue":      {"sum":        {"field": "amount"}},
        "unique_customers":   {"cardinality": {"field": "customer_id"}}
      }
    }
  }
}

相关性调优

函数评分 根据时效性和流行度提升权重：

GET /products/_search
{
  "query": {
    "function_score": {
      "query": {"match": {"name": "headphones"}},
      "functions": [
        {"gauss": {"created_at": {"origin": "now", "scale": "30d", "decay": 0.5}}, "weight": 1.5},
        {"field_value_factor": {"field": "rating", "factor": 1.2, "modifier": "sqrt", "missing": 3.0}}
      ],
      "boost_mode": "multiply",
      "score_mode": "sum"
    }
  }
}

固定查询 确保赞助或特色产品始终排在首位：

{"query": {"pinned": {"ids": ["featured-001"], "organic": {"match": {"name": "headphones"}}}}}

摄取管道

摄取管道在索引前转换文档，无需外部 ETL：

PUT /_ingest/pipeline/enrich-products
{
  "processors": [
    {"lowercase": {"field": "brand"}},
    {"trim":      {"field": "name"}},
    {"set":       {"field": "indexed_at", "value": "{{_ingest.timestamp}}"}},
    {"remove":    {"field": ["raw_data"], "ignore_missing": true}}
  ]
}

高可用部署

跨可用区的分片分配感知：

PUT /products/_settings
{
  "index.routing.allocation.awareness.attributes": "zone",
  "index.routing.allocation.awareness.force.zone.values": "us-east-1a,us-east-1b,us-east-1c"
}

专用主节点的节点配置（opensearch.yml）：

node.attr.zone: us-east-1a
cluster.routing.allocation.awareness.attributes: zone
node.master: true
node.data: false
indices.breaker.total.limit: 70%

用于日志索引的索引生命周期管理（ISM）：

PUT /_plugins/_ism/policies/log-policy
{"policy": {"default_state": "hot", "states": [
  {"name": "hot",    "actions": [{"rollover": {"min_size": "50gb"}}],
   "transitions": [{"state_name": "warm", "conditions": {"min_index_age": "3d"}}]},
  {"name": "warm",   "actions": [{"replica_count": {"number_of_replicas": 0}},
                                  {"force_merge": {"max_num_segments": 1}}],
   "transitions": [{"state_name": "delete", "conditions": {"min_index_age": "30d"}}]},
  {"name": "delete", "actions": [{"delete": {}}], "transitions": []}
]}}

使用 Python 进行批量索引

对于高吞吐量写入，始终使用 bulk API：

from opensearchpy import OpenSearch, helpers

client = OpenSearch(hosts=[{"host": "opensearch", "port": 9200}])

def generate_actions(records):
    for r in records:
        yield {"_index": "products", "_id": r["product_id"], "_source": r}

helpers.bulk(client, generate_actions(records), chunk_size=500, request_timeout=30)

搜索性能提示

• 对非评分条件使用 filter 上下文（而非 must）——结果会被缓存
• 避免使用 from+size 进行深度分页；改用带排序键的 search_after
• 使用请求体中的 "profile": true 分析慢查询
• 批量加载期间禁用副本，之后再重新启用
• 对只读历史索引执行 OPTIMIZE（强制合并为 1 个段）

结论

OpenSearch 在精心部署时能提供强大的全文搜索和分析功能。显式映射可防止模式混乱。带 filter 子句的 bool 查询可利用查询缓存。聚合无需单独的分析数据库即可实现分面导航。函数评分和固定查询让团队能够控制相关性。具有分片感知和 ISM 策略的高可用配置可确保集群在规模上具有弹性和成本效益。