Skip to content

Troubleshooting

This guide is for operations and development personnel, systematically summarizing common problems and troubleshooting paths for GoVector in production environments, covering installation and deployment, configuration errors, performance issues, data consistency, logging and location techniques, as well as community support and issue reporting processes. GoVector provides both embedded library and independent microservice modes, both using Qdrant-compatible API as entry point, based on BoltDB (bbolt) persistence and HNSW/Flat index underneath.

Project Structure

  • Top-level modules and dependencies: Module definition, Go version requirements and external dependencies (bbolt, protobuf, hnsw).
  • Server entry: Command-line parameter parsing, storage initialization, collection loading, HTTP server startup and graceful shutdown.
  • API layer: Provides /collections, /points CRUD interfaces, responsible for request parsing, routing and response.
  • Core engine: Collection is responsible for Upsert/Search/Delete; Storage is responsible for bbolt access and metadata; models and filters define.
  • Examples and scripts: Embedded usage examples, server demo script, release packaging script.
graph TB
subgraph "Runtime"
CLI["Command-line entry
cmd/govector/main.go"]
API["HTTP API Server
api/server.go"]
CORE["Core engine
core/collection.go / core/storage.go / core/models.go"]
end
subgraph "Persistence"
BBOLT["bbolt database file"]
end
CLI --> API
API --> CORE
CORE --> BBOLT

Core Components

  • Server and command-line
  • Parse parameters such as port, database path, index type, initialize storage and collection, start HTTP service, and handle signals for graceful shutdown.
  • API layer
  • Provides collection management and point operation interfaces; decodes requests, validates, and maps errors; calls Collection to execute business logic.
  • Collection
  • Maintains thread-safe in-memory index and optional persistence; Upsert guarantees persist to disk first then update index; Delete supports delete by ID or filter.
  • Storage
  • Bucket-based storage based on bbolt; supports collection metadata, point serialization (protobuf), quantization/dequantization (SQ8).
  • Models and filtering
  • Payload, PointStruct, Filter, Condition, Range/Regex/Pref/Contain/Exact and other matching rules.

Architecture Overview

The diagram below shows the complete call chain from client to storage and the consistency guarantee strategy.

sequenceDiagram
participant C as "Client"
participant S as "API Server
api/server.go"
participant COL as "Collection
core/collection.go"
participant IDX as "Index interface
core/index.go"
participant ST as "Storage
core/storage.go"
C->>S : "PUT /collections/{name}/points"
S->>S : "Parse JSON and validate"
S->>COL : "Upsert(points)"
COL->>ST : "Write point protobuf serialize"
ST-->>COL : "Success/failure"
COL->>IDX : "Update in-memory index"
IDX-->>COL : "Success/failure"
COL-->>S : "Return result or error"
S-->>C : "HTTP response"

Detailed Component Analysis

Server and Graceful Shutdown

  • Startup phase: Parse parameters, initialize storage, create/load collection, register routes, start listening.
  • Shutdown phase: Capture system signals, construct timeout context, call http.Server.Shutdown for smooth termination.
flowchart TD
Start(["Process startup"]) --> Parse["Parse command-line parameters"]
Parse --> InitStore["Initialize storage"]
InitStore --> CreateOrLoadCol["Create or load collection"]
CreateOrLoadCol --> StartHTTP["Start HTTP service"]
StartHTTP --> WaitSignal{"Wait for signal/error"}
WaitSignal --> |Signal received| Graceful["Construct timeout context and graceful shutdown"]
WaitSignal --> |Listen error| Fatal["Log fatal error and exit"]
Graceful --> End(["Process exit"])
Fatal --> End

API Request Processing and Error Mapping

  • Upsert/Search/Delete: Parse request body, validate collection existence, call Collection methods, return 4xx/5xx based on error.
  • Error mapping: Invalid JSON returns 400; collection not found returns 404; internal error returns 500.
sequenceDiagram
participant Client as "Client"
participant Handler as "handleUpsert/handleSearch/handleDelete"
participant Col as "Collection"
participant Store as "Storage"
participant Idx as "VectorIndex"
Client->>Handler : "POST/PUT/DELETE /collections/{name}/points/*"
Handler->>Handler : "Decode JSON/extract parameters"
Handler->>Col : "Call Upsert/Search/Delete"
Col->>Store : "Read/write persistence optional"
Col->>Idx : "Update in-memory index"
Col-->>Handler : "Return result or error"
Handler-->>Client : "Encode response/status code"

Data Consistency and Rollback Strategy

  • Upsert order: Persist to storage first, then update index; if index update fails, try to delete corresponding points from storage (best effort).
  • Delete: Delete from storage first, then delete from index; supports two ways: by ID or by filter.
  • Load: Collection creation loads historical points from storage and validates dimension consistency.
flowchart TD
UStart(["Upsert entry"]) --> Validate["Validate dimension/set version"]
Validate --> Persist{"Storage enabled?"}
Persist --> |Yes| WriteDisk["Write to storage protobuf"]
Persist --> |No| SkipWrite["Skip disk write"]
WriteDisk --> UpdateIdx["Update in-memory index"]
SkipWrite --> UpdateIdx
UpdateIdx --> Ok{"Index update successful?"}
Ok --> |Yes| Done(["Done"])
Ok --> |No| Rollback["Best effort rollback: delete corresponding points from storage"] --> Err(["Return error"])

Filters and Queries

  • Supports Must/MustNot conditions, key-value matching (Exact/Range/Pref/Contain/Regex), range comparison and regex matching.
  • Queries pass filter to index layer for execution.
flowchart TD
FStart(["Filter evaluation"]) --> Nil{"Filter empty?"}
Nil --> |Yes| Pass["Pass match all"]
Nil --> |No| MustEval["Evaluate Must conditions one by one"]
MustEval --> MustPass{"All satisfied?"}
MustPass --> |No| Fail["Reject"]
MustPass --> |Yes| MustNotEval["Evaluate MustNot conditions"]
MustNotEval --> NotAny{"Any satisfied?"}
NotAny --> |Yes| Fail
NotAny --> |No| Pass

Dependency Analysis

  • External dependencies
  • bbolt: Local key-value database, providing bucket and transaction capabilities.
  • protobuf: Point structure serialization.
  • hnsw: HNSW graph index implementation.
  • Internal coupling
  • API depends on Collection; Collection depends on Storage and VectorIndex interface; Storage depends on bbolt and protobuf.
  • Circular dependencies
  • No circular imports found; module boundaries are clear.
graph LR
API["api/server.go"] --> COL["core/collection.go"]
COL --> ST["core/storage.go"]
COL --> IDX["core/index.go"]
ST --> BB["bbolt"]
ST --> PB["protobuf"]
COL --> HNSW["hnsw index"]

Performance Considerations

  • Index selection
  • HNSW: Suitable for large-scale high-dimensional vectors, higher approximate search throughput; Flat: Small scale or low-dimensional scenarios, exact but slower.
  • Quantization (SQ8)
  • Storage layer supports vector quantization, reducing disk usage and loading cost, automatic dequantization during query.
  • Concurrency and locking
  • Collection provides read-write lock protection externally, avoiding data races from concurrent writes.
  • I/O and serialization
  • Protobuf serialization/deserialization overhead is controllable; batch Upsert can reduce transaction count.
  • Monitoring suggestions
  • CPU/memory/disk I/O: Combine system tools with application logs.
  • Network latency: Use pressure test tools to simulate real traffic, observe P95/P99 latency.

Troubleshooting Guide

1. Installation and Deployment Issues

  • Symptom: Cannot start server or executable not found
  • Troubleshooting points
    • Confirm Go is installed and version requirements are met.
    • Use correct build command or distribution package.

  • Reference paths

  • Symptom: Homebrew installation failure or abnormal version

  • Troubleshooting points
    • Check if version number and checksum in formula are consistent with latest release.

  • Reference paths

  • Symptom: Insufficient permissions to write database file

  • Troubleshooting points
    • Ensure running user has read/write permissions to the directory where database file is located.
  • Reference paths

2. Configuration Errors

  • Symptom: Error "invalid distance metric" when creating collection
  • Troubleshooting points
    • Only allows cosine/euclidean/dot (case-insensitive).

  • Reference paths

  • Symptom: Collection parameters (such as M/efconstruction/efsearch/k) invalid

  • Troubleshooting points