Skip to content

API Reference

This document is the complete API reference for GoVector, covering HTTP REST interfaces, data models, filter syntax, error codes, Qdrant compatibility notes, startup and configuration, and best practices for common use cases. GoVector provides a Qdrant-compatible REST API that supports collection management, point upsert, similarity search, and deletion, with built-in HNSW approximate nearest neighbor indexing and persistent storage.

Project Structure

  • api package: Provides HTTP API server, exposing REST endpoints for collections and point operations.
  • core package: Core data models, collections, indexes (HNSW/Flat), storage (BoltDB+Protobuf), and mathematical metrics.
  • cmd/govector: Standalone service entry point, starts HTTP server and loads default collections.
  • Tests: Cover API behavior and error codes, verifying endpoint correctness.
graph TB
subgraph "API Layer"
S["Server
api/server.go"]
end
subgraph "Business Layer"
C["Collection
core/collection.go"]
F["FlatIndex
core/flat_index.go"]
H["HNSWIndex
core/hnsw_index.go"]
I["VectorIndex Interface
core/index.go"]
end
subgraph "Storage Layer"
ST["Storage(BoltDB)
core/storage.go"]
P["Protobuf Serialization
core/proto/*.proto"]
end
subgraph "External Dependencies"
BB["bbolt
go.etcd.io/bbolt"]
HNSW["coder/hnsw
github.com/coder/hnsw"]
end
S --> C
C --> I
I --> F
I --> H
C --> ST
ST --> BB
H --> HNSW
ST --> P

Core Components

  • Server: HTTP API server responsible for routing and error handling; manages collection mapping and storage.
  • Collection: Collection abstraction encapsulating Upsert/Search/Delete with thread safety; selects Flat or HNSW index.
  • VectorIndex Interface: Unified index capabilities (Upsert/Search/Delete/Count/GetIDsByFilter/DeleteByFilter).
  • Storage: bbolt-based persistence using Protobuf serialization for points; supports SQ8 quantization.
  • Data Models: PointStruct, ScoredPoint, Filter/Condition/RangeValue, etc., compatible with Qdrant Payload structure and filter syntax.

Architecture Overview

The diagram below shows the call chain from HTTP requests to indexing and storage, along with collection loading and persistence flows.

sequenceDiagram
participant Client as "Client"
participant API as "Server.handleXxx
api/server.go"
participant Col as "Collection
core/collection.go"
participant Idx as "VectorIndex(HNSW/Flat)
core/index.go"
participant St as "Storage(BoltDB)
core/storage.go"
Client->>API : "HTTP Request"
API->>Col : "Lookup by collection name"
alt Collection does not exist
API-->>Client : "404 Not Found"
else Exists
API->>Col : "Execute business operation (Upsert/Search/Delete)"
Col->>Idx : "Call index interface"
Col->>St : "Read/write persistence as needed"
Col-->>API : "Return result"
API-->>Client : "200 OK + JSON Response"
end

Detailed Component Analysis

HTTP API Endpoints Overview

  • Collection Management
  • POST /collections: Create collection
  • DELETE /collections/{name}: Delete collection
  • GET /collections: List collections
  • GET /collections/{name}: Get collection info
  • Point Operations
  • PUT /collections/{name}/points: Batch write/update points
  • POST /collections/{name}/points/search: Vector search
  • POST /collections/{name}/points/delete: Delete points by ID or filter

Endpoint Specifications and Response Format

  • POST /collections
  • Function: Create collection
  • Request body fields
    • name: string, collection name
    • vector_size: integer, vector dimension (must be positive)
    • distance: string, distance metric, supports "cosine"/"euclidean"/"dot" or case variants
    • hnsw: boolean, whether to enable HNSW (optional)
    • parameters: object, HNSW parameters (optional)
    • m: integer, maximum connections
    • ef_construction: integer, construction candidate list size
    • ef_search: integer, search candidate list size
    • k: integer, number of neighbors to return
  • Success response: 200, contains collection config and parameters

  • Errors:

    • 400: JSON parsing failed, vector_size not positive, unsupported distance
    • 409: Collection already exists
    • 500: Internal error

  • DELETE /collections/{name}

  • Function: Delete collection
  • Success response: 200, contains completion status and deleted collection name

  • Error: 404 (collection does not exist)

  • GET /collections

  • Function: List all collections with basic info (name, dimension, metric, point count)

  • Success response: 200

  • GET /collections/{name}

  • Function: Get detailed info for specified collection
  • Success response: 200

  • Error: 404 (collection does not exist)

  • PUT /collections/{name}/points

  • Function: Batch upsert points
  • Request body fields
    • points: array, elements are PointStruct
  • Success response: 200, contains status and operation completion flag

  • Errors: 400 (JSON parsing failed), 404 (collection does not exist), 500 (internal error)

  • POST /collections/{name}/points/search

  • Function: Vector search
  • Request body fields
    • vector: array, query vector
    • limit: integer, return top K results, default 10
    • filter: object, Payload filter conditions (optional)
  • Success response: 200, contains status and result array (ScoredPoint)

  • Errors: 400 (JSON parsing failed), 404 (collection does not exist), 500 (internal error)

  • POST /collections/{name}/points/delete

  • Function: Delete points (by ID or by filter)
  • Request body fields
    • points: string array, delete by ID (choose one)
    • filter: object, delete by filter (choose one)
  • Success response: 200, contains status, completion flag, and delete count
  • Errors: 400 (JSON parsing failed), 404 (collection does not exist), 500 (internal error)

Data Models and Filter Syntax

  • PointStruct
  • id: string, unique identifier
  • version: integer, version number (timestamp/incrementing)
  • vector: float array, the vector

  • payload: object, metadata (for filtering)

  • ScoredPoint

  • id/version/score/payload

  • Filter and Condition

  • must/must_not: arrays, list of conditions

  • Condition

    • key: string, key in payload
    • type: string, condition type
    • exact: exact match
    • range: range comparison (gt/gte/lt/lte)
    • prefix: prefix match (string)
    • contains: contains match (array/string)
    • regex: regex match (string)
    • match: object, for exact/prefix/contains/regex
    • range: object, for range

  • Payload Filter Matching Rules

  • must: all conditions must be satisfied
  • must_not: none of the conditions can be satisfied
  • No filter provided: treated as match all

Processing Flow and Algorithm Highlights

  • Upsert Write Flow
  • Validate dimension and set version
  • Write to storage first (Storage), then update in-memory index (Index)

  • Failure rollback: try to delete from storage to maintain consistency

  • Search Retrieval Flow

  • Validate query vector dimension
  • HNSW: traverse graph first, then apply Payload filter, finally compute exact scores

  • Flat: brute force traversal, sort by metric then truncate

  • Delete Deletion Flow

  • If points provided: delete by ID
  • If filter provided: get ID list from index first, then delete
  • Delete from storage first, then from index
flowchart TD
Start(["Enter handler"]) --> Parse["Parse request body (JSON)"]
Parse --> Valid{"Parameters valid?"}
Valid --> |No| Err400["Return 400"]
Valid --> |Yes| Lookup["Locate collection"]
Lookup --> Exists{"Collection exists?"}
Exists --> |No| Err404["Return 404"]
Exists --> |Yes| Op{"Specific operation"}
Op --> Upsert["Upsert"]
Op --> Search["Search"]
Op --> Delete["Delete"]
Upsert --> U1["Validate dimension/set version"]
U1 --> U2["Write to storage"]
U2 --> U3["Update index"]
U3 --> Ok["Return 200"]
Search --> S1["Validate query dimension"]
S1 --> S2["Index search (with filter)"]
S2 --> Ok
Delete --> D1["Determine target IDs (by ID/by filter)"]
D1 --> D2["Delete from storage"]
D2 --> D3["Delete from index"]
D3 --> Ok

Error Codes and Semantics

  • 400 Bad Request: JSON parsing failed, invalid parameters (e.g., vector_size ≤ 0, unsupported distance metric)
  • 404 Not Found: Collection does not exist
  • 409 Conflict: Collection already exists (during creation)
  • 500 Internal Server Error: Internal errors (storage failure, index update failure, load failure, etc.)

Authentication, Rate Limiting, and Versioning

  • Authentication: Current implementation does not include built-in authentication; recommended to implement authentication via reverse proxy (e.g., Nginx/TLS) or gateway.
  • Rate Limiting: No built-in rate limiting; can be implemented via upstream gateway or container orchestration tools.
  • Versioning: Point structure includes version field for version tracking; storage layer provides nanosecond granularity timestamp version generation.

Startup and Configuration

  • Standalone service startup
  • Port: -port, default 18080
  • Database path: -db, default govector.db
  • Service behavior
  • Loads collection metadata from storage and rebuilds collections on startup
  • Supports graceful shutdown (context-controlled timeout)

Code Examples (Path Reference)

Dependency Analysis

graph LR
A["api/server.go"] --> B["core/collection.go"]
B --> C["core/index.go"]
C --> D["core/hnsw_index.go"]
C --> E["core/flat_index.go"]
B --> F["core/storage.go"]
F --> G["go.etcd.io/bbolt"]
D --> H["github.com/coder/hnsw"]
F --> I["google.golang.org/protobuf"]

Performance Considerations

  • Index Selection
  • HNSW: Suitable for large-scale data, low search latency, supports ef_search, m and other parameter tuning
  • Flat: Small-scale data, exact but O(n) search
  • Persistence and Serialization
  • Uses bbolt + Protobuf, stable write order; Upsert writes to storage first then updates index, ensuring consistency
  • Filter Strategy
  • HNSW search uses "over-sampling + post-filtering" strategy; the heavier the filter, the larger the over-fetch coefficient
  • Quantization
  • Optional SQ8 quantization reduces disk usage, loaded and dequantized on demand

Troubleshooting Guide

  • 400 Errors
  • JSON parsing failed: Check request body format
  • vector_size not positive: Ensure greater than 0
  • Distance not supported: Only cosine/euclidean/dot supported (case insensitive)
  • 404 Errors
  • Collection does not exist: Confirm collection name and creation process
  • 409 Errors
  • Name conflict when creating collection: Change name or delete first
  • 500 Errors
  • Storage/index exception: Check service logs; confirm database file permissions and disk space
  • Empty search results
  • Check if vector dimension matches the collection
  • Filter conditions too strict causing no matches even after over-sampling

Conclusion

This API reference document is based on the existing implementation in the repository, systematically documenting GoVector's Qdrant-compatible REST API, data models, filter syntax, and error codes, with startup configuration, performance, and troubleshooting recommendations. For production environments, it is recommended to implement authentication and rate limiting via reverse proxy, and select appropriate indexes and parameters based on data scale.

Appendix: Qdrant Compatibility Comparison

  • Collection Management
  • POST /collections: Create collection (compatible)
  • DELETE /collections/{name}: Delete collection (compatible)
  • GET /collections: List collections (compatible)
  • GET /collections/{name}: Get collection info (compatible)
  • Point Operations
  • PUT /collections/{name}/points: Upsert (compatible)
  • POST /collections/{name}/points/search: Search (compatible)
  • POST /collections/{name}/points/delete: Delete (compatible)
  • Filter Syntax
  • Supports exact/range/prefix/contains/regex with must/must_not combinations (compatible)
  • Response Structure
  • Uniformly uses "status":"ok" and "result" field wrapper (compatible style)
  • Differences
  • This implementation does not include built-in authentication and rate limiting; must be implemented via gateway or reverse proxy

  • Quantization and collection metadata persistence are extended capabilities, not Qdrant standard fields

  • core/models.go:81-106